Difference: Condor (22 vs. 23)

Revision 232010-12-16 - WilliamSeligman

Line: 1 to 1
 
META TOPICPARENT name="LinuxCluster"

Batch Services at Nevis

Line: 90 to 90
 log = mySimulation-$(Process).log
Changed:
<
<
These lines define where the job's output, error, and log files are written. If you submit one job, the above lines are fine. If you submit 10,000 jobs, you'll create 30,000 files. If you submit mySimulation1, mySimulation2, ... you'll create an indefinite number of files. Sooner or later you'll fill up your home directory. Since you share the home directory on your server with everyone else in your working group, that means everyone in your group will be affected.
>
>
These lines define where the job's output, error, and log files are written. If you submit one job, the above lines are fine. If you submit 10,000 jobs, you'll create 30,000 files. If you submit mySimulation1, mySimulation2, ... you'll create an indefinite number of files. If initialdir points to a sub-directory of your home directory, sooner or later you'll fill up the /home partition on your server. That means everyone in your group will be affected.
  The general solution is to not write your output files into your home directory. Every workgroup server has a /data partition, which is normally several TB in size. It's a good idea to make sure your output files are written to this partition.
Changed:
<
<
You can do this by:
  • submitting your job from a directory on the /data partition;
>
>
You can do this via one of the following:
  • submitting your job from a directory on the /data partition; e.g., /a/data/<server>/<username>/
 
  • using the initialdir command to tell condor where inputs and outputs are located.
Deleted:
<
<
Don't forget to create a directory with a name like /a/data/<server>/<username>/ before you submit your first job.
 

Use the vanilla environment

Unless you've specifically used the condor_compile command to compile your programs, you must submit your jobs in the "vanilla" universe. Any program that uses shared libraries cannot use condor_compile, and this includes most of the physics software at Nevis. Therefore, you are almost certainly required to have the following line in a command script:

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback