Difference: Condor (16 vs. 17)

Revision 172010-09-14 - WilliamSeligman

Line: 1 to 1
META TOPICPARENT name="LinuxCluster"

Batch Services at Nevis

Line: 159 to 159
  This means that condor will only send you an e-mail if there's an error while running the job.

Do you want to use up all your disk space?

At the end of most condor batch files, you'll see lines that look like this:

output   = mySimulation--$(Process).out
error    = mySimulation--$(Process).err
log      = mySimulation-$(Process).log

These lines define where the job's output, error, and log files are written. If you submit one job, the above lines are fine. If you submit 10,000 jobs, you'll create 30,000 files. If you submit mySimulation1, mySimulation2, ... you'll create an indefinite number of files. Sooner or later you'll fill up your home directory. Since you share the home directory on your server with everyone else in your working group, that means everyone in your group will be affected.

The general solution is to not write your output files into your home directory. Every workgroup server has a /data partition, which is normally several TB in size. It's a good idea to make sure your output files are written to this partition.

You can do this by:

  • submitting your job from a directory on the /data partition;
  • explicitly writing your output files to a directory on the /data partion; e.g.,
output   =  /a/data/<server>/<username>/mySimulation--$(Process).out
error    =  /a/data/<server>/<username>/mySimulation--$(Process).err
log      =  /a/data/<server>/<username>/mySimulation-$(Process).log

Don't forget to create /a/data/<server>/<username>/ before you submit your first job.


Use the vanilla environment

Unless you've specifically used the condor_compile command to compile your programs, you must submit your jobs in the "vanilla" universe. Any program that uses shared libraries cannot use condor_compile, and this includes most of the physics software at Nevis. Therefore, you are almost certainly required to have the following line in a command script:

This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback