Difference: DiskSharing (3 vs. 4)

Revision 42011-11-10 - WilliamSeligman

Line: 1 to 1
META TOPICPARENT name="LinuxCluster"

Disk sharing and the condor batch system

Line: 52 to 52
  • You develop your programs and scripts in your home directory. You run everything interactively. Your output files can be written to /scratch to avoid filling up the disk.
  • When you're ready to start running batch jobs, transfer your libraries to /a/share/<file-server>/$USER; e.g., if you're user jsmith and you're working on amsterdam.nevis.columbia.edu, you'll put your libraries and condor scripts in /a/share/amsterdam/jsmith; perhaps you'll write your job output and log files to /a/data/amsterdam/jsmith.
    • Don't forget to edit your scripts with the new file paths.
    • You may want to re-compile your code in the new location, to make sure the programs knows the new location of your shared libraries.
  If you think about this some more, you'll realize that this doesn't keep you from crashing the file server; you can still create a job that writes to the /data partition. But you'll only crash a file server, not a login server; the entire cluster won't crash. You'll only be in trouble with your group, not with everyone at Nevis.
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback