Difference: FilesOnCluster (1 vs. 2)

Revision 22013-11-27 - TimothyAndeen

Line: 1 to 1

Removing Files from xrootd

On the xenia cluster batch system we have 130TB of space. Please be sure to check how full this if you are starting to upload a large dataset. The easiest way to check is just df -h from the command line on xenia. The system automatically trys to balance the files across the nodes, but it is not always successful. Often one node fills up faster than the others. You can check each node by running the script clusterDF.sh I have attached below. If you see that any of the nodes are >95% full then some datasets need deleted. The first thing to do is to remove your old datasets that you no longer need. This can easily be done (across the cluster) with the script I have attached, xrd_rmrf.sh. You can run it just like

./xrd_rmrf.sh /path/to/your/dataset

where the path leaves out "/data0" or "/xrootdfs" at the front, for example just "/atlas/dq2/..." or "/xrootd/mc11/...".

-- TimothyAndeen - 2013-11-27


META FILEATTACHMENT attachment="clusterDF.sh" attr="" comment="" date="1385577624" name="clusterDF.sh" path="clusterDF.sh" size="316" user="TimothyAndeen" version="1"
META FILEATTACHMENT attachment="xrd_rmrf.sh" attr="" comment="" date="1385577624" name="xrd_rmrf.sh" path="xrd_rmrf.sh" size="235" user="TimothyAndeen" version="1"
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback