Introduction to the VERITAS analysis with Event Display (ED)


Reference material

Getting the data

Simply run this script:

/a/home/tehanu/santander/reu/getCrabrun.sh

This should create a link on $VERITAS_USER_DATA_DIR/data/d20121013 to the VBF file with the Crab data we'll be using. If we run this command:

> ls -l $VERITAS_USER_DATA_DIR/data/d20121013

we should get something like:

total 0
lrwxrwxrwx 1 santander veritas 63 Jun 10 09:27 64080.cvbf -> /a/data/tehanu/santander/veritas/data/data/d20121013/64080.cvbf

Setting up an ED analysis

Setting up your environment

1) Log in on tehanu and execute the following command:

> cp /a/home/tehanu/santander/reu/myprofile_v3.ED ~/.myprofile

2) Then, you need to create some auxiliary folders for Eventdisplay to work. Executing this little script will do that for you and will show you where to find the folders it creates:

> /a/home/tehanu/santander/reu/create-aux-dirs.sh

3) Either log out and log back in or execute:

> source ~/.bashrc

That's it! Now, assuming steps 1-3 did not give you any trouble, let's check that ED works.

4) First, you need to let ED know which data you are going to be working with, VERITAS or CTA. This is also known as setting the observatory.

There are a couple of handy aliases in your .myprofile that you can use: This command will set CTA as your observatory:

> setcta

Similarly, for VERITAS:

> setvts

5) Checking that we have access to ED executables and ED version:

> evndisp -v

This should output:

v480a

You should also run this command to make sure the condor tools are properly setup:

> submit_dag.py

The output should be:

usage: %prog [options] [-h] [-s SOURCE | -r RUN | -l LIST] [--rerun] [--nobdt]
                       [--runprocs PROCS] [--cuts CUTS] [--parfile PARFILE]
                       [--source_type SOURCE_TYPE] [--bkg_model BKG_MODEL]
%prog [options]: error: You have to either specify an IC source (-s) or a runlist (-l)

This is the script we'll use to submit our jobs to the cluster.

Note: The machine on which the DATA_PATH defined in .myprofile and create-aux-dirs.sh is located must match the machine on which condor is run. For this setup, both are located on tehanu.

Submitting your first job to the cluster

Assuming that you have the Crab run we downloaded recently (run #64080). Prepare a runlist file using your favorite text editor containing one run ID number per file. For instance, create a file called runlist.txt that contains a single line so that when you do:

> cat runlist.txt

the output should be

64080

The next step is to submit the job to the cluster, we'll use submit_dag.py for that this way:

> submit_dag.py -l runlist.txt

The output should look similar to this:

----------------------------------------
 >>> Condor options <<<
rb_radius: 0.6
reco: GEO
source_type: PointSource
bdt: True
minfilesize: 200000
rerun: False
anasum_runlist_dir: /a/data/vetch/santander/condor/anasum_runlists
bkg_model: RB
source: None
re_mindist: 0.1
cuts: Soft
parfile: ANASUM.runparameter
onoff_ratio: 20
re_minareas: 2
evndis_version: v480
re_maxareas: 6
procs: ['evndisp', 'mscw', 'anasum']
condor_script_path: /a/data/vetch/santander/condor
----------------------------------------
----------------------------------------------------------------------------------------------------
Preparing to analyse run 64080...
Output file:  /a/data/vetch/santander/veritas/data/64080.root
Command:  ./SPANALYSIS.evndisp_pedestal_events.sh 64080 1234 0
./SPANALYSIS.evndisp_tzeros.sh 64080 1234 0
./SPANALYSIS.evndisp.sh 64080 1234 0

EVNDISP sys:  /a/data/tehanu/nieto/software//evndispversions/v480b
Condor exec file: /a/data/vetch/santander/condor/vts-execs/vts-evndisp-64080.sh
Condor output file: /a/data/vetch/santander/condor/vts-out/vts-evndisp-64080.out
Condor error file: /a/data/vetch/santander/condor/vts-error/vts-evndisp-64080.error
Condor log file: /a/data/vetch/santander/condor/vts-logs/vts-evndisp-64080.log
Output file:  /a/data/vetch/santander/veritas/data/64080.mscw.root
Getting MSCW table
/a/home/tehanu/santander/bin/getAtmonRun.sh 64080
MSCW Table:  table-v480-auxv01-GRISU-SW6-ATM22-V6-GEO.root
Command:  $EVNDISPSYS/bin/mscw_energy -tablefile table-v480-auxv01-GRISU-SW6-ATM22-V6-GEO.root -inputfile $VERITAS_USER_DATA_DIR/64080.root -arrayrecid=0 -writeReconstructedEventsOnly=1
EVNDISP sys:  /a/data/tehanu/nieto/software//evndispversions/v480b
Condor exec file: /a/data/vetch/santander/condor/vts-execs/vts-mscw-64080.sh
Condor output file: /a/data/vetch/santander/condor/vts-out/vts-mscw-64080.out
Condor error file: /a/data/vetch/santander/condor/vts-error/vts-mscw-64080.error
Condor log file: /a/data/vetch/santander/condor/vts-logs/vts-mscw-64080.log
Output file:  /a/data/vetch/santander/veritas/data/64080.anasum.root
Creating ANASUM script
/a/home/tehanu/santander/bin/getAtmonRun.sh 64080
Gamma/hadron cut file:  ANASUM.GammaHadron-Cut-NTel2-PointSource-Soft-TMVA-BDT.dat
Effective area file:  effArea-v480-auxv01-GRISU-SW6-Cut-NTel2-PointSource-Soft-TMVA-BDT-GEO-V6-ATM22-T1234.root
Gamma/hadron cut file: ANASUM.GammaHadron-Cut-NTel2-PointSource-Soft-TMVA-BDT.dat
Radial acceptance file: radialAcceptance-v480-auxv01-GRISU-SW6-Cut-NTel2-Soft-GEO-V6-T1234.root
Effective area file: effArea-v480-auxv01-GRISU-SW6-Cut-NTel2-PointSource-Soft-TMVA-BDT-GEO-V6-ATM22-T1234.root
Command:  $EVNDISPSYS/bin/anasum  -l /a/data/vetch/santander/condor/anasum_runlists/anasum-64080.list -d $VERITAS_USER_DATA_DIR -o /a/data/vetch/santander/veritas/data/64080.anasum.root -f ANASUM.runparameter
EVNDISP sys:  /a/data/tehanu/nieto/software//evndispversions/v480b
Condor exec file: /a/data/vetch/santander/condor/vts-execs/vts-anasum-64080.sh
Condor output file: /a/data/vetch/santander/condor/vts-out/vts-anasum-64080.out
Condor error file: /a/data/vetch/santander/condor/vts-error/vts-anasum-64080.error
Condor log file: /a/data/vetch/santander/condor/vts-logs/vts-anasum-64080.log
Submitting DAG
/a/data/tehanu/santander/veritas/ana/condor/condor_tools.py:123: DeprecationWarning: The popen2 module is deprecated.  Use the subprocess module.
  import popen2
Renaming rescue DAGs newer than number 0
Renaming /a/data/vetch/santander/condor/vts-dag/vts-64080.dag.rescue001
Submitting job(s).
Logging submit event(s).
1 job(s) submitted to cluster 289525.

-----------------------------------------------------------------------
File for submitting this DAG to Condor           : /a/data/vetch/santander/condor/vts-dag/vts-64080.dag.condor.sub
Log of DAGMan debugging messages                 : /a/data/vetch/santander/condor/vts-dag/vts-64080.dag.dagman.out
Log of Condor library output                     : /a/data/vetch/santander/condor/vts-dag/vts-64080.dag.lib.out
Log of Condor library error messages             : /a/data/vetch/santander/condor/vts-dag/vts-64080.dag.lib.err
Log of the life of condor_dagman itself          : /a/data/vetch/santander/condor/vts-dag/vts-64080.dag.dagman.log

-----------------------------------------------------------------------
----------------------------------------------------------------------------------------------------

Once this is done, you should be able to check on the status of your submission using condor_q

> condor_q $USER

If you are using submit_dag.py, three jobs are submitted to the Condor system (one for each ED stage, namely evndisp, mscw, and anasum). The jobs have internal dependencies, so that when evndisp finishes, the mscw stage is started right after.

Running

> condor_q $USER -dag

will return

-- Submitter: tehanu.nevis.columbia.edu : <129.236.252.111:36147> : tehanu.nevis.columbia.edu
 ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD
289525.0   <your user>       6/10 09:36   0+00:02:58 R  0   7.3  condor_dagman
289526.0   <your user>       6/10 09:37   0+00:02:37 R  0   0.0  vts-evndisp-64080.

This shows that the 'A' job depends on the DAG manager that is in charger or running the three stages for that run.

More details on condor are available here.

Combining anasum files

Once all the anasum stages have been processed, you can combine all of the anasum files using this script, which you can copy to your folder:

/a/home/tehanu/santander/reu/combine_anasum.sh

You can run by giving a runlist and output file name as the parameters:

./combined_anasum.sh runlist.txt myoutputfile.root

Once this is done, you can copy the ROOT script here:

/a/home/tehanu/santander/reu/printResults.C

and edit it so that the filename points to the results file you just created.

More information on the functions called in printResults.C are available in the ED manual wiki.

Comments


Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r6 - 2017-06-02 - MarcosSantander
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback