Difference: EventDisplayAnalysis (1 vs. 6)

Revision 62017-06-02 - MarcosSantander

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Introduction to the VERITAS analysis with Event Display (ED)

Line: 27 to 28
  1) Log in on tehanu and execute the following command:
Changed:
<
<
> cp ~santander/reu/myprofile_v3.ED ~/.myprofile
>
>
> cp /a/home/tehanu/santander/reu/myprofile_v3.ED ~/.myprofile
  2) Then, you need to create some auxiliary folders for Eventdisplay to work. Executing this little script will do that for you and will show you where to find the folders it creates:

Revision 52017-04-27 - AriBrill

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Introduction to the VERITAS analysis with Event Display (ED)

Line: 27 to 27
  1) Log in on tehanu and execute the following command:
Changed:
<
<
> cp ~santander/reu/myprofile.ED ~/.myprofile
>
>
> cp ~santander/reu/myprofile_v3.ED ~/.myprofile
  2) Then, you need to create some auxiliary folders for Eventdisplay to work. Executing this little script will do that for you and will show you where to find the folders it creates:
Line: 69 to 69
 %prog [options]: error: You have to either specify an IC source (-s) or a runlist (-l)

This is the script we'll use to submit our jobs to the cluster.

Added:
>
>
Note: The machine on which the DATA_PATH defined in .myprofile and create-aux-dirs.sh is located must match the machine on which condor is run. For this setup, both are located on tehanu.
 

Submitting your first job to the cluster

Assuming that you have the Crab run we downloaded recently (run #64080). Prepare a runlist file using your favorite text editor containing one run ID number per file. For instance, create a file called runlist.txt that contains a single line so that when you do:

Line: 185 to 187
  Once all the anasum stages have been processed, you can combine all of the anasum files using this script, which you can copy to your folder:
Changed:
<
<
/a/home/tehanu/santander/reu/combine_anasum.sh
>
>
/a/home/tehanu/santander/reu/combine_anasum.sh
  You can run by giving a runlist and output file name as the parameters:
Changed:
<
<
./combined_anasum.sh runlist.txt myoutputfile.root
>
>
./combined_anasum.sh runlist.txt myoutputfile.root
  Once this is done, you can copy the ROOT script here:
Changed:
<
<
/a/home/tehanu/santander/reu/printResults.C
>
>
/a/home/tehanu/santander/reu/printResults.C
  and edit it so that the filename points to the results file you just created.

Revision 42016-06-16 - MarcosSantander

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Introduction to the VERITAS analysis with Event Display (ED)

Line: 11 to 11
  Simply run this script:
Changed:
<
<
/a/home/tehanu/santander/reu/getCrabrun.sh
>
>
/a/home/tehanu/santander/reu/getCrabrun.sh
 
Changed:
<
<
This should create a link on $VERITAS_USER_DATA_DIR/data/d20121013 to the VBF file with the Crab data we'll be using. If we run this command:
>
>
This should create a link on $VERITAS_USER_DATA_DIR/data/d20121013 to the VBF file with the Crab data we'll be using. If we run this command:
 
Changed:
<
<
> ls -l $VERITAS_USER_DATA_DIR/data/d20121013
>
>
> ls -l $VERITAS_USER_DATA_DIR/data/d20121013
  we should get something like:
Deleted:
<
<
 
total 0
Changed:
<
<
lrwxrwxrwx 1 santander veritas 63 Jun 10 09:27 64080.cvbf -> /a/data/tehanu/santander/veritas/data/data/d20121013/64080.cvbf
>
>
lrwxrwxrwx 1 santander veritas 63 Jun 10 09:27 64080.cvbf -> /a/data/tehanu/santander/veritas/data/data/d20121013/64080.cvbf
 

Setting up an ED analysis

Setting up your environment

1) Log in on tehanu and execute the following command:

Changed:
<
<
> cp ~santander/reu/myprofile.ED ~/.myprofile
>
>
> cp ~santander/reu/myprofile.ED ~/.myprofile
  2) Then, you need to create some auxiliary folders for Eventdisplay to work. Executing this little script will do that for you and will show you where to find the folders it creates:
Changed:
<
<
> /a/home/tehanu/santander/reu/create-aux-dirs.sh
>
>
> /a/home/tehanu/santander/reu/create-aux-dirs.sh
  3) Either log out and log back in or execute:
Changed:
<
<
> source ~/.bashrc
>
>
> source ~/.bashrc
  That's it! Now, assuming steps 1-3 did not give you any trouble, let's check that ED works.

4) First, you need to let ED know which data you are going to be working with, VERITAS or CTA. This is also known as setting the observatory.

Changed:
<
<
There are a couple of handy aliases in your .myprofile that you can use: This command will set CTA as your observatory:
>
>
There are a couple of handy aliases in your .myprofile that you can use: This command will set CTA as your observatory:
 
Changed:
<
<
> setcta
>
>
> setcta
  Similarly, for VERITAS:
Changed:
<
<
> setvts
>
>
> setvts
  5) Checking that we have access to ED executables and ED version:
Changed:
<
<
> evndisp -v
>
>
> evndisp -v
  This should output:
Changed:
<
<
v480a
>
>
v480a
  You should also run this command to make sure the condor tools are properly setup:
Changed:
<
<
> submit_dag.py
>
>
> submit_dag.py
  The output should be:
Deleted:
<
<
 
usage: %prog [options] [-h] [-s SOURCE | -r RUN | -l LIST] [--rerun] [--nobdt]
                       [--runprocs PROCS] [--cuts CUTS] [--parfile PARFILE]
Line: 74 to 71
 This is the script we'll use to submit our jobs to the cluster.

Submitting your first job to the cluster

Changed:
<
<
Assuming that you have the Crab run we downloaded recently (run #64080). Prepare a runlist file using your favorite text editor containing one run ID number per file. For instance, create a file called runlist.txt that contains a single line so that when you do:
>
>
Assuming that you have the Crab run we downloaded recently (run #64080). Prepare a runlist file using your favorite text editor containing one run ID number per file. For instance, create a file called runlist.txt that contains a single line so that when you do:
 
Changed:
<
<
> cat runlist.txt
>
>
> cat runlist.txt
  the output should be
Changed:
<
<
64080
>
>
64080
 
Changed:
<
<
The next step is to submit the job to the cluster, we'll use submit_dag.py for that this way:
>
>
The next step is to submit the job to the cluster, we'll use submit_dag.py for that this way:
 
Changed:
<
<
> submit_dag.py -l runlist.txt
>
>
> submit_dag.py -l runlist.txt
  The output should look similar to this:
Deleted:
<
<
 
----------------------------------------
 >>> Condor options <<<
Line: 165 to 161
 

Changed:
<
<
Once this is done, you should be able to check on the status of your submission using condor_q
>
>
Once this is done, you should be able to check on the status of your submission using condor_q
 
Changed:
<
<
> condor_q $USER
>
>
> condor_q $USER
 
Changed:
<
<
If you are using submit_dag.py, three jobs are submitted to the Condor system (one for each ED stage, namely evndisp, mscw, and anasum). The jobs have internal dependencies, so that when evndisp finishes, the mscw stage is started right after.
>
>
If you are using submit_dag.py, three jobs are submitted to the Condor system (one for each ED stage, namely evndisp, mscw, and anasum). The jobs have internal dependencies, so that when evndisp finishes, the mscw stage is started right after.
  Running
Changed:
<
<
> condor_q $USER -dag
>
>
> condor_q $USER -dag
  will return
Deleted:
<
<
 
-- Submitter: tehanu.nevis.columbia.edu : <129.236.252.111:36147> : tehanu.nevis.columbia.edu
 ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD
Line: 186 to 181
 This shows that the 'A' job depends on the DAG manager that is in charger or running the three stages for that run.

More details on condor are available here.

Added:
>
>

Combining anasum files

Once all the anasum stages have been processed, you can combine all of the anasum files using this script, which you can copy to your folder:

/a/home/tehanu/santander/reu/combine_anasum.sh

You can run by giving a runlist and output file name as the parameters:

./combined_anasum.sh runlist.txt myoutputfile.root

Once this is done, you can copy the ROOT script here:

/a/home/tehanu/santander/reu/printResults.C

and edit it so that the filename points to the results file you just created.

 
Added:
>
>
More information on the functions called in printResults.C are available in the ED manual wiki.
 

Comments


<--/commentPlugin-->

Revision 32016-06-10 - MarcosSantander

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Introduction to the VERITAS analysis with Event Display (ED)

Changed:
<
<
>
>

 

Reference material

Line: 11 to 11
  Simply run this script:
Changed:
<
<
/a/home/tehanu/santander/reu/getCrabrun.sh
>
>
/a/home/tehanu/santander/reu/getCrabrun.sh

This should create a link on $VERITAS_USER_DATA_DIR/data/d20121013 to the VBF file with the Crab data we'll be using. If we run this command:

> ls -l $VERITAS_USER_DATA_DIR/data/d20121013

we should get something like:

total 0
lrwxrwxrwx 1 santander veritas 63 Jun 10 09:27 64080.cvbf -> /a/data/tehanu/santander/veritas/data/data/d20121013/64080.cvbf
 
Deleted:
<
<
This should create a link on $VERITAS_USER_DATA_DIR/data/d20121013 to the VBF file with the Crab data we'll be using
 

Setting up an ED analysis

Setting up your environment

1) Log in on tehanu and execute the following command:

Changed:
<
<
> cp ~santander/reu/myprofile.ED ~/.myprofile
>
>
> cp ~santander/reu/myprofile.ED ~/.myprofile
  2) Then, you need to create some auxiliary folders for Eventdisplay to work. Executing this little script will do that for you and will show you where to find the folders it creates:
Changed:
<
<
> /a/home/tehanu/santander/reu/create-aux-dirs.sh
>
>
> /a/home/tehanu/santander/reu/create-aux-dirs.sh
  3) Either log out and log back in or execute:
Changed:
<
<
> source ~/.bashrc
>
>
> source ~/.bashrc
  That's it! Now, assuming steps 1-3 did not give you any trouble, let's check that ED works.

4) First, you need to let ED know which data you are going to be working with, VERITAS or CTA. This is also known as setting the observatory.

Changed:
<
<
There are a couple of handy aliases in your .myprofile that you can use: This command will set CTA as your observatory:
>
>
There are a couple of handy aliases in your .myprofile that you can use: This command will set CTA as your observatory:
 
Changed:
<
<
> setcta
>
>
> setcta
  Similarly, for VERITAS:
Changed:
<
<
> setvts
>
>
> setvts
  5) Checking that we have access to ED executables and ED version:
Changed:
<
<
> evndisp -v
>
>
> evndisp -v
  This should output:
Changed:
<
<
v480a
>
>
v480a
  You should also run this command to make sure the condor tools are properly setup:
Changed:
<
<
> _submit_dag.py_
>
>
> submit_dag.py
  The output should be:
Changed:
<
<
usage: %prog [options] [-h] [-s SOURCE | -r RUN | -l LIST] [--rerun] [--nobdt]
[--runprocs PROCS] [--cuts CUTS] [--parfile PARFILE]
[--source_type SOURCE_TYPE] [--bkg_model BKG_MODEL]
%prog [options]: error: You have to either specify an IC source (-s) or a runlist (-l)
>
>
usage: %prog [options] [-h] [-s SOURCE | -r RUN | -l LIST] [--rerun] [--nobdt]
                       [--runprocs PROCS] [--cuts CUTS] [--parfile PARFILE]
                       [--source_type SOURCE_TYPE] [--bkg_model BKG_MODEL]
%prog [options]: error: You have to either specify an IC source (-s) or a runlist (-l)
  This is the script we'll use to submit our jobs to the cluster.

Submitting your first job to the cluster

Changed:
<
<
Assuming that you have the Crab run we downloaded recently (run #64080). Prepare a runlist file using your favorite text editor containing one run ID number per file. For instance, create a file called runlist.txt that contains a single line so that when you do:
>
>
Assuming that you have the Crab run we downloaded recently (run #64080). Prepare a runlist file using your favorite text editor containing one run ID number per file. For instance, create a file called runlist.txt that contains a single line so that when you do:
 
Changed:
<
<
> cat runlist.txt
>
>
> cat runlist.txt
  the output should be
Changed:
<
<
64080
>
>
64080
 
Changed:
<
<
The next step is to submit the job to the cluster, we'll use submit_dag.py for that this way:
>
>
The next step is to submit the job to the cluster, we'll use submit_dag.py for that this way:
 
Changed:
<
<
> submit_dag.py -l runlist.txt
>
>
> submit_dag.py -l runlist.txt
  The output should look similar to this:
Deleted:
<
<

----------------------------------------
>>> Condor options <<<
rb_radius: 0.6
reco: GEO
source_type: PointSource
bdt: True
minfilesize: 200000
rerun: True
anasum_runlist_dir: /a/data/tehanu/santander/veritas/ana/anasum_runlist_dir
bkg_model: RB
source: None
re_mindist: 0.1
cuts: Soft
parfile: ANASUM.runparameter
onoff_ratio: 20
re_minareas: 2
evndis_version: v480
re_maxareas: 6
procs: ['evndisp', 'mscw', 'anasum']
condor_script_path: /a/data/tehanu/santander/veritas/ana/condor
----------------------------------------
----------------------------------------------------------------------------------------------------
Preparing to analyse run 64080...
Output file: /a/data/tehanu/santander/veritas/data/64080.root
Command: ./SPANALYSIS.evndisp_pedestal_events.sh 64080 1234 0
./SPANALYSIS.evndisp_tzeros.sh 64080 1234 0
./SPANALYSIS.evndisp.sh 64080 1234 0

EVNDISP sys: /a/data/tehanu/santander/software/EventDisplay/v480a
Condor exec file: /a/data/tehanu/santander/veritas/ana/condor/vts-execs/vts-evndisp-64080.sh
Condor output file: /a/data/tehanu/santander/veritas/ana/condor/vts-out/vts-evndisp-64080.out
Condor error file: /a/data/tehanu/santander/veritas/ana/condor/vts-error/vts-evndisp-64080.error
Condor log file: /a/data/tehanu/santander/veritas/ana/condor/vts-logs/vts-evndisp-64080.log
Output file: /a/data/tehanu/santander/veritas/data/64080.mscw.root
Getting MSCW table
/a/home/tehanu/santander/bin/getAtmonRun.sh 64080
MSCW Table: table-v480-auxv01-GRISU-SW6-ATM22-V6-GEO.root
Command: $EVNDISPSYS/bin/mscw_energy -tablefile table-v480-auxv01-GRISU-SW6-ATM22-V6-GEO.root -inputfile $VERITAS_USER_DATA_DIR/64080.root -arrayrecid=0 -writeReconstructedEventsOnly=1
EVNDISP sys: /a/data/tehanu/santander/software/EventDisplay/v480a
Condor exec file: /a/data/tehanu/santander/veritas/ana/condor/vts-execs/vts-mscw-64080.sh
Condor output file: /a/data/tehanu/santander/veritas/ana/condor/vts-out/vts-mscw-64080.out
Condor error file: /a/data/tehanu/santander/veritas/ana/condor/vts-error/vts-mscw-64080.error
Condor log file: /a/data/tehanu/santander/veritas/ana/condor/vts-logs/vts-mscw-64080.log
Output file: /a/data/tehanu/santander/veritas/data/64080.anasum.root
Creating ANASUM script
/a/home/tehanu/santander/bin/getAtmonRun.sh 64080
Gamma/hadron cut file: GammaHadron -Cut-NTel2-PointSource-Soft-TMVA-BDT.dat
Effective area file: effArea-v480-auxv01-GRISU-SW6-Cut-NTel2-PointSource-Soft-TMVA-BDT-GEO-V6-ATM22-T1234.root
Gamma/hadron cut file: GammaHadron -Cut-NTel2-PointSource-Soft-TMVA-BDT.dat
Radial acceptance file: radialAcceptance-v480-auxv01-GRISU-SW6-Cut-NTel2-Soft-GEO-V6-T1234.root
Effective area file: effArea-v480-auxv01-GRISU-SW6-Cut-NTel2-PointSource-Soft-TMVA-BDT-GEO-V6-ATM22-T1234.root
Command: $EVNDISPSYS/bin/anasum -l /a/data/tehanu/santander/veritas/ana/anasum_runlist_dir/anasum-64080.list -d $VERITAS_USER_DATA_DIR -o /a/data/tehanu/santander/veritas/data/64080.anasum.root -f ANASUM.runparameter
EVNDISP sys: /a/data/tehanu/santander/software/EventDisplay/v480a
Condor exec file: /a/data/tehanu/santander/veritas/ana/condor/vts-execs/vts-anasum-64080.sh
Condor output file: /a/data/tehanu/santander/veritas/ana/condor/vts-out/vts-anasum-64080.out
Condor error file: /a/data/tehanu/santander/veritas/ana/condor/vts-error/vts-anasum-64080.error
Condor log file: /a/data/tehanu/santander/veritas/ana/condor/vts-logs/vts-anasum-64080.log
Submitting DAG

 
Changed:
<
<
Once this is done, you should be able to check on the status of your submission using condor_q
>
>
----------------------------------------
 >>> Condor options <<<
rb_radius: 0.6
reco: GEO
source_type: PointSource
bdt: True
minfilesize: 200000
rerun: False
anasum_runlist_dir: /a/data/vetch/santander/condor/anasum_runlists
bkg_model: RB
source: None
re_mindist: 0.1
cuts: Soft
parfile: ANASUM.runparameter
onoff_ratio: 20
re_minareas: 2
evndis_version: v480
re_maxareas: 6
procs: ['evndisp', 'mscw', 'anasum']
condor_script_path: /a/data/vetch/santander/condor
----------------------------------------
----------------------------------------------------------------------------------------------------
Preparing to analyse run 64080...
Output file:  /a/data/vetch/santander/veritas/data/64080.root
Command:  ./SPANALYSIS.evndisp_pedestal_events.sh 64080 1234 0
./SPANALYSIS.evndisp_tzeros.sh 64080 1234 0
./SPANALYSIS.evndisp.sh 64080 1234 0

EVNDISP sys:  /a/data/tehanu/nieto/software//evndispversions/v480b
Condor exec file: /a/data/vetch/santander/condor/vts-execs/vts-evndisp-64080.sh
Condor output file: /a/data/vetch/santander/condor/vts-out/vts-evndisp-64080.out
Condor error file: /a/data/vetch/santander/condor/vts-error/vts-evndisp-64080.error
Condor log file: /a/data/vetch/santander/condor/vts-logs/vts-evndisp-64080.log
Output file:  /a/data/vetch/santander/veritas/data/64080.mscw.root
Getting MSCW table
/a/home/tehanu/santander/bin/getAtmonRun.sh 64080
MSCW Table:  table-v480-auxv01-GRISU-SW6-ATM22-V6-GEO.root
Command:  $EVNDISPSYS/bin/mscw_energy -tablefile table-v480-auxv01-GRISU-SW6-ATM22-V6-GEO.root -inputfile $VERITAS_USER_DATA_DIR/64080.root -arrayrecid=0 -writeReconstructedEventsOnly=1
EVNDISP sys:  /a/data/tehanu/nieto/software//evndispversions/v480b
Condor exec file: /a/data/vetch/santander/condor/vts-execs/vts-mscw-64080.sh
Condor output file: /a/data/vetch/santander/condor/vts-out/vts-mscw-64080.out
Condor error file: /a/data/vetch/santander/condor/vts-error/vts-mscw-64080.error
Condor log file: /a/data/vetch/santander/condor/vts-logs/vts-mscw-64080.log
Output file:  /a/data/vetch/santander/veritas/data/64080.anasum.root
Creating ANASUM script
/a/home/tehanu/santander/bin/getAtmonRun.sh 64080
Gamma/hadron cut file:  ANASUM.GammaHadron-Cut-NTel2-PointSource-Soft-TMVA-BDT.dat
Effective area file:  effArea-v480-auxv01-GRISU-SW6-Cut-NTel2-PointSource-Soft-TMVA-BDT-GEO-V6-ATM22-T1234.root
Gamma/hadron cut file: ANASUM.GammaHadron-Cut-NTel2-PointSource-Soft-TMVA-BDT.dat
Radial acceptance file: radialAcceptance-v480-auxv01-GRISU-SW6-Cut-NTel2-Soft-GEO-V6-T1234.root
Effective area file: effArea-v480-auxv01-GRISU-SW6-Cut-NTel2-PointSource-Soft-TMVA-BDT-GEO-V6-ATM22-T1234.root
Command:  $EVNDISPSYS/bin/anasum  -l /a/data/vetch/santander/condor/anasum_runlists/anasum-64080.list -d $VERITAS_USER_DATA_DIR -o /a/data/vetch/santander/veritas/data/64080.anasum.root -f ANASUM.runparameter
EVNDISP sys:  /a/data/tehanu/nieto/software//evndispversions/v480b
Condor exec file: /a/data/vetch/santander/condor/vts-execs/vts-anasum-64080.sh
Condor output file: /a/data/vetch/santander/condor/vts-out/vts-anasum-64080.out
Condor error file: /a/data/vetch/santander/condor/vts-error/vts-anasum-64080.error
Condor log file: /a/data/vetch/santander/condor/vts-logs/vts-anasum-64080.log
Submitting DAG
/a/data/tehanu/santander/veritas/ana/condor/condor_tools.py:123: DeprecationWarning: The popen2 module is deprecated.  Use the subprocess module.
  import popen2
Renaming rescue DAGs newer than number 0
Renaming /a/data/vetch/santander/condor/vts-dag/vts-64080.dag.rescue001
Submitting job(s).
Logging submit event(s).
1 job(s) submitted to cluster 289525.

-----------------------------------------------------------------------
File for submitting this DAG to Condor           : /a/data/vetch/santander/condor/vts-dag/vts-64080.dag.condor.sub
Log of DAGMan debugging messages                 : /a/data/vetch/santander/condor/vts-dag/vts-64080.dag.dagman.out
Log of Condor library output                     : /a/data/vetch/santander/condor/vts-dag/vts-64080.dag.lib.out
Log of Condor library error messages             : /a/data/vetch/santander/condor/vts-dag/vts-64080.dag.lib.err
Log of the life of condor_dagman itself          : /a/data/vetch/santander/condor/vts-dag/vts-64080.dag.dagman.log

-----------------------------------------------------------------------
----------------------------------------------------------------------------------------------------

Once this is done, you should be able to check on the status of your submission using condor_q

> condor_q $USER

If you are using submit_dag.py, three jobs are submitted to the Condor system (one for each ED stage, namely evndisp, mscw, and anasum). The jobs have internal dependencies, so that when evndisp finishes, the mscw stage is started right after.

Running

> condor_q $USER -dag

will return

-- Submitter: tehanu.nevis.columbia.edu : <129.236.252.111:36147> : tehanu.nevis.columbia.edu
 ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD
289525.0   <your user>       6/10 09:36   0+00:02:58 R  0   7.3  condor_dagman
289526.0   <your user>       6/10 09:37   0+00:02:37 R  0   0.0  vts-evndisp-64080.
 
Changed:
<
<
> condor_q $USER
>
>
This shows that the 'A' job depends on the DAG manager that is in charger or running the three stages for that run.
  More details on condor are available here.

Revision 22016-06-09 - MarcosSantander

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Introduction to the VERITAS analysis with Event Display (ED)

Line: 7 to 7
 
Added:
>
>

Getting the data

Simply run this script:

/a/home/tehanu/santander/reu/getCrabrun.sh

This should create a link on $VERITAS_USER_DATA_DIR/data/d20121013 to the VBF file with the Crab data we'll be using

 

Setting up an ED analysis

Setting up your environment

1) Log in on tehanu and execute the following command:

Changed:
<
<
> cp ~santander/reu/myprofile.ED ~/.myprofile
>
>
> cp ~santander/reu/myprofile.ED ~/.myprofile
  2) Then, you need to create some auxiliary folders for Eventdisplay to work. Executing this little script will do that for you and will show you where to find the folders it creates:
Changed:
<
<
> /a/home/tehanu/santander/reu/create-aux-dirs.sh
>
>
> /a/home/tehanu/santander/reu/create-aux-dirs.sh
  3) Either log out and log back in or execute:
Changed:
<
<
> source ~/.bashrc
>
>
> source ~/.bashrc
  That's it! Now, assuming steps 1-3 did not give you any trouble, let's check that ED works.
Line: 28 to 35
  There are a couple of handy aliases in your .myprofile that you can use: This command will set CTA as your observatory:
Changed:
<
<
> setcta
>
>
> setcta
  Similarly, for VERITAS:
Changed:
<
<
> setvts
>
>
> setvts
  5) Checking that we have access to ED executables and ED version:
Changed:
<
<
> evndisp -v
>
>
> evndisp -v
  This should output:
Changed:
<
<
v480a
>
>
v480a
  You should also run this command to make sure the condor tools are properly setup:
Changed:
<
<
> submit_dag.py
>
>
> _submit_dag.py_
  The output should be:
Changed:
<
<
usage: %prog [options] [-h] [-s SOURCE | -r RUN | -l LIST] [--rerun] [--nobdt]
[--runprocs PROCS] [--cuts CUTS] [--parfile PARFILE]
[--source_type SOURCE_TYPE] [--bkg_model BKG_MODEL]
%prog [options]: error: You have to either specify an IC source (-s) or a runlist (-l)
>
>
usage: %prog [options] [-h] [-s SOURCE | -r RUN | -l LIST] [--rerun] [--nobdt]
[--runprocs PROCS] [--cuts CUTS] [--parfile PARFILE]
[--source_type SOURCE_TYPE] [--bkg_model BKG_MODEL]
%prog [options]: error: You have to either specify an IC source (-s) or a runlist (-l)
  This is the script we'll use to submit our jobs to the cluster.

Submitting your first job to the cluster

Assuming that you have the Crab run we downloaded recently (run #64080). Prepare a runlist file using your favorite text editor containing one run ID number per file. For instance, create a file called runlist.txt that contains a single line so that when you do:

Changed:
<
<
> cat runlist.txt
>
>
> cat runlist.txt
  the output should be
Changed:
<
<
64080
>
>
64080
  The next step is to submit the job to the cluster, we'll use submit_dag.py for that this way:
Changed:
<
<
> submit_dag.py -l runlist.txt
>
>
> submit_dag.py -l runlist.txt
  The output should look similar to this:
Changed:
<
<
----------------------------------------
>>> Condor options <<<
rb_radius: 0.6
reco: GEO
source_type: PointSource
bdt: True
minfilesize: 200000
rerun: True
anasum_runlist_dir: /a/data/tehanu/santander/veritas/ana/anasum_runlist_dir
bkg_model: RB
source: None
re_mindist: 0.1
cuts: Soft
parfile: ANASUM.runparameter
onoff_ratio: 20
re_minareas: 2
evndis_version: v480
re_maxareas: 6
procs: ['evndisp', 'mscw', 'anasum']
condor_script_path: /a/data/tehanu/santander/veritas/ana/condor
----------------------------------------
----------------------------------------------------------------------------------------------------
Preparing to analyse run 64080...
Output file: /a/data/tehanu/santander/veritas/data/64080.root
Command: ./SPANALYSIS.evndisp_pedestal_events.sh 64080 1234 0
./SPANALYSIS.evndisp_tzeros.sh 64080 1234 0
./SPANALYSIS.evndisp.sh 64080 1234 0
>
>
----------------------------------------
>>> Condor options <<<
rb_radius: 0.6
reco: GEO
source_type: PointSource
bdt: True
minfilesize: 200000
rerun: True
anasum_runlist_dir: /a/data/tehanu/santander/veritas/ana/anasum_runlist_dir
bkg_model: RB
source: None
re_mindist: 0.1
cuts: Soft
parfile: ANASUM.runparameter
onoff_ratio: 20
re_minareas: 2
evndis_version: v480
re_maxareas: 6
procs: ['evndisp', 'mscw', 'anasum']
condor_script_path: /a/data/tehanu/santander/veritas/ana/condor
----------------------------------------
----------------------------------------------------------------------------------------------------
Preparing to analyse run 64080...
Output file: /a/data/tehanu/santander/veritas/data/64080.root
Command: ./SPANALYSIS.evndisp_pedestal_events.sh 64080 1234 0
./SPANALYSIS.evndisp_tzeros.sh 64080 1234 0
./SPANALYSIS.evndisp.sh 64080 1234 0
 
Changed:
<
<
EVNDISP sys: /a/data/tehanu/santander/software/EventDisplay/v480a
Condor exec file: /a/data/tehanu/santander/veritas/ana/condor/vts-execs/vts-evndisp-64080.sh
Condor output file: /a/data/tehanu/santander/veritas/ana/condor/vts-out/vts-evndisp-64080.out
Condor error file: /a/data/tehanu/santander/veritas/ana/condor/vts-error/vts-evndisp-64080.error
Condor log file: /a/data/tehanu/santander/veritas/ana/condor/vts-logs/vts-evndisp-64080.log
Output file: /a/data/tehanu/santander/veritas/data/64080.mscw.root
Getting MSCW table
/a/home/tehanu/santander/bin/getAtmonRun.sh 64080
MSCW Table: table-v480-auxv01-GRISU-SW6-ATM22-V6-GEO.root
Command: $EVNDISPSYS/bin/mscw_energy -tablefile table-v480-auxv01-GRISU-SW6-ATM22-V6-GEO.root -inputfile $VERITAS_USER_DATA_DIR/64080.root -arrayrecid=0 -writeReconstructedEventsOnly=1
EVNDISP sys: /a/data/tehanu/santander/software/EventDisplay/v480a
Condor exec file: /a/data/tehanu/santander/veritas/ana/condor/vts-execs/vts-mscw-64080.sh
Condor output file: /a/data/tehanu/santander/veritas/ana/condor/vts-out/vts-mscw-64080.out
Condor error file: /a/data/tehanu/santander/veritas/ana/condor/vts-error/vts-mscw-64080.error
Condor log file: /a/data/tehanu/santander/veritas/ana/condor/vts-logs/vts-mscw-64080.log
Output file: /a/data/tehanu/santander/veritas/data/64080.anasum.root
Creating ANASUM script
/a/home/tehanu/santander/bin/getAtmonRun.sh 64080
Gamma/hadron cut file: GammaHadron -Cut-NTel2-PointSource-Soft-TMVA-BDT.dat
Effective area file: effArea-v480-auxv01-GRISU-SW6-Cut-NTel2-PointSource-Soft-TMVA-BDT-GEO-V6-ATM22-T1234.root
Gamma/hadron cut file: GammaHadron -Cut-NTel2-PointSource-Soft-TMVA-BDT.dat
Radial acceptance file: radialAcceptance-v480-auxv01-GRISU-SW6-Cut-NTel2-Soft-GEO-V6-T1234.root
Effective area file: effArea-v480-auxv01-GRISU-SW6-Cut-NTel2-PointSource-Soft-TMVA-BDT-GEO-V6-ATM22-T1234.root
Command: $EVNDISPSYS/bin/anasum -l /a/data/tehanu/santander/veritas/ana/anasum_runlist_dir/anasum-64080.list -d $VERITAS_USER_DATA_DIR -o /a/data/tehanu/santander/veritas/data/64080.anasum.root -f ANASUM.runparameter
EVNDISP sys: /a/data/tehanu/santander/software/EventDisplay/v480a
Condor exec file: /a/data/tehanu/santander/veritas/ana/condor/vts-execs/vts-anasum-64080.sh
Condor output file: /a/data/tehanu/santander/veritas/ana/condor/vts-out/vts-anasum-64080.out
Condor error file: /a/data/tehanu/santander/veritas/ana/condor/vts-error/vts-anasum-64080.error
Condor log file: /a/data/tehanu/santander/veritas/ana/condor/vts-logs/vts-anasum-64080.log
Submitting DAG
>
>
EVNDISP sys: /a/data/tehanu/santander/software/EventDisplay/v480a
Condor exec file: /a/data/tehanu/santander/veritas/ana/condor/vts-execs/vts-evndisp-64080.sh
Condor output file: /a/data/tehanu/santander/veritas/ana/condor/vts-out/vts-evndisp-64080.out
Condor error file: /a/data/tehanu/santander/veritas/ana/condor/vts-error/vts-evndisp-64080.error
Condor log file: /a/data/tehanu/santander/veritas/ana/condor/vts-logs/vts-evndisp-64080.log
Output file: /a/data/tehanu/santander/veritas/data/64080.mscw.root
Getting MSCW table
/a/home/tehanu/santander/bin/getAtmonRun.sh 64080
MSCW Table: table-v480-auxv01-GRISU-SW6-ATM22-V6-GEO.root
Command: $EVNDISPSYS/bin/mscw_energy -tablefile table-v480-auxv01-GRISU-SW6-ATM22-V6-GEO.root -inputfile $VERITAS_USER_DATA_DIR/64080.root -arrayrecid=0 -writeReconstructedEventsOnly=1
EVNDISP sys: /a/data/tehanu/santander/software/EventDisplay/v480a
Condor exec file: /a/data/tehanu/santander/veritas/ana/condor/vts-execs/vts-mscw-64080.sh
Condor output file: /a/data/tehanu/santander/veritas/ana/condor/vts-out/vts-mscw-64080.out
Condor error file: /a/data/tehanu/santander/veritas/ana/condor/vts-error/vts-mscw-64080.error
Condor log file: /a/data/tehanu/santander/veritas/ana/condor/vts-logs/vts-mscw-64080.log
Output file: /a/data/tehanu/santander/veritas/data/64080.anasum.root
Creating ANASUM script
/a/home/tehanu/santander/bin/getAtmonRun.sh 64080
Gamma/hadron cut file: GammaHadron -Cut-NTel2-PointSource-Soft-TMVA-BDT.dat
Effective area file: effArea-v480-auxv01-GRISU-SW6-Cut-NTel2-PointSource-Soft-TMVA-BDT-GEO-V6-ATM22-T1234.root
Gamma/hadron cut file: GammaHadron -Cut-NTel2-PointSource-Soft-TMVA-BDT.dat
Radial acceptance file: radialAcceptance-v480-auxv01-GRISU-SW6-Cut-NTel2-Soft-GEO-V6-T1234.root
Effective area file: effArea-v480-auxv01-GRISU-SW6-Cut-NTel2-PointSource-Soft-TMVA-BDT-GEO-V6-ATM22-T1234.root
Command: $EVNDISPSYS/bin/anasum -l /a/data/tehanu/santander/veritas/ana/anasum_runlist_dir/anasum-64080.list -d $VERITAS_USER_DATA_DIR -o /a/data/tehanu/santander/veritas/data/64080.anasum.root -f ANASUM.runparameter
EVNDISP sys: /a/data/tehanu/santander/software/EventDisplay/v480a
Condor exec file: /a/data/tehanu/santander/veritas/ana/condor/vts-execs/vts-anasum-64080.sh
Condor output file: /a/data/tehanu/santander/veritas/ana/condor/vts-out/vts-anasum-64080.out
Condor error file: /a/data/tehanu/santander/veritas/ana/condor/vts-error/vts-anasum-64080.error
Condor log file: /a/data/tehanu/santander/veritas/ana/condor/vts-logs/vts-anasum-64080.log
Submitting DAG
 

Once this is done, you should be able to check on the status of your submission using condor_q

Changed:
<
<
> condor_q $USER
>
>
> condor_q $USER
  More details on condor are available here.

Revision 12016-06-08 - MarcosSantander

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="WebHome"

Introduction to the VERITAS analysis with Event Display (ED)

Reference material

Setting up an ED analysis

Setting up your environment

1) Log in on tehanu and execute the following command:

> cp ~santander/reu/myprofile.ED ~/.myprofile

2) Then, you need to create some auxiliary folders for Eventdisplay to work. Executing this little script will do that for you and will show you where to find the folders it creates:

> /a/home/tehanu/santander/reu/create-aux-dirs.sh

3) Either log out and log back in or execute:

> source ~/.bashrc

That's it! Now, assuming steps 1-3 did not give you any trouble, let's check that ED works.

4) First, you need to let ED know which data you are going to be working with, VERITAS or CTA. This is also known as setting the observatory.

There are a couple of handy aliases in your .myprofile that you can use: This command will set CTA as your observatory:

> setcta

Similarly, for VERITAS:

> setvts

5) Checking that we have access to ED executables and ED version:

> evndisp -v

This should output:

v480a

You should also run this command to make sure the condor tools are properly setup:

> submit_dag.py

The output should be:

usage: %prog [options] [-h] [-s SOURCE | -r RUN | -l LIST] [--rerun] [--nobdt]
[--runprocs PROCS] [--cuts CUTS] [--parfile PARFILE]
[--source_type SOURCE_TYPE] [--bkg_model BKG_MODEL]
%prog [options]: error: You have to either specify an IC source (-s) or a runlist (-l)

This is the script we'll use to submit our jobs to the cluster.

Submitting your first job to the cluster

Assuming that you have the Crab run we downloaded recently (run #64080). Prepare a runlist file using your favorite text editor containing one run ID number per file. For instance, create a file called runlist.txt that contains a single line so that when you do:

> cat runlist.txt

the output should be

64080

The next step is to submit the job to the cluster, we'll use submit_dag.py for that this way:

> submit_dag.py -l runlist.txt

The output should look similar to this:

----------------------------------------
>>> Condor options <<<
rb_radius: 0.6
reco: GEO
source_type: PointSource
bdt: True
minfilesize: 200000
rerun: True
anasum_runlist_dir: /a/data/tehanu/santander/veritas/ana/anasum_runlist_dir
bkg_model: RB
source: None
re_mindist: 0.1
cuts: Soft
parfile: ANASUM.runparameter
onoff_ratio: 20
re_minareas: 2
evndis_version: v480
re_maxareas: 6
procs: ['evndisp', 'mscw', 'anasum']
condor_script_path: /a/data/tehanu/santander/veritas/ana/condor
----------------------------------------
----------------------------------------------------------------------------------------------------
Preparing to analyse run 64080...
Output file: /a/data/tehanu/santander/veritas/data/64080.root
Command: ./SPANALYSIS.evndisp_pedestal_events.sh 64080 1234 0
./SPANALYSIS.evndisp_tzeros.sh 64080 1234 0
./SPANALYSIS.evndisp.sh 64080 1234 0

EVNDISP sys: /a/data/tehanu/santander/software/EventDisplay/v480a
Condor exec file: /a/data/tehanu/santander/veritas/ana/condor/vts-execs/vts-evndisp-64080.sh
Condor output file: /a/data/tehanu/santander/veritas/ana/condor/vts-out/vts-evndisp-64080.out
Condor error file: /a/data/tehanu/santander/veritas/ana/condor/vts-error/vts-evndisp-64080.error
Condor log file: /a/data/tehanu/santander/veritas/ana/condor/vts-logs/vts-evndisp-64080.log
Output file: /a/data/tehanu/santander/veritas/data/64080.mscw.root
Getting MSCW table
/a/home/tehanu/santander/bin/getAtmonRun.sh 64080
MSCW Table: table-v480-auxv01-GRISU-SW6-ATM22-V6-GEO.root
Command: $EVNDISPSYS/bin/mscw_energy -tablefile table-v480-auxv01-GRISU-SW6-ATM22-V6-GEO.root -inputfile $VERITAS_USER_DATA_DIR/64080.root -arrayrecid=0 -writeReconstructedEventsOnly=1
EVNDISP sys: /a/data/tehanu/santander/software/EventDisplay/v480a
Condor exec file: /a/data/tehanu/santander/veritas/ana/condor/vts-execs/vts-mscw-64080.sh
Condor output file: /a/data/tehanu/santander/veritas/ana/condor/vts-out/vts-mscw-64080.out
Condor error file: /a/data/tehanu/santander/veritas/ana/condor/vts-error/vts-mscw-64080.error
Condor log file: /a/data/tehanu/santander/veritas/ana/condor/vts-logs/vts-mscw-64080.log
Output file: /a/data/tehanu/santander/veritas/data/64080.anasum.root
Creating ANASUM script
/a/home/tehanu/santander/bin/getAtmonRun.sh 64080
Gamma/hadron cut file: GammaHadron -Cut-NTel2-PointSource-Soft-TMVA-BDT.dat
Effective area file: effArea-v480-auxv01-GRISU-SW6-Cut-NTel2-PointSource-Soft-TMVA-BDT-GEO-V6-ATM22-T1234.root
Gamma/hadron cut file: GammaHadron -Cut-NTel2-PointSource-Soft-TMVA-BDT.dat
Radial acceptance file: radialAcceptance-v480-auxv01-GRISU-SW6-Cut-NTel2-Soft-GEO-V6-T1234.root
Effective area file: effArea-v480-auxv01-GRISU-SW6-Cut-NTel2-PointSource-Soft-TMVA-BDT-GEO-V6-ATM22-T1234.root
Command: $EVNDISPSYS/bin/anasum -l /a/data/tehanu/santander/veritas/ana/anasum_runlist_dir/anasum-64080.list -d $VERITAS_USER_DATA_DIR -o /a/data/tehanu/santander/veritas/data/64080.anasum.root -f ANASUM.runparameter
EVNDISP sys: /a/data/tehanu/santander/software/EventDisplay/v480a
Condor exec file: /a/data/tehanu/santander/veritas/ana/condor/vts-execs/vts-anasum-64080.sh
Condor output file: /a/data/tehanu/santander/veritas/ana/condor/vts-out/vts-anasum-64080.out
Condor error file: /a/data/tehanu/santander/veritas/ana/condor/vts-error/vts-anasum-64080.error
Condor log file: /a/data/tehanu/santander/veritas/ana/condor/vts-logs/vts-anasum-64080.log
Submitting DAG

Once this is done, you should be able to check on the status of your submission using condor_q

> condor_q $USER

More details on condor are available here.

Comments


<--/commentPlugin-->
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback