Research Journal: Jamil Batshoun

CTA simulations with gammapy

Questions

1. (Answered during 7/6 meeting)July 2: In section 5 of the paper they talk about optimizing the analysis using pSCT data alone. I'm not sure where they got the significance for the full dataset described in Table 3 (page 9). The analysis was first optimized using 2.2 hours of concurrent VERITAS and pSCT, but there is at least 17.6 of ON/OFF obsersvations. Is the 8.6 significance derived from the 2.2 hours of data or from the 17.6 hours based on analysis optimized from the 2.2 hours?
2. June 30: Does "ZEN_PNT" refer to the zenith angle?
3. June 29: What does gti/GTI mean in the case of the HDU_TYPE/NAME?

July 7, 2021

1. What was done:
   1. Continued to read DC-1 closeout document introduction, gps, and gammapy sections
   2. Tried to look at new irf files downloaded 9 days ago.
2. Things left to do:
3. Notes:
   1. GAMMAPY_DATA environment variable does not contain the right path despite export line in .myprofile being correct.

1. What was done:
   1. Nevis VERITAS/CTA meeting at 4pm
   2. Read DC-1 closeout document introduction, gps, and gammapy section
2. Things left to do:
   1. Put my slides on the twiki (requested by Reshmi)
3. Notes:
   1. Noticed that fitsviewer and gammapy(the code is in cta.ipynb) can both display the information inside the obs & hdu index fits files.
   2. DC1 document notes that irf files for the cta 1dc are in a CALDB folder which I checked and confirmed. Also in that folder are the irf files that Massimo downloaded for me.

1. What was done:
   1. Finished reading "Detection of the Crab Nebula with the 9.7m prototype Schwarzschild-Couder telescope".
2. Things left to do:
3. Notes:
   1. Reading: They used 17.6 hours of concurrent ON/OFF observations from the pSCT alone to optimize the analysis further. They were able to get a final significance of 7.3. This compares to 8.6 from the full dataset. I'm not sure what counts as the full dataset though.

1. What was done:
   1. Started reading "Detection of the Crab Nebula with the 9.7m prototype Schwarzschild-Couder telescope".
2. Things left to do:
   1. Finish reading paper and take notes.
3. Notes:

1. What was done:
   1. Took notes on the obs_index file
2. Things left to do:
   1. check-in with Massimo
3. Notes:
   1. obs_index notes(again found after pressing the all button under the table column):
      1. Column names: OBS_ID, RA_PNT, DEC_PNT, GLON_PNT, GLAT_PNT ZEN_PNT
      2. OBS_ID: again the observation number
      3. RA_PNT: assuming this is the RA the telescope is pointing at
      4. DEC_PNT: assuming the same as above but for the declination
      5. GLON_PNT: assuming this is the galactic longitude pointing direction
      6. GLAT_PNT: assuming this is galactic latitude
      7. ZEN_PNT: assuming this is the zenith angle pointing direction but i'm not sure
4. Questions:
   1. Does "ZEN_PNT" refer to the zenith angle?

1. What was done:
   1. Learned how to start fitsviewer-----> ./fv #must be inside fv5.5.2 directory
   2. Took notes on what I saw in hdu_index
   3. Made a text file version of the fits file.
2. Things left to do:
   1. Take notes on obs_index.fits.gz
3. Notes:
   1. Was not able to use the ./fv outside of the fv5.5.2 directory. Possibly can only start fitsviewer from that directory. This must be because it's a script and not an actual linux command.
   2. hdu_index notes (from pressing the all button under the table column):
      1. Column names: OBS_ID, HDU_TYPE, HDU_CLASS, FILE_DIR, FILE_NAME, HDU_NAME
      2. OBS_ID: the observation number
      3. HDU_TYPE possible values: events, gti, aeff, psf, bkg
      4. HDU_CLASS possible values: events, gti, aeff_2d, edisp_2d, psf_3gauss, bkg_3d
      5. FILE_NAME: so far i only see some combination of irf_file.fits and gps_baseline_ (observation number here).fits
      6. HDU_NAME possible values: EVENTS, GTI, EFFECTIVE AREA, ENERGY DISPERSION, POINT SPREAD FUNCTION, BACKGROUND
4. Questions:
   1. What does gti/GTI mean in the case of the HDU_TYPE/NAME?

1. What was done:
   1. Met with Massimo to copy cta files over to my directory and had ds9 set up to view fits files.
      1. Massimo sent me an email with a pdf of the CTA DC1 to read.
   2. Nevis VERITAS/CTA meeting
   3. installed fitsviewer.
2. Things left to do:
   1. check index files using fitsviewer. Take notes on these.
3. Notes:
   1. I'm not currently sure about how to start fitsviewer. I'm going to check the fitsviewer website for help.

1. What was done:
   1. Completed tutorials:
      1. Overview – overview.ipynb
         CTA with gammapy – cta.ipynb
         3D map simulation – simulate_3d.ipynb
         1D spectrum simulation – spectrum_simulation.ipynb
         Point source sensitivity – cta_sensitivity.ipynb
   2. Went through some exercises in the tutorials. Included plots I added as attatchments to this page.
   3. Attended VERITAS summer meeting. Went to blazar science and ACG talks.
2. Things left to do:
   1. Complete exercises from the tutorials i've completed so far to practice what they covered.
   2. Will go through irf section of cta.ipynb again to take notes on the plots the code produces.
   3. Need to complete taking notes on the papers I have read/need to read so far.

1. What was done today:
   1. Attended 1st half of today's session of the VERITAS summer meeting.
   2. Attended REU talk
   3. Took notes on cta.ipynb. Worked on exercises in overview.ipynb but struggled getting through the first exercise.
      1. Made a sky map of hess observations!
2. Things left to do:
   1. Finish notes on cta.ipynb
3. Notes
   1. overview.ipynb exercise notes: my best guess is that i am supposed to use SphericalCircle from astropy.visualization.wcsaxes.
   2. ^^^Figured it out

1. What was done today:
   1. read about fits, found the standard paper for fits
   2. read about the icrs, dec, and ra
   3. looked for a formal definition of the dl3-dr1 data from hess. did not find much info about what a data level is
   4. ran into same issue with the path associated with the GAMMAPY_DATA environment variable
      1. I fixed this by adding a line to my .profile on the nevis computers. I don't see this becoming a problem again.
   5. completed tutorial 2
2. What is left to do:
   1. Read up on background modeling.
   2. Understand what the classes of gammapy do
   3. get a better understanding of how the analysis (both high level and lower level gammapy api)

1. What was done today:
   1. Went over again the analysis steps found in tutorial "analysis 1"
2. Things left to do:
   1. started working through tutorial "overview"
3. Notes:
   1. How higher-level analysis works in gammapy:
      1. tutorial intro intro definition:
         1. (data reduction): observations reduced to a specified geometry (1D/3D). Combined with IRFS, this makes a dataset
         2. (Analysis): set a physical model and fit it to the datasets
      2. definition based on code:
         1. create config object for the settings used in your analysis
         2. set config object's type, stack, geom, background, fit, and flux_point settings settings
         3. (optional) write config object to a YAML file to use later.
         4. select observations to use
         5. data reduction
         6. (optional) save dataset to disk
         7. create model_config object with model settings
         8. set model_config object to analysis object
         9. use run_fit on analysis object
         10. create model best fit yaml file.
         11. estimate flux points
   2. questions:
      1. what is the model-best-fit.yaml file for? what about the model-best-fit_covariance.dat file?
      2. is data really reduced?

1. What was done today:
   1. Added gammapy resources to .myprofile file
   2. started working on gammapy tutorials
      1. code to start tutorials:

         1. ssh jbatshoun@tehanu.nevis.columbia.edu -L 9888:localhost:9888

         2. module load gammapy/0.18
         3. gammapy_start
         4. jupyter notebook --no-browser --port=9888
         5. copy & paste resulting link to chrome and use jupyter notebook normally
      2. COMPLETED: 1st gammapy tutorial analysis_1.ipynb
2. Things left to do:
   1. create a .myprofile and copy contents of .profile into it.
   2. Remove gammapy line in .profile so that it is the same as when I first started (This might be unnecessary).
3. Notes:
   1. Useful commands
      1. To check where the "juypter notebook" command is being used from (i.e. gammapy vs python): which jupyter-notebook
   2. Made an error when executing "export $GAMMAPY_DATA". executed in jbatshoun directory and not datasets directory. Noticed in tutorial what should have been the real path and made corrections.

1. What was done today:
2. Things Left to do:
3. Notes:
   1. aa5351-06: Is the IRF a good measure of telescope performance? what about the psf?

1. Read through gammapy and cta explanation of IRFs
2. Read gammapy overview

June 7, 2021 (Meeting Massimo, Jamil)

1. Introduction to project
2. To-do list
   1. https://docs.gammapy.org/0.18.2/
   2. Read overview carefully
   3. Installation: no need to worry about it, we have shared tools on the nevis cluster
      1. log into the nevis cluster
      2. module load gammapy/0.18
      3. gammapy_start
   4. download tutorial datasets (gammapy download tutorials --release 0.18.2 , export GAMMAPY_DATA=<>)
   5. go through tutorials
3. Weekly meeting with Massimo: Thursdays, 11 AM

-- Massimo Capasso - 2021-06-27