Difference: FermiPyAnalysis (1 vs. 6)

Revision 62020-02-27 - AriBrill

Line: 1 to 1
 
META TOPICPARENT name="FermiAnalysis"

Fermi Analysis with FermiPy

Line: 6 to 6
 

Installation and Setup

Changed:
<
<
1. Set up your environment for using the Fermipy pipeline as follows. Open the file .myprofile in your home directory in a text editor and add the following lines:
>
>
The following instructions will set up an installation of the Fermi Science Tools (fermitools) and Fermipy using conda. Conda is a package manager that will automatically manage all of the dependencies needed to use the software and encapsulate them into an environment, which must be activated before use.

First, set up your environment for using the Fermipy pipeline by running the following commands:

 
Changed:
<
<
export FERMI_ANALYSIS_DIR=/a/data/tehanu/$USER/fermi_analysis export FERMIPIPE=/a/home/tehanu/brill/fermipipe alias fermisetup="source $FERMIPIPE/setup_fermi.sh" alias fermianalysis="nohup python $FERMIPIPE/run_analysis.py $FERMI_ANALYSIS_DIR/pipeline_config.yml &"
>
>
cat /a/home/tehanu/brill/fermipipe/bash_setup.txt >> ~/.myprofile source ~/.bashrc
 
Changed:
<
<
For these to take effect, either run source ~/.bashrc, or log out and log back in again.
>
>
In addition to initializing the conda environment for the fermi code for your account, this will provide you with several definitions and commands (you can see them by opening with a text editor the file .myprofile in your home directory).
 
Changed:
<
<
NOTE: If your home directory is on Milne or another Nevis machine other than tehanu, see the special instructions "Customizing the Installation" below. If you're unsure where your home directory is, type echo $HOME.
>
>
FERMI_ANALYSIS_DIR=/a/data/tehanu/$USER/fermi_analysis: directory where your analysis will occur by default
 
Changed:
<
<
2. Install the Fermi Science Tools (fermitools) and Fermipy using conda. Conda is a package manager that will automatically manage all of the dependencies needed to use the software and encapsulate them into an environment, which must be activated before use. The setup_pipeline script will download and install fermitools and fermipy into an environment named "fermi". By default, the software will be installed to /a/data/tehanu/$USER/miniconda3, where $USER is your username. Run the following command anywhere:
>
>
fermisetup: set up your environment before running analyses
 
Changed:
<
<
$FERMIPIPE/setup_pipeline.sh
>
>
fermianalysis="nohup python $FERMIPIPE/run_analysis.py $FERMI_ANALYSIS_DIR/pipeline_config.yml &": run an analysis
 
Changed:
<
<
The script should take several minutes to run. Ignore any message saying to run source ~/.bashrc again, the script does that for you automatically.
>
>
NOTE: If your home directory is on Milne or another Nevis machine other than tehanu, see the special instructions "Customizing the Installation" below. If you're unsure where your home directory is, type echo $HOME.
 
Changed:
<
<
3. Create a directory for running your analysis, and copy the configuration file templates there:
>
>
Next, create a directory for running your analysis, and copy the configuration file templates there:
 
mkdir $FERMI_ANALYSIS_DIR
Line: 37 to 35
  Customizing the Installation
Changed:
<
<
If you'd like to perform your analysis somewhere besides the tehanu data partition, in step 1, change the FERMI_ANALYSIS_DIR to your preferred directory. For example, to perform the analysis in your home directory (recommended if your account is on milne), write:
>
>
If you'd like to perform your analysis somewhere besides the tehanu data partition, change the FERMI_ANALYSIS_DIR to your preferred directory in your .myprofile file. For example, to perform the analysis in your home directory (recommended if your account is on milne), write:
 
export FERMI_ANALYSIS_DIR=$HOME/fermi_analysis
Changed:
<
<
If you'd like to change the installation path of miniconda, in step 2, run the setup script using the -p flag to change the path, as follows:
>
>
If you need to recreate the conda environment from scratch, use the setup_pipeline script to download and install fermitools and fermipy into an environment named "fermi". By default, the software will be installed to /a/data/tehanu/$USER/miniconda3, where $USER is your username. Run the following command anywhere:

$FERMIPIPE/setup_pipeline.sh

The script should take several minutes to run. Ignore any message saying to run source ~/.bashrc again, the script does that for you automatically.

If you'd like to change the installation path of miniconda, run the setup script using the -p flag to change the path, as follows:

 
$FERMIPIPE/setup_pipeline.sh -p <CONDA_PATH>
Line: 65 to 71
 

Running Your Analysis

Changed:
<
<
Command to set up for running the analysis, including activating the conda environment:
>
>
First, set your environment and activate the conda environment:
 
fermisetup
Changed:
<
<
Command to run the analysis using your config files:
>
>
Now you can run an analysis using your config files:
 
fermianalysis

Revision 52020-02-19 - AriBrill

Line: 1 to 1
 
META TOPICPARENT name="FermiAnalysis"

Fermi Analysis with FermiPy

Line: 10 to 10
 
export FERMI_ANALYSIS_DIR=/a/data/tehanu/$USER/fermi_analysis
Changed:
<
<
export FERMIPIPE=/a/home/tehanu/brill/fermi_pipeline
>
>
export FERMIPIPE=/a/home/tehanu/brill/fermipipe
 alias fermisetup="source $FERMIPIPE/setup_fermi.sh" alias fermianalysis="nohup python $FERMIPIPE/run_analysis.py $FERMI_ANALYSIS_DIR/pipeline_config.yml &"

Revision 42020-02-19 - AriBrill

Line: 1 to 1
 
META TOPICPARENT name="FermiAnalysis"

Fermi Analysis with FermiPy

Line: 27 to 27
  The script should take several minutes to run. Ignore any message saying to run source ~/.bashrc again, the script does that for you automatically.
Changed:
<
<
3. Create a directory for running your analysis, and copy the configuration templates there:
>
>
3. Create a directory for running your analysis, and copy the configuration file templates there:
 
mkdir $FERMI_ANALYSIS_DIR
Line: 51 to 51
  For example, to install to your home directory (recommended for milne), use -p $HOME/miniconda3. You can also use this option to create the fermi environment in an existing conda installation, if you have one, provided it doesn't already have an environment named fermi.
Changed:
<
<

Configuring an Analysis

>
>

Configuring Your Analysis

 
Changed:
<
<
Text
>
>
When running an analysis with Fermipy, all parameters are set using a configuration file. Different sections contains the parameters for the data selection, likelihood fit, source model, and so on. There are many configuration parameters you can adjust when running an analysis. In your analysis directory, you should have a file called config.yml, which you can use as a template for your Fermipy configuration file. The settings in it are meant for a standard setup, which you can adjust as needed. The Fermipy documentation lists all possible configuration settings.
 
Changed:
<
<

Running an Analysis

>
>
To perform an analysis, you will have to set a minimum of three parameters: the start time, the stop time, and the name of the target to analyze (if your target is not a known Fermi source, you can define the target using coordinates instead). The analysis start and stop times must be provided in Fermi mission elapsed time (MET). NASA provides a handy time converter that you can use to convert between a number of calendar/time formats and MET.
 
Changed:
<
<
Set up for running the Fermi analysis, including activating the conda environment:
>
>
The pipeline around Fermipy is designed in a similar way. In your analysis directory, you should have a file called pipeline_config.yml, which you can use as a template for your pipeline configuration file.

fermipy_config: the name of your Fermipy config file

prefix: the name to be used for the output directory (within the fermi_analysis directory) and as a prefix for your output files; shouldn't contain spaces

Running Your Analysis

Command to set up for running the analysis, including activating the conda environment:

 
fermisetup
Changed:
<
<
Run the analysis using your config files:
>
>
Command to run the analysis using your config files:
 
fermianalysis
Changed:
<
<
The fermianalysis command runs the analysis in the background, using nohup. This means that it will continue to run even if you close your terminal, and that instead of printing the output to the screen, saves it to a file called nohup.out.
>
>
The fermianalysis command runs the analysis in the background, using nohup. This means that it will continue to run even if you close your terminal, and that instead of printing the output to the screen, it saves it to a file called nohup.out.

Useful links

 
Changed:
<
<

Links

>
>
Fermipy documentation
  Fermipy Tutorials from Fermi Summer School

Revision 32020-02-19 - AriBrill

Line: 1 to 1
 
META TOPICPARENT name="FermiAnalysis"

Fermi Analysis with FermiPy

Line: 25 to 25
 $FERMIPIPE/setup_pipeline.sh
Changed:
<
<
The script should take several minutes to run.
>
>
The script should take several minutes to run. Ignore any message saying to run source ~/.bashrc again, the script does that for you automatically.
  3. Create a directory for running your analysis, and copy the configuration templates there:
Line: 57 to 57
 

Running an Analysis

Changed:
<
<
Text
>
>
Set up for running the Fermi analysis, including activating the conda environment:

fermisetup

Run the analysis using your config files:

fermianalysis

The fermianalysis command runs the analysis in the background, using nohup. This means that it will continue to run even if you close your terminal, and that instead of printing the output to the screen, saves it to a file called nohup.out.

 

Links

Changed:
<
<
Fermi Summer School FermiPy Tutorials
>
>
Fermipy Tutorials from Fermi Summer School

NASA mission time converter

Managing conda environments

  -- Ari Brill - 2020-02-17

Revision 22020-02-18 - AriBrill

Line: 1 to 1
 
META TOPICPARENT name="FermiAnalysis"

Fermi Analysis with FermiPy

Line: 9 to 9
 1. Set up your environment for using the Fermipy pipeline as follows. Open the file .myprofile in your home directory in a text editor and add the following lines:
Deleted:
<
<
export FERMIPIPE=/a/home/tehanu/brill/fermi_pipeline
 export FERMI_ANALYSIS_DIR=/a/data/tehanu/$USER/fermi_analysis
Added:
>
>
export FERMIPIPE=/a/home/tehanu/brill/fermi_pipeline
 alias fermisetup="source $FERMIPIPE/setup_fermi.sh" alias fermianalysis="nohup python $FERMIPIPE/run_analysis.py $FERMI_ANALYSIS_DIR/pipeline_config.yml &"

For these to take effect, either run source ~/.bashrc, or log out and log back in again.

Changed:
<
<
2. Install the Fermi Science Tools (fermitools) and Fermipy using conda. Conda is a package manager that will automatically manage all of the dependencies needed to use the software and encapsulate them into an environment, which must be activated before use. The following command will download and install fermitools and fermipy into an environment named "fermi", located in /a/data/tehanu/$USER/miniconda3, where $USER is your username:
>
>
NOTE: If your home directory is on Milne or another Nevis machine other than tehanu, see the special instructions "Customizing the Installation" below. If you're unsure where your home directory is, type echo $HOME.

2. Install the Fermi Science Tools (fermitools) and Fermipy using conda. Conda is a package manager that will automatically manage all of the dependencies needed to use the software and encapsulate them into an environment, which must be activated before use. The setup_pipeline script will download and install fermitools and fermipy into an environment named "fermi". By default, the software will be installed to /a/data/tehanu/$USER/miniconda3, where $USER is your username. Run the following command anywhere:

 
$FERMIPIPE/setup_pipeline.sh
Added:
>
>
The script should take several minutes to run.
 3. Create a directory for running your analysis, and copy the configuration templates there:
Line: 31 to 35
 cp $FERMIPIPE/pipeline_config.yml $FERMI_ANALYSIS_DIR
Added:
>
>
Customizing the Installation

If you'd like to perform your analysis somewhere besides the tehanu data partition, in step 1, change the FERMI_ANALYSIS_DIR to your preferred directory. For example, to perform the analysis in your home directory (recommended if your account is on milne), write:

export FERMI_ANALYSIS_DIR=$HOME/fermi_analysis

If you'd like to change the installation path of miniconda, in step 2, run the setup script using the -p flag to change the path, as follows:

$FERMIPIPE/setup_pipeline.sh -p <CONDA_PATH>

For example, to install to your home directory (recommended for milne), use -p $HOME/miniconda3. You can also use this option to create the fermi environment in an existing conda installation, if you have one, provided it doesn't already have an environment named fermi.

 

Configuring an Analysis

Text

Revision 12020-02-17 - AriBrill

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="FermiAnalysis"

Fermi Analysis with FermiPy


Installation and Setup

1. Set up your environment for using the Fermipy pipeline as follows. Open the file .myprofile in your home directory in a text editor and add the following lines:

export FERMIPIPE=/a/home/tehanu/brill/fermi_pipeline
export FERMI_ANALYSIS_DIR=/a/data/tehanu/$USER/fermi_analysis
alias fermisetup="source $FERMIPIPE/setup_fermi.sh"
alias fermianalysis="nohup python $FERMIPIPE/run_analysis.py $FERMI_ANALYSIS_DIR/pipeline_config.yml &"

For these to take effect, either run source ~/.bashrc, or log out and log back in again.

2. Install the Fermi Science Tools (fermitools) and Fermipy using conda. Conda is a package manager that will automatically manage all of the dependencies needed to use the software and encapsulate them into an environment, which must be activated before use. The following command will download and install fermitools and fermipy into an environment named "fermi", located in /a/data/tehanu/$USER/miniconda3, where $USER is your username:

$FERMIPIPE/setup_pipeline.sh

3. Create a directory for running your analysis, and copy the configuration templates there:

mkdir $FERMI_ANALYSIS_DIR
cp $FERMIPIPE/config.yml $FERMI_ANALYSIS_DIR
cp $FERMIPIPE/pipeline_config.yml $FERMI_ANALYSIS_DIR

Configuring an Analysis

Text

Running an Analysis

Text

Links

Fermi Summer School FermiPy Tutorials

-- Ari Brill - 2020-02-17

Comments

<--/commentPlugin-->
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback