LST1Analysis MVA2
LST1 Analysis after Software School 2024
Contents
MVA Analysis pages
Creating DL3 in the La Palma IT cluster
School scripts/notebooks: /fefs/aswg/workspace/analysis-school-2024/cta-lstchain
NOTE: the home directory has limited quota. Should use /fefs/aswg/workspace/monica.vazquez for the output instead!
Open remote connection
ssh cp03 source /fefs/aswg/software/conda/etc/profile.d/conda.sh conda activate lstchain-v0.10.7 jupyter notebook --no-browser Link for browser in printout: http://localhost:88XX/?token=YYYYYYYYYYYY In another terminal: ssh -NfL localhost:88XX:localhost:88XX monica.vazquez@cp03
Select good quality data
Run notebook: /home/monica.vazquez/LST1SCHOOL2024/notebooks/data_quality.ipynb
data_quality notebook settings to select crab data taken in the night 2023/01/17:
files = glob.glob("/fefs/aswg/data/real/OSA/DL1DataCheck_LongTerm/v0.10/2023????/DL1_datacheck_2023*.h5")
source_coordinates = SkyCoord.from_name("Crab Nebula")
max_zenith = 35 * u.deg
first_date = 20230117
last_date = 20230117
To check the most recent version of all the data use this location:
/fefs/aswg/data/real/OSA/DL1DataCheck_LongTerm/night_wise/all/
Copy the output good run list:
/fefs/aswg/data/real/DL1/20230117/v0.10/tailcut84/dl1_LST-1.Run11699.h5 /fefs/aswg/data/real/DL1/20230117/v0.10/tailcut84/dl1_LST-1.Run11700.h5 /fefs/aswg/data/real/DL1/20230117/v0.10/tailcut84/dl1_LST-1.Run11701.h5 /fefs/aswg/data/real/DL1/20230117/v0.10/tailcut84/dl1_LST-1.Run11702.h5 /fefs/aswg/data/real/DL1/20230117/v0.10/tailcut84/dl1_LST-1.Run11703.h5
to /home/monica.vazquez/LST1SCHOOL2024/helpful_scripts/file_list.txt
DL1 to DL2
cd /home/monica.vazquez/LST1SCHOOL2024/helpful_scripts ./launch_dl1_dl2.sh /fefs/aswg/data/models/AllSky/20230927_v0.10.4_crab_tuned/dec_2276 /fefs/aswg/data/models/AllSky/20230927_v0.10.4_crab_tuned/dec_2276/lstchain_config_2023-09-27.json # check batch job status squeue –u monica.vazquez # check they completed well sacct -u monica.vazquez
Processing v0.9 DL1 data
Process v0.9 DL1 files as if they were v0.10. Use lstchain v0.10 for the analysis, and v0.10 versions of the MC products (RF models and DL2).
Recommended MC trainings
Model files are located in: /fefs/aswg/data/models/AllSky/
| Recommended trainings | ||
|---|---|---|
|
Source |
Model |
json |
| Crab Nebula tuned | 20230927_v0.10.4_crab_tuned | lstchain_config_2023-09-27.json |
| Standard Dark complete production | 20230901_v0.10.4_allsky_base_prod | lstchain_config_2023-09-01.json |
| Standard Dark recent/incomplete! | 20240131_allsky_v0.10.5_all_dec_base | lstchain_config_2024-01-31.json |
Exploration of DL2 data
Run notebook: /home/monica.vazquez/LST1SCHOOL2024/notebooks/explore_DL2.ipynb
IRF creation
Use the test MC of the same declination line used in the DL2 step
cd /home/monica.vazquez/LST1SCHOOL2024/helpful_scripts cp /fefs/aswg/workspace/analysis-school-2024/cta-lstchain/docs/examples/irf_tool_config.json .
The default json has energy dependent gammaness and theta containment cuts of 70%
"gh_efficiency": 0.7 "theta_containment": 0.7
cd /home/monica.vazquez/LST1SCHOOL2024/helpful_scripts mkdir IRF DL2MC=/fefs/aswg/data/mc/DL2/AllSky/20230927_v0.10.4_crab_tuned/ ./launch_irf_creation.sh $DL2MC/TestingDataset/dec_2276 ./irf_tool_config.json gh_eff_0.7_th_cont_0.7
DL2 to DL3
cd /home/monica.vazquez/LST1SCHOOL2024/helpful_scripts cp /fefs/aswg/workspace/analysis-school-2024/cta-lstchain/docs/examples/irf_dl3_tool_config.json . IRFDIR=IRF/20230927_v0.10.4_crab_tuned/dec_2276/gh_eff_0.7_th_cont_0.7 ./launch_dl2_dl3.sh irf_dl3_tool_config.json ./DL2 $IRFDIR Crab\ Nebula
(if source name has a blank space you have to escape it, using a backslash)
Create DL3 index files
cd /home/monica.vazquez/LST1SCHOOL2024/helpful_scripts/DL3 lstchain_create_dl3_index_files -d ./DL3 -p dl3\*.fits.gz
In case the DL3 are unzipped, redo index file creation command:
gzip -d ./DL3/dl3\*.fits.gz lstchain_create_dl3_index_files --overwrite -d ./DL3 -p dl3\*.fits
Output files to be used together with DL3 in gammapy:
hdu-index.fits.gz and obs-index.fits.gz
Running DL3 gammapy analysis locally
#mamba create -c conda-forge -n lstchain-v0.10.7 python=3.11 lstchain=0.10.7 conda activate lstchain-v0.10.7 #transfer the DL3 files to laptop scp cp03:/home/monica.vazquez/LST1SCHOOL2024/helpful_scripts/DL3/*fits* /Users/monicava/Documents/CODE/LST1ANALYSIS/DL3/Crab_20230117
Run locally the notebooks:
- Theta2: plot_theta2_from_dl3.ipynb
Change/check:
base_dir = Path("/Users/monicava/Documents/CODE/LST1ANALYSIS/DL3")
label="Crab_20230117"
dl3_path = base_dir / label
target_position = SkyCoord.from_name("Crab Nebula")
- SED & LC: post_DL3_analysis.ipynb
Change/check:
base_dir = Path("/Users/monicava/Documents/CODE/LST1ANALYSIS/DL3")
label="Crab_20230117"
dl3_path = base_dir / label
filename = dl3_path / "dl3_LST-1.Run11699.fits"
target_position = SkyCoord.from_name("Crab", frame='icrs')
