Difference between revisions of "Fast Offline Analysis Guide"
(→Production of DL3 files for a quick-look analysis) |
(→Quick-look next-day analysis) |
||
| Line 5: | Line 5: | ||
Description of the tasks for the fast offline analysis (FOA) | Description of the tasks for the fast offline analysis (FOA) | ||
| + | |||
| + | Perform the analysis of the data from ToO/alert observations (AGN ToO, Galactic transient sources, ) | ||
| Line 29: | Line 31: | ||
=== 1. Selection of the DL1 real data files === | === 1. Selection of the DL1 real data files === | ||
| + | |||
| + | Use the notebook [https://github.com/cta-observatory/cta-lstchain/blob/main/notebooks/data_quality.ipynb cta-lstchain/notebooks/data_quality.ipynb] | ||
| + | |||
| + | *The version of the notebook used at the school (from lstchain v0.10.7) is at: | ||
| + | /fefs/aswg/workspace/analysis-school-2024/ | ||
| + | |||
| + | *Copy it to a directory of your choice in the IT cluster, and open it with jupyter notebook. (check these instructions if you don’t know how to execute a jupyter notebook on a remote machine) | ||
| + | *Look for the “USER INPUT” cells in the notebook and do the following: | ||
| + | *In the “path to the necessary datacheck files” cell, set “files” to load all the datacheck files of 2023 data processed with version v0.10 of lstchain (explained in the notebook) | ||
| + | *check that the source is set to “Crab Nebula” | ||
| + | *set max_zenith = 35 * u.deg | ||
| + | *set first_date = 20230117 | ||
| + | *set last_date = 20230117 | ||
| + | |||
| + | This will select Crab observations (in standard wobble mode) up to 35 deg in zenith taken | ||
| + | in the night of Jan 17, 2023. At the end of the notebook you will get a list of “good runs”: | ||
| + | |||
| + | /fefs/aswg/data/real/DL1/20230117/v0.10/tailcut84/dl1_LST-1.Run11699.h5 | ||
| + | /fefs/aswg/data/real/DL1/20230117/v0.10/tailcut84/dl1_LST-1.Run11700.h5 | ||
| + | /fefs/aswg/data/real/DL1/20230117/v0.10/tailcut84/dl1_LST-1.Run11701.h5 | ||
| + | /fefs/aswg/data/real/DL1/20230117/v0.10/tailcut84/dl1_LST-1.Run11702.h5 | ||
| + | /fefs/aswg/data/real/DL1/20230117/v0.10/tailcut84/dl1_LST-1.Run11703.h5 | ||
| + | |||
| + | Just copy this list and using a text editor paste it into a file called “file_list.txt” | ||
| + | that you will need later (yes, I know, the file could be written by the notebook directly…). | ||
| + | If you want to select data from any time in the whole sample you have to change the input | ||
| + | in the notebook cell “path to the necessary datacheck files”. The directory below | ||
| + | contains links to (the most recent version of) all datacheck files: | ||
| + | /fefs/aswg/data/real/OSA/DL1DataCheck_LongTerm/night_wise/all/ | ||
| + | |||
| + | *See this school session for more details on the data quality selection | ||
=== 2. Production of the real data DL2 files === | === 2. Production of the real data DL2 files === | ||
Revision as of 17:42, 13 April 2024
Contents
Quick-look next-day analysis
Work in progress...
Description of the tasks for the fast offline analysis (FOA)
Perform the analysis of the data from ToO/alert observations (AGN ToO, Galactic transient sources, )
Production of DL3 files for a quick-look analysis
Analysis guide made by Abelardo Moralejo (IFAE) after the LST-1 Analysis 2024 School
Outline of the procedure:
1. Start from DL1 files automatically produced by the on-site analysis, LSTOSA (each night's data is typically ready by noon next day)
2. Select the relevant files for a given source, period and zenith distance range
3. Use already existing standard RF models to create DL2 real data files
4. Use the already existing MC test DL2 files (corresponding to the same "production" as the RF models) to create the IRF files
5. Produce the DL3 files
1. Selection of the DL1 real data files
Use the notebook cta-lstchain/notebooks/data_quality.ipynb
- The version of the notebook used at the school (from lstchain v0.10.7) is at:
/fefs/aswg/workspace/analysis-school-2024/
- Copy it to a directory of your choice in the IT cluster, and open it with jupyter notebook. (check these instructions if you don’t know how to execute a jupyter notebook on a remote machine)
- Look for the “USER INPUT” cells in the notebook and do the following:
- In the “path to the necessary datacheck files” cell, set “files” to load all the datacheck files of 2023 data processed with version v0.10 of lstchain (explained in the notebook)
- check that the source is set to “Crab Nebula”
- set max_zenith = 35 * u.deg
- set first_date = 20230117
- set last_date = 20230117
This will select Crab observations (in standard wobble mode) up to 35 deg in zenith taken in the night of Jan 17, 2023. At the end of the notebook you will get a list of “good runs”:
/fefs/aswg/data/real/DL1/20230117/v0.10/tailcut84/dl1_LST-1.Run11699.h5 /fefs/aswg/data/real/DL1/20230117/v0.10/tailcut84/dl1_LST-1.Run11700.h5 /fefs/aswg/data/real/DL1/20230117/v0.10/tailcut84/dl1_LST-1.Run11701.h5 /fefs/aswg/data/real/DL1/20230117/v0.10/tailcut84/dl1_LST-1.Run11702.h5 /fefs/aswg/data/real/DL1/20230117/v0.10/tailcut84/dl1_LST-1.Run11703.h5
Just copy this list and using a text editor paste it into a file called “file_list.txt” that you will need later (yes, I know, the file could be written by the notebook directly…). If you want to select data from any time in the whole sample you have to change the input in the notebook cell “path to the necessary datacheck files”. The directory below contains links to (the most recent version of) all datacheck files:
/fefs/aswg/data/real/OSA/DL1DataCheck_LongTerm/night_wise/all/
- See this school session for more details on the data quality selection
