Difference between revisions of "OP313 Dec2023 Jan2024 LST1 analysis details"
(→Systematic on the energy scale) |
(→Systematic on the energy scale) |
||
| (9 intermediate revisions by the same user not shown) | |||
| Line 19: | Line 19: | ||
* 2024-02-16 | OP 313 meeting: https://indico.cta-observatory.org/event/5412/contributions/44159/attachments/25331/37042/OP313_LST_20240215.pdf | * 2024-02-16 | OP 313 meeting: https://indico.cta-observatory.org/event/5412/contributions/44159/attachments/25331/37042/OP313_LST_20240215.pdf | ||
* 2024-02-19 | LST analysis call: https://indico.cta-observatory.org/event/5418/contributions/44198/attachments/25346/37068/OP313_LSTreco_20240219-1.pdf | * 2024-02-19 | LST analysis call: https://indico.cta-observatory.org/event/5418/contributions/44198/attachments/25346/37068/OP313_LSTreco_20240219-1.pdf | ||
| − | + | * 2024-05-21 | LST general meeting (1/2): https://indico.cta-observatory.org/event/5371/contributions/45583/attachments/25909/38070/OP313_plan_collMeeting.pdf | |
| + | * 2024-05-21 | LST general meeting (2/2): https://indico.cta-observatory.org/event/5371/contributions/45583/attachments/25909/38061/OP313_LST_GM_20240520_v1.pdf | ||
=== Data-taking information === | === Data-taking information === | ||
* NSB level: all data was taken in dark NSB conditions | * NSB level: all data was taken in dark NSB conditions | ||
| − | * Zenith angles: | + | * Zenith angles: 5-54 deg (December 2023 data have zenith angle > 30 deg) |
=== Good-quality selected dataset === | === Good-quality selected dataset === | ||
| Line 150: | Line 151: | ||
== Systematic on the energy scale == | == Systematic on the energy scale == | ||
| − | |||
| − | Test the effect of a +/- | + | Test the effect of a +/-15% shift on a possible bias on the absolute energy scale between simulations and observed data. This is the typical value used in currents IACTs and also it was found by Federico Di Pierro et al. to be the energy scale mismatch between LST-1 and MAGIC. |
This can be done at different analysis levels: | This can be done at different analysis levels: | ||
* Post DL3 in Gammapy (e.g. have a look at https://github.com/bkhelifi/CTAO-CTAC_Meeting_Granada_2023/blob/main/Crab_simulations_systematic_errors_Solution.ipynb) | * Post DL3 in Gammapy (e.g. have a look at https://github.com/bkhelifi/CTAO-CTAC_Meeting_Granada_2023/blob/main/Crab_simulations_systematic_errors_Solution.ipynb) | ||
| − | * | + | [[File:SED_OP313_energy_scale_gammapy.png|900px]] |
| − | + | ||
| + | * Scaling of true energy in IRFs by ±15% | ||
| + | |||
| + | [[File:SED_OP313_energy_scale_pyirf.png|900px]] | ||
| + | |||
| + | {| class="wikitable" | ||
| + | |+ Best-fit spectral parameters after scaling the true energy by ±15% | ||
| + | |- | ||
| + | ! Energy true scale !! Index !! Amplitude [x 10<sup>-8</sup> TeV<sup>-1</sup> s<sup>-1</sup> cm<sup>-2</sup>] | ||
| + | |- | ||
| + | | +15% || 1.78 ± 0.55 || 1.80 ± 0.30 | ||
| + | |- | ||
| + | | -15% || 2.57 ± 0.38 || 0.60 ± 0.10 | ||
| + | |} | ||
| + | |||
| − | + | * DL1: scaling the pixel charges | |
| + | Using rescale_dl1_charge | ||
| + | |||
| + | Analysis changing the amount of light. Trigger and fluctuations are not affected, but all the rest, starting from cleaning are. Better to try this first before producing low-level MC. | ||
| + | |||
| + | * Simtel files (simulating new MC productions scaling the light yield, e.g. +/- 15 %): before doing this test, first try the approach at DL1 level scaling the charge. | ||
== Test on MAGIC vs LST-1 with the same common GTIs == | == Test on MAGIC vs LST-1 with the same common GTIs == | ||
| Line 169: | Line 188: | ||
[[File:OP313_same_GTIs_SED.png|600px]] | [[File:OP313_same_GTIs_SED.png|600px]] | ||
| + | |||
| + | In the common GTI light curve there is one night (mjd 60296) in which MAGIC has a UL, and LST a point. And LST's excess is actually pretty high (TS = 18.0 +/- 4.2, vs TS=0.58+/-0.77 in MAGIC). Mismatch to be investigated. | ||
| + | |||
| + | [[File:Magic_lst1_LC_common_gti.png|600px]] | ||
Latest revision as of 09:50, 26 July 2024
Contents
- 1 LST analysis OP313 Dec 2023 and Jan 2024
- 1.1 Links to presentation of the LST-1 analysis in several analysis and OP313 meetings
- 1.2 Data-taking information
- 1.3 Good-quality selected dataset
- 1.4 lstchain settings
- 1.5 Gammapy settings
- 1.6 Analysis cross-check
- 1.7 Test on background normalization systematics
- 1.8 Test varying the efficiency of gamma-ray selection cuts
- 1.9 Systematic on the energy scale
- 1.10 Test on MAGIC vs LST-1 with the same common GTIs
LST analysis OP313 Dec 2023 and Jan 2024[edit]
Wiki page with general information on all results for the paper: December_January_paper
LST-1 analyzers:
- Seiya Nozaki (MPP)
- Jorge Otero-Santos (IAA-CSIC)
- Mireia Nievas-Rosillo (IAC)
- Daniel Morcuende (IAA-CSIC)
Links to presentation of the LST-1 analysis in several analysis and OP313 meetings[edit]
- 2023-12-20 | LST EGAL meeting: https://indico.cta-observatory.org/event/4778/contributions/43117/attachments/24993/36374/OP313_Dec_flare-2.pdf
- 2024-01-18 | LST EGAL meeting: https://indico.cta-observatory.org/event/4779/contributions/43526/attachments/25079/36510/OP313_EgalMeeting_20240118%20(1).pdf
- 2024-01-19 | OP 313 Kick-off meeting: https://indico.cta-observatory.org/event/5330/contributions/43423/attachments/25097/36541/OP313_LST_Jan2024.pdf
- 2024-01-30 | LST analysis call: https://indico.cta-observatory.org/event/5377/contributions/43825/attachments/25187/36697/OP313_LST_analysis_20230130.pdf
- 2024-02-02 | OP 313 meeting: https://indico.cta-observatory.org/event/5379/contributions/43829/attachments/25221/36786/OP313_LST_20240202.pdf
- 2024-02-16 | OP 313 meeting: https://indico.cta-observatory.org/event/5412/contributions/44159/attachments/25331/37045/20240216_OP313_LST_srcdep.pdf
- 2024-02-16 | OP 313 meeting: https://indico.cta-observatory.org/event/5412/contributions/44159/attachments/25331/37042/OP313_LST_20240215.pdf
- 2024-02-19 | LST analysis call: https://indico.cta-observatory.org/event/5418/contributions/44198/attachments/25346/37068/OP313_LSTreco_20240219-1.pdf
- 2024-05-21 | LST general meeting (1/2): https://indico.cta-observatory.org/event/5371/contributions/45583/attachments/25909/38070/OP313_plan_collMeeting.pdf
- 2024-05-21 | LST general meeting (2/2): https://indico.cta-observatory.org/event/5371/contributions/45583/attachments/25909/38061/OP313_LST_GM_20240520_v1.pdf
Data-taking information[edit]
- NSB level: all data was taken in dark NSB conditions
- Zenith angles: 5-54 deg (December 2023 data have zenith angle > 30 deg)
Good-quality selected dataset[edit]
The list of data runs below was obtained using the notebook https://github.com/cta-observatory/cta-lstchain/blob/main/notebooks/data_quality.ipynb.
We used default settings, but min_drdi_at_422pe = 1.2 instead of the default 1.5 to recover 1.7 h from January 2024 where low statistics limit us. Still, the data selected have a relative light yield above ~80%.
Also, short data runs (< 5 min) were excluded, because usually short runs are a symptom of problems during data acquisition.
'2023-12-09': [15942, 15943, 15944, 15945], '2023-12-10': [15975, 15976, 15977, 15978], '2023-12-11': [16007, 16008, 16009, 16010], '2023-12-12': [16042, 16043, 16044, 16045], '2023-12-13': [16079, 16080, 16081, 16082, 16083], '2023-12-14': [16114, 16116, 16117, 16118, 16119], '2023-12-15': [16150, 16151, 16152, 16153, 16154, 16155, 16156], '2023-12-16': [16188, 16189, 16191, 16192, 16193], '2023-12-17': [16220, 16221, 16222, 16223, 16224, 16225], '2023-12-18': [16240, 16241, 16242, 16243, 16244, 16246], '2024-01-09': [16293, 16294, 16295], '2024-01-13': [16350, 16351, 16352, 16353, 16354, 16356, 16357, 16358, 16359], '2024-01-17': [16391, 16392], '2024-01-19': [16412, 16413, 16414, 16415], '2024-01-21': [16418, 16419, 16420]
In total, there are 19.8 hours effective time.
- Relative light yield of the selected data:
Number of runs (% is w.r.t. those in Sky region & zenith range): In the requested Sky region and range of dates: 90 + zenith in requested range: 90 + NSB in requested range: 90 (100.0%) + FF and pedestal interleaved events are present: 90 (100.0%) + Stable pointing: 90 (100.0%) + dR/dI fit P-value ok: 90 (100.0%) + dR/dI LS periodogram ok: 87 (96.7%) + dR/dI index ok: 85 (94.4%) + dR/dI rate ok: 83 (92.2%) + intensity threshold ok: 82 (92.2%)
lstchain settings[edit]
- lstchain version: v0.10.7
- MC production: 20230901_v0.10.4_allsky_base_prod
- RF models: /fefs/aswg/data/models/AllSky/20230901_v0.10.4_allsky_base_prod/dec_3476/
- DL2 MC for IRF calculation: /fefs/aswg/data/mc/DL2/AllSky/20230901_v0.10.4_allsky_base_prod/TestingDataset/dec_3476
- Used IRF interpolation
- Selection cuts IRF and DL3: 70% efficiency in both gammaness and theta gamma-ray selection cuts (Mireia used 68% theta containment instead). Intensity > 50 p.e.
- Energy threshold: ~ 40 GeV. Calculated from DL2 testing gamma MC (several nodes with similar zenith as the bulk of the telescope pointing positions: 32.059_az_102.217, 37.814_az_90, 43.197_az_87.604, 47.952_az_90, 52.374_az_110.312), weighting the default MC spectrum by a log parabola calculated from OP313 (alpha: -3.7, beta: -1.1, amplitude: 3.0e-09 TeV s-1 cm-2, E_0: 100 GeV) and assuming the same selection cuts as in the DL3 files (intensity>50 p.e., 70% efficiency in gammaness cut, and 70% efficiency in theta cut). Treatment of the DL2 MC was done using functions from
lstchain.mcmodule
Gammapy settings[edit]
- Gammapy version: v1.1
- Energy binning:
reco energy axis: 5 GeV - 10 TeV, 5 bins per decade true energy axis: 1 GeV - 20 TeV, 10 bins per decade
- 3 off positions for SED and LC calculation
- Safe mask: 5% of max. Aeff
- Energy range for spectral fitting: 70 GeV - 1 TeV (In Mireia's analysis, the fit range starts at around 60 GeV, because uses a slightly different reconstructed energy binning)
- redshift: 0.997
Analysis cross-check[edit]
Cross-checks of different LST-1 analysis results.
Cross-check of LST-1 SED from OP313 during high-flux period (Dec 2023):
Cross-check of LST-1 LC from OP313 (Dec 9th 2023 to Jan 21th 2024):
| Analyzer | Analysis method | Spectral model | EBL model | Index | Amplitude [x 10-8 TeV-1 s-1 cm-2] | E_ref [GeV] | alpha_norm |
|---|---|---|---|---|---|---|---|
| Seiya | source-dependent | PWL+EBL | Saldana-Lopez 2021 | 2.37 ± 0.88 | 0.82 ± 0.14 | 100 | 1.0 |
| Jorge | source-independent | PWL+EBL | Saldana-Lopez 2021 | 2.07 ± 0.51 | 1.15 ± 0.13 | 100 | 1.0 |
| Mireia | source-independent | PWL+EBL | Dominguez 2011 | 1.86 ± 0.50 | 0.93 ± 0.11 | 100 | 1.0 |
| Daniel | source-independent | PWL+EBL | Saldana-Lopez 2021 | 2.00 ± 0.45 | 1.14 ± 0.13 | 100 | 1.0 |
Test on background normalization systematics[edit]
The relative difference between the background counts in two off-source regions at the same distance from the center of the field of view (0.4 deg), and equidistant from the OP313 location is 0.31% ± 0.14% (counts in the OFF region at 90 deg: 487702 ± 698, and in the OFF region at 270 deg: 486203 ± 697). Therefore we tested the effect that a factor ±0.5% in the background normalization has in the SED calculation (see best-fit model spectral parameters calculated after considering the change in background normalization for source-independent analysis, assuming PWL+EBL Saldana-Lopez 2021 model).
| Background normalization | Index | Amplitude [x 10-8 TeV-1 s-1 cm-2] | E_ref [GeV] | alpha_norm |
|---|---|---|---|---|
| +0.5% | 2.12 ± 0.58 | 0.89 ± 0.14 | 100 | 1.0 |
| -0.5% | 1.86 ± 0.37 | 1.39 ± 0.14 | 100 | 1.0 |
Test varying the efficiency of gamma-ray selection cuts[edit]
We tested the effect of using several efficiencies in selection cuts. Gammaness (50%-90%) and theta (70%-90%). The hatched band in the plot below encompasses all statistical uncertainty bands of the best-fit model for each combination of efficiencies.
Systematic on the energy scale[edit]
Test the effect of a +/-15% shift on a possible bias on the absolute energy scale between simulations and observed data. This is the typical value used in currents IACTs and also it was found by Federico Di Pierro et al. to be the energy scale mismatch between LST-1 and MAGIC.
This can be done at different analysis levels:
- Post DL3 in Gammapy (e.g. have a look at https://github.com/bkhelifi/CTAO-CTAC_Meeting_Granada_2023/blob/main/Crab_simulations_systematic_errors_Solution.ipynb)
- Scaling of true energy in IRFs by ±15%
| Energy true scale | Index | Amplitude [x 10-8 TeV-1 s-1 cm-2] |
|---|---|---|
| +15% | 1.78 ± 0.55 | 1.80 ± 0.30 |
| -15% | 2.57 ± 0.38 | 0.60 ± 0.10 |
- DL1: scaling the pixel charges
Using rescale_dl1_charge
Analysis changing the amount of light. Trigger and fluctuations are not affected, but all the rest, starting from cleaning are. Better to try this first before producing low-level MC.
- Simtel files (simulating new MC productions scaling the light yield, e.g. +/- 15 %): before doing this test, first try the approach at DL1 level scaling the charge.
Test on MAGIC vs LST-1 with the same common GTIs[edit]
Test made with MAGIC (Axel's) DL3 files and LST-1 (Mireia's) DL3 files, cutting them to a common GTI selection. In total around 7.4 hours (a bit less of livetime for LST). Except at the lowest energies, the agreement is good within uncertainties and the individual bins match well.
In the common GTI light curve there is one night (mjd 60296) in which MAGIC has a UL, and LST a point. And LST's excess is actually pretty high (TS = 18.0 +/- 4.2, vs TS=0.58+/-0.77 in MAGIC). Mismatch to be investigated.
