Difference between revisions of "OP313 Dec2023 Jan2024 LST1 analysis details"

From my_wiki
Jump to: navigation, search
m (→‎Analysis cross-check)
(→‎Systematic on the energy scale)
 
(41 intermediate revisions by 2 users not shown)
Line 3: Line 3:
 
'''Wiki page with general information on all results for the paper''': [[December_January_paper]]
 
'''Wiki page with general information on all results for the paper''': [[December_January_paper]]
  
 +
LST-1 analyzers:
 +
* Seiya Nozaki (MPP)
 +
* Jorge Otero-Santos (IAA-CSIC)
 +
* Mireia Nievas-Rosillo (IAC)
 +
* Daniel Morcuende (IAA-CSIC)
 +
 +
=== Links to presentation of the LST-1 analysis in several analysis and OP313 meetings ===
 +
 +
* 2023-12-20 | LST EGAL meeting: https://indico.cta-observatory.org/event/4778/contributions/43117/attachments/24993/36374/OP313_Dec_flare-2.pdf
 +
* 2024-01-18 | LST EGAL meeting: https://indico.cta-observatory.org/event/4779/contributions/43526/attachments/25079/36510/OP313_EgalMeeting_20240118%20(1).pdf
 +
* 2024-01-19 | OP 313 Kick-off meeting: https://indico.cta-observatory.org/event/5330/contributions/43423/attachments/25097/36541/OP313_LST_Jan2024.pdf
 +
* 2024-01-30 | LST analysis call: https://indico.cta-observatory.org/event/5377/contributions/43825/attachments/25187/36697/OP313_LST_analysis_20230130.pdf
 +
* 2024-02-02 | OP 313 meeting: https://indico.cta-observatory.org/event/5379/contributions/43829/attachments/25221/36786/OP313_LST_20240202.pdf
 +
* 2024-02-16 | OP 313 meeting: https://indico.cta-observatory.org/event/5412/contributions/44159/attachments/25331/37045/20240216_OP313_LST_srcdep.pdf
 +
* 2024-02-16 | OP 313 meeting: https://indico.cta-observatory.org/event/5412/contributions/44159/attachments/25331/37042/OP313_LST_20240215.pdf
 +
* 2024-02-19 | LST analysis call: https://indico.cta-observatory.org/event/5418/contributions/44198/attachments/25346/37068/OP313_LSTreco_20240219-1.pdf
 +
* 2024-05-21 | LST general meeting (1/2): https://indico.cta-observatory.org/event/5371/contributions/45583/attachments/25909/38070/OP313_plan_collMeeting.pdf
 +
* 2024-05-21 | LST general meeting (2/2): https://indico.cta-observatory.org/event/5371/contributions/45583/attachments/25909/38061/OP313_LST_GM_20240520_v1.pdf
  
 
=== Data-taking information ===
 
=== Data-taking information ===
  
 
* NSB level: all data was taken in dark NSB conditions
 
* NSB level: all data was taken in dark NSB conditions
−
* Zenith angles: 15-54 deg (December 2023 data have zenith angle > 30 deg)
+
* Zenith angles: 5-54 deg (December 2023 data have zenith angle > 30 deg)
  
 
=== Good-quality selected dataset ===
 
=== Good-quality selected dataset ===
Line 46: Line 64:
 
[[File:Evolution_rel_light_yield_op313.png|800px]]  
 
[[File:Evolution_rel_light_yield_op313.png|800px]]  
  
−
[[File:Rel_light_yield_op313.png|500px]]  
+
[[File:Rel_light_yield_op313.png|500px]]
−
 
 
−
 
 
  
 +
Number of runs (% is w.r.t. those in Sky region & zenith range):
 +
    In the requested Sky region and range of dates: 90
 +
  + zenith in requested range: 90
 +
  + NSB in requested range: 90 (100.0%)
 +
  + FF and pedestal interleaved events are present: 90 (100.0%)
 +
  + Stable pointing: 90 (100.0%)
 +
  + dR/dI fit P-value ok: 90 (100.0%)
 +
  + dR/dI LS periodogram ok: 87 (96.7%)
 +
  + dR/dI index ok: 85 (94.4%)
 +
  + dR/dI rate ok: 83 (92.2%)
 +
  + intensity threshold ok: 82 (92.2%)
  
 
=== lstchain settings ===
 
=== lstchain settings ===
Line 57: Line 84:
 
* DL2 MC for IRF calculation: /fefs/aswg/data/mc/DL2/AllSky/20230901_v0.10.4_allsky_base_prod/TestingDataset/dec_3476
 
* DL2 MC for IRF calculation: /fefs/aswg/data/mc/DL2/AllSky/20230901_v0.10.4_allsky_base_prod/TestingDataset/dec_3476
 
* Used IRF interpolation
 
* Used IRF interpolation
−
* Energy threshold: ~ 40 GeV
+
* Selection cuts IRF and DL3: 70% efficiency in both gammaness and theta gamma-ray selection cuts (Mireia used 68% theta containment instead). Intensity > 50 p.e.
−
* Selection cuts IRF and DL3: 70% efficiency in both gammaness and theta gamma-ray selection cuts. Intensity > 50 p.e.
+
* Energy threshold: ~ 40 GeV. Calculated from DL2 testing gamma MC (several nodes with similar zenith as the bulk of the telescope pointing positions: 32.059_az_102.217, 37.814_az_90, 43.197_az_87.604, 47.952_az_90, 52.374_az_110.312), weighting the default MC spectrum by a log parabola calculated from OP313 (alpha: -3.7, beta: -1.1, amplitude: 3.0e-09 TeV s-1 cm-2, E_0: 100 GeV) and assuming the same selection cuts as in the DL3 files (intensity>50 p.e., 70% efficiency in gammaness cut, and 70% efficiency in theta cut). Treatment of the DL2 MC was done using functions from <code>lstchain.mc</code> module
 +
[[File:Op313_energy_threshold_MC.png|400px]]
  
 
=== Gammapy settings ===
 
=== Gammapy settings ===
Line 67: Line 95:
 
* 3 off positions for SED and LC calculation
 
* 3 off positions for SED and LC calculation
 
* Safe mask: 5% of max. Aeff
 
* Safe mask: 5% of max. Aeff
−
* Energy range for spectral fitting: 70 GeV - 1 TeV
+
* Energy range for spectral fitting: 70 GeV - 1 TeV (In Mireia's analysis, the fit range starts at around 60 GeV, because uses a slightly different reconstructed energy binning)
−
* EBL model: Saldaña-Lopez 2021
+
* redshift: 0.997
  
 
==Analysis cross-check==
 
==Analysis cross-check==
Line 74: Line 102:
 
Cross-checks of different LST-1 analysis results.
 
Cross-checks of different LST-1 analysis results.
  
−
* Source-dependent analysis (Seiya)
 
  
−
alpha: 2.37 +/- 0.88
+
Cross-check of LST-1 SED from OP313 during high-flux period (Dec 2023):
−
E_ref: 100 GeV
+
 
−
alpha_norm: 1.0
+
[[File:Xcheck_SED_OP313.png|750px]]
−
redshift: 0.997
+
 
−
amplitude: (0.82 +/- 0.14)e-08 TeV-1 s-1 cm-2
+
Cross-check of LST-1 LC from OP313 (Dec 9th 2023 to Jan 21th 2024):
 +
 
 +
[[File:Xcheck_LC_OP313.png|850px]]
 +
 
 +
 
 +
{| class="wikitable"
 +
|+ Best-fit spectral parameters
 +
|-
 +
! Analyzer !! Analysis method !! Spectral model !! EBL model !! Index !! Amplitude [x 10<sup>-8</sup> TeV<sup>-1</sup> s<sup>-1</sup> cm<sup>-2</sup>] !! E_ref [GeV] !! alpha_norm
 +
|-
 +
| Seiya || source-dependent || PWL+EBL || Saldana-Lopez 2021 || 2.37 ± 0.88 || 0.82 ± 0.14 || 100 || 1.0
 +
|-
 +
| Jorge || source-independent || PWL+EBL || Saldana-Lopez 2021 || 2.07 ± 0.51 || 1.15 ± 0.13 || 100 || 1.0
 +
|-
 +
| Mireia || source-independent || PWL+EBL || Dominguez 2011 || 1.86 ± 0.50 || 0.93 ± 0.11 || 100 || 1.0
 +
|-
 +
| Daniel || source-independent || PWL+EBL || Saldana-Lopez 2021 || 2.00 ± 0.45 || 1.14 ± 0.13 || 100 || 1.0
 +
|}
 +
 
 +
== Test on background normalization systematics ==
 +
 
 +
The relative difference between the background counts in two off-source regions at the same distance from the center of the field of view (0.4 deg), and equidistant from the OP313 location is 0.31% ± 0.14% (counts in the OFF region at 90 deg: 487702 ± 698, and in the OFF region at 270 deg: 486203 ± 697). Therefore we tested the effect that a factor ±0.5% in the background normalization has in the SED calculation (see best-fit model spectral parameters calculated after considering the change in background normalization for source-independent analysis, assuming PWL+EBL Saldana-Lopez 2021 model).
 +
 
 +
 
 +
[[File:Bkg_normalization_sys_op313.png|600px]]
 +
 
 +
 
 +
{| class="wikitable"
 +
|+ Best-fit spectral parameters after scaling the background normalization ±0.5%
 +
|-
 +
! Background normalization !! Index !! Amplitude [x 10<sup>-8</sup> TeV<sup>-1</sup> s<sup>-1</sup> cm<sup>-2</sup>] !! E_ref [GeV] !! alpha_norm
 +
|-
 +
| +0.5% || 2.12 ± 0.58 || 0.89 ± 0.14 || 100 || 1.0
 +
|-
 +
| -0.5% || 1.86 ± 0.37 || 1.39 ± 0.14 || 100 || 1.0
 +
|}
 +
 
 +
== Test varying the efficiency of gamma-ray selection cuts  ==
 +
We tested the effect of using several efficiencies in selection cuts. Gammaness (50%-90%) and theta (70%-90%). The hatched band in the plot below encompasses all statistical uncertainty bands of the best-fit model for each combination of efficiencies.
 +
 
 +
[[File:SED_OP313_several_efficiencies.png|900px]]
 +
 
 +
== Systematic on the energy scale  ==
 +
 
 +
 
 +
Test the effect of a +/-15% shift on a possible bias on the absolute energy scale between simulations and observed data. This is the typical value used in currents IACTs and also it was found by Federico Di Pierro et al. to be the energy scale mismatch between LST-1 and MAGIC.
 +
 
 +
This can be done at different analysis levels:
 +
 
 +
* Post DL3 in Gammapy (e.g. have a look at https://github.com/bkhelifi/CTAO-CTAC_Meeting_Granada_2023/blob/main/Crab_simulations_systematic_errors_Solution.ipynb)
 +
[[File:SED_OP313_energy_scale_gammapy.png|900px]]
 +
 
 +
* Scaling of true energy in IRFs by ±15%
 +
 
 +
[[File:SED_OP313_energy_scale_pyirf.png|900px]]
 +
 
 +
{| class="wikitable"
 +
|+ Best-fit spectral parameters after scaling the true energy by ±15%
 +
|-
 +
! Energy true scale !! Index !! Amplitude [x 10<sup>-8</sup> TeV<sup>-1</sup> s<sup>-1</sup> cm<sup>-2</sup>]
 +
|-
 +
| +15% || 1.78 ± 0.55 || 1.80 ± 0.30
 +
|-
 +
| -15% || 2.57 ± 0.38 || 0.60 ± 0.10
 +
|}
 +
 
 +
 
 +
* DL1: scaling the pixel charges
 +
Using rescale_dl1_charge
 +
 
 +
Analysis changing the amount of light. Trigger and fluctuations are not affected, but all the rest, starting from cleaning are. Better to try this first before producing low-level MC.
 +
 
 +
* Simtel files (simulating new MC productions scaling the light yield, e.g. +/- 15 %): before doing this test, first try the approach at DL1 level scaling the charge.
 +
 
 +
== Test on MAGIC vs LST-1 with the same common GTIs ==
 +
 
 +
Test made with MAGIC (Axel's) DL3 files and LST-1 (Mireia's) DL3 files, cutting them to a common GTI selection. In total around 7.4 hours (a bit less of livetime for LST). Except at the lowest energies, the agreement is good within uncertainties and the individual bins match well.
 +
 
 +
[[File:OP313_same_GTIs.png|600px]]
  
−
* Source-independent analysis (Jorge)
+
[[File:OP313_same_GTIs_SED.png|600px]]
−
alpha: 2.07 +/- 0.51
 
−
E_ref: 100 GeV
 
−
alpha_norm: 1.0
 
−
redshift: 0.997
 
−
amplitude: (1.15 +/- 0.13)e-08 TeV-1 s-1 cm-2
 
  
−
[[File:Xcheck_SED_OP313.png|800px|thumb|left|Cross-check of LST-1 SED from OP313 during high-flux period (Dec 2023).]]
+
In the common GTI  light curve there is one night (mjd 60296) in which MAGIC has a UL, and LST a point. And LST's excess is actually pretty high (TS = 18.0 +/- 4.2, vs TS=0.58+/-0.77 in MAGIC). Mismatch to be investigated.
−
[[File:Xcheck_LC_OP313.png|500px|thumb|left|Cross-check of LST-1 LC from OP313 (Dec 9th 2023 to Jan 21th 2024).]]
 
  
−
<br>
+
[[File:Magic_lst1_LC_common_gti.png|600px]]
−
[[December_January_paper|Go back to wiki page with general information on all results for the paper]]
 

Latest revision as of 09:50, 26 July 2024

LST analysis OP313 Dec 2023 and Jan 2024[edit]

Wiki page with general information on all results for the paper: December_January_paper

LST-1 analyzers:

  • Seiya Nozaki (MPP)
  • Jorge Otero-Santos (IAA-CSIC)
  • Mireia Nievas-Rosillo (IAC)
  • Daniel Morcuende (IAA-CSIC)

Links to presentation of the LST-1 analysis in several analysis and OP313 meetings[edit]

Data-taking information[edit]

  • NSB level: all data was taken in dark NSB conditions
  • Zenith angles: 5-54 deg (December 2023 data have zenith angle > 30 deg)

Good-quality selected dataset[edit]

The list of data runs below was obtained using the notebook https://github.com/cta-observatory/cta-lstchain/blob/main/notebooks/data_quality.ipynb.

We used default settings, but min_drdi_at_422pe = 1.2 instead of the default 1.5 to recover 1.7 h from January 2024 where low statistics limit us. Still, the data selected have a relative light yield above ~80%.

Also, short data runs (< 5 min) were excluded, because usually short runs are a symptom of problems during data acquisition.


Qualitycuts1 op313.png

Qualitycuts op313 2.png 


'2023-12-09': [15942, 15943, 15944, 15945],
'2023-12-10': [15975, 15976, 15977, 15978],
'2023-12-11': [16007, 16008, 16009, 16010],
'2023-12-12': [16042, 16043, 16044, 16045],
'2023-12-13': [16079, 16080, 16081, 16082, 16083],
'2023-12-14': [16114, 16116, 16117, 16118, 16119],
'2023-12-15': [16150, 16151, 16152, 16153, 16154, 16155, 16156],
'2023-12-16': [16188, 16189, 16191, 16192, 16193],
'2023-12-17': [16220, 16221, 16222, 16223, 16224, 16225],
'2023-12-18': [16240, 16241, 16242, 16243, 16244, 16246],
'2024-01-09': [16293, 16294, 16295],
'2024-01-13': [16350, 16351, 16352, 16353, 16354, 16356, 16357, 16358, 16359],
'2024-01-17': [16391, 16392],
'2024-01-19': [16412, 16413, 16414, 16415],
'2024-01-21': [16418, 16419, 16420]

In total, there are 19.8 hours effective time.

  • Relative light yield of the selected data:

Evolution rel light yield op313.png

Rel light yield op313.png 

Number of runs (% is w.r.t. those in Sky region & zenith range): 
   In the requested Sky region and range of dates:	 90
  + zenith in requested range:				 90
  + NSB in requested range:				 90 (100.0%) 
  + FF and pedestal interleaved events are present:	 90 (100.0%)
  + Stable pointing:					 90 (100.0%)
  + dR/dI fit P-value ok:				 90 (100.0%)
  + dR/dI LS periodogram ok:				 87 (96.7%)
  + dR/dI index ok:					 85 (94.4%)
  + dR/dI rate ok:					 83 (92.2%)
  + intensity threshold ok:				 82 (92.2%)

lstchain settings[edit]

  • lstchain version: v0.10.7
  • MC production: 20230901_v0.10.4_allsky_base_prod
  • RF models: /fefs/aswg/data/models/AllSky/20230901_v0.10.4_allsky_base_prod/dec_3476/
  • DL2 MC for IRF calculation: /fefs/aswg/data/mc/DL2/AllSky/20230901_v0.10.4_allsky_base_prod/TestingDataset/dec_3476
  • Used IRF interpolation
  • Selection cuts IRF and DL3: 70% efficiency in both gammaness and theta gamma-ray selection cuts (Mireia used 68% theta containment instead). Intensity > 50 p.e.
  • Energy threshold: ~ 40 GeV. Calculated from DL2 testing gamma MC (several nodes with similar zenith as the bulk of the telescope pointing positions: 32.059_az_102.217, 37.814_az_90, 43.197_az_87.604, 47.952_az_90, 52.374_az_110.312), weighting the default MC spectrum by a log parabola calculated from OP313 (alpha: -3.7, beta: -1.1, amplitude: 3.0e-09 TeV s-1 cm-2, E_0: 100 GeV) and assuming the same selection cuts as in the DL3 files (intensity>50 p.e., 70% efficiency in gammaness cut, and 70% efficiency in theta cut). Treatment of the DL2 MC was done using functions from lstchain.mc module

Op313 energy threshold MC.png

Gammapy settings[edit]

  • Gammapy version: v1.1
  • Energy binning:
reco energy axis: 5 GeV - 10 TeV, 5 bins per decade
true energy axis: 1 GeV - 20 TeV, 10 bins per decade
  • 3 off positions for SED and LC calculation
  • Safe mask: 5% of max. Aeff
  • Energy range for spectral fitting: 70 GeV - 1 TeV (In Mireia's analysis, the fit range starts at around 60 GeV, because uses a slightly different reconstructed energy binning)
  • redshift: 0.997

Analysis cross-check[edit]

Cross-checks of different LST-1 analysis results.


Cross-check of LST-1 SED from OP313 during high-flux period (Dec 2023):

Xcheck SED OP313.png

Cross-check of LST-1 LC from OP313 (Dec 9th 2023 to Jan 21th 2024):

Xcheck LC OP313.png


Best-fit spectral parameters
Analyzer Analysis method Spectral model EBL model Index Amplitude [x 10-8 TeV-1 s-1 cm-2] E_ref [GeV] alpha_norm
Seiya source-dependent PWL+EBL Saldana-Lopez 2021 2.37 ± 0.88 0.82 ± 0.14 100 1.0
Jorge source-independent PWL+EBL Saldana-Lopez 2021 2.07 ± 0.51 1.15 ± 0.13 100 1.0
Mireia source-independent PWL+EBL Dominguez 2011 1.86 ± 0.50 0.93 ± 0.11 100 1.0
Daniel source-independent PWL+EBL Saldana-Lopez 2021 2.00 ± 0.45 1.14 ± 0.13 100 1.0

Test on background normalization systematics[edit]

The relative difference between the background counts in two off-source regions at the same distance from the center of the field of view (0.4 deg), and equidistant from the OP313 location is 0.31% ± 0.14% (counts in the OFF region at 90 deg: 487702 ± 698, and in the OFF region at 270 deg: 486203 ± 697). Therefore we tested the effect that a factor ±0.5% in the background normalization has in the SED calculation (see best-fit model spectral parameters calculated after considering the change in background normalization for source-independent analysis, assuming PWL+EBL Saldana-Lopez 2021 model).


Bkg normalization sys op313.png


Best-fit spectral parameters after scaling the background normalization ±0.5%
Background normalization Index Amplitude [x 10-8 TeV-1 s-1 cm-2] E_ref [GeV] alpha_norm
+0.5% 2.12 ± 0.58 0.89 ± 0.14 100 1.0
-0.5% 1.86 ± 0.37 1.39 ± 0.14 100 1.0

Test varying the efficiency of gamma-ray selection cuts[edit]

We tested the effect of using several efficiencies in selection cuts. Gammaness (50%-90%) and theta (70%-90%). The hatched band in the plot below encompasses all statistical uncertainty bands of the best-fit model for each combination of efficiencies.

SED OP313 several efficiencies.png

Systematic on the energy scale[edit]

Test the effect of a +/-15% shift on a possible bias on the absolute energy scale between simulations and observed data. This is the typical value used in currents IACTs and also it was found by Federico Di Pierro et al. to be the energy scale mismatch between LST-1 and MAGIC.

This can be done at different analysis levels:

900px

  • Scaling of true energy in IRFs by ±15%

900px

Best-fit spectral parameters after scaling the true energy by ±15%
Energy true scale Index Amplitude [x 10-8 TeV-1 s-1 cm-2]
+15% 1.78 ± 0.55 1.80 ± 0.30
-15% 2.57 ± 0.38 0.60 ± 0.10


  • DL1: scaling the pixel charges

Using rescale_dl1_charge

Analysis changing the amount of light. Trigger and fluctuations are not affected, but all the rest, starting from cleaning are. Better to try this first before producing low-level MC.

  • Simtel files (simulating new MC productions scaling the light yield, e.g. +/- 15 %): before doing this test, first try the approach at DL1 level scaling the charge.

Test on MAGIC vs LST-1 with the same common GTIs[edit]

Test made with MAGIC (Axel's) DL3 files and LST-1 (Mireia's) DL3 files, cutting them to a common GTI selection. In total around 7.4 hours (a bit less of livetime for LST). Except at the lowest energies, the agreement is good within uncertainties and the individual bins match well.

OP313 same GTIs.png

OP313 same GTIs SED.png

In the common GTI light curve there is one night (mjd 60296) in which MAGIC has a UL, and LST a point. And LST's excess is actually pretty high (TS = 18.0 +/- 4.2, vs TS=0.58+/-0.77 in MAGIC). Mismatch to be investigated.

Magic lst1 LC common gti.png

Retrieved from "https://www.lst1.iac.es/wiki/index.php?title=OP313_Dec2023_Jan2024_LST1_analysis_details&oldid=11084"