Difference between revisions of "Analysis BLLac Aug 2021 Emery"

From my_wiki
Jump to: navigation, search
(→‎Analysis results -- lstchain v0.9.x LHfit PR ongoing, for LST general meeting April 2022)
(→‎Analysis -- lstchain v0.9.6dev + LHfit PR @commit 37b081e... + IRF interpolation PR, for the Summer conferences)
 
(20 intermediate revisions by the same user not shown)
Line 7: Line 7:
 
Context : BL Lacertae is a know blazar (and more precisely an IBL) located at a redshift of 0.069 with relatively low luminosity in quiescent state. It was active on multiple occasion in 2021 and in particular in July and August. An elevated state was reported in optical, X-rays and at high energy (see e.g. ATel #14773 , ATel #14774 and ATel #14777). This was followed in July by a detection of the source by LST reported in an astronomer telegram ATel #14783. Further observations of a new flux increase at the end of July/start of August (ATel #14820) lead to additional MAGIC follow up (ATel #14826), along with piggyback observations with LST-1.
 
Context : BL Lacertae is a know blazar (and more precisely an IBL) located at a redshift of 0.069 with relatively low luminosity in quiescent state. It was active on multiple occasion in 2021 and in particular in July and August. An elevated state was reported in optical, X-rays and at high energy (see e.g. ATel #14773 , ATel #14774 and ATel #14777). This was followed in July by a detection of the source by LST reported in an astronomer telegram ATel #14783. Further observations of a new flux increase at the end of July/start of August (ATel #14820) lead to additional MAGIC follow up (ATel #14826), along with piggyback observations with LST-1.
  
 +
== Analysis -- lstchain v0.9.6dev + LHfit PR @commit 37b081e... + IRF interpolation PR, for the Summer conferences ==
 +
 +
A preliminary new spectrum of the 8th of August was produced to be shown at the summer conferences of 2022. See [[BLLac_2022_summer_conferences]].
 +
The data and MC were analysed using the likelihood reconstruction method with the LHfit configuration available in the PR lstchain/data/lstchain_lhfit_config.json
 +
For the MC, an additional 42.9% NSB was added at waveform level to tune to the BL Lac field of view.
 +
The DL1 to DL2 step was done using random forest trained on the MC training line for dec3476, using the disp_vector for direction reconstruction and the parameter in the following file :
 +
 +
<div class="toccolours mw-collapsible mw-collapsed" style="width:400px; overflow:auto;">
 +
<div style="font-weight:bold;line-height:1.6;">lstchain_lhfit_config_RF_line_vect2.json</div>
 +
<div class="mw-collapsible-content">
 +
{
 +
  "source_config" : {
 +
    "EventSource": {
 +
      "allowed_tels": [1],
 +
      "max_events": null
 +
    },
 +
    "LSTEventSource": {
 +
      "default_trigger_type": "ucts",
 +
      "allowed_tels": [1],
 +
      "min_flatfield_adc": 3000,
 +
      "min_flatfield_pixel_fraction": 0.8,
 +
      "calibrate_flatfields_and_pedestals": false,
 +
      "EventTimeCalculator": {
 +
        "dragon_reference_counter": null,
 +
        "dragon_reference_time": null
 +
      },
 +
      "PointingSource":{
 +
        "drive_report_path": null
 +
      },
 +
      "LSTR0Corrections":{
 +
        "calib_scale_high_gain":1.088,
 +
        "calib_scale_low_gain":1.004,
 +
        "drs4_pedestal_path": null,
 +
        "calibration_path": null,
 +
        "drs4_time_calibration_path": null
 +
      }
 +
    }
 +
  },
 +
 +
  "events_filters": {
 +
    "intensity": [0, Infinity],
 +
    "width": [0, Infinity],
 +
    "length": [0, Infinity],
 +
    "wl": [0, Infinity],
 +
    "r": [0, Infinity],
 +
    "leakage_intensity_width_2": [0, Infinity]
 +
  },
 +
  "n_training_events": {
 +
    "gamma_regressors": 1.0,
 +
    "gamma_tmp_regressors": 0.8,
 +
    "gamma_classifier": 0.2,
 +
    "proton_classifier": 1.0
 +
  },
 +
 +
  "tailcut": {
 +
    "picture_thresh":8,
 +
    "boundary_thresh":4,
 +
    "keep_isolated_pixels":false,
 +
    "min_number_picture_neighbors":2,
 +
    "use_only_main_island":false,
 +
    "delta_time": 2
 +
  },
 +
  "tailcuts_clean_with_pedestal_threshold": {
 +
    "picture_thresh":8,
 +
    "boundary_thresh":4,
 +
    "sigma":2.5,
 +
    "keep_isolated_pixels":false,
 +
    "min_number_picture_neighbors":2,
 +
    "use_only_main_island":false,
 +
    "delta_time": 2
 +
  },
 +
  "dynamic_cleaning": {
 +
    "apply": true,
 +
    "threshold": 267,
 +
    "fraction_cleaning_intensity": 0.03
 +
  },
 +
 +
  "random_forest_energy_regressor_args": {
 +
    "max_depth": 30,
 +
    "min_samples_leaf": 10,
 +
    "n_jobs": -1,
 +
    "n_estimators": 150,
 +
    "bootstrap": true,
 +
    "criterion": "squared_error",
 +
    "max_features": "auto",
 +
    "max_leaf_nodes": null,
 +
    "min_impurity_decrease": 0.0,
 +
    "min_samples_split": 10,
 +
    "min_weight_fraction_leaf": 0.0,
 +
    "oob_score": false,
 +
    "random_state": 42,
 +
    "verbose": 0,
 +
    "warm_start": false
 +
  },
 +
 +
  "random_forest_disp_regressor_args": {
 +
    "max_depth": 30,
 +
    "min_samples_leaf": 10,
 +
    "n_jobs": -1,
 +
    "n_estimators": 150,
 +
    "bootstrap": true,
 +
    "criterion": "squared_error",
 +
    "max_features": "auto",
 +
    "max_leaf_nodes": null,
 +
    "min_impurity_decrease": 0.0,
 +
    "min_samples_split": 10,
 +
    "min_weight_fraction_leaf": 0.0,
 +
    "oob_score": false,
 +
    "random_state": 42,
 +
    "verbose": 0,
 +
    "warm_start": false
 +
  },
 +
 +
  "random_forest_disp_classifier_args": {
 +
    "max_depth": 30,
 +
    "min_samples_leaf": 10,
 +
    "n_jobs": -1,
 +
    "n_estimators": 100,
 +
    "criterion": "gini",
 +
    "min_samples_split": 10,
 +
    "min_weight_fraction_leaf": 0.0,
 +
    "max_features": "auto",
 +
    "max_leaf_nodes": null,
 +
    "min_impurity_decrease": 0.0,
 +
    "bootstrap": true,
 +
    "oob_score": false,
 +
    "random_state": 42,
 +
    "verbose": 0.0,
 +
    "warm_start": false,
 +
    "class_weight": null
 +
  },
 +
 +
  "random_forest_particle_classifier_args": {
 +
    "max_depth": 30,
 +
    "min_samples_leaf": 10,
 +
    "n_jobs": -1,
 +
    "n_estimators": 100,
 +
    "criterion": "gini",
 +
    "min_samples_split": 10,
 +
    "min_weight_fraction_leaf": 0.0,
 +
    "max_features": "auto",
 +
    "max_leaf_nodes": null,
 +
    "min_impurity_decrease": 0.0,
 +
    "bootstrap": true,
 +
    "oob_score": false,
 +
    "random_state": 42,
 +
    "verbose": 0.0,
 +
    "warm_start": false,
 +
    "class_weight": null
 +
  },
 +
 +
 +
  "energy_regression_features": [
 +
    "lhfit_log_intensity",
 +
    "lhfit_width",
 +
    "lhfit_length",
 +
    "lhfit_length_asymmetry",
 +
    "lhfit_time_gradient",
 +
    "leakage_intensity_width_2",
 +
    "lhfit_r",
 +
    "lhfit_wl",
 +
    "lhfit_psi",
 +
    "lhfit_x",
 +
    "lhfit_y",
 +
    "lhfit_phi",
 +
    "alt_tel",
 +
    "az_tel"
 +
  ],
 +
 +
  "disp_method": "disp_vector",
 +
 +
  "disp_regression_features": [
 +
    "lhfit_log_intensity",
 +
    "lhfit_width",
 +
    "lhfit_length",
 +
    "lhfit_length_asymmetry",
 +
    "lhfit_time_gradient",
 +
    "leakage_intensity_width_2",
 +
    "lhfit_r",
 +
    "lhfit_wl",
 +
    "lhfit_psi",
 +
    "lhfit_x",
 +
    "lhfit_y",
 +
    "lhfit_phi",
 +
    "alt_tel",
 +
    "az_tel"
 +
  ],
 +
 +
  "disp_classification_features": [
 +
    "lhfit_log_intensity",
 +
    "lhfit_length",
 +
    "lhfit_wl",
 +
    "lhfit_length_asymmetry",
 +
    "lhfit_psi",
 +
    "lhfit_time_gradient"
 +
  ],
 +
 +
  "particle_classification_features": [
 +
    "lhfit_log_intensity",
 +
    "lhfit_width",
 +
    "lhfit_length",
 +
    "lhfit_length_asymmetry",
 +
    "lhfit_time_gradient",
 +
    "leakage_intensity_width_2",
 +
    "lhfit_r",
 +
    "lhfit_wl",
 +
    "lhfit_psi",
 +
    "lhfit_x",
 +
    "lhfit_y",
 +
    "lhfit_phi",
 +
    "log_reco_energy",
 +
    "reco_disp_dx",
 +
    "reco_disp_dy",
 +
    "alt_tel",
 +
    "az_tel"
 +
  ],
 +
 +
  "allowed_tels": [1],
 +
  "write_pe_image": false,
 +
  "mc_image_scaling_factor": 1,
 +
  "image_extractor": "LocalPeakWindowSum",
 +
  "image_extractor_for_muons": "GlobalPeakWindowSum",
 +
  "CameraCalibrator": {
 +
    "apply_waveform_time_shift": false
 +
  },
 +
  "time_sampling_correction_path": "default",
 +
  "LocalPeakWindowSum":{
 +
    "window_shift": 4,
 +
    "window_width": 8,
 +
    "apply_integration_correction": false
 +
  },
 +
  "GlobalPeakWindowSum":{
 +
    "window_shift": 4,
 +
    "window_width": 8,
 +
    "apply_integration_correction": false
 +
  },
 +
  "timestamps_pointing":"ucts",
 +
 +
  "train_gamma_src_r_deg": [0, Infinity],
 +
 +
  "source_dependent": false,
 +
  "mc_nominal_source_x_deg": 0.4,
 +
  "mc_nominal_source_y_deg": 0.0,
 +
 +
  "volume_reducer":{
 +
    "algorithm": null,
 +
    "parameters": {
 +
    }
 +
  },
 +
  "calibration_product": "LSTCalibrationCalculator",
 +
 +
  "LSTCalibrationCalculator":{
 +
    "systematic_correction_path": null,
 +
    "squared_excess_noise_factor": 1.222,
 +
    "flatfield_product": "FlasherFlatFieldCalculator",
 +
    "pedestal_product": "PedestalIntegrator",
 +
    "PedestalIntegrator":{
 +
      "sample_size": 10000,
 +
      "sample_duration":100000,
 +
      "tel_id":1,
 +
      "time_sampling_correction_path": null,
 +
      "charge_median_cut_outliers": [-10,10],
 +
      "charge_std_cut_outliers": [-10,10],
 +
      "charge_product":"FixedWindowSum",
 +
      "FixedWindowSum":{
 +
        "window_shift": 6,
 +
        "window_width":12,
 +
        "peak_index": 18,
 +
        "apply_integration_correction": false
 +
      }
 +
    },
 +
    "FlasherFlatFieldCalculator":{
 +
      "sample_size": 10000,
 +
      "sample_duration":100000,
 +
      "tel_id":1,
 +
      "time_sampling_correction_path": null,
 +
      "charge_product":"LocalPeakWindowSum",
 +
      "charge_median_cut_outliers": [-0.5,0.5],
 +
      "charge_std_cut_outliers": [-10,10],
 +
      "time_cut_outliers": [2,38],
 +
      "LocalPeakWindowSum":{
 +
        "window_shift": 5,
 +
        "window_width":12,
 +
        "apply_integration_correction": false
 +
      }
 +
    }
 +
  },
 +
  "waveform_nsb_tuning":{
 +
    "nsb_tuning": false,
 +
    "nsb_tuning_ratio": 0.52,
 +
    "spe_location": "lstchain/data/SinglePhE_ResponseInPhE_expo2Gaus.dat"
 +
  },
 +
  "lh_fit_config": {
 +
    "sigma_s": [
 +
                ["type", "*", 1.0],
 +
                ["type", "LST_LST_LSTCam", 0.3282]
 +
            ],
 +
    "crosstalk": [
 +
                ["type", "*", 0.0],
 +
                ["type", "LST_LST_LSTCam", 0.0]
 +
            ],
 +
    "sigma_space": 3,
 +
    "sigma_time": 4,
 +
    "time_before_shower": [
 +
                ["type", "*", 0.0],
 +
                ["type", "LST_LST_LSTCam", 0.0]
 +
            ],
 +
    "time_after_shower": [
 +
                ["type", "*", 20.0],
 +
                ["type", "LST_LST_LSTCam", 20.0]
 +
            ],
 +
    "n_peaks": 0,
 +
    "no_asymmetry": false,
 +
    "use_weight": false,
 +
    "verbose": 0
 +
  }
 +
}
 +
</div></div>
 +
 +
IRFs are produced using energy dependent efficiency cut for gamma-hadron separation and theta corresponding to 80% gamma efficiency. A fixed cut at r=1 was also active.
 +
<div class="toccolours mw-collapsible mw-collapsed" style="width:400px; overflow:auto;">
 +
<div style="font-weight:bold;line-height:1.6;">irf_config</div>
 +
<div class="mw-collapsible-content">
 +
{
 +
  "EventSelector": {
 +
    "filters": {
 +
      "intensity": [0, Infinity],
 +
      "width": [0, Infinity],
 +
      "length": [0, Infinity],
 +
      "r": [0, 1],
 +
      "wl": [0.01, 1],
 +
      "leakage_intensity_width_2": [0, 1],
 +
      "event_type": [32, 32]
 +
    }
 +
  },
 +
  "DL3Cuts": {
 +
    "min_event_p_en_bin": 100,
 +
    "global_gh_cut": 0.7,
 +
    "gh_efficiency": 0.8,
 +
    "min_gh_cut": 0.1,
 +
    "max_gh_cut": 0.95,
 +
    "global_alpha_cut": 10,
 +
    "global_theta_cut": 0.2,
 +
    "theta_containment": 0.8,
 +
    "alpha_containment": 0.68,
 +
    "min_theta_cut": 0.05,
 +
    "max_theta_cut": 0.32,
 +
    "fill_theta_cut": 0.32,
 +
    "min_alpha_cut": 1,
 +
    "max_alpha_cut": 45,
 +
    "fill_alpha_cut": 45,
 +
    "allowed_tels": [1]
 +
  },
 +
  "DataBinning": {
 +
    "true_energy_min": 0.005,
 +
    "true_energy_max": 200,
 +
    "true_energy_n_bins_per_decade": 5,
 +
    "reco_energy_min": 0.005,
 +
    "reco_energy_max": 200,
 +
    "reco_energy_n_bins_per_decade": 5,
 +
    "energy_migration_min": 0.2,
 +
    "energy_migration_max": 5,
 +
    "energy_migration_n_bins": 31,
 +
    "fov_offset_min": 0.1,
 +
    "fov_offset_max": 1.1,
 +
    "fov_offset_n_edges": 9,
 +
    "bkg_fov_offset_min": 0,
 +
    "bkg_fov_offset_max": 10,
 +
    "bkg_fov_offset_n_edges": 21,
 +
    "source_offset_min": 0,
 +
    "source_offset_max": 1,
 +
    "source_offset_n_edges": 101
 +
  }
 +
}
 +
</div></div>
 +
 +
DL3 files were created using IRF interpolation on the test grid and analysed with gammapy version 0.20.1. The preliminary spectrum was obtained using 5 bins per energy decades in reco energy and for the SED flux point, twice this number for the true energy binning and fitting a log parabola spectral model to the energy range between ~25 GeV and 10 TeV.
 +
 +
The 25 GeV threshold was used for two reasons :
 +
 +
- The excess rate drops significantly below this energy
 +
 +
[[File:Events_vs_energy_BLlac8th_1.png]]
 +
 +
- Biases in flux level are seen at lower energy (too low upperlimit in standard reconstruction based analysis in the 16-25 GeV bin, in the 10-16 GeV bin for the LHfit + downward trend in the 16-25 GeV point)
 +
 +
[[File:SED_all_energy_BLlac8th_1.png]]
 +
 +
Good agreement is found with the standard reconstruction analysis as seen in [[BLLac_2022_summer_conferences]] in the 25-10000GeV range using similar cuts for IRFs+DL3 except an additional 50 p.e. intensity cut.
  
 
== Analysis results -- lstchain v0.9.x LHfit PR ongoing, for LST general meeting April 2022 ==
 
== Analysis results -- lstchain v0.9.x LHfit PR ongoing, for LST general meeting April 2022 ==
  
 
Analysis performed in the BL lac paper TF before the LST general meeting. https://www.lst1.iac.es/wiki/index.php/BLLac_paper_meeting#collaborator_meeting
 
Analysis performed in the BL lac paper TF before the LST general meeting. https://www.lst1.iac.es/wiki/index.php/BLLac_paper_meeting#collaborator_meeting
 +
 +
Current processing uses prod5_trans80 Zd20 south pointing MC only. Split between Zd20 and Zd 40 will be considered if time allows (small number of affected runs).
 +
RF used is with disp_vector direction reconstruction and a large set of parameters.
 +
 +
data file description for model parameters and points :
 +
 +
In /fefs/aswg/workspace/gabriel.emery/lhfit_withasymetry_rebasemaster17052021/notebook
 +
 +
BLlac_{all or 8th}_irf{irf number}{spectral_hypothesis}_dataset_{5219_to_5547 or 5552_to_5559}_flux_model_dict.dat
 +
 +
BLlac_{all or 8th}_irf{irf number}{spectral_hypothesis}_dataset_{5219_to_5547 or 5552_to_5559}_flux_pts.fits
 +
 +
irf number : 7 - fixed cuts, no intensity cuts.  8 - E dependent gh cut, no intensity cut. 9 - fixed cuts, 50 pe intensity cut
 +
spectral_hypothesis : LP, BPL or LP_refdecorrelation (LP with reference energy as decorrelation energy)
 +
 +
The first can be loaded with pickle to get the model parameters.
 +
The second contains, for the flare 8th of august, 7 HDU (SED, RUN_LC0039, RUN_LC0063, RUNLC_0100, SHORT_LC0039, SHORT_LC0063, SHORT_LC0100) for the SED points and lightcurves points or, for the full dataset, 2 (SED, NIGHT_LC0100).
 +
  
 
=== August 8th ===
 
=== August 8th ===
Line 21: Line 428:
  
 
- w/ intensity cut(>50 pe), LogParabolaSpectralModel
 
- w/ intensity cut(>50 pe), LogParabolaSpectralModel
 +
 +
[[File:SED_LP_50Icut_irf9.png]]
  
 
- w/ intensity cut(>50 pe), SmoothBrokenPowerLawSpectralModel
 
- w/ intensity cut(>50 pe), SmoothBrokenPowerLawSpectralModel
 +
 +
[[File:SED_BPL_50Icut_irf9.png]]
  
 
- w/o intensity cut, LogParabolaSpectralModel
 
- w/o intensity cut, LogParabolaSpectralModel
 +
 +
[[File:SED_LP_noIcut_irf7.png]]
  
 
- w/o intensity cut, SmoothBrokenPowerLawSpectralModel
 
- w/o intensity cut, SmoothBrokenPowerLawSpectralModel
  
−
Light curve on the night of Aug 8 (>0.037, 0.057, 0.089 0.039, 0.063, 0.1 TeV) (runwise)
+
[[File:SED_BLP_noIcut_irf7.png]]
 +
 
 +
./BLlac_8th_irf7BPL_dataset_5552_to_5559_flux_model_dict.dat
 +
 
 +
 
 +
Light curve on the night of Aug 8 (>0.037, 0.057, 0.089 0.039, 0.063, 0.1 TeV) (runwise), from SED :
 +
 
 +
- w/ intensity cut(>50 pe), LogParabolaSpectralModel
 +
 
 +
[[File:SED_LP_50Icut_irf9_LC_runwise.png]]
 +
 
 +
- w/ intensity cut(>50 pe), SmoothBrokenPowerLawSpectralModel
 +
 
 +
[[File:SED_BPL_50Icut_irf9_LC_runwise.png]]
  
−
Light curve on the night of Aug 8 (>0.037, 0.057, 0.089 0.039, 0.063, 0.1 TeV) with short time scale, each observation time divided by 3
+
- w/o intensity cut, LogParabolaSpectralModel
 +
 
 +
[[File:SED_LP_noIcut_irf7_LC_runwise.png]]
 +
 
 +
- w/o intensity cut, SmoothBrokenPowerLawSpectralModel
 +
 
 +
[[File:SED_BLP_noIcut_irf7_LC_runwise.png]]
  
−
DL3 tools config :
+
./BLlac_8th_irf7BPL_dataset_5552_to_5559_flux_pts.fits  -> HUDs "RUN_LC0xxx"
  
 +
Light curve on the night of Aug 8 (>0.037, 0.057, 0.089 0.039, 0.063, 0.1 TeV) with short time scale, each observation time divided by 3
  
  
 
''' LP w/ intensity > 50 pe '''
 
''' LP w/ intensity > 50 pe '''
  
−
 
+
[[File:SED_LP_50Icut_irf9_LC_3perrun.png]]
  
 
''' sBPL w/ intensity > 50 pe '''
 
''' sBPL w/ intensity > 50 pe '''
  
 +
[[File:SED_BPL_50Icut_irf9_LC_3perrun.png]]
  
 +
''' LP w/o intensity cut '''
  
−
''' LP w/o intensity cut '''
+
[[File:SED_LP_noIcut_irf7_LC_3perrun.png]]
  
 +
''' sBPL w/o intensity cut '''
  
 +
[[File:SED_BLP_noIcut_irf7_LC_3perrun.png]]
  
−
''' sBPL w/o intensity cut '''
+
./BLlac_8th_irf7BPL_dataset_5552_to_5559_flux_pts.fits  -> HUDs "SHORT_LC0xxx"
  
  
Line 62: Line 499:
 
''' LP w/ intensity > 50 pe '''
 
''' LP w/ intensity > 50 pe '''
  
−
 
+
[[File:july_august_SED_LP_refdecorelation_50Icut_irf9.png]]  [[File:july_august_SED_LP_refdecorelation_50Icut_irf9_LC_nightwise.png]]
  
 
''' sBPL w/ intensity > 50 pe '''
 
''' sBPL w/ intensity > 50 pe '''
  
 +
[[File:july_august_SED_BPL_50Icut_irf9.png]]  [[File:july_august_SED_BPL_50Icut_irf9_LC_nightwise.png]]
  
  
 
''' LP w/o intensity cut '''
 
''' LP w/o intensity cut '''
  
 +
[[File:july_august_SED_LP_refdecorelation_noIcut_irf7.png]]  [[File:july_august_SED_LP_refdecorelation_noIcut_irf7_LC_nightwise.png]]
  
 +
''' sBPL w/o intensity cut '''
  
−
''' sBPL w/o intensity cut '''
+
[[File:july_august_SED_BPL_noIcut_irf7.png]]  [[File:july_august_SED_BPL_noIcut_irf7_LC_nightwise.png]]
  
 
==Previous analysis status ==
 
==Previous analysis status ==

Latest revision as of 09:18, 6 July 2022

Main page for the analysis of the observations of BL Lacertae of August 2021 by the LST1 using the likelihood reconstruction method.

LST data analysis responsible : Gabriel Emery using the LH fit reconstruction pipeline.

Other analysis : See the common page for links to each analysis BLLac

Context : BL Lacertae is a know blazar (and more precisely an IBL) located at a redshift of 0.069 with relatively low luminosity in quiescent state. It was active on multiple occasion in 2021 and in particular in July and August. An elevated state was reported in optical, X-rays and at high energy (see e.g. ATel #14773 , ATel #14774 and ATel #14777). This was followed in July by a detection of the source by LST reported in an astronomer telegram ATel #14783. Further observations of a new flux increase at the end of July/start of August (ATel #14820) lead to additional MAGIC follow up (ATel #14826), along with piggyback observations with LST-1.

Analysis -- lstchain v0.9.6dev + LHfit PR @commit 37b081e... + IRF interpolation PR, for the Summer conferences[edit]

A preliminary new spectrum of the 8th of August was produced to be shown at the summer conferences of 2022. See BLLac_2022_summer_conferences. The data and MC were analysed using the likelihood reconstruction method with the LHfit configuration available in the PR lstchain/data/lstchain_lhfit_config.json For the MC, an additional 42.9% NSB was added at waveform level to tune to the BL Lac field of view. The DL1 to DL2 step was done using random forest trained on the MC training line for dec3476, using the disp_vector for direction reconstruction and the parameter in the following file :

lstchain_lhfit_config_RF_line_vect2.json

{ 

 "source_config" : {
   "EventSource": {
     "allowed_tels": [1],
     "max_events": null
   },
   "LSTEventSource": {
     "default_trigger_type": "ucts",
     "allowed_tels": [1],
     "min_flatfield_adc": 3000,
     "min_flatfield_pixel_fraction": 0.8,
     "calibrate_flatfields_and_pedestals": false,
     "EventTimeCalculator": {
       "dragon_reference_counter": null,
       "dragon_reference_time": null
     },
     "PointingSource":{
       "drive_report_path": null
     },
     "LSTR0Corrections":{
       "calib_scale_high_gain":1.088,
       "calib_scale_low_gain":1.004,
       "drs4_pedestal_path": null,
       "calibration_path": null,
       "drs4_time_calibration_path": null
     }
   }
 },

 "events_filters": {
   "intensity": [0, Infinity],
   "width": [0, Infinity],
   "length": [0, Infinity],
   "wl": [0, Infinity],
   "r": [0, Infinity],
   "leakage_intensity_width_2": [0, Infinity]
 },
 "n_training_events": {
   "gamma_regressors": 1.0,
   "gamma_tmp_regressors": 0.8,
   "gamma_classifier": 0.2,
   "proton_classifier": 1.0
 },

 "tailcut": {
   "picture_thresh":8,
   "boundary_thresh":4,
   "keep_isolated_pixels":false,
   "min_number_picture_neighbors":2,
   "use_only_main_island":false,
   "delta_time": 2
 },
 "tailcuts_clean_with_pedestal_threshold": {
   "picture_thresh":8,
   "boundary_thresh":4,
   "sigma":2.5,
   "keep_isolated_pixels":false,
   "min_number_picture_neighbors":2,
   "use_only_main_island":false,
   "delta_time": 2
 },
 "dynamic_cleaning": {
   "apply": true,
   "threshold": 267,
   "fraction_cleaning_intensity": 0.03
 },

 "random_forest_energy_regressor_args": {
   "max_depth": 30,
   "min_samples_leaf": 10,
   "n_jobs": -1,
   "n_estimators": 150,
   "bootstrap": true,
   "criterion": "squared_error",
   "max_features": "auto",
   "max_leaf_nodes": null,
   "min_impurity_decrease": 0.0,
   "min_samples_split": 10,
   "min_weight_fraction_leaf": 0.0,
   "oob_score": false,
   "random_state": 42,
   "verbose": 0,
   "warm_start": false
 },

 "random_forest_disp_regressor_args": {
   "max_depth": 30,
   "min_samples_leaf": 10,
   "n_jobs": -1,
   "n_estimators": 150,
   "bootstrap": true,
   "criterion": "squared_error",
   "max_features": "auto",
   "max_leaf_nodes": null,
   "min_impurity_decrease": 0.0,
   "min_samples_split": 10,
   "min_weight_fraction_leaf": 0.0,
   "oob_score": false,
   "random_state": 42,
   "verbose": 0,
   "warm_start": false
 },

 "random_forest_disp_classifier_args": {
   "max_depth": 30,
   "min_samples_leaf": 10,
   "n_jobs": -1,
   "n_estimators": 100,
   "criterion": "gini",
   "min_samples_split": 10,
   "min_weight_fraction_leaf": 0.0,
   "max_features": "auto",
   "max_leaf_nodes": null,
   "min_impurity_decrease": 0.0,
   "bootstrap": true,
   "oob_score": false,
   "random_state": 42,
   "verbose": 0.0,
   "warm_start": false,
   "class_weight": null
 },

 "random_forest_particle_classifier_args": {
   "max_depth": 30,
   "min_samples_leaf": 10,
   "n_jobs": -1,
   "n_estimators": 100,
   "criterion": "gini",
   "min_samples_split": 10,
   "min_weight_fraction_leaf": 0.0,
   "max_features": "auto",
   "max_leaf_nodes": null,
   "min_impurity_decrease": 0.0,
   "bootstrap": true,
   "oob_score": false,
   "random_state": 42,
   "verbose": 0.0,
   "warm_start": false,
   "class_weight": null
 },



 "energy_regression_features": [
   "lhfit_log_intensity",
   "lhfit_width",
   "lhfit_length",
   "lhfit_length_asymmetry",
   "lhfit_time_gradient",
   "leakage_intensity_width_2",
   "lhfit_r",
   "lhfit_wl",
   "lhfit_psi",
   "lhfit_x",
   "lhfit_y",
   "lhfit_phi",
   "alt_tel",
   "az_tel"
 ],

 "disp_method": "disp_vector",

 "disp_regression_features": [
   "lhfit_log_intensity",
   "lhfit_width",
   "lhfit_length",
   "lhfit_length_asymmetry",
   "lhfit_time_gradient",
   "leakage_intensity_width_2",
   "lhfit_r",
   "lhfit_wl",
   "lhfit_psi",
   "lhfit_x",
   "lhfit_y",
   "lhfit_phi",
   "alt_tel",
   "az_tel"
 ],

 "disp_classification_features": [
   "lhfit_log_intensity",
   "lhfit_length",
   "lhfit_wl",
   "lhfit_length_asymmetry",
   "lhfit_psi",
   "lhfit_time_gradient"
 ],

 "particle_classification_features": [
   "lhfit_log_intensity",
   "lhfit_width",
   "lhfit_length",
   "lhfit_length_asymmetry",
   "lhfit_time_gradient",
   "leakage_intensity_width_2",
   "lhfit_r",
   "lhfit_wl",
   "lhfit_psi",
   "lhfit_x",
   "lhfit_y",
   "lhfit_phi",
   "log_reco_energy",
   "reco_disp_dx",
   "reco_disp_dy",
   "alt_tel",
   "az_tel"
 ],

 "allowed_tels": [1],
 "write_pe_image": false,
 "mc_image_scaling_factor": 1,
 "image_extractor": "LocalPeakWindowSum",
 "image_extractor_for_muons": "GlobalPeakWindowSum",
 "CameraCalibrator": {
   "apply_waveform_time_shift": false
 },
 "time_sampling_correction_path": "default",
 "LocalPeakWindowSum":{
   "window_shift": 4,
   "window_width": 8,
   "apply_integration_correction": false
 },
 "GlobalPeakWindowSum":{
   "window_shift": 4,
   "window_width": 8,
   "apply_integration_correction": false
 },
 "timestamps_pointing":"ucts",

 "train_gamma_src_r_deg": [0, Infinity],

 "source_dependent": false,
 "mc_nominal_source_x_deg": 0.4,
 "mc_nominal_source_y_deg": 0.0,

 "volume_reducer":{
   "algorithm": null,
   "parameters": {
   }
 },
 "calibration_product": "LSTCalibrationCalculator",

 "LSTCalibrationCalculator":{
   "systematic_correction_path": null,
   "squared_excess_noise_factor": 1.222,
   "flatfield_product": "FlasherFlatFieldCalculator",
   "pedestal_product": "PedestalIntegrator",
   "PedestalIntegrator":{
     "sample_size": 10000,
     "sample_duration":100000,
     "tel_id":1,
     "time_sampling_correction_path": null,
     "charge_median_cut_outliers": [-10,10],
     "charge_std_cut_outliers": [-10,10],
     "charge_product":"FixedWindowSum",
     "FixedWindowSum":{
       "window_shift": 6,
       "window_width":12,
       "peak_index": 18,
       "apply_integration_correction": false
     }
   },
   "FlasherFlatFieldCalculator":{
     "sample_size": 10000,
     "sample_duration":100000,
     "tel_id":1,
     "time_sampling_correction_path": null,
     "charge_product":"LocalPeakWindowSum",
     "charge_median_cut_outliers": [-0.5,0.5],
     "charge_std_cut_outliers": [-10,10],
     "time_cut_outliers": [2,38],
     "LocalPeakWindowSum":{
       "window_shift": 5,
       "window_width":12,
       "apply_integration_correction": false
     }
   }
 },
 "waveform_nsb_tuning":{
   "nsb_tuning": false,
   "nsb_tuning_ratio": 0.52,
   "spe_location": "lstchain/data/SinglePhE_ResponseInPhE_expo2Gaus.dat"
 },
 "lh_fit_config": {
   "sigma_s": [
               ["type", "*", 1.0],
               ["type", "LST_LST_LSTCam", 0.3282]
           ],
   "crosstalk": [
               ["type", "*", 0.0],
               ["type", "LST_LST_LSTCam", 0.0]
           ],
   "sigma_space": 3,
   "sigma_time": 4,
   "time_before_shower": [
               ["type", "*", 0.0],
               ["type", "LST_LST_LSTCam", 0.0]
           ],
   "time_after_shower": [
               ["type", "*", 20.0],
               ["type", "LST_LST_LSTCam", 20.0]
           ],
   "n_peaks": 0,
   "no_asymmetry": false,
   "use_weight": false,
   "verbose": 0
 }

}

IRFs are produced using energy dependent efficiency cut for gamma-hadron separation and theta corresponding to 80% gamma efficiency. A fixed cut at r=1 was also active.

irf_config

{ 

 "EventSelector": {
   "filters": {
     "intensity": [0, Infinity],
     "width": [0, Infinity],
     "length": [0, Infinity],
     "r": [0, 1],
     "wl": [0.01, 1],
     "leakage_intensity_width_2": [0, 1],
     "event_type": [32, 32]
   }
 },
 "DL3Cuts": {
   "min_event_p_en_bin": 100,
   "global_gh_cut": 0.7,
   "gh_efficiency": 0.8,
   "min_gh_cut": 0.1,
   "max_gh_cut": 0.95,
   "global_alpha_cut": 10,
   "global_theta_cut": 0.2,
   "theta_containment": 0.8,
   "alpha_containment": 0.68,
   "min_theta_cut": 0.05,
   "max_theta_cut": 0.32,
   "fill_theta_cut": 0.32,
   "min_alpha_cut": 1,
   "max_alpha_cut": 45,
   "fill_alpha_cut": 45,
   "allowed_tels": [1]
 },
 "DataBinning": {
   "true_energy_min": 0.005,
   "true_energy_max": 200,
   "true_energy_n_bins_per_decade": 5,
   "reco_energy_min": 0.005,
   "reco_energy_max": 200,
   "reco_energy_n_bins_per_decade": 5,
   "energy_migration_min": 0.2,
   "energy_migration_max": 5,
   "energy_migration_n_bins": 31,
   "fov_offset_min": 0.1,
   "fov_offset_max": 1.1,
   "fov_offset_n_edges": 9,
   "bkg_fov_offset_min": 0,
   "bkg_fov_offset_max": 10,
   "bkg_fov_offset_n_edges": 21,
   "source_offset_min": 0,
   "source_offset_max": 1,
   "source_offset_n_edges": 101
 }

}

DL3 files were created using IRF interpolation on the test grid and analysed with gammapy version 0.20.1. The preliminary spectrum was obtained using 5 bins per energy decades in reco energy and for the SED flux point, twice this number for the true energy binning and fitting a log parabola spectral model to the energy range between ~25 GeV and 10 TeV.

The 25 GeV threshold was used for two reasons :

- The excess rate drops significantly below this energy

Events vs energy BLlac8th 1.png

- Biases in flux level are seen at lower energy (too low upperlimit in standard reconstruction based analysis in the 16-25 GeV bin, in the 10-16 GeV bin for the LHfit + downward trend in the 16-25 GeV point)

SED all energy BLlac8th 1.png

Good agreement is found with the standard reconstruction analysis as seen in BLLac_2022_summer_conferences in the 25-10000GeV range using similar cuts for IRFs+DL3 except an additional 50 p.e. intensity cut.

Analysis results -- lstchain v0.9.x LHfit PR ongoing, for LST general meeting April 2022[edit]

Analysis performed in the BL lac paper TF before the LST general meeting. https://www.lst1.iac.es/wiki/index.php/BLLac_paper_meeting#collaborator_meeting

Current processing uses prod5_trans80 Zd20 south pointing MC only. Split between Zd20 and Zd 40 will be considered if time allows (small number of affected runs). RF used is with disp_vector direction reconstruction and a large set of parameters.

data file description for model parameters and points :

In /fefs/aswg/workspace/gabriel.emery/lhfit_withasymetry_rebasemaster17052021/notebook

BLlac_{all or 8th}_irf{irf number}{spectral_hypothesis}_dataset_{5219_to_5547 or 5552_to_5559}_flux_model_dict.dat

BLlac_{all or 8th}_irf{irf number}{spectral_hypothesis}_dataset_{5219_to_5547 or 5552_to_5559}_flux_pts.fits

irf number : 7 - fixed cuts, no intensity cuts. 8 - E dependent gh cut, no intensity cut. 9 - fixed cuts, 50 pe intensity cut spectral_hypothesis : LP, BPL or LP_refdecorrelation (LP with reference energy as decorrelation energy)

The first can be loaded with pickle to get the model parameters. The second contains, for the flare 8th of august, 7 HDU (SED, RUN_LC0039, RUN_LC0063, RUNLC_0100, SHORT_LC0039, SHORT_LC0063, SHORT_LC0100) for the SED points and lightcurves points or, for the full dataset, 2 (SED, NIGHT_LC0100).


August 8th[edit]

Group strategy :

Run list 5552--5559

SED on the night of Aug 8

- w/ intensity cut(>50 pe), LogParabolaSpectralModel

SED LP 50Icut irf9.png

- w/ intensity cut(>50 pe), SmoothBrokenPowerLawSpectralModel

SED BPL 50Icut irf9.png

- w/o intensity cut, LogParabolaSpectralModel

SED LP noIcut irf7.png

- w/o intensity cut, SmoothBrokenPowerLawSpectralModel

SED BLP noIcut irf7.png

./BLlac_8th_irf7BPL_dataset_5552_to_5559_flux_model_dict.dat


Light curve on the night of Aug 8 (>0.037, 0.057, 0.089 0.039, 0.063, 0.1 TeV) (runwise), from SED :

- w/ intensity cut(>50 pe), LogParabolaSpectralModel

SED LP 50Icut irf9 LC runwise.png

- w/ intensity cut(>50 pe), SmoothBrokenPowerLawSpectralModel

SED BPL 50Icut irf9 LC runwise.png

- w/o intensity cut, LogParabolaSpectralModel

SED LP noIcut irf7 LC runwise.png

- w/o intensity cut, SmoothBrokenPowerLawSpectralModel

SED BLP noIcut irf7 LC runwise.png

./BLlac_8th_irf7BPL_dataset_5552_to_5559_flux_pts.fits -> HUDs "RUN_LC0xxx"

Light curve on the night of Aug 8 (>0.037, 0.057, 0.089 0.039, 0.063, 0.1 TeV) with short time scale, each observation time divided by 3


LP w/ intensity > 50 pe

SED LP 50Icut irf9 LC 3perrun.png

sBPL w/ intensity > 50 pe

SED BPL 50Icut irf9 LC 3perrun.png

LP w/o intensity cut

SED LP noIcut irf7 LC 3perrun.png

sBPL w/o intensity cut

SED BLP noIcut irf7 LC 3perrun.png

./BLlac_8th_irf7BPL_dataset_5552_to_5559_flux_pts.fits -> HUDs "SHORT_LC0xxx"


July and August[edit]

Group strategy :

Run list

daily light curve above 100 GeV (for each SED?)


LP w/ intensity > 50 pe

July august SED LP refdecorelation 50Icut irf9.png July august SED LP refdecorelation 50Icut irf9 LC nightwise.png

sBPL w/ intensity > 50 pe

July august SED BPL 50Icut irf9.png July august SED BPL 50Icut irf9 LC nightwise.png


LP w/o intensity cut

July august SED LP refdecorelation noIcut irf7.png July august SED LP refdecorelation noIcut irf7 LC nightwise.png

sBPL w/o intensity cut

July august SED BPL noIcut irf7.png July august SED BPL noIcut irf7 LC nightwise.png

Previous analysis status[edit]

The dataset analysed in below is the August dataset from August 2nd to August 13th.

Clear detection and variability are obtained. The brightest night allows for the extraction of a spectrum with a threshold around 20-30 GeV.

Both a source independent and source dependent analysis are used with the likelihood method (and standard reconstruction crosscheck).

Run list[edit]

August 2nd to 13th :

5443,5444,5447,5448,5449,5450, 5454,5455,5456,5457,5458,5460,5466, 5481,5482, 5486,5488, 5546,5547, 5552,5553,5554,5555,5556,5557,5558,5559, 5592,5593,5594,5595, 5688,5689,5690, 5705,5706, 5732,5733,5734,5735,5736

August 8th : 5552,5553,5554,5555,5556,5557,5558,5559

LH-fit analysis (Shown at the LST general meeting Fall 2021)[edit]

MC it-cluster : /fefs/aswg/data/mc/DL0/20200629_prod5_trans_80/particle/zenith_20deg/south_pointing/
Code https://github.com/gabemery/cta-lstchain/tree/lhfit_wl_less1_and_numexpr
Config r0-dl1 it-cluster : /home/gabriel.emery/cta/lstchaindev/cta-lstchain-lhfit/lstchain/data/lstchain_lhfit_config_data_v15.json
Processed DL1 files it-cluster : /fefs/aswg/workspace/gabriel.emery/lhfit_withasymetry_rebasemaster17052021/data/DL1/v15/merged/


Analysis results -- lstchain v0.7.3[edit]

Presentation at LST general meeting (19/11/2021) https://indico.cta-observatory.org/event/3671/contributions/31294/attachments/20338/28385/LHfit_BLlac_flare_analysis_LSTGeneralMeetingFall2021.pdf

Main results :

Nice spectra extending to quite low energy (down to a couple of tens of GeV depending on the time window and analysis) and connecting, even if imperfectly, with the contemporaneous data from Fermi-LAT. MWL lightcurves display common and independent variability features. Analysis in source independent/dependent mode with LHfit/standard shown compatible spectra and similar light curves on the 8th. The spectra are also compatible with the simultaneous MAGIC data. Only in slides : ~3 minutes binned LC of the LST-1 LHfit analysis of the 8th shows similar variability as the run wise LC. LC below and above 100 GeV don't differ significantly.

- 2-13th of August MWL :

Srcindep 2 13 aug mwl lhfit bllac.png

- 8th of August MWL:

Srcindep 8 aug mwl lhfit bllac.png

- Multiple LST analysis and MAGIC comparison (AUg 8th):


MultianalysisandMAGIC 8 aug bllac.png

Retrieved from "https://www.lst1.iac.es/wiki/index.php?title=Analysis_BLLac_Aug_2021_Emery&oldid=6127"