Difference between revisions of "GRB221009A src-ind analysis moon"
(18 intermediate revisions by the same user not shown) | |||
Line 11: | Line 11: | ||
=General information= | =General information= | ||
− | * We have two different | + | * We have two different analyses for GRB221009A. First, the analysis with strong moon conditions, which occurred on October 10th (20221010, run_id=9602-9607) and 12th (20221012, run_id=9613-9617) (see section LST Observations in [[GRB221009A]]). Second, the analysis with dark/mild moon conditions, which data was taken between October 15th to October 28th. However, notice that data taking in the position of GRB2210009A was extended up to November. Thus, these observations contain fantastic background (probably). |
* The analysis is done using lstchain v0.9.9 or lstchain v0.9.10. The version release between them does not affect dramatically the offsite analysis. | * The analysis is done using lstchain v0.9.9 or lstchain v0.9.10. The version release between them does not affect dramatically the offsite analysis. | ||
+ | * Slides presented at the GRB221009A meetings about the source-independent moon analysis: | ||
+ | : [https://indico.cta-observatory.org/event/4414/#2-source-independent-data-anal Start] | ||
+ | : [https://indico.cta-observatory.org/event/4521/#3-source-independent-data-anal Moon analysis I] | ||
+ | : [https://indico.cta-observatory.org/event/4522/#3-source-independent-data-anal Moon analysis II] | ||
+ | : [https://indico.cta-observatory.org/event/4590/#3-source-independent-data-anal Moon analysis III] | ||
=Strong moon analysis= | =Strong moon analysis= | ||
− | Since the observations were done in strong moon conditions, the HV gain was reduced to 70%/50% of the nominal | + | Since the observations were done in strong moon conditions, the HV gain was reduced to 70% / 50% of the nominal gain (?) (see Elog for more information). By the time the data taking and the analysis was done, this analysis is not standard. |
− | == | + | ==Problems== |
− | Due to the | + | Due to the non-standard observation conditions, several aspects are affected. Are the following (probably more problems are missing): |
(Some of the things listed here come from an internal communication with Abelardo, thanks!) | (Some of the things listed here come from an internal communication with Abelardo, thanks!) | ||
− | :* Pedestal pixels: several Pixels can be tagged as unusable using the standard calibration config. | + | :* Pedestal pixels: several Pixels can be tagged as unusable using the standard calibration config. This will produce several pixels that are not calibrated. |
− | :* F-factor: Using lower HV, probably the F-factor will change since for a signal in the PMT, there will be more fluctuations due to the moon conditions. Do we have the characterisation of the PMTs in those conditions? Meaning, we do not have the gain fluctuations curve for | + | :* F-factor: Using lower HV, probably the F-factor will change since for a signal in the PMT, there will be more fluctuations due to the moon conditions. Do we have the characterisation of the PMTs in those conditions? Meaning, we do not have the gain fluctuations curve for these conditions. Then, due to the wrong F-factor, we will get the wrong conversion factors for data. This will act as an additional noise with an F-factor-style noise (gain fluctuations) that we should consider in the MC simulations. |
:* MC-real data match | :* MC-real data match | ||
− | ::* To match the correct gain response of the PMTs for simulation, the gain fluctuations curve for these conditions must be known. So, the simulations will not follow the real response if we do not have this curve. This will induce an addition noise with | + | ::* To match the correct gain response of the PMTs for simulation, the gain fluctuations curve for these conditions must be known. So, the simulations will not follow the real response if we do not have this curve. This will induce an addition noise with an "F-factor-style noise" (gain fluctuations) between MC and real data. |
− | ::* The high noise conditions will produce that we have high fluctuations in the PMTs. As | + | ::* The high noise conditions will produce that we have high fluctuations in the PMTs. As the standard cleaning method is the tail cut cleaning with pedestal std condition, several pixels will have increased pixel picture/boundary threshold because of the high std of the pedestal pixels. Then, this will produce an inhomogeneous camera response with changing image cleaning values along the camera. On the other hand, the MC cleaning will not have this kind of response because the pedestal std conditions are not used in the MC data because this condition is intended for stars (which are not considered in the current MC). Then, there will be a mismatch between the data and MC. |
− | ::* NSB tuning: The standard analysis procedure matches the NSB tuning of the data (Moon, stars in the FoV, ...) at the DL1 stage. The problem | + | ::* NSB tuning: The standard analysis procedure matches the NSB tuning of the data (Moon, stars in the FoV, ...) at the DL1 stage. The problem with doing so is that the added NSB will not be as similar to the actual NSB added in the PMT waveform. Also, if we do not assess and correct the additional F-factor-style noise mentioned above, it will contribute as a noise between MC and data to match the script, but due to the different behaviour, it will not be correctly filtered out using the current NSB tuning. (And maybe even the tuned MC with that NSB parameters will not be as good as the output if we would have only photon noise) |
* How to face these problems? | * How to face these problems? | ||
− | :* Pedestal pixels: I think Franca solved this issue increasing the nominal range of pedestal pixels to be tagged as usable. Not sure... | + | :* Pedestal pixels: I think Franca solved this issue by increasing the nominal range of pedestal pixels to be tagged as usable. Not sure... |
:* F-factor: One could consider applying an empirical correction for such high-NSB data. | :* F-factor: One could consider applying an empirical correction for such high-NSB data. | ||
Line 52: | Line 57: | ||
== General information == | == General information == | ||
− | Based on the problems we face in this analysis. | + | Based on the problems we face in this analysis. The following procedure is used (detailed information in each section): |
* 1. Calibrations: First, Franca reprocessed the DL1 data (taken on Oct. 10th) to have a better calibration and more usable pixels. | * 1. Calibrations: First, Franca reprocessed the DL1 data (taken on Oct. 10th) to have a better calibration and more usable pixels. | ||
− | * 2. Image cleaning: The runs at DL1 stage | + | * 2. Image cleaning: The runs at the DL1 stage were analysed to find the best image cleaning for each run that satisfied that only about 4% of the pedestal pixels will raise their picture thresholds due to the pedestal std dev condition, and not more than 5% pedestals survive the cleaning. The former condition is the restrictive one in work. The std value for the image cleaning with pedestal std is set to 2.5 (standard one). Next, we apply the best cleaning to DL1 real data. |
− | * 3. NSB tuning: We found the noise required to apply to the MC data in order to match the NSB conditions of the real data. Both the NSB tuning at the waveform level and at DL1 stage were used (using the tail cut cleaning values). However, for the former, the tuned MC gammas have a strange pixel charge distribution and the computation time is really high (between 1-2 days or 12h-1day if we consider lstchain PR 1066. Nevertheless, if we consider the latter code, all MC tuned files show the strange pixel charge distribution). We decided to proceed with the NSB tuning at DL1 stage | + | * 3. NSB tuning: We found the noise required to apply to the MC data in order to match the NSB conditions of the real data. Both the NSB tuning at the waveform level and at the DL1 stage were used (using the tail-cut cleaning values). However, for the former, the tuned MC gammas have a strange pixel charge distribution and the computation time is really high (between 1-2 days or 12h-1day if we consider lstchain PR 1066. Nevertheless, if we consider the latter code, all MC tuned files show the strange pixel charge distribution). We decided to proceed with the NSB tuning at the DL1 stage |
− | * 4. MC productions: based on | + | * 4. MC productions: based on steps 2 and 3, we decided the produce different MC production to analyse the data. |
=== Calibrations === | === Calibrations === | ||
− | You can see here the difference between the DL1 files produced with the standard calibration configuration (standard when the data was taken, v0.9) and the DL1 file recalibrated by Franca (specific paths can be found later) for a run taken | + | You can see here the difference between the DL1 files produced with the standard calibration configuration (standard when the data was taken, v0.9) and the DL1 file recalibrated by Franca (specific paths can be found later) for a run taken on Oct. 10th. |
<div><ul> | <div><ul> | ||
Line 70: | Line 75: | ||
</ul></div> | </ul></div> | ||
− | Due to high number of unusable pixels in the standard files (v0.9), the resulting picture threshold is really high because the picture threshold set to the unusable pixels is the maximum pedestal threshold value of the pedestal pixel in the camera. | + | Due to the high number of unusable pixels in the standard files (v0.9), the resulting picture threshold is really high because the picture threshold set to the unusable pixels is the maximum pedestal threshold value of the pedestal pixel in the camera. |
− | On the other hand, few unusable pixels are tagged in the standard files for runs recorded | + | On the other hand, a few unusable pixels are tagged in the standard files for runs recorded on Oct. 12th. |
<div><ul> | <div><ul> | ||
Line 79: | Line 84: | ||
</ul></div> | </ul></div> | ||
+ | Note: the calibration should be improved in the near future with Cat B calibration (ongoing work by Franca). | ||
=== Image cleaning === | === Image cleaning === | ||
− | You can see how the picture threshold value evolve for the different runs | + | You can see how the picture threshold value evolve for the different runs on Oct. 10th and 12th. The picture threshold increase as we increase the run_id for a given day because the moon was rising. Notice that the picture threshold for run 9613 is higher than the rest of the runs on Oct. 12th because the HV was reduced to 50%, not to 70% of the nominal value as in the rest of the runs. The picture threshold is estimated using <Q_{ped}> + 2.5*\sigma_{ped}. The value is set to the value that corresponds to the 96% of the CDF. |
<div><ul> | <div><ul> | ||
Line 88: | Line 94: | ||
<li style="display: inline-block;"> [[File:image-cleaning20221012.jpeg|thumb|none|600px|Run-wise picture threshold distribution for runs taken in Oct. 12th.]] </li> | <li style="display: inline-block;"> [[File:image-cleaning20221012.jpeg|thumb|none|600px|Run-wise picture threshold distribution for runs taken in Oct. 12th.]] </li> | ||
</ul></div> | </ul></div> | ||
+ | |||
+ | The boundary threshold is estimated using <Q_{ped}> + 1*\sigma_{ped}. The value is set to CDF=0.96 as for the case of the picture threshold. | ||
=== NSB tuning === | === NSB tuning === | ||
− | Here you can see the pixel charge distribution between real data and tuned MC data using NSB tuning at the waveform (right and wrong tuning?). The NSB tuning at the DL1 stage is added to the right plot. | + | Here you can see the pixel charge distribution between real data (cosmics) and tuned MC gamma data using NSB tuning at the waveform (right and wrong tuning?). The NSB tuning at the DL1 stage is added to the right plot using the standard script of NSB tuning. |
+ | |||
+ | <div><ul> | ||
+ | <li style="display: inline-block;"> [[File:pixel-charge-dist-NSBwaveform.jpg|thumb|none|450px|Pixel charge dist for real cosmics (orange, run 9602) and tuned MC gamma data for two runs, one with strange distribution (green) and another closer to the real data distribution (blue).]] </li> | ||
+ | <li style="display: inline-block;"> [[File:pixel-charge-dist-NSBDL1.jpg|thumb|none|400px|Pixel charge dist for real cosmics (orange, run 9602) and tuned MC gamma data using the tuning at the waveform level (green) and at DL1 stage (blue).]] </li> | ||
+ | </ul></div> | ||
+ | |||
+ | However, the NSB tuning script is not intended to be applied for high NSB tuning due to the hard-coded bounds in the charge range (see scripts). | ||
+ | To solve these problems, I extended the bounds and modified manually the parameters to match the MC-biased pixel pedestal distribution to the real data pedestals. Manually I checked that the peak of the distributions match using a linear scale, in other words, I prioritised the matching of the peaks over the tails of the distribution. Unfortunately, the match at the tails of the distributions is not perfect (in log scale). | ||
+ | |||
+ | <div><ul> | ||
+ | <li style="display: inline-block;"> [[File:pedestals9602-DL1NSB-linear.png|thumb|none|350px|Pedestal charge distribution for cosmics data (orange, run 9602) and tuned MC pedestals (biased extractor). Plot with y-axis in linear scale.]] </li> | ||
+ | <li style="display: inline-block;"> [[File:pedestals9602-DL1NSB-log.png|thumb|none|350px|Pedestal charge distribution for cosmics data (orange, run 9602) and tuned MC pedestals (biased extractor). Plot with y-axis in log scale.]] </li> | ||
+ | </ul></div> | ||
<div><ul> | <div><ul> | ||
− | <li style="display: inline-block;"> [[File: | + | <li style="display: inline-block;"> [[File:gammas9602-DL1NSB-linear.png|thumb|none|350px|Pixel charge distribution for real cosmics (orange) and tuned MC gammas at the DL1 stage with tuned DL1 NSB parameters (blue). Plot with y-axis in linear scale.]] </li> |
− | <li style="display: inline-block;"> [[File: | + | <li style="display: inline-block;"> [[File:gammas9602-DL1NSB-log.png|thumb|none|350px|Pixel charge distribution for real cosmics (orange) and tuned MC gammas at the DL1 stage with tuned DL1 NSB parameters (blue). Plot with y-axis in log scale.]] </li> |
</ul></div> | </ul></div> | ||
− | It is clear that MC data and real data do not fully match | + | It is clear that MC data and real data do not fully match the pixel charge distribution. However, the modified DL1 NSB tuning looks better than the results with the standard NSB tuning script. |
+ | |||
==Monte Carlo information== | ==Monte Carlo information== | ||
* Link to MC files used (base): /fefs/aswg/data/mc/DL1/AllSky/20221027_v0.9.9_base_prod/* | * Link to MC files used (base): /fefs/aswg/data/mc/DL1/AllSky/20221027_v0.9.9_base_prod/* | ||
− | ** Particle types: standard particle types for AllSky MC | + | ** Particle types: standard particle types for AllSky MC production |
** DEC band: dec_2276 | ** DEC band: dec_2276 | ||
=== MC productions === | === MC productions === | ||
− | + | Productions with: | |
− | |||
::* lstchain v0.9.10 | ::* lstchain v0.9.10 | ||
::* lstMCpipe v0.10 | ::* lstMCpipe v0.10 | ||
+ | |||
+ | Based on the image cleaning and NSB tuning. We decided to produce the following MC productions. | ||
* Date: 20221010 | * Date: 20221010 | ||
− | :* MC production 1 | + | :* MC production 1: 20230205_GRB221009A_moon_strong_ph1_NSBDL1 |
::* Runs to analyse (run_id): 9602-9604 | ::* Runs to analyse (run_id): 9602-9604 | ||
− | ::* Tail cut cleaning (picture thd., boundary thd.): | + | ::* Tail cut cleaning (picture thd., boundary thd.): XX, XX |
::* NSB tuning (from mean parameters values using run 9603 subruns): | ::* NSB tuning (from mean parameters values using run 9603 subruns): | ||
"increase_nsb": true, | "increase_nsb": true, | ||
− | "extra_noise_in_dim_pixels": | + | "extra_noise_in_dim_pixels": XXX, |
− | "extra_bias_in_dim_pixels": | + | "extra_bias_in_dim_pixels": XXX, |
"transition_charge": 8, | "transition_charge": 8, | ||
− | "extra_noise_in_bright_pixels": | + | "extra_noise_in_bright_pixels": XXX, |
::* Configuration files | ::* Configuration files | ||
− | fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ | + | /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ |
− | /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ | + | /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ |
− | :* MC production 2 | + | :* MC production 2: 20230205_GRB221009A_moon_strong_ph2_NSBDL1 |
::* Runs to analyse (run_id): 9605-9607 | ::* Runs to analyse (run_id): 9605-9607 | ||
− | ::* Tail cut cleaning (picture thd., boundary thd.): | + | ::* Tail cut cleaning (picture thd., boundary thd.): XX, XX |
− | ::* NSB tuning (from mean parameters values using run | + | ::* NSB tuning (from mean parameters values using run 9606 subruns): |
"increase_nsb": true, | "increase_nsb": true, | ||
− | "extra_noise_in_dim_pixels": | + | "extra_noise_in_dim_pixels": XXX, |
− | "extra_bias_in_dim_pixels": | + | "extra_bias_in_dim_pixels": XXX, |
"transition_charge": 8, | "transition_charge": 8, | ||
− | "extra_noise_in_bright_pixels": | + | "extra_noise_in_bright_pixels": XXX, |
::* Configuration files | ::* Configuration files | ||
− | fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ | + | /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ |
− | /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ | + | /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ |
* Date: 20221012 | * Date: 20221012 | ||
− | :* MC production 1 | + | :* MC production 1: 20230205_GRB221009A_moon_strong_ph3_NSBDL1 |
::* Runs to analyse (run_id): 9613 | ::* Runs to analyse (run_id): 9613 | ||
− | ::* Tail cut cleaning (picture thd., boundary thd.): | + | ::* Tail cut cleaning (picture thd., boundary thd.): XX, XX |
::* NSB tuning (from mean parameters values using run 9613 subruns): | ::* NSB tuning (from mean parameters values using run 9613 subruns): | ||
"increase_nsb": true, | "increase_nsb": true, | ||
− | "extra_noise_in_dim_pixels": | + | "extra_noise_in_dim_pixels": XXX, |
− | "extra_bias_in_dim_pixels": | + | "extra_bias_in_dim_pixels": XXX, |
"transition_charge": 8, | "transition_charge": 8, | ||
− | "extra_noise_in_bright_pixels": | + | "extra_noise_in_bright_pixels": XXX, |
::* Configuration files | ::* Configuration files | ||
− | fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ | + | /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ |
− | /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ | + | /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ |
− | :* MC production 2 | + | :* MC production 2: 20230205_GRB221009A_moon_strong_ph4_NSBDL1 |
::* Runs to analyse (run_id): 9614-9615 | ::* Runs to analyse (run_id): 9614-9615 | ||
− | ::* Tail cut cleaning (picture thd., boundary thd.): | + | ::* Tail cut cleaning (picture thd., boundary thd.): XX, XX |
− | ::* NSB tuning: | + | ::* NSB tuning (from mean parameters values using run 9614 and 9615 subruns): |
"increase_nsb": true, | "increase_nsb": true, | ||
− | "extra_noise_in_dim_pixels": | + | "extra_noise_in_dim_pixels": XXX, |
− | "extra_bias_in_dim_pixels": | + | "extra_bias_in_dim_pixels": XXX, |
"transition_charge": 8, | "transition_charge": 8, | ||
− | "extra_noise_in_bright_pixels": | + | "extra_noise_in_bright_pixels": XXX, |
::* Configuration files | ::* Configuration files | ||
− | fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ | + | /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ |
− | /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ | + | /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ |
− | :* MC production 3 | + | :* MC production 3: 20230205_GRB221009A_moon_strong_ph5_NSBDL1 |
::* Runs to analyse (run_id): 9616-9617 | ::* Runs to analyse (run_id): 9616-9617 | ||
− | ::* Tail cut cleaning (picture thd., boundary thd.): | + | ::* Tail cut cleaning (picture thd., boundary thd.): XX, XX |
− | ::* NSB tuning: | + | ::* NSB tuning (from mean parameters values using run 9616 and 9617 subruns): |
"increase_nsb": true, | "increase_nsb": true, | ||
− | "extra_noise_in_dim_pixels": | + | "extra_noise_in_dim_pixels": XXX, |
− | "extra_bias_in_dim_pixels": | + | "extra_bias_in_dim_pixels": XXX, |
"transition_charge": 8, | "transition_charge": 8, | ||
− | "extra_noise_in_bright_pixels": | + | "extra_noise_in_bright_pixels": XXX, |
::* Configuration files | ::* Configuration files | ||
− | fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ | + | /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/XX |
− | /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ | + | /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/XX |
==DL1 data== | ==DL1 data== | ||
− | + | * original DL1a files | |
/fefs/aswg/workspace/franca.cassol/data/real/DL1/20221010_test1/dl1_LST-1.Run* | /fefs/aswg/workspace/franca.cassol/data/real/DL1/20221010_test1/dl1_LST-1.Run* | ||
Line 189: | Line 212: | ||
* Date: 20221010 | * Date: 20221010 | ||
:* MC production 1 | :* MC production 1 | ||
− | ::* Produced DL1b data: | + | ::* Produced DL1b data: tailcutXX-XX (with cleaning based on pedestal RMS), dynamic cleaning |
− | /fefs/aswg/workspace/arnau.aguasca/Analysis/results/real/DL1/20221010/srcind/v0.9.9/ | + | /fefs/aswg/workspace/arnau.aguasca/Analysis/results/real/DL1/20221010/srcind/v0.9.9/XX |
− | /fefs/aswg/workspace/MC_data_simlink/DL1/AllSky/ | + | /fefs/aswg/workspace/MC_data_simlink/DL1/AllSky/XX |
* Date: 20221010 | * Date: 20221010 | ||
:* MC production 2 | :* MC production 2 | ||
− | ::* Produced DL1b data: | + | ::* Produced DL1b data: tailcutXX-XX (with cleaning based on pedestal RMS), dynamic cleaning |
− | /fefs/aswg/workspace/arnau.aguasca/Analysis/results/real/DL1/20221010/srcind/v0.9.9/ | + | /fefs/aswg/workspace/arnau.aguasca/Analysis/results/real/DL1/20221010/srcind/v0.9.9/XXX |
− | /fefs/aswg/workspace/MC_data_simlink/DL1/AllSky/ | + | /fefs/aswg/workspace/MC_data_simlink/DL1/AllSky/XXX |
− | + | * original DL1a files | |
/fefs/aswg/data/real/DL1/20221012/v0.9/tailcut84/dl1_LST-1.Run0961* | /fefs/aswg/data/real/DL1/20221012/v0.9/tailcut84/dl1_LST-1.Run0961* | ||
Line 207: | Line 230: | ||
* Date: 20221012 | * Date: 20221012 | ||
:* MC production 1 | :* MC production 1 | ||
− | ::* Produced DL1b data: | + | ::* Produced DL1b data: tailcutXX-XX (with cleaning based on pedestal RMS), dynamic cleaning |
− | /fefs/aswg/workspace/arnau.aguasca/Analysis/results/real/DL1/ | + | /fefs/aswg/workspace/arnau.aguasca/Analysis/results/real/DL1/20221012/srcind/v0.9.9/XXX |
− | /fefs/aswg/workspace/MC_data_simlink/DL1/AllSky/ | + | /fefs/aswg/workspace/MC_data_simlink/DL1/AllSky/XX |
* Date: 20221012 | * Date: 20221012 | ||
:* MC production 2 | :* MC production 2 | ||
− | ::* Produced DL1b data: | + | ::* Produced DL1b data: tailcutXX-XX (with cleaning based on pedestal RMS), dynamic cleaning |
− | /fefs/aswg/workspace/arnau.aguasca/Analysis/results/real/DL1/ | + | /fefs/aswg/workspace/arnau.aguasca/Analysis/results/real/DL1/20221012/srcind/v0.9.9/XXX |
− | /fefs/aswg/workspace/MC_data_simlink/DL1/AllSky/ | + | /fefs/aswg/workspace/MC_data_simlink/DL1/AllSky/XX |
* Date: 20221012 | * Date: 20221012 | ||
:* MC production 3 | :* MC production 3 | ||
− | ::* Produced DL1b data: | + | ::* Produced DL1b data: tailcutXX-XX (with cleaning based on pedestal RMS), dynamic cleaning |
− | /fefs/aswg/workspace/arnau.aguasca/Analysis/results/real/DL1/ | + | /fefs/aswg/workspace/arnau.aguasca/Analysis/results/real/DL1/20221012/srcind/v0.9.9/XX |
− | /fefs/aswg/workspace/MC_data_simlink/DL1/AllSky/ | + | /fefs/aswg/workspace/MC_data_simlink/DL1/AllSky/XX |
==Random forest== | ==Random forest== | ||
Line 227: | Line 250: | ||
* lstchain-0.9.10 | * lstchain-0.9.10 | ||
− | * source-indep ( | + | * source-indep (the standard MC files for AllSky MC production) |
− | /fefs/aswg/workspace/MC_data_simlink/models/AllSky/ | + | /fefs/aswg/workspace/MC_data_simlink/models/AllSky/XX |
+ | /fefs/aswg/workspace/MC_data_simlink/models/AllSky/XX | ||
+ | /fefs/aswg/workspace/MC_data_simlink/models/AllSky/XX | ||
+ | /fefs/aswg/workspace/MC_data_simlink/models/AllSky/XX | ||
+ | /fefs/aswg/workspace/MC_data_simlink/models/AllSky/XX | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
==DL2 data== | ==DL2 data== | ||
+ | |||
+ | NOTHING DONE FROM THERE | ||
+ | |||
Information about your DL2 data and settings such as: specifc versions of lstchain, specfic .json version used. | Information about your DL2 data and settings such as: specifc versions of lstchain, specfic .json version used. | ||
Line 279: | Line 318: | ||
=== Theta2 plot === | === Theta2 plot === | ||
− | |||
− | |||
− | |||
− | |||
=== Spectral results === | === Spectral results === | ||
− | |||
− |
Latest revision as of 14:28, 7 March 2023
Back to the Data analysis page
Go back to Transient Working Group.
Go Back to Gamma-Ray Bursts (GRBs).
Go back to GRB221009A.
- Analysis by A. Aguasca-Cabot (Universitat de Barcelona - arnau.aguasca@fqa.ub.edu)
General information[edit]
- We have two different analyses for GRB221009A. First, the analysis with strong moon conditions, which occurred on October 10th (20221010, run_id=9602-9607) and 12th (20221012, run_id=9613-9617) (see section LST Observations in GRB221009A). Second, the analysis with dark/mild moon conditions, which data was taken between October 15th to October 28th. However, notice that data taking in the position of GRB2210009A was extended up to November. Thus, these observations contain fantastic background (probably).
- The analysis is done using lstchain v0.9.9 or lstchain v0.9.10. The version release between them does not affect dramatically the offsite analysis.
- Slides presented at the GRB221009A meetings about the source-independent moon analysis:
Strong moon analysis[edit]
Since the observations were done in strong moon conditions, the HV gain was reduced to 70% / 50% of the nominal gain (?) (see Elog for more information). By the time the data taking and the analysis was done, this analysis is not standard.
Problems[edit]
Due to the non-standard observation conditions, several aspects are affected. Are the following (probably more problems are missing): (Some of the things listed here come from an internal communication with Abelardo, thanks!)
- Pedestal pixels: several Pixels can be tagged as unusable using the standard calibration config. This will produce several pixels that are not calibrated.
- F-factor: Using lower HV, probably the F-factor will change since for a signal in the PMT, there will be more fluctuations due to the moon conditions. Do we have the characterisation of the PMTs in those conditions? Meaning, we do not have the gain fluctuations curve for these conditions. Then, due to the wrong F-factor, we will get the wrong conversion factors for data. This will act as an additional noise with an F-factor-style noise (gain fluctuations) that we should consider in the MC simulations.
- MC-real data match
- To match the correct gain response of the PMTs for simulation, the gain fluctuations curve for these conditions must be known. So, the simulations will not follow the real response if we do not have this curve. This will induce an addition noise with an "F-factor-style noise" (gain fluctuations) between MC and real data.
- The high noise conditions will produce that we have high fluctuations in the PMTs. As the standard cleaning method is the tail cut cleaning with pedestal std condition, several pixels will have increased pixel picture/boundary threshold because of the high std of the pedestal pixels. Then, this will produce an inhomogeneous camera response with changing image cleaning values along the camera. On the other hand, the MC cleaning will not have this kind of response because the pedestal std conditions are not used in the MC data because this condition is intended for stars (which are not considered in the current MC). Then, there will be a mismatch between the data and MC.
- NSB tuning: The standard analysis procedure matches the NSB tuning of the data (Moon, stars in the FoV, ...) at the DL1 stage. The problem with doing so is that the added NSB will not be as similar to the actual NSB added in the PMT waveform. Also, if we do not assess and correct the additional F-factor-style noise mentioned above, it will contribute as a noise between MC and data to match the script, but due to the different behaviour, it will not be correctly filtered out using the current NSB tuning. (And maybe even the tuned MC with that NSB parameters will not be as good as the output if we would have only photon noise)
- How to face these problems?
- Pedestal pixels: I think Franca solved this issue by increasing the nominal range of pedestal pixels to be tagged as usable. Not sure...
- F-factor: One could consider applying an empirical correction for such high-NSB data.
- MC-real data match (1): We need the gain fluctuations curve.
- MC-real data match (2): the levels of tail cut cleaning should be high enough such that only a handful of pixels get raised thresholds, and therefore data and MC are more or less similar.
- MC-real data match (NSB): use the NSB tuning at the waveform level, or make specific MC simulations matching directly the NSB of the runs.
General information[edit]
Based on the problems we face in this analysis. The following procedure is used (detailed information in each section):
- 1. Calibrations: First, Franca reprocessed the DL1 data (taken on Oct. 10th) to have a better calibration and more usable pixels.
- 2. Image cleaning: The runs at the DL1 stage were analysed to find the best image cleaning for each run that satisfied that only about 4% of the pedestal pixels will raise their picture thresholds due to the pedestal std dev condition, and not more than 5% pedestals survive the cleaning. The former condition is the restrictive one in work. The std value for the image cleaning with pedestal std is set to 2.5 (standard one). Next, we apply the best cleaning to DL1 real data.
- 3. NSB tuning: We found the noise required to apply to the MC data in order to match the NSB conditions of the real data. Both the NSB tuning at the waveform level and at the DL1 stage were used (using the tail-cut cleaning values). However, for the former, the tuned MC gammas have a strange pixel charge distribution and the computation time is really high (between 1-2 days or 12h-1day if we consider lstchain PR 1066. Nevertheless, if we consider the latter code, all MC tuned files show the strange pixel charge distribution). We decided to proceed with the NSB tuning at the DL1 stage
- 4. MC productions: based on steps 2 and 3, we decided the produce different MC production to analyse the data.
Calibrations[edit]
You can see here the difference between the DL1 files produced with the standard calibration configuration (standard when the data was taken, v0.9) and the DL1 file recalibrated by Franca (specific paths can be found later) for a run taken on Oct. 10th.
Due to the high number of unusable pixels in the standard files (v0.9), the resulting picture threshold is really high because the picture threshold set to the unusable pixels is the maximum pedestal threshold value of the pedestal pixel in the camera.
On the other hand, a few unusable pixels are tagged in the standard files for runs recorded on Oct. 12th.
Note: the calibration should be improved in the near future with Cat B calibration (ongoing work by Franca).
Image cleaning[edit]
You can see how the picture threshold value evolve for the different runs on Oct. 10th and 12th. The picture threshold increase as we increase the run_id for a given day because the moon was rising. Notice that the picture threshold for run 9613 is higher than the rest of the runs on Oct. 12th because the HV was reduced to 50%, not to 70% of the nominal value as in the rest of the runs. The picture threshold is estimated using <Q_{ped}> + 2.5*\sigma_{ped}. The value is set to the value that corresponds to the 96% of the CDF.
The boundary threshold is estimated using <Q_{ped}> + 1*\sigma_{ped}. The value is set to CDF=0.96 as for the case of the picture threshold.
NSB tuning[edit]
Here you can see the pixel charge distribution between real data (cosmics) and tuned MC gamma data using NSB tuning at the waveform (right and wrong tuning?). The NSB tuning at the DL1 stage is added to the right plot using the standard script of NSB tuning.
However, the NSB tuning script is not intended to be applied for high NSB tuning due to the hard-coded bounds in the charge range (see scripts). To solve these problems, I extended the bounds and modified manually the parameters to match the MC-biased pixel pedestal distribution to the real data pedestals. Manually I checked that the peak of the distributions match using a linear scale, in other words, I prioritised the matching of the peaks over the tails of the distribution. Unfortunately, the match at the tails of the distributions is not perfect (in log scale).
It is clear that MC data and real data do not fully match the pixel charge distribution. However, the modified DL1 NSB tuning looks better than the results with the standard NSB tuning script.
Monte Carlo information[edit]
- Link to MC files used (base): /fefs/aswg/data/mc/DL1/AllSky/20221027_v0.9.9_base_prod/*
- Particle types: standard particle types for AllSky MC production
- DEC band: dec_2276
MC productions[edit]
Productions with:
- lstchain v0.9.10
- lstMCpipe v0.10
Based on the image cleaning and NSB tuning. We decided to produce the following MC productions.
- Date: 20221010
- MC production 1: 20230205_GRB221009A_moon_strong_ph1_NSBDL1
- Runs to analyse (run_id): 9602-9604
- Tail cut cleaning (picture thd., boundary thd.): XX, XX
- NSB tuning (from mean parameters values using run 9603 subruns):
"increase_nsb": true, "extra_noise_in_dim_pixels": XXX, "extra_bias_in_dim_pixels": XXX, "transition_charge": 8, "extra_noise_in_bright_pixels": XXX,
- Configuration files
/fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/
- MC production 2: 20230205_GRB221009A_moon_strong_ph2_NSBDL1
- Runs to analyse (run_id): 9605-9607
- Tail cut cleaning (picture thd., boundary thd.): XX, XX
- NSB tuning (from mean parameters values using run 9606 subruns):
"increase_nsb": true, "extra_noise_in_dim_pixels": XXX, "extra_bias_in_dim_pixels": XXX, "transition_charge": 8, "extra_noise_in_bright_pixels": XXX,
- Configuration files
/fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/
- Date: 20221012
- MC production 1: 20230205_GRB221009A_moon_strong_ph3_NSBDL1
- Runs to analyse (run_id): 9613
- Tail cut cleaning (picture thd., boundary thd.): XX, XX
- NSB tuning (from mean parameters values using run 9613 subruns):
"increase_nsb": true, "extra_noise_in_dim_pixels": XXX, "extra_bias_in_dim_pixels": XXX, "transition_charge": 8, "extra_noise_in_bright_pixels": XXX,
- Configuration files
/fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/
- MC production 2: 20230205_GRB221009A_moon_strong_ph4_NSBDL1
- Runs to analyse (run_id): 9614-9615
- Tail cut cleaning (picture thd., boundary thd.): XX, XX
- NSB tuning (from mean parameters values using run 9614 and 9615 subruns):
"increase_nsb": true, "extra_noise_in_dim_pixels": XXX, "extra_bias_in_dim_pixels": XXX, "transition_charge": 8, "extra_noise_in_bright_pixels": XXX,
- Configuration files
/fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/ /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/
- MC production 3: 20230205_GRB221009A_moon_strong_ph5_NSBDL1
- Runs to analyse (run_id): 9616-9617
- Tail cut cleaning (picture thd., boundary thd.): XX, XX
- NSB tuning (from mean parameters values using run 9616 and 9617 subruns):
"increase_nsb": true, "extra_noise_in_dim_pixels": XXX, "extra_bias_in_dim_pixels": XXX, "transition_charge": 8, "extra_noise_in_bright_pixels": XXX,
- Configuration files
/fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/XX /fefs/aswg/workspace/arnau.aguasca/scripts/lstMCpipe/v0.10.X/XX
DL1 data[edit]
- original DL1a files
/fefs/aswg/workspace/franca.cassol/data/real/DL1/20221010_test1/dl1_LST-1.Run* /fefs/aswg/data/mc/DL1/AllSky/20221027_v0.9.9_base_prod/*
- Date: 20221010
- MC production 1
- Produced DL1b data: tailcutXX-XX (with cleaning based on pedestal RMS), dynamic cleaning
/fefs/aswg/workspace/arnau.aguasca/Analysis/results/real/DL1/20221010/srcind/v0.9.9/XX /fefs/aswg/workspace/MC_data_simlink/DL1/AllSky/XX
- Date: 20221010
- MC production 2
- Produced DL1b data: tailcutXX-XX (with cleaning based on pedestal RMS), dynamic cleaning
/fefs/aswg/workspace/arnau.aguasca/Analysis/results/real/DL1/20221010/srcind/v0.9.9/XXX /fefs/aswg/workspace/MC_data_simlink/DL1/AllSky/XXX
- original DL1a files
/fefs/aswg/data/real/DL1/20221012/v0.9/tailcut84/dl1_LST-1.Run0961* /fefs/aswg/data/mc/DL1/AllSky/20221027_v0.9.9_base_prod/*
- Date: 20221012
- MC production 1
- Produced DL1b data: tailcutXX-XX (with cleaning based on pedestal RMS), dynamic cleaning
/fefs/aswg/workspace/arnau.aguasca/Analysis/results/real/DL1/20221012/srcind/v0.9.9/XXX /fefs/aswg/workspace/MC_data_simlink/DL1/AllSky/XX
- Date: 20221012
- MC production 2
- Produced DL1b data: tailcutXX-XX (with cleaning based on pedestal RMS), dynamic cleaning
/fefs/aswg/workspace/arnau.aguasca/Analysis/results/real/DL1/20221012/srcind/v0.9.9/XXX /fefs/aswg/workspace/MC_data_simlink/DL1/AllSky/XX
- Date: 20221012
- MC production 3
- Produced DL1b data: tailcutXX-XX (with cleaning based on pedestal RMS), dynamic cleaning
/fefs/aswg/workspace/arnau.aguasca/Analysis/results/real/DL1/20221012/srcind/v0.9.9/XX /fefs/aswg/workspace/MC_data_simlink/DL1/AllSky/XX
Random forest[edit]
- lstchain-0.9.10
- source-indep (the standard MC files for AllSky MC production)
/fefs/aswg/workspace/MC_data_simlink/models/AllSky/XX /fefs/aswg/workspace/MC_data_simlink/models/AllSky/XX /fefs/aswg/workspace/MC_data_simlink/models/AllSky/XX /fefs/aswg/workspace/MC_data_simlink/models/AllSky/XX /fefs/aswg/workspace/MC_data_simlink/models/AllSky/XX
DL2 data[edit]
NOTHING DONE FROM THERE
Information about your DL2 data and settings such as: specifc versions of lstchain, specfic .json version used.
Example:
- lstchain-0.7.6.dev242+g2086cb9
- source-indep
/fefs/aswg/workspace/seiya.nozaki/data/MC/v0.7.5/tailcut84_dynamic_bllac/srcindep/DL2/data/ /fefs/aswg/workspace/seiya.nozaki/data/BLLac/v0.7.3/tailcut84_dynamic_v075/srcindep/DL2/data/dl2_LST-1.Run0XXXX_merged.h5
- source-dep
/fefs/aswg/workspace/seiya.nozaki/data/MC/v0.7.5/tailcut84_dynamic_bllac/srcdep/DL2/data/ /fefs/aswg/workspace/seiya.nozaki/data/BLLac/v0.7.3/tailcut84_dynamic_v075/srcdep/DL2/data/dl2_LST-1.Run0XXXX_merged.h5
DL3 data selection[edit]
Information about your DL3 data selection.
Example
- intensity > 50
- r: [0, 1 ]
- wl: [0.1, 1 ]
- leakage_intensity_width_2: [0, 0.2 ]
- source-indep
- fixed_gh_cut: 0.3
- fixed_theta_cut: 0.2
- source-dep
- fixed_gh_cut: 0.7
- fixed_alpha_cut: 10
High-level analysis[edit]
Please put any information about the production of higher level analysis here.
Example
- lstchain to generate source-dep IRF, DL3
- Science Tool: gammapy 0.18.2
- point-like IRF, 1D analysis
Analysis Results[edit]
Please place higher-level analysis results (Spectra, SkyMaps, Lightcurves, etc) here.