DL1 data check

Where to find the DL1 data check products

The DL1 data check products can be found inside this Browsable directory Always use the most recent version vx.y of the check products which is available for the night of interest. As of March 2024, the most recent version is v0.10.

Inside each of the night-wise YYYYMMDD directories, e.g. v0.10/20231118, you can find three files: DL1_datacheck_20231118.[html, log, h5], which contain information relative to the whole night of observations (for the sky runs that were successfully processed by lstosa). There is also a pdf subdirectory which contains one .pdf file (with further useful information) per run. More details on these products are provided below.

Reporting issues

If problems are found in the data, experts should be contacted as soon as possible. This can be done via Slack (in the appropriate channel, depending on the problem) and / or submitting a reply to the ELOG entry of the corresponding night. Whenever a problem is found in the data check it is convenient to check the ELOG to verify if the shift crew reported anything that may be related to the problem (e.g. a car flash).

The html DL1_datacheck file

This is the most relevant one for checking the good behaviour of the telescope during the night. It is an interactive webpage showing various quantities (both pixel-wise and camera-wise) averaged over the whole run. Many of the plots have a slider which allows you to select the run number.

Here is an example: DL1_datacheck_20231214.html. Go to datacheck/dl1/v0.10 to find the latest available nights.

General comments:

• Some of the quantities are displayed together with limits (min/max, indicated by dashed orange and red lines) within which good data are usually found. The limits may change through the night (from run to run), because the limits are sometimes pointing-dependent (e.g. rates of showers - "cosmics" - depend significantly on the zenith angle). Note that the current limits were set in 2021 and some of them may require regular updates. The values are as of now hard-coded in the lstchain script lstchain_longterm_dl1_check.py, which is the one which produces the night-wise DL1 check outputs.

• The limits, and even the computation of some of the data check quantities, like e.g. everything related to muon rings, will not be valid for data taken under very high NSB conditions (which can be identified by the large values of the mean and std deviation of charges recorded in interleaved pedestal events). If limits are exceeded for such data, there is no reason to conclude that the telescope is malfunctioning.

• The quantities displayed in the html file are all run averages, there is no information about the evolution within a run. Inside the "pdf" directory under each of the YYYYMMDD folders you will find a pdf file for each of the runs of the night. They display the evolution subrun-by-subrun (i.e., every few seconds) of some quantities, and may be helpful to understand some of the possible issues spotted when checking the run-wise averages in the html file.

• By moving the mouse over the plots you can obtain useful information, like run number, pixel / cluster number, numerical value of the displayed quantity, etc.

What to check in each tab of the DL1 datacheck html file?

1. Event rates
   • This tab shows the average rate (vs. run number) of different types of events: interleaved pedestals; interleaved flatfield; "cosmics" (i.e. physics triggers, which are mostly showers when everything works well); and "contained muon rings" (those cosmics recognized as muons, and not truncated by the camera edges).
   • The limits for cosmics and muon rings are for data taken in good conditions (and with not too high NSB). Data taken in moon conditions (with reduced HV in the PMTs) will be beyond limits, even if the telescope is perfectly fine.
   • The rate of interleaved pedestals and interleaved FF is smaller than 100 Hz because of dead time. If they fall below ~90 Hz it may be due to increased dead time (e.g. because of occasional DAQ issues). Sometimes they may be completely missing - this may indicate either a problem with the event tagging, or perhaps the interleaved events were not even generated. This is a serious problem and should be reported immediately.
   • Only sky runs are displayed in the html file. If you see gaps in the time axis of the plots, it may be simply due to a pause in observations. If the run numbers before and after the gap are not consecutive (you can know the run number placing the mouse over a point) it may mean that non-sky runs (for example, "pedcal" calibration runs) were taken. You can check this in the ELOG of the corresponding night (use the "Find" utility if it is not the latest night). If the missing runs are not calibrations, but sky runs, then the LST on-site analysis team should be informed.
2. Trigger tags (both tabs)
   • these tabs are obsolete since 20231206, when we started version 6 of the Event builder!
   • TBD! (note for developers): keep the EvB5-specific info (like UCTS jumps) for older data in a dedicated tab. Replace the rest by a tab showing checks that the recorded images for the different event types look like they should (fraction of events surviving cleaning may be a good observable).

List of known problems

Below is a (very incomplete!) list of some runs with known problems based on the data checks, which should not be used for higher data analysis (unless some special treatment is applied to solve their issues). You can use this to add the problems you spot in the data.

• 20201123/v0.9/tailcut84/dl1_LST-1.Run03038 "CDTS not connected"
• 20210513/v0.9/tailcut84/dl1_LST-1.Run04738 No pedestals (No TIB no CDTS)
• 20220302/v0.9/tailcut84/dl1_LST-1.Run07217 No pedestals
• 20220501/v0.9/tailcut84/dl1_LST-1.Run08053 No pedestals “TIB and CDTS are both 0”
• 20220503/v0.9/tailcut84/dl1_LST-1.Run08091 “Broken Camera Configuration, Data not usable most likely”
• 20220504/v0.9/tailcut84/dl1_LST-1.Run08121 CDTS and TIB are 0
• 20220916/v0.9/tailcut84/dl1_LST-1.Run09228,29,30,31,32 No pedestals according to data check, UCTS and TIB seem k.o. despite no Log message about it
• 20220929/v0.9/tailcut84/dl1_LST-1.Run09419 No pedestals, UCTS and TIB seem k.o. despite no Log message about it
• 20221018/v0.9/tailcut84/dl1_LST-1.Run09706 No pedestals, UCTS and TIB k.o.
• 20221020/v0.9/tailcut84/dl1_LST-1.Run09789 No pedestals, UCTS and TIB k.o.
• 20221024/v0.9/tailcut84/dl1_LST-1.Run09882 No pedestals, UCTS and TIB k.o.
• 20221026/v0.9/tailcut84/dl1_LST-1.Run09925 No pedestals, UCTS and TIB k.o. Not even mentioned in ELOG
• 20221028/v0.9/tailcut84/dl1_LST-1.Run09978 No pedestals, UCTS and TIB k.o.
• 20221101/v0.9/tailcut84/dl1_LST-1.Run10060 No pedestals, UCTS and TIB k.o.
• 20221102/v0.9/tailcut84/dl1_LST-1.Run10106 No pedestals, UCTS and TIB k.o. Runs to test DAQ problems!
• 20221102/v0.9/tailcut84/dl1_LST-1.Run10147 No pedestals, UCTS and TIB k.o. Runs to test DAQ problems!
• 20221102/v0.9/tailcut84/dl1_LST-1.Run10188 No pedestals, UCTS and TIB k.o. Runs to test DAQ problems!
• 20221104/v0.9/tailcut84/dl1_LST-1.Run10304 No pedestals, UCTS and TIB k.o. (DAQ stability tests)
• 20221105/v0.9/tailcut84/dl1_LST-1.Run10353 No pedestals, UCTS and TIB k.o. No message in Log, but anyway this is a high moon run, rate ~1k, Extremely low FF charges (for the whole night actually), and probably very low HV.