Image cleaning and parametrization

Real Data DL1 production

Real data will be stored in the IT Container following the structure discussed in the Redmine issue https://forge.in2p3.fr/issues/37938, which includes directories whose name depends on the date. The convention UTC+12, by which days change at noon of the next day, is used at DAQ. For instance, for the data taken in the night 2019_11_23 to 2019_11_24 the selected date would be 2019_11_23 and main directories:

/fefs/aswg/data/real/DL1/20191123/v00 for the default analysis

v00 can take the form of XXX_NN where XXX is the initials of a name and NN a version number (e,g, JLC_01 for the first version of Jose Luis Contreras) when it is an alternative version run by an analysis shifter)

--Jose-Luis (talk) 18:09, 22 November 2019 (UTC)

Standard DL1 v0.4.4 production for 2019 and 2020 data (edited: March 6, 2020)

Files are in /fefs/aswg/data/real/DL1/<observation_date>/v0.4.4_v00 , produced with lstchain v0.4.4

Note: these v0.4.4 data are for the first time processed with the intended LocalPeakWindowSum, same as MC.

Standard lstchain configuration file was used: cta-lstchain/lstchain/data/lstchain_standard_config.json. Hence these cleaning settings were applied:

 "tailcut": {
   "picture_thresh":6,
   "boundary_thresh":3,
   "keep_isolated_pixels":false,
   "min_number_picture_neighbors":1
 }

Find all the details about which runs where analyzed in the tables below (DL1 Analysis logbook).

Additional comments

• DRS4 & CALIB runs from 2020_01_17 and 2020_01_25 are not suitable for producing DRS4 pedestal and calibration files so DRS4 & CALIB runs from other days (2020_01_18 and 2020_01_27 respectively) were used to process the data runs from those days.

• Time calibration file time_calibration.Run1625.0000.hdf5 from 2019_11_24 was used to process all the DL1 files.

• By default (from January 2020 on), ucts time-stamps "timestamps_pointing":"ucts" are used to interpolate pointing coordinates.

• However for Nov 2019 data, dragon-based time-stamps were used for the interpolation: "timestamps_pointing":"dragon"

• Corrected new drive log files were used for Nov 2019: /fefs/aswg/scripts-osa/corrected_drive_logs_Nov19/

--Daniel.morcuende (talk) 11:31, 6 March 2020 (UTC)

Standard DL1 v0.4.3 production for 2019 and 2020 data (edited: January 2020)

Files are in /fefs/aswg/data/real/DL1/<observation_date>/v0.4.3_v00 , produced with lstchain v0.4.3

Standard lstchain configuration file was used. You can find it here:

/fefs/aswg/software/virtual_env/ctasoft/cta-lstchain/lstchain/data/lstchain_standard_config.json

Cleaning levels: 6-3

 "tailcut": {
   "picture_thresh":6,
   "boundary_thresh":3,
   "keep_isolated_pixels":false,
   "min_number_picture_neighbors":1
 }

2020 DL1 Analysis logbook

Date	DRS4	CALIB	Runs ON	Runs OFF	Source	Conditions	Comments	Incidences
2020_01_14	1785	1786	1787	1788	Crab	Dark
2020_01_15	1789	1791	1795, 1796, 1797, 1799, 1800 (moon)	1798, 1801 (moon)	Crab	Dark + Moon	Piggy-back MAGIC (moon)	1795 wrong DTs
2020_01_17	1830	1831	1813, 1814, 1817, 1818, 1819, 1820, 1821, 1824, 1825, 1826, 1827	1815, 1816, 1822, 1823	Crab	Dark	Piggy-back MAGIC	1813, 1827 (1 subrun), 1814 (no UCTS). Problems with calib files.
2020_01_18	1830	1831	1832, 1833, 1834, 1835, 1836, 1843, 1844	1837, 1840, 1841, 1842	Crab	Dark	Piggy-back MAGIC
2020_01_25	1872	1873	1863	-	Crab	Dark		Problems with calib runs. Data may not be good either.
2020_01_27	1872	1873	1874, 1875, 1876, 1878, 1879, 1880	1877, 1881	Crab	Dark		Some moon giving high rates
2020_01_28	1882	1883	1887, 1888, 1891, 1892	1893	Crab	Dark		1887, 1888 (no UCTS)
2020_01_31	1915	1917	1918, 1919,1921	1920	Mrk421	Dark
2020_02_01	1922	1923	1925, 1926, 1928, 1929, 1931, 1932	1927, 1930, 1933	Crab	Moon	OFF runs 1927, 1930 and 1933 to be shared with the CTA real-time analysis team for purposes of software development
2020_02_02	1944	1947	1948, 1951, 1952, 1954, 1955	1949, 1953	Crab	Moon	Piggy-back MAGIC
2020_02_13	1967	1968	1970	1971	Crab	Dark		Discarded run 1969 (taken without CDTS)
2020_02_15	1985	1986	1987, 1988, 1991	1990, 1992	Crab	Dark	Simultaneous observations with MAGIC	Discarded run 1989 (HV not properly set). Strange pattern in L1 rates in first runs due to MAGIC LIDAR.
2020_02_17	1994	1995	1996, 1997, 1999, 2000, 2002, 2003	1998, 2001, 2004	Crab	Dark	Simultaneous observations with MAGIC	Cloudy weather starting from run 1999
2020_02_18	2005	2006	2007, 2008, 2010, 2011, 2013, 2014	2009, 2012	Crab	Dark	Good weather. Simultaneous observations with MAGIC (SumT until 23:00 UTC)
2020_02_27	2028	2030	2031, 2032	2033	Crab	Dark	Good weather
2020_02_28	2034	2035	2036, 2037	2038	Crab	Dark	No interleaved pedestal events (?)

2019 DL1 Analysis logbook

Date	DRS4	CALIB	Runs ON	Runs OFF	Source	Conditions	Comments	Incidences
2019_11_23	1611	1614	1616, 1618, 1620, 1622	1615, 1617, 1619, 1621	Crab	Dark	Data taken without EvB online corrections (except for first DRS4 run)	Good weather
2019_11_24	1623	1625	1627, 1629, 1631, 1632, 1633	1626, 1628, 1630	Crab	Dark	Data taken with EvB online corrections (except for DRS4 run 1624)	Low transmission9km (<0.5) up to run 1629
2019_11_26	1645	1646	1648, 1649, 1650, 1652, 1653	1647, 1651	Crab	Dark	Data taken with EvB online corrections	Good weather
2019_11_29	1696	1697	1701, 1702	1700	Crab	Dark	Data taken with EvB online corrections

MC DL1 production

DL1 MC production to use can be found there for point source gammas:

• /fefs/aswg/data/mc/DL1/20190415/gamma/south_pointing/

Replace gamma with gamma_diffuse, proton or electron.

--Thomas Vuillaume (talk) 15:02, 21 November 2019 (UTC)