Incidents 2019

Enter comments in reverse time order

Date	What happened?	Corrective measures	Comments	Documents	Incident Closed?
2019-11-27	aluminum tube in the upper structure close to the elevation chain is loosen	To repair	The structure should be safe, but there is a danger of the tube detaching completely after more movement. The way the tube is positioned there is almost no chance it will detach in park position. Still, any movement by people in the rear part of the telescope must be limited to the minimum possible amount, meaning only the people who will repair the tube are allowed back there.		No
2019-09-02	Rubber seal in the frontal part of the camera is damaged.	To repair	Rubber was damaged on 08-04-2019. The rubber was cut by LST team on 02.09.2019. The rubber was removed by Ciemat on Nov. 2019		Yes
2019-07-18	IT container door(s) not/very hard to open	To repair the lock. The door was repaired by Schneider on July.2020			YES
2019-07-08	Online event display does not work anymore	To check by experts	Connection with EVBv2 -> v3 transition?		No
2019-06-11	Coating protection of concrete beam is damaged in several points.	To repair the coating		bubbles	No
2019-06-09	Fly Wheel 2 from UC 1 was stopped		There are vibration and rotation Alerts in the MBC parameters. Hitachi repaired the FW on 18.07.19		Yes
2019-06-07	Fire Alarm from IT Container was pre-activated. Power was switched off	To reset the Fire Alarm and the ceiling sensor. General power was switched on acording the IT manual but UPS was switched off. UPS was switched on with technical assistand from Schneider (by phone). Inrow 2 and Inrow 3 are not working properly.	Technician from Schneider needs to check the Inrow 2 and Inrow 3. Inrow 3 was repaired by Schneider on july.2019. Inrow 2 and 3 were repaired by Schneider on October 2019. The Fire Alarm Panel was repaired by Schneider on Oct.2019		Yes
2019-05-27	Casana received an overweight alarm from cherry picker. Cherry Picker stopped. After that, cherry picker was blocked.	Opein checked and repaired the cherry picker on 29-05-19.			Yes
2019-04-11	Loss of Dragon Firmware	To open the camera back and upload Dragon Firmware to every module	Following the result on April 9th for the debug of 10M clock gap problem, Seiya wanted to install this firmware to all modules. During the attempt to upload firmware to the modules, a mistake happened and two scripts were accessing same modules in the same time. As a result 242 Dragons lost the firmware and cannot be accessed via ethernet.	Yellow are modules which can still be accessed via ethernet	No
2019-04-08	The shutter of camera was damaged when it was moved while the telescope was pointing to about 20 degrees zenith angle	To repair camera shutter	During camera test at lower zenith angle +-20 degree camera communication was lost due to a TIB error and in order to recover the control, a camera reset was performed and CaCo sent the automatic command of closing the shutter to put the camera in Safe. As the telescope zenith angle was too low the shutter did not close but the motors continue rotating up to the motor end switch stopped them. Telescope was sent to parking and manually closed the shutter. During the next try to open the shutter, the motors sounded as they were rotating but no movement in the blades were observed. After reviewing the comments and photos sent by Oscar we think that: 1.- the connection between axis and first blade is broken. This is normal as these pieces are the mechanical fuse of the system. Those are the weaker point in order to avoid damaging or dismounting the shutter blades. As a consequence it is not possible to move the shutter. Maybe some damage appear in the first blades, but cannot be evaluated from the pictures. 2.- as the blades in the top side are not properly fixed, we don’t know if they can displace during the movement of the telescope and damage any other component of the camera. “On April 23 Miguel Polo and Carlos Díaz from CIEMAT travelled to the observatory and carried out the following operations: 1.- Visual inspection of the shutter: during the inspection it was confirmed that the pieces of union between the shaft and the blades were broken. These pieces are broken from a certain strength in such a way that they prevent further damage. 2.- Once the problem was identified, the damaged parts were replaced. 3.- During the first tests, a very strong friction between the blades and the guides was identified. This friction was due to a deformation of the support structure of the blind. This deformation was corrected, reducing the friction to normal levels. 4.- The system was verified Open / Close several times in the presence of the Commissioning Leader. The blind remains in normal operating condition with the restrictions that were applied before the incident. No more actions are required.”		Yes
2019-04-03	There was water inside the UC1 (roof, floor, wall and door)	To dry the water and to repair the container.	Local crew switched off the UC1 on 03.04.2019. Technicial from Hitachi checked the UC1. Aparently everything was fine but the equipments were not swichted on because there was a lot of water inside the container. Metal plates were sealed with new silicone. Several holes around the door were detected on 04.04.2019. Technicial from Hitachi swiched on the UC1 on 06.06.2019. Technicial from Hitachi Malasia checked the UC1. There was not water. Container was checked with Cerrapalma too on 09.04.2019. Sensor of water was tested by technicial from Hitachi and it was working fine. 12.04.2019 Hitachi repaired the container on 18.07.19		Yes
2019-04-03	The fire alarm of UC1 rang. There was snow	To change the external button of fire alarm	Technician from Hitachi checked the fire alarm and he detected that the external button was damaged. Button was changed by Hitachi on 10.04.2019		yes
2019-03-12	Camera access tower platform is blocked (opened)	To repair	After the drive test, the platform was not possible to be closed		yes
2019-03-02	Cherry Picker lost about 1 liter of oil. One rubber piece was broken	To repair the rubber piece and to refill the Oil	Rubber piece was changed by Opein on 04.03.2019. Opein put oil into the machine and they tested the machine too.		yes
2019-02-20	UPS from IT container stopped	To reset the fuse TOP UPS on 21.02.2019. To change one intelligence module	3 alarms were received from UPS when the main fuse was resetted. Alarms were sent to Schneider and one Intelligent Module needs to be changed by Schneider. UPS is working with the Redundant Intelligence Module. The damaged module was changed by Schneider on 27.2.19		Yes
2019-02-18	Perimeter fence was broken by wind	To repair the fence	Fence was repaired by Cerrajeria Bediesta on 06.03.2019	27.02.2019	Yes
2019-02-16	Power cut into Commissioning Container after bad weather. The problem was in line X1.6	To check/find the problem	UPS´S were started one per one on 18.02.2019. Kaplan modificated the power connection into the Commissioning Container on 27.02.2019. Now there are 2 power lines by UPS´S. Roof of Commissioning Container was sealed by Opein on 04.03.2019		Yes
2019-02-05	The glue of one mirror (position: -8, -6; fixed point) got loose	To Change the mirror (ICRR/CIEMAT/CaSana)	Mirror was changed by CaSana on 06.02.2019		Yes
2019-01-23	One of the four Elevation breaks is stuck during drive tests by LAPP	To Repair (MPI/CaSana)	C-Lock Washer of the shaft fell into the disks. The break had to be removed and C-Lock Washer put back. Repaired by CaSana on 25.1.2019	C-Lock Washer Break cylinder	Yes
2019-01-22	Water in electrical boxes of the ALS	Dried boxes, called Ayumar	Need to replace the boxes. Plastic boxes of IP66 were installed by Ayumar on 21.02.2019. Electronic pieces have not been changed.	ALS box ALS IP66 box	Yes
2019-01-09	There are electrical discharges in the commissioning container	To improve the earthing connection	Some elements of Commissioning Container are not connected to grounding. Several elements were connected to grounding by Kaplan on 16.04.2019. Metal floor was installed by Casana on July.2019		Yes