Difference between revisions of "Incidents 2019"

From my_wiki
Jump to: navigation, search
(Enter comments in reverse time order)
(Enter comments in reverse time order)
 
(10 intermediate revisions by 3 users not shown)
Line 6: Line 6:
  
 
|- valign="top"
 
|- valign="top"
| 2019-06-09 || Fly Wheel 2 from UC 1 was stopped  ||  || There are vibration and rotation Alerts in the MBC parameters. || [[File:FW2UC1.jpg|300px|thumb|center|]]|| No
+
| 2019-11-27 || aluminum tube in the upper structure close to the elevation chain is loosen|| To repair  || The structure should be safe, but there is a danger of the tube detaching completely after more movement. The way the tube is positioned there is almost no chance it will detach in park position. Still, any movement by people in the rear part of the telescope must be limited to the minimum possible amount, meaning only the people who will repair the tube are allowed back there.  ||  || No
 +
 
 +
 
 +
|- valign="top"
 +
| 2019-09-02 || Rubber seal in the frontal part of the camera is damaged. || To repair  || Rubber was damaged on 08-04-2019. The rubber was cut by LST team on 02.09.2019. The rubber was removed by Ciemat on Nov. 2019  || [[File:IMG_20190902_103403.jpg|120px|thumb|center|]] || Yes
 +
 
 +
|- valign="top"
 +
| 2019-07-18 || IT container door(s) not/very hard to open || To repair the lock.  || The door was repaired by Schneider on July.2020 || || YES
 +
 
 +
 
 +
|- valign="top"
 +
| 2019-07-08 || Online event display does not work anymore || To check by experts  || Connection with EVBv2 -> v3 transition? || || No
 +
 
 +
 
 +
 
 +
|- valign="top"
 +
| 2019-06-11 || Coating protection of concrete beam is damaged in several points. || To repair the coating  ||  || [[File:paint1.jpg|120px|thumb|rigth|]] [[File:paint2.jpg|120px|thumb|left|]][[File:paint3.jpg|120px|thumb|center|]][[File:paint4.jpg|120px|thumb|rigth|bubbles]] [[File:paint5.jpg|120px|thumb|center|]]|| No
 +
 
 +
|- valign="top"
 +
| 2019-06-09 || Fly Wheel 2 from UC 1 was stopped  ||  || There are vibration and rotation Alerts in the MBC parameters. Hitachi repaired the FW on 18.07.19|| [[File:IMG_8920.jpg|300px|thumb|center|]]|| Yes
 +
 
 +
|- valign="top"
 +
| 2019-06-07 || Fire Alarm from IT Container was pre-activated. Power was switched off || To reset the Fire Alarm and the ceiling sensor. General power was switched on acording the IT manual but UPS was switched off. UPS was switched on with technical assistand from Schneider (by phone). Inrow 2 and Inrow 3 are not working properly. || Technician from Schneider needs to check the Inrow 2 and Inrow 3. Inrow 3 was repaired by Schneider on july.2019. Inrow 2 and 3 were repaired by Schneider on October 2019. The Fire Alarm Panel was repaired by Schneider on Oct.2019 || [[File:WhatsApp_Image_2019-06-07_at_11.26.38.jpeg|300px|thumb|center|]]|| Yes
  
 
|- valign="top"
 
|- valign="top"
| 2019-06-07 || Fire Alarm from IT Container was pre-activated. Power was switched off || To reset the Fire Alarm and the ceiling sensor. General power was switched on acording the IT manual but UPS was switched off. UPS was switched on with technical assistand from Schneider (by phone). Inrow 2 and Inrow 3 are not working properly. || Technician from Schneider needs to check the Inrow 2 and Inrow 3. || || No
+
| 2019-05-27 || Casana received an overweight alarm from cherry picker. Cherry Picker stopped. After that, cherry picker was blocked. || Opein checked and repaired the cherry picker on 29-05-19. || || ||   Yes
  
 
|- valign="top"
 
|- valign="top"
Line 27: Line 49:
  
 
|- valign="top"
 
|- valign="top"
| 2019-04-03 || There was water inside the UC1 (roof, floor, wall and door) || To dry the water and to repair the container. || Local crew switched off the UC1 on 03.04.2019. Technicial from Hitachi checked the UC1. Aparently everything was fine but the equipments were not swichted on because there was a lot of water inside the container. Metal plates were sealed with new silicone. Several holes around the door were detected on 04.04.2019. Technicial from Hitachi swiched on the UC1 on 06.06.2019. Technicial from Hitachi Malasia checked the UC1. There was not water. Container was checked with Cerrapalma too on 09.04.2019. Sensor of water was tested by technicial from Hitachi and it was working fine. 12.04.2019 || [[File:IMG_20190405_123151.jpg|130px|thumb|left|]] [[File:UC1.JPG|130px|thumb|right|]] [[File:IMG_20190405_122919.jpg|130px|thumb|left|]] [[File:IMG_20190405_123427.jpg |130px |thumb|right|]][[File:Image_2019-04-12_at_13.30.42.jpeg|130px|thumb|left|]] [[File:Image_2019-04-12_at_13.30.43.jpeg  |130px|thumb|right|]] || No
+
| 2019-04-03 || There was water inside the UC1 (roof, floor, wall and door) || To dry the water and to repair the container. || Local crew switched off the UC1 on 03.04.2019. Technicial from Hitachi checked the UC1. Aparently everything was fine but the equipments were not swichted on because there was a lot of water inside the container. Metal plates were sealed with new silicone. Several holes around the door were detected on 04.04.2019. Technicial from Hitachi swiched on the UC1 on 06.06.2019. Technicial from Hitachi Malasia checked the UC1. There was not water. Container was checked with Cerrapalma too on 09.04.2019. Sensor of water was tested by technicial from Hitachi and it was working fine. 12.04.2019 Hitachi repaired the container on 18.07.19||[[File:IMG_20190403_120832.jpg|140px|thumb|right|]] [[File:IMG_20190405_123151.jpg|130px|thumb|left|]] [[File:UC1.JPG|130px|thumb|right|]] [[File:IMG_20190405_122919.jpg|130px|thumb|left|]] [[File:IMG_20190405_123427.jpg |130px |thumb|right|]][[File:Image_2019-04-12_at_13.30.42.jpeg|130px|thumb|left|]] [[File:Image_2019-04-12_at_13.30.43.jpeg  |130px|thumb|right|]] || Yes
  
 
|- valign="top"
 
|- valign="top"
Line 39: Line 61:
  
 
|- valign="top"
 
|- valign="top"
| 2019-02-20 || UPS from IT container stopped || To reset the fuse TOP UPS on 21.02.2019. To change one intelligence module || 3 alarms were received from UPS when the main fuse was resetted. Alarms were sent to Schneider and one Intelligent Module needs to be changed by Schneider. UPS is working with the Redundant Intelligence Module. The damaged module was changed by Schneider on 27.2.19 || [[File:20190221.alarm11.IT.jpg|140px|thumb|left|]] [[File:20190221.alarm23.IT.jpg|140px|thumb|right|]] [[File:20190221.alarm33.IT.jpg|150px|thumb|center|]]  || yes
+
| 2019-02-20 || UPS from IT container stopped || To reset the fuse TOP UPS on 21.02.2019. To change one intelligence module || 3 alarms were received from UPS when the main fuse was resetted. Alarms were sent to Schneider and one Intelligent Module needs to be changed by Schneider. UPS is working with the Redundant Intelligence Module. The damaged module was changed by Schneider on 27.2.19 || [[File:20190221.alarm11.IT.jpg|140px|thumb|left|]] [[File:20190221.alarm23.IT.jpg|140px|thumb|right|]] [[File:20190221.alarm33.IT.jpg|150px|thumb|center|]]  || Yes
  
 
|- valign="top"
 
|- valign="top"
| 2019-02-18 || Perimeter fence was broken by wind || To repair the fence || Fence was repaired by Cerrajeria Bediesta on 06.03.2019  || [[File:IMG-20190218-WA0009.jpg|140px|thumb|left|27.02.2019]]  || yes
+
| 2019-02-18 || Perimeter fence was broken by wind || To repair the fence || Fence was repaired by Cerrajeria Bediesta on 06.03.2019  || [[File:IMG-20190218-WA0009.jpg|140px|thumb|left|27.02.2019]]  || Yes
  
 
|- valign="top"
 
|- valign="top"
| 2019-02-16 || Power cut into Commissioning Container after bad weather. The problem was in line X1.6 || To check/find the problem || UPS´S were started one per one on 18.02.2019. Kaplan modificated the power connection into the Commissioning Container on 27.02.2019. Now there are 2 power lines by UPS´S. Roof of Commissioning Container was sealed by Opein on 04.03.2019|| [[File:IMG_20190227_164243.jpg|130px|thumb|left|]] [[File:IMG_20190227_164234.jpg|130px|thumb|right|]] || yes
+
| 2019-02-16 || Power cut into Commissioning Container after bad weather. The problem was in line X1.6 || To check/find the problem || UPS´S were started one per one on 18.02.2019. Kaplan modificated the power connection into the Commissioning Container on 27.02.2019. Now there are 2 power lines by UPS´S. Roof of Commissioning Container was sealed by Opein on 04.03.2019|| [[File:IMG_20190227_164243.jpg|130px|thumb|left|]] [[File:IMG_20190227_164234.jpg|130px|thumb|right|]] || Yes
  
 
|- valign="top"
 
|- valign="top"
| 2019-02-05 || The glue of one mirror (position: -8, -6; fixed point) got loose || To Change the mirror (ICRR/CIEMAT/CaSana)  || Mirror was changed by CaSana on 06.02.2019 || [[File:20190205_mirror01.JPG|140px|thumb|left|]] [[File:20190205_mirror02.JPG|140px|thumb|right|]] [[File:20190205_mirror03.JPG|150px|thumb|center|]] || yes
+
| 2019-02-05 || The glue of one mirror (position: -8, -6; fixed point) got loose || To Change the mirror (ICRR/CIEMAT/CaSana)  || Mirror was changed by CaSana on 06.02.2019 || [[File:20190205_mirror01.JPG|140px|thumb|left|]] [[File:20190205_mirror02.JPG|140px|thumb|right|]] [[File:20190205_mirror03.JPG|150px|thumb|center|]] || Yes
  
 
|- valign="top"
 
|- valign="top"
| 2019-01-23 || One of the four Elevation breaks is stuck during drive tests by LAPP || To Repair (MPI/CaSana)  || C-Lock Washer of the shaft fell into the disks. The break had to be removed and C-Lock Washer put back. Repaired by CaSana on 25.1.2019 || [[File:Break 1.jpg|140px|thumb|left|C-Lock Washer]] [[File:Break cylinder.jpg|140px|thumb|right|Break cylinder]] [[File:Break cover.jpg|150px|thumb|center|]] || yes
+
| 2019-01-23 || One of the four Elevation breaks is stuck during drive tests by LAPP || To Repair (MPI/CaSana)  || C-Lock Washer of the shaft fell into the disks. The break had to be removed and C-Lock Washer put back. Repaired by CaSana on 25.1.2019 || [[File:Break 1.jpg|140px|thumb|left|C-Lock Washer]] [[File:Break cylinder.jpg|140px|thumb|right|Break cylinder]] [[File:Break cover.jpg|150px|thumb|center|]] || Yes
  
 
|- valign="top"
 
|- valign="top"
| 2019-01-22 || Water in electrical boxes of the ALS || Dried boxes, called Ayumar  || Need to replace the boxes. Plastic boxes of IP66 were installed by Ayumar on 21.02.2019. Electronic pieces have not been changed  || [[File:IMG 20190122 121250.jpg|250px|thumb|center|ALS box]] [[File:IP66 box ALS.jpg|250px|thumb|center|ALS IP66 box]] || yes
+
| 2019-01-22 || Water in electrical boxes of the ALS || Dried boxes, called Ayumar  || Need to replace the boxes. Plastic boxes of IP66 were installed by Ayumar on 21.02.2019. Electronic pieces have not been changed. || [[File:IMG 20190122 121250.jpg|250px|thumb|center|ALS box]] [[File:IP66 box ALS.jpg|250px|thumb|center|ALS IP66 box]] || Yes
  
 
|- valign="top"
 
|- valign="top"
| 2019-01-09 || There are electrical discharges in the commissioning container || To improve the earthing connection  || Some elements of Commissioning Container are not connected to grounding. Several elements were connected to grounding by Kaplan on 16.04.2019  || [[File:IMG_20190416_121338.jpg|130px|thumb|rigth|]] [[File:IMG_20190416_113445.jpg|130px|thumb|left|]] || Yes
+
| 2019-01-09 || There are electrical discharges in the commissioning container || To improve the earthing connection  || Some elements of Commissioning Container are not connected to grounding. Several elements were connected to grounding by Kaplan on 16.04.2019. Metal floor was installed by Casana on July.2019  || [[File:IMG_20190416_121338.jpg|130px|thumb|rigth|]] [[File:IMG_20190416_113445.jpg|130px|thumb|left|]] || Yes

Latest revision as of 09:37, 26 August 2020

Enter comments in reverse time order[edit]

Date What happened? Corrective measures Comments Documents Incident Closed?
2019-11-27 aluminum tube in the upper structure close to the elevation chain is loosen To repair The structure should be safe, but there is a danger of the tube detaching completely after more movement. The way the tube is positioned there is almost no chance it will detach in park position. Still, any movement by people in the rear part of the telescope must be limited to the minimum possible amount, meaning only the people who will repair the tube are allowed back there. No


2019-09-02 Rubber seal in the frontal part of the camera is damaged. To repair Rubber was damaged on 08-04-2019. The rubber was cut by LST team on 02.09.2019. The rubber was removed by Ciemat on Nov. 2019
IMG 20190902 103403.jpg
Yes
2019-07-18 IT container door(s) not/very hard to open To repair the lock. The door was repaired by Schneider on July.2020 YES


2019-07-08 Online event display does not work anymore To check by experts Connection with EVBv2 -> v3 transition? No


2019-06-11 Coating protection of concrete beam is damaged in several points. To repair the coating
Paint1.jpg
Paint2.jpg
Paint3.jpg
bubbles
Paint5.jpg
No
2019-06-09 Fly Wheel 2 from UC 1 was stopped There are vibration and rotation Alerts in the MBC parameters. Hitachi repaired the FW on 18.07.19
IMG 8920.jpg
Yes
2019-06-07 Fire Alarm from IT Container was pre-activated. Power was switched off To reset the Fire Alarm and the ceiling sensor. General power was switched on acording the IT manual but UPS was switched off. UPS was switched on with technical assistand from Schneider (by phone). Inrow 2 and Inrow 3 are not working properly. Technician from Schneider needs to check the Inrow 2 and Inrow 3. Inrow 3 was repaired by Schneider on july.2019. Inrow 2 and 3 were repaired by Schneider on October 2019. The Fire Alarm Panel was repaired by Schneider on Oct.2019
WhatsApp Image 2019-06-07 at 11.26.38.jpeg
Yes
2019-05-27 Casana received an overweight alarm from cherry picker. Cherry Picker stopped. After that, cherry picker was blocked. Opein checked and repaired the cherry picker on 29-05-19. Yes
2019-04-11 Loss of Dragon Firmware To open the camera back and upload Dragon Firmware to every module Following the result on April 9th for the debug of 10M clock gap problem, Seiya wanted to install this firmware to all modules. During the attempt to upload firmware to the modules, a mistake happened and two scripts were accessing same modules in the same time. As a result 242 Dragons lost the firmware and cannot be accessed via ethernet.
Alive modules.png
Yellow are modules which can still be accessed via ethernet
No


2019-04-08 The shutter of camera was damaged when it was moved while the telescope was pointing to about 20 degrees zenith angle To repair camera shutter During camera test at lower zenith angle +-20 degree camera communication was lost due to a TIB error and in order to recover the control, a camera reset was performed and CaCo sent the automatic command of closing the shutter to put the camera in Safe. As the telescope zenith angle was too low the shutter did not close but the motors continue rotating up to the motor end switch stopped them. Telescope was sent to parking and manually closed the shutter. During the next try to open the shutter, the motors sounded as they were rotating but no movement in the blades were observed. After reviewing the comments and photos sent by Oscar we think that: 1.- the connection between axis and first blade is broken. This is normal as these pieces are the mechanical fuse of the system. Those are the weaker point in order to avoid damaging or dismounting the shutter blades. As a consequence it is not possible to move the shutter. Maybe some damage appear in the first blades, but cannot be evaluated from the pictures. 2.- as the blades in the top side are not properly fixed, we don’t know if they can displace during the movement of the telescope and damage any other component of the camera.

“On April 23 Miguel Polo and Carlos Díaz from CIEMAT travelled to the observatory and carried out the following operations: 1.- Visual inspection of the shutter: during the inspection it was confirmed that the pieces of union between the shaft and the blades were broken. These pieces are broken from a certain strength in such a way that they prevent further damage. 2.- Once the problem was identified, the damaged parts were replaced. 3.- During the first tests, a very strong friction between the blades and the guides was identified. This friction was due to a deformation of the support structure of the blind. This deformation was corrected, reducing the friction to normal levels. 4.- The system was verified Open / Close several times in the presence of the Commissioning Leader. The blind remains in normal operating condition with the restrictions that were applied before the incident. No more actions are required.”

Image 2019-04-12 at 09.15.31 (2).jpeg
Yes
2019-04-03 There was water inside the UC1 (roof, floor, wall and door) To dry the water and to repair the container. Local crew switched off the UC1 on 03.04.2019. Technicial from Hitachi checked the UC1. Aparently everything was fine but the equipments were not swichted on because there was a lot of water inside the container. Metal plates were sealed with new silicone. Several holes around the door were detected on 04.04.2019. Technicial from Hitachi swiched on the UC1 on 06.06.2019. Technicial from Hitachi Malasia checked the UC1. There was not water. Container was checked with Cerrapalma too on 09.04.2019. Sensor of water was tested by technicial from Hitachi and it was working fine. 12.04.2019 Hitachi repaired the container on 18.07.19
IMG 20190403 120832.jpg
IMG 20190405 123151.jpg
UC1.JPG
IMG 20190405 122919.jpg
IMG 20190405 123427.jpg
Image 2019-04-12 at 13.30.42.jpeg
Image 2019-04-12 at 13.30.43.jpeg
Yes
2019-04-03 The fire alarm of UC1 rang. There was snow To change the external button of fire alarm Technician from Hitachi checked the fire alarm and he detected that the external button was damaged. Button was changed by Hitachi on 10.04.2019
IMG 20190403 120832.jpg
yes
2019-03-12 Camera access tower platform is blocked (opened) To repair After the drive test, the platform was not possible to be closed
IMG 20190319 113421.jpg
yes
2019-03-02 Cherry Picker lost about 1 liter of oil. One rubber piece was broken To repair the rubber piece and to refill the Oil Rubber piece was changed by Opein on 04.03.2019. Opein put oil into the machine and they tested the machine too.
3fcd6b9c-e890-4fd1-b4f0-c8e2d85b00ce.jpg
E15e3e9f-ade1-436e-ad77-7f36203c71cf.jpg
yes
2019-02-20 UPS from IT container stopped To reset the fuse TOP UPS on 21.02.2019. To change one intelligence module 3 alarms were received from UPS when the main fuse was resetted. Alarms were sent to Schneider and one Intelligent Module needs to be changed by Schneider. UPS is working with the Redundant Intelligence Module. The damaged module was changed by Schneider on 27.2.19
20190221.alarm11.IT.jpg
20190221.alarm23.IT.jpg
20190221.alarm33.IT.jpg
Yes
2019-02-18 Perimeter fence was broken by wind To repair the fence Fence was repaired by Cerrajeria Bediesta on 06.03.2019
27.02.2019
Yes
2019-02-16 Power cut into Commissioning Container after bad weather. The problem was in line X1.6 To check/find the problem UPS´S were started one per one on 18.02.2019. Kaplan modificated the power connection into the Commissioning Container on 27.02.2019. Now there are 2 power lines by UPS´S. Roof of Commissioning Container was sealed by Opein on 04.03.2019
IMG 20190227 164243.jpg
IMG 20190227 164234.jpg
Yes
2019-02-05 The glue of one mirror (position: -8, -6; fixed point) got loose To Change the mirror (ICRR/CIEMAT/CaSana) Mirror was changed by CaSana on 06.02.2019
20190205 mirror01.JPG
20190205 mirror02.JPG
20190205 mirror03.JPG
Yes
2019-01-23 One of the four Elevation breaks is stuck during drive tests by LAPP To Repair (MPI/CaSana) C-Lock Washer of the shaft fell into the disks. The break had to be removed and C-Lock Washer put back. Repaired by CaSana on 25.1.2019
C-Lock Washer
Break cylinder
Break cover.jpg
Yes
2019-01-22 Water in electrical boxes of the ALS Dried boxes, called Ayumar Need to replace the boxes. Plastic boxes of IP66 were installed by Ayumar on 21.02.2019. Electronic pieces have not been changed.
ALS box
ALS IP66 box
Yes
2019-01-09 There are electrical discharges in the commissioning container To improve the earthing connection Some elements of Commissioning Container are not connected to grounding. Several elements were connected to grounding by Kaplan on 16.04.2019. Metal floor was installed by Casana on July.2019
IMG 20190416 121338.jpg
IMG 20190416 113445.jpg
Yes