Difference between revisions of "Incidents 2020"
(→Enter comments in reverse time order) |
MartinWill (talk | contribs) |
||
(18 intermediate revisions by 5 users not shown) | |||
Line 5: | Line 5: | ||
! style="width: 100px;" | Date !! style="width: 210px;" | What happened? !! style="width: 150px;" | Corrective measures !! style="width: 300px;" | Comments !! style="width: 400px;" | Documents !! style="width: 100px;"| Incident Closed? | ! style="width: 100px;" | Date !! style="width: 210px;" | What happened? !! style="width: 150px;" | Corrective measures !! style="width: 300px;" | Comments !! style="width: 400px;" | Documents !! style="width: 100px;"| Incident Closed? | ||
+ | |- valign="top" | ||
+ | | 2020-10-19 || Camera front scaffolding loosened in the morning on 19 October as seen on internal webcams. [http://www.lst1.iac.es/elog/LST+commissioning/1167 ELOG Entry] || Park and secure scaffolding || Shunsuke S., Enrique G., and Stefan C. decided to go to the site together for safety. Took additional ropes and straps. Before going up the camera access tower, they performed an inspection of the scaffolding from the ground. It looked that a rope is still connecting the fence and the scaffolding. On the tower, they performed quick inspection before starting the fix. The west scaffolding moved to the camera direction, but it did not hit the camera. It was stopped by the newly installed foot barriers around the camera hole in the platform. The brake of the west scaffolding was not locked. And the ropes of west and east scaffoldings had been fixed to the horizontal rail, which allows horizontal movement of the scaffolding. So they put back the west scaffolding in where it should be and locked the brake. After that they fixed the west / east scaffolding by two ropes. It does not allow horizontal movement any more. The final design of the scaffolding foresees an additional brake using the locking and sensor mechanism on the side of the scaffolding on 20.10.20. || [[File:Scaffolding.jpg|170px|thumb|left|Scaffolding moved towards camera, stopped by foot barrier.]] [[File:Brake west.jpg|170px|thumb|right|Opened brake on scaffolding.]] [[File:Ropes before and after.jpg|170px|thumb|right|Good and Bad example of tying rope, Attachment before the storm was only with one strap on horizontal bar. (the right side like in this picture). After incident, attachment with two straps, one from handrail and from vertical frame of the scaffolding both on a vertical struct of the fence of the camera tower.]] [[File:Fixed scaffolding.jpg|170px|thumb|left|Additional strap on the right side.]] || YES | ||
+ | |||
+ | |- valign="top" | ||
+ | | 2020-08-19 || Elevation drive got stuck at ~60° zd after the night. Four persons at hand wheels and remote expert intervention were needed to bring telescope and camera in safe state around 9:30 (local) in the morning. [[http://www.lst1.iac.es/elog/LST+commissioning/1013 ELOG Entry 1]] [[http://www.lst1.iac.es/elog/LST+commissioning/1017 ELOG Entry 2]] || To prevent problems like this in the future (in addition to regular inspection and maintenance of the drive and arc) probably some brushes can be added on top, and in front and rear of the elevation drive, similar to the brushes at the azmiuth bogies. Also the zinc coating should be removed or renewed, it is flaking off and therefore does not fulfill its intended function, but the debris may have caused the blocking of the drive. Possibly, also allow local crew to release elevation brakes in emergency cases. || When trying to go to park in position at end of night, the elevation movement got stuck. Telescope was in part-out position for hours before, but moved to different sky directions without notable problems earlier in the night. At parking-in via eGui, the telescope got stuck very close to the park-in position. Shifters went to park-out again, and then tried to park in again using the semi-expert drive interface. This time, the telescope got stuck at very beginning of the movement (around 75 Zd). Shifters tried a third time, with same result and contacted experts. Another park-out was done with M.Will present and loud noise was heard when the telescope tried to move. After several movements, telescope got stuck at 60 deg. After being in contact with remote experts, noticed that the motors were overtorqued and overheated to more than 100 deg C. After cooling down the motors connected hand wheels and with a lot of strength and 4 people finally managed to park the telescope. On the inspection two days after, large amount of debris was found on the arch where the guiding wheels of the elevation drive move within the arch. Debris could be sanded down and smoothed. Another test was made in the evening, when the arc was again sanded and thoroughly cleaned from debris again between 50deg and 95deg Zd. Afterwards another cycle of park-out, Zd 30, park-in confirmed that there was no more scraping sounds at any point. It was concluded that the problem is fixed, although the cause still not perfectly clear. As for the cause, no larger debris or piece could be found that looks like it could have fallen into the drive. It is possible that it was a slow process over several days or weeks where small metallic debris was embedded and compacted on the guiding wheels and at one point finally started to scrape so much on the surface of the arc that the motors overheated and overtorqued. Loose zinc coating was found on the steel tubes directly above the back arc which may have flaked off and fallen into the elevation drive. A single flake should not be hard enough to cause a problem, but possibly an accumulation over several weeks might cause a problem. || [[File:20200820 093638.jpg|120px|thumb|center|]] [[File:20200820 092408.jpg |120px|thumb|center|]] [[File:20200819 101002.jpg |120px|thumb|center|]] [[File:20200819 100954.jpg |120px|thumb|center|]] More images at [[https://drive.google.com/drive/folders/1nnlUArumAFzSBtbWWfpkxyMtL6rtoiXx Google Drive]] || YES | ||
|- valign="top" | |- valign="top" | ||
− | | 2020- | + | | 2020-07-18 || The ladder used to uncover the temporary camera cover has been forgotten in front of the camera. Once the platform opened, the ladder fell but remained on the upper platform. [[http://www.lst1.iac.es/elog/LST+commissioning/947 ELOG Entry]] || To check and repair (if apply) || As can be seen in the attached webcam archive images, the ladder caught on the tension rope and tilted towards the camera frame. It did not touch the plexiglas, it probably rested on the carbon fiber frame until the camera started moving upwards, at which point it fell across the hole in the top platform. We noticed the problem, make sure everyting was safe and climbed to inspect the platform. The ladder was in a stable but not secure position at the edge of the top platform. To make sure it couldn't fall to the lower platform, we performed the following action: close manually the platform slowly, retrieve the ladder and fix it on the ground, open manually the platform, park the telescope normally (in automatic mode), inspect the camera frame and the platform. With the darkness and the unsure situation of the ladder on the edge of the platform we did not take images but opted to secure the ladder quickly and calmly to avoid it falling further. The attached webcam image should give some idea how it landed and how close the top of the ladder is to the edge. It was already dark when we did the inspection. The camera and camera housing look undammaged. There is damage to the locking system of the ladder hatch (the part that hit the floor first) which can be repaired, and a deep scratch in the metal sheet at the place where the ladder hit the platform. There seems to be no damage on the tension rope. There is a small damage on the carbon fiber camera frame but we need to inspect it during day light to see if it is a new damage or old scratch. We will perform a deeper inspection with day light. We will leave the telescope parked tonight and perform test only with the tower open. No damages were found || [[File:Camera4_20200718_2040.jpeg|120px|thumb|center|]] [[File:Camera4_20200718_2045.jpeg|120px|thumb|center|]] || YES |
|- valign="top" | |- valign="top" | ||
− | | 2020-02-23 || Incident 8: The camera | + | | 2020-02-23 || Incident 8: The camera shutter was broken after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00. || To repair || LST people installed a temporal cover on 25.02.20. CIEMAT team checked the camera and they installed a better temporal cover on 26.02.20|| [[File:IMG_20200225_141909.jpg|120px|thumb|center|]] [[File:IMG_20200225_113146.jpg|120px|thumb|center|]] || YES |
|- valign="top" | |- valign="top" | ||
− | | 2020-02-23 || | + | | 2020-02-23 || Incident 7: The camera access tower was opened after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00.|| To check and to repair and to close|| LST people on site and Casana checked the platform. Everything was fine and they closed the platform from the control in the camera access tower on 25.02.20 || [[File:DSC_0436.JPG|120px|thumb|center|]] || YES |
+ | |||
|- valign="top" | |- valign="top" | ||
− | | 2020-02-23 || Incident 6: The camera maintenance room was moved after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00. || To relocate | + | | 2020-02-23 || Incident 6: The camera maintenance room was moved after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00. || To relocate || The room was relocated by Casana|| [[File:IMG 20200225 094855.jpg|120px|thumb|center|]] [[File:IMG 20200225 094830.jpg|120px|thumb|center|]] || YES |
|- valign="top" | |- valign="top" | ||
− | | 2020-02-23 || Incident 5: Several signs were removed and broken after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00. || To repair || | + | | 2020-02-23 || Incident 5: Several signs were removed and broken after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00. || To repair || New signs were installed by local LST staff || || YES |
|- valign="top" | |- valign="top" | ||
− | | 2020-02-23 || Incident 4: 2 piles from the fence were moved (the fence is not opened) after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00.|| To repair || | + | | 2020-02-23 || Incident 4: 2 piles from the fence were moved (the fence is not opened) after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00.|| To repair || Fence was repaired by Bediesta on June 2020 || [[File:IMG 20200225 102924.jpg|120px|thumb|center|]] || YES |
|- valign="top" | |- valign="top" | ||
Line 28: | Line 34: | ||
|- valign="top" | |- valign="top" | ||
− | | 2020-02-23 || Incident 2: The roof of the chiller was damaged after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00. || To repair || The roof was repaired by CASANA|| [[File: | + | | 2020-02-23 || Incident 2: The roof of the chiller was damaged after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00. || To repair || The roof was repaired by CASANA|| || YES |
+ | |||
+ | |- valign="top" | ||
+ | | 2020-02-23 || Incident 1: Several cable trays were damaged after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00.|| To repair || Several cable trays were repaired on 28.02.20. Other cables trayes were repaired by Bediesta on June 2020|| [[File:IMG_20200303_122248.jpg|120px|thumb|center|]] || YES | ||
|- valign="top" | |- valign="top" | ||
− | | 2020- | + | | 2020-01-28 || There are drops of water from the humidifier in the wall of the IT container || To repair || Schneider and LST Team checked the humidifier and there was a loop in the drainage pipe. The loop was removed. The bottle was removed and it was broken. A new bottle was installed but the new holes are looking to the ceiling and they produce condensation on the ceiling. The humidifier cannot be connected in this situation because the water can wet the leak rope. The humidifier was repaied by Schneider on week 30 (2020)|| [[File:IMG-20200212-WA0006.jpg|120px|thumb|center|]] || YES |
Latest revision as of 10:13, 22 January 2021
Enter comments in reverse time order[edit]
Date | What happened? | Corrective measures | Comments | Documents | Incident Closed? |
---|---|---|---|---|---|
2020-10-19 | Camera front scaffolding loosened in the morning on 19 October as seen on internal webcams. ELOG Entry | Park and secure scaffolding | Shunsuke S., Enrique G., and Stefan C. decided to go to the site together for safety. Took additional ropes and straps. Before going up the camera access tower, they performed an inspection of the scaffolding from the ground. It looked that a rope is still connecting the fence and the scaffolding. On the tower, they performed quick inspection before starting the fix. The west scaffolding moved to the camera direction, but it did not hit the camera. It was stopped by the newly installed foot barriers around the camera hole in the platform. The brake of the west scaffolding was not locked. And the ropes of west and east scaffoldings had been fixed to the horizontal rail, which allows horizontal movement of the scaffolding. So they put back the west scaffolding in where it should be and locked the brake. After that they fixed the west / east scaffolding by two ropes. It does not allow horizontal movement any more. The final design of the scaffolding foresees an additional brake using the locking and sensor mechanism on the side of the scaffolding on 20.10.20. | YES | |
2020-08-19 | Elevation drive got stuck at ~60° zd after the night. Four persons at hand wheels and remote expert intervention were needed to bring telescope and camera in safe state around 9:30 (local) in the morning. [ELOG Entry 1] [ELOG Entry 2] | To prevent problems like this in the future (in addition to regular inspection and maintenance of the drive and arc) probably some brushes can be added on top, and in front and rear of the elevation drive, similar to the brushes at the azmiuth bogies. Also the zinc coating should be removed or renewed, it is flaking off and therefore does not fulfill its intended function, but the debris may have caused the blocking of the drive. Possibly, also allow local crew to release elevation brakes in emergency cases. | When trying to go to park in position at end of night, the elevation movement got stuck. Telescope was in part-out position for hours before, but moved to different sky directions without notable problems earlier in the night. At parking-in via eGui, the telescope got stuck very close to the park-in position. Shifters went to park-out again, and then tried to park in again using the semi-expert drive interface. This time, the telescope got stuck at very beginning of the movement (around 75 Zd). Shifters tried a third time, with same result and contacted experts. Another park-out was done with M.Will present and loud noise was heard when the telescope tried to move. After several movements, telescope got stuck at 60 deg. After being in contact with remote experts, noticed that the motors were overtorqued and overheated to more than 100 deg C. After cooling down the motors connected hand wheels and with a lot of strength and 4 people finally managed to park the telescope. On the inspection two days after, large amount of debris was found on the arch where the guiding wheels of the elevation drive move within the arch. Debris could be sanded down and smoothed. Another test was made in the evening, when the arc was again sanded and thoroughly cleaned from debris again between 50deg and 95deg Zd. Afterwards another cycle of park-out, Zd 30, park-in confirmed that there was no more scraping sounds at any point. It was concluded that the problem is fixed, although the cause still not perfectly clear. As for the cause, no larger debris or piece could be found that looks like it could have fallen into the drive. It is possible that it was a slow process over several days or weeks where small metallic debris was embedded and compacted on the guiding wheels and at one point finally started to scrape so much on the surface of the arc that the motors overheated and overtorqued. Loose zinc coating was found on the steel tubes directly above the back arc which may have flaked off and fallen into the elevation drive. A single flake should not be hard enough to cause a problem, but possibly an accumulation over several weeks might cause a problem. | More images at [Google Drive] | YES |
2020-07-18 | The ladder used to uncover the temporary camera cover has been forgotten in front of the camera. Once the platform opened, the ladder fell but remained on the upper platform. [ELOG Entry] | To check and repair (if apply) | As can be seen in the attached webcam archive images, the ladder caught on the tension rope and tilted towards the camera frame. It did not touch the plexiglas, it probably rested on the carbon fiber frame until the camera started moving upwards, at which point it fell across the hole in the top platform. We noticed the problem, make sure everyting was safe and climbed to inspect the platform. The ladder was in a stable but not secure position at the edge of the top platform. To make sure it couldn't fall to the lower platform, we performed the following action: close manually the platform slowly, retrieve the ladder and fix it on the ground, open manually the platform, park the telescope normally (in automatic mode), inspect the camera frame and the platform. With the darkness and the unsure situation of the ladder on the edge of the platform we did not take images but opted to secure the ladder quickly and calmly to avoid it falling further. The attached webcam image should give some idea how it landed and how close the top of the ladder is to the edge. It was already dark when we did the inspection. The camera and camera housing look undammaged. There is damage to the locking system of the ladder hatch (the part that hit the floor first) which can be repaired, and a deep scratch in the metal sheet at the place where the ladder hit the platform. There seems to be no damage on the tension rope. There is a small damage on the carbon fiber camera frame but we need to inspect it during day light to see if it is a new damage or old scratch. We will perform a deeper inspection with day light. We will leave the telescope parked tonight and perform test only with the tower open. No damages were found | YES | |
2020-02-23 | Incident 8: The camera shutter was broken after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00. | To repair | LST people installed a temporal cover on 25.02.20. CIEMAT team checked the camera and they installed a better temporal cover on 26.02.20 | YES | |
2020-02-23 | Incident 7: The camera access tower was opened after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00. | To check and to repair and to close | LST people on site and Casana checked the platform. Everything was fine and they closed the platform from the control in the camera access tower on 25.02.20 | YES
| |
2020-02-23 | Incident 6: The camera maintenance room was moved after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00. | To relocate | The room was relocated by Casana | YES | |
2020-02-23 | Incident 5: Several signs were removed and broken after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00. | To repair | New signs were installed by local LST staff | YES | |
2020-02-23 | Incident 4: 2 piles from the fence were moved (the fence is not opened) after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00. | To repair | Fence was repaired by Bediesta on June 2020 | YES | |
2020-02-23 | Incident 3: A cover from the camera chiller was damaged after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00. | To repair | The cover was installed on 25.02.20 | YES | |
2020-02-23 | Incident 2: The roof of the chiller was damaged after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00. | To repair | The roof was repaired by CASANA | YES | |
2020-02-23 | Incident 1: Several cable trays were damaged after a wind storm, the maximum wind gust speed reported by the MAGIC weather station during the storm was 157 km/h at ~8:00. | To repair | Several cable trays were repaired on 28.02.20. Other cables trayes were repaired by Bediesta on June 2020 | YES | |
2020-01-28 | There are drops of water from the humidifier in the wall of the IT container | To repair | Schneider and LST Team checked the humidifier and there was a loop in the drainage pipe. The loop was removed. The bottle was removed and it was broken. A new bottle was installed but the new holes are looking to the ceiling and they produce condensation on the ceiling. The humidifier cannot be connected in this situation because the water can wet the leak rope. The humidifier was repaied by Schneider on week 30 (2020) | YES |