As far as I've heard there was not personnel manually interacting with the power system from the point that grid power was lost to when the UPS's finally gave out (it was a Sunday afternoon). I'm assuming that if someone was there engineers would have realized that the gen sets juice was going to /dev/null . I believe Fisher Plaza now has maintainence staff on-site 24x7 until all the parts are fixed. If anyone has a set of questions to ask colo providers about their power system best practices, or more simply background information about how facility power systems work for the 'smaller fish' here that have less exposure to running generators, etc... I'd greatly appreciate a reference. This event has reminded me that I can't be complacent and blindly hope that the people downstream from me (colo providers, etc...) are on their game 100%. Jim Popovitch wrote:
Michael K. Smith wrote:
It was a breaker in the main bypass from city power to the generators. The breaker failed to close so the generators happily fed power to nowhere. Then, everyone's UPS failed and down we/they went. The outage lasted approximately 26 minutes.
Nobody checked to make sure that at least one of the UPSs showed a status of "ONLINE" instead of "ONBATTERY"? Were there no UPSs configured to alert during continued and extended PF? Surely people didn't just trust the sound/vibration of the running generator.
-Jim P.