Just a heads up to anyone else colo'd at 365 TPA1/TAMSFLDE. Currently seeing floor temps of ~105F as reported by equipment. Started yesterday at ~5:30PM eastern. 2nd AC failure in the last 30 days. They have not sent any advisory notices as of yet.
Oof. Get ready to replace all spinning media you may have there. -George Sent from my iPhone
On Jun 12, 2023, at 4:06 PM, Nick Olsen <nick@141networks.com> wrote:
Just a heads up to anyone else colo'd at 365 TPA1/TAMSFLDE. Currently seeing floor temps of ~105F as reported by equipment. Started yesterday at ~5:30PM eastern. 2nd AC failure in the last 30 days. They have not sent any advisory notices as of yet.
Hello all,
From our side (365 Data Centers) We saw the issue start at a little after 6:30 PM yesterday. We had a component failure and we saw a rise in temperature. We brought additional capacity online while we were working with our vendors on getting replacement hardware. The hardware has now been replaced and the system is back online. This unit and others in this suite are on the block for replacement this year as we add cooling capacity to our various spaces within the Franklin Exchange building.
On a side note, this was not related to the issue mentioned from earlier in the month. This was a straight component failure and replacement. Any customers impacted can reach out to the 365 Customer Service Center for details and documentation. Thank you, James Ashton VP Network Engineering 365 Data Centers From: "NANOG mailing list" <nanog@nanog.org> To: "George Herbert" <george.herbert@gmail.com> Cc: "NANOG mailing list" <nanog@nanog.org> Sent: Monday, June 12, 2023 8:03:10 PM Subject: Re: 365 Datacenters Tampa AC Failure Issue started a little after 2am, I was hard down till about 11:30am (servers did a high temp shutdown) On Jun 12, 2023 8:01 PM, Michael Spears <michael@spears.io> wrote: Yep there's issues over there. They had some compressors go down. Should be getting back to normal now... Hasn't been a good month for them in regards to cooling.. On Jun 12, 2023 7:15 PM, George Herbert <george.herbert@gmail.com> wrote: BQ_BEGIN Oof. Get ready to replace all spinning media you may have there. -George Sent from my iPhone
On Jun 12, 2023, at 4:06 PM, Nick Olsen <nick@141networks.com> wrote:
Just a heads up to anyone else colo'd at 365 TPA1/TAMSFLDE. Currently seeing floor temps of ~105F as reported by equipment. Started yesterday at ~5:30PM eastern. 2nd AC failure in the last 30 days. They have not sent any advisory notices as of yet.
BQ_END
James, Any word on how redundant systems failed at the same time? Or why the secondary was untested and not known-working? It seems like HVAC maintenance was done by the night crew which was quietly eliminated at some point. On Tue, Jun 13, 2023 at 12:02 AM James Ashton <james@gitflorida.com> wrote:
Hello all, From our side (365 Data Centers) We saw the issue start at a little after 6:30 PM yesterday. We had a component failure and we saw a rise in temperature. We brought additional capacity online while we were working with our vendors on getting replacement hardware. The hardware has now been replaced and the system is back online. This unit and others in this suite are on the block for replacement this year as we add cooling capacity to our various spaces within the Franklin Exchange building.
On a side note, this was not related to the issue mentioned from earlier in the month. This was a straight component failure and replacement.
Any customers impacted can reach out to the 365 Customer Service Center for details and documentation.
Thank you, James Ashton VP Network Engineering 365 Data Centers
------------------------------ *From: *"NANOG mailing list" <nanog@nanog.org> *To: *"George Herbert" <george.herbert@gmail.com> *Cc: *"NANOG mailing list" <nanog@nanog.org> *Sent: *Monday, June 12, 2023 8:03:10 PM *Subject: *Re: 365 Datacenters Tampa AC Failure
Issue started a little after 2am, I was hard down till about 11:30am (servers did a high temp shutdown)
On Jun 12, 2023 8:01 PM, Michael Spears <michael@spears.io> wrote:
Yep there's issues over there. They had some compressors go down. Should be getting back to normal now... Hasn't been a good month for them in regards to cooling..
On Jun 12, 2023 7:15 PM, George Herbert <george.herbert@gmail.com> wrote:
Oof. Get ready to replace all spinning media you may have there.
-George
Sent from my iPhone
On Jun 12, 2023, at 4:06 PM, Nick Olsen <nick@141networks.com> wrote:
Just a heads up to anyone else colo'd at 365 TPA1/TAMSFLDE. Currently seeing floor temps of ~105F as reported by equipment. Started yesterday at ~5:30PM eastern. 2nd AC failure in the last 30 days. They have not sent any advisory notices as of yet.
Is this the building with the lizard mural? On Mon, Jun 12, 2023 at 4:03 PM Nick Olsen <nick@141networks.com> wrote:
Just a heads up to anyone else colo'd at 365 TPA1/TAMSFLDE. Currently seeing floor temps of ~105F as reported by equipment. Started yesterday at ~5:30PM eastern. 2nd AC failure in the last 30 days. They have not sent any advisory notices as of yet.
-- -- Joe Hamelin, W7COM, Tulalip, WA, 360-474-7474
participants (5)
-
George Herbert -
James Ashton -
Joe Hamelin -
Michael Spears -
Nick Olsen