Hi, Has anyone had issues with Arista switches rebooting out of the blue, when there isn't even a sufficient load on them to be a CPU or memory issue? We have a couple Arista 7280s both SR and CR that have had this behaviour, this is the second time we see this issue and just wanted to see if this is something anyone else is experiencing with this platfrom Thanks, Javier Gutierrez Guerra
Hey Javier,
Has anyone had issues with Arista switches rebooting out of the blue, when there isn't even a sufficient load on them to be a CPU or memory issue? We have a couple Arista 7280s both SR and CR that have had this behaviour, this is the second time we see this issue and just wanted to see if this is something anyone else is experiencing with this platfrom
You may not realise, but you are asking 'has anyone seen software crash'. I've seen software crash many times. You should contact Arista support and provide them with relevant information so they can help you with the issue. -- ++ytti
Hi, Has anyone had issues with Arista switches rebooting out of the blue, when there isn't even a sufficient load on them to be a CPU or memory issue? We have a couple Arista 7280s both SR and CR that have had this behaviour, this is the second time we see this issue and just wanted to see if this is something anyone else is experiencing with this platfrom Thanks,
We found a bug on the 64 port x 100gig model that if you insert a quad twinax 10gig fanout cable in many of the ports it will trigger a reboot. Some select ports are okay and supported, but the ones that are not would trigger a reset. Issue was immediate to the cable being inserted. No idea if this was patched or not. - Ethan
On 5/4/20 4:02 PM, Ethan O'Toole wrote:
We found a bug on the 64 port x 100gig model that if you insert a quad twinax 10gig fanout cable in many of the ports it will trigger a reboot.
Some select ports are okay and supported, but the ones that are not would trigger a reset. Issue was immediate to the cable being inserted. No idea if this was patched or not.
Did you contact the vendor and did they commit to a fix? I can't imagine a vendor not wanting to fix an easily reproducible bug such as this. -- Bryan Fields 727-409-1194 - Voice http://bryanfields.net
Did you contact the vendor and did they commit to a fix? I can't imagine a vendor not wanting to fix an easily reproducible bug such as this.
It was a while ago, and the vendor was aware of the issue. Arista had the info on which specific ports would accept the cable. An interesting moment when all the activity LEDs stop flashing green, go Orange for a second across the board, then continue to stay dark for a while as the switch boots back up. - Ethan O'Toole
Hi, On 5/5/20 4:02 am, Ethan O'Toole wrote:
We found a bug on the 64 port x 100gig model that if you insert a quad twinax 10gig fanout cable in many of the ports it will trigger a reboot.
Timing, seems like there is a similar issue for Juniper QFX51110-48S devices, just saw PR 1499422: This technical support bulletin(TSB) has been opened to inform Juniper network customers about PR1499422. On the QFX5110-48S device inserting any QSFP-100G-transceiver on one or more network ports will cause FPC state to flap. In the problem state FPC will go down as soon as the 100G link comes up and FPC flap will be seen every 90 seconds.
On Mon, 4 May 2020 at 23:06, Ethan O'Toole <telmnstr@757.org> wrote:
We found a bug on the 64 port x 100gig model that if you insert a quad twinax 10gig fanout cable in many of the ports it will trigger a reboot.I
I've seen a similar issue in another vendor, where specific SFP inserted would reload the linecard. This was because the SFP didn't answer fast enough to I2C queries and the polling code couldn't handle the error so it crashed the whole linecard. Vendor didn't fix the code, because it didn't happen on vendor optic, while obviously they must have understood they can't guarantee vendor optic answers in a timely manner in I2C. I2C is a pretty terrible bus, particularly if you try to actually hang everything off of a single I2C bus, single misbehaving speaker and you might get your power supplies offline. Hopefully we'll move to I3C or 10SPE Ethernet soon. Or maybe some sort of I2C switch where every connection is on its own bus. -- ++ytti
❦ 5 mai 2020 09:09 +03, Saku Ytti:
We found a bug on the 64 port x 100gig model that if you insert a quad twinax 10gig fanout cable in many of the ports it will trigger a reboot.I
I've seen a similar issue in another vendor, where specific SFP inserted would reload the linecard. This was because the SFP didn't answer fast enough to I2C queries and the polling code couldn't handle the error so it crashed the whole linecard. Vendor didn't fix the code, because it didn't happen on vendor optic, while obviously they must have understood they can't guarantee vendor optic answers in a timely manner in I2C.
We had a similar issue, but vendor fixed the issue (despite it only happened with cheap third-party optics). If we talk about the same vendor, it was fixed in 17.3R4, 17.4R3, 17.3R3-S4 and 18.1R3-S4. It's PR 1425893 (not public). -- Man is the only animal that blushes -- or needs to. -- Mark Twain
On 5/5/20 2:09 AM, Saku Ytti wrote:
I2C is a pretty terrible bus, particularly if you try to actually hang everything off of a single I2C bus, single misbehaving speaker and you might get your power supplies offline. Hopefully we'll move to I3C or 10SPE Ethernet soon. Or maybe some sort of I2C switch where every connection is on its own bus.
I was of the impression that, due to addressing, I2C bus switches are already typically used between each transceiver port and the system bus (hopefully one of many) that connects the micro to the ports (hopefully relatively dedicated to that purpose). That's certainly been the case in all of the switches I've torn down and, last time I checked at least the SFP specification, there wasn't much of a way around it since there was each transceiver uses the same, fixed addresses for ID and DDM. But yes, I2C, while very useful, is the devil. Clock stretching is particularly annoying along with the requisite use of open-drain drivers to accomplish it. I was not aware of 10SPE, though...looks very useful (for lots of purposes). Physical multi-drop on low-cost cabling is quite useful. -- Brandon Martin
participants (7)
-
Brandon Martin -
Bryan Fields -
Chris -
Ethan O'Toole -
Javier Gutierrez Guerra -
Saku Ytti -
Vincent Bernat