Curious to hear if the community has had any real-world experience using Mellanox/Cumulus (nVidia) for L2/L3 things outside of the datacenter. Like other vendors, notably Arista, they seem to be trying to move out of the datacenter and target SPs and the layer 3 market. Personally, I think Arista has worked out most of the kinks over the last few years, and we've been happy with their L3 solutions (e.g., the 7280s) While Mellanox's chipset is intriguing, I get a sense of feature-itis from their marketing. (BGP, OSPF, NAT, we do it all etc.) No IS-IS support, I'm told ... Anybody using these in production in an SP environment? And if so, any opinions, good or bad? Feel free to reach out off-list if you prefer. Thank you, - bryan
On 02/11/2020 17:52, Bryan Holloway wrote:
Anybody using these in production in an SP environment? And if so, any opinions, good or bad?
I haven't used them in an SP environment precisely because the Mellanox hardware - while miles better than equivalent Broadcom designs - does not cater to anyone with more than the most basic of QoS requirements. Realistically, the ASIC designs are brilliant for data centre/storage/HPC use, but they do not have (last I was briefed) any hardware that would replace even an access switch, let alone a capable border router. I would *love* for that to change, so please correct me if I'm outdated. On the software side, Cumulus Linux is very capable, and a joy to work with. However, the business case to support even the Broadcom DNX range (e.g. Arista 7280R) just wasn't there /before/ their acquisition. Again, if that's changed it would be a fine software suite to investigate. Regards, -- Tom
One of my CDN caching providers sent a Mellanox SN2700 with their servers. Seems to be running well. They manage them, I just give them rack, power, and a couple 10 gig links into my core -Aaron -----Original Message----- From: NANOG <nanog-bounces+aaron1=gvtc.com@nanog.org> On Behalf Of Tom Hill Sent: Wednesday, November 4, 2020 9:37 AM To: nanog@nanog.org Subject: Re: Mellanox / Cumulus On 02/11/2020 17:52, Bryan Holloway wrote:
Anybody using these in production in an SP environment? And if so, any opinions, good or bad?
I haven't used them in an SP environment precisely because the Mellanox hardware - while miles better than equivalent Broadcom designs - does not cater to anyone with more than the most basic of QoS requirements. Realistically, the ASIC designs are brilliant for data centre/storage/HPC use, but they do not have (last I was briefed) any hardware that would replace even an access switch, let alone a capable border router. I would *love* for that to change, so please correct me if I'm outdated. On the software side, Cumulus Linux is very capable, and a joy to work with. However, the business case to support even the Broadcom DNX range (e.g. Arista 7280R) just wasn't there /before/ their acquisition. Again, if that's changed it would be a fine software suite to investigate. Regards, -- Tom
On 04/11/2020 16:02, aaron1@gvtc.com wrote:
One of my CDN caching providers sent a Mellanox SN2700 with their servers. Seems to be running well. They manage them, I just give them rack, power, and a couple 10 gig links into my core
At this point, we may descend into a "what does SP mean" debate, but I am not at all surprised that CDN providers would like the Spectrum ASICs: they're excellent at pushing lots of bits, quickly. Because they lack buffer-laden features (by design) a lot of SPs - say, access ISPs - will stop and say, "What? This is a DC switch!" -- Tom
participants (3)
-
aaron1＠gvtc.com -
Bryan Holloway -
Tom Hill