Howdy! Looking to replace some edge routers for my small ISP. With all the various SDN platforms available along with various choices of bare-metal hardware platforms, im thinking i may go this route instead of going with Cisco/Juniper/Etc. I only need a handful of 10G uplinks. The SuperMicro SSE-G3648B and the Penguin Arctica Network switches appear to fit my needs. I am eyeing Cumulus Linux to run on these, but that isn’t set in stone. They’ll likely be getting 2 full tables along with some peers. Has anyone run SuperMicro or Penguin hardware with Cumulus in this type of scenario? What were your experiences? How is BGP convergence time on x86 hardware these days? Any insight would be appreciated. Thank You, Mike
Howdy!
Looking to replace some edge routers for my small ISP. With all the various SDN platforms available along with various choices of bare-metal hardware platforms, im thinking i may go this route instead of going with Cisco/Juniper/Etc.
I only need a handful of 10G uplinks. The SuperMicro SSE-G3648B and the Penguin Arctica Network switches appear to fit my needs.
I am eyeing Cumulus Linux to run on these, but that isn’t set in stone.
They’ll likely be getting 2 full tables along with some peers. afaik if these are consistent with other t2/tomahawk/helix switches
On 11/18/17 17:55, mike.lyon@gmail.com wrote: they're roughly 200K alpm routes installed or as few as 16K. if you install selected routes and otherwise default or this is a peering only router, that can get you pretty far but it's not a full table by any means. https://docs.cumulusnetworks.com/display/DOCS/Routing (cumulous details on route table size and alpm routes)
Has anyone run SuperMicro or Penguin hardware with Cumulus in this type of scenario?
What were your experiences? How is BGP convergence time on x86 hardware these days?
control plane on the switches mentioned about is 2GB of ram and a dual core atom, so it's fine more or less for a datacenter TOR. It's not particularly powerful. I find out particular tooling doesn't run that well in 4GB any more so your milage may vary. the supermicro should bear a striking resemblance to the dell 3048 that was splayed open here http://eoinpk.blogspot.com/2015/08/under-hood-of-dell-s3048-on.html
Any insight would be appreciated.
Thank You, Mike
On 2017-11-19 02:55, mike.lyon@gmail.com wrote:
Howdy!
Looking to replace some edge routers for my small ISP. With all the various SDN platforms available along with various choices of bare-metal hardware platforms, im thinking i may go this route instead of going with Cisco/Juniper/Etc.
I only need a handful of 10G uplinks. The SuperMicro SSE-G3648B and the Penguin Arctica Network switches appear to fit my needs.
I am eyeing Cumulus Linux to run on these, but that isn’t set in stone.
They’ll likely be getting 2 full tables along with some peers.
Has anyone run SuperMicro or Penguin hardware with Cumulus in this type of scenario?
What were your experiences? How is BGP convergence time on x86 hardware these days?
Any insight would be appreciated.
Thank You, Mike
Replacing a edge-router with a switch is nothing new, however make sure you actually replace it with the correct one. The Supermicro looks like any generic Helix4-switch and is a ToR-switch for the datacenter. Its not very fitting for edge-routing. It does not have buffers at all and would make your sub-speed connections perform like shit, and also it has a tiny LPM table so you wont be able to fit anywhere near a full table in there It seems that you want a cost-effective 1G solution given that you linked SSE-G3648B? Merchant-switch silicon and edge-routing isn't very competitive on 1G/10G, both because traditional legacy-routers is somewhat cheap for 1G applications and also that 1G is virtually non-existant in datacenter enviroments these days so its hard to leverage the economy-of-scale from there on these swithces. Look at Nokias portfolio for 1G/10G routers, they still care in that segment and is in Europe a very popular choice for broadband buildouts, as is Huaweis smaller NetEngines but that might not fly that well in the US. Juniper MX150 might also work depending on how much 1G you need, but you likely need more. If you bump it up a notch to 10G/100G or 100G only the market for routing-merchant-silicon looks much better. I guess the most famous platform is the Arista 7280R that was the first Broadcom-based box that accepted 1M routes, had big buffers and didnt cost the equivalent of a bunch of new cars for a 1Tbit of capacity like J/C/N/H would charge you for a equivalent linecard to their edge-portfolios. Cisco quickly released NCS550 productline as an answer, Huawei released CE6870-line (but didnt do the LEM/LPM hack that C/A did for full tables to protect NetEngine BU), Juniper pushes QFX10K which is somewhat equal to a Broadcom Jericho-based box. The only Whitebox-vendor i know off that actually has a Jericho (qumran) based box is Agema with the AGC7648S, not sure which stand-alone NOS that actually supports this box fully. Now Jericho+ is also out and Jericho2 is around the corner so i guess we will see alot bigger and even more competetive switch-routers based on these chips. But it doesent really help much if you are operating in 1G/10G space. -- hugge
Which is sad because I believe there are a ton of people using old gear (lacking modern features and security) because the old gear meets price and performance requirements. Although obviously much smaller networks (and thus potential with each one), it's easy to say there are more 1G\10G ISPs than there are 100G ISPs. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Fredrik Korsbäck" <hugge@nordu.net> To: nanog@nanog.org Sent: Sunday, November 19, 2017 1:46:53 AM Subject: Re: Commodity routers/switches On 2017-11-19 02:55, mike.lyon@gmail.com wrote:
Howdy!
Looking to replace some edge routers for my small ISP. With all the various SDN platforms available along with various choices of bare-metal hardware platforms, im thinking i may go this route instead of going with Cisco/Juniper/Etc.
I only need a handful of 10G uplinks. The SuperMicro SSE-G3648B and the Penguin Arctica Network switches appear to fit my needs.
I am eyeing Cumulus Linux to run on these, but that isn’t set in stone.
They’ll likely be getting 2 full tables along with some peers.
Has anyone run SuperMicro or Penguin hardware with Cumulus in this type of scenario?
What were your experiences? How is BGP convergence time on x86 hardware these days?
Any insight would be appreciated.
Thank You, Mike
Replacing a edge-router with a switch is nothing new, however make sure you actually replace it with the correct one. The Supermicro looks like any generic Helix4-switch and is a ToR-switch for the datacenter. Its not very fitting for edge-routing. It does not have buffers at all and would make your sub-speed connections perform like shit, and also it has a tiny LPM table so you wont be able to fit anywhere near a full table in there It seems that you want a cost-effective 1G solution given that you linked SSE-G3648B? Merchant-switch silicon and edge-routing isn't very competitive on 1G/10G, both because traditional legacy-routers is somewhat cheap for 1G applications and also that 1G is virtually non-existant in datacenter enviroments these days so its hard to leverage the economy-of-scale from there on these swithces. Look at Nokias portfolio for 1G/10G routers, they still care in that segment and is in Europe a very popular choice for broadband buildouts, as is Huaweis smaller NetEngines but that might not fly that well in the US. Juniper MX150 might also work depending on how much 1G you need, but you likely need more. If you bump it up a notch to 10G/100G or 100G only the market for routing-merchant-silicon looks much better. I guess the most famous platform is the Arista 7280R that was the first Broadcom-based box that accepted 1M routes, had big buffers and didnt cost the equivalent of a bunch of new cars for a 1Tbit of capacity like J/C/N/H would charge you for a equivalent linecard to their edge-portfolios. Cisco quickly released NCS550 productline as an answer, Huawei released CE6870-line (but didnt do the LEM/LPM hack that C/A did for full tables to protect NetEngine BU), Juniper pushes QFX10K which is somewhat equal to a Broadcom Jericho-based box. The only Whitebox-vendor i know off that actually has a Jericho (qumran) based box is Agema with the AGC7648S, not sure which stand-alone NOS that actually supports this box fully. Now Jericho+ is also out and Jericho2 is around the corner so i guess we will see alot bigger and even more competetive switch-routers based on these chips. But it doesent really help much if you are operating in 1G/10G space. -- hugge
On 11/19/17 07:36, Mike Hammett wrote:
Which is sad because I believe there are a ton of people using old gear (lacking modern features and security) because the old gear meets price and performance requirements. Although obviously much smaller networks (and thus potential with each one), it's easy to say there are more 1G\10G ISPs than there are 100G ISPs. Feature demand drives per port costs that are not very competitively achieved on 1Gb/s switches. On the plus side the per-port cost of the 10Gb/s and mixed 10/100Gb/s switches with usefully rich features continues to slide. Some use of L2 devices for port demux for bigger iron has been done in the past, I imagine it still works for a number of use cases (cisco sells fabric extenders under a similar rational).
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
----- Original Message -----
From: "Fredrik Korsbäck" <hugge@nordu.net> To: nanog@nanog.org Sent: Sunday, November 19, 2017 1:46:53 AM Subject: Re: Commodity routers/switches
On 2017-11-19 02:55, mike.lyon@gmail.com wrote:
Howdy!
Looking to replace some edge routers for my small ISP. With all the various SDN platforms available along with various choices of bare-metal hardware platforms, im thinking i may go this route instead of going with Cisco/Juniper/Etc.
I only need a handful of 10G uplinks. The SuperMicro SSE-G3648B and the Penguin Arctica Network switches appear to fit my needs.
I am eyeing Cumulus Linux to run on these, but that isn’t set in stone.
They’ll likely be getting 2 full tables along with some peers.
Has anyone run SuperMicro or Penguin hardware with Cumulus in this type of scenario?
What were your experiences? How is BGP convergence time on x86 hardware these days?
Any insight would be appreciated.
Thank You, Mike
Replacing a edge-router with a switch is nothing new, however make sure you actually replace it with the correct one.
The Supermicro looks like any generic Helix4-switch and is a ToR-switch for the datacenter. Its not very fitting for edge-routing. It does not have buffers at all and would make your sub-speed connections perform like shit, and also it has a tiny LPM table so you wont be able to fit anywhere near a full table in there
It seems that you want a cost-effective 1G solution given that you linked SSE-G3648B?
Merchant-switch silicon and edge-routing isn't very competitive on 1G/10G, both because traditional legacy-routers is somewhat cheap for 1G applications and also that 1G is virtually non-existant in datacenter enviroments these days so its hard to leverage the economy-of-scale from there on these swithces.
Look at Nokias portfolio for 1G/10G routers, they still care in that segment and is in Europe a very popular choice for broadband buildouts, as is Huaweis smaller NetEngines but that might not fly that well in the US. Juniper MX150 might also work depending on how much 1G you need, but you likely need more.
If you bump it up a notch to 10G/100G or 100G only the market for routing-merchant-silicon looks much better. I guess the most famous platform is the Arista 7280R that was the first Broadcom-based box that accepted 1M routes, had big buffers and didnt cost the equivalent of a bunch of new cars for a 1Tbit of capacity like J/C/N/H would charge you for a equivalent linecard to their edge-portfolios.
Cisco quickly released NCS550 productline as an answer, Huawei released CE6870-line (but didnt do the LEM/LPM hack that C/A did for full tables to protect NetEngine BU), Juniper pushes QFX10K which is somewhat equal to a Broadcom Jericho-based box. The only Whitebox-vendor i know off that actually has a Jericho (qumran) based box is Agema with the AGC7648S, not sure which stand-alone NOS that actually supports this box fully.
Now Jericho+ is also out and Jericho2 is around the corner so i guess we will see alot bigger and even more competetive switch-routers based on these chips. But it doesent really help much if you are operating in 1G/10G space.
participants (4)
-
Fredrik Korsbäck -
joel jaeggli -
Mike Hammett -
mike.lyon＠gmail.com