Suggestion for Layer 3, all SFP+ switches
Hello, we're looking for some L3 switches to be used as distribution devices. They should have all (at leaast 24) SFP+ ports 10G and at least a couple of upstream ports 40G capable, but what is most important, they should be able to run MPLS, EoMPLS and VPLS. Is there any device you would suggest us? We where thinking about NEXUS but I'm sure there are also others, even if it is not so easy to find them on the Internet. Thank you in advance for your help . Giuseppe Spanò Datacast Srl
Brocade/arris is eager for business these days. They have a nice switch (10g ports with 40g stacking) that should meet your needs with very aggressive pricing. Eric Sent from my iPhone
On Apr 18, 2018, at 5:26 AM, Giuseppe Spanò - Datacast Srl <spano@datacast.it> wrote:
Hello,
we're looking for some L3 switches to be used as distribution devices. They should have all (at leaast 24) SFP+ ports 10G and at least a couple of upstream ports 40G capable, but what is most important, they should be able to run MPLS, EoMPLS and VPLS. Is there any device you would suggest us? We where thinking about NEXUS but I'm sure there are also others, even if it is not so easy to find them on the Internet.
Thank you in advance for your help .
Giuseppe Spanò Datacast Srl
On 04/18/2018 03:49 PM, Eric Litvin wrote:
Brocade/arris is eager for business these days. They have a nice switch (10g ports with 40g stacking) that should meet your needs with very aggressive pricing.
Does the Brocade/Foundry-lineage stuff that went to Arris actually do MPLS? I didn't think ICX did any MPLS. The SLX (and MLX) line that went to Extreme does but is perhaps overkill (it will also do Internet-scale FIB). The SLX9540 is a 48 port SFP+ pizza box that also has 6 40/100Gb QSFP+/28 ports on it. You'd need the "advanced feature" license for MPLS, and I don't know how mature the MPLS code is. Pricing I've seen is pretty good for what you get, but again it may be overkill. Juniper has some nice boxes in the EX series with at least MPLS L2-endpoint functionality that might also be an option for this sort of thing, but I don't know any models off the top of my head. -- Brandon Martin
Ruckus ICX switches do not do MPLS. They meet all the other requirements listed, but unfortunately MPLS was listed as the most important one. On Wed, Apr 18, 2018 at 3:01 PM Brandon Martin <lists.nanog@monmotha.net> wrote:
On 04/18/2018 03:49 PM, Eric Litvin wrote:
Brocade/arris is eager for business these days. They have a nice switch (10g ports with 40g stacking) that should meet your needs with very aggressive pricing.
Does the Brocade/Foundry-lineage stuff that went to Arris actually do MPLS? I didn't think ICX did any MPLS.
The SLX (and MLX) line that went to Extreme does but is perhaps overkill (it will also do Internet-scale FIB). The SLX9540 is a 48 port SFP+ pizza box that also has 6 40/100Gb QSFP+/28 ports on it. You'd need the "advanced feature" license for MPLS, and I don't know how mature the MPLS code is. Pricing I've seen is pretty good for what you get, but again it may be overkill.
Juniper has some nice boxes in the EX series with at least MPLS L2-endpoint functionality that might also be an option for this sort of thing, but I don't know any models off the top of my head. -- Brandon Martin
-- -- Hunter Fuller Network Engineer VBH Annex B-5 +1 256 824 5331 Office of Information Technology The University of Alabama in Huntsville Systems and Infrastructure
Juniper ACX 5048 is what we use though you need to license 10g ports (ACX5K-L-1X10GE) and VPN (ACX5K-L-IPVPN) QFX does MPLS but I'm pretty sure it doesn't do VPLs. ns -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Brandon Martin Sent: Wednesday, April 18, 2018 3:01 PM To: nanog@nanog.org Subject: Re: Suggestion for Layer 3, all SFP+ switches On 04/18/2018 03:49 PM, Eric Litvin wrote:
Brocade/arris is eager for business these days. They have a nice switch (10g ports with 40g stacking) that should meet your needs with very aggressive pricing.
Does the Brocade/Foundry-lineage stuff that went to Arris actually do MPLS? I didn't think ICX did any MPLS. The SLX (and MLX) line that went to Extreme does but is perhaps overkill (it will also do Internet-scale FIB). The SLX9540 is a 48 port SFP+ pizza box that also has 6 40/100Gb QSFP+/28 ports on it. You'd need the "advanced feature" license for MPLS, and I don't know how mature the MPLS code is. Pricing I've seen is pretty good for what you get, but again it may be overkill. Juniper has some nice boxes in the EX series with at least MPLS L2-endpoint functionality that might also be an option for this sort of thing, but I don't know any models off the top of my head. -- Brandon Martin
look at these... * Juniper ACX5048 - I've deployed about ~50 of these over the last couple years and they are great boxes. I'm using them as mpls p/pe running L3VPN (v4 and tested 6vpe), L2VPN (manual martini l2circuits and bgp-ad rfc4762, I'll say that IOS XR asr9k has an occasional problem with vpls pw towards acx5048 rfc4762, overcome with bounce of ldp neighbor only when needed, not sure who to blame) (48) 10 gig sfp+ (6) 40 gig qsfp+ - or these can run as (24) 10 gig ports using a break-out cable * Cisco NCS5K and its variants as I think there are a few....you might find a 10/40 gig option here, however I recall the one I tested with in my lab a few years ago had 10/100 gig. I will say that when I tested it a few year back that I wasn't ready for prime time, but, in cisco defense, they've had a few years to make improvements on it and I should, and you should, look into it. * Juniper ACX5400 (ACX5448) - I'm seeing this advertised on juniper.net now...new box - I want one for my lab - (48) 10 gig - (4) 100 gig, check if you can slide a 40 gig optic into that qsfp slot * Juniper ACX5k+ - unsure if it's advertised yet by Juniper... this is a new box - I want one for my lab - lots of 10 gig and I recall some 25 or 40 or 100, I don't recall https://www.mail-archive.com/nanog@nanog.org/msg93672.html or google - 1/2u 100g Metro-E Aggregation Switch * Juniper EX4550 - I've ran these are virtual chassis paired top-of-rack in my small data centers with rock solid performance - with multiple cdn caches sitting behind them. Now, I have tested L3VPN some years back, and I've heard they also do L2VPN martini manual pw's...i'm about to give them another go in mpls testing... check back with my shortly if you wanna know how it goes. Yesterday I pushed in my first EX4550-EM-2QSFP 40 gig module into my lab EX4550 in preparation for new supercore (100 gig mx960's) and had to upgrade junos from v12 to v13 (wasn't avail so went with 14.1X53-D46.7) and now optic was visible in that 40 gig module in the ex4550 - (32) 10 gig ports - (2) 40 gig - Aaron
What is your budget? I know on the low end many operators are using the Huawei S6720S-26Q-EI-24S-AC. You can get these new for $2500 to $3500, and the support all the features and port counts you requested. The also have a lifetime warranty that includes advanced replacement (10 days), TAC support, and software support all for free if you buy through official channels. It support MPLS, and also VXLAN. Extreme seems to have some good options, but I doubt they are that low cost. For Juniper you need to look at the ACX series which is expensive. Like the ACX5048 which list price is $40k not that anyone pays list, and that's before port licenses. The EX series does not have proper MPLS support. Cisco has mutliple options, but mainly the NCS based on your port count I think. Supposely the C3850 and C9500 now support MPLS? There is a new 16 port 10G version of the C9500. I haven't looked into Nexus switches. Does Nexus support full MPLS? HPE has some low cost options. In their FlexFabric and FlexNetwork lines that support MPLS in Comware V7. Who else are we missing? MPLS support really cuts down this list, but I agree its a critical feature for most service providers. On Wed, Apr 18, 2018 at 7:26 AM, Giuseppe Spanò - Datacast Srl < spano@datacast.it> wrote:
Hello,
we're looking for some L3 switches to be used as distribution devices. They should have all (at leaast 24) SFP+ ports 10G and at least a couple of upstream ports 40G capable, but what is most important, they should be able to run MPLS, EoMPLS and VPLS. Is there any device you would suggest us? We where thinking about NEXUS but I'm sure there are also others, even if it is not so easy to find them on the Internet.
Thank you in advance for your help .
Giuseppe Spanò Datacast Srl
I've been testing IPInfusion OcNOS running on Dell Z9100 and S4048. I've run a couple of test cases using MPLS LDP signaled port based and VLAN based VPWS (pseudowire / e-line / xconnect / Juniper CCC) and VPLS (e-lan) over an OSPFv2 IGP. It's working well between Dell/IPI to Dell/IPI boxes. We have had issues with the VPLS between Dell/IPI to Juniper/JunOS where the circuit will show up on the Juniper and down on the Dell. If we clear LDP session on the Dell, it comes back up right away. This seems to be similar to what Aaron is seeing in his multi-vendor environment. The price on the Dell hardware is really good. The features included with OcNOS are much better than FTOS9. If you aren't partial to Dell, you can run OcNOS on a variety of other whitebox switches, like EdgeCore. I haven't tested MP-BGP and L3VPN or BFD yet, but that is supposedly supported in OcNOS as well. -ben On Wed, Apr 18, 2018 at 8:32 PM, Colton Conor <colton.conor@gmail.com> wrote:
What is your budget?
I know on the low end many operators are using the Huawei S6720S-26Q-EI-24S-AC. You can get these new for $2500 to $3500, and the support all the features and port counts you requested. The also have a lifetime warranty that includes advanced replacement (10 days), TAC support, and software support all for free if you buy through official channels. It support MPLS, and also VXLAN.
Extreme seems to have some good options, but I doubt they are that low cost.
For Juniper you need to look at the ACX series which is expensive. Like the ACX5048 which list price is $40k not that anyone pays list, and that's before port licenses. The EX series does not have proper MPLS support.
Cisco has mutliple options, but mainly the NCS based on your port count I think. Supposely the C3850 and C9500 now support MPLS? There is a new 16 port 10G version of the C9500. I haven't looked into Nexus switches. Does Nexus support full MPLS?
HPE has some low cost options. In their FlexFabric and FlexNetwork lines that support MPLS in Comware V7.
Who else are we missing? MPLS support really cuts down this list, but I agree its a critical feature for most service providers.
On Wed, Apr 18, 2018 at 7:26 AM, Giuseppe Spanò - Datacast Srl < spano@datacast.it> wrote:
Hello,
we're looking for some L3 switches to be used as distribution devices. They should have all (at leaast 24) SFP+ ports 10G and at least a couple of upstream ports 40G capable, but what is most important, they should be able to run MPLS, EoMPLS and VPLS. Is there any device you would suggest us? We where thinking about NEXUS but I'm sure there are also others, even if it is not so easy to find them on the Internet.
Thank you in advance for your help .
Giuseppe Spanò Datacast Srl
Ben, The Dell options intrigue me. First question is who do you talk to at Dell about their solutions as most sales guys just seem to know their laptop and server lines? How does Dell's pricing compare with Edge-Core. Considering most of the hardware is the same Broadcom chipset, what are the reasons you went with Dell over someone like Edge-Core or the other OEM's? I have looked into IPInfusion OcNOS and feature wise it looks nice, but by the time you pay IPInfusion OcNOS for the software and an OEM for the hardware the costs adds up to a Cisco/Juniper equalivent model. Dell seems interesting as I think they include an OS for free, but then you can load IPInfusion OcNOS or Cumulus or others onto it if that doesn't meet your need. You mentioned FTOS9. Doesn't Dell now have OS10 version? On Thu, Apr 19, 2018 at 8:28 AM, Ben Bartsch <uwcableguy@gmail.com> wrote:
I've been testing IPInfusion OcNOS running on Dell Z9100 and S4048. I've run a couple of test cases using MPLS LDP signaled port based and VLAN based VPWS (pseudowire / e-line / xconnect / Juniper CCC) and VPLS (e-lan) over an OSPFv2 IGP. It's working well between Dell/IPI to Dell/IPI boxes. We have had issues with the VPLS between Dell/IPI to Juniper/JunOS where the circuit will show up on the Juniper and down on the Dell. If we clear LDP session on the Dell, it comes back up right away. This seems to be similar to what Aaron is seeing in his multi-vendor environment. The price on the Dell hardware is really good. The features included with OcNOS are much better than FTOS9. If you aren't partial to Dell, you can run OcNOS on a variety of other whitebox switches, like EdgeCore.
I haven't tested MP-BGP and L3VPN or BFD yet, but that is supposedly supported in OcNOS as well.
-ben
On Wed, Apr 18, 2018 at 8:32 PM, Colton Conor <colton.conor@gmail.com> wrote:
What is your budget?
I know on the low end many operators are using the Huawei S6720S-26Q-EI-24S-AC. You can get these new for $2500 to $3500, and the support all the features and port counts you requested. The also have a lifetime warranty that includes advanced replacement (10 days), TAC support, and software support all for free if you buy through official channels. It support MPLS, and also VXLAN.
Extreme seems to have some good options, but I doubt they are that low cost.
For Juniper you need to look at the ACX series which is expensive. Like the ACX5048 which list price is $40k not that anyone pays list, and that's before port licenses. The EX series does not have proper MPLS support.
Cisco has mutliple options, but mainly the NCS based on your port count I think. Supposely the C3850 and C9500 now support MPLS? There is a new 16 port 10G version of the C9500. I haven't looked into Nexus switches. Does Nexus support full MPLS?
HPE has some low cost options. In their FlexFabric and FlexNetwork lines that support MPLS in Comware V7.
Who else are we missing? MPLS support really cuts down this list, but I agree its a critical feature for most service providers.
On Wed, Apr 18, 2018 at 7:26 AM, Giuseppe Spanò - Datacast Srl < spano@datacast.it> wrote:
Hello,
we're looking for some L3 switches to be used as distribution devices. They should have all (at leaast 24) SFP+ ports 10G and at least a couple of upstream ports 40G capable, but what is most important, they should be able to run MPLS, EoMPLS and VPLS. Is there any device you would suggest us? We where thinking about NEXUS but I'm sure there are also others, even if it is not so easy to find them on the Internet.
Thank you in advance for your help .
Giuseppe Spanò Datacast Srl
Aren't there issues/concerns with Huawei ? I think we pay about $10k with discounts and about (4) 10 gig port license to slow start our deployment of ACX5048's.... 10 gig east , 10 gig west , dual 10's facing FTTH OLT (Calix E7) -Aaron
Yes, there are issues/concerns with using Huawei in the USA, but in the rest of the world they are the number 2 vendor. Also, $3500 for that box with lifetime support and warranty (their TAC is in Plano, Texas) vs $10,000 for an ACX5048 onetime plus at least $1500 a year for JTAC seems like a big difference! ACX has 48 ports vs 24 in the Huawei, but you have to licenses each one of the ports on the ACX making the total cost even higher. Sounds like your ACX cost more than you E7 that its feeding! On Thu, Apr 19, 2018 at 11:07 AM, Aaron Gould <aaron1@gvtc.com> wrote:
Aren't there issues/concerns with Huawei ?
I think we pay about $10k with discounts and about (4) 10 gig port license to slow start our deployment of ACX5048's.... 10 gig east , 10 gig west , dual 10's facing FTTH OLT (Calix E7)
-Aaron
Thanks Colton, Since I live in the US, and work for a boss that’s nervous (concerned) about those things, then I comply. I remember mentioning Huawei as an option recently in a meeting and the boss and a few other fellow engineers were nervous and resistant to it. I tend to feel the same. I see you started a thread on comparing those 2 (zte and Huawei) … and was immediately met with cautionary/warning statements about these some things... from Suresh and Curtis. So I wonder if because of all this, are ZTE and Huawei sales being adversely affected in the US? …it would seem so, but thought I’d ask y’all. Google - China Showdown Huawei vs ZTE http://seclists.org/nanog/2018/Apr/293 - Aaron
Well, if the US government spies on everyone using exported cisco hardware, why wouldn't the PRC do the same? On 20 April 2018 at 08:59, Aaron Gould <aaron1@gvtc.com> wrote:
Thanks Colton, Since I live in the US, and work for a boss that’s nervous (concerned) about those things, then I comply. I remember mentioning Huawei as an option recently in a meeting and the boss and a few other fellow engineers were nervous and resistant to it. I tend to feel the same.
I see you started a thread on comparing those 2 (zte and Huawei) … and was immediately met with cautionary/warning statements about these some things... from Suresh and Curtis.
So I wonder if because of all this, are ZTE and Huawei sales being adversely affected in the US? …it would seem so, but thought I’d ask y’all.
Google - China Showdown Huawei vs ZTE
http://seclists.org/nanog/2018/Apr/293
- Aaron
Thank you very much to everyone. The budget is around 3000-5000 $ each, possibly. There are many devices that could match our needs but as usual the dark side of this market is the platforms compatibility. We deployed many Mikrotik and Ericsson devices, hope they will "match" with a Cisco or Juniper or Huawey device with regards to MPLS, EoMPLS, VPLS etc... Anyway your kind help is really very appreciated, we'll decide for one and will test it, no way out I think. Giuseppe Il 19/04/18 03:32, Colton Conor ha scritto:
What is your budget?
I know on the low end many operators are using the Huawei S6720S-26Q-EI-24S-AC. You can get these new for $2500 to $3500, and the support all the features and port counts you requested. The also have a lifetime warranty that includes advanced replacement (10 days), TAC support, and software support all for free if you buy through official channels. It support MPLS, and also VXLAN.
Extreme seems to have some good options, but I doubt they are that low cost.
For Juniper you need to look at the ACX series which is expensive. Like the ACX5048 which list price is $40k not that anyone pays list, and that's before port licenses. The EX series does not have proper MPLS support.
Cisco has mutliple options, but mainly the NCS based on your port count I think. Supposely the C3850 and C9500 now support MPLS? There is a new 16 port 10G version of the C9500. I haven't looked into Nexus switches. Does Nexus support full MPLS?
HPE has some low cost options. In their FlexFabric and FlexNetwork lines that support MPLS in Comware V7.
Who else are we missing? MPLS support really cuts down this list, but I agree its a critical feature for most service providers.
On Wed, Apr 18, 2018 at 7:26 AM, Giuseppe Spanò - Datacast Srl <spano@datacast.it <mailto:spano@datacast.it>> wrote:
Hello,
we're looking for some L3 switches to be used as distribution devices. They should have all (at leaast 24) SFP+ ports 10G and at least a couple of upstream ports 40G capable, but what is most important, they should be able to run MPLS, EoMPLS and VPLS. Is there any device you would suggest us? We where thinking about NEXUS but I'm sure there are also others, even if it is not so easy to find them on the Internet.
Thank you in advance for your help .
Giuseppe Spanò Datacast Srl
The ZTE 5960 with 48x SFP+ and 4x QSFP28 (40G and 100G capable) will do it within the budget listed. We use it for MPLS and VPLS. Regards Baldur Den tor. 19. apr. 2018 18.17 skrev Giuseppe Spanò - Datacast Srl < spano@datacast.it>:
Thank you very much to everyone.
The budget is around 3000-5000 $ each, possibly. There are many devices that could match our needs but as usual the dark side of this market is the platforms compatibility. We deployed many Mikrotik and Ericsson devices, hope they will "match" with a Cisco or Juniper or Huawey device with regards to MPLS, EoMPLS, VPLS etc...
Anyway your kind help is really very appreciated, we'll decide for one and will test it, no way out I think.
Giuseppe
Il 19/04/18 03:32, Colton Conor ha scritto:
What is your budget?
I know on the low end many operators are using the Huawei S6720S-26Q-EI-24S-AC. You can get these new for $2500 to $3500, and the support all the features and port counts you requested. The also have a lifetime warranty that includes advanced replacement (10 days), TAC support, and software support all for free if you buy through official channels. It support MPLS, and also VXLAN.
Extreme seems to have some good options, but I doubt they are that low cost.
For Juniper you need to look at the ACX series which is expensive. Like the ACX5048 which list price is $40k not that anyone pays list, and that's before port licenses. The EX series does not have proper MPLS support.
Cisco has mutliple options, but mainly the NCS based on your port count I think. Supposely the C3850 and C9500 now support MPLS? There is a new 16 port 10G version of the C9500. I haven't looked into Nexus switches. Does Nexus support full MPLS?
HPE has some low cost options. In their FlexFabric and FlexNetwork lines that support MPLS in Comware V7.
Who else are we missing? MPLS support really cuts down this list, but I agree its a critical feature for most service providers.
On Wed, Apr 18, 2018 at 7:26 AM, Giuseppe Spanò - Datacast Srl <spano@datacast.it <mailto:spano@datacast.it>> wrote:
Hello,
we're looking for some L3 switches to be used as distribution devices. They should have all (at leaast 24) SFP+ ports 10G and at least a couple of upstream ports 40G capable, but what is most important, they should be able to run MPLS, EoMPLS and VPLS. Is there any device you would suggest us? We where thinking about NEXUS but I'm sure there are also others, even if it is not so easy to find them on the Internet.
Thank you in advance for your help .
Giuseppe Spanò Datacast Srl
Colton,
On 19 Apr 2018, at 03:32, Colton Conor <colton.conor@gmail.com> wrote:
Cisco has mutliple options, but mainly the NCS based on your port count I think. Supposely the C3850 and C9500 now support MPLS? There is a new 16 port 10G version of the C9500. I haven't looked into Nexus switches. Does Nexus support full MPLS?
UADP based platforms, both older (C3650/3850) and newer (C9xxx) do support MPLS encap and VXLAN encap and can be extended in future to support others. There are new 9xxx based off UADP 3.0 with 40G and 100G ports: https://www.cisco.com/c/en/us/products/collateral/switches/catalyst-9500-ser... <https://www.cisco.com/c/en/us/products/collateral/switches/catalyst-9500-series-switches/datasheet-c78-738978.html> Nexus 7k supports MPLS with LDP while Nexus 9k supports MPLS but with SR (IGP) or BGP-LU (no LDP support). -- Łukasz Bromirski CCIE R&S/SP #15929, CCDE #2012::17, PGP Key ID: 0xFD077F6A
Łukasz, Out of all those Cisco models, which meets the OP requirements of " (at least 24) SFP+ ports 10G and at least a couple of upstream ports 40G capable" and a " The budget is around 3000-5000 $ each, possibly. "? The Nexus 7000's look very large with the smallest being 3U in size, so I doubt they would meet the budget requirement. The Nexus 9000 series seems to have 1U versions. Assuming he is fine with using Segmented Routing instead of LDP, any models that fit the bill price wise? Are there any Nexus products that are lower cost that the Catalyst (C3650/3850) and newer (C9xxx)? The Catalyst UADP based platforms seem nice, but most are requiring DNA licensing driving up initial cost. On Thu, Apr 19, 2018 at 11:50 AM, Łukasz Bromirski <lukasz@bromirski.net> wrote:
Colton,
On 19 Apr 2018, at 03:32, Colton Conor <colton.conor@gmail.com> wrote:
Cisco has mutliple options, but mainly the NCS based on your port count I think. Supposely the C3850 and C9500 now support MPLS? There is a new 16 port 10G version of the C9500. I haven't looked into Nexus switches. Does Nexus support full MPLS?
UADP based platforms, both older (C3650/3850) and newer (C9xxx) do support MPLS encap and VXLAN encap and can be extended in future to support others. There are new 9xxx based off UADP 3.0 with 40G and 100G ports: https://www.cisco.com/c/en/us/products/collateral/switches/c atalyst-9500-series-switches/datasheet-c78-738978.html
Nexus 7k supports MPLS with LDP while Nexus 9k supports MPLS but with SR (IGP) or BGP-LU (no LDP support).
-- Łukasz Bromirski CCIE R&S/SP #15929, CCDE #2012::17, PGP Key ID: 0xFD077F6A
participants (11)
-
Aaron Gould -
Baldur Norddahl -
Ben Bartsch -
Brandon Martin -
Colton Conor -
Eric Litvin -
Giuseppe Spanò - Datacast Srl -
Hunter Fuller -
Luke Guillory -
Michael Crapse -
Łukasz Bromirski