Hi, Im looking to deploy more mpls in my network. I like the Cisco 3600X series but the low density of 10g ports has me wanting to consider perhaps others. I would love a minimum of 4 10g ports but of course more is better. Cost would also be a factor. What are people using these days? Thanks. Mike-
Mike, Nor sure how much you know about SDN or if you are in anywhere near being able to make the transition, but white-label switches may be a deciding factor for you. In fact you may be able to do it without SDN, but I cannot say for sure as we have ours configured in SDN mode. We use Edge-Core 5712-54X switches for our 10G (48x 10G and 6x 40G) and I cannot recommend them enough. Combined with Picos from Pica8, we can do SDN/SD-WAN as well as MPLS at around 6,000 per switch plus maintenance. If you want more information or contacts, hit me up offline (or rather off the mailing list). -Todd
On Apr 6, 2016, at 5:02 PM, Mike <mike-nanog@tiedyenetworks.com> wrote:
Hi,
Im looking to deploy more mpls in my network. I like the Cisco 3600X series but the low density of 10g ports has me wanting to consider perhaps others. I would love a minimum of 4 10g ports but of course more is better. Cost would also be a factor. What are people using these days?
Thanks.
Mike-
I just subscribed to NANOG and I already learned something new. I did not know about WhiteBox switches. I like how the price tag is plainly visible to all. This "contact a salesperson" requirement gets old very quickly. On Wed, 6 Apr 2016 22:12:37 -0400 David Bass <davidbass570@gmail.com> wrote:
Interesting. What SDN controller are you using?
Seems like quite a few are moving to white box switches...
On Apr 6, 2016, at 9:53 PM, Todd Crane <todd.crane@n5tech.com> wrote:
Edge-Core 5712-54X
ZTE 5950 serie MPLS routing switches are about 1500 to 3000 USD depending on configuration. They have a 4x 10G subcard. The CLI is very Cisco like. The ZTE 5960 has 32 or 64 x 10G but starts at 5k-6k. Regards Baldur Den 7. apr. 2016 02.05 skrev "Mike" <mike-nanog@tiedyenetworks.com>:
Hi,
Im looking to deploy more mpls in my network. I like the Cisco 3600X series but the low density of 10g ports has me wanting to consider perhaps others. I would love a minimum of 4 10g ports but of course more is better. Cost would also be a factor. What are people using these days?
Thanks.
Mike-
On 7/Apr/16 02:02, Mike wrote:
Hi,
Im looking to deploy more mpls in my network. I like the Cisco 3600X series but the low density of 10g ports has me wanting to consider perhaps others. I would love a minimum of 4 10g ports but of course more is better. Cost would also be a factor. What are people using these days?
Cisco ASR920. 4x 10Gbps uplink ports + 24x customer-facing ports. All IP/MPLS capable. Mark.
Dear Mr. Mike, I would recommend HP A5500-HI. It's a very capable L3 routing switch (12k FIB) aswell as MPLS forwarder - both P and PE. It has two 10 GbE SFP+ ports and is expandable to a total of six if you add two modules. The price is about 2000 dollars (new) and you can stack them aswell using regular M/SMF. It's also got 24 one gig SFP ports in addition to four one gig RJ45 copper ports. Best regards, Jonas Bjork Senior network engineer Sent from my iPhone
On 7 apr. 2016, at 02:02, Mike <mike-nanog@tiedyenetworks.com> wrote:
Hi,
Im looking to deploy more mpls in my network. I like the Cisco 3600X series but the low density of 10g ports has me wanting to consider perhaps others. I would love a minimum of 4 10g ports but of course more is better. Cost would also be a factor. What are people using these days?
Thanks.
Mike-
Do the Juniper EX switches support MPLS? I know they have models with multiple 10G ports on them. There is also the QFX series. On Wed, Apr 6, 2016 at 7:02 PM, Mike <mike-nanog@tiedyenetworks.com> wrote:
Hi,
Im looking to deploy more mpls in my network. I like the Cisco 3600X series but the low density of 10g ports has me wanting to consider perhaps others. I would love a minimum of 4 10g ports but of course more is better. Cost would also be a factor. What are people using these days?
Thanks.
Mike-
I know the 4500/4550 does but it requires a license. On Apr 12, 2016 8:07 AM, "Colton Conor" <colton.conor@gmail.com> wrote:
Do the Juniper EX switches support MPLS? I know they have models with multiple 10G ports on them. There is also the QFX series.
On Wed, Apr 6, 2016 at 7:02 PM, Mike <mike-nanog@tiedyenetworks.com> wrote:
Hi,
Im looking to deploy more mpls in my network. I like the Cisco 3600X series but the low density of 10g ports has me wanting to consider perhaps others. I would love a minimum of 4 10g ports but of course more is better. Cost would also be a factor. What are people using these days?
Thanks.
Mike-
On 12/Apr/16 15:04, Colton Conor wrote:
Do the Juniper EX switches support MPLS? I know they have models with multiple 10G ports on them.
They do, but (deliberately) broken. I wouldn't try it.
There is also the QFX series.
Not that I know of, but the ACX is a QFX-derivative (Broadcom chipset, approach with caution). Mark.
Do the Juniper EX switches support MPLS? I know they have models with multiple 10G ports on them.
They do, but (deliberately) broken. I wouldn't try it.
EX4600 does MPLS just fine, nothing else really does in the EX series.. EX4200 can do 1 label. The EX4600 featureset is pretty much the same as QFX5100 in addition to supporting MACSEC.
There is also the QFX series.
Not that I know of, but the ACX is a QFX-derivative (Broadcom chipset, approach with caution).
QFX5100 works fine for MPLS.. ACX5k is QFX5100 hardware, but a different train of software, and it's a bit different. QFX5100 is a great P and lightweight PE.. -- Tim
On 12/Apr/16 15:22, Tim Jackson wrote:
QFX5100 works fine for MPLS.. ACX5k is QFX5100 hardware, but a different train of software, and it's a bit different. QFX5100 is a great P and lightweight PE..
As a P, fine (except if you're doing NG-MVPN, of course, which would make it a poor branch router). The "lightweight PE" is where my concern comes in. And if the EX4600 is the same as the QFX in this regard, same problem, i.e., if the OP is expecting all PE functionality he'd get on an MX in this unit, he needs to reset his expectations. Mark.
On 4/12/16, 9:22 AM, "NANOG on behalf of Tim Jackson" <nanog-bounces@nanog.org on behalf of jackson.tim@gmail.com> wrote:
(Broadcom chipset, approach with caution).
QFX5100 works fine for MPLS.. [snip] QFX5100 is a great P and lightweight PE..
WG] For some values of "fine" and "great" perhaps, but emphasis on the "lightweight" is important, as its suitability is heavily dependent on your intended use case. Use it with a few thousand routes and nothing particularly exotic as far as features go and you should be fine. However, there are sometimes little gotchas where established features (esp in MPLS) either are missing or behave differently in subtle ways compared with more traditional JunOS routers like the MX. Some of these are limitations in the Broadcom chipset and some are driven by customer demand prioritizing feature completion. Test carefully, and regard the higher-end multidimensional/route scale numbers with healthy skepticism. Wes George Anything below this line has been added by my company’s mail server, I have no control over it. ----------- ________________________________ This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.
Someone told me to check out extreme networks, cisco or Ciena for the more cost effective mpls kit. Any advice on which of the three would have the most cost effective 10G MPLS switch? Cisco's MPLS switch is the ASR 920 right? On Tue, Apr 12, 2016 at 4:13 PM, George, Wes <wesley.george@twcable.com> wrote:
On 4/12/16, 9:22 AM, "NANOG on behalf of Tim Jackson" <nanog-bounces@nanog.org on behalf of jackson.tim@gmail.com> wrote:
(Broadcom chipset, approach with caution).
QFX5100 works fine for MPLS.. [snip] QFX5100 is a great P and lightweight PE..
WG] For some values of "fine" and "great" perhaps, but emphasis on the "lightweight" is important, as its suitability is heavily dependent on your intended use case. Use it with a few thousand routes and nothing particularly exotic as far as features go and you should be fine. However, there are sometimes little gotchas where established features (esp in MPLS) either are missing or behave differently in subtle ways compared with more traditional JunOS routers like the MX. Some of these are limitations in the Broadcom chipset and some are driven by customer demand prioritizing feature completion.
Test carefully, and regard the higher-end multidimensional/route scale numbers with healthy skepticism.
Wes George
Anything below this line has been added by my company’s mail server, I have no control over it. -----------
________________________________
This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.
On 13/Apr/16 02:29, Colton Conor wrote:
Someone told me to check out extreme networks, cisco or Ciena for the more cost effective mpls kit. Any advice on which of the three would have the most cost effective 10G MPLS switch?
Cisco's MPLS switch is the ASR 920 right?
The useful ones are the ASR920 and ME3600X/3800X. The ASR920 is the way forward, and is generally half the price of the ME3600X. Mark.
How does the ASR 903 compare to the 920? When we got pricing for the ASR 903 it was more expensive than a real ASR 9k router. On Tue, Apr 12, 2016 at 11:49 PM, Mark Tinka <mark.tinka@seacom.mu> wrote:
On 13/Apr/16 02:29, Colton Conor wrote:
Someone told me to check out extreme networks, cisco or Ciena for the more cost effective mpls kit. Any advice on which of the three would have the most cost effective 10G MPLS switch?
Cisco's MPLS switch is the ASR 920 right?
The useful ones are the ASR920 and ME3600X/3800X.
The ASR920 is the way forward, and is generally half the price of the ME3600X.
Mark.
On 13/Apr/16 20:30, Colton Conor wrote:
How does the ASR 903 compare to the 920? When we got pricing for the ASR 903 it was more expensive than a real ASR 9k router.
Feature-wise, it's more mature than the ASR920, as it came before. Personally, I find it more of a device where you need a mix-and-match, e.g., at a RAN site. Not my kind of thing; I focus purely on Ethernet in a small form factor, which the ASR920 does very well. But I'd move this query to c-nsp. There are a bunch of good folk there that use the ASR903 and can speak more authoritatively about it than I can. Mark.
While following that Arista chat... That reminded me of that little afternoon project years ago. So I decided to find new hamsters, fire up that VM, refresh the DB's and from the view point of a tiny 7206VXR/G1 with 2 T3 peers... The amount of superfluous subnet advertisement drop to ~120k from ~166k from the previous snapshot. And this is the distribution by country. country | superfluous --------------------+------------- United States | 28254 Brazil | 10012 China | 7537 India | 6449 Russian Federation | 4524 Korea, Republic of | 4062 Saudi Arabia | 3297 Australia | 2989 Indonesia | 2878 Hong Kong | 2251 Thailand | 2093 Canada | 2019 Taiwan | 1955 Ukraine | 1877 Singapore | 1856 Bulgaria | 1488 Argentina | 1436 Japan | 1403 Mexico | 1351 Chile | 1271 (Damn Canada, can't break top 10 again). PS: "Superfluous" is a nice way to say that the best path of a subnet is the same as his supernet. And yes I'm aware of the Weekly Routing Report, I was just curious to see it by country =D. ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 04/13/16 15:17, Mark Tinka wrote:
On 13/Apr/16 20:30, Colton Conor wrote:
How does the ASR 903 compare to the 920? When we got pricing for the ASR 903 it was more expensive than a real ASR 9k router. Feature-wise, it's more mature than the ASR920, as it came before.
Personally, I find it more of a device where you need a mix-and-match, e.g., at a RAN site. Not my kind of thing; I focus purely on Ethernet in a small form factor, which the ASR920 does very well.
But I'd move this query to c-nsp. There are a bunch of good folk there that use the ASR903 and can speak more authoritatively about it than I can.
Mark.
Alain Hebert wrote:
PS: "Superfluous" is a nice way to say that the best path of a subnet is the same as his supernet.
... from the point of view of the paths that you see, which is to say two egress paths. Someone else on the internet may have a different set of bgp views which will give a different set of results for the bgp decision process. The more paths you receive from different sources, the more likely it is that this list of 120k "superfluous" prefixes will converge towards zero. You're right that it's often not necessary to accept all paths, and your fib view can optimised in a way that your rib shouldn't be. All these things can be used to drop the forwarding lookup engine resource requirements, although it is important to understand that there is no such thing as a free lunch and if you do this, there might well be edge cases which could cause your optimisation to fail and things to blow up horribly in your face. Still, it's an interesting thing to examine. Nick
On 2016-04-29 12:48, Nick Hilliard wrote:
Alain Hebert wrote:
PS: "Superfluous" is a nice way to say that the best path of a subnet is the same as his supernet.
... from the point of view of the paths that you see, which is to say two egress paths. Someone else on the internet may have a different set of bgp views which will give a different set of results for the bgp decision process. The more paths you receive from different sources, the more likely it is that this list of 120k "superfluous" prefixes will converge towards zero.
You're right that it's often not necessary to accept all paths, and your fib view can optimised in a way that your rib shouldn't be. All these things can be used to drop the forwarding lookup engine resource requirements, although it is important to understand that there is no such thing as a free lunch and if you do this, there might well be edge cases which could cause your optimisation to fail and things to blow up horribly in your face. Still, it's an interesting thing to examine.
Nick
What Nick said is basically what I was asking about in the Arista thread. Are there new edge cases and new failure modes that are introduced by this strategy? It seems like you'd have to recompute the minimal set of forwarding rules each time a prefix is added or removed, and a single update may cause you to have to do many adds/removes to bring your compressed rules into sync, like when a hole is punched in an aggregated prefix. I'm curious about specific failure modes that can result from this, if anyone can share examples/experience with it. Thanks, Laszlo
Laszlo Hanyecz wrote:
I'm curious about specific failure modes that can result from this, if anyone can share examples/experience with it.
The canonical pathological case is where the deaggregated prefixes are affected by upstream topology changes and suddenly your optimisations which saved you N% of forwarding lookup table capacity are wiped out to zero and you end up with no ability to look up next-hops. Nick
Just to be clear, this isn't (to my knowledge) something that Arista is doing and so the risk described doesn't affect the products that were discussed on that thread. On Fri, Apr 29, 2016 at 9:30 AM, Nick Hilliard <nick@foobar.org> wrote:
Laszlo Hanyecz wrote:
I'm curious about specific failure modes that can result from this, if anyone can share examples/experience with it.
The canonical pathological case is where the deaggregated prefixes are affected by upstream topology changes and suddenly your optimisations which saved you N% of forwarding lookup table capacity are wiped out to zero and you end up with no ability to look up next-hops.
Nick
Den 29. apr. 2016 15.31 skrev "Nick Hilliard" <nick@foobar.org>:
Laszlo Hanyecz wrote:
I'm curious about specific failure modes that can result from this, if anyone can share examples/experience with it.
The canonical pathological case is where the deaggregated prefixes are affected by upstream topology changes and suddenly your optimisations which saved you N% of forwarding lookup table capacity are wiped out to zero and you end up with no ability to look up next-hops.
With two uplinks that is highly unlikely to the point of being impossible. There is no topology change upstream that can cause a situation where it is not possible to do a high degree of aggregation of the full default free routing table before loading it in the FIB. Regards Baldur
Baldur Norddahl wrote:
With two uplinks that is highly unlikely to the point of being impossible. There is no topology change upstream that can cause a situation where it is not possible to do a high degree of aggregation of the full default free routing table before loading it in the FIB.
which is why I qualified this in a previous posting:
The more paths you receive from different sources, the more likely it is that this list of 120k "superfluous" prefixes will converge towards zero.
Agreed that small numbers of paths are most unlikely to create the conditions for this problem to occur. Nick
On 29 April 2016 at 13:25, Nick Hilliard <nick@foobar.org> wrote:
The more paths you receive from different sources, the more likely it is that this list of 120k "superfluous" prefixes will converge towards zero.
Agreed that small numbers of paths are most unlikely to create the conditions for this problem to occur.
If these compression schemes are implemented, and our compressed count is near the limit of hardware, it creates interesting new attack vector for attackers. Pump carefully crafted updated to global table and watch networks melt. I think compression makes more sense in controlled environments, but controlled environments with large scale are likely to be exact matches (i.e. bunch of host routes) not LPM anyhow. I'm not optimistic about the technology. -- ++ytti
On 29 April 2016 at 22:25, Nick Hilliard <nick@foobar.org> wrote:
Baldur Norddahl wrote:
With two uplinks that is highly unlikely to the point of being impossible. There is no topology change upstream that can cause a situation where it is not possible to do a high degree of aggregation of the full default free routing table before loading it in the FIB.
which is why I qualified this in a previous posting:
The more paths you receive from different sources, the more likely it is that this list of 120k "superfluous" prefixes will converge towards zero.
Agreed that small numbers of paths are most unlikely to create the conditions for this problem to occur.
I agree that a larger number of peers makes the situation more complicated. It might warrant more studies. Your thesis is that there might be a problem, but mine is there likely is not. Let me argue why. We can consider networks of various sizes: 1) the dual homed network with full tables 2) the lightly peered ISP with more than two full tables 3) the well peered ISP 4) tier 1 backbone provider Each of those might experience different gain from the proposal and indeed it is likely that the backbone provider would not be interested in the solution no matter what. Even so the proposal could help deliver considerable cheaper hardware solutions to say #1 and #2 class providers. We already agree that the #1 class provider will not see an external event that can explode the number of needed FIB entries after compression. The #2 class provider is not much different. The number of routes he takes in as peering routes as opposed to transit are few. If he runs his network with proper max routes on every BGP session, there is nothing a free peer can do to wreck havok. Any entity with say max routes 50 can only break up a max of 50 of your optimized FIB entries and while that can cascade such a /16 breaks into a series of /17, /18, /19, ..., /24 that will never add up to anything that is a problem. In any case the real problem here will be a rogue peer injecting fake routes into your network. Can the more than two transit providers with full tables become a problem? No not really. These guys are all sending mostly the same routes to you and anything large happening will be reflected on all your transits. There is also the point about the weekly routing report: BGP routing table entries examined: 593320 Prefixes after maximum aggregation (per Origin AS): 217357 Deaggregation factor: 2.73 Unique aggregates announced (without unneeded subnets): 290159 Now can you really say any one entity has the power to magically make all that aggregation disappear just so he can crash your network? I will put that in the "impossible" and "the net already crashed long before that" categories. There is a trend that some network are deaggregating their prefixes. Why not use software to aggregate that right back to what it ought to be before loading the routes into FIB? According to the above stat, that would save at least half the FIB memory and make some routers able to handle full tables for very much longer (possible forever). Regards, Baldur
On Tue Apr 12, 2016 at 07:29:54PM -0500, Colton Conor wrote:
Someone told me to check out extreme networks, cisco or Ciena for the more cost effective mpls kit. Any advice on which of the three would have the most cost effective 10G MPLS switch?
I'm using Extreme switches for VPLS - the X460 will give you up to 6 x 10G ports, and the X670 will give you 48 x 10G ports (and 4 x 40G ports). I've not tried them as P nodes (we use Cisco for that), or for any other MPLS features (L3VPN), but for VPLS they're working well for us. When we started using them, they were significantly cheaper than Cisco alternatives. Simon
Hi, L2VPN works also pretty well with the Extremes (X670). Only one thing doesn't work: LACP BPDU forwarding for the customer. This is caused by the method how Extreme let you configure the L2VPN on those small boxes. best regards Jürgen Jaritsch Head of Network & Infrastructure ANEXIA Internetdienstleistungs GmbH Telefon: +43-5-0556-300 Telefax: +43-5-0556-500 E-Mail: JJaritsch@anexia-it.com Web: http://www.anexia-it.com Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601 -----Ursprüngliche Nachricht----- Von: NANOG [mailto:nanog-bounces@nanog.org] Im Auftrag von Simon Lockhart Gesendet: Mittwoch, 13. April 2016 08:23 An: Colton Conor Cc: nanog@nanog.org Betreff: Re: mpls switches On Tue Apr 12, 2016 at 07:29:54PM -0500, Colton Conor wrote:
Someone told me to check out extreme networks, cisco or Ciena for the more cost effective mpls kit. Any advice on which of the three would have the most cost effective 10G MPLS switch?
I'm using Extreme switches for VPLS - the X460 will give you up to 6 x 10G ports, and the X670 will give you 48 x 10G ports (and 4 x 40G ports). I've not tried them as P nodes (we use Cisco for that), or for any other MPLS features (L3VPN), but for VPLS they're working well for us. When we started using them, they were significantly cheaper than Cisco alternatives. Simon
On Tuesday, 12 April, 2016 14:04, "Colton Conor" <colton.conor@gmail.com> said:
Do the Juniper EX switches support MPLS? I know they have models with multiple 10G ports on them. There is also the QFX series.
The EXes can also run in a "fabric extender" mode to the MX (and others?). Depending on geographical footprint and requirements, this might be worth a look. Regards, Tim.
participants (19)
-
Alain Hebert -
Baldur Norddahl -
Colton Conor -
Coyo Stormcaller -
David Bass -
George, Wes -
Jonas Bjork -
Josh Reynolds -
Jürgen Jaritsch -
Laszlo Hanyecz -
Mark Tinka -
Mike -
Nick Hilliard -
Ryan Woolley -
Saku Ytti -
Simon Lockhart -
Tim Jackson -
tim＠pelican.org -
Todd Crane