On 10/31/18 4:56 PM, Aaron wrote:

It won't hold a full table. 256,000 IPv4 and 64,000 IPv6 routes.

That was changed earlier this year AFAIK. The website was slow to get updated but has been updated now. Current claim is 1.5M IPv4 and 140k IPv6. You need the "advanced feature license" to get access to that. -- Brandon Martin

If you buy brocade, be sure to also by a license for securecrt so that backspace works over ssh... also, just don't do brocade... ever. On Thu, Nov 1, 2018 at 9:31 AM Ryan Hamel <Ryan.Hamel@quadranet.com> wrote:

140K IPv6 equates to about 560K IPv4 routes, leaving the end user with 940K IPv4, which is not a lot of ceiling space considering we're at 741K IPv4 + and 60K IPv6 (240k IPv4 equivalent) now (941K total). This will leave you with 559K. I am not sure what the OP has for peering but with trying to keep 20% of TCAM space free, and keeping up with the current rate of rise according to CIDR-report, I'd say 4 years product lifetime if the OS has excellent TCAM management.

Considering how the device looks like a switch and the SLX9850 uses Broadcom sillicon, I'm thinking it must use the Jericho chipset or some variant to get that kind of performance. In the end, your mileage may vary.

-- Ryan Hamel Network Engineer ryan.hamel@quadranet.com | +1 (888) 578-2372 x201 QuadraNet Enterprises, LLC. | Dedicated Servers, Colocation, Cloud

-----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Brandon Martin Sent: Wednesday, October 31, 2018 3:08 PM To: nanog@nanog.org Subject: Re: Brocade SLX Internet Edge

On 10/31/18 4:56 PM, Aaron wrote:

...

It won't hold a full table. 256,000 IPv4 and 64,000 IPv6 routes.

That was changed earlier this year AFAIK. The website was slow to get updated but has been updated now. Current claim is 1.5M IPv4 and 140k IPv6. You need the "advanced feature license" to get access to that. -- Brandon Martin

On 10/31/18 6:37 PM, Christopher Morrow wrote:

If you buy brocade, be sure to also by a license for securecrt so that backspace works over ssh... also, just don't do brocade... ever.

Works fine for me using OpenSSH in most Linux-y terminal emulators (Konsole, Linux console, Gnome terminal). I didn't do any special configuration. Now, over serial, enjoy your ctrl-H unless you do some remapping. I've never had any real problems with the hardware. The software can leave something to be desired especially on the old Foundry stuff that can't run the modern software, but if you just want it to push packets all day long, they seem to be pretty stable. Only bug I've been bitten with recently is apparent CAM corruption when manipulating large ACLs, but that was on the old (EOL) FCX platform. It's stable as long as you don't CHANGE things, and networks never change, right? (/s) Netiron seems to be more stable but definitely lacking control plane features, especially for MPLS, including some major ones that I gather the big C and J have had available for quite a while. I'm curious how things will diverge now that the "switching" line is at Ruckus/Arris while the "routing" line is at Extreme. I can't say I was ever a fan of Extreme's software, either, and I don't really have enough experience with Arris gear to comment. Certainly like any vendor's box, know what you're getting. It's a packet pusher. On a similar/related topic, has anyone used the Juniper MX204? It seems to occupy roughly the same space as the SLX9540. Less bandwidth, but JunOS is presumably more fully featured in terms of Internet-scale stuff one might want. -- Brandon Martin

I’m just going to echo what a few others have been saying. Brocade (now Extreme) have come a long way since the Foundry days; and the SLX isn’t based on the old Netiron code. The platform is a completely different animal. I’ve been a happy Brocade customer for a while now.

On 10/31/18 6:30 PM, Ryan Hamel wrote:

140K IPv6 equates to about 560K IPv4 routes, leaving the end user with 940K IPv4, which is not a lot of ceiling space considering we're at 741K IPv4 + and 60K IPv6 (240k IPv4 equivalent) now (941K total). This will leave you with 559K. I am not sure what the OP has for peering but with trying to keep 20% of TCAM space free, and keeping up with the current rate of rise according to CIDR-report, I'd say 4 years product lifetime if the OS has excellent TCAM management.

I'm actually in the process of spec'ing one of these (if indeed it's appropriate) for a limited full-Internet-routes application and indeed these are the questions I've been asking of my rep. On "classic Netiron" (MLX, etc.) the numbers they often quoted were actually somewhat pessimistic in that they were one of their stock TCAM profiles, and you actually ended up with BOTH the IPv4 and IPv6 route counts simultaneously.

Considering how the device looks like a switch and the SLX9850 uses Broadcom sillicon, I'm thinking it must use the Jericho chipset or some variant to get that kind of performance. In the end, your mileage may vary.

I want to say it's a Qumran. They apparently have a bigger SLX pizzabox in the works that claims 4M IPv4 FIB and some stupid amount of buffering (8GB IIRC?). I know that's a Qumran, but that also seems like a truly huge amount of TCAM, so I dunno if that's with "typical aggregation" or some other shady trick. -- Brandon Martin

I think Extreme is doing the same thing with their Extreme OptiScale™ that Arista is doing with their Arista FlexRoute™ and EOS NetDB™. They are both using Broadcom Jericho /Qurman with extenal TCAM, but still has a hardware limitiation on route table size. Then in software they filer right? Question is who has a better solution Arista or Extreme for this? Also, the question is can any whitebox vendors do the same thing, with the same Broadcom switch you can buy for around $9k new. Another question, could you even consider these with the Juniper MX204 coming in at $20k? On Thu, Nov 1, 2018 at 8:04 AM Kevin Burke <kburke@burlingtontelecom.com> wrote:

Thanks for everyone who responded on and off list.

As a small company that is happy to still be in business the pricing is too good to ignore. A “gently used” ASR-9006 is something like $45k for one plus a shelf spare. A brand new SLX 9540 is something like $30k for one plus a shelf spare.

There were some common things. Software is behind where we would like. The occasional bug like that SSH one. Also there are some relatively common features like IPv6 outbound ACL and BGP MED that aren’t there. This stuff isn't a showstopper but I will take this a sign of things to come.

As for the notes about full tables. Different vendors seem to have used different techniques to get past the hard FIB limit that we are all used to. I had the same question when pawing through the spec sheets. So I asked the sales rep:

“We can support 1.5M routes…..

*These platforms support all of the requirements detailed above for Internet routing. In particular, they support a table size of 1.5 million IP routes today, ensuring headroom for the next 5-7 years. This scale is made possible through our new technology called Extreme OptiScale™ for Internet Routing that optimizes programmable hardware and software capabilities to accelerate innovation and deliver investment protection.*

https://www.extremenetworks.com/extreme-networks-blog/internet-routing-in-th... <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.extremenetworks.com_extreme-2Dnetworks-2Dblog_internet-2Drouting-2Din-2Dthe-2Denterprise_&d=DwMGaQ&c=H-_4ND6xHpqgzcyjh8aroNpgOyofdYsHrgIgJ94I47w&r=8KBJJnyFAhW7tSYonou-hrzJp0AgKIxIroKtTA850eU&m=KQ3bFBpmJxMvk6-gUGIquclVowK5s1yqe6K3DhY5MZw&s=3rfpGQzZ3dJx3KZvPnF21DuFdz569Krg0zdmo1W_Cfg&e=> ”

Kevin Burke

802-540-0979

Burlington Telecom - City of Burlington

200 Church St, Burlington, VT 05401

-----Original Message----- From: NANOG <nanog-bounces@nanog.org> On Behalf Of Kevin Burke Sent: Wednesday, October 31, 2018 4:02 PM To: nanog@nanog.org Subject: Brocade SLX Internet Edge

Does anyone have any success with the Brocade SLX 9540 or similar? Its going to be taking full BGP tables from two Tier1's and some peering.

The specs and sales rep says its fine, but the price makes me think its too good to be true.

We are trying to shepherd an old Cat 6509 out of our core.

Kevin Burke

802-540-0979

Burlington Telecom - City of Burlington

200 Church St, Burlington, VT 05401

Nicolas Fevrier has a very detailed blog post on how Cisco handles the prefixes on their Broadcom Jericho based NCS 5500 gear. https://xrdocs.io/cloud-scale-networking/tutorials/2017-08-03-understanding-... I'm pretty sure the principle is more or less the same for the Jericho based platforms on Arista and Extreme. Best regards, Chris On 01.11.18 15:31, Saku Ytti wrote:

Hey,

They all do in principle the same thing. There are memories for longest path lookup and memories for exact lookup. I believe the trick is to put specific prefix size, like /24 to exact lookup table, relieving the LPM table stress greatly. Then in parallel ask both, and take more specific result.

There are variation to this, like having multiple separate exact match tables, and populating each with different prefix size, and so forth.

Juniper on PTX is doing something quite different, they are asking on-chip bloom filter about hint on where to query, reducing query count they need to do towards high latency off-chip memory.

MX is doing yet something different, having JNPR proprietary memory ASIC (no longer plain (RL)DRAM).

ASR9k is still just TCAM (for all ezchip generations, unsure about lightspeed). On Thu, 1 Nov 2018 at 15:26, Colton Conor <colton.conor@gmail.com> wrote:

...

I think Extreme is doing the same thing with their Extreme OptiScale™ that Arista is doing with their Arista FlexRoute™ and EOS NetDB™. They are both using Broadcom Jericho /Qurman with extenal TCAM, but still has a hardware limitiation on route table size. Then in software they filer right?

Question is who has a better solution Arista or Extreme for this?

Also, the question is can any whitebox vendors do the same thing, with the same Broadcom switch you can buy for around $9k new.

Another question, could you even consider these with the Juniper MX204 coming in at $20k?

On Thu, Nov 1, 2018 at 8:04 AM Kevin Burke <kburke@burlingtontelecom.com> wrote:

...

Thanks for everyone who responded on and off list.

As a small company that is happy to still be in business the pricing is too good to ignore. A “gently used” ASR-9006 is something like $45k for one plus a shelf spare. A brand new SLX 9540 is something like $30k for one plus a shelf spare.

There were some common things. Software is behind where we would like. The occasional bug like that SSH one. Also there are some relatively common features like IPv6 outbound ACL and BGP MED that aren’t there. This stuff isn't a showstopper but I will take this a sign of things to come.

As for the notes about full tables. Different vendors seem to have used different techniques to get past the hard FIB limit that we are all used to. I had the same question when pawing through the spec sheets. So I asked the sales rep:

“We can support 1.5M routes…..

These platforms support all of the requirements detailed above for Internet routing. In particular, they support a table size of 1.5 million IP routes today, ensuring headroom for the next 5-7 years. This scale is made possible through our new technology called Extreme OptiScale™ for Internet Routing that optimizes programmable hardware and software capabilities to accelerate innovation and deliver investment protection.

https://www.extremenetworks.com/extreme-networks-blog/internet-routing-in-the-enterprise/”

Kevin Burke

802-540-0979

Burlington Telecom - City of Burlington

200 Church St, Burlington, VT 05401

-----Original Message----- From: NANOG <nanog-bounces@nanog.org> On Behalf Of Kevin Burke Sent: Wednesday, October 31, 2018 4:02 PM To: nanog@nanog.org Subject: Brocade SLX Internet Edge

Does anyone have any success with the Brocade SLX 9540 or similar? Its going to be taking full BGP tables from two Tier1's and some peering.

The specs and sales rep says its fine, but the price makes me think its too good to be true.

We are trying to shepherd an old Cat 6509 out of our core.

Kevin Burke

802-540-0979

Burlington Telecom - City of Burlington

200 Church St, Burlington, VT 05401

I have no horse in this race, however one need only look at the NYIIX outages list to see how well the Brocade/Extreme SLX platform works on at-scale service provider networks... On Thu, Nov 1, 2018 at 2:55 PM Blake Hudson <blake@ispn.net> wrote:

Chris Welti wrote on 11/1/2018 10:03 AM:

...

Nicolas Fevrier has a very detailed blog post on how Cisco handles the prefixes on their Broadcom Jericho based NCS 5500 gear. https://xrdocs.io/cloud-scale-networking/tutorials/2017-08-03-understanding-...

I'm pretty sure the principle is more or less the same for the Jericho based platforms on Arista and Extreme.

Best regards, Chris

I love the nitty gritty detail in this author's post and I'm glad he concludes by stating clearly that while the base card (spec sheet says: "On-chip tables for 256K IPv4 or 64K IPv6 routes" and "On-chip tables for 786K IPv4 host routes, MAC, and labels") can actually hold a full BGP table today when configured appropriately, Cisco still recommends the scale cards for that application (spec sheet says: "FIB scale up 2M IPv4 or 512K IPv6 routes" and "On-chip tables for 786K IPv4 host routes, MAC, and labels").

I do have to wonder about the internal expansion of each /23 route into two /24 routes in their FIB algorithm, as I would have thought Cisco would have attempted to go the opposite way, but I'm sure Cisco has their reasons.