It has been an interesting discussion. Always willing to see what others are doing in this space and evaluate it to what we're doing / thinking about going forward. We were just quoted 250k+ for a Cisco ASR9902 with one route processor card (list. not our price). There can be 2 rps, but they don't talk to each other -- who thought that was a good idea. We were just looking for something that could take a couple of full tables and had 2 - 4 100g connections. We're still using ASR920s (12 10g ports, rather uncommon) at most of our pops but have seen the writing on the wall that 10g will not be enough at some point and 100g will be necessary. I did have an interesting conversation with Ribbon (we have a C15 phone switch) about their Neptune platform. Surprisingly affordable when you look at what Cisco is charging. Though they didn't have as many 100g interfaces as I'd like -- can't see using them as a BGP speaking router, but for internal transport stuff, it was definitely attractive. Again, always nice to see what others are using / considering for similar stuff. Too often all we hear about are the 'really big guys' and how they're deploying X for 400g now, etc. Shawn -----Original Message----- From: "Mike Hammett" <nanog@ics-il.net> Sent: Tuesday, December 24, 2024 2:38pm To: "Saku Ytti" <saku@ytti.fi> Cc: "NANOG" <nanog@nanog.org> Subject: Re: Distributed Router Fabrics "what benefits is OP seeing here when it comes to pizzabox"I'm more learning and questioning than stating. I've thoroughly enjoyed the thread. One of the main advantages I saw from the outset was that I could start with a single box and then grow if needed. Other than recabling, if not planned for accordingly, it seems like I can still do that. You would have an increased cost once you had to add a fabric box, but you've already had some amount of scale to get there. With a chassis system, you have the larger cost up front before you even know how you're going to scale. It's more difficult to plan what sized solution and no matter what you do, you'll probably pick the wrong one. -----Mike Hammett[ Intelligent Computing Solutions ]( http://www.ics-il.com/ )[ ]( https://www.facebook.com/ICSIL )[ ]( https://plus.google.com/+IntelligentComputingSolutionsDeKalb )[ ]( https://www.linkedin.com/company/intelligent-computing-solutions )[ ]( https://twitter.com/ICSIL )[ Midwest Internet Exchange ]( http://www.midwest-ix.com/ )[ ]( https://www.facebook.com/mdwestix )[ ]( https://www.linkedin.com/company/midwest-internet-exchange )[ ]( https://twitter.com/mdwestix )[ The Brothers WISP ]( http://www.thebrotherswisp.com/ )[ ]( https://www.facebook.com/thebrotherswisp )[ ]( https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg ) From: "Saku Ytti" <saku@ytti.fi> To: "Tom Beecher" <beecher@beecher.cc> Cc: "NANOG" <nanog@nanog.org> Sent: Tuesday, December 24, 2024 9:28:06 AM Subject: Re: Distributed Router Fabrics On Tue, 24 Dec 2024 at 17:22, Tom Beecher <beecher@beecher.cc> wrote:
It's possible I s/chip/ in my head with a different meaning than you intended, and I am answering a different question.
I generally won't put all LAG members on the same ASIC, or even same linecard, for failure domain reasons. I also don't really care about possible challenges with BFD there, because I just use micro-BFD on members + min-links.
Quite, it depends what is important for your case. You may want to put all in one chip for better feature parity in terms of QoS, counters et.al., especially if you want them to fail as one, because you're doing it purely for capacity, not for redundancy. And indeed without uBFD, you're going to run LACP over one interface in one chip at most, anyhow, and with uBFD each member are going to run their own, anyhow. So I wonder, what benefits is OP seeing here when it comes to pizzabox? To me pizzzabox seems identical here to chassis box with LACP spanning only single chip. -- ++ytti