BGP topological vs centralized route reflector
Dears Am trying to find some documents and practical implementations regarding bgp topological vs centralized route reflector(In-band vs out-of-band) Any good shares are appreciated
Hey, in-band, out-of-band is bit of misnomers to me. You mean in-path or out-of-path. Main advantage of out-of-path is that you decouple FIB and RIB scaling requirements and feature requirements. Your backbone device does not need to be qualified for large RIB or BGP at all. And when you do need more RIB scaling, you can upgrade out-of-path without any network interruption. Main advantage of in-path is maturity and simplicity, you don't need ORR and you might not need AddPath. On Wed, Feb 13, 2019 at 7:37 PM Mohammad Khalil <eng.mssk@gmail.com> wrote:
Dears Am trying to find some documents and practical implementations regarding bgp topological vs centralized route reflector(In-band vs out-of-band)
Any good shares are appreciated
-- ++ytti
On 13/Feb/19 20:00, Saku Ytti wrote:
Main advantage of out-of-path is that you decouple FIB and RIB scaling requirements and feature requirements. Your backbone device does not need to be qualified for large RIB or BGP at all. And when you do need more RIB scaling, you can upgrade out-of-path without any network interruption.
We've ran this for years (Cisco CSR1000v, since 2014), and our biggest problem has been server hardware failure. Failing fans, sensitivity to higher temperatures that routers can weather better... that sort of thing. Other than that, run this as a VM in your favourite hypervisor and you're good to go. Can't recommend it enough. Mark.
Hi, Unlucky as always, we had issues with the chassis of a MX104 about every years since we installed. I thinking the vibration from the train track above our location might be having an effect on connectors in those chassis, but we never got a "autopsy" report back from JNP about the chassis we swapped. Oddly luck, we have ~40 VM servers in the rack beside it with a mix of mechanical and SSDs drive with 0 issues for the same time span. So mileage may vary. ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 2/14/19 12:15 AM, Mark Tinka wrote:
On 13/Feb/19 20:00, Saku Ytti wrote:
Main advantage of out-of-path is that you decouple FIB and RIB scaling requirements and feature requirements. Your backbone device does not need to be qualified for large RIB or BGP at all. And when you do need more RIB scaling, you can upgrade out-of-path without any network interruption. We've ran this for years (Cisco CSR1000v, since 2014), and our biggest problem has been server hardware failure. Failing fans, sensitivity to higher temperatures that routers can weather better... that sort of thing.
Other than that, run this as a VM in your favourite hypervisor and you're good to go. Can't recommend it enough.
Mark.
On 14/Feb/19 14:04, Alain Hebert wrote:
Hi,
Unlucky as always, we had issues with the chassis of a MX104 about every years since we installed.
Are you using the MX104 as a route reflector? If so, make one of the VM's your alternative for this function :-). If you're not doing any non-Ethernet services on your MX104, and are struggling with the control plane, I'd propose moving to the MX204. Mark.
To not get off-topic too much, since you mentioned MX204, please tell me, do you know if it is a nice MPLS P/PE box ? If so, is it quite capable in its ability to do L3 VPN's, L2 VPN's (l2circuit mainly, but also curious of vpls, evpn). Actually I'm considering it as a router for my ENNI hand-offs to 3rd party neighboring networks where I hand-off vlans (double tagged) for various enterprise customers and cbh towers, etc.... then I would carry that probably in a l2circuit from that MX204 to the utter-most parts of my mpls cloud. I would want to police at that subinterface (unit) level to limit traffic for obviously what they buy. MX204 be good for that ? Thanks Mark -Aaron -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Mark Tinka Sent: Thursday, February 14, 2019 7:09 AM To: nanog@nanog.org Subject: Re: BGP topological vs centralized route reflector On 14/Feb/19 14:04, Alain Hebert wrote:
Hi,
Unlucky as always, we had issues with the chassis of a MX104 about every years since we installed.
Are you using the MX104 as a route reflector? If so, make one of the VM's your alternative for this function :-). If you're not doing any non-Ethernet services on your MX104, and are struggling with the control plane, I'd propose moving to the MX204. Mark.
On 14/Feb/19 17:02, Aaron Gould wrote:
To not get off-topic too much, since you mentioned MX204, please tell me, do you know if it is a nice MPLS P/PE box ? If so, is it quite capable in its ability to do L3 VPN's, L2 VPN's (l2circuit mainly, but also curious of vpls, evpn).
We've started ordering them as our new peering and border routers. However, we shall also be using them as high-capacity Metro-E routers, where customers need 10Gbps hand-off, with a 100Gbps backbone.
Actually I'm considering it as a router for my ENNI hand-offs to 3rd party neighboring networks where I hand-off vlans (double tagged) for various enterprise customers and cbh towers, etc.... then I would carry that probably in a l2circuit from that MX204 to the utter-most parts of my mpls cloud. I would want to police at that subinterface (unit) level to limit traffic for obviously what they buy.
MX204 be good for that ?
I'm sure it will be - it's an MPC7 in a cage :-). Mark.
On Fri, Feb 15, 2019 at 9:55 AM Mark Tinka <mark.tinka@seacom.mu> wrote:
MX204 be good for that ?
I'm sure it will be - it's an MPC7 in a cage :-).
Anyone know why MX204 has so few ports? It seems like it only has WAN side used, leaving FAB side entirely unused, throwing away 50% of free capacity. MX80/MX104 have both sides for revenue ports. I would GLADLY take 50% more ports in MX204, without taking any more PPS or QoS bandwidth. Is this because we as a community are so anal towards vendors about PPS performance that JNPR marketing forbade them making pizza-box MPC7 using all the capacity in fears of people being angry about not being able to do good PPS on all ports? As far as I understand, it would have been zero cost to have double ports in MX204, if you don't want to use them, there is capex efficient vendor-agnostic, single-spare solution[0] to turn any platform back into full PPS platform. I want my free ports, in metro application you are limited by your east+west capacity and you can never see more PPS, but you want to add more edges. [0] http://z.ip.fi/BVLE -- ++ytti
On 15/Feb/19 10:40, Saku Ytti wrote:
Is this because we as a community are so anal towards vendors about PPS performance that JNPR marketing forbade them making pizza-box MPC7 using all the capacity in fears of people being angry about not being able to do good PPS on all ports?
As far as I understand, it would have been zero cost to have double ports in MX204, if you don't want to use them, there is capex efficient vendor-agnostic, single-spare solution[0] to turn any platform back into full PPS platform. I want my free ports, in metro application you are limited by your east+west capacity and you can never see more PPS, but you want to add more edges.
I'm with you - but from what I can imagine, Juniper did not envisage this box being used in high-density Metro-E applications (which I wouldn't mind doing by planting a bunch of customers on 10Gbps that, in an ideal world, would oversubscribe the 100Gbps uplinks, but in real life, won't). If someone from Juniper is reading this thread, I'd take the feedback and have an "MX204-ME" style box designed with more port density on the platform without having to increase pps or the uplink. Mark.
Anyone know why MX204 has so few ports? It seems like it only has WAN side used, leaving FAB side entirely unused, throwing away 50% of free capacity.
The usable port configs are also quite tricky. Juniper have had to make a tool to validate the configurations (https://apps.juniper.net/home/port-checker/). For example, using 4x40G disables all of the 10G ports however using 3x40G and 1x100G gives you all of the 10G ports
MX80/MX104 have both sides for revenue ports.
They are, however, not Trio - rather just commodity CPUs. Routing re-convergence times are shockingly high - in the region of 5-10 minutes for MX80 with a full table vs 30 seconds (ish) for 204
I would GLADLY take 50% more ports in MX204, without taking any more PPS or QoS bandwidth.
You can add switches (EX or QFX) as line cards using Fusion, to add more port density. I've heard some good things about Fusion, though I'm always wary of proprietary clustering technology having been bitten by VC a few times. You can also just trunk some VLANs up to switches if you don't want to buy the Fusion license
On Fri, Feb 15, 2019 at 10:54 AM Phil Lavin <phil.lavin@cloudcall.com> wrote:
MX80/MX104 have both sides for revenue ports.
They are, however, not Trio - rather just commodity CPUs. Routing re-convergence times are shockingly high - in the region of 5-10 minutes for MX80 with a full table vs 30 seconds (ish) for 204
They are normal 1st gen trio boxes, same as MPC1, MPC2, MPC3 originals were. You may be confused about the fact that their control plane is freescale, instead of intel. -- ++ytti
They are normal 1st gen trio boxes, same as MPC1, MPC2, MPC3 originals were. You may be confused about the fact that their control plane is freescale, instead of intel.
Sorry, yes - you're right. Re-convergence times are, however, still awful. Though if you're not handling a lot of routes, that may not be a huge problem for you
On 15/Feb/19 10:54, Phil Lavin wrote:
They are, however, not Trio - rather just commodity CPUs. Routing re-convergence times are shockingly high - in the region of 5-10 minutes for MX80 with a full table vs 30 seconds (ish) for 204
They are Trio. It's the control plane which is not Intel... Freescale.
You can add switches (EX or QFX) as line cards using Fusion, to add more port density. I've heard some good things about Fusion, though I'm always wary of proprietary clustering technology having been bitten by VC a few times. You can also just trunk some VLANs up to switches if you don't want to buy the Fusion license
Nasty, but doable (with or without Fusion). I'd prefer not to, but I'm getting old so my resolve could be questioned :-). Mark.
Saku Ytti Sent: Friday, February 15, 2019 8:41 AM
On Fri, Feb 15, 2019 at 9:55 AM Mark Tinka <mark.tinka@seacom.mu> wrote:
MX204 be good for that ?
I'm sure it will be - it's an MPC7 in a cage :-).
Anyone know why MX204 has so few ports? It seems like it only has WAN side used, leaving FAB side entirely unused, throwing away 50% of free capacity.
I don't think aiming for good PPS is the case. See KB33477, if the "spare" capacity was there to boost the available pps budget for the artificially limited number of ports I don't think there would be any KB33477. Maybe some other architectural challenge, don't know? Also this could be asked of any platform from any vendor, just give us 48x 40/100GE ports for each NPU and we'll figure out what to do with those. Maybe there will be use-cases where I enable all of them as I don't expect much traffic/pps to be generated on each port and maybe there will be cases where I enable just one port as I expect 64b frames @ line-rate and have 100k lines in the filter matching for packet size. You actually reminded me of the A9K-24X10GE vs A9K-36X10GE (yes please, I'll have 36 ports variant and I'll decide what to do with them). adam
I seem to remember there were some good old (really old now) books on BGP discussing various design aspects of BGP infrastructure including topology-based or address-based route reflection, etc... Topology-based doesn’t need to mean in-path you can still have a whole fleet of out-of-the-path RRs servicing say south-east region. My advice is: -Keep your RR-1 infra separate form RR-2 infra (when RR1&RR2=same cluster). -Keep your Internet-RRs and Services-RRs infrastructures separate (ships in night), which is very easy to pull off with current virtualized RRs. -Type-1 RDs will help you simulate full-mesh. adam From: NANOG <nanog-bounces@nanog.org> On Behalf Of Mohammad Khalil Sent: Sunday, February 10, 2019 8:10 PM To: nanog@nanog.org Subject: BGP topological vs centralized route reflector Dears Am trying to find some documents and practical implementations regarding bgp topological vs centralized route reflector(In-band vs out-of-band) Any good shares are appreciated
yes From: Jason Lixfeld <jason+nanog@lixfeld.ca> Sent: Tuesday, February 19, 2019 4:06 PM To: adamv0025@netconsultings.com Cc: Mohammad Khalil <eng.mssk@gmail.com>; nanog@nanog.org Subject: Re: BGP topological vs centralized route reflector Hi Adam, On Feb 19, 2019, at 10:28 AM, <adamv0025@netconsultings.com <mailto:adamv0025@netconsultings.com> > <adamv0025@netconsultings.com <mailto:adamv0025@netconsultings.com> > wrote: -Type-1 RDs will help you simulate full-mesh. By “Type-1 RD”, are you referring to a unique RD per PE?
participants (8)
-
Aaron Gould -
adamv0025＠netconsultings.com -
Alain Hebert -
Jason Lixfeld -
Mark Tinka -
Mohammad Khalil -
Phil Lavin -
Saku Ytti