Can I do this in EVPN? (Multihome to more different CEs)
All, I have a bit of a networking design challenge, and I thing EVPN is the right answer, but despite spending the last week reading loads of resources about it, I can't quite get my head around one aspect. I'm trying to genericise the design a bit here, but what I've got is... I have multiple layer two broadcast domains that I need to link together over a layer 3 network. The broadcast domains consist of multiple switches carrying multiple vlans spanning multiple locations (think of it like a customer campus network). I need to interconnect with each broadcast domain in two different locations. (so two PEs to two CEs), and link it back to a datacentre in another city. In the simple case, using EVPN, I see that I can run active-standby multihoming, configuring one ESI for the customer campus network. If one of my PEs fails, or one of the customer CEs fails, then EVPN will fail over to the other link. However, the failure scenario I need to deal with is if a layer two link fails between two locations within the customer campus, the two halves of the now split broadcast domain still need to be able to communicate with the datacentre (but do not need to be able to communicate with each other). Every example I can see for EVPN shows multihoming to a single CE, and I can't find anywhere an example which deals with a "split" ES. Is there a solution to this problem? Many thanks in advance, Simon
You should be able to setup a VPLS between 3 (or more) devices. Something like this -- Example: VFI on a PE Device The following example shows a virtual forwarding instance (VFI) configuration: Device(config)# l2 vfi vfi110 manual Device(config-vfi)# vpn id 110 Device(config-vfi)# neighbor 172.16.10.2 4 encapsulation mpls Device(config-vfi)# neighbor 10.16.33.33 encapsulation mpls Device(config-vfi)# neighbor 198.51.100.44 encapsulation mpls Device(config-vfi)# bridge-domain 100 Device(config-vfi)# end The following example shows a VFI configuration for a hub-and-spoke configuration: Device(config)# l2 vfi VPLSA manual Device(config-vfi)# vpn id 110 Device(config-vfi)# neighbor 10.9.9.9 encapsulation mpls Device(config-vfi)# neighbor 192.0.2.12 encapsulation mpls Device(config-vfi)# neighbor 203.0.113.4 encapsulation mpls no-split-horizon Device(config-vfi)# bridge-domain 100 Device(config-vfi)# end -----Original Message----- From: "Simon Lockhart" <simon@slimey.org> Sent: Thursday, February 9, 2023 2:47am To: nanog@nanog.org Subject: Can I do this in EVPN? (Multihome to more different CEs) All, I have a bit of a networking design challenge, and I thing EVPN is the right answer, but despite spending the last week reading loads of resources about it, I can't quite get my head around one aspect. I'm trying to genericise the design a bit here, but what I've got is... I have multiple layer two broadcast domains that I need to link together over a layer 3 network. The broadcast domains consist of multiple switches carrying multiple vlans spanning multiple locations (think of it like a customer campus network). I need to interconnect with each broadcast domain in two different locations. (so two PEs to two CEs), and link it back to a datacentre in another city. In the simple case, using EVPN, I see that I can run active-standby multihoming, configuring one ESI for the customer campus network. If one of my PEs fails, or one of the customer CEs fails, then EVPN will fail over to the other link. However, the failure scenario I need to deal with is if a layer two link fails between two locations within the customer campus, the two halves of the now split broadcast domain still need to be able to communicate with the datacentre (but do not need to be able to communicate with each other). Every example I can see for EVPN shows multihoming to a single CE, and I can't find anywhere an example which deals with a "split" ES. Is there a solution to this problem? Many thanks in advance, Simon
On Thu Feb 09, 2023 at 11:54:28AM -0500, Shawn L wrote:
You should be able to setup a VPLS between 3 (or more) devices. Something like this --
[snip] Thanks - I'm not committed to EVPN, so VPLS could work too. Would VPLS handle loop avoidance for me? (i.e. if I have two VPLS PE connections into the same broadcast domain on the customer side) Simon
VPLS doesn't handle loop avoidance. At least, not apart from split horizon rules. I assume that them properly connecting routers only and doing dynamic routing over your service is out of the question? (Even _just_ doing this doesn't completely solve the challenge though.) It sounds to me like your customer is needing two separate services. One to provide connectivity to other sites at layer 2, and another to provide backup connectivity within their single campus at layer 2. I would suggest that you treat these as two separate services, because there's nothing in EVPN that's going to notice on the PE side of the equation that the customer has a break in the middle of their network. Maybe consider offering these two services in combination: 1) layer 2 VPN service (VPWS / single pseudowire) between the two sides of their campus. You would need to ensure L2CP transparency (or tunneling) for STP and they would need to run STP across the link to keep their campus whole 2) EVPN with ESI in single-active mode, as you had mentioned. ------- Original Message ------- On Thursday, February 9th, 2023 at 11:56 AM, Simon Lockhart <simon@slimey.org> wrote:
On Thu Feb 09, 2023 at 11:54:28AM -0500, Shawn L wrote:
You should be able to setup a VPLS between 3 (or more) devices. Something like this --
[snip]
Thanks - I'm not committed to EVPN, so VPLS could work too. Would VPLS handle loop avoidance for me? (i.e. if I have two VPLS PE connections into the same broadcast domain on the customer side)
Simon
_______________________ Jason R. Rokeach m: 603.969.5549 e: jason@rokea.ch tg: jasonrokeach Sent with ProtonMail secure email. Get my PGP Public Key.
The solution we've deployed is to use a VXLAN termination device at each location requiring multi-path redundancy. Run VXLAN over isolated L3 domains, let IS-IS or OSPF handle path selection, including ECMP if desired. If multi-chassis redundancy is required, pick a platform that can do MLAG or similar. So for example, I have two sites with multiple VLANs needing to be interconnected, and for whatever reason I can't just use a LAG (distance, lack of transparent L2 service, whatever). We would put an Arista 7k-series pizzabox at each end, one end could be an MLAG pair. Terminate two L2 or L3 services on the singleton box, terminate each service onto one half of the MLAG pair at the other site. Run an IGP (ideally IS-IS with BFD, but YMNV) and ECMP and happens automatically, as does handling single-path failures. This could equally be a MLAG-to-MLAG setup if you have too much money and need to use some up. Cisco vPC does essentially the same thing, as does Juniper's VC. Extreme has something similar, too. STP does not get transported across the VXLAN transport, so you now avoid all the inherent problems with long-distance or multi-site STP bridging. -Adam Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca Chat with me on Teams: athompson@merlin.mb.ca
-----Original Message----- From: NANOG <nanog-bounces+athompson=merlin.mb.ca@nanog.org> On Behalf Of Jason R. Rokeach via NANOG Sent: February 9, 2023 1:11 PM Cc: nanog@nanog.org Subject: Re: Can I do this in EVPN? (Multihome to more different CEs)
VPLS doesn't handle loop avoidance. At least, not apart from split horizon rules.
I assume that them properly connecting routers only and doing dynamic routing over your service is out of the question? (Even _just_ doing this doesn't completely solve the challenge though.)
It sounds to me like your customer is needing two separate services. One to provide connectivity to other sites at layer 2, and another to provide backup connectivity within their single campus at layer 2. I would suggest that you treat these as two separate services, because there's nothing in EVPN that's going to notice on the PE side of the equation that the customer has a break in the middle of their network. Maybe consider offering these two services in combination: 1) layer 2 VPN service (VPWS / single pseudowire) between the two sides of their campus. You would need to ensure L2CP transparency (or tunneling) for STP and they would need to run STP across the link to keep their campus whole 2) EVPN with ESI in single-active mode, as you had mentioned.
------- Original Message ------- On Thursday, February 9th, 2023 at 11:56 AM, Simon Lockhart <simon@slimey.org> wrote:
On Thu Feb 09, 2023 at 11:54:28AM -0500, Shawn L wrote:
You should be able to setup a VPLS between 3 (or more) devices. Something like this --
[snip]
Thanks - I'm not committed to EVPN, so VPLS could work too. Would VPLS handle loop avoidance for me? (i.e. if I have two VPLS PE connections into the same broadcast domain on the customer side)
Simon
_______________________ Jason R. Rokeach m: 603.969.5549 e: jason@rokea.ch tg: jasonrokeach
Sent with ProtonMail secure email. Get my PGP Public Key.
Hi Simon, This looks like an interesting project. What are you ultimately trying to achieve? Stretching an Ethernet segment between sites shouldn't be the first choice, but I imagine you're having to deal with some legacy use case and your hands are tied. I'm a little curious on what that might look like. Best, Josh On Thu, Feb 9, 2023, 02:50 Simon Lockhart <simon@slimey.org> wrote:
All,
I have a bit of a networking design challenge, and I thing EVPN is the right answer, but despite spending the last week reading loads of resources about it, I can't quite get my head around one aspect.
I'm trying to genericise the design a bit here, but what I've got is...
I have multiple layer two broadcast domains that I need to link together over a layer 3 network. The broadcast domains consist of multiple switches carrying multiple vlans spanning multiple locations (think of it like a customer campus network).
I need to interconnect with each broadcast domain in two different locations. (so two PEs to two CEs), and link it back to a datacentre in another city.
In the simple case, using EVPN, I see that I can run active-standby multihoming, configuring one ESI for the customer campus network. If one of my PEs fails, or one of the customer CEs fails, then EVPN will fail over to the other link.
However, the failure scenario I need to deal with is if a layer two link fails between two locations within the customer campus, the two halves of the now split broadcast domain still need to be able to communicate with the datacentre (but do not need to be able to communicate with each other).
Every example I can see for EVPN shows multihoming to a single CE, and I can't find anywhere an example which deals with a "split" ES.
Is there a solution to this problem?
Many thanks in advance,
Simon
participants (5)
-
Adam Thompson -
Jason R. Rokeach -
Joshua Miller -
Shawn L -
Simon Lockhart