David Sinn Sent: Friday, June 12, 2020 4:42 PM
On Jun 12, 2020, at 8:26 AM, Saku Ytti <saku@ytti.fi> wrote:
On Fri, 12 Jun 2020 at 18:16, David Sinn <dsinn@dsinn.com> wrote:
The label stack question is about the comparisons between the two extremes of SR that you can be in. You either label your packet just for it's ultimate destination or you apply the stack of the points you want to pass through.
In the former case you are, at the forwarding plane, equal to what you see with traditional MPLS today, with every node along the path needing to know how to reach the end-point. Yes, you have lowered label space from traditional MPLS, but that can be done with site-cast labels already. And, while the nodes don't have to actually swap labels, when you look at commodity implementations (across the last three generations since you want to do this with what is deployed, not wholesale replace the network) a null swap still ends up eating a unique egress next-hop entry. So from a hardware perspective, you haven't improved anything. Your ECMP group count is high.
In the extreme latter case, you have to, on ingress, place the full stack of every "site" you want to pass through. That has the benefits of "sites" only need labels for their directly connected sites, so you have optimized the implications on commodity hardware. However, you now have a label stack that can be quite tall. At least if you want to walk the long-way around
Yes this is where each node needs to have a label uniquely identifying every LSP passing through it. Saku, With IP header you don't need this, Consider this: PE1 to PE2 via 3 P-core nodes With ECMP in IP, then PE1 just needs single FEC the DST-IP of PE2, which will be load-shared across all 3 paths. Using MPLS If you need to uniquely identify each path you need 3 FECs (3 LSPs one via each P core node), now imagine you have 100K possible paths across the fabric -that's a lot of FECs on PE1 or any node in the fabric where each has to have a unique label for every possible unique path via the core that the particular node is part of. the
world, say due to failure. On top, that depth of label stack means devices in the middle can't look at the original payload to make ECMP decisions. So you can turn to entropy labels, but that sort of makes matters worse.
David, You can use hierarchy of tunnels to help with the deep label-stack imposition problem. adam
On Mon, 15 Jun 2020 at 12:24, <adamv0025@netconsultings.com> wrote:
Yes this is where each node needs to have a label uniquely identifying every LSP passing through it. Saku, With IP header you don't need this, Consider this: PE1 to PE2 via 3 P-core nodes With ECMP in IP, then PE1 just needs single FEC the DST-IP of PE2, which will be load-shared across all 3 paths. Using MPLS If you need to uniquely identify each path you need 3 FECs (3 LSPs one via each P core node), now imagine you have 100K possible paths across the fabric -that's a lot of FECs on PE1 or any node in the fabric where each has to have a unique label for every possible unique path via the core that the particular node is part of.
Are we talking about specific implementations of fundamentals? It sounds like we are talking about a specific case where IP next-hop is unilist of N next-hops, and MPLS next-hop is a single item without indirection? This is not a fundamental difference, this is implementation detail. There is no particular reason MPLS next-hop couldn't be unilist of N destinations. I think people are too focused on thinking IP and MPLS have some inherent magical property differences, they don't. We only care about lookup cost (again IP can be made cheap with IPinIPinIP tunnels and telling LSR devices all lookups are LEM host lookups) and we care about key width. Rest is implementation detail. Yes on typical case there are some biases in IP and MPLS, but these can be rendered away leaving the fundamental differences. Briding the gap from MPLS to IP is far smaller than bridging the gap from IP to MPLS, there is so much added value which depends on MPLS tunnels. -- ++ytti
From: Saku Ytti <saku@ytti.fi> Sent: Monday, June 15, 2020 10:31 AM
On Mon, 15 Jun 2020 at 12:24, <adamv0025@netconsultings.com> wrote:
Yes this is where each node needs to have a label uniquely identifying every LSP passing through it. Saku, With IP header you don't need this, Consider this: PE1 to PE2 via 3 P-core nodes With ECMP in IP, then PE1 just needs single FEC the DST-IP of PE2, which will be load-shared across all 3 paths. Using MPLS If you need to uniquely identify each path you need 3 FECs (3 LSPs one via each P core node), now imagine you have 100K possible paths across the fabric -that's a lot of FECs on PE1 or any node in the fabric where each has to have a unique label for every possible unique path via the core that the particular node is part of.
Are we talking about specific implementations of fundamentals? It sounds like we are talking about a specific case where IP next-hop is unilist of N next- hops, and MPLS next-hop is a single item without indirection? This is not a fundamental difference, this is implementation detail. There is no particular reason MPLS next-hop couldn't be unilist of N destinations.
Yes it can indeed, and that's moving towards the centre between the extreme cases that David laid out. It's about how granular one wants to be in identifying an end-to-end path between a pair of edge nodes. I agree with you that MPLS is still better than IP, and I tried to illustrate that even enumerating every possible paths using deep label stack is not a problem (and even that can be alleviated using hierarchy of LSPs). adam
On Mon, 15 Jun 2020 at 12:46, <adamv0025@netconsultings.com> wrote:
Yes it can indeed, and that's moving towards the centre between the extreme cases that David laid out. It's about how granular one wants to be in identifying an end-to-end path between a pair of edge nodes. I agree with you that MPLS is still better than IP, and I tried to illustrate that even enumerating every possible paths using deep label stack is not a problem (and even that can be alleviated using hierarchy of LSPs).
The entirety of my point is, if we were rational, we'd move towards increasingly efficient solutions. And technically everything we do in MPLS tunnels, we can do in IP tunnels and converse. Should we imagine a future where all features and functions are supported in both, it's clear we should want to do MPLS tunnels. Just the [IGP][BGP-LU] 8B overhead, compared to IP 40B overhead should drive the point home, and ultimately, that's the only difference, rest is implementation. And I'm saddened we've been marketed snake-oil like SRv6 with fake promises of inherent advantages or simplicity 'just IP'. We can do better than MPLS, absolutely. But IP is worse. -- ++ytti
From: Saku Ytti <saku@ytti.fi> Sent: Monday, June 15, 2020 11:02 AM
On Mon, 15 Jun 2020 at 12:46, <adamv0025@netconsultings.com> wrote:
Yes it can indeed, and that's moving towards the centre between the extreme cases that David laid out. It's about how granular one wants to be in identifying an end-to-end path between a pair of edge nodes. I agree with you that MPLS is still better than IP, and I tried to illustrate that even enumerating every possible paths using deep label stack is not a problem (and even that can be alleviated using hierarchy of LSPs).
The entirety of my point is, if we were rational, we'd move towards increasingly efficient solutions. And technically everything we do in MPLS tunnels, we can do in IP tunnels and converse. Should we imagine a future where all features and functions are supported in both, it's clear we should want to do MPLS tunnels. Just the [IGP][BGP-LU] 8B overhead, compared to IP 40B overhead should drive the point home, and ultimately, that's the only difference, rest is implementation.
And I'm saddened we've been marketed snake-oil like SRv6 with fake promises of inherent advantages or simplicity 'just IP'.
We can do better than MPLS, absolutely. But IP is worse.
Yes I absolutely agree, Not to mention this whole thread is focused solely on next-hop identification -which is just the lowest of the layers of abstraction in the vertical stack. We haven’t talked about other "entities" that need identification like: VPNs, applications, policies (yes I'm looking at you VXLAN!) etc... - all of which are way better identified by a simple label rather than IPinIPinIP.... adam
On 15/Jun/20 12:13, adamv0025@netconsultings.com wrote:
Not to mention this whole thread is focused solely on next-hop identification -which is just the lowest of the layers of abstraction in the vertical stack. We haven’t talked about other "entities" that need identification like: VPNs, applications, policies (yes I'm looking at you VXLAN!) etc... - all of which are way better identified by a simple label rather than IPinIPinIP....
The problem is if you want to have MPLS services signaled over IPv6, you first need an IPv6 control plane that will signal the labels that will support those services, i.e., LDPv6 and RSVPv6. Right now, all vendors that support LDPv6 do so only for pure MPLS-IPv6 forwarding. If you want l2vpnv6, l3vpnv6, EVPNv6, 4PE, 4VPE, TE-FRRv6, e.t.c., you can't get that today. You'd have to signal those services over LDPv4 or RSVPv4. The guys did a great gap analysis back in 2015, when LDPv6 began to appear in IOS XR and Junos, that showed the challenges toward an IPv6-only MPLS network. I believe much of this is still relevant in 2020: https://tools.ietf.org/html/rfc7439 I have to commend Vishwas, together with both Rajiv's, for kicking this off way back in 2008. The road has been long and hard. Mark.
From: Mark Tinka <mark.tinka@seacom.mu> Sent: Monday, June 15, 2020 4:07 PM
On 15/Jun/20 12:13, adamv0025@netconsultings.com wrote:
Not to mention this whole thread is focused solely on next-hop identification -which is just the lowest of the layers of abstraction in the vertical stack. We haven’t talked about other "entities" that need identification like: VPNs, applications, policies (yes I'm looking at you VXLAN!) etc... - all of which are way better identified by a simple label rather than IPinIPinIP....
The problem is if you want to have MPLS services signaled over IPv6, you first need an IPv6 control plane that will signal the labels that will support those services, i.e., LDPv6 and RSVPv6.
Right now, all vendors that support LDPv6 do so only for pure MPLS-IPv6 forwarding. If you want l2vpnv6, l3vpnv6, EVPNv6, 4PE, 4VPE, TE-FRRv6, e.t.c., you can't get that today. You'd have to signal those services over LDPv4 or RSVPv4.
Hence my earlier comment on why I think it's not commercially feasible to switch to v6 control plane, (the only thing I'd be getting is MPLS-IPv6 which I already have (or have had) via L3VPN-6VPE), time will tell if I ever need to make the switch. But I'm thankful to you for doing the "ice breaking" for the rest of the community. adam
On 16/Jun/20 12:00, adamv0025@netconsultings.com wrote:
Hence my earlier comment on why I think it's not commercially feasible to switch to v6 control plane,
Personally, I've never been a fan of a single-stack backbone. I can, however, understand the use-case where a new or growing network is unable to obtain anymore IPv4 space and don't want to use RFC 1918 space (yuck!).
(the only thing I'd be getting is MPLS-IPv6 which I already have (or have had) via L3VPN-6VPE),
Well, not quite. What you currently have with 6PE is IPv6 tunneled inside MPLSv4 which runs over IPv4. While you get the benefits of MPLS forwarding for your IPv6 traffic, you now create a condition where your IPv6 network is in the hands of your IPv4 network. Granted, there are many folk that run 6PE, so whether the fate-sharing is of concern to you or not is an exercise left up to the reader. Personally, I'd rather avoid fate-sharing whenever I can. On the other hand, MPLSv6 is native, runs over IPv6 and does not depend on IPv4 at all. Ultimately, plenty of energy will need to go into supporting the additional VPN services that go beyond plain-old MPLSv6 switching. But that can only be promoted with the vendors after we jump the first hurdle of deploying the 1st application, basic MPLSv6 switching; get that widely adopted, and create more awareness within the vendor community about its overall viability. 80% of our network currently runs LDPv6 and switches IPv6 traffic in MPLSv6. Our immediate task is to get the remaining 20% supported (IOS XE) as well.
But I'm thankful to you for doing the "ice breaking" for the rest of the community.
As Eriq La Salle unashamedly claimed in the 1988 Eddie Murphy picture, Coming to America, "You know me, anything for the kids :-)". Mark.
From: Mark Tinka <mark.tinka@seacom.mu> Sent: Tuesday, June 16, 2020 12:09 PM
On 16/Jun/20 12:00, adamv0025@netconsultings.com wrote:
Hence my earlier comment on why I think it's not commercially feasible to switch to v6 control plane,
Personally, I've never been a fan of a single-stack backbone. I can, however, understand the use-case where a new or growing network is unable to obtain anymore IPv4 space and don't want to use RFC 1918 space (yuck!).
Actually I was exactly I that situation and v4 RFC 1918 space worked out just fine.
(the only thing I'd be getting is MPLS-IPv6 which I already have (or have had) via L3VPN-6VPE),
Well, not quite.
What you currently have with 6PE is IPv6 tunneled inside MPLSv4 which runs over IPv4. While you get the benefits of MPLS forwarding for your IPv6 traffic, you now create a condition where your IPv6 network is in the hands of your IPv4 network. Granted, there are many folk that run 6PE, so whether the fate-sharing is of concern to you or not is an exercise left up to the reader. Personally, I'd rather avoid fate-sharing whenever I can.
I've been dependent solely on v4 all my life and I still am. But I see your fate-sharing argument, similar to my argument around separate iBGP infrastructure (Route-Reflector plane) for Internet vs for other customer private VPN prefixes. But in the multiplanar iBGP case one plane is statistically more likely to fail than the other, whereas in case of v4 vs v6 control plane I'd say it's actually the NEW v6 that's more likely hiding some unforeseen bug. So let me ask the following "devil's advocate" type of question, under the assumption that the LDPv6 is a new thing (not as proven as LDPv4), are you taking dependency away by splitting control plane to v4 and v6 or actually adding dependency - where the NEW v6 control plane components could negatively affect the existing v4 control plane components after all they share a common RE (or even RDP in Junos).
On the other hand, MPLSv6 is native, runs over IPv6 and does not depend on IPv4 at all.
Ultimately, plenty of energy will need to go into supporting the additional VPN services that go beyond plain-old MPLSv6 switching. But that can only be promoted with the vendors after we jump the first hurdle of deploying the 1st application, basic MPLSv6 switching; get that widely adopted, and create more awareness within the vendor community about its overall viability.
80% of our network currently runs LDPv6 and switches IPv6 traffic in MPLSv6. Our immediate task is to get the remaining 20% supported (IOS XE) as well.
But I'm thankful to you for doing the "ice breaking" for the rest of the community.
As Eriq La Salle unashamedly claimed in the 1988 Eddie Murphy picture, Coming to America, "You know me, anything for the kids :-)".
That was a good movie :D :D :D adam
On 16/Jun/20 14:24, adamv0025@netconsultings.com wrote:
Actually I was exactly I that situation and v4 RFC 1918 space worked out just fine.
In that way, you are braver than me. But hey, if you need IPv4 and can't get the public stuff, I won't fault you for going with the private stuff :-).
I've been dependent solely on v4 all my life and I still am. But I see your fate-sharing argument, similar to my argument around separate iBGP infrastructure (Route-Reflector plane) for Internet vs for other customer private VPN prefixes. But in the multiplanar iBGP case one plane is statistically more likely to fail than the other, whereas in case of v4 vs v6 control plane I'd say it's actually the NEW v6 that's more likely hiding some unforeseen bug. So let me ask the following "devil's advocate" type of question, under the assumption that the LDPv6 is a new thing (not as proven as LDPv4), are you taking dependency away by splitting control plane to v4 and v6 or actually adding dependency - where the NEW v6 control plane components could negatively affect the existing v4 control plane components after all they share a common RE (or even RDP in Junos).
Well, that's a bottomless rabbit hole that go could all to the way to the data centre providing A+B power feeds, but connected to a single grid on the outside. At some point, redundancy stops making sense and eats into your margins as much as it does your sanity :-). But back to the question at hand, even with 6PE, you can't avoid running a dual-stack network... you'd need to at the edge. So if your goal is to use 6PE to avoid running IPv6 in some native form anywhere in your network, that won't work out, as I'm sure you know :-). But more importantly, I, as have many others on this group, have been running IPv6 since about 2003 (others, even longer, I'm sure). IPv6, in and of itself, has never been an issue for me. The problems have always been the ancillary services that need to run on top of it in order to work. For the past 17 years, my IPv6 headaches have been about feature parity with IPv4, mostly, in routers and switches (in general-purpose server OS's too, but those got fixed much more quickly): * DNS over IPv6 took a while to arrive. * TACACS+ over IPv6 took a while to arrive. * IPv6 ACL's took a while to get proper support. * SNMP over IPv6 took a while to arrive. * NTP over IPv6 took a while to arrive. * SSH over IPv6 took a while to arrive. * OSPFv3 Authentication was very clunky. * Multi Topoloy IS-IS support was very clunky. You get the idea. I've always operated a native dual-stack network, so having to go back and upgrade routers every so often when one of the above limitations got fixed in a later revision of code was tiresome, but worthwhile. We take a lot of these things for granted in 2020, but it was no joke more than a decade ago. So for me, I've never really experienced any problems from basic IPv6 that have negatively impacted IPv4. The corner case I am aware of that didn't even bother IPv4 was Ethernet switches and some popular Chinese GPON AN's that silently dropped Ethernet frames carrying IPv6 packets because they did not know how to handle the 0x86DD EtherType. But AFAIK, these have all been since fixed. So based on pure experience, I don't expect this "32-year old new IPv6" thing to be hiding some unforeseen bug that will break our IPv4 network :-). LDPv6 was first implemented in IOS XR, Junos and SR-OS in 2015/2016, so it has been around for a while. The biggest challenge was with IOS XR in 2015 (5.3.0) which didn't support dual-stack TLV's. So if the LDP neighbor negotiated LDPv4 and LDPv6 in the same LDP session, IOS XR didn't know what to do. It could do LDPv4-only or LDPv6-only, and not both. That issue was fixed in IOS XR 6.0.1, when the Dual-Stack Capability TLV feature was introduced. That was May of 2016, so also not that new. Mark.
On 15/Jun/20 12:13, adamv0025@netconsultings.com wrote:
Not to mention this whole thread is focused solely on next-hop identification -which is just the lowest of the layers of abstraction in the vertical stack. We haven’t talked about other "entities" that need identification like: VPNs, applications, policies (yes I'm looking at you VXLAN!) etc... - all of which are way better identified by a simple label rather than IPinIPinIP....
The problem is if you want to have MPLS services signaled over IPv6, you first need an IPv6 control plane that will signal the labels that will support those services, i.e., LDPv6 and RSVPv6. Right now, all vendors that support LDPv6 do so only for pure MPLS-IPv6 forwarding. If you want l2vpnv6, l3vpnv6, EVPNv6, 4PE, 4VPE, TE-FRRv6, e.t.c., you can't get that today. You'd have to signal those services over LDPv4 or RSVPv4. The guys did a great gap analysis back in 2015, when LDPv6 began to appear in IOS XR and Junos, that showed the challenges toward an IPv6-only MPLS network. I believe much of this is still relevant in 2020: https://tools.ietf.org/html/rfc7439 I have to commend Vishwas, together with both Rajiv's, for kicking this off way back in 2008. The road has been long and hard. Mark.
participants (3)
-
adamv0025@netconsultings.com
-
Mark Tinka
-
Saku Ytti