Devil's Advocate - Segment Routing, Why?
Hi all. When the whole SR concept was being first dreamed up, I was mildly excited about it. But then real life happened and global deployment (be it basic SR-MPLS or SRv6) is what it is, and I became less excited. This was back in 2015. All the talk about LDPv6 this and last week has had me reflecting a great deal on where we are, as an industry, in why we are having to think about SR and all its incarnations. So, let me be the one that stirs up the hornets' nest... Why do we really need SR? Be it SR-MPLS or SRv6 or SRv6+? I've heard a lot about "network programmability", e.t.c., but can anyone point me toward a solution that actually does this in the way that it has been touted for years? A true flow that shows the implementation of "network programming" over any incarnation of SR? Perhaps one a customer can go to the shop and grab off the shelf? A lot of kit does not currently support SR, be it in hardware or software. So are operators expected to dispose of boxes that are happily moving MPLS frames along with no complaints, and replace them with some newfangled creations that will support SR in code and silicon? At whose cost? Not just money, but time, people and working the day-to-day kinks out? I've heard about "end-to-end service chaining" as a use-case for SR. To service-chain what? Classic telco's don't offer complex over-the-top services that operate at a such a scale that "service chaining" in SR would make lives easier. More than half of the traffic we are carrying is coming in over the public Internet, and not some private VPN. And if "service chaining" makes sense to the cloud and content operators who run humongous data centres where the servers significantly outnumber the routing/switching/transport gear, I'd naively posit that they have built a myriad of custom, in-house solutions, systems, tools and controllers to do all the "service chaining" they could ever need, and have been at it for more than 10 years, if not more, all to manage an MPLS/DWDM backbone. So what off-the-shelf "service chaining" controller are they going to walk into the shop and pay money for? If I had to think of the number of network, content and cloud operators who have either said they've deployed some kind of SR, or intend to, you're looking at probably 10% - 15% of a market. What about the other 85% - 90% of the operators whose requirements are so simple, thinking about dumping existing boxes, systems, tools and solutions that work very well in order to join the SR club doesn't seem feasible. What problems are 90% of the operators running MPLS having that SR will truly fix, given that they don't operate large, distributed data centres or have a 5G license? What's even more wild, is that there are equally a number of networks that are stalling IPv6 deployment, for some reason or other, meaning it will probably take us another 1 to 2 decades to see worldwide adoption of IPv6. If SRv6 or SRv6+ is "where the market is dying to go", and a bunch of operators don't have IPv6 in their plans, what gives? To be clear, I'm not against SR; what has to come will come. What I am less enthused about is being forced into an all-or-nothing scenario for the going concern of my network. For those that are keen on SR, give them SR. But for those who would prefer to keep things simple in networks that are not about to fall over and die, let's have LDPv6 and let's implement RFC 7439. Then let the operators choose. On a personal note, it's a pity Juniper gave in to the SRv6 fight, despite all the initial resistance. If they'd gone a different direction and simply implemented RFC 7439 (they have LDPv6 already), not only would that have put Cisco under serious pressure, but it would have solved the problems of many network operators that are desperately looking to go IPv6-only, and still maintain the rich MPLS services they and their customers have grown to like. Mark.
Hey,
Why do we really need SR? Be it SR-MPLS or SRv6 or SRv6+?
I don't like this, SR-MPLS and SRv6 are just utterly different things to me, and no answer meaningfully applies to both. I would ask, why do we need LDP, why not use IGP to carry labels? Less state, protocols, SLOC, cost, bug surface And we get more features to boot, with LDP if you want LFA, you need to form tLDP to every Q-space node, on top of your normal LDP, because you don't know label view from anyone else but yourself. With SR by nature you know the label view for everyone, thus you have full LFA coverage for free, by-design. Also by-design IGP/LDP Sync. So no need to justify it by any magic new things, it's just a lot simpler than LDP, you don't need to need new things to justify SR-MPLS, you need to want to do existing things while reducing complexity and state. -- ++ytti
On Wed, 17 Jun 2020 at 18:42, Saku Ytti <saku@ytti.fi> wrote:
Hey,
Why do we really need SR? Be it SR-MPLS or SRv6 or SRv6+?
I don't like this, SR-MPLS and SRv6 are just utterly different things to me, and no answer meaningfully applies to both.
I don't understand the point of SRv6. What equipment can support IPv6 routing, but can't support MPLS label switching? I'm a big fan of SR-MPLS however.
And we get more features to boot, with LDP if you want LFA, you need to form tLDP to every Q-space node, on top of your normal LDP, because you don't know label view from anyone else but yourself. With SR by nature you know the label view for everyone, thus you have full LFA coverage for free, by-design.
Not just this, but the LFA path is always the post-convergence path. You don't get microloops. You can implement TE on top if that is your thing. No need to run RSVP. Another protocol you don't need to run. You don't need to throw out all your old kit, and replace with new in one go. You can incrementally roll it out, and leave islands of LDP where needed. LDP-SR interworking is pretty simple. We are currently introducing it into our core. It will probably be a while before we fully phase out LDP, but its definitely on the roadmap. Regards, Dave
On 17/Jun/20 20:40, Dave Bell wrote:
I don't understand the point of SRv6. What equipment can support IPv6 routing, but can't support MPLS label switching?
Indeed. Anything that can support LDPv4 today can support LDPv6, in hardware. SRv6 and SRv6+ is a whole other issue, not to mention the amount of work needed to write code for it.
Not just this, but the LFA path is always the post-convergence path. You don't get microloops.
You can implement TE on top if that is your thing. No need to run RSVP. Another protocol you don't need to run.
You don't need to throw out all your old kit, and replace with new in one go. You can incrementally roll it out, and leave islands of LDP where needed. LDP-SR interworking is pretty simple.
We are currently introducing it into our core. It will probably be a while before we fully phase out LDP, but its definitely on the roadmap.
Happy to hear, and I have nothing against your choice if you are happy with it. But for a network that may not see the need in spending cycles doing yet-another roll out, it tastes funny when you are forced down a new path. Mark.
Anything that can support LDPv4 today can support LDPv6, in hardware.
While I am trying to stay out of this interesting discussion the above statement is not fully correct. Yes in the MPLS2MPLS path you are correct, But ingress and egress switching vectors are very different for LDPv6 as you need to match on IPv6 vs LDPv4 ingress where you match on IPv4 to map it to correct label stack rewrite. Example: If your hardware ASICs do not support IPv6 while support IPv4 - LDPv4 will work just fine while LDPv6 will have a rather a bit of hard time :) Cheers, R.
On 18/Jun/20 00:29, Robert Raszuk wrote:
Example: If your hardware ASICs do not support IPv6 while support IPv4 - LDPv4 will work just fine while LDPv6 will have a rather a bit of hard time :)
Well, safe to say that if your box doesn't support IPv6, MPLSv6 is probably the least of your worries :-). Mark.
On Wed, Jun 17, 2020, at 20:40, Dave Bell wrote:
I don't understand the point of SRv6. What equipment can support IPv6 routing, but can't support MPLS label switching?
A whole ocean of "datacenter" hardware, from pretty much evey vendor. Because many of them automatically link MPLS to RSVP and IPv4 L3VPN (which may still be an interesting feature in datacenter), many try to stay as far away from it as possible. Othen than some scared C-level guys imposing this, I don't really see a good reason (lack of market demand, which is sometimes invoked, doesn't stand).
where needed. LDP-SR interworking is pretty simple.
Mapping servers or something else ?
On Fri, 19 Jun 2020 at 10:24, Radu-Adrian Feurdean <nanog@radu-adrian.feurdean.net> wrote:
I don't understand the point of SRv6. What equipment can support IPv6 routing, but can't support MPLS label switching?
A whole ocean of "datacenter" hardware, from pretty much evey vendor. Because many of them automatically link MPLS to RSVP and IPv4 L3VPN (which may still be an interesting feature in datacenter), many try to stay as far away from it as possible. Othen than some scared C-level guys imposing this, I don't really see a good reason (lack of market demand, which is sometimes invoked, doesn't stand).
I'm sure such devices exist, I can't name any from top of my head. But this market perversion is caused by DC people who did not understand networks and suffer from not-invented-here. Everyone needsa tunnel solution, but DC people decided before looking into or understanding the topic that MPLS is bad and complex, let's invent something new. Then we re-invented solutions that already had _MORE_ efficient solutions in MPLS, and a lot of those technologies are now becoming established in DC space, creating confusion in SP space. Maybe these inferior technologies will win, due to the marketing strength of DC solutions. Or maybe DC will later figure out the fundamental aspect in tunneling cost, and invent even-better-MPLS, which is entirely possible now that we have a bit more understanding how we use MPLS. -- ++ytti
On 19/Jun/20 09:50, Saku Ytti wrote:
I'm sure such devices exist, I can't name any from top of my head. But this market perversion is caused by DC people who did not understand networks and suffer from not-invented-here. Everyone needsa tunnel solution, but DC people decided before looking into or understanding the topic that MPLS is bad and complex, let's invent something new. Then we re-invented solutions that already had _MORE_ efficient solutions in MPLS, and a lot of those technologies are now becoming established in DC space, creating confusion in SP space.
Maybe these inferior technologies will win, due to the marketing strength of DC solutions. Or maybe DC will later figure out the fundamental aspect in tunneling cost, and invent even-better-MPLS, which is entirely possible now that we have a bit more understanding how we use MPLS.
Let me work out how to print all this on a t-shirt. Mark.
Saku Ytti Sent: Friday, June 19, 2020 8:50 AM
On Fri, 19 Jun 2020 at 10:24, Radu-Adrian Feurdean <nanog@radu- adrian.feurdean.net> wrote:
I don't understand the point of SRv6. What equipment can support IPv6 routing, but can't support MPLS label switching?
Maybe these inferior technologies will win, due to the marketing strength of DC solutions. Or maybe DC will later figure out the fundamental aspect in tunneling cost, and invent even-better-MPLS, which is entirely possible now that we have a bit more understanding how we use MPLS.
Looking back at history (VXLAN or the Google's Espresso "architecture") I'm not holding my breath for anything reasonable coming out of the DC camp.... adam
On 19/Jun/20 09:20, Radu-Adrian Feurdean wrote:
A whole ocean of "datacenter" hardware, from pretty much evey vendor.
You mean the ones deliberately castrated so that we can create a specific "DC vertical", even if they are, pretty much, the same box a service provider will buy, just given a darker color so it can glow more brightly in the data centre night? Mark.
On Fri, Jun 19, 2020, at 10:11, Mark Tinka wrote:
On 19/Jun/20 09:20, Radu-Adrian Feurdean wrote:
A whole ocean of "datacenter" hardware, from pretty much evey vendor.
You mean the ones deliberately castrated so that we can create a specific "DC vertical", even if they are, pretty much, the same box a service provider will buy, just given a darker color so it can glow more brightly in the data centre night?
Yes, exactly that one. Which also happens to spill outside the DC area, because the main "vertical" allows it to be sold at lower prices.
On Wed, Jun 17, 2020 at 11:40 AM Dave Bell <me@geordish.org> wrote:
On Wed, 17 Jun 2020 at 18:42, Saku Ytti <saku@ytti.fi> wrote:
Hey,
Why do we really need SR? Be it SR-MPLS or SRv6 or SRv6+?
I don't like this, SR-MPLS and SRv6 are just utterly different things to me, and no answer meaningfully applies to both.
I don't understand the point of SRv6. What equipment can support IPv6 routing, but can't support MPLS label switching?
I'm a big fan of SR-MPLS however.
One of the advantages cited for SRv6 over MPLS is that the packet contains
a record of where it has been.
One of the advantages cited for SRv6 over MPLS is that the packet contains
a record of where it has been.
Not really ... packets are not tourists in a bus. First there are real studies proving that most large production networks for the goal of good TE only need to place 1, 2 or 3 hops to traverse through. Rest is the shortest path between those hops. Then even if you place those node SIDs you have no control which interfaces are chosen as outbound. There is often more then one IGP ECMP path in between. You would need to insert adj. SIDs which does require pretty fine level of controller's capabilities to start with. I just hope that no one sane proposes that now all packets should get encapsulated in a new IPv6 header while entering a transit ISP network and carry long list of hop by hop adjacencies it is to travel by. Besides even if it would it would be valid only within given ASN and had no visibility outside. Thx, R.
On 20/Jun/20 00:41, Anoop Ghanwani wrote:
One of the advantages cited for SRv6 over MPLS is that the packet contains a record of where it has been.
I can't see how advantageous that is, or how possible it would be to implement, especially for inter-domain traffic. Mark.
----- On Jun 20, 2020, at 2:27 PM, Mark Tinka <mark.tinka@seacom.mu> wrote: Hi Mark,
On 20/Jun/20 00:41, Anoop Ghanwani wrote:
One of the advantages cited for SRv6 over MPLS is that the packet contains a record of where it has been.
I can't see how advantageous that is,
That will be very advantageous in a datacenter environment, or any other environment dealing with a lot of ECMP paths. I can't tell you how often during my eBay time I've been troubleshooting end-to-end packetloss between hosts in two datacenters where there were at least 10 or more layers of up to 16 way ECMP between them. Having a record of which path is being taken by a packet is very helpful to determine the one with a crappy transceiver.
or how possible it would be to implement,
That work is already underway, albeit not specifically for MPLS. For example, I've worked with an experimental version of In-Band Network Telemetry (INT) as described in this draft: https://tools.ietf.org/html/draft-kumar-ippm-ifa-02 I even demonstrated a very basic implementatoin during SuperCompute 19 in Denver last year. Most people who were interested in the demo were academics however, probably because it wasn't a real networking event. Note that there are several caveats that come with this draft and previous versions, and that it is still very much work in progress. But the potential is huge, at least in the DC.
especially for inter-domain traffic.
That's a different story, but not entirely impossible. A probe packet can be sent across AS borders, and as long as the two NOCs are cooperating, the entire path can be reconstructed. Thanks, Sabri
On 21/Jun/20 00:54, Sabri Berisha wrote:
That will be very advantageous in a datacenter environment, or any other environment dealing with a lot of ECMP paths.
I can't tell you how often during my eBay time I've been troubleshooting end-to-end packetloss between hosts in two datacenters where there were at least 10 or more layers of up to 16 way ECMP between them. Having a record of which path is being taken by a packet is very helpful to determine the one with a crappy transceiver.
That work is already underway, albeit not specifically for MPLS. For example, I've worked with an experimental version of In-Band Network Telemetry (INT) as described in this draft: https://tools.ietf.org/html/draft-kumar-ippm-ifa-02
I even demonstrated a very basic implementatoin during SuperCompute 19 in Denver last year. Most people who were interested in the demo were academics however, probably because it wasn't a real networking event.
Note that there are several caveats that come with this draft and previous versions, and that it is still very much work in progress. But the potential is huge, at least in the DC.
Alright, we'll wait and see, then.
That's a different story, but not entirely impossible. A probe packet can be sent across AS borders, and as long as the two NOCs are cooperating, the entire path can be reconstructed.
Yes, for once-off troubleshooting, I suppose that would work. My concern is if it's for normal day-to-day operations. But who knows, maybe someone will propose that too :-). Mark.
On Jun 20, 2020, at 2:27 PM, Mark Tinka <mark.tinka@seacom.mu> wrote:
On 20/Jun/20 00:41, Anoop Ghanwani wrote:
One of the advantages cited for SRv6 over MPLS is that the packet contains a record of where it has been.
I can't see how advantageous that is, or how possible it would be to implement, especially for inter-domain traffic.
Mark.
Since the packet is essentially source-routed, and the labels aren’t popped off the way they are in MPLS, but preserved in the hop by hop headers (AIUI), the implementation isn’t particularly difficult. Owen
On 17/06/2020 18:38, Saku Ytti wrote:
Why do we really need SR? Be it SR-MPLS or SRv6 or SRv6+? I don't like this, SR-MPLS and SRv6 are just utterly different things to me, and no answer meaningfully applies to both.
I would ask, why do we need LDP, why not use IGP to carry labels?
Less state, protocols, SLOC, cost, bug surface
And we get more features to boot, with LDP if you want LFA, you need to form tLDP to every Q-space node, on top of your normal LDP, because you don't know label view from anyone else but yourself. With SR by nature you know the label view for everyone, thus you have full LFA coverage for free, by-design. Also by-design IGP/LDP Sync.
So no need to justify it by any magic new things, it's just a lot simpler than LDP, you don't need to need new things to justify SR-MPLS, you need to want to do existing things while reducing complexity and state.
Unsurprisingly, there would be no way on Earth that I could have said that better, so you shall find only loud cheering from over here. -- Tom
On 17/Jun/20 19:38, Saku Ytti wrote:
I don't like this, SR-MPLS and SRv6 are just utterly different things to me, and no answer meaningfully applies to both.
I know they are different, but that was on purpose, because even with SR-MPLS, there are a couple of things to consider: * IOS XR does not appear to support SR-OSPFv3. * IOS XE does not appear to support SR-ISISv6. * IOS XE does not appear to support SR-OSPFv3. * Junos does not appear to support SR-OSPFv3. * MPLS/VPN service signaling in IPv6-only networks also has gaps in SR. So for networks that run OSPF and don't run Juniper, they'd need to move to IS-IS in order to have SR forward IPv6 traffic in an MPLS encapsulation. Seems like a bit of an ask. Yes, code needs to be written, which is fine by me, as it also does for LDPv6.
I would ask, why do we need LDP, why not use IGP to carry labels?
Less state, protocols, SLOC, cost, bug surface
I'd be curious to understand what bugs you've suffered with LDP in the last 10 or so years, that likely still have open tickets. Yes, we all love less state, I won't argue that. But it's the same question that is being asked less and less with each passing year - what scales better in 2020, OSPF or IS-IS. That is becoming less relevant as control planes keep getting faster and cheaper. I'm not saying that if you are dealing with 100,000 T-LDP sessions you should not consider SR, but if you're not, and SR still requires a bit more development (never mind deployment experience), what's wrong with having LDPv6? If it makes near-as-no-difference to your control plane in 2020 or 2030 as to whether your 10,000-node network is running LDP or SR, why not have the choice?
And we get more features to boot, with LDP if you want LFA, you need to form tLDP to every Q-space node, on top of your normal LDP, because you don't know label view from anyone else but yourself. With SR by nature you know the label view for everyone, thus you have full LFA coverage for free, by-design. Also by-design IGP/LDP Sync.
So no need to justify it by any magic new things, it's just a lot simpler than LDP, you don't need to need new things to justify SR-MPLS, you need to want to do existing things while reducing complexity and state.
Again, it's a question of scale and requirements. Some large networks don't run any RSVP, while some small networks do. I'm not saying let's not do SR; but for those who want something mature, and for those who want something new, I don't see a reason why the choice can't be left up to the operator. Routers, in 2020, still ship with RIPv2. If anyone wants to use it (as I am sure there are some that do), who are we to stand in their way, if it makes sense for them? Mark.
On Thu, 18 Jun 2020 at 01:17, Mark Tinka <mark.tinka@seacom.mu> wrote:
IOS XR does not appear to support SR-OSPFv3. IOS XE does not appear to support SR-ISISv6. IOS XE does not appear to support SR-OSPFv3. Junos does not appear to support SR-OSPFv3.
The IGP mess we are in is horrible, but I can't blame SR for it. It's really unacceptable we spend NRE hours developing 3 identical IGP (OSPFv2, OSPFv3, ISIS). We all pay a 300-400% premium for a single IGP. In a sane world, we'd retire all of them except OSPFv3 and put all NRE focus on there or move some of the NRE dollars to some other problems we have, perhaps we would have room to support some different non-djikstra IGP. In a half sane world, IGP code, 90% of your code would be identical, then you'd have adapter/ospfv2 adapter/ospfv3 adapter/isis which translates internal struct to wire and wire to internal struct. So any features you code, come for free to all of them. But no one is doing this, it's 300% effort, and we all pay a premium for that. In a quarter sane world we'd have some CIC, common-igp-container RFC and then new features like SR would be specified as CIC-format, instead of OSPFv2, OSPFv3, ISIS and BGP. Then each OSPFv2, OSPFv3, ISIS and BGP would have CIC-to-x RFC. So people introducing new IGP features do not need to write 4 drafts, one is enough. I would include IPv4+IPv6 my-igp-of-choice SR in my RFP. Luckily ISIS is supported on platforms I care about for IPV4+IPV6, so I'm already there.
MPLS/VPN service signaling in IPv6-only networks also has gaps in SR.
I don't understand this.
So for networks that run OSPF and don't run Juniper, they'd need to move to IS-IS in order to have SR forward IPv6 traffic in an MPLS encapsulation. Seems like a bit of an ask. Yes, code needs to be written, which is fine by me, as it also does for LDPv6.
And it's really just adding TLV, if it already does IPv4 all the infra should be in place, only thing missing is transporting the information. Adding TLV to IGP is a lot less work than LDPv6.
I'd be curious to understand what bugs you've suffered with LDP in the last 10 or so years, that likely still have open tickets.
3 within a year. - PR1436119 - PR1428081 - PR1416032 I don't have IOS-XR LDP bugs within a year, but we had a bunch back when going from 4 to 5. And none of these are cosmetic, these are blackholing. I'm not saying LDP is bad, it's just, of course more code lines you exercise more bugs you see. But yes, LDP has a lot of bug surface compared to SR, but in _your network_ lot of that bug surface and complexity is amortised complexity. So status quo bias is strong to keep running LDP, it is simpler _NOW_ as a lot of the tax has been paid and moving to an objectively simpler solution carries risk, as its complexity is not amortised yet.
Yes, we all love less state, I won't argue that. But it's the same question that is being asked less and less with each passing year - what scales better in 2020, OSPF or IS-IS. That is becoming less relevant as control planes keep getting faster and cheaper.
I don't think it ever was relevant.
I'm not saying that if you are dealing with 100,000 T-LDP sessions you should not consider SR, but if you're not, and SR still requires a bit more development (never mind deployment experience), what's wrong with having LDPv6? If it makes near-as-no-difference to your control plane in 2020 or 2030 as to whether your 10,000-node network is running LDP or SR, why not have the choice?
I can't add anything to the upside of going from LDP to SR that I've not already said. You get more by spending less, it's win:win. Only reason to stay in LDP is status quo bias which makes short term sense.
Routers, in 2020, still ship with RIPv2. If anyone wants to use it (as I am sure there are some that do), who are we to stand in their way, if it makes sense for them?
RIP might make sense in some deployments, because it's essentially stateless (routes age out, no real 'session') so if you have 100k VM per router that you need to support and you want dynamic routing, RIP might be the least resistance solution with the highest scale. Timing wheels should help it scale and maintain great number of timers. -- ++ytti
On 18/Jun/20 07:25, Saku Ytti wrote:
The IGP mess we are in is horrible, but I can't blame SR for it. It's really unacceptable we spend NRE hours developing 3 identical IGP (OSPFv2, OSPFv3, ISIS). We all pay a 300-400% premium for a single IGP.
In a sane world, we'd retire all of them except OSPFv3 and put all NRE focus on there or move some of the NRE dollars to some other problems we have, perhaps we would have room to support some different non-djikstra IGP.
In a half sane world, IGP code, 90% of your code would be identical, then you'd have adapter/ospfv2 adapter/ospfv3 adapter/isis which translates internal struct to wire and wire to internal struct. So any features you code, come for free to all of them. But no one is doing this, it's 300% effort, and we all pay a premium for that.
In a quarter sane world we'd have some CIC, common-igp-container RFC and then new features like SR would be specified as CIC-format, instead of OSPFv2, OSPFv3, ISIS and BGP. Then each OSPFv2, OSPFv3, ISIS and BGP would have CIC-to-x RFC. So people introducing new IGP features do not need to write 4 drafts, one is enough.
While I don't have a real opinion on how to fix the IGP mess, the point is we sit with it now. Getting all these fixed is going to increase the bug surface area for some time to come as both vendors and operators work the kinks out, in addition to SR's own kinks. Yes, it's all par for the course for new features, which is why I'd also like to have an alternative that has been baked in for many years to give me an option for stability, as we roll the new kid out. I probably will deploy SR-MPLS at some point in my lifetime, but I'm not feeling awfully comfortable to do so right now; and yet I do need MPLSv6 forwarding.
I would include IPv4+IPv6 my-igp-of-choice SR in my RFP. Luckily ISIS is supported on platforms I care about for IPV4+IPV6, so I'm already there.
Which is great for you, me, and a ton of other folk that run IS-IS on Juniper. What about folk that don't have Juniper, or run OSPF? I know, not your or my problem, but the Internet isn't just a few networks.
I don't understand this.
I mean the same gaps that exist in RFC 7439, for would-be IPv6-only MPLS networks.
And it's really just adding TLV, if it already does IPv4 all the infra should be in place, only thing missing is transporting the information. Adding TLV to IGP is a lot less work than LDPv6.
What we theorize as "should be easy" can turn out to be a whole discussion with the vendors about it being months or years of work. Not being inside their meeting rooms, I can't quite challenge how they present the task. Fundamentally, LDPv6 already has 5+ years in implementation (and LDPv4 is 20 years old), inter-op issues seem to be mostly fixed, and for what we need it to do, it's working very well. There are probably as many networks running SR-MPLS as there are running LDPv6, likely fewer if your SR deployment doesn't yet support OSPFv3 or SR-ISISv6. I concede that for some networks looking to go SR-MPLS, label distribution state reduction is probably higher up on the agenda than MPLSv6 forwarding. For me, I'd like the option to have both, and decide whether my network is in a position to handle the additional state required for LDPv6, if I feel that I'd prefer to deal with a protocol that has had more exposure to the sun. Ultimately, boxes with LDPv6 have been shipping for some time, and we have a ton of them deployed and running for a while now. If it comes down to kicking out the 20% that won't support it because of an all-or-nothing vendor approach on a platform without full SR-MPLS support for all IGP's, it is what it is.
3 within a year. - PR1436119 - PR1428081 - PR1416032
I don't have IOS-XR LDP bugs within a year, but we had a bunch back when going from 4 to 5. And none of these are cosmetic, these are blackholing.
I'm not saying LDP is bad, it's just, of course more code lines you exercise more bugs you see.
But yes, LDP has a lot of bug surface compared to SR, but in _your network_ lot of that bug surface and complexity is amortised complexity. So status quo bias is strong to keep running LDP, it is simpler _NOW_ as a lot of the tax has been paid and moving to an objectively simpler solution carries risk, as its complexity is not amortised yet.
And FWIW, if some operators are willing to benefit from all the experience that has gone into developing and maintaining LDP, while we let SR settle down, I don't see why that choice shouldn't be there. I'm not saying it should be an SR vs. LDP debate like it was BGP-signaling vs. LDP-signaling for VPLS 12+ years ago. All I'm saying is for those who want to go bleeding edge with SR, go for it. For those who prefer to gracefully transition toward SR over time by settling on LDP that has been in the field for a minute, go for it too. I won't claim to know whether LDP or SR have a smaller or larger bug surface area. What I do know is that there will be plenty of bugs for SR, as there have been for MPLS and all related protocols in the last 20+ years. From my side, I'd prefer to give SR the time it needs to get all of its Vitamin D, but don't oppose anyone that prefers to deploy it.
I can't add anything to the upside of going from LDP to SR that I've not already said. You get more by spending less, it's win:win. Only reason to stay in LDP is status quo bias which makes short term sense.
I can't argue the usefulness of reducing label distribution state in MPLS. Heck, that is what got me excited about SR back in 2013, and also what caused me to pump the brakes on the noise I was making to vendors about developing LDPv6 (which started in 2008), because I was finally going to get native MPLSv6 forwarding in SR without all the LDP/RSVP fluff. But, things took their own turn, and with the IGP mess that it currently is, we are where we are. Thankfully, some vendors did develop LDPv6 anyway, so we got MPLSv6 in the end as SR was still in the embryo. If I'm still in the game in half-a-decade from now or so, I will very likely dump LDP and move to SR-MPLS. I'm just not too comfortable doing so now because IGP support is not where it needs to be, and it still has to through its own life cycle of bugs and fixes, which will be quite an effort as global deployment is still far behind LDP and RSVP.
RIP might make sense in some deployments, because it's essentially stateless (routes age out, no real 'session') so if you have 100k VM per router that you need to support and you want dynamic routing, RIP might be the least resistance solution with the highest scale. Timing wheels should help it scale and maintain great number of timers.
I guess my point was the vendors won't be dumping RIP, even if general conensus is to avoid it whenever possible. If I'm not concerned about LDP state, and protocol stability is more important to me in the near-to-medium term, we'd be remiss to start a culture of taking that choice away. Because the next time vendors get bored with what they've built and sold and decide that SR or some other feature has seen enough light of day, let's dream up something else to shout about between the 2030 - 2040 decade, they'll have the had the experience of cornering operators into making rash decisions, and they'd never let us forget it. Mark.
On Thu, 18 Jun 2020 at 10:13, Mark Tinka <mark.tinka@seacom.mu> wrote:
Which is great for you, me, and a ton of other folk that run IS-IS on Juniper. What about folk that don't have Juniper, or run OSPF?
I know, not your or my problem, but the Internet isn't just a few networks.
Yes work left to be done. Ultimately the root problem is, no one cares about IPv6. But perhaps work with vendors in parallel to LDPv6 to get them to fix OSPFv3 and/or ISIS.
I'm not saying it should be an SR vs. LDP debate like it was BGP-signaling vs. LDP-signaling for VPLS 12+ years ago. All I'm saying
FWIW I am definitely saying that, and it should be IGP+BGP. I do accept and realise a lot of platforms only did and do Martini not Kompella, so reality isn't quite there. -- ++ytti
On 18/Jun/20 09:30, Saku Ytti wrote:
Yes work left to be done. Ultimately the root problem is, no one cares about IPv6. But perhaps work with vendors in parallel to LDPv6 to get them to fix OSPFv3 and/or ISIS.
Yes, this. Vendor feedback for those not supporting LDPv6 is that there is no demand for it. And like I said in the previous thread, LDPv6 demand is not about LDPv6, it's about IPv6. If the majority of the high-paying vendors' favorite customers that pay for CGN's continue to do so, what incentive do they have to ask for IPv6. The T-Mobile US's of the world are few and far between, sadly. I suppose I would not be unwilling to push the vendors to support SR-OSPFv3 and SR-ISISv6 as I am also pushing them to support LDPv6 where it is lacking, because at some point in the future, I do want to deploy SR-MPLS in the same way I envisioned doing so back in 2014. I just need to take it on a few dates first before I bring it home to meet the folks :-).
FWIW I am definitely saying that, and it should be IGP+BGP. I do accept and realise a lot of platforms only did and do Martini not Kompella, so reality isn't quite there.
That was me in 2013/2014. Dump LDP, dump RSVP, get SR deployed, forward IPv4 natively in MPLSv4, and IPv6 natively in MPLSv6. But life happened. Nonetheless, I will go SR-MPLS in many years to come, after I'm feeling comfortable about it. That's a promise. But until then, I'd like trusted, stable IPv4-IPv6 MPLS forwarding parity. I have never cared much for VPLS because I thought it was a very messy piece of tech. from Day 1. And while EVPN makes more sense, for our market, more than 98% of the traffic we sell is IP-based, so we have no demand for mp2mp Ethernet VPN's. But for those that adore VPLS (or EVPN), let them have the choice of LDP or BGP, which both Cisco and Juniper, after years of muscle-flexing, both ended up agreeing on anyway, despite all the fuss. So the LDPv6 vs. SR-MPLS vs. SRv6 vs. SRv6+ posturing is a rehash of those LDP vs. BGP days, which just wastes everyone's time. Mark.
From: NANOG <nanog-bounces@nanog.org> On Behalf Of Mark Tinka Sent: Thursday, June 18, 2020 8:13 AM
There are probably as many networks running SR-MPLS as there are running LDPv6, likely fewer if your SR deployment doesn't yet support OSPFv3 or SR- ISISv6. I concede that for some networks looking to go SR-MPLS, label distribution state reduction is probably higher up on the agenda than MPLSv6 forwarding. For me, I'd like the option to have both, and decide whether my network is in a position to handle the additional state required for LDPv6, if I feel that I'd prefer to deal with a protocol that has had more exposure to the sun.
You do have the LDP vs SR choice (in v4 anyways) yes there's not a good 1:1 feature parity with v6, but the important point is the current state is not the end state, this is a pretty dynamic industry that I'm sure is converging/evolving towards a v4:v6 parity, however the pace may be, which is understandable considering the scope of ground to be covered. Yes you're right in acknowledging that we're not living in a perfect world and that choices are limited, but it's been like that since ever yet we managed to thrive by analysing our options and striving for optimal strategies year by year. adam
On 18/Jun/20 13:23, adamv0025@netconsultings.com wrote:
You do have the LDP vs SR choice (in v4 anyways) yes there's not a good 1:1 feature parity with v6, but the important point...
But the lack of IPv4/IPv6 parity is a crucial one. There is only so long we can stretch IPv4, if one can still manage the tangible and intangible costs of doing so. But that's for another discussion.
is the current state is not the end state, this is a pretty dynamic industry that I'm sure is converging/evolving towards a v4:v6 parity, however the pace may be, which is understandable considering the scope of ground to be covered.
Which I am fine with - if you give me a time line to say LDPv6, SR-OSPFv3 and SR-ISISv6 will be available on X date, I can manage my operation and expectations accordingly. But if you say, "No LDPv6, no SR-OSPFv3, no SR-ISISv6... only SRv6", then that's an entirely different issue. The good news is there currently is choice on the matter, but upending hundreds or thousands of boxes to prove that point should really be a last resort, as there are more pressing things we all have to deal with.
Yes you're right in acknowledging that we're not living in a perfect world and that choices are limited, but it's been like that since ever yet we managed to thrive by analysing our options and striving for optimal strategies year by year.
We can thank NAT44, CIDR, DHCP and PPPoE for that strategy over the years :-). IPv6 is the future, and at some point, we'll have to stop hiding from it. Mark.
From: Mark Tinka <mark.tinka@seacom.mu> Sent: Thursday, June 18, 2020 12:51 PM
On 18/Jun/20 13:23, adamv0025@netconsultings.com wrote:
is the current state is not the end state, this is a pretty dynamic industry that I'm sure is converging/evolving towards a v4:v6 parity, however the pace may be, which is understandable considering the scope of ground to be covered.
Which I am fine with - if you give me a time line to say LDPv6, SR-OSPFv3 and SR-ISISv6 will be available on X date, I can manage my operation and expectations accordingly.
But if you say, "No LDPv6, no SR-OSPFv3, no SR-ISISv6... only SRv6", then that's an entirely different issue.
The good news is there currently is choice on the matter, but upending hundreds or thousands of boxes to prove that point should really be a last resort, as there are more pressing things we all have to deal with.
Hence our current strategy is to stay on IPv4 control-plane (and IPv4 management plane) as it suits, and for the foreseeable future will suite, all our needs (which are to transport v4&v6 data packets via L2&L3 MPLS VPN services), there are simply more important projects than to experiment with v6 control-plane, like for instance perfecting/securing the v6 customer facing services (delivered over the underlying v4 signalled MPLS infrastructure, that no customer really cares about). But I understand your frustrations case it seems like you're taking the bullet for us late adopters and in a sense you are, cause say in 10 years from now when I decide to migrate to v6 control-plane and management-plane as then it might be viewed as common courtesy, it will be all there on a silver plate waiting for me allowing for a relatively effortless and painless move. All thanks to you fighting the good fight today.
Yes you're right in acknowledging that we're not living in a perfect world and that choices are limited, but it's been like that since ever yet we managed to thrive by analysing our options and striving for optimal strategies year by year.
We can thank NAT44, CIDR, DHCP and PPPoE for that strategy over the years :-).
IPv6 is the future, and at some point, we'll have to stop hiding from it.
And I'd say the future is now, cause there is an actual need for v6 services. But need for v6 control & management plane? - It's not like operators are losing business opportunities not having that. (they might even be viewed as conservative->stable, which might be preferred by some customers). adam
On 18/Jun/20 14:30, adamv0025@netconsultings.com wrote:
Hence our current strategy is to stay on IPv4 control-plane (and IPv4 management plane) as it suits, and for the foreseeable future will suite, all our needs (which are to transport v4&v6 data packets via L2&L3 MPLS VPN services), there are simply more important projects than to experiment with v6 control-plane, like for instance perfecting/securing the v6 customer facing services (delivered over the underlying v4 signalled MPLS infrastructure, that no customer really cares about).
Fair enough.
But I understand your frustrations case it seems like you're taking the bullet for us late adopters and in a sense you are, cause say in 10 years from now when I decide to migrate to v6 control-plane and management-plane as then it might be viewed as common courtesy, it will be all there on a silver plate waiting for me allowing for a relatively effortless and painless move. All thanks to you fighting the good fight today.
You better hope and pray I don't run out wine. Equipment manufacturers make me drink, and I like my wine :-).
And I'd say the future is now, cause there is an actual need for v6 services. But need for v6 control & management plane? - It's not like operators are losing business opportunities not having that. (they might even be viewed as conservative->stable, which might be preferred by some customers).
Well, the other way to look at it, especially if you are a Broadband or mobile network operator, is what your plan is when you can no longer stretch the IPv4 you have, can no longer obtain IPv4 from an RIR, and can't afford to buy IPv4 on the open market. For mobile operators, paying US$50 million/year in CGN line cards and licensing is not even a rounding error on the books. But the telco space has been under pressure for some time now, further amplified and accelerated this Coronavirus pandemic. Even though mobile networks are ATM machines printing money for the shareholders, they probably made more money in the days of SMS than they do now building and selling 4G/5G packet cores. At some point, that US$50 million/year is going to start getting some ex-co and Board level visibility, as capex spend begins to pinch revenue because of the data demands of subscribers, and the ever-falling ARPU's to go along with it. Perhaps, at that point, massive IPv6 deployment in the mobile space is what will wake everyone else up, as the race to grab on to every $$ tightens. Mark.
From: Saku Ytti Sent: Thursday, June 18, 2020 6:26 AM
On Thu, 18 Jun 2020 at 01:17, Mark Tinka <mark.tinka@seacom.mu> wrote:
Yes, we all love less state, I won't argue that. But it's the same question that is being asked less and less with each passing year - what scales better in 2020, OSPF or IS-IS. That is becoming less relevant as control planes keep getting faster and cheaper.
I don't think it ever was relevant.
In 99% of cases, there are cases however where supporting 1M+ routes in IGP is one of the viable options to consider, or running multi-100k of LSPs through a core node... But these are core MPLS networks that have no boundaries cause these literally wrap around the globe, and access to custom code. adam
Hi Saku, To your IGP point let me observe that OSPF runs over IP and ISIS does not. That is first fundamental difference. There are customers using both all over the world and therefore any suggestion to just use OSPFv3 is IMHO quite unrealistic. Keep in mind that OSPF hierarchy is 2 (or 3 with super area) while in IETF there is ongoing work to extend ISIS to 8 levels. There is a lot of fundamental differences between those two (or three) IGPs and I am sure many folks on the lists know them. Last there is a lot of enterprise networks happily using IPv4 RFC1918 all over their global WAN and DCs infrastructure and have no reason to deploy IPv6 there any time soon. If you are serious about converging to a single IGP I would rather consider look towards OpenR type of IGP architecture with message bus underneath. Thx, R. On Thu, Jun 18, 2020 at 7:26 AM Saku Ytti <saku@ytti.fi> wrote:
On Thu, 18 Jun 2020 at 01:17, Mark Tinka <mark.tinka@seacom.mu> wrote:
IOS XR does not appear to support SR-OSPFv3. IOS XE does not appear to support SR-ISISv6. IOS XE does not appear to support SR-OSPFv3. Junos does not appear to support SR-OSPFv3.
The IGP mess we are in is horrible, but I can't blame SR for it. It's really unacceptable we spend NRE hours developing 3 identical IGP (OSPFv2, OSPFv3, ISIS). We all pay a 300-400% premium for a single IGP.
In a sane world, we'd retire all of them except OSPFv3 and put all NRE focus on there or move some of the NRE dollars to some other problems we have, perhaps we would have room to support some different non-djikstra IGP.
In a half sane world, IGP code, 90% of your code would be identical, then you'd have adapter/ospfv2 adapter/ospfv3 adapter/isis which translates internal struct to wire and wire to internal struct. So any features you code, come for free to all of them. But no one is doing this, it's 300% effort, and we all pay a premium for that.
In a quarter sane world we'd have some CIC, common-igp-container RFC and then new features like SR would be specified as CIC-format, instead of OSPFv2, OSPFv3, ISIS and BGP. Then each OSPFv2, OSPFv3, ISIS and BGP would have CIC-to-x RFC. So people introducing new IGP features do not need to write 4 drafts, one is enough.
I would include IPv4+IPv6 my-igp-of-choice SR in my RFP. Luckily ISIS is supported on platforms I care about for IPV4+IPV6, so I'm already there.
MPLS/VPN service signaling in IPv6-only networks also has gaps in SR.
I don't understand this.
So for networks that run OSPF and don't run Juniper, they'd need to move to IS-IS in order to have SR forward IPv6 traffic in an MPLS encapsulation. Seems like a bit of an ask. Yes, code needs to be written, which is fine by me, as it also does for LDPv6.
And it's really just adding TLV, if it already does IPv4 all the infra should be in place, only thing missing is transporting the information. Adding TLV to IGP is a lot less work than LDPv6.
I'd be curious to understand what bugs you've suffered with LDP in the last 10 or so years, that likely still have open tickets.
3 within a year. - PR1436119 - PR1428081 - PR1416032
I don't have IOS-XR LDP bugs within a year, but we had a bunch back when going from 4 to 5. And none of these are cosmetic, these are blackholing.
I'm not saying LDP is bad, it's just, of course more code lines you exercise more bugs you see.
But yes, LDP has a lot of bug surface compared to SR, but in _your network_ lot of that bug surface and complexity is amortised complexity. So status quo bias is strong to keep running LDP, it is simpler _NOW_ as a lot of the tax has been paid and moving to an objectively simpler solution carries risk, as its complexity is not amortised yet.
Yes, we all love less state, I won't argue that. But it's the same question that is being asked less and less with each passing year - what scales better in 2020, OSPF or IS-IS. That is becoming less relevant as control planes keep getting faster and cheaper.
I don't think it ever was relevant.
I'm not saying that if you are dealing with 100,000 T-LDP sessions you should not consider SR, but if you're not, and SR still requires a bit more development (never mind deployment experience), what's wrong with having LDPv6? If it makes near-as-no-difference to your control plane in 2020 or 2030 as to whether your 10,000-node network is running LDP or SR, why not have the choice?
I can't add anything to the upside of going from LDP to SR that I've not already said. You get more by spending less, it's win:win. Only reason to stay in LDP is status quo bias which makes short term sense.
Routers, in 2020, still ship with RIPv2. If anyone wants to use it (as I am sure there are some that do), who are we to stand in their way, if it makes sense for them?
RIP might make sense in some deployments, because it's essentially stateless (routes age out, no real 'session') so if you have 100k VM per router that you need to support and you want dynamic routing, RIP might be the least resistance solution with the highest scale. Timing wheels should help it scale and maintain great number of timers.
-- ++ytti
On 18/Jun/20 12:28, Robert Raszuk wrote:
To your IGP point let me observe that OSPF runs over IP and ISIS does not. That is first fundamental difference. There are customers using both all over the world and therefore any suggestion to just use OSPFv3 is IMHO quite unrealistic.
Are you saying that OSPF houses that want IPv6 should just move to IS-IS. Don't get me wrong, I support that very much as I think IS-IS is a great IGP. That said, while it's good to convince OSPF operators to consider IS-IS, it's not our place to force them to use it. Also, OSPFv3-only for your dual-stack IGP needs is a supported capability. Last time I tested it in Juniper in 2010/2011, it worked well. I don't know if anyone is actually running IPv4 and IPv6 on OSPFv3 only, but it does work.
Keep in mind that OSPF hierarchy is 2 (or 3 with super area) while in IETF there is ongoing work to extend ISIS to 8 levels. There is a lot of fundamental differences between those two (or three) IGPs and I am sure many folks on the lists know them.
15+ years ago, I'd have said that one protocol may have been suited to a specific task than another due to the control plane limitations of the day. In 2020, with the state-of-the-art of control planes today, it near as makes no difference, IMHO.
Last there is a lot of enterprise networks happily using IPv4 RFC1918 all over their global WAN and DCs infrastructure and have no reason to deploy IPv6 there any time soon.
No wonder the vendors aren't seeing any LDPv6, SR-ISISv6 or SR-OSPFv3 demand :-). Mark.
On Thu, 18 Jun 2020 at 13:28, Robert Raszuk <robert@raszuk.net> wrote:
To your IGP point let me observe that OSPF runs over IP and ISIS does not. That is first fundamental difference. There are customers using both all over the world and therefore any suggestion to just use OSPFv3 is IMHO quite unrealistic. Keep in mind that OSPF hierarchy is 2 (or 3 with super area) while in IETF there is ongoing work to extend ISIS to 8 levels. There is a lot of fundamental differences between those two (or three) IGPs and I am sure many folks on the lists know them. Last there is a lot of enterprise networks happily using IPv4 RFC1918 all over their global WAN and DCs infrastructure and have no reason to deploy IPv6 there any time soon.
I view the 802.3 and CLNS as liability, not an asset. People who actually route CLNS are a dying breed, think just DCN of a legacy optical. Many platforms have no facilities to protect ISIS, any connected attacker can kill the box. Nokia handles generated packets classification by assigning DSCP value to application then DSCP to forwarding-class, which precludes from configuring ISIS qos. Very few people understand how ISIS works before ISIS PDU is handed to them, world from 802.3 to that is largely huge pile of hacks, instead of complete CLNS stack implementation. There is no standard way to send large frames over 802.3, so there is non-standard way to encap ISIS for those links. Also due to lack of LSP roll-over, ISIS is subject to a horrible attack vector which is very difficult to troubleshoot and solve. -- ++ytti
On Wed, 17 Jun 2020 at 23:19, Mark Tinka <mark.tinka@seacom.mu> wrote:
Yes, we all love less state, I won't argue that. But it's the same question that is being asked less and less with each passing year - what scales better in 2020, OSPF or IS-IS. That is becoming less relevant as control planes keep getting faster and cheaper.
I'm not saying that if you are dealing with 100,000 T-LDP sessions you should not consider SR, but if you're not, and SR still requires a bit more development (never mind deployment experience), what's wrong with having LDPv6? If it makes near-as-no-difference to your control plane in 2020 or 2030 as to whether your 10,000-node network is running LDP or SR, why not have the choice?
I'm going to kick the nest in the other direction now :D ... There would be no need to massively scale an IGP or worry about running LDPv4 + LDv6 or SR MPLS if we had put more development time into MPLS over UDP. I think it's a great technology which solves a lot of problems and I've been itching to deploy it for ages now, but vendor support for it is nowhere near the level of MPLS over Ethernet. Cheers, James.
On Wed, 17 Jun 2020 at 18:08, Mark Tinka <mark.tinka@seacom.mu> wrote:
Hi all.
When the whole SR concept was being first dreamed up, I was mildly excited about it. But then real life happened and global deployment (be it basic SR-MPLS or SRv6) is what it is, and I became less excited. This was back in 2015.
All the talk about LDPv6 this and last week has had me reflecting a great deal on where we are, as an industry, in why we are having to think about SR and all its incarnations.
So, let me be the one that stirs up the hornets' nest...
Why do we really need SR? Be it SR-MPLS or SRv6 or SRv6+?
I am clearly very far behind on my emails, but of the emails I've read so far in this thread though you have mentioned at least twice: On Wed, 17 Jun 2020 at 18:08, Mark Tinka <mark.tinka@seacom.mu> wrote:
What I am less enthused about is being forced
On Wed, 17 Jun 2020 at 23:22, Mark Tinka <mark.tinka@seacom.mu> wrote:
it tastes funny when you are forced
Mark, does someone have a gun to your head? Are you in trouble? Blink 63 times for yes, 64 times for no ;) Cheers, James.
On Tue, 30 Jun 2020 at 22:07, Mark Tinka <mark.tinka@seacom.com> wrote:
On 30/Jun/20 20:37, James Bensley wrote:
Mark, does someone have a gun to your head? Are you in trouble? Blink 63 times for yes, 64 times for no ;)
You're pretty late to this party, mate...
True, but what's changed in two weeks with regards to LDv6 and SR? What was your use case / requirement for LDv6 - to remove the full table v6 feed from your core or to remove IPv4 from your IGP or both? Cheers, James.
On 1/Jul/20 09:10, James Bensley wrote:
True, but what's changed in two weeks with regards to LDv6 and SR?
What was your use case / requirement for LDv6 - to remove the full table v6 feed from your core or to remove IPv4 from your IGP or both?
Give me a year to work this and report back, hopefully at a NANOG lectern near you :-). Mark.
participants (12)
-
adamv0025＠netconsultings.com -
Anoop Ghanwani -
Dave Bell -
James Bensley -
Mark Tinka -
Mark Tinka -
Owen DeLong -
Radu-Adrian Feurdean -
Robert Raszuk -
Sabri Berisha -
Saku Ytti -
Tom Hill