Ah yes, I would say LDPv6 and/or SR/MPLS IPv6. SRv6 reads like a science project. Either way, I would like to achieve a full IPv6 control plane. On Wed, Jun 10, 2020 at 2:46 PM Saku Ytti <saku@ytti.fi> wrote:

I'm pretty sure that one or more of Mark, Gert or Tim are thinking SR/MPLS IPv6 when they say SRv6?

No one in their right minds thinks SRv6 is a good idea, terrible snake oil and waste of NRE. SR/MPLS IPv6 of course is terrific.

LDPv6 and SRv6 seem like an odd couple, LDPv6 SR/MPLS IPv6 seem far more reasonable couple to choose from. I have my favorite.

On Wed, 10 Jun 2020 at 21:32, Tim Durack <tdurack@gmail.com> wrote:

...

I would take either LDPv6 or SRv6 - but also need L3VPN (and now EVPN)

re-wired to use IPv6 NH.

...

I have requested LDPv6 and SRv6 many times from Cisco to migrate the

routing control plane from IPv4 to IPv6

...

I have lots of IPv6 address space. I don't have a lot of IPv4 address

space. RFC1918 is not as big as it seems. Apparently this is hard to grasp...

...

(This is primarily IOS-XE - can't afford the IOS-XR supercars)

On Wed, Jun 10, 2020 at 1:20 PM Mark Tinka <mark.tinka@seacom.mu> wrote:

...

Hi all.

Just want to sample the room and find out if anyone here - especially

those running an LDP-based BGPv4-free core (or something close to it) - would be interested in LDPv6, in order to achieve the same for BGPv6?

...

...

A discussion I've been having with Cisco on the matter is that they do

not "see any demand" for LDPv6, and thus, won't develop it (on IOS XE). Meanwhile, it is actively developed, supported and maintained on IOS XR since 5.3.0, with new features being added to it as currently as 7.1.1.

...

Needless to say, a bunch of other vendors have been supporting it for a

while now - Juniper, Nokia/ALU, Huawei, even HP.

...

IOS XR supporting LDPv6 notwithstanding, Cisco's argument is that "the

world" is heavily focused on deploying SRv6 (Segment Routing). While I know of one or two questionable deployments, I'm not entirely sure much of the world is clamouring to deploy SR, based on all the polls we've done at various NOG meetings and within the general list-based operator community

...

So I just wanted to hear from this operator community on whether you

would be interested in having LDPv6 support to go alongside your LDPv4 deployments, especially if you run native dual-stack backbones. Or if your focus is totally on SRv6. Or if you don't care either way :-). Thanks.

...

Mark.

-- Tim:>

-- ++ytti

-- Tim:>

On 10/Jun/20 21:36, Phil Bedard wrote:

In its simplest form without TE paths, there isn't much to SRv6. You use a v6 address as an endpoint and a portion of the address to specify a specific VPN service. You completely eliminate the label distribution protocol.

A BGPv6-free core is a decent use-case for us. Much simplicity has been enjoyed by removing that in the IPv4 world. Mark.

On 11/Jun/20 06:51, Saku Ytti wrote:

Every HW designer has sighed in relief when I've said I don't care about it, because it's also very HW unfriendly, like IPv6 generally. Unfortunately SRv6 is somewhat easy to market with the whole 'it's simple, just IP' spiel.

Mine have sighed in disbelief that I don't share their vision of an SR(v6) world :-). What's funny is that the ASR920 does not support SRv6, which I think is more due to hardware limitations than a lack of coding kiddies. Conversely, you don't need new bits of hardware to support LDPv6. Today, there is no box that supports LDPv4 that cannot support LDPv6, by just extending the code. No further hardware needed. Instead, I'm being asked to "upgrade" to the NCS540 so that I can get LDPv6. You, sometimes, have to wonder in what world these folk live. It's a Coronavirus era now - people want to hold on to all the $$ they don't have, and only spend it where they will extract the most value. Boeing and Airbus are struggling to reach customers that have pending deliveries; now isn't the time for vendors to posture. We need value, not product. Mark.

On 11/Jun/20 09:42, Saku Ytti wrote:

I don't like to conflate these two; SR is great, SRv6 is horrible abomination. SR is what MPLS should have been day1, but it probably was easier to market LDP than to say 'we need to change all IGP protocols'.

Fair point. Mark.

Just to clarify the only routers who potentially need to inspect or do anything with those headers are endpoints who require information in the extension header or hops in an explicit path. In the simple example I gave, there are no extension headers at all. I'm pretty agnostic to IPv6 SR-MPLS and SRv6, but just want to clarify that. I'm not going to claim inserting/swapping v6 extension headers is what all routers made in the last 20 years are especially good at. ( But it's not impossible to do it in some shipping devices today at wire rate with deterministic latency. As for normal v6 forwarding, the way most higher speed routers made recently work there is little difference in latency since the encapsulation for the packet is done in a common function at the end of the pipeline and the lookups are often in the same memory space. NPUs are also being built today with enough on-package memory to hold larger routing tables. Whether a packet has to be buffered on-chip vs. off-chip has a much larger impact on latency/PDV than a forwarding lookup. Thanks, Phil On 6/11/20, 5:07 AM, "NANOG on behalf of Nick Hilliard" <nanog-bounces@nanog.org on behalf of nick@foobar.org> wrote: Saku Ytti wrote on 11/06/2020 05:51: > Unfortunately SRv6 is somewhat easy to market with the whole 'it's > simple, just IP' spiel. it's not "just IP": it's ipv6 with per-router push / pop operations on ipv6 extension headers, i.e. high touch in areas which are known to be deeply troublesome on hardware. In this regard alone, the specification is problematic enough that it's unearthed a bug in the IPv6 standard (rfc8200). Nick

On 6/11/20, 1:19 PM, "Saku Ytti" <saku@ytti.fi> wrote: On Thu, 11 Jun 2020 at 19:49, Phil Bedard <bedard.phil@gmail.com> wrote: > As for normal v6 forwarding, the way most higher speed routers made recently work there is little difference in latency since the encapsulation for the packet is done in a common function at the end of the pipeline and the lookups are often in the same memory space. NPUs are also being built today with enough on-package memory to hold larger routing tables. Whether a packet has to be buffered on-chip vs. off-chip has a much larger impact on latency/PDV than a forwarding lookup. On-package is not important, on-chip or off-chip is what matters, i.e. do you eat SERDES to connect memory or not. [pmb] Sorry meant to say on-die, not on-package. Typically the time it takes to do those lookups are built into the system specs to attain the performance you need with deterministic latency within a certain bounds. There are certainly corner cases where you make tradeoffs, especially now that single NPUs are 10+ Tbps, but it's not really an MPLS vs. IPv4 vs. IPv6 thing. The other key is to do those types of accesses in a single pass, not traverse multiple hierarchy levels or do multiple operations. If you are tunneling then the table for any of those types is going to small on a mid-point router.

Phil Bedard wrote on 11/06/2020 17:49:

Just to clarify the only routers who potentially need to inspect or do anything with those headers are endpoints who require information in the extension header or hops in an explicit path. In the simple example I gave, there are no extension headers at all.

perhaps, but no-one planning to use srv6 is going to invest in kit which can handle srv6 but not the TE component. Or deploy srv6 on existing kit which can't handle TE. Nick

On 10/Jun/20 20:45, Saku Ytti wrote:

I'm pretty sure that one or more of Mark, Gert or Tim are thinking SR/MPLS IPv6 when they say SRv6?

Oh, not at all, Saku.

No one in their right minds thinks SRv6 is a good idea, terrible snake oil and waste of NRE. SR/MPLS IPv6 of course is terrific.

LDPv6 and SRv6 seem like an odd couple, LDPv6 SR/MPLS IPv6 seem far more reasonable couple to choose from. I have my favorite.

I've been tracking SR since it began making "waves" in Paris at a previous MPLS/SDN/IPv6 Congress meeting a a 2013, or thereabouts. Plenty of promise about being a decent alternative to LDPv6 (which had been on my mind since 2008). In the end, as you rightly point out, it's been much ado about nothing. To this day, I am yet to hear from a single operator about all the chanting I hear from vendors - and not just SRv6, but SR in general. 7 years and counting. Mark.

On Wed, Jun 10, 2020, at 20:51, Mark Tinka wrote:

Well, according to them, SRv6 is winning customers over, and nobody wants LDPv6. Then again, they have LDPv6 in IOS XR; figures.

Well, given their (Cisco's) braindead policy regarding non-implementation of LDPv6 on XE, no wonder people are looking for alternatives, and SRv6 is one of them. And don't forget SRv6 is also heavily associated (marketing-wise) with 5G.... Back to our friends and their policy: It happens that in certain regions of the world, if you want to be an ISP other than the "establishment" (== incumbent + "first alternatives" that started 20-25 years ago), you MUST have LNS (if you want to stay in business). If like many, you are kind of stuck with Cisco because it's Cisco, the only decent solution to have LNS is ASR1K (running XE). Also add ASR920 which has a number of uses. Also, in order to stay in business, you may want to offer L3VPN services, which brings you to doing MPLS. You say MPLS, you say LDP, and there you go, your backbone remains v4-based for the next eternity. There also seems to be a lack of global vision. Tyry to ask your favourite vendor what do you need in order to be able to offer IPv4-L3VPN, IPv6-L3VPN and L2VPN (mainly point-to-point - NO MAC learning) over a backbone that does NOT use any single IPv4 address (backbone-side). Because you can do it on a backbone that does not use any single IPv*6* address, but you may want to go forwards, not backwards. Add a LNS in the mix (the v4 addresses for the LNS go in VRFs - that's not backbone). Add a money, rack space and power needed constraints in the mix. This exercise looks challenging with other vendors too, but with Cisco it's just impossible. Of course, Cisco says there is no demand for one simple reason : the people talking with Cisco account managers (or whatever they are called) are only rarely those that care about technical stuff. They may want some features on the CPEs (like "ui uant SDWAN"), but for anything else (including backbone equipment) they only want lower prices. You end up with everybody having to deal with a specific platform in real life to dream about a specific feature, yet the vendor to consider that "nobody wants it".

On 11/Jun/20 10:32, adamv0025@netconsultings.com wrote:

Hey Mark, My stance is that should I go with anything "new" for label distribution the MPLS SR/SPRING is getting to a point where it might be mature enough.

"Getting to a point" doesn't really work if you are actively running a network today :-). While I do agree that going with the new thing is always a good plan, one has to truly consider the overall gain vs. labour required to get there. Going from static routing to an IGP + BGP makes sense when you scale up. Switching from Distribute Lists to Prefix Lists makes sense when you scale up. Route summarization after you dump your old Cisco 2501 for an ASR9901 doesn't add value any longer. You get the idea. The position about not needing a label distribution protocol in SR is actually quite sexy. But considering how powerful router control planes are nowadays, especially when they are being virtualized on or off chassis, I just don't see the gains expected by removing LDP and going SR, on a box and code that supports both. Yes, if you are talking about dumping a spaghetti of RSVP tunnels, that makes sense as there is a gain in day-to-day network administration. But in this case, we are just speaking about LDP. 10 years ago, we worried about how well an IP network would scale running OSPF or IS-IS. With control planes what they are today, who really cares anymore? You may recall we've been running CSR1000v for route reflection since 2014 - millions of routes being carried everyday, converging in seconds. We've never had to think about those boxes until last year when we did the server hardware refresh as a matter of course, not because anything was struggling. What I'm saying is not all new tech. NEEDS to get deployed just because it's new. If that was the case, we'd all be running Inter-AS DSCP over IPoDWDM :-).

Also "BGP free core" means internet won't talk to your core -i.e. free to use private addressing -so no need for v6 at all in the "underlay" (as hipsters call it these days).

Careful with that one - Cisco's proposal to me was to run my IPv6 network on link-local :-). Don't encourage them, hehe.

Alternatively using public "infrastructure subnet" (i.e. not advertised to the Internet) for a "BGP free core", the aim is to make money of the core -what additional revenue stream am I getting by enabling v6 in the underlay/management plane that would offset the pain of dealing with the increased bug surface?

I don't know about you, but my BGP-free core is inaccessible from the world even if it lives in public-IPv4 land. That's how pure MPLS forwarding works. You'd have be "inside" to reach it (IGP). Also, if you link every feature to a revenue stream, you'll never deploy RPKI or DNSSEC :-).

And with regards to the XE/XR discrepancies, I mentioned my prophecy a number of times, I think XE future in SP products portfolio is next to none.

Which is fine - but customers are spending real money and need to keep real networks running with real costs for real years. If Cisco want to kick IOS XE to the side, let customers know so we can make informed decisions about where to purchase gear. The current state-of-the-art is that kit you buy today is probably good years after standard depreciation policies, probably longer. If Cisco's model is to throw boxes away sooner than that like they did in the old days, that is not consistent with where the tech. has gotten to in the past 2 decades. Mark.

On 11/Jun/20 14:17, adamv0025@netconsultings.com wrote:

Well RPKI or DNSSEC is at least adding a value, even something you can brag about to your customers (we are part of the solution not part of the problem etc...).

Bragging doesn't bring in income, it's just PR :-).

But running MPLS over IPv6 in addition to already running it over IPv4, gaining new functionality or features in the process that could be ultimately monetized in providing better service to customers? -maybe even exposing the network to a new set of bugs? -I don't know, that doesn’t sound like a good use of company resources especially in these uncertain times . Maybe I'm being overly harsh here maybe if you could please explain the drivers or expected net value out of this exercise please?

Oh dear, you sound like our Finance department now; "drivers" and "net value" :-). If I take your line of reasoning, deploying SRv6 likely requires new hardware, which means $$. How much money will I charge customers for my shiny new SRv6? LDPv6 builds on LDPv4. Just like IPv6 builds on IPv4. At best, you can remove BGPv6 from your core, which lowers your administration costs in that part of the network even further, costing you less in human time running it, resources you can otherwise re-deploy to other time- and money-saving activities. At worst, you get IPv4/IPv6 feature parity, and who doesn't like that :-). And how much money did LDPv6 cost you to deploy? $0. Mark.

On 11/Jun/20 15:34, Saku Ytti wrote:

Unsure of others, but we didn't implement RPKI for shit and giggles, we implemented it, because customers wanted us to do RPKI.

I'll be honest, none of our customers ever asked us to deploy RPKI and ROV. Will they benefit from it, sure? Is it about to become part of an RFP requirements document? Probably not. Then again, the typical NTT customer probably has an engineering department that understands and demands RPKI, because they are a sizeable operator themselves. We only have a handful of customers that technically appreciate that we've deployed RPKI. For the rest, it doesn't register. Not sure if beating up on Job for months qualifies as "a customer wanting RPKI from NTT" :-). Mark.

From: Saku Ytti <saku@ytti.fi> Sent: Thursday, June 11, 2020 3:29 PM

On Thu, 11 Jun 2020 at 16:45, Mark Tinka <mark.tinka@seacom.mu> wrote:

...

Not sure if beating up on Job for months qualifies as "a customer wanting RPKI from NTT" :-).

I'm sure we can blame Job for this, why not. But probably because of his lobbying some customers are _requiring_, i.e. flat out told they will stop accepting transit offers from providers who don't do RPKI.

Case in point. On the other hand not sure if any of the customers would care whether LSPs are signalled over v4 as opposed to v6. adam

On 11/Jun/20 16:28, Saku Ytti wrote:

I'm sure we can blame Job for this, why not. But probably because of his lobbying some customers are _requiring_, i.e. flat out told they will stop accepting transit offers from providers who don't do RPKI.

As my Chad friend would say, "I like the sound of this". Mark.

On 13/Jun/20 01:32, Randy Bush wrote:

we have rov in rfps received from paying customers

I hope this becomes the norm, globally. Mark.

On 11/Jun/20 16:25, adamv0025@netconsultings.com wrote:

Good PR might ;)

I'm old school - build something customers want to use, and the money follows. Care to guess how much PR Tik Tok do :-)? But I digress.

No, my line of reasoning is if you have MPLS LSPs signalled over v4 I see no point having them signalled also over v6 in parallel.

It's not about signaling IPv4 LSP's over IPv6. LDPv4 creates IPv4 FEC's. LDPv6 creates IPv6 FEC's. The idea is to create IPv6 FEC's so that IPv6 traffic can be label-switched in the network natively, allowing you to remove BGPv6 in a native dual-stack core.

Or if your full-mesh RSVP-TE is killing your RSVP-TE or you're in need of TE, then might want to look at SR MPLS.

No RSVP-TE here, thank the Lord :-).

I'm using VPNv6 & VPNv4 so not sure I follow how LDPv6 allows for removal of BGPv6 (is it BGPv6 over v4 NHs/transport?) but then again if it works over v4...

Nothing like the real thing: In IPv4-land, you get this: 24005  49017       105.16.Y.Z/32     Te0/0/0/2    105.16.A.B    8614773             49017       105.16.Y.Z/32     Te0/1/0/0    105.16.C.D   8121753             49017       105.16.Y.Z/32     Te0/7/0/5    105.16.E.F    9964543 In IPv6-land, you get this: 2c0f:feb0:X:Y::Z/128 25071   25458          BE1          Link-local                                               25458          BE2          Link-local In Junos land, it looks like this: 7967               *[LDP/9] 1w5d 01:21:05, metric 1                         >  to fe80::de38:e1ff:fe15:dc02 via ae2.0, Swap 145674 If I run an IPv6 traceroute toward a host that sits behind a router doing LDPv6, this is what happens: run traceroute 2c0f:feb0:2f:ff00:WWWW:XXXX:YYYY:ZZZZ traceroute6 to 2c0f:feb0:2f:ff00:WWWW:XXXX:YYYY:ZZZZ (2c0f:feb0:2f:ff00:WWWW:XXXX:YYYY:ZZZZ) from 2c0f:feb0:1:2::112, 64 hops max, 12 byte packets  1  ae-5-0.er6-01-fra.de.seacomnet.com (2c0f:feb0:1:2::111)  165.567 ms  166.158 ms  166.924 ms      MPLS Label=145786 CoS=0 TTL=1 S=1  2  xe-0-0-0-0.cr6-02-mrs.fr.seacomnet.com (2c0f:feb0:1:2::f9)  165.682 ms 2c0f:feb0:1:2::279 (2c0f:feb0:1:2::279)  165.817 ms  168.123 ms      MPLS Label=25494 CoS=0 TTL=1 S=1 <snip> As you can see, just as with IPv4, IPv6 packets are now being MPLS-switched in the core, allowing you to remove BGPv6 in the core and simplify operations in that area of the network. So this is native MPLSv6. It's not 6PE or 6VPE. Your IPv4 domain could fall over and your MPLSv6 network will still be alive, because it's neither 6PE nor 6VPE. And vice versa - your IPv6 network could die, and your MPLSv4 will be unaffected.

I knew there's a bit of OCD hidden in this at some level :)

Safe to say the Internet is one big OCD project :-). My OCD is sleeping at 3AM, in peace, lockdown or not, hehe.

Apart from X months worth of functionality, performance, scalability and interworking testing -network wide code upgrades to address the bugs found during the testing process and then finally rollout across the core and possibly even migration from LDPv4 to LDPv6, involving dozens of people from Arch, Design, OPS, Project management, etc... with potential for things to break while making changes in live network.

Which you wouldn't have to do with SRv6, because you trust the vendors? And no, LDPv6 does not call for one to migrate from LDPv4. They can live together, side-by-side, just as native dual-stack IPv4/IPv6 does. We are running LDPv4 and LDPv6 side-by-side, with no problems, between IOS XR and Junos, as you can see above. This is a live network, carrying revenue-generating, production traffic. It's not a fantasy. Mark.

On 11/Jun/20 23:45, adamv0025@netconsultings.com wrote:

Right I see what you are striving to achieve is migrate from BGP in a core to a BGP free core but not leveraging 6PE or 6VPE?

Yes sir.

So considering you already had v4 FECs wouldn't it be simpler to do 6PE/6VPE, what do you see as drawbacks of these compared to native MPLSv6 please?

Because 6PE, for us, adds a lot more complexity in how we design the network. But most importantly, it creates a dependency for the success of IPv6 on IPv4. If my IPv4 network were to break, for whatever reason, it would take my IPv6 network down with it. Years back, there was a nasty bug in the ASR920 that set an upper limit on the MPLS label space it created FEC's for. Since Juniper sometimes uses higher label numbers than Cisco, traffic between that ASR920 and our Juniper network was blackholed. It took weeks to troubleshoot, Cisco sent some engineering code, I confirmed it fixed the issue, and it was rolled out generally. During that time when the ASR920 was unavailable on IPv4, it was still reachable on IPv6. Other issues are also with the ASR920 and ME3600X/3800X routers, where 0/0 and ::/0 are the last routes to be programmed into FIB when you run BGP-SD. It can be a while until those boxes can reach the rest of the world via default. IPv6 will get there faster. I also remember another issue, back in 2015, where a badly-written IPv4 ACL kicked one of our engineers out of the box. Thankfully, he got back in via IPv6. I guess what I'm saying is we don't want to fate-share. IPv4 and IPv6 can operate independently. A failure mode in one of them does not necessarily propagate to the other, in a native, dual-stack network. You can deploy something in your IPv6 control/data plane without impacting IPv4, and vice versa, if you want to roll out gracefully, without impacting the other protocol. 6PE simply has too many moving parts to setup, comparing to just adding an IPv6 address to a router interface and updating your IGP. Slap on LDPv6 for good measure, and you've achieved MPLSv6 forwarding without all the 6PE faffing.

Well my point was that if v4 FECs would be enough to carry v6 traffic then I wouldn't need SRv6 nor LDPv6, hence I'm curious to hear from you about the benefits of v6 FEC over v4 FEC (or in other words MPLSv6 vs 6PE/6VPE).

No need for 6PE deployment and day-to-day operation complexity. A simplified and more native tunneling for IPv6-in-MPLSv6, rather than IPv6-in-MPLSv4-on-IPv4. No inter-dependence between IPv6 and IPv4. Easier troubleshooting if one of the protocols is misbehaving, because then you are working on just one protocol, and not trying to figure if IPv4 or MPLSv4 are breaking IPv6, or vice versa. For me, those 4 simple points help me sleep well at 3AM, meaning I can stay up longer having more wine, in peace :-). Mark.