- NANOG - lists.nanog.org

Re: [c-nsp] LDPv6 Census Check
by Saku Ytti 12 Jun '20

12 Jun '20

On Fri, 12 Jun 2020 at 18:42, David Sinn <dsinn(a)dsinn.com> wrote: > But why do you need full table lookup in the middle of the network? Why place that class of gear where it's not needed? Some people do collapsed core. But this is getting bit theoretical, because we definitely could do this on IP, we could do some lookups on on-chip and some off-chip to do both, should market want it. > The label stack question is about the comparisons between the two extremes of SR that you can be in. You either label your packet just for it's ultimate destination or you apply the stack of the points you want to pass through. Quite, but transit won't inspect the stack, it doesn't have to care about it, so it can be very deep. > In the former case you are, at the forwarding plane, equal to what you see with traditional MPLS today, with every node along the path needing to know how to reach the end-point. Yes, you have lowered label space from traditional MPLS, but that can be done with site-cast labels already. And, while the nodes don't have to actually swap labels, when you look at commodity implementations (across the last three generations since you want to do this with what is deployed, not wholesale replace the network) a null swap still ends up eating a unique egress next-hop entry. So from a hardware perspective, you haven't improved anything. Your ECMP group count is high. I don't disagree. What I'm trying to say, however you tunnel, you have the same issues. If you need to tunnel, then MPLS is better tunnel than IP. Ultimately both can be made LEM on-chip should market want it, so difference left is what is the overhead of the tunnel, and MPLS wins here handsdown. This is objectively true, now what practical market reality is, that may be different, because market doesn't optimise for best solution. > So, yes, MPLS works fine if you want to buy big iron boxes. But that come at a cost. So the point about MPLS is always better is not accurate. Engineering is about trade-offs and there are trade-offs to be made when you optimize in a different direction and that points away from MPLS and back to IPIP Always if you need tunnel. Because the fundamental question is how much overhead we have, and what is our key width. In both IP tunnel and MPLS tunnel cases we will assume LEM lookup, to keep the lookup cheap. -- ++ytti

1 0

IPv4 Broker / Service -
by edwin.mallette＠gmail.com 12 Jun '20

12 Jun '20

Hi Nanog, I have need of a reputable IPv4 broker or service - personal experience with said broker would be preferred. These would be for small blocks - /23, 24s - in the US, so ARIN. I know, I know, IPv6 for life and all that and I agree, but . you know, the business. I'm happy to take responses off-list, but I would really appreciate any recommendations. Thanks! Ed

7 6

Re: [c-nsp] LDPv6 Census Check
by Saku Ytti 12 Jun '20

12 Jun '20

On Fri, 12 Jun 2020 at 18:16, David Sinn <dsinn(a)dsinn.com> wrote: > I'm not sure what implementation you are saying doesn't exist. The Broadcom XGS line is all on-die. The two largest cloud providers are using them in their transport network (to the best of my understanding). So I'm not sure if your saying that no one is using small boxes like I'm describing or what. And it doesn't have to be MPLS over IP. That is one option, but IPIP is another. I'm saying implementation which has off-chip and supports putting some on-chip. So that you could have full table lookup for edge packets and and fast exact lookup for others. Of course we do have platforms which do have large LEM tables off-chip. > Again, feel free to look at only one small aspect and say that it is completely better in all cases. MPLS is not better in wide ECMP cases, full stop. SR doesn't help that when you actually look at the problems at massive scale as I have done. You continually are on the trade-off spectrum of irrationally deep label stacks or enumeration of all of the possible paths through the network and burn all of your next-hop re-writes. At least if you want high-radiux, single chip systems. So this is not sentimentally around a protocol, it's the practical reality when you look at the problems at scale using commodity components. So if you want to optimize for costs and power (which is operational costs), MPLS is not where it is at. I'm not sure why this deep label stack keeps popping, if we need multiple levels of tunneling, we need it in IP too, and it's almost more expensive in IP. Cases I can think of in SR, you'll only loop top label or two, even if you might have 10 labels. For every apples to apples cases MPLS tunnels are superior to IP tunnels. If you want cheap very small FIB backbone, then all traffic will need to be IP tunneled to egress, and you get all the MPLS problems, and you get more overhead and larger keys (larger keys is not a big deal). Now if discussion is do we need tunnelling at all, the discussion is very different. -- ++ytti

1 0

Re: [c-nsp] LDPv6 Census Check
by Mark Tinka 11 Jun '20

11 Jun '20

On 11/Jun/20 17:32, David Sinn wrote: > Respectfully, that is deployment dependent. In a traditional SP topology that focuses on large do everything boxes, where the topology is fairly point-to-point and you only have a small handful of nodes at a PoP, labels can be fast, cheap and easy. Given the lack of ECMP/WECMP, they remain fairly efficient within the hardware. > > However if you move away from large multi-chip systems, which hide internal links which can only be debugged and monitored if you know the the obscure, often different ways in which they are partially exposed to the operator, and to a system of fixed form-factor, single chip systems, labels fall apart at scale with high ECMP. I'm curious about this statement - have you hit practical ECMP issues with label switching at scale? We have ECMP'ed label switch paths with multiple paths for a single FEC all over the place, and those work fine both on Cisco and Junos (of all sizes), both for IPv4 and IPv6 FEC's. Have done for years. Unless I misunderstand your concern. Mark.

1 0

RE: [c-nsp] LDPv6 Census Check
by adamv0025＠netconsultings.com 11 Jun '20

11 Jun '20

> From: David Sinn > Sent: Thursday, June 11, 2020 4:32 PM > > However if you move away from large multi-chip systems, > to a system of fixed form-factor, single chip systems, labels fall > apart at scale with high ECMP. Needing to enumerate every possible path > within the network or having to have a super-deep label stack removes all of > the perceived benefits of cheap and simple. > Looks like the deployments you describe are large DC Clos/Benes fabric, then the potentially deep label imposition would be done by the VMs right? On transit nodes the 64x ECMP or super-deep labels is no problem for NPU/lookup process as it's always just top label lookup and resolving to a single egress interface. adam

1 0

Re: [c-nsp] LDPv6 Census Check
by Saku Ytti 11 Jun '20

11 Jun '20

On Thu, 11 Jun 2020 at 21:04, David Sinn <dsinn(a)dsinn.com> wrote: > You've made my point for me. If you are building the core of your network out of MX's, to turn a phrase, in a past life "I fully support my competitors to do so". Large numbers of small boxes, as they have already shown in the data-center, have major cost, control and operational advantages over a small number of large ones. They also expose the inherent problems of label-switching and where IP has it's merits. Except this implementation does not exist, but we can argue that is missing feature. We can argue we should be able to tell the lookup engine this CIDR is on-chip and it's host routes only. This is certainly doable, and would make IP tunnels like MPLS tunnels for lookup cost, just larger lookup key, which is not significant cost. But even if we had this (we don't, we have for MPLS) IP would be still inferior, it is more tunneling overhead, i.e. I need more overspeed. Technically MPLS is just better tunneling header. I can understand sentimental arguments for IPv4 and market seems to appreciate those arguments particularly well. -- ++ytti

1 0

Re: [c-nsp] LDPv6 Census Check
by Saku Ytti 11 Jun '20

11 Jun '20

On Thu, 11 Jun 2020 at 18:32, David Sinn <dsinn(a)dsinn.com> wrote: > However if you move away from large multi-chip systems, which hide internal links which can only be debugged and monitored if you know the the obscure, often different ways in which they are partially exposed to the operator, and to a system of fixed form-factor, single chip systems, labels fall apart at scale with high ECMP. Needing to enumerate every possible path within the network or having to have a super-deep label stack removes all of the perceived benefits of cheap and simple. The arguments about IP lookups being slow is one best left to the 1990's when it was true. Fixed pipeline systems have proven this to be false. It continues to be very much true. IP lookups require external memory, which takes SERDES, which could be used for revenue otherwise. IP lookups are slow, expensive and complex, fundamentally, no amount of advancement will change this fundamental nature. Sure we can come up with all kind of implementations which bridge the gap, but the gap is there. If we take say JNPR MX, your lookup speed isn't limited by the instruction count on the PPE, the PPE spends most of its time sleeping, when the platform is fully PPS congested, the PPE is waiting for the memory to return! -- ++ytti

1 0

Re: [c-nsp] LDPv6 Census Check
by Mark Tinka 11 Jun '20

11 Jun '20

On 11/Jun/20 14:43, Gert Doering wrote: > > Not "in addition to" but "to throw out the IPv4 part" :-) That's another view-point, yes. There are plenty of networks that can't afford to keep buying IPv4 on the open market. They want to go single-stack IPv6. Today, you would need to build a 6PE network if you want MPLS support in your IPv6-only network, which still requires IPv4. Private IPv4 address space works, but some people like it, some don't, and more importantly, you still carry around IPv4. You could try the SRv6 gravy, but now you probably need new kit. What's the SRv6 business-case? Well, one would say you save millions of $$ not buying IPv4, but now you've spent money supporting SRv6. LDPv6 can be supported by any platform that supports MPLS today. No money out the door. Cisco would need to develop code for both LDPv6 and SRv6 anyway, so no harm done on that side. As we used to say in Vladivostok, "It's simple physics :-)". > (If I had LDPv6 today, I wouldn't actually change the existing network > today. But for the next round of rebuilding things, it would be something > to consider...) Give your favorite Cisco AM a shout-out. I promise, they are probably too young to remember your 6500/7600 tickets :-). Mark.

1 0

Re: [c-nsp] LDPv6 Census Check
by Mark Tinka 11 Jun '20

11 Jun '20

On 11/Jun/20 13:03, Drikus Brits wrote: > We're deploying some new POPs with a mix of IOS-XR as borders, BNG and > PEs and IOS-XE for LNSs. LDPv6 would be awesome to have availabke on > IOS-XE alongside LDPv4. We're being pushed by Cisco to go one flavour > or another of SR as well by Cisco, but i'd much prefer to have LDPv4 & > LDPv6 right now. > > We have some buying power with Cisco at the moment, so let me know how > I can help from the land 'down under'. Thanks, Drikus. If you can get your AM to get in touch with the ASR920 and ASR1000 BU's to ask for LDPv6, that will help a great deal. Cisco are saying "no one" is asking for LDPv6, forgetting that it's more about IPv6, than other lower/higher-level services it can support, LDPv6 included. As with you, they are trying to shove SRv6 down our throats, which can only come on an NCS540 for the moment, which is odd since that runs IOS XR, which supports LDPv6. So basically, swap out tons of boxes to get a feature that can easily be coded for in existing hardware... makes you wonder about the thought-process; It's been a crazy two weeks with our Cisco AM's and the platform TME's on heated calls about this that it's almost old testament biblical :-). It's at the stage where it's going up the chain to the BU's who will make the final call. So if they can get some noise from you as well, it certainly won't hurt. You don't need new hardware features on the ASR920 or ASR1000 to support LDPv6. You might do for SRv6. Either way, both platforms would need both features developed in code, and I think it's safe to assume which one will be practically easier to build and roll-out to the entire MPLS customer base, today. Mark.

1 0

Re: [c-nsp] LDPv6 Census Check
by Saku Ytti 11 Jun '20

11 Jun '20

On Thu, 11 Jun 2020 at 12:57, Robert Raszuk <robert(a)raszuk.net> wrote: > Nope that was not the main reason. > > Main reason was the belief that labels MUST be locally significant - and not domain wide unique. Just look at Juniper's SRm6 or now SRH ... they keep this notion of locally significant SIDs. It is deep in their DNA ... still. Seems weird, because neither LDP or SR implies globally significant labels, implementation choice. What SR does imply is a continuous block of labels of equal size in domain. > Now to your runt that MPLS is great because of exact match perhaps you missed it but number of solutions on the table (including RbR[**] I recently proposed) use exact match 4B locator based lookup in the v6 packets to get from segment end to segment end. Which is making a simple thing a complex thing. > On the other hand your comments about greatness of MPLS ... simplified data plane and depending on the hardware difference in jitter (in sub ms ranges - if that even matters) comes up with a lot of control plane complexity when you want to build a network across all continents, yet keep it scoped from IGP to areas or levels. No summarization in MPLS in FECs is something we should not sweep under the carpet. MPLS complexity is trivial in control-plane and forwarding-plane compared to IPV6 or SRv6 or RbR[**]. I'm not saying we can't do better than MPLS, but the proposal here is blatantly worse. -- ++ytti

1 0