From: David Sinn Sent: Friday, June 12, 2020 4:19 PM

...

On Jun 11, 2020, at 2:02 PM, Mark Tinka <mark.tinka@seacom.mu> wrote:

On 11/Jun/20 17:32, David Sinn wrote:

...

Respectfully, that is deployment dependent. In a traditional SP topology that focuses on large do everything boxes, where the topology is fairly

to-point and you only have a small handful of nodes at a PoP, labels can be fast, cheap and easy. Given the lack of ECMP/WECMP, they remain fairly efficient within the hardware.

...

...

However if you move away from large multi-chip systems, which hide

internal links which can only be debugged and monitored if you know the

point- the

obscure, often different ways in which they are partially exposed to the operator, and to a system of fixed form-factor, single chip systems, labels fall apart at scale with high ECMP.

...

I'm curious about this statement - have you hit practical ECMP issues with label switching at scale?

Yes. Path enumeration when you use mult-tier Clos topologies within a PoP causes you many, many problem.

Hi David, Can you be more specific please? Maybe some examples with numbers. I can see how you might run out of L2 rewrite/adjacency table space on a particular node if you enumerate every possible path downstream of it (especially on leaf nodes), cause that number is has a dependency on the size of the fabric in terms of the total number of links in the fabric (which balloons quickly). Let's focus on the alternate case then, (the deep label-stack one) At each node in the multi-tier Clos (which I assume is the Russ White's butterfly model? But any Clos or Benes fabric needs the same) you need to program a label to uniquely identify each egress interface ,so now there's this nice one-to-one relationship between label and egress interface. Now the depth of the label-stack depends on the number of hops the packet needs to traverse across the fabric. But how deep could the fabric realistically be? Even in the "butterfly" model with separate pods instead of leaf nodes I counted 9 hops (that's not ultra-deep is it)? It's the VM doing label imposition as programmed by the fabric controller (all in SW so can go as deep as you want) and all fabric nodes are just popping top label so no big deal. adam