Uuuuuu that's sounds great! Exactly what I was wondering. Create a VRF called BFD-RouteServer on each PE, on that VRF on each PE activate the IPs of Route-Server, and activate de BFD Sessions with the participants connected to that PE. And then, add a PBR deviation of BFD coming from participants to that Route-Server IP replicas. Having that weird Anycast of BFD deployed, now comes the part of making Route-Servers being aware of the status of those Distributed BFD Sessions. Em sáb., 21 de dez. de 2024, 22:23, Nick Hilliard <nick@foobar.org> escreveu:

Douglas Fischer wrote on 22/12/2024 00:29:

Does any colleague have any clue about BFD session status propagation over BGP?

this internet draft discusses the problem space

https://datatracker.ietf.org/doc/draft-ietf-idr-rs-bfd/

Unfortunately there are no known implementations of the protocol.

Nick

Tom Beecher wrote on 22/12/2024 02:51:

Definitely will be interesting to read the list discussion about this. My first reaction was why would you even need this, so def curious.

There are two main target situations: firstly, when a router unexpectedly drops off an ixp platform, this won't be explicitly signaled to the other routers on the fabric, which can mean that packets to that device will be black-holed until all the others bgp hold timers kick in. This would typically happen after 90-180s (e.g. hello time: 30-60s). The second situation would be to deal with forwarding plane incongruence on IXPs, i.e. where router A can reach RS, router B can reach RS, router A cannot reach router B due to a problem on the IXP fabric itself. Thankfully this style of problem has become quite unusual over the last several years. I'm not sure it's a all-round good solution to either of these problems, in the "be careful what you wish for, because you might get it" sense. There are going to be router platforms out there which won't handle hundreds of BFD sessions reliably, so if the protocol were widely supported, it's not clear that it would help or harm interdomain routing stability due to the ability of routers to handle large numbers of BFD sessions, particularly where there were situations where all the sessions could be triggered simultaneously. As a separate issue, hold timers should generally be of a comparable order of magnitude to the non-availability effect they're attempting to mitigate. Inter-domain routing convergence is often measured in minutes rather than seconds. So even if the protocol layer worked at IXPs without causing control plane meltdown, it's still a mechanism which which has a trigger timer two orders of magnitude faster than the general case of DFZ reconvergence. I can't see that this would help overall inter-domain routing stability. Nick

All, This thread touches on day one bgp architecture bug where the BGP spec is too vague on what should be considered as valid next hop. Most implementations today go as far as checking if the next hop can be resolved in RIB and if so consider the path as valid and eligible for best path selection. Well clearly that approach is not working well in IXP setup as there can be peers on a connected subnet which went down 179 seconds ago and we are still blackholing by pushing the traffic to such next hops. In reality the situation get's worse even in intradomain cases where operators use encapsulation via the domain while IGP does install the route to a next hop. The encap path can be broken while the BGP thinks it is all cool and keep blasting traffic towards it. It has been observed as a real issue in number of networks and Rajiv few years back wrote a draft to fix it .. https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-bestpath-selection-... Well as lot's of useful real BGP protocol extensions in IDR it died. So to the top of this thread IMHO we should not put BFD state in BGP. That's as ugly as BGP-LS. Instead each peer should be able to periodically test reachability to a peer (bgp nh) in the data plane and only then consider the path is valid for BGP best path selection. Would running even ICMP to a nh peer every second be an overkill ? I don't think so but I am sure there would be some who think it would. Best, R. On Sun, Dec 22, 2024 at 2:05 PM Mark Tinka <mark@tinka.africa> wrote:

On 12/22/24 14:15, Nick Hilliard wrote:

...

As a separate issue, hold timers should generally be of a comparable order of magnitude to the non-availability effect they're attempting to mitigate. Inter-domain routing convergence is often measured in minutes rather than seconds. So even if the protocol layer worked at IXPs without causing control plane meltdown, it's still a mechanism which which has a trigger timer two orders of magnitude faster than the general case of DFZ reconvergence. I can't see that this would help overall inter-domain routing stability.

I think this is the fundamental question.

BGP is stable and scales well given its global scope, not only because it turns like a tanker, but because we accept that it turns like a tanker.

Now, in a world of TikTok Brain and Uber Eats where we are used to getting what we want instantly, imposing that on to BGP, even if sneakily, is probably not something we want. At least not at a global scale.

Mark.

Hi Bill,

I'm missing something.

Wouldn't the route server send withdrawals and updates to the rest of the participants as soon as its hold timer with the lost router expires?

I believe the case here is about a situation where peers can talk to RS just fine (no bgp session goes down) however reachability between them is broken. RS to peer path does not need to be congruent with date plane path. Besides, large IXs sometimes use encapsulation between peer's ports while packets to control plane RS may not be subject to it.

Could this not be accelerated by the IXP asking the participants to keep low keepalive and hold timers with the route server?

Not in the above case.

How would your solution help when two participants at an IXP have chosen -not- to bilaterally peer, thus needing the route server to intermediate? They're going to agree to build BFD sessions even though they don't want BGP sessions? That... doesn't make sense.

It will not. That is why IMO peers should detect NH reachability by unidirectional OAMs. And as mentioned above this would be applicable much wider then in the IXPs. Thx, R.

...

The second situation would be to deal with forwarding plane incongruence on IXPs, i.e. where router A can reach RS, router B can reach RS, router A cannot reach router B due to a problem on the IXP fabric itself. Thankfully this style of problem has become quite unusual over the last several years.

Doesn't seem like it would solve the bouncy link problem.

Absent bouncy links, simply having a reasonable time out for arp and ND will assure the router quickly finds its neighbor unreachable, which is applied as backpressure into BGP.

Regards, Bill Herrin

--

William Herrin bill@herrin.us https://bill.herrin.us/