On 21 March 2018 at 13:10, Jason Lixfeld <jason+nanog@lixfeld.ca> wrote:
Hey,
For those running BFD on your land-based point-to-point links, I’m interested in hearing about what factors you consider when deciding how to configure your timers and multiplier.
On paper, BFD between two devices over a local or metro dark fibre or wave seems pretty trivial: Assuming your gear can a) support echo mode b) hardware offloads echo processing c) automatically treats echos as vital and puts them into the appropriate high priority queue, then setting the timers down to their lowest possible values (3ms on some of the gear that I’ve seen) and some low multiplier seems more than reasonable. But?
From another angle, your link isn’t dark fibre or a wave but, for example, ethernet over some sort of IP based L2 Transport, and is still a low (sub 1ms) one-way latency local or metro link. How do you set your timers, and what do you base that on?
From yet another angle, what if your link is a long-haul wave, or for that matter a wave of any distance that imposes a one-way latency that is higher than the minimum tx and rx timers that are supported by your gear? We’ll assume an unprotected wave, because I’m sure if it’s protected, you have no choice but to consider the one-way latency of the longest of the two segments.
I made some assumptions above about support for echo mode and hardware offload, but what if (some of) your gear doesn’t support some or all of that stuff? How do you factor your configuration decisions?
Thanks!
Going back to the original question;
From another angle, your link isn’t dark fibre or a wave but, for example, ethernet over some sort of IP based L2 Transport, and is still a low (sub 1ms) one-way latency local or metro link. How do you set your timers, and what do you base that on?
Personally I don't care if it's a wavelength, dark fibre or L2 VPN service. I don't treat them differently based on the underlying connectivity type. The SLAs are probably more important. But if we are paying for say 10G of capacity on a link which is say a 10G pseudowire from another carrier, I treat it the same as a dark fibre connected to 10G transceivers at each end. Wave lengths are generally more stable in my opinion, we did have a 10G L2 Ethernet circuit from a carrier that was a pseudowire from them essentially, and their PE was under a DDoS attack so our L2 VPN service was affected (because the pseudowire was flapping up and down). But once the circuit is up and running for a while, if you're regularly pushing somewhere near the max circuit bandwidth and monitoring circuit latency, you'll get a feel for "how good" the carrier is and then adjust from there. Generally speaking though, if the carrier is "good" I treat DF/lamda/L2 circuits the same with regards the BFD/IGP tuning.
I made some assumptions above about support for echo mode and hardware offload, but what if (some of) your gear doesn’t support some or all of that stuff? How do you factor your configuration decisions?
Elsewhere in the thread you have mentioned that you are using Cisco ME3600 devices. If you disable BFD echo mode you will be able to get low timers on these devices. Echo mode is enabled by default on IOS when you enable BFD under an interface, which these devices don't support, so you need to explicitly disable it. See the min/max/avg BFD timers below between two ME devices when the interfaces are configured with "bfd interval 50 min_rx 50 multiplier 3": ME3600#show bfd neighbors interface te0/2 details ... Session state is UP and using echo function with 50 ms interval. Session Host: Software ... Rx Count: 72, Rx Interval (ms) min/max/avg: 1/4976/4323 last: 2348 ms ago Tx Count: 74, Tx Interval (ms) min/max/avg: 1/4968/4217 last: 1436 ms ago If you add the command "no bfd echo" to the interface you should see the following min/max/avg BFD timers: ME3600#show bfd neighbors interface te0/2 details ... Session state is UP and not using echo function. Session Host: Software ... Rx Count: 3314443, Rx Interval (ms) min/max/avg: 1/72/47 last: 36 ms ago Tx Count: 3310865, Tx Interval (ms) min/max/avg: 1/72/47 last: 40 ms ago We have a mixture of devices and they don't all support BFD echo mode. We have for example Cisco ASR9000s that support both echo / no echo mode, so it may have one interface towards a Juniper MX running BFD echo mode and one interface towards a Cisco ME which runs no echo mode. It's working fine for us. Cheers, James.