L3 ECMP over links with different RTT
Hi, Is it foolish to build a L3 ECMP connection between 2 iBGP routers with one of the links having a 50% longer RTT? We're building fiber between 2 locations with the option of going both north and south around the great lakes. One link will have a 20ms RTT and one with around 35ms. The majority of our peering and transit will be on one side, with the enterprise datacenter on the other. The design right now would use OSPF as an IGP over these links with iBGP peering on loopbacks. Both links are 1G. Any thoughts on doing this different/better would be appreciated. Thanks.
On 10 mei 2011, at 16:28, Dikkema, Michael (Business Technology) wrote:
Is it foolish to build a L3 ECMP connection between 2 iBGP routers with one of the links having a 50% longer RTT?
No problem at all as long as you don't do per-packet load balancing but something that makes sure a single flow only goes over a single link. There are many ways to skin that particular cat, best practice is based on the 5-tuple (source and dest addresses and ports and the protocol number) which will give you the best chance of having a similar load on both links as long as you have at least some 1000 flows at any given time. With less granular load balancing there's a much bigger risk that one link will be full and the other more or less idle unless you have very, very many flows. You may want to use VLANs so you can load balance some stuff as per the above and manually balance some other stuff to go over the faster link.
Per flow is generally the best method, and allows the employ of CEF (or the equivalent). I've done load balancing in this method, and in others I've configured active/standby for the reasons specified. It depends on whether you need true link redundancy more than the latency will affect traffic. Another option, of course, is to apply PBR to get your low latency queues to use the preferred link. I've done that as well, using EEM to remove the forced next hop if the interface drops. Sincerely, Brian A . Rettke RHCT, CCDP, CCNP, CCIP Network Engineer, CableONE Internet Services -----Original Message----- From: Iljitsch van Beijnum [mailto:iljitsch@muada.com] Sent: Tuesday, May 10, 2011 7:37 AM To: Dikkema, Michael (Business Technology) Cc: nanog@nanog.org Subject: Re: L3 ECMP over links with different RTT On 10 mei 2011, at 16:28, Dikkema, Michael (Business Technology) wrote:
Is it foolish to build a L3 ECMP connection between 2 iBGP routers with one of the links having a 50% longer RTT?
No problem at all as long as you don't do per-packet load balancing but something that makes sure a single flow only goes over a single link. There are many ways to skin that particular cat, best practice is based on the 5-tuple (source and dest addresses and ports and the protocol number) which will give you the best chance of having a similar load on both links as long as you have at least some 1000 flows at any given time. With less granular load balancing there's a much bigger risk that one link will be full and the other more or less idle unless you have very, very many flows. You may want to use VLANs so you can load balance some stuff as per the above and manually balance some other stuff to go over the faster link.
participants (3)
-
Dikkema, Michael (Business Technology) -
Iljitsch van Beijnum -
Rettke, Brian