Load balance over multiple bgp feeds
I was wondering what everyone does to load balance over multiple bgp feeds. We currently have 5 bgp feeds with 2 providers. Do you just randomly pick networks, or use something like netflow to try and pick the best path. Mike
On Thu, Apr 21, 2005 at 06:47:36PM -0500, Mike Hyde wrote:
I was wondering what everyone does to load balance over multiple bgp feeds. We currently have 5 bgp feeds with 2 providers. Do you just randomly pick networks, or use something like netflow to try and pick the best path.
A lot of it depends on what you're trying to balance for (cost? performance?), the size of each link, the type of traffic you're dealing with, and other factors. My main experience has been at a couple of stub networks, and I don't claim to be an expert on this. YMMV. I've generally been able to get surprisingly accurate results in terms of how much traffic to send out one link or another just by using route-maps and some simple netflow analysis, plus traceroute / ping. I don't know if Netflow will give you much information in terms of the best path, but it can give you an idea of how much traffic you're sending / receiving to various ASes. I've found ehnt[1] pretty helpful for this. Even a few simple rules (all traffic to AOL goes through X, all traffic passing through 701 goes through Y) can often get you pretty close to where you want to be. This is a sledgehammer / shotgun type tool... but it has worked well for me. Of course you need to take into consideration the size of your pipes and how the loss of connectivity to one provider may affect your rules. And of course there are tools which claim to do this for you (for a "small" price), Route Science, Internap's appliance (whatever it's called), etc. I've not had any personal experience with these devices; check the NANOG archives for plenty of opinions. I'm not sure if there are any open source tools that do a good job of this, but you might see the NANOG thread with the subject "Open Source BGP Route Optimization?"[2] from May 2004 - came up from a quick google search I did. For content type stuff, I haven't found performance to be a huge deal most of the time... I imagine with VOIP and other applications, it might be more of a concern, though. /w [1] http://ehnt.sourceforge.net/ [2] Looks like this is the first item in the thread: http://www.cctec.com/maillists/nanog/current/msg00719.html
On Thu, Apr 21, 2005 at 05:06:27PM -0700, Will Yardley wrote:
On Thu, Apr 21, 2005 at 06:47:36PM -0500, Mike Hyde wrote:
[ Sorry for the self-followup. ]
I was wondering what everyone does to load balance over multiple bgp feeds. We currently have 5 bgp feeds with 2 providers. Do you just randomly pick networks, or use something like netflow to try and pick the best path.
A lot of it depends on what you're trying to balance for (cost? performance?), the size of each link, the type of traffic you're dealing with, and other factors.
Also, my experience has been in fairly content-heavy networks. Shaping inbound traffic can be a little more difficult (lots of information on this online). If your provider supports communities, they may allow you to prepend selectively to certain ASes or in certain geographical locations -- and of course you can play around with prepending.
I've generally been able to get surprisingly accurate results in terms of how much traffic to send out one link or another just by using route-maps and some simple netflow analysis, plus traceroute / ping.
Probably obvious, but if your providers support communities, this can be extremely helpful... you can often use these to do some slightly less kludgy traffic-shaping - "prioritize traffic learned from provider X's customers, but not from their peers", or "prioritize provider X's peers in our city, but de-prioritize their peers in the Bay area". In addition to internal documentation, many providers put information on the communities they support in the routing-registry entry for their AS. w
Mike,
I was wondering what everyone does to load balance over multiple bgp feeds. We currently have 5 bgp feeds with 2 providers. Do you just randomly pick networks, or use something like netflow to try and pick the best path.
You might be interested by the following papers : S. Uhlig and O. Bonaventure. Designing BGP-based outbound traffic engineering techniques for stub ASes. ACM SIGCOMM Computer Communication Review, 34(5), 2004. S. Uhlig, O. Bonaventure, and B. Quoitin. Interdomain traffic engineering with minimal BGP configurations. In 18th International Teletraffic Congress (ITC), September 2003. http://totem.info.ucl.ac.be/publications.html Best regards, Olivier
participants (3)
-
Mike Hyde -
Olivier Bonaventure -
Will Yardley