Replying to what most of the offline replies that i received said:
We wish to load balance the traffic for a block/range of IP addresses that we learn via BGP4 from our two upstream providers. The problem is that my favorite vendor does not let me install ECMP routes in case of routes learnt from extrnal BGP peers. Assuming that we are able to install EBGP ECMP routes, how do we advertise this information to our downstream peers? As far as my working knowledge of BGP4 goes, it wouldnt let me do this. I wish to understand how other network operators do this?
You don't, not usually anyway. You advertise the best path to your downstreams. If you want to 'load balance' per packet or otherwise to one or more upstreams that's an internal/your AS decision only. There's nothing to tell the downstreams about from BGP's point of view.
I think there is a need to tell my downstream peers about ASes the traffic is gonna go through. I'm thinking wildly, and it may not make a lot of sense but heres the scenario i have in mind: You load balance (per stream which is usually what most of the vendors do) and you distribute your traffic through ASes 10 and 20. Now you are advertising only one BGP path, say the one through AS 10. Isnt this a problem? Isnt "Advertise what you yourself use" one of the basic shibboleths of BGP or routing for that matter? Regards, John
How exactly you loadbalance the outgoing traffic totally depends on your hardware of choice.
You can, if you wish, send me a message offline and i will collate all the information that i receive and send out a consolidated reply for the benefit of others to this mailing list. Thanks, John
-- "Genius might be described as a supreme capacity for getting its possessors into trouble of all kinds." -- Samuel Butler
On Mon, 26 Jun 2006, John Smith wrote:
Replying to what most of the offline replies that i received said:
We wish to load balance the traffic for a block/range of IP addresses that we learn via BGP4 from our two upstream providers. The problem is that my favorite vendor does not let me install ECMP routes in case of routes learnt from extrnal BGP peers. Assuming that we are able to
you could leak from BGP to 'igp' and make sure you have both paths in the IGP. NOTE: this is possibly very dangerous... you've been warned sorta :) (why dangerous? something breaks in your leak mechanism and you drop 'full internet routes' on ospf/eigrp/isis... network go boom! it is fun to watch though.)
install EBGP ECMP routes, how do we advertise this information to our downstream peers? As far as my working knowledge of BGP4 goes, it wouldnt let me do this.
correct... BGP selects 'best path' and sends that along to it's neighbors. There is a flag on one vendor I believe to force it to send 'all paths', but this is also dangerous, or could be if misused. Perhaps someone who's used that feature could speak up?
I wish to understand how other network operators do this?
You don't, not usually anyway. You advertise the best path to your downstreams. If you want to 'load balance' per packet or otherwise to one or more upstreams that's an internal/your AS decision only. There's nothing to tell the downstreams about from BGP's point of view.
I think there is a need to tell my downstream peers about ASes the traffic is gonna go through.
There isn't a facility in bgp to tell a neighbor more than one possible aspath... or not one that most network folk use currently. I suppose for a subset of routes you might hack up some community based solution, but it'd be a horrible hack, and it'd cause you to keep churning your router configs on a very regular basis as things up stream changed. If the downstream has a connection only to you does it matter where they send packets? everything has to go through your AS to get anywhere... right? If they have a multihomed solution (you and another isp) they are going to have to decide on some other internal metric (interal to them based perhaps on non-routing-table information, like 'john has a oc-12 to provider-Y, Jim only has a T1.... send to John!') whete to send traffic.
I'm thinking wildly, and it may not make a lot of sense but heres the scenario i have in mind: You load balance (per stream which is usually
per flow... is the normal terminology I think, but sure.
what most of the vendors do) and you distribute your traffic through ASes 10 and 20. Now you are advertising only one BGP path, say the one through AS 10. Isnt this a problem? Isnt "Advertise what you yourself use" one of the basic shibboleths of BGP or routing for that matter?
BGP will only pick 'one best path', So, unless you did some local static or IGP based thing (see the leak suggestion above) you'll only really be using one path to AS10 or AS20, and only be sending internally (then externally on the other side of the network) one path. If you were sending to AS10 initially and that link failed or otherwise became 'worst path' you churn on your edge then ship an update with new path info along to your ebgp peers... They have to then churn and decide which path is 'best' and move forward. What benefit is there in sending them 2 paths? They still must remove a path and re-converge, eh? (if you could even send them 2 paths of course) Oh, and to throw in another monkey wrench... if you really wanted to do this for some reason you COULD provide ebgp-multihop peers to your border routers to all customers (ebgp neighbors) that wanted this 'service'... again, this is messy and ugly, but it'd get them multiple copies of the same route, they could then decide on 'best path' based on this information. (this also is not recommended, just a thought) (glad someone atleast replied offline :) ) -Chris
On Mon, Jun 26, 2006 at 02:06:03AM +0000, Christopher L. Morrow wrote:
There is a flag on one vendor I believe to force it to send 'all paths',
How so? BGP as protocol doesn't allow that, unless you use e.g. route distinguisher to... distinguish them. But them we're firmly into the "special hacks" realm. :-)
Perhaps someone who's used that feature could speak up?
I'd be interested in the technical implementation as well... Best regards, Daniel -- CLUE-RIPE -- Jabber: dr@cluenet.de -- dr@IRCnet -- PGP: 0xA85C8AA0
On Mon, 26 Jun 2006, Daniel Roesen wrote:
On Mon, Jun 26, 2006 at 02:06:03AM +0000, Christopher L. Morrow wrote:
There is a flag on one vendor I believe to force it to send 'all paths',
How so? BGP as protocol doesn't allow that, unless you use e.g. route
Hrm, so I could be speaking out of turn :( I was going to find some config that did this, then decided to not :( Looking around I don't see it handy :( and I was probably smoking something good when I remembered this the first time around :(
distinguisher to... distinguish them. But them we're firmly into the "special hacks" realm. :-)
apparenlty only legally available in AMS :(
Perhaps someone who's used that feature could speak up?
I'd be interested in the technical implementation as well...
sorry for the false hope :( I suppose if I'd thought through the problem: 1) all inbound ebgp/ibgp routes to local-rib's 2) decision process in BGP RIB (best path selection) 3) outbound bgp policy application/sending 4) FIB population it would have aviled my faux-pas :( Sorry about that.
I'd be interested in the technical implementation as well...
I vaguely remember Joel presenting a proposal in the Iast IETF which talked about being able to do more or less the same. A bit of googling gives me the drafts that were presented: http://www.ietf.org/internet-drafts/draft-bhatia-bgp-multiple-next-hops-00.t... http://www.ietf.org/internet-drafts/draft-bhatia-ecmp-routes-in-bgp-02.txt The first lets a BGP peer advertise multiple paths for a prefix to its peers and the second, built upon the first, proposes extensions to BGP for advertising ECMP BGP routes. These are still individual submissions, so i dont think there gonna be any implementations out there in the wild! Glen
Best regards, Daniel
-- CLUE-RIPE -- Jabber: dr@cluenet.de -- dr@IRCnet -- PGP: 0xA85C8AA0
you could leak from BGP to 'igp' and make sure you have both paths in the IGP. NOTE: this is possibly very dangerous... you've been warned sorta :) (why dangerous? something breaks in your leak mechanism and you drop 'full internet routes' on ospf/eigrp/isis... network go boom! it is fun to watch though.)
Well leaking BGP into IGP is oooooooouutttta question for obvious reasons !!! :-)
install EBGP ECMP routes, how do we advertise this information to our downstream peers? As far as my working knowledge of BGP4 goes, it wouldnt let me do this.
correct... BGP selects 'best path' and sends that along to it's neighbors. There is a flag on one vendor I believe to force it to send 'all paths', but this is also dangerous, or could be if misused. Perhaps someone who's used that feature could speak up?
I would too be interested in this!
I wish to understand how other network operators do this?
You don't, not usually anyway. You advertise the best path to your downstreams. If you want to 'load balance' per packet or otherwise to one or more upstreams that's an internal/your AS decision only. There's nothing to tell the downstreams about from BGP's point of view.
I think there is a need to tell my downstream peers about ASes the traffic is gonna go through.
There isn't a facility in bgp to tell a neighbor more than one possible aspath... or not one that most network folk use currently.
I suppose for a subset of routes you might hack up some community based solution, but it'd be a horrible hack, and it'd cause you to keep churning your router configs on a very regular basis as things up stream changed.
If the downstream has a connection only to you does it matter where they send packets? everything has to go through your AS to get anywhere... right? If they have a multihomed solution (you and another isp) they are going to have to decide on some other internal metric (interal to them based perhaps on non-routing-table information, like 'john has a oc-12 to provider-Y, Jim only has a T1.... send to John!') whete to send traffic.
I dont think its as simple as this. In the simplest case, assume that my downstream peer has a policy to reject all routes that traverse AS 20. Now i am splitting all load across AS 10 and AS 20, while i tell him that i'll be only sending the traffic through AS 10. This can create problems for my downstream peers, and in the worst case can lead to blackholes/loops.
I'm thinking wildly, and it may not make a lot of sense but heres the scenario i have in mind: You load balance (per stream which is usually
per flow... is the normal terminology I think, but sure.
Blame it on the lack of coffee and early morning blues .. it happens sometimes ! :-)
what most of the vendors do) and you distribute your traffic through ASes 10 and 20. Now you are advertising only one BGP path, say the one through AS 10. Isnt this a problem? Isnt "Advertise what you yourself use" one of the basic shibboleths of BGP or routing for that matter?
BGP will only pick 'one best path', So, unless you did some local static or IGP based thing (see the leak suggestion above) you'll only really be using one path to AS10 or AS20, and only be sending internally (then externally on the other side of the network) one path.
If you were sending to AS10 initially and that link failed or otherwise became 'worst path' you churn on your edge then ship an update with new path info along to your ebgp peers... They have to then churn and decide which path is 'best' and move forward. What benefit is there in sending them 2 paths? They still must remove a path and re-converge, eh? (if you could even send them 2 paths of course)
I dont mean to send them 2 paths! Its just a question that i am posing to the network operators and/or the vendors in this list !! Cheers, John
Oh, and to throw in another monkey wrench... if you really wanted to do this for some reason you COULD provide ebgp-multihop peers to your border routers to all customers (ebgp neighbors) that wanted this 'service'... again, this is messy and ugly, but it'd get them multiple copies of the same route, they could then decide on 'best path' based on this information. (this also is not recommended, just a thought)
(glad someone atleast replied offline :) )
-Chris
On Mon, 26 Jun 2006 06:28:41 +0000 (GMT), "John Smith" <jsmith4112003@yahoo.co.uk> said: [snip]
There isn't a facility in bgp to tell a neighbor more than one possible aspath... or not one that most network folk use currently.
I suppose for a subset of routes you might hack up some community based solution, but it'd be a horrible hack, and it'd cause you to keep churning your router configs on a very regular basis as things up stream changed.
If the downstream has a connection only to you does it matter where they send packets? everything has to go through your AS to get anywhere... right? If they have a multihomed solution (you and another isp) they are going to have to decide on some other internal metric (interal to them based perhaps on non-routing-table information, like 'john has a oc-12 to provider-Y, Jim only has a T1.... send to John!') whete to send traffic.
I dont think its as simple as this. In the simplest case, assume that my downstream peer has a policy to reject all routes that traverse AS 20. Now i am splitting all load across AS 10 and AS 20, while i tell him that i'll be only sending the traffic through AS 10. This can create problems for my downstream peers, and in the worst case can lead to blackholes/loops.
If an AS is balancing traffic outbound in weird ways that's only a problem for their customers if it doesn't work. Customers do otherwise not see any details beyond the ability to filter based on AS-paths they receive, so outbound there's no potential for "blackholes" with current standards which specify that only the best (one) path is announced. For inbound traffic you'd only have a problem if every packet did carry a recorded traversal path and there were equipment around that could make decisions based on such information. What you're asking for simply doesn't exist in today's network. Those who want to the ability control path-selection globally should participate in IETF workshops to get such functionality included in future network-architectures ;) //per -- Per Heldal http://heldal.eml.cc/
Those who want to the ability control path-selection globally should participate in IETF workshops to get such functionality included in future network-architectures ;)
future network architectures, in order to scale, will more likely have less global reach of local te granularity. randy
participants (6)
-
Christopher L. Morrow -
Daniel Roesen -
Glen Kent -
John Smith -
Per Heldal -
Randy Bush