Re: Dynamically Changing Exit Policy (iBGP)
On Mon, Oct 29, 2007 at 04:53:50PM -0400, Deepak Jain wrote:
You can "nail" down your announcements to external peers by tying their network blocks to a route-of-last resort on one of your loopbacks. This will prevent flapping externally.
Point taken, but it's actually difficult to nail down all of our routes. We have some lone /24's that are not subnetted and thus cannot be used with an 'ip route ... null0' statement. When WAN connectivity drops, the routes flap if we don't have a stable iBGP session. Thus I'd like to steer well clear of severing the iBGP session.
The weights can be added/removed automatically by using a route-map on the routes that will be added/removed by the interface going down.
Only a single internal /30 route will be removed when an interface goes down. I can't come up with a route-map implementation that would add/remove the weights to the routes already received from our eBGP neighbors. If I'm missing something, please let me know.
Normally, however, you wouldn't use iBGP for this and you'd use a heavier, link-aware internal routing protocol like ISIS or OSPF.
We use OSPF internally, but it just carries internal infrastructure addresses. I understand that OSPF is link-aware and can carry knowledge of link bandwidth, but I don't see how it would fit into our exit path policies. We accept full routes from our eBGP neighbors and it's not advisable to inject those into OSPF. Our normal policy must be best-exit, which leaves iBGP as the decision-maker unless I'm missing something. Is there a better IGP-based method of choosing a network exit path that would solve these problems? I ask because I'm curious, not because I know the answer. -- Ben Howell
Benjamin Howell wrote:
Is there a generally accepted method of automatically altering exit policies within an AS?
I'd like to dynamically change from best-exit to a "hot potato" exit policy when an internal DS3 fails. We fail over to a much lower bandwidth link and would like to avoid sending anything but internal traffic over that link. If it's not already clear, this change needs to happen automatically.
I realize that there are two means of accomplishing this:
(1) Set a weight on all routes received from the eBGP peer at each location so that it prefers the direct eBGP peer. (2) Sever the iBGP session by tying the iBGP session to an interface IP address rather than a loopback IP. When the DS3 goes down, so will the knowledge of the remote exit point.
The devil's in the details however. I can't figure out how to make the weight approach work on routes that were received prior to the circuit failure or how to remove the weights once the circuit comes back up.
Severing the iBGP session seems drastic to me, and I'm worried that our advertised routes will be dampened by peers if the internal DS3 starts flapping.
Any input from wiser peers would be greatly appreciated.
-- Ben Howell
Perhaps a drawing of your architecture might make your travails more clear? Benjamin Howell wrote:
On Mon, Oct 29, 2007 at 04:53:50PM -0400, Deepak Jain wrote:
You can "nail" down your announcements to external peers by tying their network blocks to a route-of-last resort on one of your loopbacks. This will prevent flapping externally.
Point taken, but it's actually difficult to nail down all of our routes. We have some lone /24's that are not subnetted and thus cannot be used with an 'ip route ... null0' statement. When WAN connectivity drops, the routes flap if we don't have a stable iBGP session. Thus I'd like to steer well clear of severing the iBGP session.
The weights can be added/removed automatically by using a route-map on the routes that will be added/removed by the interface going down.
Only a single internal /30 route will be removed when an interface goes down. I can't come up with a route-map implementation that would add/remove the weights to the routes already received from our eBGP neighbors. If I'm missing something, please let me know.
Normally, however, you wouldn't use iBGP for this and you'd use a heavier, link-aware internal routing protocol like ISIS or OSPF.
We use OSPF internally, but it just carries internal infrastructure addresses. I understand that OSPF is link-aware and can carry knowledge of link bandwidth, but I don't see how it would fit into our exit path policies.
We accept full routes from our eBGP neighbors and it's not advisable to inject those into OSPF. Our normal policy must be best-exit, which leaves iBGP as the decision-maker unless I'm missing something. Is there a better IGP-based method of choosing a network exit path that would solve these problems? I ask because I'm curious, not because I know the answer.
-- Ben Howell
Benjamin Howell wrote:
Is there a generally accepted method of automatically altering exit policies within an AS?
I'd like to dynamically change from best-exit to a "hot potato" exit policy when an internal DS3 fails. We fail over to a much lower bandwidth link and would like to avoid sending anything but internal traffic over that link. If it's not already clear, this change needs to happen automatically.
I realize that there are two means of accomplishing this:
(1) Set a weight on all routes received from the eBGP peer at each location so that it prefers the direct eBGP peer. (2) Sever the iBGP session by tying the iBGP session to an interface IP address rather than a loopback IP. When the DS3 goes down, so will the knowledge of the remote exit point.
The devil's in the details however. I can't figure out how to make the weight approach work on routes that were received prior to the circuit failure or how to remove the weights once the circuit comes back up.
Severing the iBGP session seems drastic to me, and I'm worried that our advertised routes will be dampened by peers if the internal DS3 starts flapping.
Any input from wiser peers would be greatly appreciated.
-- Ben Howell
On Mon, 29 Oct 2007, Benjamin Howell wrote:
On Mon, Oct 29, 2007 at 04:53:50PM -0400, Deepak Jain wrote:
You can "nail" down your announcements to external peers by tying their network blocks to a route-of-last resort on one of your loopbacks. This will prevent flapping externally.
Point taken, but it's actually difficult to nail down all of our routes. We have some lone /24's that are not subnetted and thus cannot be used with an 'ip route ... null0' statement. When WAN connectivity drops, the routes flap if we don't have a stable iBGP session. Thus I'd like to steer well clear of severing the iBGP session.
Not subnetting them doesn't mean you can't ip route a.b.c.d 255.255.255.0 null0 250 while still routing the /24s internally (with lower metric) or having them connected on some interface.
Only a single internal /30 route will be removed when an interface goes down. I can't come up with a route-map implementation that would add/remove the weights to the routes already received from our eBGP neighbors. If I'm missing something, please let me know. ...
I'd like to dynamically change from best-exit to a "hot potato" exit policy when an internal DS3 fails. We fail over to a much lower bandwidth link and would like to avoid sending anything but internal traffic over that link. If it's not already clear, this change needs to happen automatically.
Are you talking about a single internal DS3, or the more general case of "if any of our internal DS3s are down, we need to route differently"? If it's a simple case of two DS3 connected routers which are iBGP peers and also have directly connected eBGP peers, could you use route-maps to set ip next-hop on iBGP exchanged external routes (setting the ip next-hop to be the IP of the other end of the internal DS3, with a second IP of an eBGP neighbor interface)? I haven't tried it, but it seems like it might do what you want.
(1) Set a weight on all routes received from the eBGP peer at each location so that it prefers the direct eBGP peer. (2) Sever the iBGP session by tying the iBGP session to an interface IP address rather than a loopback IP. When the DS3 goes down, so will the knowledge of the remote exit point.
Another possiblility (I've never tried) would be to configure multiple iBGP sessions...one using loopback IPs, the other using the DS3 interface IPs, exchanging internal routes over both sessions, while exchanging external routes over only the second. If the DS3 goes down, the session exchanging external routes dies. I'm not sure you can do this, but I think by having different peer/endpoint IPs (loopbacks for one session, serial interface IPs for the other), it may work. It may be appropriate to move this thread to the *-nsp list appropriate for your brand of routers. ---------------------------------------------------------------------- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On Tue, Oct 30, 2007 at 12:55:24AM -0400, Jon Lewis wrote:
On Mon, 29 Oct 2007, Benjamin Howell wrote:
On Mon, Oct 29, 2007 at 04:53:50PM -0400, Deepak Jain wrote:
You can "nail" down your announcements to external peers by tying their network blocks to a route-of-last resort on one of your loopbacks. This will prevent flapping externally.
Point taken, but it's actually difficult to nail down all of our routes. We have some lone /24's that are not subnetted and thus cannot be used with an 'ip route ... null0' statement. When WAN connectivity drops, the routes flap if we don't have a stable iBGP session. Thus I'd like to steer well clear of severing the iBGP session.
Not subnetting them doesn't mean you can't ip route a.b.c.d 255.255.255.0 null0 250 while still routing the /24s internally (with lower metric) or having them connected on some interface.
Whoops, some oversights make you feel like an idiot. You're right.
Only a single internal /30 route will be removed when an interface goes down. I can't come up with a route-map implementation that would add/remove the weights to the routes already received from our eBGP neighbors. If I'm missing something, please let me know. ...
I'd like to dynamically change from best-exit to a "hot potato" exit policy when an internal DS3 fails. We fail over to a much lower bandwidth link and would like to avoid sending anything but internal traffic over that link. If it's not already clear, this change needs to happen automatically.
Are you talking about a single internal DS3, or the more general case of "if any of our internal DS3s are down, we need to route differently"?
If it's a simple case of two DS3 connected routers which are iBGP peers and also have directly connected eBGP peers, could you use route-maps to set ip next-hop on iBGP exchanged external routes (setting the ip next-hop to be the IP of the other end of the internal DS3, with a second IP of an eBGP neighbor interface)? I haven't tried it, but it seems like it might do what you want.
Indeed, I'll give it some thought. That seems like it should work. In my case, it is just two DS3 connected routers. I figured I'd leave the question open-ended though for other readers' benefit.
Another possiblility (I've never tried) would be to configure multiple iBGP sessions...one using loopback IPs, the other using the DS3 interface IPs, exchanging internal routes over both sessions, while exchanging external routes over only the second. If the DS3 goes down, the session exchanging external routes dies. I'm not sure you can do this, but I think by having different peer/endpoint IPs (loopbacks for one session, serial interface IPs for the other), it may work.
Actually this suggestion seems to be a common theme. I hadn't considered this possibility and it seems like it should work fine. David Burns also suggested this in an email that wasn't directed to the list. Thanks for everybody's input. I should have some workable options now. -- Ben Howell
participants (3)
-
Benjamin Howell -
Deepak Jain -
Jon Lewis