My apologies if this question doesn't belong here. We have a PI /24 we'd like to advertise out of our primary data center for production use. (Well, actually, we'll be advertising a more specific from our /21 assignment, so already not too friendly... but I digress.) We have a disaster recovery site which will have a clone of the myriad production servers. We'd like to fail over to that site automagically. I'm thinking advertising the same prefix and just doing several as-prepends. However, now I'm not sure if this is a polite thing to do or not. Someone mentioned to me something with MEDs, but as soon as that term was used, I started twitching, and couldn't follow the conversation. Would a "good netizen" use the as-prepend method? Or am I missing a simpler/more polite solution? -Christopher
On 16 Feb 2006, at 14:56, Christopher J. Pilkington wrote:
We have a PI /24 we'd like to advertise out of our primary data center for production use. (Well, actually, we'll be advertising a more specific from our /21 assignment, so already not too friendly... but I digress.) [...] I'm thinking advertising the same prefix and just doing several as-prepends. However, now I'm not sure if this is a polite thing to do or not.
How about advertising all of your PI from all of your perimeter routers, and configuring internal routing to ensure packets get to the rightful live/preferred data centre. Then, in the event of a failover situation, you can change the routing behaviour so that packets for the production /24 end up in the alternative dc. To work, this does imply you have good connectivity between your live and failover environments (but i guess you have this so that you can copy over state/live data between your two environments anyway..), so that you can carry customer/live data over the link too. Double plus good for you if you can get cheap transit delivered to your failover dc - you could use this for your preferred environment too ! cheers -a
* Christopher J. Pilkington:
We have a disaster recovery site which will have a clone of the myriad production servers. We'd like to fail over to that site automagically.
I'm thinking advertising the same prefix and just doing several as-prepends. However, now I'm not sure if this is a polite thing to do or not.
Can your backup servers run in parallel to your primary servers? How do you handle conflicting updates (assuming that the services are not completely stateless)?
Hello. Answering the original question, as prepends are not a definitive way to force all traffic to one site or the other. Two ways that definitely will work are. 1. Send a community to the upstream provider at the backup site that causes them to set a lower plocalpref on the route you send them vs. the route learned from the internet. 2. If you have a large enough ip block, Advertise an aggregate address from your backup site, and more specifics from your primary. I.e. advertise a /23 from your backup site and the same /23 as two /24's from the primary. -ejay Thu, 16 Feb 2006 16:10:02 +0100 Florian Weimer <fw@deneb.enyo.de> wrote:
* Christopher J. Pilkington:
We have a disaster recovery site which will have a clone of the myriad production servers. We'd like to fail over to that site automagically.
I'm thinking advertising the same prefix and just doing several as-prepends. However, now I'm not sure if this is a polite thing to do or not.
Can your backup servers run in parallel to your primary servers? How do you handle conflicting updates (assuming that the services are not completely stateless)?
Part of the question is how bad it is for you if you DO get any traffic to your backup datacenter, the connectivity between the datacenters and the datacenters connectivity to the rest of the world. Assuming that you do not have good connectivity between datacenters and that the datacenters have different connectivity to the outside world: While pre-pending should get almost all of your traffic away from your backup DC, you cannot guarantee that it will not get any traffic while the primary is still up. If your primary is connected to ISP_A and the backup is connected to ISP_B, customers connected to ISP_B MAY still flow to your backup DC (ISP_B will probably set local preference on all customer routes - you should be able to override this behavior with communities but not all providers support this (or honor it 100% of the time!)) Announcing a more specific from the primary is likely to work basically all the time (assuming a) your announcement is not too long to be listened to, b) ISP_A and ISP_B don't lose connectivity between themselves). This is not particularly polite however... Another option is just not to announce the backup datacenter until the primary one goes away - see if you can do something like BGP Conditional Advertisement (or your vendor's version of the same). Depending on just how bad having request arrive at the backup datacenter will drive just how paranoid you ned to be - if having your backup get traffic is going to make databases unhappy, etc then you MIGHT even want to consider a manual only failover - if your primary datacenter has a 20 second blip, the pain of dealing with requests that hit the backup during those 20 seconds MAY be greater than just being unavailable for 20 seconds... It all depends on your business, applications, etc, but prepending alone might not be the way to go. Warren On Feb 16, 2006, at 6:56 AM, Christopher J. Pilkington wrote:
My apologies if this question doesn't belong here.
We have a PI /24 we'd like to advertise out of our primary data center for production use. (Well, actually, we'll be advertising a more specific from our /21 assignment, so already not too friendly... but I digress.)
We have a disaster recovery site which will have a clone of the myriad production servers. We'd like to fail over to that site automagically.
I'm thinking advertising the same prefix and just doing several as-prepends. However, now I'm not sure if this is a polite thing to do or not.
Someone mentioned to me something with MEDs, but as soon as that term was used, I started twitching, and couldn't follow the conversation.
Would a "good netizen" use the as-prepend method? Or am I missing a simpler/more polite solution?
-Christopher
On Thu, 16 Feb 2006, Warren Kumari wrote:
If your primary is connected to ISP_A and the backup is connected to ISP_B, customers connected to ISP_B MAY still flow to your backup DC (ISP_B will probably set local preference on all customer routes - you should be able to override this behavior with communities but not all providers support this (or honor it 100% of the time!))
And in addition to that, even multihomed customers of ISP_B may choose the prepended route for a number of different reasons; for instance, ISP_B might be a cheaper pipe for them, or there may be a smart-ish routing device or scheme in play that overrides normal BGP decision making. -- -- Todd Vierling <tv@duh.org> <tv@pobox.com> <todd@vierling.name>
On Fri, 17 Feb 2006, Todd Vierling wrote:
On Thu, 16 Feb 2006, Warren Kumari wrote:
If your primary is connected to ISP_A and the backup is connected to ISP_B, customers connected to ISP_B MAY still flow to your backup DC (ISP_B will probably set local preference on all customer routes - you should be able to override this behavior with communities but not all providers support this (or honor it 100% of the time!))
And in addition to that, even multihomed customers of ISP_B may choose the prepended route for a number of different reasons; for instance, ISP_B might be a cheaper pipe for them, or there may be a smart-ish routing device or scheme in play that overrides normal BGP decision making.
I might be crazy, but couldn't you just prepend the route enough to effectively poison it at ingress to 'backup-isp' ? so they kept chosing the remote path and never really accept the route from local until the remote path is gone?
On Fri, 17 Feb 2006, Christopher L. Morrow wrote:
If your primary is connected to ISP_A and the backup is connected to ISP_B, customers connected to ISP_B MAY still flow to your backup DC (ISP_B will probably set local preference on all customer routes - you should be able to override this behavior with communities but not all providers support this (or honor it 100% of the time!))
And in addition to that, even multihomed customers of ISP_B may choose the prepended route for a number of different reasons; for instance, ISP_B might be a cheaper pipe for them, or there may be a smart-ish routing device or scheme in play that overrides normal BGP decision making.
I might be crazy, but couldn't you just prepend the route enough to effectively poison it at ingress to 'backup-isp' ?
Some route decision override schemes don't care what the path length is at all, or factor it in with such a low weight, such that no reasonable amount of prepending will change the situation. With the development of source traffic engineering schemes, prepending is no longer reliable as a means of affecting routing on the remote side. That perception will have to die with it (hopefully sooner rather than later). -- -- Todd Vierling <tv@duh.org> <tv@pobox.com> <todd@vierling.name>
On Fri, 17 Feb 2006, Todd Vierling wrote:
I might be crazy, but couldn't you just prepend the route enough to effectively poison it at ingress to 'backup-isp' ?
Some route decision override schemes don't care what the path length is at all, or factor it in with such a low weight, such that no reasonable amount of prepending will change the situation.
And while I actually misunderstood what you said, this is still partly correct -- ISP_B will often put a preference onto their own route at a level that outweighs path length absolutely. If ISP_B has a "don't prefer" community, that could work for ISP_B's own customers, but short of removing the advertisement from ISP_B's peers too, that doesn't always move traffic completely off of the ISP_B pipe. This is because traffic entering ISP_B destined for the network *will* usually prefer ISP_B's link in spite of the community, thanks to ISP_B not desiring to transit packets in and out of their network without a revenue point somewhere in the path.
With the development of source traffic engineering schemes, prepending is no longer reliable as a means of affecting routing on the remote side. That perception will have to die with it (hopefully sooner rather than later).
-- -- Todd Vierling <tv@duh.org> <tv@pobox.com> <todd@vierling.name>
On Feb 17, 2006, at 1:25 PM, Christopher L. Morrow wrote:
On Fri, 17 Feb 2006, Todd Vierling wrote:
On Thu, 16 Feb 2006, Warren Kumari wrote:
If your primary is connected to ISP_A and the backup is connected to ISP_B, customers connected to ISP_B MAY still flow to your backup DC (ISP_B will probably set local preference on all customer routes - you should be able to override this behavior with communities but not all providers support this (or honor it 100% of the time!))
And in addition to that, even multihomed customers of ISP_B may choose the prepended route for a number of different reasons; for instance, ISP_B might be a cheaper pipe for them, or there may be a smart-ish routing device or scheme in play that overrides normal BGP decision making.
I might be crazy, but couldn't you just prepend the route enough to effectively poison it at ingress to 'backup-isp' ? so they kept chosing the remote path and never really accept the route from local until the remote path is gone?
Not really - horrendous ASCII art below: Customer / \ / \ ISP_A ---------ISP_B \ / \ / DC1 DC2 Assuming DC is AS_65530, ISP_A is AS_655301 ISP_B is AS_655302 and DC_2 prepends 5 (or some other "large" number) times: Under "normal" conditions: ISP_A sees: 192.0.2.0/24 -- 65530 i (direct from DC1) ISP_B sees 192.0.2.0/24 -- 65530 65530 65530 65530 65530 i (direct from DC2) 192.0.2.0/24 -- 65531 65530 i (ISP_A -> DC_1) <= Best due to AS_PATH Customer sees: 192.0.2.0/24 -- 65531 65530 i (ISP_A -> DC1) <=Best due to AS_PATH 192.0.2.0/24 -- 65532 65531 65530 i (ISP_B -> ISP_A -> DC1) If ISP_B sets Local-Pref on customer routers: ISP_A sees: 192.0.2.0/24 -- 65530 i (direct from DC1) ISP_B sees: 192.0.2.0/24 -- 65530 65530 65530 65530 65530 i (direct from DC2) <- Best due to Local-Pref 192.0.2.0/24 -- 65531 65530 i (ISP_A -> DC_1) Customer sees: 192.0.2.0/24 -- 65532 65530 65530 65530 65530 65530 i (ISP_B -> DC2) 192.0.2.0/24 -- 65531 65530 i (ISP_A -> DC_1) <- Best due to AS_PATH This means that any traffic that enters ISP_B (eg: Customer is singly homed to ISP_B, their connection to ISP_A goes down or they adjust local_pref to prefer ISP_B) will go to DC2. The problem is that Local-Pref trumps basically all other conditions in the BGP decision process - if ISP_B adjusts it it will be prefered in their network no matter how many times you prepend. Warren
On Fri, 17 Feb 2006, Warren Kumari wrote:
On Feb 17, 2006, at 1:25 PM, Christopher L. Morrow wrote:
I might be crazy, but couldn't you just prepend the route enough to effectively poison it at ingress to 'backup-isp' ? so they kept chosing the remote path and never really accept the route from local until the remote path is gone?
Not really - horrendous ASCII art below:
Customer / \ / \ ISP_A ---------ISP_B \ / \ / DC1 DC2
see, I knew I was crazy :( where is that halabi book when I needed it. Did the original poster ever say if he/she was able to accept incidental traffic to the backup DC ?
participants (7)
-
Andy Davidson -
Christopher J. Pilkington -
Christopher L. Morrow -
Ejay Hire -
Florian Weimer -
Todd Vierling -
Warren Kumari