Thanks to all for the on and off list replies, they've been helpful. We get full BGP routes from all upstream connections (currently they are all different providers). The upstream bandwidth is cheaper at site 2 than at site 1 and the private backnet connection is a fixed cost so when previously considering the multi-ASN approach we would plan for each site using the other as a transit/gateway using eBGP but put preference on sending out via site 2 and maybe prepend site 1 AS on the local upstream SP so incoming favors site 2 as well (we're already doing this preferential routing anyways). I don't particularly care for the allow routes for our own ASN arrive from an upstream BGP session especially when it seems like all carriers would need to be cooperative on this, which may not be a big deal overall but adds another layer of complexity and difficulty if we change/add/remove carriers later on. What if they don't all support it, change their policies, or upgrade to a new version of router code that makes the default/expected behavior interfere. I am thinking the multiple ASN route is the cleanest but the idea of letting a default gateway (via static route maybe) out the local upstream connection to reach the other site when the backnet link is down sounds like it would work with minimal to no headaches but it just some how seems like a duct tape job. Does this sort of technique have any significant flaws or concerns associated with it? -----Original Message----- From: Adam Greene [mailto:maillist@webjogger.net] Sent: Saturday, June 06, 2009 8:38 AM To: nanog@nanog.org Subject: Re: Multi site BGP Routing design Hi all, We actually have a very similar setup to what Justin asked about, with the exception that we advertise only some of our netblocks to one provider and the rest to the other. If one of the providers fails, we then advertise all netblocks through the provider which is still up. If the private link between our two locations fails, the two halves of our network communicate via the Internet.
From what Justin described, I would think he would be able to keep a single
ASN and configure his network so that if the private link goes down, the two newly disconnected halves of his network advertise only the netblocks they can still "see" (i.e. the ones on their half). As long as his internal network is set up with dynamic routing (iBGP / OSPF) the two halves should realize they have to get to the other half via the Internet. In our case, we don't get full routing tables from our providers, just default routes. Perhaps in Justin's case something as simple as a floating static route via the Internet to the other half of the network would take care of any ASN weirdness. It doesn't sound like he really needs his border routers to speak BGP with each other while the private link is down. If he wanted to remove the BGP session entirely under these circumstances, he could do the iBGP peering between RFC 1918 addresses and thus force the iBGP session to go down if the private link fails. Thanks, Adam ----- Original Message ----- From: "Saqib Ilyas" <msaqib@gmail.com> To: <nanog@nanog.org> Sent: Saturday, June 06, 2009 8:21 AM Subject: Re: Multi site BGP Routing design
For a given interconnection between the upstream ISPs for the two site, once the direct link goes down, the time required for site A to learn the new route to site B and vice versa would be different with the different proposed solutions, right? Thanks and best regards
On Sat, Jun 6, 2009 at 12:40 PM, Ivan Pepelnjak <ip@ioshints.info> wrote:
To rephrase the OP's question, would it be BCP to acquire a second ASN, and without further de-aggregating, continue advertising each site's IP space to the DFZ, but from dissimilar ASs as opposed to the same one?
This would definitely be the best approach. You're not introducing new IP prefixes and you're not extending AS paths, so the net effect on the global BGP routing is zero (OK, you might have to use the 4 byte AS number :).
Just make sure that both ISPs you connect to allow you to advertise "transit" prefixes. If site A public link goes down, but the private link is up, site B will advertise its own address space plus site A's address space with an extra AS number in the AS path (and the upstream ISP might filter that).
Ivan
-- Muhammad Saqib Ilyas PhD Student, Computer Science and Engineering Lahore University of Management Sciences