Are there any practical issues with announcing the same route behind different ASNs? Shortly I'll have two seperate sites (EU, US) announcing their own space behind their own ASNs but have a desire to anycast a particular network out of both locations as well. (This is just my attempted to now have to deal with GRE tunnels between sites that aren't logically connected anyways and using the same ASN).
Are there any practical issues with announcing the same route behind different ASNs?
No.
Shortly I'll have two seperate sites (EU, US) announcing their own space behind their own ASNs but have a desire to anycast a particular network out of both locations as well.
(This is just my attempted to now have to deal with GRE tunnels between sites that aren't logically connected anyways and using the same ASN).
Check 192.88.99.0/24. It is an anycasted prefix for 6to4 tunneling. No AS number was assigned for 6to4, thus it has inconsistent AS origin, and works without any problems. Regards, james
On 6-Dec-2006, at 13:03, James Jun wrote:
Check 192.88.99.0/24. It is an anycasted prefix for 6to4 tunneling. No AS number was assigned for 6to4, thus it has inconsistent AS origin, and works without any problems.
Well, without any problems that a consistent origin AS would fix, anyway. Joe
On Wed, Dec 06, 2006 at 09:38:10AM -0800, matthew zeier wrote:
Are there any practical issues with announcing the same route behind different ASNs?
there are any nubmer of self-appointed routing police who will instruct you on their particular brand of correct behaviour.
Shortly I'll have two seperate sites (EU, US) announcing their own space behind their own ASNs but have a desire to anycast a particular network out of both locations as well.
(This is just my attempted to now have to deal with GRE tunnels between sites that aren't logically connected anyways and using the same ASN).
this is done today for the AS112 servers. --bill
On 6-Dec-2006, at 13:05, bmanning@karoshi.com wrote:
this is done today for the AS112 servers.
Actually, I think the origin AS of the AS112 prefix 192.175.48.0/24 is intended to be consistent, and the view from route-views.oregon- ix.net doesn't contradict that theory, in practice. This isn't a rabid endorsement of using consistent origin ASes; merely an observation, since you mentioned it. Joe route-views.oregon-ix.net>show ip bgp 192.175.48.0 BGP routing table entry for 192.175.48.0/24, version 92957 Paths: (44 available, best #40, table Default-IP-Routing-Table) Not advertised to any peer 3277 112, (aggregated by 112 194.85.103.253) 194.85.4.55 from 194.85.4.55 (194.85.4.16) Origin IGP, localpref 100, valid, external Community: 3277:65400 3277:65401 701 10913 10515 112 157.130.10.233 (inaccessible) from 157.130.10.233 (137.39.3.60) Origin incomplete, localpref 100, valid, external 286 1257 8674 112 134.222.85.45 from 134.222.85.45 (134.222.85.45) Origin IGP, localpref 100, valid, external Community: 286:85 286:800 286:3031 286:4001 6395 6453 3557 112 216.140.2.59 from 216.140.2.59 (216.140.2.59) Origin IGP, metric 20, localpref 100, valid, external Community: 6395:200 16150 112 217.75.96.60 from 217.75.96.60 (217.75.96.60) Origin IGP, metric 0, localpref 100, valid, external Community: 16150:90 16150:63392 16150:64520 16150:65308 16150:65320 16150:65330 6079 174 27552 112 207.172.6.162 from 207.172.6.162 (207.172.6.162) Origin IGP, metric 6, localpref 100, valid, external 7018 10515 112 12.0.1.63 from 12.0.1.63 (12.0.1.63) Origin incomplete, localpref 100, valid, external Community: 7018:2000 2905 701 10913 10515 112 196.7.106.245 from 196.7.106.245 (196.7.106.245) Origin incomplete, metric 0, localpref 100, valid, external 5511 3557 112 193.251.245.6 from 193.251.245.6 (193.251.245.6) Origin IGP, localpref 100, valid, external 6395 174 27552 112 216.140.8.59 from 216.140.8.59 (216.140.8.59) Origin IGP, metric 20, localpref 100, valid, external Community: 6395:200 12956 10429 22548 112 213.140.32.146 from 213.140.32.146 (213.140.32.146) Origin IGP, localpref 100, valid, external Community: 10429:110 10429:151 12956:1330 12956:2010 12956:2960 12956:3043 12956:3076 12956:3117 12956:3120 12956:3126 12956:3128 12956:3238 12956:3298 12956:3305 12956:3488 12956:3556 12956:3570 12956:3620 12956:3666 12956:3723 12956:3880 12956:3886 12956:4723 12956:4726 12956:4729 12956:4743 12956:7225 12956:15820 12956:15822 852 6461 3557 112 154.11.98.225 from 154.11.98.225 (154.11.98.225) Origin IGP, metric 0, localpref 100, valid, external Community: 852:180 852 6461 3557 112 154.11.11.113 from 154.11.11.113 (154.11.11.113) Origin IGP, metric 0, localpref 100, valid, external Community: 852:180 6939 112 216.218.252.145 from 216.218.252.145 (216.218.255.241) Origin IGP, localpref 100, valid, external 2914 3557 112 129.250.0.85 from 129.250.0.85 (129.250.0.85) Origin IGP, metric 92, localpref 100, valid, external Community: 2914:410 2914:2000 2914:3000 5650 2914 3557 112 74.40.7.36 from 74.40.7.36 (74.40.0.11) Origin IGP, metric 0, localpref 100, valid, external 5650 174 3557 112 74.40.7.35 from 74.40.7.35 (207.173.112.63) Origin IGP, metric 0, localpref 100, valid, external 6539 19318 112 216.18.63.137 from 216.18.63.137 (216.18.63.137) Origin IGP, localpref 100, valid, external 2914 3557 112 129.250.0.11 from 129.250.0.11 (129.250.0.88) Origin IGP, metric 6, localpref 100, valid, external Community: 2914:410 2914:2000 2914:3000 11608 3557 112 207.246.129.13 from 207.246.129.13 (207.246.129.15) Origin IGP, localpref 100, valid, external Community: 11608:444 11608:801 11608:1008 11608:6601 1668 6461 3557 112 66.185.128.48 from 66.185.128.48 (66.185.128.48) Origin IGP, metric 504, localpref 100, valid, external 4513 19318 112 209.10.12.125 (inaccessible) from 209.10.12.125 (209.10.12.125) Origin IGP, metric 8203, localpref 100, valid, external 4513 19318 112 209.10.12.28 (inaccessible) from 209.10.12.28 (209.10.12.31) Origin IGP, metric 0, localpref 100, valid, external 6079 174 27552 112 207.172.6.20 from 207.172.6.20 (207.172.6.20) Origin IGP, metric 0, localpref 100, valid, external 4513 3557 112 209.10.12.156 (inaccessible) from 209.10.12.156 (209.10.12.156) Origin IGP, metric 0, localpref 100, valid, external 3356 12956 10429 22548 112 4.68.1.166 from 4.68.1.166 (4.68.1.166) Origin IGP, metric 0, localpref 100, valid, external Community: 3356:3 3356:22 3356:100 3356:123 3356:575 3356:2008 10429:110 10429:151 12956:1330 12956:2010 12956:2960 12956:3043 12956:3076 12956:3117 12956:3120 12956:3126 12956:3128 12956:3238 12956:3298 12956:3305 12956:3488 12956:3556 12956:3570 12956:3620 12956:3666 12956:3723 12956:3880 12956:3886 12956:4723 12956:4726 12956:4729 12956:4743 12956:7225 12956:15820 12956:15822 3549 4589 112, (suppressed due to dampening) 208.51.134.254 from 208.51.134.254 (67.17.81.162) Origin IGP, metric 15808, localpref 100, valid, external Community: 3549:4667 3549:31724 4589:2 4589:631 4589:13104 Dampinfo: penalty 11598, flapped 5345 times in 2d15h , reuse in 00:04:33 5459 16150 112 195.66.232.239 from 195.66.232.239 (195.66.232.239) Origin IGP, localpref 100, valid, external Community: 5459:1 5459:60 16150:1 16150:90 16150:63392 16150:64520 16150:65308 16150:65330 293 6509 2884 112 134.55.200.1 from 134.55.200.1 (134.55.200.1) Origin incomplete, localpref 100, valid, external Community: 293:15 293:42 3257 112 213.200.87.254 from 213.200.87.254 (213.200.87.40) Origin IGP, metric 10, localpref 100, valid, external 2493 3602 812 19318 112 206.186.255.223 from 206.186.255.223 (206.186.255.223) Origin IGP, localpref 100, valid, external Community: 812:2 3602:65011 3602:65535 11537 6509 2884 112 198.32.8.196 from 198.32.8.196 (198.32.8.196) Origin incomplete, metric 260, localpref 100, valid, external Community: 11537:2501 1221 4637 3557 112 203.62.252.26 from 203.62.252.26 (203.62.252.26) Origin IGP, localpref 100, valid, external 6453 3557 112 195.219.96.239 from 195.219.96.239 (195.219.96.239) Origin IGP, localpref 100, valid, external 6453 3557 112 207.45.223.244 from 207.45.223.244 (207.45.196.93) Origin IGP, localpref 100, valid, external 2497 7500 7500 7500 112 202.232.0.2 from 202.232.0.2 (202.232.0.2) Origin IGP, localpref 100, valid, external 3333 112 193.0.0.56 from 193.0.0.56 (193.0.0.56) Origin IGP, localpref 100, valid, external 3303 16150 112 164.128.32.11 from 164.128.32.11 (164.128.32.11) Origin IGP, localpref 100, valid, external Community: 3303:1004 3303:1006 16150:90 16150:64520 16150:65308 16150:65330 1239 9009 112 144.228.241.81 from 144.228.241.81 (144.228.241.81) Origin IGP, metric 4294967294, localpref 100, valid, external Community: 1239:123 1239:5000 1239:5090 3292 112 195.215.109.252 from 195.215.109.252 (83.88.48.14) Origin IGP, localpref 100, valid, external, best Community: 3292:1104 3292:1906 22388 11537 6509 2884 112 192.203.116.253 from 192.203.116.253 (192.203.116.253) Origin incomplete, localpref 100, valid, external Community: 11537:2501 22388:100 3561 12956 10429 22548 112 206.24.210.99 from 206.24.210.99 (206.24.210.99) Origin IGP, localpref 100, valid, external 7660 2500 7500 7500 112 203.181.248.168 from 203.181.248.168 (203.181.248.168) Origin IGP, localpref 100, valid, external 6509 2884 112 205.189.32.44 from 205.189.32.44 (205.189.32.44) Origin incomplete, localpref 100, valid, external Community: 2884:112 6509:65001 6509:65030 route-views.oregon-ix.net>
On Wed, 06 Dec 2006 09:38:10 -0800 matthew zeier <mrz@velvet.org> wrote:
Are there any practical issues with announcing the same route behind different ASNs?
This is known as Multiple Origin AS of which you should be able to find plenty of discussion and articles about. It's not uncommon and as far as I know generally doesn't cause any operational problems in and of itself, though doing it should be well thought out and understood since depending on how things fit into the routing topology, packets may not flow as you expect.
Shortly I'll have two seperate sites (EU, US) announcing their own space behind their own ASNs but have a desire to anycast a particular network out of both locations as well.
In the talk on zonecheck.fr in reference to testing for authoritative DNS server set diversity at the OARC meeting, something similar to this came up: <http://public.oarci.net/oarc/workshop-2006/agenda/> That was not part of the public portion, but the slides are available. Since I basically asked that question of the presenter when AS origin diversity was highlighted as one of the tests I'll summarize what I think is a reasonable concensus on the issue in that forum. Having a single origin ASes in the NS RRset may indicate insufficient network connectivity diversity. This is commonly the case where a single AS represents a network at a geographically isolated insitution. In this case it may be appropriate to house a server on another network prefix with a different origin AS and upstream connectivity. In the case of larger networks or anycast however, this may not be such a useful measure of diversity and in fact many large DNS service providers use a single origin AS for all their server instances. One might still argue in those cases that multiple origin ASes might help mitigate problematic local policy decisions such as load balancing that is done based on an ASN or perhaps due to incorrect AS path filters, but I think most would agree that in practice that is a pretty weak argument. John
On Wed, 6 Dec 2006, John Kristoff wrote:
On Wed, 06 Dec 2006 09:38:10 -0800 matthew zeier <mrz@velvet.org> wrote:
Are there any practical issues with announcing the same route behind different ASNs?
This is known as Multiple Origin AS of which you should be able to find plenty of discussion and articles about. It's not uncommon and as far as I know generally doesn't cause any operational problems in and of itself, though doing it should be well thought out and understood since depending on how things fit into the routing topology, packets may not flow as you expect.
As others have said, the origin AS doesn't matter much, but routing topology does. I'm assuming your goal in anycasting the service is at least in part to speed response times. You want clients to go to the closest server. Internet routing tends to follow financial relationships. If a network has traffic to send somewhere, it's best to get paid for it, second best not to have to pay, and having to pay to send it somewhere is a last resort. Customer routes tend to have higher local preferences than peer routes, and peer routes tend to have higher local preferences than transit routes. When local preferences are equal, AS path length comes into play, but in this age of "global" ASes, AS path length doesn't have much to do with distance. For end sites in a single developed world location, this doesn't matter so much. "Global" ASes tend to interconnect "globally" (with some exceptions) and more local ASes tend to interconnect within their coverage area. Hot potato routing tends to produce reasonably direct paths, no matter which networks the traffic goes through along the way. With anycast you have to be a bit more careful. If you get transit from different networks in different places, you may find that your incoming traffic is following customer relationships in undesirable ways. That is, if your transit provider in the US is a big global network, or a customer of a big global network, or a customer of a customer, any traffic that hits that big global network in Europe will flow downstream into your anycast node in the US. Meanwhile, you'll have a different set of customer relationships pulling some of your US traffic into Europe. An easy way around this is to be consistent about your transit and peering arrangements across locations. If your anycast network has transit from a network in one location, get transit from them in your other locations, and let hot potato routing do its thing. For cases where this isn't practical, or where you want more control over who is sending traffic to a node, declare the node to be a "peering only" node and make sure your peers aren't leaking the anycast routes to their out of region upstreams. -Steve
An easy way around this is to be consistent about your transit and peering arrangements across locations. If your anycast network has transit from a network in one location, get transit from them in your other locations, and let hot potato routing do its thing. For cases where this isn't practical, or where you want more control over who is sending traffic to a node, declare the node to be a "peering only" node and make sure your peers aren't leaking the anycast routes to their out of region upstreams.
Or prepend (gasp!) like the Dickens for your "peering only" node to have transit, but be the route chosen after transit/connectivity to your "transit/peering ok" node fails. Deepak
On Wed, 6 Dec 2006, Deepak Jain wrote:
An easy way around this is to be consistent about your transit and peering arrangements across locations. If your anycast network has transit from a network in one location, get transit from them in your other locations, and let hot potato routing do its thing. For cases where this isn't practical, or where you want more control over who is sending traffic to a node, declare the node to be a "peering only" node and make sure your peers aren't leaking the anycast routes to their out of region upstreams.
Or prepend (gasp!) like the Dickens for your "peering only" node to have transit, but be the route chosen after transit/connectivity to your "transit/peering ok" node fails.
Doesn't work. Local preference gets considered before AS path. -Steve
On Wed, 6 Dec 2006, matthew zeier wrote:
Are there any practical issues with announcing the same route behind different ASNs?
Shortly I'll have two seperate sites (EU, US) announcing their own space behind their own ASNs but have a desire to anycast a particular network out of both locations as well.
What is the problem you're trying to solve that you think inconsistent origin AS announcements of the same network would solve which announcements (from same locations) with same AS would not solve?
(This is just my attempted to now have to deal with GRE tunnels between sites that aren't logically connected anyways and using the same ASN).
Yes, understood. But note that different originating AS does not mean traffic you expect to see in one location would not show up in another. The only difference maybe when you're peering with same network from these two locations but then you can provide distinct routing policies using communities. BTW - My suggestion is to consider doing it with 3 ASs, i.e. you have "AS2 AS1" as starting path in location 1 with AS2 being what you use for peering and AS1 being prepended to the routes. At location 2 you do it as "AS3 AS1" with AS3 being used for peering and AS1 prepended. Then to the outside it does not look like you have multiple unconnected locations. But that may not achieve quite the same effect in routing table for anycasting though... --- William Leibzon Elan Networks william@elan.net
william(at)elan.net wrote:
What is the problem you're trying to solve that you think inconsistent origin AS announcements of the same network would solve which announcements (from same locations) with same AS would not solve?
Mostly to avoid GRE tunnels and the added "complexity" therein. My real question was whether anyone actually cared about inconsistent AS paths and did something (drop traffic) because of it. Appears that it's not much of an issue.
My real question was whether anyone actually cared about inconsistent AS paths and did something (drop traffic) because of it. Appears that it's not much of an issue.
Not any significantly sized AS at least.. IMO, filtering inconsistent AS would be more "too much time on hands" and hectic process than building and updating prefix-list to drop top 10 deaggregators listed in cidr-report.org :) In operational sense, there should not be any problems in reaching almost the entire internet in my experience. There are some issues with incon. AS_PATHs as others noted, but technically speaking, BGP in itself doesn't care about it in its route selection process. It cares about AS_PATH length (and AS loop) during its selection process, not what number ends in a path. This is a common confusion by many low/mid-level engineers and sales engineers who claim their network having lower AS number somehow makes them more trafficked and preferred AS :) But what Steve suggested is probably more important operationally for what you're trying to do. Transit/peer/customer relationships often catch first-timers doing anycast by surprise when routes don't go where they want to go in some cases. Regards, james
On Wed, Dec 06, 2006 at 09:38:10AM -0800, matthew zeier wrote:
Are there any practical issues with announcing the same route behind different ASNs? [snip]
In addition to all the sound advice already provided, I would add that if you decide to do something unusual, make sure the documentation trail [publicly-reachable data in whois/irr/dns/etc] is clear. Be aware that from the outside there is no way to tell "clever" from "malicious" and that's why some of the derisively-called routing police get in a twist. Cheers, Joe -- RSUC / GweepNet / Spunk / FnB / Usenix / SAGE
participants (9)
-
bmanning＠karoshi.com -
Deepak Jain -
James Jun -
Joe Abley -
Joe Provo -
John Kristoff -
matthew zeier -
Steve Gibbard -
william(at)elan.net