Re: consistent policy != consistent announcements
If a provider/AS does not have the policy of dumping hot potatos to other peers for the traffic to its customers who happen to multihome to those peers. Then there is a simple way to do it. They can have policy of always favor its own customer routes over the routes learned from other peers. This will avoid the inconsistent announcement by eliminating the cases that some part of the AS favors routes M from it's customer and other part favors M from another peer, thus announce consistent routes to other peers. It's very easy to manage this policy. Just assign lower (less favored) local_pref to routes learned from peers than those learned from customers. It's AS based. By using the default local_pref value, one only need to assign a lower than default local_pref value to the peer AS at border routers. A good side effect of this policy is that your customers routes will be protected from being blackingholed from careless ISPs leaking/advertising your customers routes but have no way to reach them. --Jessica ------- Forwarded Message Return-Path: owner-nanog@merit.edu Received: from interlock.ans.net (interlock.ans.net [147.225.5.5]) by cannes.aa.ans.net (8.8.5/8.8.3) with SMTP id QAA24798 for <jyy@cannes.aa.ans.net>; Thu, 13 Mar 1997 16:58:11 -0500 (EST) Received: by interlock.ans.net id AA25369 (InterLock SMTP Gateway 3.0 for regional-techsers@ans.net); Thu, 13 Mar 1997 16:57:00 -0500 Received: by interlock.ans.net (Internal Mail Agent-5); Thu, 13 Mar 1997 16:57:00 -0500 Received: by interlock.ans.net (Internal Mail Agent-4); Thu, 13 Mar 1997 16:57:00 -0500 Received: by interlock.ans.net (Internal Mail Agent-3); Thu, 13 Mar 1997 16:57:00 -0500 Received: by interlock.ans.net (Internal Mail Agent-2); Thu, 13 Mar 1997 16:57:00 -0500 Received: by interlock.ans.net (Internal Mail Agent-1); Thu, 13 Mar 1997 16:57:00 -0500 Message-Id: <v03101919af4e08805b01@[152.160.213.42]> In-Reply-To: <CMM.0.90.2.858281493.vaf@valinor.barrnet.net> References: Your message of Thu, 13 Mar 1997 10:46:06 -0500 (EST) Mime-Version: 1.0 Date: Thu, 13 Mar 1997 16:41:41 -0500 To: Vince Fuller <vaf@valinor.barrnet.net> From: "John G. Scudder" <jgs@ieng.com> Subject: Re: consistent policy != consistent announcements Cc: Sean Donelan <SEAN@sdg.dra.com>, nanog@merit.edu Sender: owner-nanog@merit.edu Content-Type: text/plain; charset="us-ascii" Content-Length: 2715 At 11:31 AM -0800 3/13/97, Vince Fuller wrote:
Well, if that "candidate path" is, in fact, one of Randy's customer, then I would expect him to "cold-potato" route toward it. One would think that to be part of the service that his customer is purchasing.
I think that his internal routing policy and service to his customer is really irrelevant to the question at hand, except insofar as its side-effects affect you.
I certainly shouldn't have to do "cold-potato" routing to his customer, as that customer isn't purchasing anything from me.
Ah, I finally see the problem. In essence, Randy's (announcement) policy assumes that a destination is either in the set of customer routes OR the set of peer routes, but not both. But in this case we have a destination which is in both. The side-effect ends up making you "cold-potato" route to destinatons in the (customer + peer) intersection. The question is, does it make sense for a destination to be considered to be in both sets? If "peer" is supposed to mean "not-customer" then the answer is probably no, there is (should be) no intersection between "customer" and "not-customer". If this is right, the problem is that sometimes the AS path is an overly blunt instrument to determine the customer-ness of a route. In the case where using the AS path doesn't allow the route to be categorized correctly, the only way *I* can see of fixing it is using manual policy (sorry). Presumably this could be done with a minimum of pain by something like (assumes M should properly be categorized as "customer"): (at router peering with customer C in Randy's example): write "customer" community onto all routes from C (at router peering with peer P in Randy's example): write "customer" community onto all routes from (P M) write "peer" community onto all other routes from P (at routers sending routes to external neighbors) send routes with "customer" community to all peer routers send routes with "peer" community to all customer routers send routes with "customer" community to all customer routers The special-casing is limited to the guy who peers with P, who has to decide to write "customer" onto (P M) routes. Presumably one could (semi) automatically detect when a new special case is needed by gathering routing table snapshots from border routers from time to time, and finding routes which are colored inconsistently (identical destinations, different communities). Regards, - --John - -- John Scudder email: jgs@ieng.com Internet Engineering Group, LLC phone: (313) 669-8800 122 S. Main, Suite 280 fax: (313) 669-8661 Ann Arbor, MI 48104 www: http://www.ieng.com ------- End of Forwarded Message
Excuse me for responding myself here. Just like to clarify my previous message a bit. The note below does not intend to solve Randy's problem for his chosen policy. It rather intends to describe a different policy used by many ISPs which won't run into the same problem. IMO, ISPs who are engaging in the 'hot potato' routing practice have the obligation to announce consistent routes to its peers at different peering points. It may require an ISP to change it's internal policy to achive this with reasonable maintance work (like the one described below) or to excercise whatever internal policy it decide but do more work (configuration management,etc) to fulfill the obligation. --Jessica speaking for myself. Date: Fri, 14 Mar 1997 12:48:49 EST To: jgs@ieng.com cc: vaf@valinor.barrnet.net, Sean Donelan <SEAN@sdg.dra.com>, nanog@merit. ***edu From: Jessica Yu <jyy@ans.net> Subject: Re: consistent policy != consistent announcements Return-Path: owner-nanog@merit.edu Sender: owner-nanog@merit.edu Content-Type: text Content-Length: 5173 If a provider/AS does not have the policy of dumping hot potatos to other peers for the traffic to its customers who happen to multihome to those peers. Then there is a simple way to do it. They can have policy of always favor its own customer routes over the routes learned from other peers. This will avoid the inconsistent announcement by eliminating the cases that some part of the AS favors routes M from it's customer and other part favors M from another peer, thus announce consistent routes to other peers. It's very easy to manage this policy. Just assign lower (less favored) local_pref to routes learned from peers than those learned from customers. It's AS based. By using the default local_pref value, one only need to assign a lower than default local_pref value to the peer AS at border routers. A good side effect of this policy is that your customers routes will be protected from being blackingholed from careless ISPs leaking/advertising your customers routes but have no way to reach them. --Jessica
IMO, ISPs who are engaging in the 'hot potato' routing practice have the obligation to announce consistent routes to its peers at different peering points. It may require an ISP to change it's internal policy to achive this with reasonable maintance work (like the one described below) or to excercise whatever internal policy it decide but do more work (configuration management,etc) to fulfill the obligation.
Peers get announced customer routes. In broad brush-strokes, if you don't consistently prefer customer routes over peer routes, and don't reannounce peer routes, you are bound to announce routes inconsistently. If you do reannounce peer routes, you get into all sorts of possible messiness (see below). This much is presumably obvious. Is there any *customer-led* reason why one might not want to prefer customer routes over peer routes? (i.e. not "it saves me doing some backhaul as I can dump the traffic off to the customers other provider"). If we and X (my peer) both transit Y, I would complain if X didn't prefer customer routes over my (peer) routes. Internally-to-X generated traffic (news being an obvious example) or other X-customer traffic which gets into X's network near a peering point far from where X provides transit to Y, I get to do backhaul for. Not nice, not only from a traffic point of view but also a debugging one. Also makes asymmetry worse. I'd also complain if X started announcing routes like <us> <Y>. Mainly from bitter experience where incompetents announced routes like that then blackholed the traffic. So I'd be interested to know why the customer might prefer anything other than "prefer customer routes over peer routes". Alex Bligh Xara Networks
Alex, I haven't seen all of the email on this subject, so forgive me if this was all ready mentioned or if it "goes with out saying" but one faily common case would be when a multi-homed customer prefers on provider as primary and the other/s as a backup. The primary provider would prefer customer announced routes over that of any peers, while the backup providers would set there preferences to normal customers at highest, then peers, and then the customer designated backup at lowest priority. -- Rusty
IMO, ISPs who are engaging in the 'hot potato' routing practice have the obligation to announce consistent routes to its peers at different peering points. It may require an ISP to change it's internal policy to achive this with reasonable maintance work (like the one described below) or to excercise whatever internal policy it decide but do more work (configuration management,etc) to fulfill the obligation.
Peers get announced customer routes. In broad brush-strokes, if you don't consistently prefer customer routes over peer routes, and don't reannounce peer routes, you are bound to announce routes inconsistently. If you do reannounce peer routes, you get into all sorts of possible messiness (see below). This much is presumably obvious.
Is there any *customer-led* reason why one might not want to prefer customer routes over peer routes? (i.e. not "it saves me doing some backhaul as I can dump the traffic off to the customers other provider").
If we and X (my peer) both transit Y, I would complain if X didn't prefer customer routes over my (peer) routes. Internally-to-X generated traffic (news being an obvious example) or other X-customer traffic which gets into X's network near a peering point far from where X provides transit to Y, I get to do backhaul for. Not nice, not only from a traffic point of view but also a debugging one. Also makes asymmetry worse.
I'd also complain if X started announcing routes like <us> <Y>. Mainly from bitter experience where incompetents announced routes like that then blackholed the traffic.
So I'd be interested to know why the customer might prefer anything other than "prefer customer routes over peer routes".
Alex Bligh Xara Networks
Alex, I haven't seen all of the email on this subject, so forgive me if this was all ready mentioned or if it "goes with out saying" but one faily common case would be when a multi-homed customer prefers on provider as primary and the other/s as a backup. The primary provider would prefer customer announced routes over that of any peers, while the backup providers would set there preferences to normal customers at highest, then peers, and then the customer designated backup at lowest priority.
Absolutely. But then while primary is still up one would assume the (primary) peer routes are preferred throughout the network of the backup provider so it won't be announing anything anywhere (so nothing announced would be inconsistent). I guess what I meant was "is there any case where a customer would want a provider to prefer customer routes in some parts of the provider net and peer routes elsewhere?". To answer my own question: One possible scenario for the above is if the customer had their own high capacity ungongested East to West link, but couldn't get peering. Instead the customer takes connectivity from two large providers, A and B, one on the East and one on the West. As the large providers' backbones are congested, the customer elects to do their own backhaul (may be they are a content provider style customer and most of the high volume traffic is outbound and handed off hot-potato). So if A and B have peer routes which are a substantial % of the internet, and the A link is West coast and the B East coast, the customer may prefer to get traffic from A's East coast customers through B, rather than suffer a congested backhaul through A. And vice-versa. However I'd have thought the above scenario was relatively rare. Alex Bligh Xara Networks
Is there any *customer-led* reason why one might not want to prefer customer routes over peer routes? (i.e. not "it saves me doing some backhaul as I can dump the traffic off to the customers other provider").
Yes. Suppose that I am "M", and I have two providers "A" and "B". The links M/A and M/B are expensive international links, much lower bandwidth than I would like, and prone to congestion. Further suppose that A is a customer of R, and B is a peer of R. For load balancing reasons, I would like R to send some of my traffic via A and some via B. Since I pay A and B for transit, and A pays R for transit, and A and B both agree to play along with my desire to load balance, it's reasonable for us to ask R to do this. From R's point of view, their customer A and their indirect customer M have asked them to treat peer routes (via B) and customer routes (via A) to destinations in M as being equivalent. --apb (Alan Barrett)
Yes. Suppose that I am "M", and I have two providers "A" and "B". The links M/A and M/B are expensive international links, much lower bandwidth than I would like, and prone to congestion. Further suppose that A is a customer of R, and B is a peer of R. For load balancing reasons, I would like R to send some of my traffic via A and some via B. Since I pay A and B for transit, and A pays R for transit, and A and B both agree to play along with my desire to load balance, it's reasonable for us to ask R to do this.
From R's point of view, their customer A and their indirect customer M have asked them to treat peer routes (via B) and customer routes (via A) to destinations in M as being equivalent.
This is just plain difficult to orchestrate, if one asks R to treat some subset of peer routes as transit routes. Far better to adopt the reverse solution of having R de-preference the routes received from A to be an equivalent localperf to those received from B. That is, give special treatment to a subset of customer roues, and continue to treat all peer routes the same. RFC1998 provides a very nice example of this, as implemented by one provider (MCI). Nevertheless, this is not a panacea. Peers of R may still receive routes with different as-paths, but as-path lengths and origin codes (which matter for selection) should at least be the same, which should be sufficient. --jhawk
participants (5)
-
Alan Barrett -
Alex.Bligh -
Jessica Yu -
John Hawkinson -
Rusty Zickefoose