Re: Request for Comments on a topological address block for N. Calif.
From: asp@uunet.uu.net (Andrew Partan) Lets assume that everyone on the Boston area has a geographic address and that AlterNet and some other ISP (SmallGuy) have some customers in this area, and that AlterNet and SmallGuy peer at some Boston area exchange point. ... Now if I peer with some other ISP in some other area (say someone in San Francisco) ... I have now suddenly offered transit for SmallGuy between San Francisco and Boston. Well, there is a "fix", but this whole topic of traffic is a major can of worms. I assume the basic concept is similar to the existing US 'phone system, in that each provider should do the long-haul for their own customers only (unless some suitable peering arrangement is in place), to prevent "dumping" the long-haul load (i.e. provider P gets the traffic, immediately hands it to Q, who has the carry the bits across the globe...)? (This, of course, does lead to asmmetric routes. I thought I recalled a message from Curtis that talked about that, as part of a longer discussion of this whole topic, but can't find it now....) One fix to do this is for a provider not to accept traffic from another provider unless that traffic is destined to an "appropriate" customer (exact definition varies) of the first provider (or a customer of a peer provider of the first provider, one which has a transit agreement with the first provider). The definition of "appropriate" is tricky; does this mean any customer (which still leaves you vulnerable to "dumping" of long-haul load to customers of the dumpee), or just "local" customers? If the latter, would you rather the customer not get the traffic at all (if the traffic source is attached to a provider which doesn't have an attachment to the destination's provider, near the destination - we'll call this source's provider a minimal-connect provider)? If not, how do you prevent "dumping"? In geographic addressing this means that you don't accept traffic for another metro from a provider unless you have a peering arrangement with them, otherwise you wind up doing the long-haul. This stops "dumping", true, but still leaves your customer not getting traffic from minimal-connect customer. You can do more or less the same thing in connecvity-based addressing, and you have the same cut-off-minimal-connect-guys/allow-long-haul-dumping choice... (Just out of interest, does anyone know how the phone companies did it? I.e. if I'm a new inter-lATA carrier, and I don't yet serve all LATA's, how does it work if I'm one of their customers and I try and call one of those LATA's?) If SmallGuy is not paying me for transit, then I am not going to do this. Ah, there's the rub! What you really want, it seems, is traffic settlements (since I don't think we can adopt/enforce a simplistic you-always-carry-your- customers-traffic-pretty-much-to-the-destination model, which would get rid of any need for settlements) but right now the Internet doesn't have them, for a whole host of good reasons. For one, let's go back to the case of two large providers, P and Q. Right now, P could "dump" its long-haul traffic on Q, knowing Q will give it back sooner or later (unless they each try to dump on the other, which will get them into a routign loop :-). However, what about customers of R, which isn't connected to P, but is to Q? The Internet currently assumes that traffic takes something close to the best path, so we assume that P->R traffic will in fact transit Q, without explicit arrangements between P and Q about it. So Q does have to accept traffic from P which is not to customers of P ("appropriate" or not). There's an interesting analogy here between traffic settlements and routing setllements; the Internet will work fine with no administrative overhead if everyone is reasonable. However, if people try and take advantage of the system, to a point where administrative controls are needed, the overhead of those controls will be substanial, to the point where the cure might be as bad as the disease. Hmm.... Noel
From: asp@uunet.uu.net (Andrew Partan)
Lets assume that everyone on the Boston area has a geographic address and that AlterNet and some other ISP (SmallGuy) have some customers in this area, and that AlterNet and SmallGuy peer at some Boston area exchange point. ... Now if I peer with some other ISP in some other area (say someone in San Francisco) ... I have now suddenly offered transit for SmallGuy between San Francisco and Boston.
Well, there is a "fix", but this whole topic of traffic is a major can of worms.
There's two kinds of traffic that Alternet can provide to SmallGuy: a) non-transit peering, meaning Alternet announces only routes it originates to SmallGuy (and vice-versa, though that's for SmallGuy to ensure!) and takes SmallGuy's route announcements and floods them through Alternet's ASes (but no others!), or b) transit peering, meaning Alternet announces to SmallGuy all routes that SmallGuy is paying to get (Alternet's plus everyone else's presumably, though transit-to-some-but-not-al ASes can also be done, and where Alternet propagates all of SmallGuys route announcements to the ASes SmallGuy buys access to. [Buying access to some ASes but not the whole Internet as seen by Alternet can be hard since some of these ASes may not be directly connected to Alternet, in which case SmallGuy must also pay for transit to the ASes between Alternet and the ones he desires. In practice SmallGuy will buy transit to the whole Internet as heard of by Alternet.] [Non-transit peering between Sprint and Alternet means that Alternet and Sprint announce their own routes plus their transit customers' to each other. This is so because in effect, Alternet's routes to its transit customers can be considered to be Alternet's for the sake of the definition I gave above in (a).] This should make the can of worms vanish.
(This, of course, does lead to asmmetric routes. [...]
Assymetric routes can be avoided.
One fix to do this is for a provider not to accept traffic from another provider unless that traffic is destined to an "appropriate" customer (exact definition varies) of the first provider (or a customer of a peer provider of the first provider, one which has a transit agreement with the first provider).
[I'll assume permission to use the following names for the scenario below: Alternet, Sprint, MCI, SmallGuy. May I? :) Any resemblance to reality is pure coincidence. :)] If Alternet and Sprint have a non-transit peering agreement, then Alternet announces all of its routes, including those of its transit customers, to Sprint and vice-versa. Now if MCI also peers with Sprint and Alternet and f everyone sticks to the definition of transit and non-transit peering (i.e. MCI, Alternet and Sprint configure their routers appropriately) then there's no way that Sprint will transit SmallGuy's traffic to MCI, because MCI won't hear any routes to SmallGuy through Sprint and Alternet will only hear MCI's routes where it peers with MCI (if Alternet heard MCI routes elsewhere, such as through Sprint due to a configuration error, it would still use the route through MCI as it would have a shorter AS-path). There is then no route assymetry in this system. In practice this means that everyone has to be careful to filter *outbound* announcements so that peers hear only what they are meant to hear. So Alternet will have outbound filters that allow only routes it originates plus routes originated by its transit customers to make it to Sprint, and vice-versa. Also, in practice, a new provider that is trying to get access to the whole net will have to arrange to buy transit through someone else until it has enough peering arrangements in place with other providers that it can offer decent reachability to its transit customers. Large providers like Sprint and Alternet end up using large configurations to put this sort of routing policy together.
The definition of "appropriate" is tricky; does this mean any customer (which still leaves you vulnerable to "dumping" of long-haul load to customers of the dumpee), or just "local" customers? If the latter, would you rather the customer not get the traffic at all (if the traffic source is attached to a provider which doesn't have an attachment to the destination's provider, near the destination - we'll call this source's provider a minimal-connect provider)? If not, how do you prevent "dumping"?
By carefully controlling outbound announcements, see above. It's true that a bad player can rig the system so that a non-transit peer will transit one side of the traffic to some other destination (e.g. MCI could put static routes in its routers for SmallGuy pointing to Sprint; Sprint will pick up traffic for SmallGuy from MCI because Sprint has a route to SmallGuy, but traffic between SmallGuy and MCI cannot go through Sprint without Alternet's help). In practice noone cheats like this.
In geographic addressing this means that you don't accept traffic for another metro from a provider unless you have a peering arrangement with them, otherwise you wind up doing the long-haul. This stops "dumping", true, but still leaves your customer not getting traffic from minimal-connect customer. You can do more or less the same thing in connecvity-based addressing, and you have the same cut-off-minimal-connect-guys/allow-long-haul-dumping choice...
If geographic addresses don't match the underlying topology, then dumping is hard to prevent without fragmenting the net or allowing CIDR holes or dropping aggregation, which is to say abandon geographically assigned addresses.
If SmallGuy is not paying me for transit, then I am not going to do this.
Ah, there's the rub! What you really want, it seems, is traffic settlements (since I don't think we can adopt/enforce a simplistic you-always-carry-your- customers-traffic-pretty-much-to-the-destination model, which would get rid of any need for settlements) but right now the Internet doesn't have them, for a whole host of good reasons.
Finely detailed traffic summaries cannot be kept on the net the way telephone calls made can be logged by phone companies, at least not right now. If detailed summaries are ever introduced, the logging will be done at the interface between carriers and their customers, not at exchange points (IMHO), and the summaries will be in the form of packets transited to a certain AS. Or will the sort of router used at exchange points ever have enough oomph to do this sort of traffic logging (source AS, destination AS, packet count).
There's an interesting analogy here between traffic settlements and routing setllements; the Internet will work fine with no administrative overhead if everyone is reasonable. However, if people try and take advantage of the system, to a point where administrative controls are needed, the overhead of those controls will be substanial, to the point where the cure might be as bad as the disease. Hmm....
Sure, but if some players collude to have others transit what they shouldn't and this is discovered, then it's all over for the rogue players. You see, even without a central authority it is possible to have some justice in the system.
Noel
Nick
There's two kinds of traffic that Alternet can provide to SmallGuy:
[ a) deleted...standard peering ]
b) transit peering, meaning Alternet announces to SmallGuy all routes that SmallGuy is paying to get (Alternet's plus everyone else's presumably, though transit-to-some-but-not-al ASes can also be done, and where Alternet propagates all of SmallGuys route announcements to the ASes SmallGuy buys access to.
[Buying access to some ASes but not the whole Internet as seen by Alternet can be hard since some of these ASes may not be directly connected to Alternet, in which case SmallGuy must also pay for transit to the ASes between Alternet and the ones he desires. In practice SmallGuy will buy transit to the whole Internet as heard of by Alternet.]
Unfortunately, this is not particular feasible in any great amount (at least on a Cisco). The differnet filter-lists needed for each peering session would make your configuration downright un-wieldy after a few such arrangements. I starting having to deal with three filter-lists to update on the mae-east net99 router for one similar project, and each time I added a new peer, all three of them had to be updated a little differently. Had it gotten any worse, I would have had to build some script to automatically update my filter-lists for me. Even if it was easy, I really wouldn't want to wade through the configuration file when done. They are getting bad enough already. Dave -- Dave Siegel President, RTD Systems & Networking, Inc. (520)318-0696 Systems Consultant -- Unix, LANs, WANs, Cisco dsiegel@rtd.com User Tracking & Acctg -- "Written by an ISP, http://www.rtd.com/ for an ISP."
participants (3)
-
Dave Siegel -
jnc＠ginger.lcs.mit.edu -
Nick Williams