On Fri, 01 June 2001, Steve Schaefer wrote:
How is it that UUNet is getting a free ride? The ILEC's customer pays for that line every month. (OK, the customer underpays, but the ILEC gets to make it up in access charges as part of the regulation process.)
The only reason that the traffic is unbalanced is that the ILEC's wrote the rules to favor that traffic arrangement. Too late to whine about it.
Since on the Internet the sender pays for sent traffic, and the receiver pays for received traffic, I've never understood the argument advanced by BBN/Genuity, UUNET and now apparently C&W that unbalanced traffic means someone is getting a free ride. If the customer pays flat-rate, you collect the same amount of money no matter how little traffic they send or receive. The 95% charging used by some providers is the greater of *either* inbound or outbound traffic. So imbalanced traffic to or from your customers is paid by your customers. So, can anyone explain why C&W, UUNET or Genuity care about traffic balance, other than to limit competition by providers who are better at attracting particular types of customers than them? If you are good at being a webhoster, your traffic will have one profile. If you are good at being an access provider, your traffic will have another profile. If you are mediocre at everything, I guess your traffic will be balanced.
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of Sean Donelan Sent: June 1, 2001 10:28 PM To: nanog@merit.edu Subject: RE: C&W Peering Problem?
If the customer pays flat-rate, you collect the same amount of money no matter how little traffic they send or receive. The 95% charging used by some providers is the greater of *either* inbound or outbound traffic. So imbalanced traffic to or from your customers is paid by your customers.
I know this was beaten to death a few threads ago, but there are ISPs (eg: one of ours) that bill 95th percentile on the TOTAL of inbound and outbound.
So, can anyone explain why C&W, UUNET or Genuity care about traffic balance, other than to limit competition by providers who are better at attracting particular types of customers than them? If you are good at being a webhoster, your traffic will have one profile. If you are good at being an access provider, your traffic will have another profile.
The real reason is probably $$$ plain and simple, but... My understanding, based on talking to some people who run networks like @Home which are totally access providers, is that the theory they use it this. Let's say you have network A, a big access network, and network H, a hosting network. If the two networks peer in San Jose, Dallas, Chicago, New York, and Washington, DC, and network H's biggest data centers are in San Jose but network A's biggest customer base is in New York, that means that network H sends lots of traffic through the San Jose peering link, and then network A needs to carry tons of traffic on their backbone all the way to New York. Meanwhile, network A sends acks and similar things to network H, and a majority of those go through the New York peering link, and are then taken back to San Jose on network H. The problem, the way network A sees it, is that they might need to get an OC48 between San Jose and New York, whereas network H can get away with an OC3/OC12 on the same path. Thus, network A finds it unjust that they have to pay all this money for this OC48 when network H, which is the network sending them all this traffic, can get away with a much cheaper circuit, and thus they use this excuse to try and bill network H in order to make as much money as possible. Thus the "free ride" argument.. If traffic is balanced, then I guess each side feels like they're using equivalent amounts of the other's network, and thus that it cancels out... (of course, in a real life implementation, this is presumably significantly more complicated, and I think at that point the logic vanishes in favour of simple greed) Vivien -- Vivien M. vivienm@dyndns.org Assistant System Administrator Dynamic DNS Network Services http://www.dyndns.org/
It seems as if a good solution to this kind of an imbalance would be for the peering partners to agree to listen to each others' MEDs. That way access provider B gets the traffic handed to them a close to the destination as possible. True, that means they need a beefier backbone, but the value in the peering arrangements they don't lose as a result of imbalanced traffic would seem to compensate. Are there any known iplementations like this among backbone providers? -C
My understanding, based on talking to some people who run networks like @Home which are totally access providers, is that the theory they use it this. Let's say you have network A, a big access network, and network H, a hosting network. If the two networks peer in San Jose, Dallas, Chicago, New York, and Washington, DC, and network H's biggest data centers are in San Jose but network A's biggest customer base is in New York, that means that network H sends lots of traffic through the San Jose peering link, and then network A needs to carry tons of traffic on their backbone all the way to New York. Meanwhile, network A sends acks and similar things to network H, and a majority of those go through the New York peering link, and are then taken back to San Jose on network H. The problem, the way network A sees it, is that they might need to get an OC48 between San Jose and New York, whereas network H can get away with an OC3/OC12 on the same path. Thus, network A finds it unjust that they have to pay all this money for this OC48 when network H, which is the network sending them all this traffic, can get away with a much cheaper circuit, and thus they use this excuse to try and bill network H in order to make as much money as possible. Thus the "free ride" argument..
--------------------------- Christopher A. Woodfield rekoil@semihuman.com PGP Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xB887618B
At 12:19 AM 6/2/01 -0400, Vivien M. wrote:
I know this was beaten to death a few threads ago, but there are ISPs (eg: one of ours) that bill 95th percentile on the TOTAL of inbound and outbound.
I almost got caught by this one a few months ago. I was fixing to sign a contract with Exodus for a 100bT circuit when I noticed some funny-looking language and asked some probing questions, and then realized that I had to double their quoted rates before comparing them to everyone else. This moved them from the front of the pack back to UU-land. UUNet is another story. They not only charge significantly more than everyone else, but they calculate 95th percentile on the higher of incoming and outgoing rather than the average. When I asked my salesperson why she couldn't give me a competitive rate, she said "Because we're UUNet." She seemed pretty taken aback when I explained to her that UUNet actually had a pretty bad reputation in NetAdmin circles and I wasn't interested in paying a premium for their name. She still declined to give me a competitive rate. I hear that InFlow charges for average traffic rather than 95th percentile. They're not a backbone, but I wouldn't be surprised to see backbone networks start doing that before too long. It would require some excess capacity, but they would probably make more money in the long run.
On 1 Jun 2001, Sean Donelan wrote:
So, can anyone explain why C&W, UUNET or Genuity care about traffic balance, other than to limit competition by providers who are better at attracting particular types of customers than them?
You have the cart before the horse (effect before cause), there are really two principles that come before the example policy effect above. They are truisms. For the purposes of the rules below the term monopolistic peering refers to core networks that have policies that would limit their peers to 10 or so networks IF they were uniformly applied to all their current peers (which they are invariably not (even though a few might be snubbed for general purposes of crassness, ergo C&W depeering a few arbitrarily)). 1) The first rule of monopolistic peering is that the policy MUST overwhelmingly favor the writer of the policy. This is a truism, no company defines a policy that requires them to pay settlements, only vice versa. For example, this means that for all the lip service paid to settlement based peering compensating the parties equitably you will not find any that allow both parties to be paid. The few settlement based peering contracts I've heard of typically are written as transit fading to settlement free peering when some goal is met (the "correct" ratio and the "correct" quantity of traffic). That is, only one side ever gets compensated. If UUnet offers you a settlement based peering contract you can bet they will not extend you the exact same compensation if you manage to turn the tables on them and attract customers that suck traffic instead of push. Otherwise people would rush out and implement that as a business model. 2) The second rule of monopolistic peering is that the policy MUST be written in a way that allows you to severely limit who you peer with. This is a truism, in order to get the number of qualifying networks down to 10 or so one must write sufficiently restrictive policies. See, Sean, your error was assuming the rationalization accompanying the rules you see in the more restrictive peering policies legitimately represents the effect of the the policy. heh. ;) Hrmmm... By the way, many large networks are not monopolistic. Some have hundreds of peers and continue to evolve and grow their networks, taking into account new market entrants that become sufficiently established. Presumably these networks will provide lower latency and more direct routes (asuming that is what you want) than once upon a time tier one networks that have aggressively shrunk the number of strategic business partners (um, peers, yeah them, who without your network becomes rather boring). +------------------- H U R R I C A N E - E L E C T R I C -------------------+ | Mike Leber Direct Internet Connections Voice 510 580 4100 | | Hurricane Electric Web Hosting Colocation Fax 510 580 4151 | | mleber@he.net http://www.he.net | +---------------------------------------------------------------------------+
On Fri, Jun 01, 2001 at 07:28:03PM -0700, Sean Donelan wrote:
So, can anyone explain why C&W, UUNET or Genuity care about traffic balance, other than to limit competition by providers who are better at attracting particular types of customers than them? If you are good at being a webhoster, your traffic will have one profile. If you are good at being an access provider, your traffic will have another profile.
There are several reasons to care about traffic ratio. Where I think the mistake is made is that providers are looking at ratio, but that the ratios they use are fixed regardless of the type of network they are evaluating. That said, it's hard to get more flexable guidelines past the lawyers and bean counters, particularly in a large orginization. Here's a few interesting cases, first, the ratio problem. A1--------------B1---User | | | | | | server---A2--------------B2 Consider "A" is west coast, "B" is east coast. User requests flow B1, B2, A2, while reponses flow A2, A1, B1. Provider 1 ends up carrying more bits a longer distance, and thus incurs a higher cost. There are several responses to this argument, each with their own problems: * That's what you get for building and end user network. If you don't like it, build a data center network. Most people don't like suggesting their business model is broken. * Use BGP MEDs to make the return route A2, B2, B1. This moves the cost to network 2, which may or may not be fair. Many times provider 1 does not trust provider 2 to do this properly. Even when they do, sometimes it is impossible. BBN and ATT are good examples. If someone sends you a single /8, you have no choice but to hot potato it out, as meds make no sense. The only solution is deaggregation, which has a large number of other problems. * A settlement should be paid from network 2 to network 1. This is possibly acceptable, if it comes in the form of a settlement. Often the pricing resembles transit, below. * Network 2 should buy transit from network 1. Most of the medium to large networks are trying to be transit free, and reject this outright. Also, it's quite likely they would by transit from, well, anyone else just so network 1 doesn't get the money from it. There is an important factor here many of the depeering crowd are missing. The overall traffic ratio of your network is more or less fixed, and is determined by your customer base. Unless you can convert peers to customers (which I have never seen someone be successful in doing on any scale), you will simply move the problem around. That is, if you're 2:1 with Sprint, and depeer 5 10:1 guys, they may well buy transit from Sprint, moving them to 3:1 (due to traffic volume). Now what, depeer Sprint? Most people from their billing software can add up all customer in and all customer out. If your ratio is under that number, you will _NEVER_ reach it, no matter what you do. Since individual peers will be different, you probably want your limit to be about twice your customer ratio, at a minimum. This is why I believe you have to evaulate people based on value. Consider someone like @home peering with someone like Globix. One is a pure end user provider, that in fact prevents most of it's users from running servers and the like. The other is a pure web hosting company, with lots of content and almost no users. These two networks _cannot exist without each other_. If they refused to peer with each other based on ratio, it would be utter folly. Clearly there is great value to both of them in peering, even though the ratio may well be 10:1 or higher. One of the funniest results of the ratio dance is that it may well create more competitiors for a large network. A tight ratio (eg, 1.5:1) is really a requirement that you have a similar customer mix, so you have a similar amount of in+out traffic. How many web hosting networks, who didn't want to compete for end users have been forced to go after end users big time? How many access only networks have done things to attract server users? Companies that could have enjoyed much less competition have forced people to compete with them by ratio. Equally interesting to me is the "minimum traffic" numbers that many large networks want to put forth. Some of them are quite high, with major networks requiring well over a gig of actual traffic to qualify for peering. This has the effect of pushing the restrictive peering policies down to smaller providers. If a smaller provider has a lot of peers, they send less traffic to any individual peer. One of the easiest ways to get that traffic level is to pull peering with a bunch of transit customers of the network you need to increase traffic with, which of course increases your reliance on that paritcular peer. One could wonder if some large providers pushed C&W due to the lack of traffic between their networks (since we know C&W had some issues where they couldn't grow their network last year, and had trouble turning up new customers) and that wasn't one of the catalysts for this most recent action. -- Leo Bicknell - bicknell@ufp.org Systems Engineer - Internetworking Engineer - CCIE 3440 Read TMBG List - tmbg-list-request@tmbg.org, www.tmbg.org
so, if it is primarily the content providers that these "de-peer" rampage folks have their middle finger directed toward; if hosting provider brand X does content caching close to peering locations (thus the return path very likely being symetrical and neither party is hauling the traffic farther than "necessary"), will the "de-peer" folks consider different ratios or no ratios at all? ignoring the how/what part of doing the caching. Sat, Jun 02, 2001 at 02:24:41PM -0400, Leo Bicknell:
On Fri, Jun 01, 2001 at 07:28:03PM -0700, Sean Donelan wrote:
So, can anyone explain why C&W, UUNET or Genuity care about traffic balance, other than to limit competition by providers who are better at attracting particular types of customers than them? If you are good at being a webhoster, your traffic will have one profile. If you are good at being an access provider, your traffic will have another profile.
There are several reasons to care about traffic ratio. Where I think the mistake is made is that providers are looking at ratio, but that the ratios they use are fixed regardless of the type of network they are evaluating. That said, it's hard to get more flexable guidelines past the lawyers and bean counters, particularly in a large orginization.
Here's a few interesting cases, first, the ratio problem.
A1--------------B1---User | | | | | | server---A2--------------B2
Consider "A" is west coast, "B" is east coast. User requests flow B1, B2, A2, while reponses flow A2, A1, B1. Provider 1 ends up carrying more bits a longer distance, and thus incurs a higher cost.
There are several responses to this argument, each with their own problems:
* That's what you get for building and end user network. If you don't like it, build a data center network. Most people don't like suggesting their business model is broken.
* Use BGP MEDs to make the return route A2, B2, B1. This moves the cost to network 2, which may or may not be fair. Many times provider 1 does not trust provider 2 to do this properly. Even when they do, sometimes it is impossible. BBN and ATT are good examples. If someone sends you a single /8, you have no choice but to hot potato it out, as meds make no sense. The only solution is deaggregation, which has a large number of other problems.
* A settlement should be paid from network 2 to network 1. This is possibly acceptable, if it comes in the form of a settlement. Often the pricing resembles transit, below.
* Network 2 should buy transit from network 1. Most of the medium to large networks are trying to be transit free, and reject this outright. Also, it's quite likely they would by transit from, well, anyone else just so network 1 doesn't get the money from it.
There is an important factor here many of the depeering crowd are missing. The overall traffic ratio of your network is more or less fixed, and is determined by your customer base. Unless you can convert peers to customers (which I have never seen someone be successful in doing on any scale), you will simply move the problem around. That is, if you're 2:1 with Sprint, and depeer 5 10:1 guys, they may well buy transit from Sprint, moving them to 3:1 (due to traffic volume). Now what, depeer Sprint?
Most people from their billing software can add up all customer in and all customer out. If your ratio is under that number, you will _NEVER_ reach it, no matter what you do. Since individual peers will be different, you probably want your limit to be about twice your customer ratio, at a minimum.
This is why I believe you have to evaulate people based on value. Consider someone like @home peering with someone like Globix. One is a pure end user provider, that in fact prevents most of it's users from running servers and the like. The other is a pure web hosting company, with lots of content and almost no users.
These two networks _cannot exist without each other_. If they refused to peer with each other based on ratio, it would be utter folly. Clearly there is great value to both of them in peering, even though the ratio may well be 10:1 or higher.
One of the funniest results of the ratio dance is that it may well create more competitiors for a large network. A tight ratio (eg, 1.5:1) is really a requirement that you have a similar customer mix, so you have a similar amount of in+out traffic. How many web hosting networks, who didn't want to compete for end users have been forced to go after end users big time? How many access only networks have done things to attract server users? Companies that could have enjoyed much less competition have forced people to compete with them by ratio.
Equally interesting to me is the "minimum traffic" numbers that many large networks want to put forth. Some of them are quite high, with major networks requiring well over a gig of actual traffic to qualify for peering. This has the effect of pushing the restrictive peering policies down to smaller providers. If a smaller provider has a lot of peers, they send less traffic to any individual peer. One of the easiest ways to get that traffic level is to pull peering with a bunch of transit customers of the network you need to increase traffic with, which of course increases your reliance on that paritcular peer.
One could wonder if some large providers pushed C&W due to the lack of traffic between their networks (since we know C&W had some issues where they couldn't grow their network last year, and had trouble turning up new customers) and that wasn't one of the catalysts for this most recent action.
-- Leo Bicknell - bicknell@ufp.org Systems Engineer - Internetworking Engineer - CCIE 3440 Read TMBG List - tmbg-list-request@tmbg.org, www.tmbg.org
participants (7)
-
Albert Meyer -
Christopher A. Woodfield -
john heasley -
Leo Bicknell -
Mike Leber -
Sean Donelan -
Vivien M.