August 1998 - Test - lists.nanog.org

Generation of traffic in "settled" peering arrangement
by John Curran 23 Aug '98

23 Aug '98

At 08:47 PM 08/21/1998 -0700, Mike Leber wrote: >... >And any suitably informed party to a peering arrangement like this can >simply offer free hosting to search engine crawlers until they are >extracting the desired amount cash from their peer via settlements. Folks, Think very carefully about what happens as traffic increases... Given that the context is the introduction of "settlement-based" peering for traffic imbalance in a shortest-exit world, then all of the various schemes for generation of traffic (web crawlers, backup services, etc) are no problem whatsoever, and may actually been seen as desirable... Background: Imagine a nationwide Internet provider with customer A on the west coast and customer B on the east coast. Customer B sends queries to customer A's server, and receives quite a bit of data in return. (all of this traffic stays on the provider's backbone in this case) Presume for the moment that the provider charges the customers (A&B) on a usage-basis which takes both sending and received traffic levels into consideration. Now, one could charge each of A and B the full cost of the traffic exchanged (which would effectively be "double dipping" and quite profitable :-) but a competitive market makes it far more likely that the rate per traffic measure will be such that A & B together pay the costs associated with their traffic exchange Now, given that A & B are both on the provider's network, everything is fine. When we hypothesize that A is instead connected to a peer network, we don't know whether costs will be recovered. In a shortest- exit world, traffic is quickly exchanged to the original provider's network and hence the costs of transmission to B are almost identical to the case where the A was a customer. Given that there's only one customer (B) now paying for these costs, this presents a potential problem. Customer B expects to pay their fair share of the costs, but doesn't expect to pay a larger portion of the total costs simply because A is connected to another provider. The truth is, there is actually nothing wrong with this situation if the traffic to/from the peer is of the same order of magnitude. To elaborate: a peer provides traffic which must be carried without corresponding sender-side revenue. On the other hand, the peer also allows the Internet provider to provide a similar level of sender-paid traffic which (by nature of being handed off) has very little actual transmission costs. This is why typical peers are overall neutral to cost recovery, and why peers which send far more traffic than received are problematic. There are a few options that have been mentioned to this situation, including the "receiver pays full freight / sending is free" model, the use of something other than shortest-exit routing, and introduction of settlements for such traffic. For purposes of this thread, we're trying to determine if the settlement approach is doomed to failure due to ability to generate additional traffic flows through innovation. Assessment: Presuming that a peer which is a net exporter of traffic begins to address this by encouraging additional traffic from customers to its network, this would result in more sender-paid originating traffic which quickly moves to the peer. Such traffic is (as noted above) quite profitable, as it includes full cost recover but little actual costs due to shortest to the peer. This is actually good for both networks, and us back to an Internet where more traffic is considered a good thing by ISP's at both ends of the connection. /John p.s. The fact that the sender of traffic should be paying some portion of the resulting costs is not a surprise to anyone; many of the content companies that I've spoken to believe they already are paying more as traffic increases, and were quite surprised to find that it doesn't actually make it to the networks which bear the brunt of the traffic carriage. p.p.s. As noted, departure from shortest-exit is also another approach which may provide some answers to this situation, but that's a different topic which deserves its own thread. This message is simply noting that settling for peering traffic is quite viable, despite assertions to the contrary regarding traffic generation. As long as you're billing the senders on your network for increased usage (and handing it off shortest-exit), increased traffic is good thing.

1 0

Re: BBN/GTEI
by Bruce Hahne 23 Aug '98

23 Aug '98

Dave Rand wrote: >> Assuming it does matter, in which direction does the value flow? and Michael Dillon responds: > Here's the complex part. The value is not expressed in bits and it depends > on the destination within the peer's network. I am assuming that we can > map the IPv4 address space by city and that we can set some value to each > intercity link. This means that a stream of bits entering a peers network > in San Jose with a final destination in San Jose would be free. But if the > stream of bits was destined for Santa Cruz there would be a small cost. I don't see that you've answered his question about value flow. You're describing how to set a value on a stream of packets without answering the question of who should pay. Should we charge the sender or the receiver? Back at the 1995 APRICOT conference there was a mention of an IETF WG (likely now dissolved) which was specifically working on the settlements issue, and their answer to this question was "it depends". For every packet you have to determine a packet owner who is responsible for paying. The owner could vary per application, per site, and/or per end user. I could even imagine that responsibility for payment could be split between sender and receiver: 50/50, 90/10, choose your favorite. If I'm downloading a copy of sendmail, I'm probably the "owner" of those ftp packets because Berkeley makes sendmail available as a public service and doesn't want to pay settlement costs to reach me. If I'm downloading a product white paper, maybe I expect the company that wrote the white paper to pay transit costs to reach me because it's a marketing expense that they should absorb. If somebody is sending me spam, I want the spammer to be the owner of those SMTP packets because I'm certainly not going to pay a settlement fee so that people can spam me. Back to Dave Rand again: >Down this path lies madness. Perhaps. On the other hand, there are already multiple Silicon Valley startups whose business models revolve entirely around metering every packet. And several well-known SF Bay area companies are already paying on a strictly metered (per Gb) basis to deliver web and ftp content to customers and potential customers. Metering in a multi-provider environment generally requires settlements. >On-net facilities have made distance-sensitive pricing in most metro areas a >thing of the past. Sub $0.04 per DS0-mile long-haul rates have made long >haul circuits very affordable. High speed WDM dark fiber routes will make >long haul very, very cheap in the next 2 years. This is the old "bandwidth is so cheap that it's free" argument. True, bandwidth is now inexpensive enough for a single DS0 that it's becoming counterproductive, from a marketing standpoint, for the telco to attempt to differentiate based on distance for a voice call. But if we're ever to arrive at the day when I can turn on my 100Mbps video-over-IP feed to my home PC over my FTTH connection, and expect to actually get reasonable performance, we're probably going to need both metering and inter-provider settlements. Bruce Hahne hahne(a)netcom.com

5 10

welcome to the one size fits all Internet discussion swamp
by Jerry Scharf 23 Aug '98

23 Aug '98

I think we're being more than a little silly here. Person A says that the financial model for internet peering should be client funded, and B says no is should be content provider. folks, you can both be right for various situations. Most newspapers and magazines are largely paid for by the advertiser. Most subsciption services are paid for mostly by the clients. In each case the line is not simple, and the delivery media adapts to the money flow and vice versa. The problem comes from one of two major insanity swamps. a) deciding that there is one model that should be used for pricing. b) trying to decide how to classify a given flow, and the contractual concepts involved. Maybe I'm just hardened to it all, but anything I can't describe in a contract with terms, costs and redress is too soft for me to chage money. That's why we have these simple minded peering agreements. Anything beyond the mindless trading starts you onto a slippery slope from which I see no return. Phone settlements are 100x simpler than this stuff, and you see as the cost of haul drops this will become a major part of total cost of a phone call. I can understand bandwidth times distance hauled in simple terms, so saying I may not accept nearest exit routing from someone else is within my scope of simple minded. my last mail on this whole subject, jerry

1 0

Fwd: Re: BBN Peering issues - FCC Notice of Inquiry on subject
by Roeland M.J. Meyer 22 Aug '98

22 Aug '98

It looks like an ISP finally kicked the hornet's nest. Just when the USG thinks it can get out of the Internet business, an ISP does this and the FCC may think it can step in. For those that do not follow NANOG (yes Michael, I know you're there) A recent announcement, from BBN, has caused a very large bit of churn. At the bottom-end there are the pipes that keep it all flowing. It's not about IP assignments, or names, at this level. It's about whom will talk to whom, and for how much. Earlier, when we went to CIDR routing, we also got stuck with non-portable IP addresses. This locked us into certain ISP-based routing hierarchies. There was a tremour, earlier this year, when Sprint began aggrigating their BGP announcements differently. That was only a foreshadowing of things to come. Regardless of how we all view the Internet, if the peering points stop peering, then we all get segregated into a very few large proprietary ponds. These ponds will be unable to communicate with each other. All of these peering points are in the hands of a few commercial entities, mostly telcos. Mostly they are "gentelmen's agreements". The problem is that this appears to be becoming unraveled. For the record, BBN claims that its new owners (GTEI) are coercing this announcement. Since this is a typical telco move, I'm inclined to believe them. There are really two problems here, one is the peering system and the other is FCC involvement. If the FCC determines that it has jurisdiction over these issues then it may decide that it also has jurisdiction over names and IP assignments. In which case the IFWP proceedings are for nought. Magaziner may as well pack it up and go home too. The problem is that the FCC only has jurisdiction in the US. How will it work extra-territorially, existing treaties? If so, then this is the trigger that could cause an internet version of the marines landing in Panama. I hope this reaches y'all out there in Singapore. >To: nanog(a)merit.edu >Reply-To: nlj(a)bellcore.com >Subject: Re: BBN Peering issues - FCC Notice of Inquiry on subject >Date: Wed, 12 Aug 1998 16:46:43 -0400 >From: "Nick Lordi" <nlj(a)bellcore.com> >Sender: owner-nanog(a)merit.edu > > >Now's your chance, the FCC just released a Notice of Inquiry, FCC 98-187, >"Inquiry Concerning the Deployment of Advanced Telecommunications >Capabilities..." available in a variety of formats via > > http://www.fcc.gov/ccb/706/ > >Take a look at paragraph 79: > > What can and should the Commission do to preserve efficient > peering arrangements among Internet companies, especially in the face > of consolidations of large proprietary gateways? We ask for comment > whether the Commission should monitor or have authority over > peering arrangements to assure that the public interest is served. > >But be careful, commenting could be akin to opening "Pandora's Box" ;-) > >Nick > ___________________________________________________ Roeland M.J. Meyer, ISOC (InterNIC RM993) e-mail: <mailto:rmeyer@mhsc.com>rmeyer(a)mhsc.com Internet phone: hawk.mhsc.com Personal web pages: <http://www.mhsc.com/~rmeyer>www.mhsc.com/~rmeyer Company web-site: <http://www.mhsc.com/>www.mhsc.com/ ___________________________________________ SecureMail from MHSC.NET is coming soon!

2 1

RE: Transaction Based Settlements Encourage Waste (was Re: BBN/GT EI)
by Derek Elder 22 Aug '98

22 Aug '98

And this is precisely why the RBOC's are refusing to pay the settlements to the CLEC's. I personally agree that settlement based peering won't work (with current technology). Whoever figures out a good solution gets a truckload of beers from me ;) > > In fact, this already happens with telco termination charges. > For example, > there are CLECs that call up dialup ISPs and offer free PRIs. > They plan > to make more on termination charges than they could possibly > get from the > ISPs in a competitive environment. Aside from the fact that > they don't > have copper in the ground, this is the reason CLECs don't > want residential > customers, since they would be the ones paying termination > fees on originated > calls. When enough CLECs are concentrating on businesses > that only have > incoming calls (ISPs, tech support centers, 900-style > services, etc), Bell > will scream to the FCC that termination charges need to be > reevaluated. > With this approach, eventually *no one* will want customers > that make lots > of outbound calls -- let the other providers have them and > then collect the > money when they call your customers. > > The point is it is an unstable environment. Unlike > capitalism where our > innate greedy nature serves to balance the system, in this > case the greedy > nature serves to unbalance the system by promoting inefficient use of > resources. > > John Tamplin Traveller > Information Services > jat(a)Traveller.COM 2104 West Ferry Way > 256/705-7007 - FAX 256/705-7100 Huntsville, AL 35801 > >

1 0

Re: Transaction Based Settlements Encourage Waste (was Re: BBN/GTEI)
by dlr＠bungi.com 22 Aug '98

22 Aug '98

[In the message entitled "Transaction Based Settlements Encourage Waste (was Re: BBN/GTEI)" on Aug 22, 1:06, Mike Leber writes:] > > Oh, and before anybody comes up with a simplistic (and flawed) rebuttal > such as banning the large web crawlers from settlements, there are plenty > of other legitimate automated methods of getting flows in either > direction. > All I can say is that if settlements are based on who is receiving the traffic, there will be huge increase in the amount of money that networks pay to attract everyone's favortive smurf target, the IRC servers. A 2-day-long 100 Mbps smurf will be _encouraged_ :-) -- Dave Rand dlr(a)bungi.com http://www.bungi.com

3 2

Re: Transaction Based Settlements Encourage Waste (was Re: BBN/GTEI)
by owen＠DeLong.SJ.CA.US 22 Aug '98

22 Aug '98

> On Sat, 22 Aug 1998, Mike Leber wrote: > > > If your split isn't 50/50 (in other words settlement free) I guarantee > > anybody with half a brain will balance out their traffic by hosting the > > correct type of applications so that you owe them money. > > Of course if everybody does this, then the traffic will balance out. Seems > to be a good thing to create the incentive, no? > > > In fact, TRANSACTION BASED SETTLEMENTS ENCOURAGE WASTE to the point where > > it should be obvious to the casual observer that anybody reasonably > > informed would never suggest them. > > Then I must be unreasonably informed because I think that some form of > exchanging money to even out imbalances would give providers more choice. > No, I think your just missing his point, whether deliberately or not... The point was that althoug the traffic flows would appear balanced, they would be artificially balanced by wasteful automatons asking for or delivering traffic with no benefit just for the sake of consuming bandwidth in the opposite direction in order to balance assymetry. > > The factors that are the cause of the inherent problem with transactions > > based settlements are: > > > > 1) Either party can end up paying. > > 2) Transactions are generated by machines. > > 3) Machines can generate transactions which are either positive flow or > > negative flow. (web servers vs web crawlers as two simplistic examples). > > OK. So now you've convinced us that network traffic shres a number of > characteristics with electric power distribution. Yet the power companies > seem to manage quite well with settlements. I wouldn't want to slavishly > imitate their system any more than I want to slavishly imitate the long > distance telcos. But these things can be made to work. > There's a difference. Power is inherently a one-way flow, and whichever way it is flowing obviously has a TO is purchasing from FROM value system built in. This is not inherently true in IP traffic, and so the model doesn't work as well. > > Oh, and before anybody comes up with a simplistic (and flawed) rebuttal > > such as banning the large web crawlers from settlements, there are plenty > > of other legitimate automated methods of getting flows in either > > direction. > > They aren't legitimate if the sole purpose of the method is to create > traffic; that would be fraud. A webcrawler would be fine if it was for a > search engine but not if you just ran it and discarded the data. > It's not fraud if you create real traffic. It's only fraud if you claim traffic that didn't exist. Reread the legal definition of fraud. > -- > Michael Dillon - Internet & ISP Consulting > Memra Communications Inc. - E-mail: michael(a)memra.com > Check the website for my Internet World articles - http://www.memra.com > > > Owen

1 0

Re: Transaction Based Settlements Encourage Waste (was Re: BBN/GTEI)
by William Allen Simpson 22 Aug '98

22 Aug '98

> From: Michael Dillon <michael(a)memra.com> > They aren't legitimate if the sole purpose of the method is to create > traffic; that would be fraud. A webcrawler would be fine if it was for a > search engine but not if you just ran it and discarded the data. > Are you telling me that your model requires examination of the content of packets, and auditing the "legitimate" internal practices of the hosts? Are you telling me that I should not duplicate packets, as that would be "fraud"? Are you telling me that I should not fill my pipe with duplicate Acks, as that would be "fraud"? Michael, go home, take the weekend off, and in your spare time, re-read RFCs 1 to 1123. Take special note that duplication of packets is permitted, even encouraged. Your model does not correspond to the Internet. WSimpson(a)UMich.edu Key fingerprint = 17 40 5E 67 15 6F 31 26 DD 0D B9 9B 6A 15 2C 32

3 2

RE: BBN/GTEI
by owen＠DeLong.SJ.CA.US 22 Aug '98

22 Aug '98

> If we look at any other downstream, or end, customer, they still pay their > provider for a full circuit's worth of bandwidth. Even if they are not > symmetrically using their bandwidth. So, you pay not only the telco, but > your upstream provider for use of data in both directions. ISPs use > customers like this to not only balance their traffic patterns with other > services that suck massive bandwidth (such as universities and dialup), but > to provide additional revenue (by not discounting a circuit in half because > they primarily use traffic headed in one direction) > OK, I completely agree with you so far. > A network that has chosen a business model that is tailored towards outbound > only transit should be prepared to pay for transit the same way their > customers do. Their customers should pay a premium to have a network that > is better connected than they could provide themselves, not so their > upstream can reap the benefits of not having to pay for their own transit. > These companies have opted towards a business model that does not balance > traffic, thus making it difficult to be a "peer" with a network. > I don't get the connection. True, Exodus has a business model tailored towards OUTBOUND flows, but not OUTBOUND TRANSIT. The outbound flows are exactly a portion of the INBOUND traffic above. Nothing else. > This, of course, brings up many, many interesting questions about which > companies have the right to be at the top of the network pyramid. Although > it seems the market place has already determined this, at least a majority > of it. > That's assuming a pyramid is necessary, desirable, etc. In fact, there's no real need for such a pyramid, but at the moment, there does seem to be a small number of larger providers with increasing numbers of providers as you reduce the size of said provider. > This shift in costs have been a long time coming, you can't charge your > customers for bandwidth, and then not expect to have to pay for it yourself. > Exactly. That's exactly why I think that Exodus/BBN are peers and that the assymetrical nature of the relationship is mostly irrelevant at this point. As you've pointed out, the BBN customer pays BBN for a full circuits worth of bandwidth. Exodus provides to BBN what those BBN customers request from Exodus. Exodus does not charge those BBN customers (or BBN) for the requests or related ACKs, nor does Exodus charge the BBN customers or BBN for the data being delivered across Exodus network to the point where BBN has told Exodus they would like to receive the traffic for that customer. Given that, I don't see how you say anyone is currently charging customers for bandwidht and not paying for it themseleves. Exodus runs a very large national network, with very high growth rates. > -jamie > jamie(a)networked.org > > The views stated above are mine and do not reflect those > of my employer. > The views I have expressed are mine and may or may not coincide with my employer. Owen > -----Original Message----- > From: Michael Dillon [mailto:michael@memra.com] > Sent: Friday, August 21, 1998 7:44 PM > To: nanog(a)merit.edu > Subject: Re: BBN/GTEI > > > On Fri, 21 Aug 1998, Karl Denninger wrote: > > No, its actually becoming MORE suitable. Instead of burning the entire > > circuit in both directions, you're only burning half of it now (one > > direction). > > You still have to pay for the whole circuit. > > -- > Michael Dillon - Internet & ISP Consulting > Memra Communications Inc. - E-mail: michael(a)memra.com > Check the website for my Internet World articles - http://www.memra.com > > >

1 0

RE: GTE BBN - Rand originated BUSINESS MODEL discussion
by Richard Irving 22 Aug '98

22 Aug '98

You cannot configure your router from this message... warning. This is continuation of the "BLUE SKY INTERNET BUSINESS MODEL" discussion. Obviously I am missing something..( I may be naive on this) Why would you want to charge hosters for hits ? (seriously !) I cannot conceive charging an author of a popular book for everyone who reads a copy.... Most could never afford it.... Worse, popularity would create hardhship. Now streaming UDP, (UNI/MULTI) growing in popularity so rapidly for multimedia distribution, could be originator based. More of a Television Industry method... The origin would want to make sure they have the credit card of the user who is about to tune into their latest "multimedia transmission" before they start the *push* based application. (Back to "no surprises" concept) Also, to someone else's point... (in a *much* more *real world now* discussion :) In my *opinion*: Your right, details of peering should be between peers. It already *is* between telco's... Next "Lata" (Peer AS's ?) transactions should go to the owner of the physical. (Which if they pull to you, is them...) (If you pull to them, it is you... If you choose not to charge, your choice! ) (Bonus: Owner of last physical collects the *hypothetical* acks fee's.) However, in my heart, I do not feel that the users are *not* responsible... They performed an "ACTION" to cause that data to flow... The hoster did *not*...... Remember, NO surprise bills! I really feel that this is a better philosophy.. I would hate someone being able to call my grandmother collect, without her having the option to say *NO*.... Also, popularity *should be* *desirable* on the internet. But, lets look at the economics of it... THE WORLD could decide to descend on "IGGY the IGUANA" ( a simple mid-west families site hosted on our circuits, *extremely popular* with children worldwide..) Now, should that family be billed 1 million at the end of the month ? Or, should the provider be billed 1 million. Which of these should be punished for providing interesting content ? Or , how about, the 1 billion users who went there ?? What if they were all billed about .001 cents for that transaction, would they be discouraged? I Don't think so... You billed the same 1 million dollars, but everybody paid a price about equal to the service's they received. (and with the hypothetical ack fee's, the ISP could afford to put up pipe to support IGGY's popularity) ("Morris the Cat" sure never "hurt" for cat food :\ Regarding complexity of DS0m/s billing..... Certainly this could be approached with a "region uniform pricing" to the end users... "Surf AnyWhere in the Continental North America, for a simple flat fee .0004c DS0/s! With "Sprat Cents" no more confusing bills" (Any relationship to a real companies approach is purely hysterical, er.. hypothetical ;) But, that wouldn't mean that the *providers* wouldn't settle up in a more controlled detailed fashion... (or maybe not, interesting....) "Cross MCI pathways for a simple .0001 cents a DS0/second, good anywhere in continental america, and your MCI local region remains free" Finally, a fiscal motive to peer. Also, given a hypothetical unit DS0mile/seconds.. What would prevent someone from routing poorly to increase the "cost of call", as the "maximum throughput" (which in turn would generate volume in terms of DS0 *mile* seconds) does not go down *that* linearly with distance... maybe you could throw in a *real* shoe... Mile bands are based on *geography*, not physical path taken. ;) Or even upon "regions" (AS's with SUB-'AS's, a model already currently in use [check your AS-SETS], and regional LATA's). This sure could motivate inter-provider meshing..... And assist in reducing "across the street, via MAE-E" routing... Thoughts? The more efficiently your route your data, the greater the margin, as the costs are reduced... Less efficient ? Your loss, higher cost of call.. You absorb it. (In terms of reduced margins) Now data loss becomes an issue.. This unfortunately would be a challenging issue, as it is the pipe of the ISP connected to a "poor quality" NSP that will be consumed by lossy fabric. Perhaps the accumulated CDR would result in fiscal punishment to the lossy NSP.. "5000 DS0/s were delivered to said NSP, 4000 DS0/s were completed resulting in 150000 DS0m/s".. collected. Hrmmmmm.. A reason to change NSP's ? After all, that NSP is losing you money. NSP's who have a greater completion rate would be a competitive angle... Better circuit, better management = greater DSO/s completion ratio = Competitive advantage. "Use NSP SOMEPROVIDER. We guarantee a 98% DS0/s completion ratio, if *your* DS0/s completion ratio drops below this threshold we will pay you any ways." small print follows, DS0 Second payments based on average "DS0 miles" accumulated on existing completed seconds of traffic, UDP exclusion clause, etc, etc... Maybe I should get some coffee.... I am taking this too seriously. :\

2 2