It's evident, from reading through all of the responses to my questions on exchange of Internet traffic, that there are a lot of problems in this space, but not a lot of agreement on how to address them. Here are some summary observations based on group-wide communication, and notes sent to me directly. 1) More needs to be done to leverage locality of traffic Some great examples were given wherein simple cross-state connections were accomplished via an exchange point some one thousand miles away, a clearly sub-optimal arrangement. Fixing this problem was a point of some contention. Some said to forget about national exchange points, and just make private peering arrangements. Others noted that local exchange points are springing up in the areas that need them. Often these exchange points are the product of local ISPs working cooperatively, sometimes with the assistance of a corporate sponsor. Still others noted some commercial enterprises that are building an infrastructure and business model around the need for decent peering arrangements. To add complexity to this issue, differences arose over implementation details of a national peering system that preserved locality of traffic. Some advocated a hierarchical model, tiered if you will, that frankly sounded a lot like 5-level PSTN switching model. Others advocated a flatter model, although how that model relieves traffic at the core, and leverages locality remains to be seen. 2) Who dropped the packet? You can't fit 20 gallons of water in a ten gallon hat, at least not all at once. Oversubscription generated some heated comments from various factions. It is clear that one person's oversubscription is another person's unethical business practice. So, just how do we handle the delicate topic of oversubscription? Where is it okay to oversubscribe? Where can it absolutely not be tolerated? Oversubscription is fundamental to the design of many network switches, and is an essential aspect of many network technologies, and business plans. It's here. It's reality. How does one use it responsibly? A lot of finger pointing was done in the area of highly asymmetric and grossly underprovisioned connections to the Internet. What policies might mitigate such problems? Thoughts? 3) Internet SLAs and customer verification There was a modest amount of chest beating around verifiable levels of service. Nobody came right out and proposed Internet SLAs, but some did suggest that it would be nice to give the customer the tools for verifying the level of performance delivered by their access provider. This is an interesting angle, and we've seen evidence in the industry of some providers giving the customer limited ability to access this kind of information. NDAs seems to be at the heart of this problem. What tools do the ISPs need to provide this kind of information to the customer? Do the ISPs even want to provide this information? Is it pointless to provide this information since so much of the customer's total performance picture is dependent on factors that can't be controlled by the ISP? dp David R. Pickett Northchurch Communications Inc 5 Corporate Drive Andover, MA 01810 978-691-4649
"Pickett, David" writes:
1) More needs to be done to leverage locality of traffic
In the long run, why are we assuming there will be locality of traffic? It is true that the old PSTN has locality of traffic, but it doesn't have flat rate pricing, or the usage patterns that the Internet has. I argue that users are rarely more likely to be trying to download a web page from near to their homes than from far away. If there is locality, it is probably weak, and in the long run would only account for a fraction of the traffic. Perry
The moving finger of Perry E Metzger, having written:
Perry> "Pickett, David" writes: >> 1) More needs to be done to leverage locality of traffic Perry> In the long run, why are we assuming there will be locality of Perry> traffic? Perry> It is true that the old PSTN has locality of traffic, but it doesn't Perry> have flat rate pricing, or the usage patterns that the Internet has. I Perry> argue that users are rarely more likely to be trying to download a web Perry> page from near to their homes than from far away. If there is Perry> locality, it is probably weak, and in the long run would only account Perry> for a fraction of the traffic. Perry> Perry As to how much of the traffic is local, about half of the people I know at least check email from home. Some are "weekend and evening telecommuters", making quite extensive use of "local connectivity". How this compares to the generic web surfing, I can't say. The CAIDA and NLANR folks say that http traffic is the most significant portion of today's Internet. Perhaps someone who is actually running a local exchange can report on how much traffic they are carrying that is now not being sent to a MAE-equivalent? I think that actual experience and hard data will surprise us all. --tep
I'd guestimate that local peering and stuff accounts for as much as 5 to 15% of our traffic.
Perhaps someone who is actually running a local exchange can report on how much traffic they are carrying that is now not being sent to a MAE-equivalent? I think that actual experience and hard data will surprise us all.
--tep
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Atheism is a non-prophet organization. I route, therefore I am. Alex Rubenstein, alex@nac.net, KC2BUO, ISP/C Charter Member Father of the Network and Head Bottle-Washer Net Access Corporation, 9 Mt. Pleasant Tpk., Denville, NJ 07834 Don't choose a spineless ISP! We have more backbone! http://www.nac.net -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
On Wed, 27 May 1998, Al Reuben wrote:
I'd guestimate that local peering and stuff accounts for as much as 5 to 15% of our traffic.
I don't see numbers quite that high with local peering, but would guestimate that 50% or more of out nntp traffic is across a local exchange. I'm hoping to move another 20%+ of my total traffic off onto AADS next week, which by no means is a local exchange, but Chicago ISP's have never been able to agree on local peering, so AADS it is for Chicago still.
Perhaps someone who is actually running a local exchange can report on how much traffic they are carrying that is now not being sent to a MAE-equivalent? I think that actual experience and hard data will surprise us all.
--tep
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Atheism is a non-prophet organization. I route, therefore I am. Alex Rubenstein, alex@nac.net, KC2BUO, ISP/C Charter Member Father of the Network and Head Bottle-Washer Net Access Corporation, 9 Mt. Pleasant Tpk., Denville, NJ 07834 Don't choose a spineless ISP! We have more backbone! http://www.nac.net -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
============================================================== Tim Flavin Internet Access for St Louis & Chicago Internet 1st, Inc Toll Free Sales & Support 800-875-3173 http://www.i1.net For more information email info@i1.net ==============================================================
Our current average suggests about 10% of our total traffic volume stays within the Austin metro area. I suspect that this may climb as high as 15% with more of the population on the net as a percentage of the total population, and due to the several examples I have of the direct competative advantage of local peering for local providers and local content offerings. At least until the social basis for communication changes to the extent that people prefer to talk to people they don't know v. people they do know. In message <Pine.BSF.3.96.980527162433.21541L-100000@iago.nac.net>, Al Reuben w rites:
I'd guestimate that local peering and stuff accounts for as much as 5 to 15% of our traffic.
Perhaps someone who is actually running a local exchange can report on how much traffic they are carrying that is now not being sent to a MAE-equivalent? I think that actual experience and hard data will surprise us all.
--tep
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Atheism is a non-prophet organization. I route, therefore I am. Alex Rubenstein, alex@nac.net, KC2BUO, ISP/C Charter Member Father of the Network and Head Bottle-Washer Net Access Corporation, 9 Mt. Pleasant Tpk., Denville, NJ 07834 Don't choose a spineless ISP! We have more backbone! http://www.nac.net -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
--- Jeremy Porter, Freeside Communications, Inc. jerry@fc.net PO BOX 80315 Austin, Tx 78708 | 512-458-9810 http://www.fc.net
On Wed, May 27, 1998 at 01:44:08PM -0400, Perry E. Metzger wrote:
1) More needs to be done to leverage locality of traffic
In the long run, why are we assuming there will be locality of traffic?
It is true that the old PSTN has locality of traffic, but it doesn't have flat rate pricing, or the usage patterns that the Internet has. I argue that users are rarely more likely to be trying to download a web page from near to their homes than from far away. If there is locality, it is probably weak, and in the long run would only account for a fraction of the traffic.
The Web is Not The Net. Please write that 100K times on your blackboard. (PS: no, you _can't_ use expect(1l) :-). The fact, however, that you're correct in your implication that it's difficult to prove how much traffic would be geographically local given the routing technologies currently available is why I was going to slip the peering in under the rug of selling the transit -- which everyone seems to be telling me won't work. On another point, it's worth noting that, _currently_, all the "good" servers are somewhere else... but this won't be the case forever. Cheers, -- jra -- Jay R. Ashworth jra@baylink.com Member of the Technical Staff Unsolicited Commercial Emailers Sued The Suncoast Freenet "Two words: Darth Doogie." -- Jason Colby, Tampa Bay, Florida on alt.fan.heinlein +1 813 790 7592 Managing Editor, Top Of The Key sports e-zine ------------ http://www.totk.com
On Wed, 27 May 1998, Jay R. Ashworth wrote:
On another point, it's worth noting that, _currently_, all the "good" servers are somewhere else... but this won't be the case forever.
Especially not if you define "good" servers as those most frequently accessed because you can set up a Squid cache at the local exchange point and if every ISP connected to the local exchange point runs a cache using yours as a parent then the "good" servers miraculously become local servers. The Australians have considerable experience at doing just that including preloading their parent caches, using cheaper one-way satellite bandwidth to load the caches (skycache.com anyone?) and setting up a national backbone between exchange points so that the exchange point caches can all have sibling relationships over a controlled network infrastructure. The folks at http://www.auix.net/ can tell you more, and if you would sign up for the NANOG meeting in Dearborn you could talk to Andrew Khoo andrew@aussie.net and find out more. -- Michael Dillon - Internet & ISP Consulting Memra Communications Inc. - E-mail: michael@memra.com http://www.memra.com - *check out the new name & new website*
On Wed, 27 May 1998, Perry E. Metzger wrote:
"Pickett, David" writes:
1) More needs to be done to leverage locality of traffic
In the long run, why are we assuming there will be locality of traffic?
I don't know how carefully this topic has been studied, but in the UK, where it is easy to distinguish local traffic from non-UK traffic, there most certainly is locality of traffic. It's easy to distinguish local traffic because nearly all UK traffic flows to the LINX, still the UK's major peering point; other traffic goes to international circuits. Most larger ISPs will have one or two LINX connections and one or more international links. It's a no-brainer to count the packets going to each. These counts ignore packets staying within a given network. As far as I know, for all larger UK ISPs (those with good peering), most traffic stays in the UK.
It is true that the old PSTN has locality of traffic, but it doesn't have flat rate pricing, or the usage patterns that the Internet has. I argue that users are rarely more likely to be trying to download a web page from near to their homes than from far away.
Terms like "near" and "far away" are uncomfortably vague in this context. However, I am sure that UK users are most likely to hit UK Web pages first, then US Web pages, with (say) French Web pages far behind either. And Calais is 20 miles away, whereas the USA is 3500 or so.
If there is locality, it is probably weak, and in the long run would only account for a fraction of the traffic.
Our experience isn't like this. The effects of locality are obvious, but locality is not always just a function of georgraphy. -- Jim Dixon VBCnet GB Ltd http://www.vbc.net tel +44 117 929 1316 fax +44 117 927 2015
In the long run, why are we assuming there will be locality of traffic?
I think the right question is: How much local traffic will there be and is that enough to make local shortcuts cost effective? Part of that discussion may involve what "local" means. Things get complicated because current practice is not to charge by the packet-mile but rather by the size of the access pipe. A local shortcut might avoid the need for a larger access pipe or give better response over an existing pipe. So we are still talking real money, it's just that we are looking through a foggy window. ----- Does anybody have back-of-envelope type numbers for per-mile link costs? What does it cost when I fetch a file or web page from the wrong side of the country or ocean? I know the cost won't show up on my bill. I'm looking for some handwaving estimate of the incremental cost to the whole system.
The moving finger of Hal Murray, having written:
Hal> I think the right question is: How much local traffic will there Hal> be and is that enough to make local shortcuts cost effective? Hal> Part of that discussion may involve what "local" means. Actually, I think a more important question will be "*where* do we mean by local"? In large metro areas, or any place with a large *number* of ISPs, there will be advantages to local interconnects. Period. Even if there is distance-insensitive pricing for their connection to a MAE-thing, I would think that they would rather save that precious bandwidth for traffic that really needs to go to the MAE-thing. It could reduce their need to buy a bigger pipe to that major interconnect, just to carry that local traffic "up-and-back". I also can't see how a local interconnect could not be cheaper than a long-haul up to a major interconnect. Considering that many of the locals are co-ops, or non-profits, with cheaper(?) local circuit costs, lower "membership" rates, etc. Also, it is a "commons" thing: If there are no "local" interconnects, then the MAE-things will always be on the verge of oversubscribed in relation to the traffic carrying ability of the technology. Are the major interconnects running at 20% or 80% (or 99%) of the capacity of the underlying fabric? I don't know for sure, but I can hazard a good guess :-) What will happen when the next 10 ISPs buy OC3+ circuits into a major interconnect? If the on-line poplulation is small, or if a majority (or oven if a large minority) of the on-line population is served by a single ISP, then a local interconnect makes less sense. Also, don't forget to count univerities as ISPs; UCSD was the largest single ISP (if you consider local customers) in San Diego until cable modems got started. Now, even though cable modems are more common, UCSD is still one of the major "targets" of those cable modem users. I'm sure that this not an unusual situation. I think that the key is the "diversity" factor of the traffic, in addition to the basic population size. --tep
At 01:44 PM 5/27/98 -0400, Perry E. Metzger wrote:
"Pickett, David" writes:
1) More needs to be done to leverage locality of traffic
In the long run, why are we assuming there will be locality of traffic?
It is true that the old PSTN has locality of traffic, but it doesn't have flat rate pricing, or the usage patterns that the Internet has. I argue that users are rarely more likely to be trying to download a web page from near to their homes than from far away. If there is locality, it is probably weak, and in the long run would only account for a fraction of the traffic.
You seems to be ignoring the single largest cause of rtaffic, e-mail. There is plenty of locality there. ___________________________________________________ Roeland M.J. Meyer, ISOC (InterNIC RM993) e-mail: <mailto:rmeyer@mhsc.com>rmeyer@mhsc.com Internet phone: hawk.mhsc.com Personal web pages: <http://www.mhsc.com/~rmeyer>www.mhsc.com/~rmeyer Company web-site: <http://www.mhsc.com/>www.mhsc.com/ ___________________________________________ SecureMail from MHSC.NET is coming soon!
But we don't want to route SPAM. At 11:05 AM 5/28/98 -0400, Phillip Vandry wrote:
You seems to be ignoring the single largest cause of rtaffic, e-mail. There is plenty of locality there.
You seem to be ignoring the single largest cause of e-mail, spam. There's not much locality there.
:-)
-Phil
___________________________________________________ Roeland M.J. Meyer, ISOC (InterNIC RM993) e-mail: <mailto:rmeyer@mhsc.com>rmeyer@mhsc.com Internet phone: hawk.mhsc.com Personal web pages: <http://www.mhsc.com/~rmeyer>www.mhsc.com/~rmeyer Company web-site: <http://www.mhsc.com/>www.mhsc.com/ ___________________________________________ SecureMail from MHSC.NET is coming soon!
At 11:05 AM -0400 5/28/98, Phillip Vandry wrote:
You seems to be ignoring the single largest cause of rtaffic, e-mail. There is plenty of locality there.
You seem to be ignoring the single largest cause of e-mail, spam. There's not much locality there.
I've been collecting spam stats for almost a year. Spam actually only accounts for a few percent of the email being sent. --Dean ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Plain Aviation, Inc dean@av8.com LAN/WAN/UNIX/NT/TCPIP/DCE http://www.av8.com We Make IT Fly! (617)242-3091 x246 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
participants (13)
-
Al Reuben -
Dean Anderson -
Hal Murray -
Jay R. Ashworth -
Jeremy Porter -
Jim Dixon -
Michael Dillon -
Perry E. Metzger -
Phillip Vandry -
Pickett, David -
Roeland M.J. Meyer -
Tim Flavin -
Tom Perrine