August 1998 - Test - lists.nanog.org

Re: Sprint/HiNet
by Kyle_Bacon/Fnsi/US＠fnsi.net 26 Aug '98

26 Aug '98

William, If I was forced to guess...I'd guess they are running ATM-DXI on a HSSI interface. Older Cisco IOS versions exhibited similar behavior to what you are seeing below. Traffic flowed fine, but traceroute traffic star'ed out all the time. K william(a)pacific.net.ph on 08/26/98 02:55:23 PM To: nanog(a)merit.edu cc: (bcc: Kyle Bacon/Fnsi/US) Subject: Sprint/HiNet Hi all! Whenever we do a traceroute from our Sprint connection to a host in the HiNet network, we get timed out in the Sprint/HiNet link, as follows: Tracing route to w3.hinet.net [168.95.1.82] over a maximum of 30 hops: 1 138 ms 116 ms 113 ms pi-ph-ts01.pacific.net.ph [210.23.234.128] 2 142 ms 111 ms 116 ms pi-ph-gw01.pacific.net.ph [210.23.234.1] 3 332 ms 330 ms 340 ms 204.59.178.25 4 375 ms 324 ms 328 ms 204.59.120.198 5 323 ms 322 ms 339 ms 144.232.1.153 6 380 ms 366 ms 365 ms 144.232.8.70 7 347 ms 373 ms 374 ms 144.232.4.142 8 * * * Request timed out. 9 546 ms 529 ms 523 ms 168.95.253.126 10 540 ms 532 ms 540 ms 168.95.253.189 11 547 ms 556 ms 547 ms w3.hinet.net [168.95.1.82] Trace complete. Line 8 is always like this and a customer of ours is complaining about it. Is this a time out or congestion or something else? William

1 0

Re: Transaction Based Settlements Encourage Waste (was Re:
by Sean Butler 26 Aug '98

26 Aug '98

vadim wrote: >In any case, nobody was able to connect backbone flow >traces for any significant amount of time. A typical >Internet load on OC-3 would produce about 7000 fps, or >500 mil flows per day. Now, crunching that data to >generate bills is going to be fun. But you're not going to do accounting on backbone links to bill customers, you'd do that at their ingress. Now, if you have customers with OC-3's, then for the time being, I agree, you probably can't bill based on flows... But for customers from dial-up speeds up to DS-3's, the technology is there to do this today. Quesion is: is it cost feasible to do so? Sean ___________________________________ Sean Butler, CCIE #3897 IBM Global Services -- OpenNet Support Phone: 8-631-9809, 813-523-7353 Fax: 8-427-5475 813-878-5475 Internet email: sebutler(a)us.ibm.com

2 1

Re: Equality and asymmetry in network designs
by Sean M. Doran 26 Aug '98

26 Aug '98

Alex Bligh wrote: | > Actually, I do not know of any non-US provider who shares trans-oceanic cost | > with a US provider. They are all considered "customers." Given that | > aggregate links across the Pacific is above 1.6 Gbps, we are out of the old | > "Sprint ICM" days. We're talking serious money at stake. | | I know of several. Yup, and there are equally several U.S.-based providers who bring fully-paid-by-them capacity into various places in Europe, two or three of which connect to European networks on a customer basis. I gather this dramatically improves their connectivity to European destinations. Sean.

1 0

Anyone know how to get Karl at this hour?
by Ryan Brooks 26 Aug '98

26 Aug '98

It appears MCS is advertising one of my blocks (along with a specific that has my primary DNS): * 204.95.0.0/18 198.32.136.5 0 1740 5646 3365 i * 198.32.176.25 0 1740 5646 3365 i *> 198.32.146.20 0 1740 5646 3365 i * 198.32.136.55 0 1740 5646 3365 i * 192.157.69.5 0 1740 5646 3365 i > > BGP routing table entry for 204.95.160.0/24, version 11953916 > Paths: (2 available, no best path, advertised over IBGP) > 1239 5646 3365 (history entry) > 192.157.69.10 from 192.157.69.10 (144.228.62.1) > Origin incomplete, metric 2, localpref 100, external > Community: 2548:667 > Dampinfo: penalty 1558, flapped 3 times in 01:36:59 > 1239 5646 3365 (history entry) > 192.157.69.9 from 192.157.69.9 (144.228.61.1) > Origin incomplete, metric 57, localpref 100, external > Community: 2548:667 > Dampinfo: penalty 1557, flapped 3 times in 01:37:09 > I emailed and called his contact number that's on both his ARIN and NIC contact info, but it's not 24x7. Help! Ryan Brooks ryan(a)inc.net

2 2

Generation of traffic in "settled" peering arrangement
by John Curran 26 Aug '98

26 Aug '98

At 08:47 PM 08/21/1998 -0700, Mike Leber wrote: >... >And any suitably informed party to a peering arrangement like this can >simply offer free hosting to search engine crawlers until they are >extracting the desired amount cash from their peer via settlements. Folks, Think very carefully about what happens as traffic increases... Given that the context is the introduction of "settlement-based" peering for traffic imbalance in a shortest-exit world, then all of the various schemes for generation of traffic (web crawlers, backup services, etc) are no problem whatsoever, and may actually been seen as desirable... Background: Imagine a nationwide Internet provider with customer A on the west coast and customer B on the east coast. Customer B sends queries to customer A's server, and receives quite a bit of data in return. (all of this traffic stays on the provider's backbone in this case) Presume for the moment that the provider charges the customers (A&B) on a usage-basis which takes both sending and received traffic levels into consideration. Now, one could charge each of A and B the full cost of the traffic exchanged (which would effectively be "double dipping" and quite profitable :-) but a competitive market makes it far more likely that the rate per traffic measure will be such that A & B together pay the costs associated with their traffic exchange Now, given that A & B are both on the provider's network, everything is fine. When we hypothesize that A is instead connected to a peer network, we don't know whether costs will be recovered. In a shortest- exit world, traffic is quickly exchanged to the original provider's network and hence the costs of transmission to B are almost identical to the case where the A was a customer. Given that there's only one customer (B) now paying for these costs, this presents a potential problem. Customer B expects to pay their fair share of the costs, but doesn't expect to pay a larger portion of the total costs simply because A is connected to another provider. The truth is, there is actually nothing wrong with this situation if the traffic to/from the peer is of the same order of magnitude. To elaborate: a peer provides traffic which must be carried without corresponding sender-side revenue. On the other hand, the peer also allows the Internet provider to provide a similar level of sender-paid traffic which (by nature of being handed off) has very little actual transmission costs. This is why typical peers are overall neutral to cost recovery, and why peers which send far more traffic than received are problematic. There are a few options that have been mentioned to this situation, including the "receiver pays full freight / sending is free" model, the use of something other than shortest-exit routing, and introduction of settlements for such traffic. For purposes of this thread, we're trying to determine if the settlement approach is doomed to failure due to ability to generate additional traffic flows through innovation. Assessment: Presuming that a peer which is a net exporter of traffic begins to address this by encouraging additional traffic from customers to its network, this would result in more sender-paid originating traffic which quickly moves to the peer. Such traffic is (as noted above) quite profitable, as it includes full cost recover but little actual costs due to shortest to the peer. This is actually good for both networks, and us back to an Internet where more traffic is considered a good thing by ISP's at both ends of the connection. /John p.s. The fact that the sender of traffic should be paying some portion of the resulting costs is not a surprise to anyone; many of the content companies that I've spoken to believe they already are paying more as traffic increases, and were quite surprised to find that it doesn't actually make it to the networks which bear the brunt of the traffic carriage. p.p.s. As noted, departure from shortest-exit is also another approach which may provide some answers to this situation, but that's a different topic which deserves its own thread. This message is simply noting that settling for peering traffic is quite viable, despite assertions to the contrary regarding traffic generation. As long as you're billing the senders on your network for increased usage (and handing it off shortest-exit), increased traffic is good thing.

5 5

Equality and asymetry in network designs
by Sean Donelan 26 Aug '98

26 Aug '98

jcurran(a)bbnplanet.COM (John Curran) writes: >p.s. The fact that the sender of traffic should be paying some portion > of the resulting costs is not a surprise to anyone; many of the > content companies that I've spoken to believe they already are > paying more as traffic increases, and were quite surprised to > find that it doesn't actually make it to the networks which > bear the brunt of the traffic carriage. As someone who pays to carry traffic to both coasts, and pays to carry traffic from both coasts to the center of the country and other places where I notice very few large providers seem to exchange traffic I am perpetually amused by this recurring discussion. If I read Mr. Curran's statement at face value I would assume networks such as DRANET which are net importers of data, and pick up the data on both coasts networks would expect to start receiving payments from large providers with large web hosting centers in cities such as Cambridge. I look at maps such as http://www.bbnplanet.com/products/maps/us_host.htm and see a network which is strong in some areas, and very weak in other areas. St. Louis seems to be a gaping hole in many large providers. Sure, the sales people always say "don't worry, any day we'll have capacity there soon" but if you want something today we have to pay our way to an outer edge of the country. Even if the network shows a "dot" in the middle of the country, when you investigate further, I've usually found limited facilities. I'm sorry if I sound cynical. But I've been down this road of ever changing requirements too many times. "Best-Exit" used to be the norm, I even have a mail message from a Sprint engineer prohibiting "Hot potato" routing if I wanted to peer with Sprint. Then the requirements changed, I modified our network to comply, and the other provider of interest comes up with a different requirement to justify it isn't a "peer" because something is "unequal." The fact is, no two networks are equal in every way. How do you compare networks with large consumer access, web hosting, or large numbers of dedicated access customers? How do you compare a network with a large backbone in the continental USA with a network with a smaller backbone in the USA plus Canada? Or how to you compare a network with lots of transatlantic connectivity, but little cross-USA backbone or trans-pacific connectivity. And the biggest question of all. The USA is still the largest net exporter of data bits in the world. How much should US providers be paying non-USA providers to accept the bits? Right now, other than some NSF-funded connectivity such as Sprint ICM, most non-USA commercial networks must pay the full cost of the inter-continental circuits. Should the Internet adopt a charging model like international voice settlements. Will we see the rise of 'call-back' services on the Internet based on asymmetric charges for inbound and outbound data flows? The simple way to avoid people 'working' the system is to keep charges as close as possible to the actual costs, so there is no incentive. But shareholders prefer companies that operate well above the marginal cost, so you see companies trying to create a pricing model which allows them to charge above cost. I like profit too, but other than certain regulated monopolies and software companies, in a competitive market with low barriers to entry profit margins are going to be narrow. One way to increase the 'spread' is to to raise your competitors costs and make it difficult for new providers to enter the market. Internal network design requirements of the other providers network seem to be the most popular, e.g. you must put pipes of certain sizes into certain locations even if it doesn't make design sense. I've noticed that most of the newest competitors trying to enter the market in the last year have resorted to buying a provider with an existing peering agreement because it was virtually impossible for the new competitor to get peering with some large providers, Level 3->Geonet, Qwest->Eunet, and then pumping up the size of the pipes through those existing interconnect. I've long said I'll agree to any settlement scheme, so long as it is reciprocal. The problem I've seen with most of the schemes going back to the ANS CO+RE charges has been the one-way nature of the settlements. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

4 3

Re: Transaction Based Settlements Encourage Waste (was Re: BBN/GTEI)
by Hank Nussbacher 26 Aug '98

26 Aug '98

At 03:48 PM 8/25/98 -0700, Sean M. Doran wrote: >> What about netflow ? > >[to Vadim Antonov who said that _metering_ traffic is intractable] > >Dear ISP - you have made an accounting error that I am able to demonstrate >in my copious traffic logs. Please credit $x to my account. > >or > >Dear Customer - our copious traffic logs indicate that you generated x traffic >last month. Please pay $x net 30 days. Thank you. > >Dispute resolution -- it's fun! You don't say. Back in 2/95 I hit the following: CSCdi27509: Frame relay counters not updated when fast switching (it was fixed for 10.0. 10.2 and 10.3 fixes were being tested at the time). The MIB stats (inOctets, outOctets) were all wrong for FR subinterface customers. Cisco listed it as a sev=3 bug even though I argued that if ISPs are billing based on this info they may open themselves to a class action suit. Everyone inherently trusts the numbers coming out of your Cisco box. As we have seen bugs in all aspects of IOS code, some dormant for 10 years (see the recent remote crash telnet bug), one would have to independently verify that the numbers Cisco MIBs, netflow of whatever produce are correct, and this procedure would have to be done every time you upgrade to a new IOS version. Ever try doing that? And every time you add a new function you would have to go back and recheck (ever turn on FR traffic shaping and see what happens? :-)) Can of worms. -Hank > > Sean. > >

1 0

Information/Help requested in locating quality Japanese ISP
by Rizzo, Mark 26 Aug '98

26 Aug '98

If there is someone out there who has a good insight of the Japanese Internet market or has done Internet business in Japan could you please drop me an e-mail. Our company is looking to co-locate some high bandwidth server equipment and we need to find the ISP/partner in Japan that has the best Japanese/PAC Rim connectivity. Thanks in advance, Mark Rizzo Manager of Online Operations Origin Systems, INC

1 0

Metered IP billing is indeed possible
by Bruce Hahne 26 Aug '98

26 Aug '98

So, umm, who here recently claimed that packet metering and billing in an ISP/IP setting is technically impossible? In my mailbox this evening I see: > Cisco, Solect Offer Usage-Based Billing Solution > > Cisco Systems, Inc., a global leader in Internet > networking, and Solect Technology Group, a leading provider > of infrastructure software solutions for service providers, > have announced current availability of IAF/NetFlow, an > advanced usage-based billing system integrating Cisco's > NetFlow software into Solect's Internet Administration > Framework (IAF) product used by high-speed, broadband > network and service providers for commercial and business > users. Solect sells telco billing systems. There are other products about to hit the market which do IP metering, and they're going to use telco billing systems as a back end as well. The metered billing steamroller is coming, people. Settlements won't be too far behind. Watch out, because the telcos have over 100 years of experience playing this game. (What was that NANOG 13 T-shirt slogan again, hm?) Bruce Hahne hahne(a)netcom.com

3 2

Buy my box to guarantee your traffic patterns!
by Jon Green 26 Aug '98

26 Aug '98

Hi. I am announcing here the formation of a new company and a new product called "PacketWarrior" which network providers can use to enforce their pre-determined traffic profiles. Not sending enough traffic to one of your peers? PacketWarrior will match their traffic bit for bit to ensure an equal balance. Sending too much traffic to your peers? Configure PacketWarrior with the addresses of web servers on your peer's networks, and it will automatically download web pages over and over until it achieves a balance! Order now for four easy payments of only $1999 each! Billed to your Visa or Mastercard, and shipped discreetly in plain brown paper wrapping. -Jon ----------------------------------------------------------------- * Jon Green * "Life's a dance * * jcgreen(a)netINS.net * you learn as you go" * * Finger for Geek Code/PGP * * * #include "std_disclaimer.h" * http://www.quadrunner.com/~jon * -------------------------------------------------------------------------

1 0