- NANOG - lists.nanog.org

Re: Ungodly packet loss rates
by John Curran 23 Oct '96

23 Oct '96

At 12:33 10/23/96, Robert Laughlin wrote: >As one who is living this senerio on a daily basis, I can tell you it's >frustrating and upsetting. We have gone so far as to test the legality >of what is happening (there *must* be someone we can sue <grin>). Public >peering works well these days as the large networks move their traffic >off the NAPs, freeing up bandwidth for the mid to smaller networks. > >The model that makes sense to me, is for the largest networks to exchange >traffic through private interconnects, and for them to treat the >aggregated NAP traffic as another large ISP. The NAP is then used for the >2nd tier and smaller providers to exchange traffic with each other, as >well as a collection point to gather up traffic for the large networks. Interesting model: presuming that the "aggregating exchange point" had a sizable backbone and could engage in shortest exit routing, there's no reason why it shouldn't work. Of course, a _single_ exchange point can't meaningfully provide any benefit of shortest exit routing, and would probably be treated as an extremely large volume customer. /John

1 0

Re: Ungodly packet loss rates
by Curtis Villamizar 23 Oct '96

23 Oct '96

Independent of the fact that the traceroute shows that packets are not taking the most direct route (both Best and BBN are at Mae-West), your complaint about high loss is valid. This is an industry problem, but the problem is not as universal as you may think. Merit has been measuring loss NAP to NAP. We've been looking at this data and trying to provide useful summaries. Its kind of drafty since Merit's data collection is not so good, but you can get a vague idea of the conditions from the page: (This page is temporary - don't point to it.) http://www.brookfield.ans.net/ans/netnow/netnow.html We're not sure of the accuracy of this data or the statistical validity. In particular 1) Merit's method of storage doesn't provide markers between data sets, 2) cummalitive probabilities of 20 packet samples are fairly worthless for looking at the probability of losses below 5%. (The perl programs are all in the same directory so if you see any bugs please tell me. :) \begin{aside} Probability of 1% to 4% loss could be estimated by assuming data sets are uncorrelated. For example, the chance of 2.5% loss or more P(2.5) is (1 - ((1 - P(5))**2)) * (1 - P(10)), or in english the estimate probability that one of two samples is zero times the probability that the other point is no worse than 5%. Or something like that. Apply Bayes theorem of joint probabilities. But is the data uncorrelated? \end{aside} Curtis ps- wrt what's being done about it, this is an attempt to provide insights into loss rates as per your point #2 but more for the purpose of meaningful measure than any kind of guarantee. 2. Develop meaningful quality-of-service standards that can be used to guarantee reasonable performance in terms of end-to-end drop rates, delays, and downtime. Though our loss is among the lowest, we're adding circuits to bring it back so it is zero most of the time, where it used to be and where it belongs. I guess this is the reason for the "state of the Internet" segment of NANOG tommorrow. Other providers can speak for themselves. pps- Its been a long enough thread already. Sorry to add to the thread. :( ------- Forwarded Message Received: from interlock.ans.net (interlock.ans.net [147.225.5.5]) by brookfield.ans.net (8.7.3/8.7.3) with SMTP id NAA05470 for <curtis(a)brookfield.ans.net>; Mon, 21 Oct 1996 13:59:17 -0400 (EDT) Received: by interlock.ans.net id AA05244 (InterLock SMTP Gateway 3.0 for regional-techsers(a)ans.net) Mon, 21 Oct 1996 13:59:45 -0400 Received: by interlock.ans.net (Internal Mail Agent-5); Mon, 21 Oct 1996 13:59:45 -0400 Received: by interlock.ans.net (Internal Mail Agent-4); Mon, 21 Oct 1996 13:59:45 -0400 Received: by interlock.ans.net (Internal Mail Agent-3); Mon, 21 Oct 1996 13:59:45 -0400 Received: by interlock.ans.net (Internal Mail Agent-2); Mon, 21 Oct 1996 13:59:45 -0400 Received: by interlock.ans.net (Internal Mail Agent-1); Mon, 21 Oct 1996 13:59:45 -0400 To: noc(a)tlg.net, help(a)uunet.uu.net, nanog(a)merit.net, ops(a)bbnplanet.com Cc: barb(a)velvet.com, ianp(a)darktower.demon.co.uk From: jbash(a)velvet.com Subject: Ungodly packet loss rates Date: Mon, 21 Oct 1996 10:42:00 -0700 Message-Id: <96Oct21.104256-0700pdt.18972-3+3(a)blue.velvet.com> Sender: owner-nanog(a)merit.edu [Resent... I stupidly used the wrong address for the NANOG list] This is being sent to the "help-line" addresses of several Internet providers because they're not providing what I consider appropriate service. It's being sent to the NANOG mailing list because it represents what I believe to be an industry-wide problem. I'm just a lowly end user, and perhaps I shouldn't intrude into the councils of the Wise and the Great, but this is just a bit ridiculous. Attached is a traceroute from my home machine to the system I'm trying to work on over a TELNET session. It looks like there's about a 40-percent overall round-trip loss rate, most or all of it apparently introduced in the Alternet and BBN Planet backbones. This is not a transient condition; it's been going on for at least several days, and similar things happen all the time. I think we can all agree that a 40-percent loss rate isn't an acceptable level of service in an IP network. It's certainly making it annoying and frustrating for me to try to work. It's also driving up the load on the network by provoking retransmissions. A corporate internal network running at that loss rate would probably be considered to be in collapse. I pay TLGnet (now Best) an agreed-upon amount of money every month, nominally in exchange for a reasonable level of Internet service. I think that part of TLGnet's obligation under that arrangement is to contract for reliable backbone service. Likewise, the other end of the path (Cisco systems, for whom I am emphatically not, *not, *NOT* speaking here) pays BBN what I suspect to be a very large amount of money indeed for DS3 service, presumably in the expectation that most of the packets that go into the DS3 will come out of the network somewhere. Alternet presumably has agreements with both TLGnet and BBN. That puts everybody on the hook. I fully understand that it's difficult to provide reliable service in an exponentially-growing network. I'm aware that everybody's already using the fastest lines they can get, and connecting the fastest routers to them. I know that links are being added. I appreciate that both lines and equipment are very expensive, and that adding lines serves to complicate an already amazingly complex router configuration situation. I understand that cash-flow issues (as well as convincing-the-bean-counters issues) are involved. I sympathize... ... but the fact remains that I'm not getting the level of service I think I'm entitled to, nor are other end users. Not only that, but if the level of service gets any lower, the Net will become so painful to use that I'll start wondering why I bother. While reducing my Net use might be good for my mental health, I don't think anybody wants to see users abandoning the Net because of poor service. So, what's to be done about it? Assuming that all technical means are being pursued, and from what I've seen on various mailing lists I believe they probably are, the only thing left is a management fix. May I make the probably-sacreligous suggestion that the industry as a whole, and the providers I've mentioned in particular, show greater concern for the quality of service provided, and specifically-- 1. Stop taking on new customers (or other traffic sources) until existing customers can be provided with an appropriate level of service. 2. Develop meaningful quality-of-service standards that can be used to guarantee reasonable performance in terms of end-to-end drop rates, delays, and downtime. 3. Reexamine both pricing levels and the Internet pricing model, to make sure that there's enough money available to fund a usable level of service. Yes, this means giving up some business. That's one of the costs of honoring your agreements... and of not alienating an entire generation of customers. Thank you for your attention. Although I usually scan at least the subject lines of messages sent to the NANOG list, I'm temporarily without access to the news server on which I ordinarily read the list. For the next few days, I won't be able to answer replies not sent to me directly. -- J. Bashinski blue% traceroute -a -q 25 -Q champagne.cisco.com traceroute to checkpoint-sj.cisco.com (171.69.10.37), 30 hops max, 40 byte packets 1 tongue.velvet.com (206.14.77.65) (2.8 ms/3.6 ms(+-0.9 ms)/15.8 ms) 25/25 (100.00%) 2 tlg-cust-link.tlg.net (140.174.151.93) (39.0 ms/47.5 ms(+-10.3 ms)/134.8 ms) 25/25 (100.00%) 3 mae-west.tlg.net (198.32.136.22) (40.7 ms/45.6 ms(+-9.2 ms)/57.0 ms) 25/25 (100.00%) 4 905.Hssi3-0.GW1.SCL1.ALTER.NET (137.39.133.89) (43.9 ms/49.1 ms(+-9.9 ms)/60.8 ms) 25/25 (100.00%) 5 Fddi0-0.CR1.SCL1.Alter.Net (137.39.19.5) (43.5 ms/48.8 ms(+-9.8 ms)/57.0 ms) 25/25 (100.00%) 6 Hssi3-0.San-Jose3.CA.Alter.Net (137.39.100.1) * * (45.3 ms/52.5 ms(+-11.1 ms)/77.6 ms) 23/25 (92.00%) 7 Fddi0-0.San-Jose6.CA.Alter.Net (137.39.27.12) * (44.4 ms/49.6 ms(+-10.2 ms)/60.8 ms) 24/25 (96.00%) 8 Hssi1-0.Palo-Alto2.CA.ALTER.NET (137.39.101.162) (46.7 ms/52.6 ms(+-10.5 ms)/61.6 ms) 25/25 (100.00%) 9 Fddi1-0.Palo-Alto3.CA.Alter.Net (137.39.47.7) * * * * * (57.9 ms/82.0 ms(+-21.6 ms)/248.5 ms) 20/25 (80.00%) 10 decwrl.bbnplanet.net (198.32.176.5) * * * * * * (50.4 ms/60.8 ms(+-14.0 ms)/68.9 ms) 19/25 (76.00%) 11 paloalto-br1.bbnplanet.net (4.0.1.57) * * * * * * (52.7 ms/63.6 ms(+-14.7 ms)/82.3 ms) 19/25 (76.00%) 12 paloalto-cisco.bbnplanet.net (131.119.0.196) * * * * * * * (57.0 ms/66.2 ms(+-15.7 ms)/80.7 ms) 18/25 (72.00%) 13 * 131.119.26.10 (131.119.26.10) * * * * * (54.8 ms/65.6 ms(+-15.1 ms)/75.6 ms) 19/25 (76.00%) 14 sj-wall-2.cisco.com (192.31.7.34) * * * * * (48.2 ms/77.2 ms(+-18.4 ms)/151.9 ms) 20/25 (80.00%) 15 * * * * * sj-eng-corp2.cisco.com (198.92.1.130) * * * * * (58.5 ms/70.3 ms(+-18.2 ms)/81.2 ms) 15/25 (60.00%) 16 * eng-atm-gw2.cisco.com (171.69.4.129) * * * * * * * * * (61.9 ms/67.1 ms(+-17.4 ms)/78.7 ms) 15/25 (60.00%) 17 sj-eng-corp1.cisco.com (171.69.5.10) * * * * * * * (52.2 ms/64.9 ms(+-15.4 ms)/81.8 ms) 18/25 (72.00%) 18 checkpoint-sj.cisco.com (171.69.10.37) * * * * * * * * * * * * * * * * (58.1 ms/78.6 ms(+-30.0 ms)/202.2 ms) 9/25 (36.00%) ------- End of Forwarded Message

1 0

Re: Ungodly packet loss rates
by John Curran 23 Oct '96

23 Oct '96

At 15:39 10/22/96, Kent W. England wrote: >... >But there isn't any gee-whiz technology that you can do at a >private interconnect that you can't do at a NAP/MAE. Open NAPs >aren't bad engineering. While there is no difference from a technology perspective, there's also no benefit to be gained by interconnecting large networks at a public (as opposed to private) interconnects. One can certainly make the argument that running large traffic flows through shared interconnects is bad engineering if a private interconnects for such traffic are available. /John

9 8

a multihomed net
by Zachary DeAquila 23 Oct '96

23 Oct '96

I was just thinking (dangerous, I know) and wondering how much extra paths to a known prefix affected the size of the routing tables under BGP. More specifically, under Cisco's implementation of BGP. The trend seems to be toward everyone multihoming for 1) stability/fault tolerance (yes, I know it's only really fault tolerant if you have separate paths and etc, but thats not the point - the trend is) and 2) shorter paths (hop counts) to disparate net.places. The hierarchial model of the net with a few big providers at the top with smaller and smaller providers underneath until you get to the the base customer is just not where we're headed (IMHO). I think we're headed more toward a fully connected graph of AS's (though we may never get there, of course - probably an asymptotic approach). So my question is, how will current routers (and routing technology) handle this? well? not well? If all the ASs in existance were to suddenly peer with each other, what would happen? would BGP explode? Massive CPU crunch? What effect would route flaps then have? Stumbling blindly through the fog, --Zachary

3 2

RE: You are right [was Re: Ungodly packet loss rates]
by Vadim Antonov 23 Oct '96

23 Oct '96

That would be a lot of fun. I always had nice laughs looking at measurement techniques so prevalent in the respective publications. --vadim From: Peter Ford <peterf(a)microsoft.com> Another pragmatic solution is to call the editors of comm week, network world, data communications and suggest that they might get a lot of mileage writing a story comparing and contrasting the performance of ISPs. They do this for routers, bridges, FR services so they can probably find a respectable consulting/measurement group to collect information. Some of this could be live empirical data and some of this could be survey. cheers, peter

3 3

Re: Who has implemented source-IP-address filtering?
by DAVE NORDLUND 23 Oct '96

23 Oct '96

> Date: Tue, 22 Oct 1996 09:48:12 -0400 (EDT) > From: Alexis Rosen <alexis(a)panix.com> > Subject: Who has implemented source-IP-address filtering? > To: nanog(a)merit.edu > No doubt this will come up in Ann Arbor, and I'm about to put up a web page > about the SYN attack and the aftermath, so I'd like anyone who is now doing > filtering of customer packets for spoofed IPs to send me a note. If you > have a plan in the works to do so but haven't yet, I'd like to hear about > that too. > > So far I know that these companies are filtering: > Panix > Net Access > Compuserve > Erol's Kansas Research and Education Network (KANREN) is starting. However, we have some multihome BGP4 things to do first. > > Someone from ANS (not Curtis) told me that he thought ANS was doing it, but > he wasn't sure and would get back to me. > > I've heard back from a few more significant players, mostly very encouraging > stuff, but I am respecting people's requests not to announce anything > publically yet. > > Send me a note if you're doing filtering. If you've done so sometime in the > last six weeks and I've forgotten you, I apologize, but things here have > been pretty insane... > > /a > > --- > Alexis Rosen Owner/Sysadmin, > PANIX Public Access Unix & Internet, NYC. > alexis(a)panix.com > Dave Nordlund d-nordlund(a)ukans.edu University of Kansas 913/864-0450 Computing Services FAX 913/864-0485 Lawrence, KS 66045 KANREN

1 0

Re; Windows/NT and inverse subnetting
by Hank Nussbacher 23 Oct '96

23 Oct '96

FYI. Hank -------------forwarded message------------ >I have to apologize for the lack of clarity since I was getting the >reports from customers. The problem is NOT in setting a prefix mask. >The problem is with inverse for subnets. From the install notes of >Windows/NT (DNS and Microsoft Windows NT 4.0) dated May 12, 1996 (page >28), under the section 'Configuring Reverse Lookup' the doc says, and >I quote: > >??? I haven't a clue. I think there is a bug??? > >end quote. First of all, from the date this would have been the NT4 *beta2* release notes, not the final release. Please ask your customer what the final release notes say -- I assume they are cleaned up somewhat. Secondly, having looked into this a bit what I've found is that there is a limitation in the DNS Administrator -- the GUI tool that front-ends our DNS server -- where you can't enter a value in reverse DNS corresponding to the smaller subnet. However, we read BIND-compatible files and if you enter the right values in the ASCII file then it works fine (according to the developer who wrote it, anyway :-). So you're really no worse off with NT than with UNIX, you only loose the ability to use the nice GUI admin if you need this feature. We'll fix the admin UI in the next NT release. Thanks. Mark Ryland Developer Relations Group Microsoft Corporation markry(a)microsoft.com

1 0

Re: Ungodly packet loss rates
by John Curran 23 Oct '96

23 Oct '96

At 20:38 10/22/96, Gordon Cook wrote: >John, you say there is no benefit to be gained by large providers using >anything other than private inter-connects. may i refine that question >slightly and ask: benefit **to whom?** to the large provider? If the >question of benefit is limited in this manner, i'd guess that your >statement is quite correct. Gordon, It's not a question of large or small; it's a question of traffic volume. Creating multi-provider interconnects makes sense, right up until the aggregation pushes the technology into a higher-cost or higher-risk category. It's hard enough getting stable high-speed equipment; intentionally bringing multiple high-speed traffic flows together when you can easily do a mesh of private interconnects makes little sense. >When you get big enough, you to can connect to the internet at the very >apex as demarcated by major naps in the us and soon in asia and europe. > >*BUT* here is my question. Don't private interconnects essentially >provide a new apex for the internet? One that pushes interconnects at the >major exchange points down a level. >... >now you may say that from a competitive point of view this makes no >difference. perhaps. But what if the big four no longer see the need to >upgrade their bandwidth INTO and OUT OF exchange points? For relatively small traffic volumes, the shared interconnects work fine. Upgrading the bandwidth INTO and OUT of these points is definitely happening, but there's often a higher payoff in taking a major flow off the public interconnects into a private interconnect. Providers do have good reason to upgrade their public interconnects but principally after this phase of major traffic migration to private interconnects has been completed. /John

1 0

Re: Cisco Memory.
by Greg Ketell 23 Oct '96

23 Oct '96

Actually it is not on CCO due to potential legal issues (we list unapproved as well as approved and that made our lawyers squeamish). However, you can call the TAC and give them your info (router model, existing RAM) and they can then look up the info on the internal TAC webserver and give you the appropriate authorized list. Before I was made painfully aware of the legal issues I posted the list to inet-access if anyone wants to dig through the archives (it was only about 2 months ago). GK At 07:22 AM 10/22/96 -0400, you wrote: >At 09:17 PM 10/21/96 -0700, Craig A. Huegen wrote: > >> >>There is a list of third-party RAM that Cisco approves, and this list is >>available on CCO *somewhere*. I don't remember where, though; Paul >>Ferguson is really good at remembering where this stuff is, though. >> > >I poked around for a few minutes without finding the 'approved alternate >vendor list', but I recall that its there *somewhere*. > >- paul > > >

1 0

Re: AADS outage
by Jeffrey S. Curtis 23 Oct '96

23 Oct '96

Joseph Klein writes: }The AADS outage started (for some) about 23:30 GMT and ended about }02:45 GMT. For us, it started at 2140Z, and as you indicate, everything came back up around 0245Z. I still have one BGP session down, though... ... and as I typed that, I got a call from AADS. Everything looked clean through the NAP, so odds are that this last remaining problem is on the other provider's end. } Did the outage at AADS have a major impact on the other }meet points? I know we dumped a bunch of traffic on MAE-East }and MAE-West. The graphs from the AADS Web page (http://nap.aads.net/~nap-stat) show that the switch tends to average about 100Mbps aggregate load over hourly sampling intervals, if that gives you any insight. Jeff -- Jeffrey S. Curtis | Internetwork Manager Argonne National Laboratory | Email: curtis(a)anl.gov 9700 South Cass Avenue, ECT-221 | Voice: 630/252-1789 Argonne, IL 60439 | Fax: 630/252-9689

1 0