- Test - lists.nanog.org

[fwd] Rats take down Stanford ...
by Paul Ferguson 23 Oct '96

23 Oct '96

A follow-up thought on redundancy issues. - paul [snip] >Date: Mon, 21 Oct 1996 12:54:05 -0700 (PDT) >From: risks(a)csl.sri.com >Subject: RISKS DIGEST 18.54 [snip] > >Date: Fri, 18 Oct 96 11:03 EST >From: William Hugh Murray <0003158580(a)mcimail.com> >Subject: Re: Rats take down Stanford ... (RISKS-18.53) > >PGN's request for redundancy brings to mind the story of the infrastructure >computer center in Trumbull, Connecticut. It is an old story but bears >repeating. > >Seems that a squirrel got into a transformer and brought down the external >power supply. The UPS kicked in, engine generators came on line, and the >center operated in this mode for about an hour and a half. At the end of >that time the external power was restored. The external power, the UPS, and >the engine generators went inot a deadly embrace. The whole thing came down >and would not come back up. > >I take two lessons from this. First, redundancy adds some complexity and a >lot of redundancy adds a lot of complexity. At some point the redundancy >begins to introduce failure modes and failure events that would not have >exited in its absence. There is an upper bound to such redundancy. > >Second, test redundant systems through to resumption of normal operations. >In this case, the operators had tested to ensure that the redundant systems >would come online in the event of a failure of the primary system. They had >not tested to see what would happen when the primary system was restored to >normal operation. > >Who would have even thought about it? I confess that I would not have. > >William Hugh Murray, New Canaan, Connecticut > [snip]

2 1

Re: Ungodly packet loss rates
by Jonathan Heiliger 23 Oct '96

23 Oct '96

On Wed, 23 Oct 1996, Paul Ferguson wrote: |} One Big Win (tm) with the creation of local interconnects (for |} instance, strategic locations in Asia & South/Central America), is that |} local content providers can find a way to swap bits without transiting |} all the way back to the United States; in my opinion, we need more |} exchanges to decrease the amount of traffic contributing to this |} problem. Unfortunately the larger network providers, which provide access to the bulk of the users haven't quite grasped this model, or are ignoring it. Some have jumped the fence and are working with content companies and/or their providers' to both (a) spread the load out and (b) focus on keeping content local within a region. The Internet is made up of users, the users are there in part, because they believe they can get better, quicker, or easier access to "content," than they could using other methods (e.g. newspaper, TV, etc.) This illustrates the point that if there isn't any content, what is the Internet but a big Email network? -jh-

1 0

Re: Ungodly packet loss rates
by Paul Ferguson 23 Oct '96

23 Oct '96

At 08:38 PM 10/22/96 -0400, Gordon Cook wrote: > >*BUT* here is my question. Don't private interconnects essentially >provide a new apex for the internet? One that pushes interconnects at the >major exchange points down a level. Sprint, MCI, UUNET, and BBN are >clearly the four largest players by market share. All of them have >multiple private interconnects with each other. perhaps ANS does as well. >AGIS certainly does not. So in this sense, the "6" have not really become >15 but rather have become 4 or maybe 5. > This begs the question: What value do 'private' interconnects/exchanges provide? I would submit that the answer isn't readily acknowledgeable, or at least, has not been explored to the extents necessary. One Big Win (tm) with the creation of local interconnects (for instance, strategic locations in Asia & South/Central America), is that local content providers can find a way to swap bits without transiting all the way back to the United States; in my opinion, we need more exchanges to decrease the amount of traffic contributing to this problem. The value of private exchanges/peering between the larger global providers is that they can get traffic off of their backbone before it reaches exchange points which are already glowing in the dark. I don't think this needs much explanation. Part of the problem is that, the more BGP peers, the more the statistical chances for instability. Unfortunately, I don't readily see a solution to this particular issue, if large providers are not economically incentivized to build more, smaller interconnect points which allow tier-2 and tier-3 providers to peer with them. Then again, I don't pretend to understand *all* of the corporate political, technical and economic, issues. ;-) - paul (en route to Ann Arbor)

1 0

Re: a multihomed net
by Ed Kern 23 Oct '96

23 Oct '96

On October 22, you wrote: > Today I multihomed a customer using only two disjoint PA spaces. > > PA1 PA2 > P1---=---Wall---=---P2 > | > ------ RFC1597 > | > Server so is anything in the above picture directly addressable through the wall?...and paul you mean rfc1918 right? ;) > > I did this whole thing with an almost-stock BSD/OS 2.1 system (other than > the "socket" command which is off the net from way back.) > server...?...just do the nat with your cissssccoooo... http://schizo.digex.net Easy to install, and it works....if people will want to use it....thats a different matter... Id like to hear opinions on this at nanog if people have any.. > So don't let's talk any longer about multihoming requiring PI space. Either way...firewall or just with the router..you are correct. Ed

1 0

Re: Ungodly packet loss rates
by John Curran 23 Oct '96

23 Oct '96

At 12:33 10/23/96, Robert Laughlin wrote: >As one who is living this senerio on a daily basis, I can tell you it's >frustrating and upsetting. We have gone so far as to test the legality >of what is happening (there *must* be someone we can sue <grin>). Public >peering works well these days as the large networks move their traffic >off the NAPs, freeing up bandwidth for the mid to smaller networks. > >The model that makes sense to me, is for the largest networks to exchange >traffic through private interconnects, and for them to treat the >aggregated NAP traffic as another large ISP. The NAP is then used for the >2nd tier and smaller providers to exchange traffic with each other, as >well as a collection point to gather up traffic for the large networks. Interesting model: presuming that the "aggregating exchange point" had a sizable backbone and could engage in shortest exit routing, there's no reason why it shouldn't work. Of course, a _single_ exchange point can't meaningfully provide any benefit of shortest exit routing, and would probably be treated as an extremely large volume customer. /John

1 0

Re: Ungodly packet loss rates
by Curtis Villamizar 23 Oct '96

23 Oct '96

Independent of the fact that the traceroute shows that packets are not taking the most direct route (both Best and BBN are at Mae-West), your complaint about high loss is valid. This is an industry problem, but the problem is not as universal as you may think. Merit has been measuring loss NAP to NAP. We've been looking at this data and trying to provide useful summaries. Its kind of drafty since Merit's data collection is not so good, but you can get a vague idea of the conditions from the page: (This page is temporary - don't point to it.) http://www.brookfield.ans.net/ans/netnow/netnow.html We're not sure of the accuracy of this data or the statistical validity. In particular 1) Merit's method of storage doesn't provide markers between data sets, 2) cummalitive probabilities of 20 packet samples are fairly worthless for looking at the probability of losses below 5%. (The perl programs are all in the same directory so if you see any bugs please tell me. :) \begin{aside} Probability of 1% to 4% loss could be estimated by assuming data sets are uncorrelated. For example, the chance of 2.5% loss or more P(2.5) is (1 - ((1 - P(5))**2)) * (1 - P(10)), or in english the estimate probability that one of two samples is zero times the probability that the other point is no worse than 5%. Or something like that. Apply Bayes theorem of joint probabilities. But is the data uncorrelated? \end{aside} Curtis ps- wrt what's being done about it, this is an attempt to provide insights into loss rates as per your point #2 but more for the purpose of meaningful measure than any kind of guarantee. 2. Develop meaningful quality-of-service standards that can be used to guarantee reasonable performance in terms of end-to-end drop rates, delays, and downtime. Though our loss is among the lowest, we're adding circuits to bring it back so it is zero most of the time, where it used to be and where it belongs. I guess this is the reason for the "state of the Internet" segment of NANOG tommorrow. Other providers can speak for themselves. pps- Its been a long enough thread already. Sorry to add to the thread. :( ------- Forwarded Message Received: from interlock.ans.net (interlock.ans.net [147.225.5.5]) by brookfield.ans.net (8.7.3/8.7.3) with SMTP id NAA05470 for <curtis(a)brookfield.ans.net>; Mon, 21 Oct 1996 13:59:17 -0400 (EDT) Received: by interlock.ans.net id AA05244 (InterLock SMTP Gateway 3.0 for regional-techsers(a)ans.net) Mon, 21 Oct 1996 13:59:45 -0400 Received: by interlock.ans.net (Internal Mail Agent-5); Mon, 21 Oct 1996 13:59:45 -0400 Received: by interlock.ans.net (Internal Mail Agent-4); Mon, 21 Oct 1996 13:59:45 -0400 Received: by interlock.ans.net (Internal Mail Agent-3); Mon, 21 Oct 1996 13:59:45 -0400 Received: by interlock.ans.net (Internal Mail Agent-2); Mon, 21 Oct 1996 13:59:45 -0400 Received: by interlock.ans.net (Internal Mail Agent-1); Mon, 21 Oct 1996 13:59:45 -0400 To: noc(a)tlg.net, help(a)uunet.uu.net, nanog(a)merit.net, ops(a)bbnplanet.com Cc: barb(a)velvet.com, ianp(a)darktower.demon.co.uk From: jbash(a)velvet.com Subject: Ungodly packet loss rates Date: Mon, 21 Oct 1996 10:42:00 -0700 Message-Id: <96Oct21.104256-0700pdt.18972-3+3(a)blue.velvet.com> Sender: owner-nanog(a)merit.edu [Resent... I stupidly used the wrong address for the NANOG list] This is being sent to the "help-line" addresses of several Internet providers because they're not providing what I consider appropriate service. It's being sent to the NANOG mailing list because it represents what I believe to be an industry-wide problem. I'm just a lowly end user, and perhaps I shouldn't intrude into the councils of the Wise and the Great, but this is just a bit ridiculous. Attached is a traceroute from my home machine to the system I'm trying to work on over a TELNET session. It looks like there's about a 40-percent overall round-trip loss rate, most or all of it apparently introduced in the Alternet and BBN Planet backbones. This is not a transient condition; it's been going on for at least several days, and similar things happen all the time. I think we can all agree that a 40-percent loss rate isn't an acceptable level of service in an IP network. It's certainly making it annoying and frustrating for me to try to work. It's also driving up the load on the network by provoking retransmissions. A corporate internal network running at that loss rate would probably be considered to be in collapse. I pay TLGnet (now Best) an agreed-upon amount of money every month, nominally in exchange for a reasonable level of Internet service. I think that part of TLGnet's obligation under that arrangement is to contract for reliable backbone service. Likewise, the other end of the path (Cisco systems, for whom I am emphatically not, *not, *NOT* speaking here) pays BBN what I suspect to be a very large amount of money indeed for DS3 service, presumably in the expectation that most of the packets that go into the DS3 will come out of the network somewhere. Alternet presumably has agreements with both TLGnet and BBN. That puts everybody on the hook. I fully understand that it's difficult to provide reliable service in an exponentially-growing network. I'm aware that everybody's already using the fastest lines they can get, and connecting the fastest routers to them. I know that links are being added. I appreciate that both lines and equipment are very expensive, and that adding lines serves to complicate an already amazingly complex router configuration situation. I understand that cash-flow issues (as well as convincing-the-bean-counters issues) are involved. I sympathize... ... but the fact remains that I'm not getting the level of service I think I'm entitled to, nor are other end users. Not only that, but if the level of service gets any lower, the Net will become so painful to use that I'll start wondering why I bother. While reducing my Net use might be good for my mental health, I don't think anybody wants to see users abandoning the Net because of poor service. So, what's to be done about it? Assuming that all technical means are being pursued, and from what I've seen on various mailing lists I believe they probably are, the only thing left is a management fix. May I make the probably-sacreligous suggestion that the industry as a whole, and the providers I've mentioned in particular, show greater concern for the quality of service provided, and specifically-- 1. Stop taking on new customers (or other traffic sources) until existing customers can be provided with an appropriate level of service. 2. Develop meaningful quality-of-service standards that can be used to guarantee reasonable performance in terms of end-to-end drop rates, delays, and downtime. 3. Reexamine both pricing levels and the Internet pricing model, to make sure that there's enough money available to fund a usable level of service. Yes, this means giving up some business. That's one of the costs of honoring your agreements... and of not alienating an entire generation of customers. Thank you for your attention. Although I usually scan at least the subject lines of messages sent to the NANOG list, I'm temporarily without access to the news server on which I ordinarily read the list. For the next few days, I won't be able to answer replies not sent to me directly. -- J. Bashinski blue% traceroute -a -q 25 -Q champagne.cisco.com traceroute to checkpoint-sj.cisco.com (171.69.10.37), 30 hops max, 40 byte packets 1 tongue.velvet.com (206.14.77.65) (2.8 ms/3.6 ms(+-0.9 ms)/15.8 ms) 25/25 (100.00%) 2 tlg-cust-link.tlg.net (140.174.151.93) (39.0 ms/47.5 ms(+-10.3 ms)/134.8 ms) 25/25 (100.00%) 3 mae-west.tlg.net (198.32.136.22) (40.7 ms/45.6 ms(+-9.2 ms)/57.0 ms) 25/25 (100.00%) 4 905.Hssi3-0.GW1.SCL1.ALTER.NET (137.39.133.89) (43.9 ms/49.1 ms(+-9.9 ms)/60.8 ms) 25/25 (100.00%) 5 Fddi0-0.CR1.SCL1.Alter.Net (137.39.19.5) (43.5 ms/48.8 ms(+-9.8 ms)/57.0 ms) 25/25 (100.00%) 6 Hssi3-0.San-Jose3.CA.Alter.Net (137.39.100.1) * * (45.3 ms/52.5 ms(+-11.1 ms)/77.6 ms) 23/25 (92.00%) 7 Fddi0-0.San-Jose6.CA.Alter.Net (137.39.27.12) * (44.4 ms/49.6 ms(+-10.2 ms)/60.8 ms) 24/25 (96.00%) 8 Hssi1-0.Palo-Alto2.CA.ALTER.NET (137.39.101.162) (46.7 ms/52.6 ms(+-10.5 ms)/61.6 ms) 25/25 (100.00%) 9 Fddi1-0.Palo-Alto3.CA.Alter.Net (137.39.47.7) * * * * * (57.9 ms/82.0 ms(+-21.6 ms)/248.5 ms) 20/25 (80.00%) 10 decwrl.bbnplanet.net (198.32.176.5) * * * * * * (50.4 ms/60.8 ms(+-14.0 ms)/68.9 ms) 19/25 (76.00%) 11 paloalto-br1.bbnplanet.net (4.0.1.57) * * * * * * (52.7 ms/63.6 ms(+-14.7 ms)/82.3 ms) 19/25 (76.00%) 12 paloalto-cisco.bbnplanet.net (131.119.0.196) * * * * * * * (57.0 ms/66.2 ms(+-15.7 ms)/80.7 ms) 18/25 (72.00%) 13 * 131.119.26.10 (131.119.26.10) * * * * * (54.8 ms/65.6 ms(+-15.1 ms)/75.6 ms) 19/25 (76.00%) 14 sj-wall-2.cisco.com (192.31.7.34) * * * * * (48.2 ms/77.2 ms(+-18.4 ms)/151.9 ms) 20/25 (80.00%) 15 * * * * * sj-eng-corp2.cisco.com (198.92.1.130) * * * * * (58.5 ms/70.3 ms(+-18.2 ms)/81.2 ms) 15/25 (60.00%) 16 * eng-atm-gw2.cisco.com (171.69.4.129) * * * * * * * * * (61.9 ms/67.1 ms(+-17.4 ms)/78.7 ms) 15/25 (60.00%) 17 sj-eng-corp1.cisco.com (171.69.5.10) * * * * * * * (52.2 ms/64.9 ms(+-15.4 ms)/81.8 ms) 18/25 (72.00%) 18 checkpoint-sj.cisco.com (171.69.10.37) * * * * * * * * * * * * * * * * (58.1 ms/78.6 ms(+-30.0 ms)/202.2 ms) 9/25 (36.00%) ------- End of Forwarded Message

1 0

Re: Ungodly packet loss rates
by John Curran 23 Oct '96

23 Oct '96

At 15:39 10/22/96, Kent W. England wrote: >... >But there isn't any gee-whiz technology that you can do at a >private interconnect that you can't do at a NAP/MAE. Open NAPs >aren't bad engineering. While there is no difference from a technology perspective, there's also no benefit to be gained by interconnecting large networks at a public (as opposed to private) interconnects. One can certainly make the argument that running large traffic flows through shared interconnects is bad engineering if a private interconnects for such traffic are available. /John

9 8

a multihomed net
by Zachary DeAquila 23 Oct '96

23 Oct '96

I was just thinking (dangerous, I know) and wondering how much extra paths to a known prefix affected the size of the routing tables under BGP. More specifically, under Cisco's implementation of BGP. The trend seems to be toward everyone multihoming for 1) stability/fault tolerance (yes, I know it's only really fault tolerant if you have separate paths and etc, but thats not the point - the trend is) and 2) shorter paths (hop counts) to disparate net.places. The hierarchial model of the net with a few big providers at the top with smaller and smaller providers underneath until you get to the the base customer is just not where we're headed (IMHO). I think we're headed more toward a fully connected graph of AS's (though we may never get there, of course - probably an asymptotic approach). So my question is, how will current routers (and routing technology) handle this? well? not well? If all the ASs in existance were to suddenly peer with each other, what would happen? would BGP explode? Massive CPU crunch? What effect would route flaps then have? Stumbling blindly through the fog, --Zachary

3 2

RE: You are right [was Re: Ungodly packet loss rates]
by Vadim Antonov 23 Oct '96

23 Oct '96

That would be a lot of fun. I always had nice laughs looking at measurement techniques so prevalent in the respective publications. --vadim From: Peter Ford <peterf(a)microsoft.com> Another pragmatic solution is to call the editors of comm week, network world, data communications and suggest that they might get a lot of mileage writing a story comparing and contrasting the performance of ISPs. They do this for routers, bridges, FR services so they can probably find a respectable consulting/measurement group to collect information. Some of this could be live empirical data and some of this could be survey. cheers, peter

3 3

Re: Who has implemented source-IP-address filtering?
by DAVE NORDLUND 23 Oct '96

23 Oct '96

> Date: Tue, 22 Oct 1996 09:48:12 -0400 (EDT) > From: Alexis Rosen <alexis(a)panix.com> > Subject: Who has implemented source-IP-address filtering? > To: nanog(a)merit.edu > No doubt this will come up in Ann Arbor, and I'm about to put up a web page > about the SYN attack and the aftermath, so I'd like anyone who is now doing > filtering of customer packets for spoofed IPs to send me a note. If you > have a plan in the works to do so but haven't yet, I'd like to hear about > that too. > > So far I know that these companies are filtering: > Panix > Net Access > Compuserve > Erol's Kansas Research and Education Network (KANREN) is starting. However, we have some multihome BGP4 things to do first. > > Someone from ANS (not Curtis) told me that he thought ANS was doing it, but > he wasn't sure and would get back to me. > > I've heard back from a few more significant players, mostly very encouraging > stuff, but I am respecting people's requests not to announce anything > publically yet. > > Send me a note if you're doing filtering. If you've done so sometime in the > last six weeks and I've forgotten you, I apologize, but things here have > been pretty insane... > > /a > > --- > Alexis Rosen Owner/Sysadmin, > PANIX Public Access Unix & Internet, NYC. > alexis(a)panix.com > Dave Nordlund d-nordlund(a)ukans.edu University of Kansas 913/864-0450 Computing Services FAX 913/864-0485 Lawrence, KS 66045 KANREN

1 0