- Test - lists.nanog.org

Re: MAE-East
by William Allen Simpson 24 Oct '95

24 Oct '95

Sarcasm warning.... > From: Sean Doran <smd(a)icp.net> > A couple quick points: the place to go to for this kind > of thing should be your direct service provider, who hopefully > has some means of communicating with upstream and peer providers > when things are going wrong somehow, or at least might be > able to give you some additional information. > It should not surprise you that I am no stranger to the folks at my "direct service provider", on either the local or national sides.... > NANOG and other lists aren't appropriate places to discuss > specific problems. OTOH, general problems, such as how big > the Internet is and how saturated well-known-things seem to > be and what might be done about it, could be OK here. > IMnsHO, the _only_ way to discuss engineering is with specific problems! NANOG is damn well the appropriate place to discuss _this_ specific problem! The problem is trying to cram the entire world's traffic through a few NAPs, instead of many MIXs. The problem is the outright failure of the design of the first NAPs, particularly the MAE-East "ethernet" and the "ATM" NAPs. The problem is scaling, and the _cheapness_ of the commercial providers in failing to provide widespread interconnections with each other. > I should also note that Sprint and MCI are in the final > phases of negotiating direct point-to-point peerings which > will pop up in several areas, and improve connectivity > between SprintLink and InternetMCI. You wouldn't be far off > in expecting this to become something of a trend, > particularly among a small number of the large heavy-traffic > providers, for whom the NAP/MAE/FIX concepts and models > don't seem to be scaling well in practice. > You mean, we might end up with a Network instead of a Treework!?!? Halleluia! Bill.Simpson(a)um.cc.umich.edu Key fingerprint = 2E 07 23 03 C5 62 70 D3 59 B1 4F 5E 1D C2 C1 A2

1 0

mae-east dampening/flapping view
by Ed Kern 23 Oct '95

23 Oct '95

Dont know if this would be of intrest to a soul (outside DIGEX), but if you ever have a need.. http://nitrous.digex.net/mae-lg.html Will let you view/use trace dampened-paths flap-statistics bgp summary version ping Comments to me directly please. Ed PS all hits will be logged so I can lock out the abuses, but all logs will remain private.

1 0

Re: MAE-East
by Brett D. Watson 23 Oct '95

23 Oct '95

That was precisely the point of my last note. I *want* to get involved in finding a reliable way to measure network performance and I *want* to be proactive at it. I absolutely hate telling customers that "pinging a cisco is not a good indicator..." but like you say "there ain't no other way". Just as ping and traceroute are not good indicators of performance, saying that they aren't good indicators is *not* acceptable to customers paying money for your service. So just for the record, I'm not working for a provider and whining that it's not our fault. I want to find a good way to measure peformance on my network and I'd love to be able to publish some results. Guy Almes contacted me about getting involved and that's just what I want to do. -brett > > A couple quick points: the place to go to for this kind > >of thing should be your direct service provider, who hopefully > >has some means of communicating with upstream and peer providers > >when things are going wrong somehow, or at least might be > >able to give you some additional information. > > Ahem, my success rate at that has been less than stellar. If you guys > (you service providers, not you Sean) are so worried about being asked > questions in public (and short of com-priv, I see no more logical place > for this than NANOG if "local" questions don't work], why don't you > work on schemes to pro-actively publish performance data? Otherwise > you *will* get called to the table based on ping and traceroute data, > there just ain't no other way. Either *you* do it, or others will do it > for you, with the tools they find most appropriate, and all your > whining will eventually be discounted. Be pro-active, or give them > better tools. You *asked* to be in the kitchen, *you* better deliver a > working network, including at the Internet-systemic level and not only > in your own service provider centric environment. I suppose that if > you guys can't figure it out yourself, there is always the opportunity > to get it fixed by someone else. Like, ahem, regulation.

1 0

Re: Initial Route Server Stats for MAE-East
by Brett D. Watson 23 Oct '95

23 Oct '95

I've only been to one IETF but wasn't the IPPM (IP porvider metric) group working on just such a problem? Basically attacking how we measure performance on an internet? Did the group come up with anything? I deal with people every day that *insist* that pinging a router (yes, we're talking Cisco's in this case) is "a good enough indication for me" that "your routers are dropping X% of my packets". Sometimes it turns out to be true and I find the problem. Many times, it's just not so. That begs the question: How do I determine whether or not I have packet loss or delay on my network (and be able to prove it beyond a reasonable doubt)? Does FTPing a 50M file across the network tell me? Does a 10,000 packet ping tell me? My point for interjecting here is to ask: Has anyone come up with *any* way to measure network performance (packet loss, throughput, delay) other than ping and traceroute? I understand Mike's comments on defining *what* we really want to measure and I'm all for trying to help define it. I ask these questions because I'd really like to find answers, not to stir the coals even more. -brett > by all means, ping all you like. please do realize that > routers have other things to do besides answer pings when > they're busy. just because a router doesn't respond to > 15% of its pings doesn't mean that it's dropping 15% of > its packets or that there is a 15% loss on a line. router > load is not router load, process switched packets are not > the same as card-level switched or cache switched. > > there aren't any easy answers. perhaps we should all just > implement "ping" servers in pops ( a 386 running > linux or bsd might suffice). > > Jeff Young > young(a)mci.net > > > > > > > > > On Sat, 21 Oct 1995, Mike O'Dell wrote: > > > > > > > remember that using pings to sample connectivity to a very busy > > > > cisco router is not a very reliable probe for several reasons. > > > > Returning pings is a low-priority task in the first place, and > > > > they are rate-limited, so if the processor is busy processing > > > > lots of BGP updates and several folks are fribbling with it > > > > using ping or SNMP, it is less than clear what they will see. > > > > > > > > -mo > > > > > > > Michael A. Nasto quips: > > > > > OK! Agreed. So then, what would you use? > > > > > > > Have you ever been in a classroom and had a student raise his hand, > > answer every question, ask intelligent questions, etc. just to prove > > to the class how smart he or she is. This is the premise of > > the 'Two Mike's Interchange' above. One says, HEY! I know ping packets > > are a lower priority than everything else in a *CISCO* router > > LOOK AT ME (wave wave). Then another kid in the class quips... > > if PING is not what you would use, give us a better utility. > > > > In fact... EVERYONE ( okay 99.73 percent :-) uses PING. After all > > router LOAD is router LOAD.... and if a few ICMP packets can't get > > back in a subjectly reasonable time.. then DUH.... "da network > > is busy......" BGP updates take bandwidth just like any other packet. > > > > Of course the 0.27 percent, zen routing gods of the universe just > > feel the load and the harmonic BGP update patterns and PING between > > the BGP updates.... for a better answer. > > > > Sorry, I could not resist.... and apologize for the satire. PING!! > > PING!! PING!! > > > > Tim > > > > -- > > +--------------------------------------------------------------------------+ > > | Tim Bass | #include<campfire.h> | > > | Principal Network Systems Engineer | for(beer=100;beer>1;beer++){ | > > | The Silk Road Group, Ltd. | take_one_down(); | > > | | pass_it_around(); | > > | http://www.silkroad.com/ | } | > > | | back_to_work(); /*never reached */ | > > +--------------------------------------------------------------------------+ >

4 3

Re: Asymmetric Routing
by Steven J. Richardson 23 Oct '95

23 Oct '95

Sean: This is misleading at best... >From list-admin(a)merit.edu Fri Oct 20 15:19:42 1995 >From: Sean Doran <smd(a)icp.net> >To: joliveto(a)cwi.net, nanog(a)merit.edu >Subject: Re: Asymmetric Routing >Message-Id: <95Oct20.142237-0400_edt.20696+16(a)chops.icp.net> >Date: Fri, 20 Oct 1995 14:22:35 -0400 >Status: R > >| Analysis of some of our routes shows the potential for asymmetric >| routing...that is...the transmit path will be different than the return path >| for packets. > >Asymmetry has been a reality for quite some time now; in fact >it was happening during the NSFNET backbone service days, >since the information in the PRDB was often in a shambles. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Not surprisingly, the PRDB was as good as the information which was submitted by customers of the NSFNET BNS, though there was a lot of checking done to try to proactively catch mistakes. As someone who was working with/supporting the NSFNET configuration process, I do not know of other customers claiming that "the PRDB was often in a shambles" via phone, email, or third-party reports. > Sean. Steve Richardson, Merit

1 0

Re: MAE-East
by Sean Doran 23 Oct '95

23 Oct '95

Bill - The information you supplied is more helpful for figuring out what you're seeing than other things I have seen. Indicating a traceroute and where you see packet loss, and when you see packet loss is a big first step in diagnosing complaints. A couple quick points: the place to go to for this kind of thing should be your direct service provider, who hopefully has some means of communicating with upstream and peer providers when things are going wrong somehow, or at least might be able to give you some additional information. NANOG and other lists aren't appropriate places to discuss specific problems. OTOH, general problems, such as how big the Internet is and how saturated well-known-things seem to be and what might be done about it, could be OK here. Your numbers are quite reasonable ones, and point to heavy congestion in the area of MAE-EAST+. A good part of that is on the DS3 between MCI and their first MAE-EAST+ router. MCI has done something reasonable about that, viz., putting a second router in place, which I am about to set up a peering with. This should divert a large amount of traffic away from the original DS3, which in turn likely should improve performance between you and PSI. I should also note that Sprint and MCI are in the final phases of negotiating direct point-to-point peerings which will pop up in several areas, and improve connectivity between SprintLink and InternetMCI. You wouldn't be far off in expecting this to become something of a trend, particularly among a small number of the large heavy-traffic providers, for whom the NAP/MAE/FIX concepts and models don't seem to be scaling well in practice. Meanwhile, expect congestion, particularly at NAPs/MAEs/etc. :( | If you want a really good a measure, instead of ping try running NTP | over the links in question. It gives a very accurate measurement! | And I've observed 40% loss peaks there too! NTP is one of two really useful network diagnostic tools, and there is another by Matt Mathis that I hesitate to advertise simply because it is somewhat CPU-unfriendly towards busy Cisco routers, while the former is more useful for long-term information gotten as a side-effect. You might want to investigate the IPPM work, btw, wrt network performance. Finally, wrt MAE-EAST, you might want to look at http://www.mfsdatanet.com/MAE/east.stats.html for an idea of the amount of traffic there. Hint: it's *lots*. Sean.

3 2

Re: Initial Route Server Stats for MAE-East
by Guy T Almes 23 Oct '95

23 Oct '95

Randy, I agree. In fact, at the interim BMWG/IPPM meeting, Vern made a presentation on his work, and it was well received. -- Guy At 07:28 AM 10/23/95 PDT, Randy Bush wrote: >hate to sound egghead, but this stuff has been the subject of non-trivial >academic (yet practical) research, and there is a fair literature base. >hard to drag out refs from here, but one interesting ongoing project is >NPD by Vern Paxson, <vern(a)ee.lbl.gov>, which might serve as a starting >point for a lit search. some evenings it is nice to take a cs break from >all the grotty hacking. >

1 0

Re: Initial Route Server Stats for MAE-East
by randy＠psg.com 23 Oct '95

23 Oct '95

hate to sound egghead, but this stuff has been the subject of non-trivial academic (yet practical) research, and there is a fair literature base. hard to drag out refs from here, but one interesting ongoing project is NPD by Vern Paxson, <vern(a)ee.lbl.gov>, which might serve as a starting point for a lit search. some evenings it is nice to take a cs break from all the grotty hacking.

1 0

Re: Initial Route Server Stats for MAE-East
by Gerald Andrew Winters 23 Oct '95

23 Oct '95

Hello, I can't resist jumping into this disscussion. I think its clear that there is much work to be done on router/IP performance. The question then is what can be done in the meantime, until said work is completed? Here is a script I wrote that will work on cisco routers. The script takes advantage of cisco's enterprise specific MIB. There are 3 SNMP variables yielding CPU busy percentage. The first is CPU busy percentage in the last 5 seconds of scheduler time. The other two are 1 and 5 minute exponentially-decayed moving averages. The CPU busy metrics can yield interesting results, for example %'s above 40% indicate a busy router. I was able to generate some interesting graphs awhile back for our cisco routers. For more information, use the oid's in the script as pointers into the cisco mib. And here is the script for anyone who is interested. --Gerald Winters <gerald(a)merit.edu> #!/usr/local/bin/perl @mib_objs = ("1.3.6.1.4.1.9.2.1.56.0", "1.3.6.1.4.1.9.2.1.57.0", "1.3.6.1.4.1.9.2.1.58.0"); for ($i = $num_values = 0; $i < 3; $i++) { foreach (`/afs/umich.edu/user/g/e/gerald/CISCO/snmpget -h 141.211.128.1 @mib_objs[$i]`) { if (/value:/) { $num_values++; @values[$i] = /\w+\W+\w+\W+(\w+)/; } } } if ($num_values == 3) { print "@values[0] @values[1] @values[2] ".`date`; } else { print "0 0 0 ".`date`; }

1 0

Re: Initial Route Server Stats for MAE-East
by Guy T Almes 23 Oct '95

23 Oct '95

I'd like to second Mike's note, particularly the call for participation. Learning how to characterize IP clouds with carefully defined metrics of delay, flow capacity, and packet loss will be real work. Note, for example, the discussion this weekend of how simple pings to a loaded router don't yield a good measure of packet-forwarding loss. The same could be said for trying to measure round-trip delay to a loaded router. If you'd like to join the IPPM email list, send mail to <ippm-request(a)advanced.org>. Notes expressing an interest in working on specific kinds of issues would be particularly appreciated. We'll also be meeting at the upcoming Dallas IETF. -- Guy At 10:06 PM 10/22/95 -0400, Mike O'Dell wrote: > >the IPPM effort within the Benchmark Methodology Working Group is >working on "IP Performance Metrics" which apply to large IP networks >viewed as the "device-under-test" and not "provider metrics" per se. >(I say this as the both the Co-Area Director and the AD responsible for >the working group.) > >understanding the performance of complex networks IS a very much >unsolved problems. people are thrashing around taking whatever >measurements they can think of with hopes of generating SOME kind of >understanding. I don't blame them at all - there isn't much else to do >right this instant. > >all this means, though, is that folks concerned with the problem need >to participate in the BMWG efforts. there is a lot of work to do here >and with some luck, we may do some science to the point of developing >some understanding about how to measure and characterize these beasts >we've built. > >A lot has been done in characterizing performance of network elements, >so now we need to draw a bigger circle around what we're testing >but still apply significant rigor and good experimental methodology >so the data will be meaningful. > > yours for more consistent confusion, > > -mo >

1 0