- Test - lists.nanog.org

Re: Keynote/Boardwatch Internet Backbone Index A better test!!!
by Hank Nussbacher 28 Jun '97

28 Jun '97

At 04:43 PM 6/27/97 -0600, Jack Rickard wrote: >Randy: > >I'm with you to a point. But NONE of the measurements were taken from >Jack's house. None were taken from Boardwatch's office. And in fact, the >measurements were taken from 27 different geographic locations. I think >that is a somewhat critical element here. We failed to come up with a >methodology that could be performed from "Jack's House" without primarily >measuring "Jack's connectivity" more than anything else. Obviously, that >is what I would have liked. Never could come up with an idea on how to do >that effectively. I'm curious - to which ISPs were these 27 sites connected? Can we get a list? Thanks, Hank

1 0

Re: Internet Backbone Index
by Daniel W McRobb 28 Jun '97

28 Jun '97

> Jack, this is a really large list, with a lot of backbiters and > pontificators. My usual policy is to collect responses for a day, and > then reply to only one or two, rather than trying to answer each message > individually. That would have helped avoid 100 messages on this topic. > > > From: Joe Shaw <jshaw(a)insync.net> > > There are enough marketing reports and half witted articles and reports > > out their to confuse the average consumer. I'd think that you'd be > > ashamed to be part of them, instead of providing a service that's closer > > to providing what you intentionally planned to do. > > > Actually, I'm a bit ashamed of the NANOG response. > > First of all, however you all might dislike it, our end users' > perception of performance _is_ based on web download speeds these days. > And the users don't distinguish network load as opposed to server load. Absolutely. However, even just from this perspective, I think the sample set of servers was: 1) too small 2) not what the typical end user sees 3) partly naive 1 is mostly obvious, or at least to me. Even as a casual surfer, I hit 100 or more sites a day (at least that's what my cache says), usually searching for something (I'm still looking for the perfect search site, but alas have had little luck on that front ;-)). 2... I don't think many users surf network provider home pages. Reachability is probably much more important to providers on that front, and servers located near inter-provider connectivity epicenters for high reachability from the rest of the world sometimes suffer on throughput to the end user. That's a reasonable tradeoff for a WWW site that's funded out of your own pocket and sees very low hit rates, uses no push technology or any high traffic generators, etc. 3... one of the assumptions in the summary is highly questionable: 'Keynote decided to measure a backbone provider's own public web server on the assumption that the provider would locate its own server in the best-performing hosting location for that provider.' I think most providers' home pages are not viewed as being in need of high performance. This is not just in terms of location; in general, I think they're given little funding outside of: - content - reliability The hit rates are low; they don't draw much attention (except 'til maybe now, if people really do go about making them better; I'd be a bit surprised at that, since it doesn't really serve the end users, who almost never look at these sites). > Maybe you _think_ the server choices were poor, but as far as I can > tell, you don't have enough data to determine that. As to criticizing > the methodology, why haven't you (collectively) proposed a better > methodology for measuring web access? I would much rather see something closer to what an end user actually sees. For example, take nlanr's (or someone else's) squid cache hit data and use that to drive a weighted measurement of the spots end users are actually surfing. Alone, it doesn't address the issues of where the problems are, but it at least gets much closer to the end user experience. I don't know anyone in my immediate vicinity who has looked at more than 3 (if any) of the WWW pages in the study; they're just not popular sites, even for those of us who work in the business (who presumably have more interest these sites for seeing what the competition is doing ;-)). Of course, you might want to trim out the obvious top sites like playboy.com and penthouse.com, unless you want a nice picture of our net community's demographics by byte (from what I've seen, it looks to be largely teenage males, but I suspect there's a bias by type of content ;-)). I'd still be very hesitant to make any judgement about a provider's performance based on such data, w/o an intimate knowledge of the topology and where things are bad. That's reasonably difficult to do without a reasonably-sized matrix of well-selected measurement points. As of yet, we don't know how well the keynote measurements adrressed this issue. > Boardwatch, from its history, comes from the user download experience. > One cannot blame them if they do their best to measure what they see. > One could instead offer to help them to update their next article. > > Second of all, there really are network performance problems. It > matters not at all if RSA (a problem I've had this week) has a powerful > WWW server, when I cannot get to it reliably because WillowSprings and > SanFrancisco are dropping packets like crazy. Certainly. I don't think anyone's really denying that, are they? (I'll admit to not knowing, having blindly deleted thru most of this thread ;-)). > So far, only one response has noted the current ongoing efforts at ISI, > LBL, Merit, NLANR, and others, to develop good network performance > measurement techniques and metrics. I urge Boardwatch to help fund > them, and to regularly publish the results! > > How many of you naysayers have actually participated in and help fund > the "scientific" studies? Put your money where your mouth is! I've put at least a small fraction of my time into work in nlanr and caida, and I think network performance measurements are a hard nut to crack, especially from the end user perspective. Real visibilty into causes of performance differences is lacking. My fear of measurements like the summary I've seen thus far from the keynote measurements is that consumers can be misled by such studies. From the summary, I think such a study is of questionable value and not something I'd want anyone I'm friendly with to use as a basis for a purchase decision. Even as an end user I give it little value from what I've seen thus far; I don't surf provider WWW sites, I surf elsewhere. I'd be much more interested in a weighted measurement of popular sites, driven from squid cache data or the like. And those of us who also offer Web-hosting services know (intimately) that it's not simple to root out network performance issues from WWW server issues. Yes, it's very possible, but it requires a vantage point that end users don't typically have. I don't really have anything 'bad' to say about the study, really (depending on how you take the above). I just don't find the summary to be of any personal value to me as an end user or as a network engineer. The summary doesn't answer any of my performance questions. Daniel ~~~~~~

1 0

RE: Internet Backbone Index
by Chris A. Icide 28 Jun '97

28 Jun '97

Considering the fact that our web site has not been a large factor in our marketing direction, and the site has received less that 1000 "hits" per week on average, it was and currently is (IMHO) a perfect choice for our application. However, in the last couple of weeks, it's been "bumped" to a 133MHz Pentium with *gasp* a fast ethernet card (as a result of a power supply fan failure in the previous box). Perhaps it was my fault, but I never imagined anyone claiming to have any knowledge whatsoever would attempt to perform a test of "backbone" providers performance based upon said vendors web server(s). I guess they call that "naive". Chris A. Icide ---------- From: Tim Gibson[SMTP:tim@taggnet.skyscape.net] Sent: Friday, June 27, 1997 2:45 AM To: Joseph T. Klein Cc: Chris A. Icide; nanog(a)merit.edu; 'Sean Donelan' Subject: Re: Internet Backbone Index On Thu, 26 Jun 1997, Joseph T. Klein wrote: > Is the NAP.NET web server still a 90MHz Pentium running BSDI? > > Chris A. Icide wrote: > > > Interesting enough, Does it REALLY matter?!?! As long as it ain't some MS product clogging the lan with needless broadcasts and it's big enough to service those that are ignorant to think a national provider's web site *IS* a measure of their backbone/service.... BSDI is a damn fine product!

3 2

Re: Keynote/Boardwatch Internet Backbone Index A better test!!!
by Jack Rickard 28 Jun '97

28 Jun '97

It's not embarrassing to me Randy. I'm in my element and having a ball. I haven't heard this much screeching in months. Go girlfriend. Jack Rickard ---------- > From: Randy Bush <randy(a)psg.com> > To: Jack Rickard <jack.rickard(a)boardwatch.com> > Cc: nanog(a)merit.edu > Subject: Re: Keynote/Boardwatch Internet Backbone Index A better test!!! > Date: Saturday, June 28, 1997 12:00 AM > > >> I want to measure the response of each ISP's marketing department. So I > >> will send a certified letter to their office of record in Deleware, and > >> report how long before I get the receipt back. Maybe I well do this for > >> 30 letters to get better data. > > I'm with you to a point. But NONE of the measurements were taken from > > Jack's house. > > So send the registered letters from 50 places. Same basic conceptual error. > You are not measuring anything like what you claim. > > But you seem not to want to hear this, so I will plonk this whole > embarrassing (to you) thread. Bye now. > > randy

2 1

Re: Internet Backbone Index
by Jack Rickard 28 Jun '97

28 Jun '97

I don't think I'm missing it. I think I'm disagreeing with it in as nice and nonconfrontational a way as I can given the crappy personality I have to work from. Splitting hairs from here to infinity on what "network" means and what the world wide web is departs rather widely from my mission here, so I'm giving it short shrift. If you don't know how ping and traceroute vary from data flows, I can't help much there either. If you want to draw a line of demarcation between a network and its performance, and a web server and its performance, you're free to do so. I just probably won't buy into it. On the actual concept that changing all the web servers will move the numbers: It might. It might not. I would probably bet at this point that there will be a lot of that going on among the non-moron crowd. I'm kind of hoping for it anyway. And then we'll see if the numbers move. My sense is that they will move some, and not as much as most seem to think. But it's true it could go the other way and be dramatic. I'm open to whatever results derive. Jack Rickard ---------- > From: Justin W. Newton <justin(a)priori.net> > To: Jack Rickard <jack.rickard(a)boardwatch.com>; Stan Barber <sob(a)academ.com>; vaden(a)texoma.net; SEAN(a)SDG.DRA.COM; nanog(a)merit.edu > Subject: Re: Internet Backbone Index > Date: Friday, June 27, 1997 2:50 PM > > > Jack, > I believe that you are missing the point that measuring web server > response time is /not/ the equivalent of measuring backbone performance. > > > At 12:45 PM 6/27/97 -0600, Jack Rickard wrote: > >They could be. The attempt is to factor that out. ALL measuring agents > >applied to ALL the backbones. And all contributed more or less equally to > >the end numbers. If a particular agent ran on a Commodore 64 with a kluged > >copy of KA9Q, and another agent ran on an Sun Solaris, both results would > >go into the result pile for all 29 measured networks. The net effect > >would be that the flaw would be in our "footprint" from which the > >measurements were taken. This footprint can only be a rough approximation > >of end user distribution anyway. It would affect absolute values relative > >to zero, but the relative indexes between networks should not be affected. > >Since we're looking at the relative relationship primarily, it wouldn't > >appear important. > > > > > >Jack Rickard > >---------- > >> From: Stan Barber <sob(a)academ.com> > >> To: Justin W. Newton <justin(a)priori.net>; Larry Vaden <vaden(a)texoma.net>; > >Sean Donelan <SEAN(a)SDG.DRA.COM>; nanog(a)merit.edu > >> Subject: Re: Internet Backbone Index > >> Date: Friday, June 27, 1997 1:54 PM > >> > >> Justin writes: > >> > ObAboutTopic: This is possibly the most flawed study on the planet. > >> > Remind me to get a fast web server. (And to think, we were going to > >put > >> > our web server in our office, behind a T-1, instead of in real estate > >near > >> > where the real bandwidth is that could be used for customers.). > >> > >> There are many studies more flawed. Consider some of the studies that > >> the Tobacco Institute has released over the years about the affects of > >> smoking. > >> > >> Concerning Internet performance, there have always been a variety of ways > >> of measuring it. It all depends on what you are really trying to measure. > >> The Keynote study is attempting to measure something to which the average > > > >> Internet user (not engineers) can relate. However, There are also > >clearly > >> the possibility of artifacts in the data because of the testing machine's > > > >> TCP stack or other issues (Vern Paxson has covered these issues at NANOG > >> and IETF meetings over the last few years). Checking their web site, > >their > >> software appears to run on top of the TCP stacks of many systems, so the > >> known artifacts of some of these platforms could be an issue. > >> > >> -- > >> Stan | Academ Consulting Services |internet: sob(a)academ.com > >> Olan | For more info on academ, see this |uucp: > >{mcsun|amdahl}!academ!sob > >> Barber | URL- http://www.academ.com/academ |Opinions expressed are only > >mine. > > > > > > ********************************************************* > Justin W. Newton voice: +1-415-482-2840 > Senior Network Architect fax: +1-415-482-2844 > PRIORI NETWORKS, INC. http://www.priori.net > Director At Large, ISP/C http://www.ispc.org > "The People You Know. The People You Trust." > *********************************************************

3 2

Re: floods
by fehnb＠jerseycape.net 28 Jun '97

28 Jun '97

Looks like the same one. As I recall, he got many, many answers. Some even addressed his question.... >Did this message get lost in a temporal warp about >3 months long or so? I seem to remember a thread >> >> I'm doing a project for school.Can you help?what is the scientific >> principle for floods?Thanks! > > Robert J. Fehn Sr.,President and CEO Senior Systems Engineer, Jersey Cape Information Systems Inc. "Access, Engineering, Microwave Links, Consulting, Design."

2 1

Re: Internet Backbone Index
by Sean Donelan 28 Jun '97

28 Jun '97

Ok, so I said I wasn't going to comment on the methodology, I lied. I wouldn't say the Keynote study is the worst ever. There are some really rotten studies in the fields of psychology and sociology. Since the Boardwatch/Keynote study didn't 'test' DRA Net, I guess I'm one of the few independent, disinterested parties to comment on the study's methods. A problem with the Keynote study is it seems very dependent on the location, type and connections of the testing platforms. Keynote mentions that connections from Dallas and Phoenix were slow to 'every' backbone site. This would indicate some systematic problem with the testing sites. Perhaps the results are even more dependent on the testing systems than the systems under test. There are also problems with outlier data points. For example, elsewhere on the Keynote site, the MCI web site had very fast access from 28 test sites (< 4secs), and very slow access from one test site in philly (> 14secs). Mixing and matching data points, if you left out the one outlier data point, MCI would have been faster than Savvis. So I don't know if the rankings are very meaningful if a single test site can have such a pronounced effect. Unlike a scientific study, there doesn't seem to be enough information to independently reproduce the results, so I'm just going from the bits and pieces I can glean from the Keynote pages, press release, and article. >Concerning Internet performance, there have always been a variety of ways >of measuring it. It all depends on what you are really trying to measure. >The Keynote study is attempting to measure something to which the average >Internet user (not engineers) can relate. However, There are also clearly >the possibility of artifacts in the data because of the testing machine's >TCP stack or other issues (Vern Paxson has covered these issues at NANOG >and IETF meetings over the last few years). Checking their web site, their >software appears to run on top of the TCP stacks of many systems, so the >known artifacts of some of these platforms could be an issue. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

3 2

Re: Keynote/Boardwatch Internet Backbone Index A better test!!!
by Jack Rickard 28 Jun '97

28 Jun '97

Randy: I'm with you to a point. But NONE of the measurements were taken from Jack's house. None were taken from Boardwatch's office. And in fact, the measurements were taken from 27 different geographic locations. I think that is a somewhat critical element here. We failed to come up with a methodology that could be performed from "Jack's House" without primarily measuring "Jack's connectivity" more than anything else. Obviously, that is what I would have liked. Never could come up with an idea on how to do that effectively. Inverse technology appears to be in the business of selling $50,000 marketing reports to order for anyone with $50,000.00. How would that be apt? Finally, I'm amazed at how "trivial it is to point out useless and devoid of meaning this is", when you haven't seen the results yet. It might be even EASIER once you've READ them. Jack Rickard ---------- > From: Randy Bush <randy(a)psg.com> > To: nanog(a)merit.edu > Subject: Re: Keynote/Boardwatch Internet Backbone Index A better test!!! > Date: Friday, June 27, 1997 3:00 PM > > I want to measure the response of each ISP's marketing department. So I > will send a certified letter to their office of record in Deleware, and > report how long before I get the receipt back. Maybe I well do this for 30 > letters to get better data. > > So, we now have an interesting report of the accessibility of various ISPs' > web servers from Jack's house. While it is trivial to point out how useless > and devoid of meaning this is, what is a wee bit harder is to describe what > would be a useful measure to Jack's audience and how that might be obtained. > > Ben Black's pointing out http://www.inversenet.com/about/backgrounder.html#2 > may be *very* apt, and somewhat ingenious, considering Jack's audience. > > Are there other good ideas or examples? > > randy

5 4

Re: Cisco access-lists ???
by Danny McPherson 28 Jun '97

28 Jun '97

to expand a bit .. i created an acl to match all prefixes longer than a /19: > access-list 101 permit ip any 255.255.240.0 0.0.15.255 and an acl to match everything else: > access-list 102 permit ip any any i created route-map "damp": > route-map damp permit 10 > match ip address 101 > set dampening 30 750 2000 90 > ! > route-map damp permit 20 > match ip address 102 > set dampening 15 750 2000 60 in the first sequence (10) of the route-map, i match all prefixes longer than a /19 (via acl 101) and apply the following dampening thresholds: half-life= 30 minutes reuse-value= 750 suppress-value= 2000 max-supress= 90 minutes in the second sequence i match everything else (via acl 102) and applied cisco's default values: halflife= 15 minutes reuse-value= 750 suppress-value= 2000 max-suppress= 60 minutes i then applied the route-map to bgp dampening: > router bgp 65534 > bgp dampening route-map damp each time a route flaps it gets a penalty - 1000. if the cumulative penalty is greater than the suppress-value (2000 here, so 3 flaps), the advertisement of the route will be suppressed. the penalty will be exponentially decayed based on the half-life. once the penalty decreases below the reuse-value (750 here), the advertisement will be unsuppressed. using cisco's defaults (the second sequence), if a route flaps a lot in a very short period then becomes stable, it will only be suppressed for a maximum of 1 hour. with the first sequence, the half-life is doubled and the maximum amount of time a route that has become stable will be suppressed is 90 minutes .. basically. back to the floods... -danny

1 0

RE: floods
by Sanjay Dani 28 Jun '97

28 Jun '97

>Here's the original one (different guy).... > >Unless its one of JF's friends getting us to waste cycles, my guess is >that these kids are entering "floods" into a search engine, and are >sending spam quantities of requests for info for their projects. I guess >"SYN floods" gets the search engine's attention... Today's "floods" poster probably meant it as a joke to spare us from the "backbone performance" drivel..

1 0