- Test - lists.nanog.org

RE: Keynote/Boardwatch Internet Backbone Index A better test!!!
by Gene Shklar 27 Jun '97

27 Jun '97

Thanks for the suggestions. We're considering doing a follow-up and perhaps making this a regular feature every 2 or 4 months thereafter. We've received a few suggestions for methodology changes/enhancements, and also several emails so far denouncing our methodology but not explaining why (which is typical of people in many areas -- politics, the environment, economics, whatever -- who disagree emotionally but not intellectually with the conclusions of a study). The current methodology generally shows how a web site connected to a particular backbone appears to the general internet population of users. The results are intended to be a guide (but not the only one) for helping web sites select or evaluate a collocation, hosting, or access provider. Your methodology suggestion would be useful to include because its results would also help end-users select their dial-up ISPs based on the backbone that those ISPs are connected to. Gene Shklar -------------------------------------------------------- Gene Shklar GeneShklar(a)keynote.com Keynote Systems, Inc. voice (415) 524-3011 Two West Fifth Avenue fax (415) 524-3099 San Mateo, CA 94402 main # (415) 524-3000 http://www.keynote.com "A great Internet application experience is all a matter of customer perspective." ---------- From: Peter Cole[SMTP:Peter.Cole@telescan.com] Sent: Friday, June 27, 1997 10:57 AM To: nanog(a)merit.edu Cc: marketing(a)keynote.com Subject: RE: Keynote/Boardwatch Internet Backbone Index A better test!!! I would like to see the test run again with the following change. >From each provider test the response time of the other 28 sites and not the providers own web server. Then average the response times for these other 28 web servers and report that average response time from that provider. The providers with good connectivity to the rest of the net should have lower average response time. P.S. One might also be interested in the top one hundred web sites average response time. Peter Cole of Telescan, Inc. (281)588-9155 Better computing through lack of sleep. > ---------- > From: Golan Ben-Oni[SMTP:bnite@tremere.ios.com] > Sent: Thursday, June 26, 1997 3:53 PM > To: nanog(a)merit.edu > Subject: Keynote/Boardwatch Internet Backbone Index > > For shits and grins: > > http://www.keynote.com/measures/backbones/backbones.html > > -Golan >

4 3

RE: Keynote/Boardwatch Internet Backbone Index A better test!!!
by Wheeler, Jesse W 27 Jun '97

27 Jun '97

So are we all going to all resort to name calling now to get our points across? Every time you deal with a dynamic object such as the Internet, where conditions can change on a minute by minute basis, benchmark results are close to useless. The only thing that benchmarks on a dynamic object can do is give you a rough estimate of the actual throughput.. The variables to be tested in something massive as the Internet and too numerous to count.. SO, lets get on with it :) ------------------------------------------------------------------------ ------------------------ Jesse W. Wheeler Quality Assurance Analyst Reynolds & Reynolds Healthcare Systems Division Email: Jesse_Wheeler(a)reyrey.com ------------------------------------------------------------------------ -------------------------------------------------------- >---------- >From: Jack Rickard[SMTP:jack.rickard@boardwatch.com] >Sent: Friday, June 27, 1997 2:00 PM >To: Ben Black; nanog(a)merit.edu >Subject: Re: Keynote/Boardwatch Internet Backbone Index A better test!!! > >I'm not a marketing droid. But only a moron would think that overall >performance would NOT affect the download of a web page, which is >essentially what you are attempting to say. > >There are certainly other ways to do it. Approximately Internet User >Population raised to Internet User Population worth. > >Jack RIckard > >=================================================================== >Jack Rickard Boardwatch Magazine >Editor/Publisher 8500 West Bowles Ave., Ste. 210 >jack.rickard(a)boardwatch.com Littleton, CO 80123 >www.boardwatch.com Voice: (303)973-6038 >=================================================================== > > >---------- >> From: Ben Black <black(a)zen.cypher.net> >> To: Craig A. Huegen <c-huegen(a)quadrunner.com> >> Cc: Jack Rickard <jack.rickard(a)boardwatch.com>; Peter Cole ><Peter.Cole(a)telescan.com>; nanog(a)merit.edu; marketing(a)keynote.com >> Subject: Re: Keynote/Boardwatch Internet Backbone Index A better test!!! >> Date: Friday, June 27, 1997 2:28 PM >> >> for an exmple of somewhat more complete and better designed benchmarks of > >> this type: >> >> http://www.inversenet.com/about/backgrounder.html#2 >> >> note that they understand the numerous factors that contribute to overall > >> performance. only a marketing droid could think downloading 50k worth of > >> web pages is somehow an indicator of overall performance. >> >> >> b3n >

1 0

Re: Internet Backbone Index
by sob＠academ.com 27 Jun '97

27 Jun '97

> They could be. The attempt is to factor that out. ALL measuring agents > applied to ALL the backbones. And all contributed more or less equally to > the end numbers. If a particular agent ran on a Commodore 64 with a kluged > copy of KA9Q, and another agent ran on an Sun Solaris, both results would > go into the result pile for all 29 measured networks. The net effect > would be that the flaw would be in our "footprint" from which the > measurements were taken. This footprint can only be a rough approximation > of end user distribution anyway. It would affect absolute values relative > to zero, but the relative indexes between networks should not be affected. > Since we're looking at the relative relationship primarily, it wouldn't > appear important. > > > Jack Rickard Jack, you appear to be saying that the ranking is the important issue, but that the actually timing numbers are not important. Will you clarify this for me? The web page does not say what the configuration of the "high performance workstations" are and there is the possibliity that the artifacts could be so large that none of the data is meaningful. If you are saying that the "high performance workstations" vary from "a Commodore 64 with a kluged copy of KA9Q, and another agent ran on an Sun Solaris", you still have not eliminated the possiblity that the artifacts are so big to make the data meaningful. [Although, I note that Keynote does not publish the fact that they run on KA9Q on a Commodore 64 on their web site, so I would assume that none of the agents really do run on that configuaration.] Could you possibly get more detail on these configurations? -- Stan | Academ Consulting Services |internet: sob(a)academ.com Olan | For more info on academ, see this |uucp: {mcsun|amdahl}!academ!sob Barber | URL- http://www.academ.com/academ |Opinions expressed are only mine.

1 0

Re: Internet Backbone Index
by Jack Rickard 27 Jun '97

27 Jun '97

They could be. The attempt is to factor that out. ALL measuring agents applied to ALL the backbones. And all contributed more or less equally to the end numbers. If a particular agent ran on a Commodore 64 with a kluged copy of KA9Q, and another agent ran on an Sun Solaris, both results would go into the result pile for all 29 measured networks. The net effect would be that the flaw would be in our "footprint" from which the measurements were taken. This footprint can only be a rough approximation of end user distribution anyway. It would affect absolute values relative to zero, but the relative indexes between networks should not be affected. Since we're looking at the relative relationship primarily, it wouldn't appear important. Jack Rickard ---------- > From: Stan Barber <sob(a)academ.com> > To: Justin W. Newton <justin(a)priori.net>; Larry Vaden <vaden(a)texoma.net>; Sean Donelan <SEAN(a)SDG.DRA.COM>; nanog(a)merit.edu > Subject: Re: Internet Backbone Index > Date: Friday, June 27, 1997 1:54 PM > > Justin writes: > > ObAboutTopic: This is possibly the most flawed study on the planet. > > Remind me to get a fast web server. (And to think, we were going to put > > our web server in our office, behind a T-1, instead of in real estate near > > where the real bandwidth is that could be used for customers.). > > There are many studies more flawed. Consider some of the studies that > the Tobacco Institute has released over the years about the affects of > smoking. > > Concerning Internet performance, there have always been a variety of ways > of measuring it. It all depends on what you are really trying to measure. > The Keynote study is attempting to measure something to which the average > Internet user (not engineers) can relate. However, There are also clearly > the possibility of artifacts in the data because of the testing machine's > TCP stack or other issues (Vern Paxson has covered these issues at NANOG > and IETF meetings over the last few years). Checking their web site, their > software appears to run on top of the TCP stacks of many systems, so the > known artifacts of some of these platforms could be an issue. > > -- > Stan | Academ Consulting Services |internet: sob(a)academ.com > Olan | For more info on academ, see this |uucp: {mcsun|amdahl}!academ!sob > Barber | URL- http://www.academ.com/academ |Opinions expressed are only mine.

2 1

Re: Network IP analysis?
by Havard.Eidnes＠runit.sintef.no 27 Jun '97

27 Jun '97

> MRTG is a great graphing tool for small applications. Unfortunately > the package does not scale very well for my purposes. The snmp polling > and data storage in the logfiles are areas that I have problems with > using MRTG. Are there any other viable alternatives to MRTG, or are > there any existing mods to mrtg that graph right out of a file rather > than polling for itself? I already have the data I need and would > like to reuse it. If possible I would like to have a package that > is separated into 3 sections. One section can be called upon to > do polling or gathering data out of a repository. The second can > be called upon to do statistics runs. The 3rd can be called to provide > graphs on demand. The package I have been fiddling with on and off for a while and which was recently named "NORDstat" may do some of what you want. It has a simple textual and easily understood (hah!) format for the raw data files, so that it should not be too difficult to convert the data you already have to this format. It has separate modules to produce graphs and tabular textual reports, and the graphs (and the tabular reports++) can be presented via a WWW interface (the WWW setup is even cache-friendly). Some of the plotting scripts fetch the data from the tabular reports, some fetch them directly from the raw data files. I have recently made the package available for testing, you can reach it (together with "live data" for NORDUnet) at http://www.nordu.net/stats/ If there is interest I may set up a few mailing lists for information dissemination about the package, typically an -announce list and list for general discussion of the package. If any of you are interested in this, send me e-mail. Regards, - Håvard

1 0

Re: Internet Backbone Index
by sob＠academ.com 27 Jun '97

27 Jun '97

Justin writes: > ObAboutTopic: This is possibly the most flawed study on the planet. > Remind me to get a fast web server. (And to think, we were going to put > our web server in our office, behind a T-1, instead of in real estate near > where the real bandwidth is that could be used for customers.). There are many studies more flawed. Consider some of the studies that the Tobacco Institute has released over the years about the affects of smoking. Concerning Internet performance, there have always been a variety of ways of measuring it. It all depends on what you are really trying to measure. The Keynote study is attempting to measure something to which the average Internet user (not engineers) can relate. However, There are also clearly the possibility of artifacts in the data because of the testing machine's TCP stack or other issues (Vern Paxson has covered these issues at NANOG and IETF meetings over the last few years). Checking their web site, their software appears to run on top of the TCP stacks of many systems, so the known artifacts of some of these platforms could be an issue. -- Stan | Academ Consulting Services |internet: sob(a)academ.com Olan | For more info on academ, see this |uucp: {mcsun|amdahl}!academ!sob Barber | URL- http://www.academ.com/academ |Opinions expressed are only mine.

1 0

RMON probes .vs. sniffer
by Peter Giza 27 Jun '97

27 Jun '97

Has/Is anyone using Axon, NetMetrix etc, or evaluated these tools as a replacement for a sniffer? We're trying to decide whether to buy one or the other. The HP product provides a lot of nice graphs etc, but how practical is it for diagnostics and monitoring for intrusion etc. TIA -pete -- --/ Peter E. Giza || Technical Consultant || ADSmart Corporation fone 508.684.3609 || phax 508.684.3618 || page 800.632.1746 || http://www.adsmart.net /--

2 1

RE: Keynote/Boardwatch Internet Backbone Index A better test!!!
by Peter Cole 27 Jun '97

27 Jun '97

I would like to see the test run again with the following change. >From each provider test the response time of the other 28 sites and not the providers own web server. Then average the response times for these other 28 web servers and report that average response time from that provider. The providers with good connectivity to the rest of the net should have lower average response time. P.S. One might also be interested in the top one hundred web sites average response time. Peter Cole of Telescan, Inc. (281)588-9155 Better computing through lack of sleep. > ---------- > From: Golan Ben-Oni[SMTP:bnite@tremere.ios.com] > Sent: Thursday, June 26, 1997 3:53 PM > To: nanog(a)merit.edu > Subject: Keynote/Boardwatch Internet Backbone Index > > For shits and grins: > > http://www.keynote.com/measures/backbones/backbones.html > > -Golan >

1 0

Keynote/Boardwatch Internet Backbone Index
by Golan Ben-Oni 27 Jun '97

27 Jun '97

For shits and grins: http://www.keynote.com/measures/backbones/backbones.html -Golan

3 2

RE: Internet Backbone Index
by Chris A. Icide 27 Jun '97

27 Jun '97

Interesting enough, Take this little tidbit. The methodology in general was performance testing of the backbone's web site. (Which in itself is probably an accurate measurement goal for the level of subscribers to that magazine) However, Nap.Net's web site was sitting behind a Fractional T1 off of our chicago node while the web page was being overhauled during this test period. So, the rating for Nap.Net reflects a Fractional (actually 22 channels) T1 performance. This is interesting in the light that Nap.Net still scored above the average and the median. Chris ---------- From: Sean Donelan[SMTP:SEAN@SDG.DRA.COM] Sent: Thursday, June 26, 1997 5:34 PM To: nanog(a)merit.edu Subject: Internet Backbone Index I guess it was only a matter of time. The Keynote/Boardwatch Internet Backbone Index http://www.keynote.com/measures/backbones/backbones.html I'm not going to comment on the methodology flaws, but it does show what will fill the information gap. I do like the line "The world's fastest network provides good service to its users only if it does a good job of connecting to the rest of the Internet." Unfortunately I don't think this study really shows that. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

4 3