Re: Keynote/Boardwatch Internet Backbone Index A better test!!!
We have all seen the results of the survey now and there is a lot of opinion that there are many flaws in the logic. So the question is what is a legitimate test? I remember At the Ann Arbor NANOG there was a report on a project to test reachability, what is the current status of that project? Is there any strong consensus to perform a similar test that is engineered without the flaws in this test? ---> Phil
I think first you need to know what your looking for in order to find it. There is a large difference between reachability from A to B, overall network performance between point A and B, backbone performance between A and B, and host performance on B. You can only attempt to find an answer if you know the question. I havent looked closely at the Keynote performance data.. but it seems it is looking at a 'general' performance check from point A/B/C/D/E etc, TO point Z. That is fine, but as with any stats, they can be easily munged to imply what is wanted, and not what is. Personally, i dont think checking the network path to www.'NSP'.net is the correct way to do it. To me, tracing the path from multiple locations TO www.'NSPs-Customer'.com is better.. altho that doesnt take into account 'peak' times, anomolies, freakish events, position of the moon.. etc.. However, i do find stats kind of entertaining to interpret, i dont think there is a way to set in stone that by doing X you can find a 'perfect' value that represents the statistics from A to B. Just my 2 cents.. -mike On Fri, 27 Jun 1997, Philip J. Nesser II wrote:
We have all seen the results of the survey now and there is a lot of opinion that there are many flaws in the logic. So the question is what is a legitimate test? I remember At the Ann Arbor NANOG there was a report on a project to test reachability, what is the current status of that project? Is there any strong consensus to perform a similar test that is engineered without the flaws in this test?
---> Phil
On Fri, 27 Jun 1997, Mike Hedlund wrote:
Personally, i dont think checking the network path to www.'NSP'.net is the correct way to do it. To me, tracing the path from multiple locations TO
Exactly. Take SprintLink as an example. www.sprintlink.net and www.sprint.com are the same box [martha.commerce.com (198.70.62.12)], but that sits off some T1 somewhere. www.sprint.net (aka ftp.sprintlink.net) seems to be in a better position network-wise to determine real network performance. It's highly unlikely that I'll burn an FE port to stick a web server up -- it's more likely that I'll put the www machine on a shared 10M segment with other corporate machines. Also, not everyone will use the fastest machine in the world as their web server -- expecially for the company (corporate) machine.
www.'NSPs-Customer'.com is better.. altho that doesnt take into account 'peak' times, anomolies, freakish events, position of the moon.. etc..
Hopefully these would be global effects that would show up in everyone's data - since KeyNote apparently performed the tests from "from 27 major metropolitan areas in the United States". Oh, does that say 'United States'? No wonder Bell Canada got screwed.
However, i do find stats kind of entertaining to interpret, i dont think there is a way to set in stone that by doing X you can find a 'perfect' value that represents the statistics from A to B.
It sounds like their metric was strictly a 'download time' mertic - maybe they also took into account the time to connect() - but either way they are depending on one machine to judge an entire network -- which is a highly flawed metric. Also, what if it takes 3 GET's to download 50k on one server, and on another it takes 20 GET's? That is a *huge* difference in overhead for the server. I wonder how much they thought about their measuring methods before they decided to implement them. So how *do* you "correctly" measure a provider's bandwidth and performance? Well, I don't think there is any simple answer to that question -- if one at all -- as Mike points out :-) I'd like to see Keynote publish some details on how, exactly, they performed the tests - and exactly what tests they performed. -Taner -- D. Taner Halicioglu taner@isi.net Programmer/Engineer/Sysadmin ISI / GlobalCenter Voice: +1 408 543 0313 Fax: +1 408 541 9878 PGP Fingerprint: 65 0D 03 A8 26 21 6D B8 23 3A D6 67 23 6E C0 36
participants (3)
-
Mike Hedlund -
Philip J. Nesser II -
Taner Halicioglu