Mark Allman: Internet measurement: what next?
Folks- I sent the following note out the Internet Measurement Research Group (of the IRTF) mailing list last week. I'd love to hear from operations folk on these sorts of question... i.e., what would you love to be able to measure that you can't do terribly effectivly today? If you're interested in participating the mailing list is imrg@ietf.org. You can see the archive, subscription info, etc. at http://imrg.grc.nasa.gov/. Thanks! allman -- Mark Allman -- BBN/NASA GRC -- http://roland.grc.nasa.gov/~mallman/ ------- Forwarded Message To: imrg@ietf.org From: Mark Allman <mallman@grc.nasa.gov> Reply-To: mallman@grc.nasa.gov Subject: Internet measurement: what next? Organization: BBN Technologies/NASA GRC Song-of-the-Day: Lucky Ones Date: Wed, 02 Jul 2003 12:30:00 -0400 Sender: mallman@guns.lerc.nasa.gov I am interested in seeing some discussion about the fairly generic topic of what sorts of outstanding problems exist in Internet measurement: What are the core problems we need to solve? What do we need to be able to measure that we cannot measure very well today? Why? What are the problems in taking measurements? What can be done about those problems? What are practical problems that prevent wide-scale network measurement? What new tools does the community need to take, use, share and store measurements? Please jump in with answers o any or all of the above. Or, even with additional big-picture sorts of questions of your own. Thanks! allman (imrg chair) - -- Mark Allman -- BBN/NASA GRC -- http://roland.grc.nasa.gov/~mallman/ ------- End of Forwarded Message
Mark Allman wrote:
Folks-
I sent the following note out the Internet Measurement Research Group (of the IRTF) mailing list last week. I'd love to hear from operations folk on these sorts of question... i.e., what would you love to be able to measure that you can't do terribly effectivly today?
I'd like to masure throughtput, latency, and jitter of the entire Internet in a 3d model accessible via VR gear, color coded links, and cool abilities to highlight a path and examine it. Oh, and for interconnect points where you have access, a VR telnet session would be nice. (a boy can dream, can't he?) -Jack
I sent the following note out the Internet Measurement Research Group (of the IRTF) mailing list last week. I'd love to hear from operations folk on these sorts of question... i.e., what would you love to be able to measure that you can't do terribly effectivly today?
As predominantly a content hoster, I'd love to know more about the path between my servers and the end user. Stuff like how much bandwidth is available (or, potentially available, to remove the congestion issue), in real time (i.e. as fast as PMTUD works). Really stuff so I can decide whether to deliver broadband or narrowband content, etc. I'd then like to know if there was congestion on that route so that when they complain that downloading stuff is slow, I can point at where the problem lies. Simon -- Simon Lockhart | Tel: +44 (0)1628 407720 (x37720) | Si fractum Technology Manager | Fax: +44 (0)1628 407701 (x37701) | non sit, noli BBC Internet Services | Email: Simon.Lockhart@bbc.co.uk | id reficere BBC Technology, Maiden House, Vanwall Road, Maidenhead. SL6 4UB. UK
SL> Date: Mon, 7 Jul 2003 19:47:53 +0100 SL> From: Simon Lockhart SL> As predominantly a content hoster, I'd love to know more about the path SL> between my servers and the end user. Stuff like how much bandwidth is SL> available (or, potentially available, to remove the congestion issue), SL> in real time (i.e. as fast as PMTUD works). Really stuff so I can decide You mean something like RouteScience [supposedly] does or ECN influencing next hop? Yes, I know, that would get ugly for transit ASes. For the degenerate case of a leaf AS, though, things become simpler... Eddy -- Brotsman & Dreger, Inc. - EverQuick Internet Division Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 (785) 865-5885 Lawrence and [inter]national Phone: +1 (316) 794-8922 Wichita _________________________________________________________________ DO NOT send mail to the following addresses : blacklist@brics.com -or- alfra@intc.net -or- curbjmp@intc.net Sending mail to spambait addresses is a great way to get blocked.
E.B. Dreger wrote: SL> Date: Mon, 7 Jul 2003 19:47:53 +0100 SL> From: Simon Lockhart SL> As predominantly a content hoster, I'd love to know more about the path SL> between my servers and the end user. Stuff like how much bandwidth is SL> available (or, potentially available, to remove the congestion issue), SL> in real time (i.e. as fast as PMTUD works). Really stuff so I can decide It would be tricky, but I've heard of using javascript (not applicable with all EU's of course) to calculate the throughput (similar to various bandwidth testing pages) and set the results in a hidden field which the user would then submit in a form. Something to ponder when designing your various forms. Of course, a better method would be to ask your visitors to provide the information by running an applet which could feed you a lot of b/w and latency information. Total capacity would be a little more difficult and various theories used to calculate it blind don't work from dialups and are questionable on broadband. With the number of people that play with SETI and other distributed systems, I was thinking it'd be interesting to build a 'net monitor based on the same premise, pulling latency information peer to peer as well as building path maps using the multiple views. While we have this to some degree, 1M 'doze boxes would provide a lot more granular detail. Overall performance through certain paths could also be determined. -Jack
On Tuesday, July 8, 2003, at 12:24AM, Jack Bates wrote:
E.B. Dreger wrote: SL> Date: Mon, 7 Jul 2003 19:47:53 +0100 SL> From: Simon Lockhart
SL> As predominantly a content hoster, I'd love to know more about the path SL> between my servers and the end user. Stuff like how much bandwidth is SL> available (or, potentially available, to remove the congestion issue), SL> in real time (i.e. as fast as PMTUD works). Really stuff so I can decide
It would be tricky, but I've heard of using javascript (not applicable with all EU's of course) to calculate the throughput (similar to various bandwidth testing pages) and set the results in a hidden field which the user would then submit in a form. Something to ponder when designing your various forms.
Of course, a better method would be to ask your visitors to provide the information by running an applet which could feed you a lot of b/w and latency information. Total capacity would be a little more difficult and various theories used to calculate it blind don't work from dialups and are questionable on broadband.
With the number of people that play with SETI and other distributed systems, I was thinking it'd be interesting to build a 'net monitor based on the same premise, pulling latency information peer to peer as well as building path maps using the multiple views. While we have this to some degree, 1M 'doze boxes would provide a lot more granular detail. Overall performance through certain paths could also be determined.
Gomez seems to be trying to do this, with a monetary incentive: http://www.porivo.com/peernetwork/jsp/index.jsp
-Jack
-- Matt Levine <matt@deliver3.com> "The Trouble with doing anything right the first time is that nobody appreciates how difficult it was." -BIX
Matt Levine wrote:
Gomez seems to be trying to do this, with a monetary incentive:
Test is narrowed to webserver performance and is limited in the actual test methods. From what I can tell, it says nothing about network performance except in the most general aspects (I can download a page from network X faster than network Y). Since it is testing webservers and workload is provided by a central site, there is no peer to peer interaction or discovery. The RIPE-NNC tests are much more related to what I was refering to, although they are limited in many reguards. -Jack
On Tuesday, July 8, 2003, at 3:59PM, Jack Bates wrote:
Matt Levine wrote:
Gomez seems to be trying to do this, with a monetary incentive: http://www.porivo.com/peernetwork/jsp/index.jsp
Test is narrowed to webserver performance and is limited in the actual test methods. From what I can tell, it says nothing about network performance except in the most general aspects (I can download a page from network X faster than network Y). Since it is testing webservers and workload is provided by a central site, there is no peer to peer interaction or discovery. The RIPE-NNC tests are much more related to what I was refering to, although they are limited in many reguards.
Indeed, but some tweaking to the software could have it act in a more broad nature, my point was simply that there's somebody trying to build a metric-oriented 'network' of end-user nodes, similar to SETI (though obviously a commercial venture, in this case).
-Jack
-- Matt Levine <matt@deliver3.com> "The Trouble with doing anything right the first time is that nobody appreciates how difficult it was." -BIX
On 08.07 14:59, Jack Bates wrote:
.... The RIPE-NNC tests are much more related to what I was refering to, although they are limited in many reguards.
If you tell us what limits you want removed we may work on that! We are definitely working towards making the results generally available; see http://www.ripe.net/ripe/docs/ripe-271.html for details of that proposal. So far we have had very positive reactions on this although some ISPs are worried about being exposed in comparison to competitors who do not participate. We will also add quite detailed measurments of DNS servers for the root and TLDs. Data has been taken for several months now. A beta version of the resulting products will be available soon. I have been thinking about a multi-tier measurement network, adding probes that do not have dedicated hardware and high precision clocks, but rather conisist of just a software package. These could be deloyed more easily. Daniel Karrenberg RIPE NCC
Daniel Karrenberg wrote:
If you tell us what limits you want removed we may work on that!
Sounds like below as if you are working on it.
We are definitely working towards making the results generally available; see http://www.ripe.net/ripe/docs/ripe-271.html for details of that proposal. So far we have had very positive reactions on this although some ISPs are worried about being exposed in comparison to competitors who do not participate.
Raw data is important and if publicly available, I think you will find a lot of people assimilating the data in ways that others haven't thought of, as well as contributing suggestions for tests to obtain data that they feel is missing for their specific purposes.
We will also add quite detailed measurments of DNS servers for the root and TLDs. Data has been taken for several months now. A beta version of the resulting products will be available soon.
I think this would be very useful, especially in conjunction with the current tests and the data available concerning route views.
I have been thinking about a multi-tier measurement network, adding probes that do not have dedicated hardware and high precision clocks, but rather conisist of just a software package. These could be deloyed more easily.
I recommend operating system independant where possible. Store with the data the information pertaining to the testing machine. ie. Was the measurement taken over a dialup (should be detectable), what OS was it, and what version of resolver was used for dns tests? This would show performance information associated with this information as well as allowing weights in data analysis. Path latencies in both directions, detmining if the path was a/symetrical, etc would also be useful. Of course, this information will change over time and must be resampled with each testing interval. Take the tests to the user, and the network's involved won't have a choice on the public nature of network performance, availability, and routing. Everyone would be affected, and it would help provide an overall understanding of the 'net as a whole. For security purposes, you could allow a blind spot, so that companies might be interested in using the testing equipment. Such a system would black out the statistics and information concerning the internal network. -Jack
Eddy, E.B. Dreger wrote:
You mean something like RouteScience [supposedly] does or ECN influencing next hop?
Guilty as charged, of course :-)
Yes, I know, that would get ugly for transit ASes.
Well, the measurement itself isn't ugly - it's just a question of what you choose to do with the information. Some more conservative folks just take the data over time, and use it for peer or transit elimination, for example.
For the degenerate case of a leaf AS, though, things become simpler...
Degenerate seems a little strong, even in its mathematical sense - rather like an auto maker calling car drivers the "degenerate" case of mechanics. But on this list, I suppose I'll let it pass :-) I'll agree that the constraints on active use of measurements for route change are lighter when you have no BGP downstreams. But it's still possible to use good end to end measurements, with very strong damping, to make changes in a transit routing context. Others in this thread have been talking about bandwidth measurement along the variety of paths you use across other peoples' networks. To me, that's a good answer to one of Mark's original questions: What do we need to be able to measure that we cannot measure very well today? That is, I consider end to end bandwidth one of those quantities we'd love to measure widely and often, but practically, we cannot. Available techniques are ugly and unscalable, but more disconcertingly, estimates of bandwidth appear to have a shelf-life comparable to certain sub-atomic particles. That's not to say it's a lost cause, of course. Many end to end paths are constrained primarily by first hop (which you can usually measure directly with SNMP) and last hop (which is roughly constant, and next to impossible to influence). For the remainder of the path in between, it's important to characterize it, but accurately assessing bandwidth is hard. Measuring probability of packet loss over time is a very practical proxy. It won't give you all that precise a projection of your goodput if you throw, say, a multi-gigabyte ftp down the pipe, but for short or light transactions, it's not bad at all. After all, once you've accounted for first and last hop, your traffic is probably a pretty minor fraction of total load on each link along the chain. Mike Lloyd CTO, RouteScience
participants (7)
-
Daniel Karrenberg -
E.B. Dreger -
Jack Bates -
Mark Allman -
Matt Levine -
Mike Lloyd -
Simon Lockhart