Announcing Project BISMark: ISP Performance Measurements from Home Routers
Hello NANOG, We've launched Project BISMark, a project that performs active performance measurements of upload and download throughput, latency, etc. from OpenWRT-based routers running inside of homes. We have tested our OpenWRT image on the NetGear WNDR 3700v2 and are currently shipping out NetGear routers with the BISMark firmware to anyone who is interested. 1. If you're interested in receiving a NetGear WNDR 3700v2 with our measurement suite, please sign up on the project site. We are planning to give out more devices in mid-July. You can watch a video about BISMark here: http://challenge.gov/challenges/114/submissions/3133-bismark-a-broadband-int... 2. Help us: if you would like to cast your vote for our tool on the project page above for the FCC Open Internet Application Challenge at the link above, we'd really appreciate your support. http://tinyurl.com/VoteBISMark 3. We're also looking for help in developing software. Project information and source code at: http://github.com/bismark-devel/ Thanks! Nick * To get a sample of the type of data we show to users, watch the video above, or check out the following example (in development): http://networkdashboard.org/device/NB105/ * Note: We do not collect any personal or private data; we only collect data from measurements that we perform, and we will share all of that data with you.
On Mon, 27 Jun 2011, Nick Feamster wrote:
We've launched Project BISMark, a project that performs active performance measurements of upload and download throughput, latency, etc. from OpenWRT-based routers running inside of homes. We have tested our OpenWRT image on the NetGear WNDR 3700v2 and are currently shipping out NetGear routers with the BISMark firmware to anyone who is interested.
Please, pretty please, with sugar on top, don't just do active measurement, but also do passive measurement of real traffic. Doing test traffic is one case, but the really important thing to look at is real traffic. I tried to get traction for this on IETF75, but there seems to be little interest. On a NAT router there is a state table, what would the performance penality be to look at TCP sequence numbers, RTTs (TCP timestamps) to be able to discern PDV and loss of the actual traffic the customer is doing? There are a lot of test suites, they solve one problem, but a passive monitoring system that would show how the real traffic is behaving would yield a lot more valuable information that just relying on active testing (which will cause harm to customer traffic when the test is run). -- Mikael Abrahamsson email: swmike@swm.pp.se
Thanks for the feedback! On Jun 28, 2011, at 6:13 AM, Mikael Abrahamsson wrote:
On Mon, 27 Jun 2011, Nick Feamster wrote:
We've launched Project BISMark, a project that performs active performance measurements of upload and download throughput, latency, etc. from OpenWRT-based routers running inside of homes. We have tested our OpenWRT image on the NetGear WNDR 3700v2 and are currently shipping out NetGear routers with the BISMark firmware to anyone who is interested.
Please, pretty please, with sugar on top, don't just do active measurement, but also do passive measurement of real traffic. Doing test traffic is one case, but the really important thing to look at is real traffic. I tried to get traction for this on IETF75, but there seems to be little interest.
We would very much like to. There are a number of reasons that regular users seem to be asking for passive measurement, such as monitoring of traffic usage of different applications (e.g., "How much is streaming eating into my usage cap?"). A few years ago, we had a tool that would do all of this with passive measurement (http://gtnoise.net/nano/), and we'd certainly like to resume this line of inquiry, if we can figure out how to address people's privacy concerns. We are developing a passive measurement suite for BISMark, which is also available on github.
On a NAT router there is a state table, what would the performance penality be to look at TCP sequence numbers, RTTs (TCP timestamps) to be able to discern PDV and loss of the actual traffic the customer is doing?
There are a lot of test suites, they solve one problem, but a passive monitoring system that would show how the real traffic is behaving would yield a lot more valuable information that just relying on active testing (which will cause harm to customer traffic when the test is run).
Definitely good points, and we've thought about this, for sure. The key question seems to be how to handle user privacy in a way that everyone can be happy with. Ultimately, we might take a survey of our users about this (e.g., certain people have said they don't mind tracking application performance/usage as long as the specific Web sites or destinations are not logged). It would be really helpful to get an understanding of what users might find acceptable, as far as passive measurements. -Nick
Hello, On Mon, Jun 27, 2011 at 9:48 PM, Nick Feamster <feamster@cc.gatech.edu> wrote:
Hello NANOG,
We've launched Project BISMark, a project that performs active performance measurements of upload and download throughput, latency, etc. from OpenWRT-based routers running inside of homes. We have tested our OpenWRT image on the NetGear WNDR 3700v2 and are currently shipping out NetGear routers with the BISMark firmware to anyone who is interested.
Is this similar to the UK (Ofcom, http://www.ofcom.org.uk/) and US (FCC, http://www.fcc.gov/) regulators scheme that is being run by Sam Knows at http://www.samknows.com/broadband/test_my_isp and http://www.samknows.com/broadband/broadband_performance ? I'd be quite interested to know the difference between them. Alex
Hi Alex, On Jun 28, 2011, at 6:30 AM, Alex Brooks wrote:
Is this similar to the UK (Ofcom, http://www.ofcom.org.uk/) and US (FCC, http://www.fcc.gov/) regulators scheme that is being run by Sam Knows at http://www.samknows.com/broadband/test_my_isp and http://www.samknows.com/broadband/broadband_performance ?
Yes, it's very closely related, and we've been in close contact with them (we have a paper upcoming at SIGCOMM that is based on the US data that we've jointly authored). One of the differences is that the platform is open / open-source. For example, anyone would be able to develop and run their own tests from the box. Another difference is that we're looking into a variety of other things (e.g., measurements of performance inside the home, deployments in other regions, maybe ultimately passive measurements if we can figure out how to balance the more sensitive aspects of that). Our goal is to have some sets of tests that are running/could be run on either platform. We're interested in helping to improve the tests that are run on the SK routers, as well. -Nick
participants (3)
-
Alex Brooks -
Mikael Abrahamsson -
Nick Feamster