lhoward@UU.NET writes:

1. Although a variety of backbones is used, the study does not say which ones. Also, even though the study does point out the assymetrical routing of a web transavtion (hot-potato), it doesn't point out that the traffic being measured is a brief web request (which is dumped to the web server's backbone ASAP) answered by a long response (10KB in this case, dumped to the querier's backbone).

Ironically, the only person I can remember actively (and publically) questioning the wisdom of hot potato routing for several applications is Curtis Villamizar. While Tony Bates and I had what seemed to be good reasons for doing closest-exit routing (among other routing policies) at the time, and there are strong economic incentives for maintaining closest-exit routing among peers, there are certainly drawbacks. Considering that the Keynote/Boardwatch study purports to help content-providers decide where to locate their content, poor performance attributable to closest-exit routing should have engineers considering the wisdom of that particular routing policy with respect to that particular market segment, depending on the focus of their businesses. The nice thing about the Keynote test is that it might get some of the cleverer people around here thinking of the Internet in a more abstract way than they might be used to. When I think in a more end-to-end fashion about comment #1 above, it seems that a provider who is rated poorly deserves the rating, even though the performance problem may be attributable to a peer's network's congested lines, broken equipment, or whatever. In effect, by taking an end-to-end approach, this test also rates networks' routing policies. If the routing policy drives traffic through a competitor's suboptimal path, and this degrades end-to-end performance, then the provider with that routing policy will receive a poorer rating than otherwise. One of the things that should be thought about is that there may be ways to improve end-to-end goodput by changing one's own network based on what one can infer from tests similar to the Keynote ones (or indeed similar to Vern Paxson's tests), as much as what one can infer from what one's own equipment is doing or seeing.

All of the previous notwithstanding, I would be interested in a better version of this study. Even more interesting would be to track how providers do over the course of several studies--who responds well to backbone congestion?

Personally, I would take this comment as a sign that the study and its publication is astoundingly successful. Finally, I would rather see people respond to congestion in a cleverer fashion, i.e. by deploying or pressing for the deployment of better WWW transport technology. There are some people with very clever ideas about caching and lazy proxies, and some clever ideas about changing HTTP in dramatic ways, that deserve some attention, as these ideas are more likely to help people and information meet more quickly in the long run than any amount of tweaking the underlying transport fabric ever will. Sean.