On Thu, Oct 31, 2019 at 1:08 PM Jeff Shultz <jeffshultz@sctcweb.com> wrote:
[snip] What has most people (from anecdotal observation) concerned is that we are usually more than one or two carriers out from an IXP where the speed test server will be, and don't have a lot of influence on paths and carriers that we aren't directly connected with.
It sounds like there would be some test method concerns there by having merely one performance-testing server. But the performance of "broadband service" is really end to end; the choice of direct carrier, their routing policies, and indirect carriers, is still an integral part of the service that should be measured --- the best last mile connection possible has no value if the provider is allowed to mess that up by undersizing peering or backhaul, whether directly, or indirectly through carriers which end-to-end traffic depends upon. Seems like a testing method might have a plethora of speed testing servers and include HTTPS bandwidth tests through websites fronted by numerous CDN nodes which are by design indistinguishable from regular traffic. Given enough varying remote test locations and a large enough number of samples over time, and providers prevented from being able to distinguish what traffic or users might be test traffic or test users and which users or traffic might be normal traffic; It seems like they ought be able to formulate an automatic analysis of the data that will limit the affect of "noise" such as one-off suboptimal routing to some destinations, involving one IXP, etc.
-- Jeff Shultz -- -JH