AT&T and Edgemarc now believe they might have shipped routers with firmware that is causing the issues. We will upgrade the firmware on Tuesday and see if that is the case. Thanks everyone for the input. On Thu, Feb 17, 2011 at 9:04 AM, Mikeal Clark <mikeal.clark@gmail.com>wrote:
The routers are Edgemarc. P/N EM-4608T4
http://www.edgewaternetworks.com/edgemarc_overview_page.htm
On Thu, Feb 17, 2011 at 8:19 AM, Ryan Finnesey < ryan.finnesey@harrierinvestments.com> wrote:
What type of hardware are they using for this BIB router?
Cheers Ryan
-----Original Message----- From: Mikeal Clark [mailto:mikeal.clark@gmail.com] Sent: Wednesday, February 16, 2011 6:16 PM To: Jim Gettys Cc: nanog@nanog.org Subject: Re: AT&T MPLS / BIB Routers
I'm building up to 3000-4000ms latency with these BIB routers. We never had this issue on the old point to points using Cisco gear.
On Wed, Feb 16, 2011 at 5:09 PM, Jim Gettys <jg@freedesktop.org> wrote:
On 02/16/2011 05:44 PM, Mikeal Clark wrote:
We just put in a AT&T MPLS and are having a pretty negative experience with the "Business in a Box" routers they are using for our smaller sites. We are seeing extremely high latency under load.
Anyone have any experience with these devices that could shed some light on this? Are they really this bad?
There is excessive buffering in all sorts of devices all over the Internet. This causes high latency under load (along with higher packet losses, and lots of other problems.
It's what I've been blogging about on http://gettys.wordpress.com. These buffers fill; and they are so large they have defeated TCP congestion avoidance to boot, with horrifying consequences.
So far, I've found this problem (almost) everywhere I've looked: o ICSI has good data that bufferbloat is endemic in DSL, Cable,
and FIOS. Delays are often measured in seconds (rather than milliseconds). o some corporate and ISP networks run without AQM, in circumstances that they should. o Windows, Mac OSX and Linux all have bufferbloat in their network stacks, at a minimum on recent network device drivers, and often elsewhere. o Every home router I've tested is horrifyingly bad. o 3g networks & 802.11 have this in spades.
Why should AT&T's MPLS be any different?
My next topic will be "transient" bufferbloat, having to do with defeating slowstart.
Come start helping fix this: please join us at bufferbloat.net, as we try to get people to fix it. Already there are some experimental patches for the Linux Intel wireless driver. Jim Gettys Bell Labs