OK, here's a wild guess from left-field. Well, at least from left-field where I made at least one game-saving catch :) We had a similar case some years back, but it was a ramp-up in overall traffic we were looking at. If you're looking at latency, it could be related to traffic (do you have traffic graphs?). One particular user that was accustomed to Windows and trying to get started with Linux was "playing games" with our NAT firewall. Rather than file a request with us for a static NAT and firewall openings for their "new" Linux server, they discovered that as long as they generated some internet traffic periodically, they could defeat the NAT translation timeout, and essentially keep a static outside IP. Problem was, they "crontab"ed a "ping" of an outside server to run once a minute. Just a "ping x.x.x.x". Windows as we know defaults to only ping 4 times then quit. Linux does not :) So you might look for some "recurring scheduled event" on the customer's end that might be cumulative rather than simply recurring. Jeff On 5/31/2013 6:25 PM, Mike wrote:
Gang,
In the interest of sharing 'the weird stuff' which makes the job of being an operator ... uh, fun? is that the right word?..., I would like to present the following two smokeping latency/packetloss plots, which are by far the weirdest I have ever seen.
These plots are from our smokeping host out to a customer location. The customer is connected via DSL and they run PPPoE over it to connect with our access concentrator. There is about 5 physical insfastructure hops between the host and customer; The switch, the BRAS, the Switch again, and then directly to the DSLAM and then customer on the end.
The 10 day plot: http://picpaste.com/10_Day_graph-YV3IdvRV.png
The 30 hour plot: http://picpaste.com/30_hour_graph-DrwzfhYJ.png
How can you possibly have consistent increase in latency like that? I'd love to hear theories (or offers of beer, your choice!).
Happy friday all!
Mike-