DSL and/or Routing Problems

Greetings NANOGers, Yesterday we starting noticing long delays on an ADSL connection. I spent most of the day trying to track down the problem and getting no where. Telco says they do not detect any problem on the line... so I am kind of lost. Anyone here have any ideas? Here are the specifics: This connection uses a Cisco 827 ADSL router and has several static IPs. All IPs show identical delays. Using other circuits between the same two locations, we do not see any delays. Normally on this DSL connection, local can ping remote with packet transit times around 60-70ms. Here is what we are seeing now: # ping -s SOMEHOST 68 25; sleep 1; ping -s SOMEHOST 68 25 PING SOMEHOST: 68 data bytes 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=0. time=105. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=1. time=9132. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=2. time=8132. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=3. time=7132. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=4. time=6132. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=5. time=5133. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=6. time=4133. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=7. time=3133. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=8. time=2133. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=9. time=1133. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=10. time=133. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=11. time=104. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=12. time=110. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=13. time=109. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=14. time=112. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=15. time=106. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=16. time=114. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=17. time=107. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=18. time=109. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=19. time=106. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=20. time=112. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=21. time=106. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=22. time=108. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=23. time=106. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=24. time=110. ms ----SOMEHOST PING Statistics---- 25 packets transmitted, 25 packets received, 0% packet loss round-trip (ms) min/avg/max = 104/1918/9132 PING SOMEHOST: 68 data bytes 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=0. time=112. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=1. time=9131. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=2. time=8132. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=3. time=7132. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=4. time=6132. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=5. time=5132. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=6. time=4133. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=7. time=3132. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=8. time=2133. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=9. time=1133. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=10. time=133. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=11. time=111. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=12. time=106. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=13. time=109. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=14. time=116. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=15. time=108. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=16. time=107. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=17. time=113. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=18. time=106. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=19. time=107. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=20. time=108. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=21. time=108. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=22. time=105. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=23. time=109. ms 76 bytes from SOMEHOST (w.x.y.z): icmp_seq=24. time=106. ms ----SOMEHOST PING Statistics---- 25 packets transmitted, 25 packets received, 0% packet loss round-trip (ms) min/avg/max = 105/1918/9131 What really has me bugged is the pattern shown by the first dozen packets... why the relatively quick first time, followed by a long but decreasing delay which repeats every time you restart the ping (that's why I provided 2 samples)? Despite the fact that Telco says there are not any line problems, we are seeing a change in DSL performance compared to our benchmark. When we first started noticing the problem yesterday, both in and out connections were using the Fast path, but compared to the benchmark, the inbound speed had dropped to 576 and the Capacity had jumped to 99%, plus we had some RS and CRC errors on both in and out connections. Later in the day, the connection switched from using the Fast path to the Interleave path (we did nothing on our end to cause this to change) and the performance settled down to what is shown below under "DSL NOW." DSL BENCHMARK: ============== ATU-R (DS) ATU-C (US) Capacity Used: 72% 21% Interleave Fast Interleave Fast Speed (kbps): 0 960 0 256 Reed-Solomon EC: 0 0 0 0 CRC Errors: 0 0 0 0 Header Errors: 0 0 0 0 Bit Errors: 0 0 BER Valid sec: 0 0 BER Invalid sec: 0 0 DSL NOW: ======== ATU-R (DS) ATU-C (US) Capacity Used: 94% 63% Interleave Fast Interleave Fast Speed (kbps): 736 0 256 0 Reed-Solomon EC: 99 0 4 0 CRC Errors: 4 0 1 0 Header Errors: 3 0 0 0 Bit Errors: 0 0 BER Valid sec: 0 0 BER Invalid sec: 0 0 This leaves me with a few questions... 1) What exactly is the difference between the Fast and Interleave paths? 2) At the same data rates, should both Fast and Interleave show similar throughput? 3) What would cause a switch from Fast to Interleave to occur? 4) If the Telco cannot find any line problems, what can account for the increased capacity use, the decreased data rates, and the data errors? And the bottom line is: Why the bizarre ping times, and could the changes observed in DSL performance account for that... plus the near doubling of average ping transit times? TIA for everyone's expert thoughts on this problem! Jon R. Kibler A.S.E.T., Inc. Charleston, SC USA (843) 849-8214 ================================================== Filtered by: TRUSTEM.COM's Email Filtering Service http://www.trustem.com/ No Spam. No Viruses. Just Good Clean Email.