I was wondering if the wonderful members of this list could provide their opinions regarding the traceroute below. I have contacted sprint several times regarding this issue and their noc keeps coming back with "no trouble found". Am I barking up the wrong up the wrong tree? I'm experiencing packetloss and large amounts of delay at hop 8. Both of my T providers say they cant do anything to help me. C:\>tracert 67.97.251.18 Tracing route to 67.97.251.18 over a maximum of 30 hops 1 <10 ms <10 ms <10 ms 192.168.1.75 2 16 ms 15 ms 16 ms host-64-179-34-81.pit.choiceone.net [64.179.34.8 1] 3 15 ms 16 ms 15 ms atm6-5-992-pitb-isp-cisco.choiceone.net [64.65.2 10.101] 4 31 ms 31 ms 16 ms chi-edge-09.inet.qwest.net [63.149.3.229] 5 31 ms 16 ms 31 ms chi-core-03.inet.qwest.net [205.171.20.125] 6 15 ms 32 ms 31 ms chi-brdr-03.inet.qwest.net [205.171.20.142] 7 16 ms 31 ms 31 ms 205.171.1.162 8 297 ms 375 ms 94 ms sl-bb20-chi-4-0.sprintlink.net [144.232.8.219] 9 47 ms 47 ms 47 ms sl-browing-20-0.sprintlink.net [144.223.241.22] 10 47 ms 46 ms 63 ms ge-2-1-0.a1.chcg.broadwing.net [216.140.15.17] 11 47 ms 63 ms 47 ms so-0-1-0.c1.gnwd.broadwing.net [216.140.14.97] 12 47 ms 62 ms 47 ms so-2-0-0.c1.wash.broadwing.net [216.140.16.22] 13 47 ms 62 ms 63 ms p0-0-0.a1.wash.broadwing.net [216.140.8.13] 14 47 ms 63 ms 62 ms p8-0-0.e0.wash.broadwing.net [216.140.8.34] 15 141 ms 125 ms 141 ms 67.97.250.166 16 * * * Request timed out. 17 * * * Request timed out. 18 * * * Request timed out. 19 * * * Request timed out. 20 * * * Request timed out. 21 * * * Request timed out. 22 * * * Request timed out. 23 * * * Request timed out. 24 * * * Request timed out. 25 * * * Request timed out. 26 * * * Request timed out. 27 * * * Request timed out. 28 * * * Request timed out. 29 * * * Request timed out. 30 * * * Request timed out. Trace complete. C:\> Joe Marr ( <mailto:joe.marr@intektechnologies.com> joe.marr@intektechnologies.com) Sr. Technology Consultant Intek Technologies Phone: 1.570.322.1915 www.intektechnologies.com
Joe, You are, in fact, barking up the wrong tree. The question should be ³when I test against my destination, am I seeing packet loss and latency?². Traceroute is not a useful tool for determining the state of an intermediate router on a path. ICMP TTL-Exceeded messages are rate limited and not handled as expeditiously as forwarded packets going THROUGH a modern router. Why? Because core routers have limited CPU, control bus, and other resources. Traceroute is (but is not always) useful for: * Determining which routers and carriers your packets are passing through * Figuring out where there is congestion on a path (this is iffy, doesn't always work, and assumes that there is actually congestion, not other things going on). Not the tool of choice for this, but sometimes the only thing you've got. Remember, if you really had packet loss or increased latency on the 8th hop, it would show up on all the other, upstream hops, and the destination. -- Daniel Golding Network and Telecommunications Strategies Burton Group On 3/18/04 11:33 AM, "Joe Marr" <joe.marr@intektechnologies.com> wrote:
I was wondering if the wonderful members of this list could provide their opinions regarding the traceroute below. I have contacted sprint several times regarding this issue and their noc keeps coming back with ³no trouble found². Am I barking up the wrong up the wrong tree?
I¹m experiencing packetloss and large amounts of delay at hop 8. Both of my T providers say they cant do anything to help me.
C:\>tracert 67.97.251.18
Tracing route to 67.97.251.18 over a maximum of 30 hops
1 <10 ms <10 ms <10 ms 192.168.1.75 2 16 ms 15 ms 16 ms host-64-179-34-81.pit.choiceone.net [64.179.34.8 1] 3 15 ms 16 ms 15 ms atm6-5-992-pitb-isp-cisco.choiceone.net [64.65.2 10.101] 4 31 ms 31 ms 16 ms chi-edge-09.inet.qwest.net [63.149.3.229] 5 31 ms 16 ms 31 ms chi-core-03.inet.qwest.net [205.171.20.125] 6 15 ms 32 ms 31 ms chi-brdr-03.inet.qwest.net [205.171.20.142] 7 16 ms 31 ms 31 ms 205.171.1.162 8 297 ms 375 ms 94 ms sl-bb20-chi-4-0.sprintlink.net [144.232.8.219] 9 47 ms 47 ms 47 ms sl-browing-20-0.sprintlink.net [144.223.241.22]
10 47 ms 46 ms 63 ms ge-2-1-0.a1.chcg.broadwing.net [216.140.15.17] 11 47 ms 63 ms 47 ms so-0-1-0.c1.gnwd.broadwing.net [216.140.14.97] 12 47 ms 62 ms 47 ms so-2-0-0.c1.wash.broadwing.net [216.140.16.22] 13 47 ms 62 ms 63 ms p0-0-0.a1.wash.broadwing.net [216.140.8.13] 14 47 ms 63 ms 62 ms p8-0-0.e0.wash.broadwing.net [216.140.8.34] 15 141 ms 125 ms 141 ms 67.97.250.166 16 * * * Request timed out. 17 * * * Request timed out. 18 * * * Request timed out. 19 * * * Request timed out. 20 * * * Request timed out. 21 * * * Request timed out. 22 * * * Request timed out. 23 * * * Request timed out. 24 * * * Request timed out. 25 * * * Request timed out. 26 * * * Request timed out. 27 * * * Request timed out. 28 * * * Request timed out. 29 * * * Request timed out. 30 * * * Request timed out.
Trace complete.
C:\> Joe Marr (joe.marr@intektechnologies.com <mailto:joe.marr@intektechnologies.com> ) Sr. Technology Consultant Intek Technologies
Phone: 1.570.322.1915 www.intektechnologies.com <http://www.intektechnologies.com>
I should have provided more information. :) I have a customer with a large citrix farm at hop 15. The customer has several remote offices, one as far away as Japan. One of their offices has been experiencing slow performance with their citrix connections. It's not an issue of congestion on either end of the link, so the only thing I can find wrong is packetloss and latency between hops 7 and 8. Granted the traceroute I sent doesn't show packet loss now, but it has in the past. I understand what your saying, but how do I go about tracking down this issue? None of their other offices have this problem. Thanks for your input. Joe urish_popeck#traceroute urish-pitts-gw.centrepc.net Type escape sequence to abort. Tracing the route to urish-pitts-gw.centrepc.net (67.97.250.166) 1 host-64-179-34-81.pit.choiceone.net (64.179.34.81) 16 msec 16 msec 16 msec 2 atm6-5-992-pitb-isp-cisco.choiceone.net (64.65.210.101) 16 msec 16 msec 16 msec 3 chi-edge-09.inet.qwest.net (63.149.3.229) 28 msec 28 msec 28 msec 4 chi-core-03.inet.qwest.net (205.171.20.125) 28 msec 32 msec 28 msec 5 chi-brdr-03.inet.qwest.net (205.171.20.142) 28 msec 28 msec 24 msec 6 205.171.1.162 28 msec 32 msec 28 msec 7 sl-bb20-chi-4-0.sprintlink.net (144.232.8.219) 60 msec 60 msec 56 msec 8 sl-browing-20-0.sprintlink.net (144.223.241.22) 56 msec 56 msec 60 msec 9 ge-2-1-0.a1.chcg.broadwing.net (216.140.15.17) 60 msec * 176 msec 10 so-0-1-0.c1.gnwd.broadwing.net (216.140.14.97) 284 msec 404 msec 92 msec 11 so-2-0-0.c1.wash.broadwing.net (216.140.16.22) 60 msec 64 msec 64 msec 12 p4-1.a0.wash.broadwing.net (216.140.8.94) 64 msec 64 msec 64 msec 13 p9-0-0.e0.wash.broadwing.net (216.140.8.18) 68 msec 68 msec 64 msec 14 urish-pitts-gw.centrepc.net (67.97.250.166) 100 msec * 100 msec urish_popeck#traceroute urish-pitts-gw.centrepc.net Type escape sequence to abort. Tracing the route to urish-pitts-gw.centrepc.net (67.97.250.166) 1 host-64-179-34-81.pit.choiceone.net (64.179.34.81) 20 msec 16 msec 24 msec 2 atm6-5-992-pitb-isp-cisco.choiceone.net (64.65.210.101) 12 msec 16 msec 16 msec 3 chi-edge-09.inet.qwest.net (63.149.3.229) 28 msec 28 msec 28 msec 4 chi-core-03.inet.qwest.net (205.171.20.125) 28 msec 28 msec 28 msec 5 chi-brdr-03.inet.qwest.net (205.171.20.142) 24 msec 28 msec 28 msec 6 205.171.1.162 28 msec 36 msec 32 msec 7 sl-bb20-chi-4-0.sprintlink.net (144.232.8.219) 216 msec 212 msec 60 msec 8 sl-browing-20-0.sprintlink.net (144.223.241.22) 144 msec * 60 msec 9 ge-2-1-0.a1.chcg.broadwing.net (216.140.15.17) 60 msec 185 msec 56 msec 10 so-0-1-0.c1.gnwd.broadwing.net (216.140.14.97) 60 msec * 64 msec 11 so-2-0-0.c1.wash.broadwing.net (216.140.16.22) 64 msec 64 msec * 12 p4-1.a0.wash.broadwing.net (216.140.8.94) 64 msec 64 msec 60 msec 13 p9-0-0.e0.wash.broadwing.net (216.140.8.18) 64 msec 64 msec 72 msec 14 urish-pitts-gw.centrepc.net (67.97.250.166) 152 msec * 140 msec urish_popeck#ping Protocol [ip]: Target IP address: 67.97.250.166 Repeat count [5]: 1000 Datagram size [100]: Timeout in seconds [2]: Extended commands [n]: Sweep range of sizes [n]: Type escape sequence to abort. Sending 1000, 100-byte ICMP Echos to 67.97.250.166, timeout is 2 seconds: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!! Success rate is 99 percent (998/1000), round-trip min/avg/max = 104/109/376 ms urish_popeck#ping Protocol [ip]: Target IP address: 144.232.8.219 Repeat count [5]: Datagram size [100]: Timeout in seconds [2]: Extended commands [n]: Sweep range of sizes [n]: Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 144.232.8.219, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 60/61/64 ms urish_popeck#ping Protocol [ip]: Target IP address: 144.232.8.219 Repeat count [5]: 1000 Datagram size [100]: Timeout in seconds [2]: Extended commands [n]: Sweep range of sizes [n]: Type escape sequence to abort. Sending 1000, 100-byte ICMP Echos to 144.232.8.219, timeout is 2 seconds: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!! Success rate is 98 percent (997/1000), round-trip min/avg/max = 60/61/416 ms Joe Marr -----Original Message----- From: Daniel Golding [mailto:dgolding@burtongroup.com] Sent: Thursday, March 18, 2004 12:00 PM To: joe.marr@intektechnologies.com; nanog@merit.edu Subject: Re: Sprint Help Joe, You are, in fact, barking up the wrong tree. The question should be ³when I test against my destination, am I seeing packet loss and latency?². Traceroute is not a useful tool for determining the state of an intermediate router on a path. ICMP TTL-Exceeded messages are rate limited and not handled as expeditiously as forwarded packets going THROUGH a modern router. Why? Because core routers have limited CPU, control bus, and other resources. Traceroute is (but is not always) useful for: * Determining which routers and carriers your packets are passing through * Figuring out where there is congestion on a path (this is iffy, doesn't always work, and assumes that there is actually congestion, not other things going on). Not the tool of choice for this, but sometimes the only thing you've got. Remember, if you really had packet loss or increased latency on the 8th hop, it would show up on all the other, upstream hops, and the destination. -- Daniel Golding Network and Telecommunications Strategies Burton Group On 3/18/04 11:33 AM, "Joe Marr" <joe.marr@intektechnologies.com> wrote:
I was wondering if the wonderful members of this list could provide their opinions regarding the traceroute below. I have contacted sprint several times regarding this issue and their noc keeps coming back with ³no trouble found². Am I barking up the wrong up the wrong tree?
I¹m experiencing packetloss and large amounts of delay at hop 8. Both of my T providers say they cant do anything to help me.
C:\>tracert 67.97.251.18
Tracing route to 67.97.251.18 over a maximum of 30 hops
1 <10 ms <10 ms <10 ms 192.168.1.75 2 16 ms 15 ms 16 ms host-64-179-34-81.pit.choiceone.net [64.179.34.8 1] 3 15 ms 16 ms 15 ms atm6-5-992-pitb-isp-cisco.choiceone.net [64.65.2 10.101] 4 31 ms 31 ms 16 ms chi-edge-09.inet.qwest.net [63.149.3.229] 5 31 ms 16 ms 31 ms chi-core-03.inet.qwest.net [205.171.20.125] 6 15 ms 32 ms 31 ms chi-brdr-03.inet.qwest.net [205.171.20.142] 7 16 ms 31 ms 31 ms 205.171.1.162 8 297 ms 375 ms 94 ms sl-bb20-chi-4-0.sprintlink.net [144.232.8.219] 9 47 ms 47 ms 47 ms sl-browing-20-0.sprintlink.net [144.223.241.22]
10 47 ms 46 ms 63 ms ge-2-1-0.a1.chcg.broadwing.net [216.140.15.17] 11 47 ms 63 ms 47 ms so-0-1-0.c1.gnwd.broadwing.net [216.140.14.97] 12 47 ms 62 ms 47 ms so-2-0-0.c1.wash.broadwing.net [216.140.16.22] 13 47 ms 62 ms 63 ms p0-0-0.a1.wash.broadwing.net [216.140.8.13] 14 47 ms 63 ms 62 ms p8-0-0.e0.wash.broadwing.net [216.140.8.34] 15 141 ms 125 ms 141 ms 67.97.250.166 16 * * * Request timed out. 17 * * * Request timed out. 18 * * * Request timed out. 19 * * * Request timed out. 20 * * * Request timed out. 21 * * * Request timed out. 22 * * * Request timed out. 23 * * * Request timed out. 24 * * * Request timed out. 25 * * * Request timed out. 26 * * * Request timed out. 27 * * * Request timed out. 28 * * * Request timed out. 29 * * * Request timed out. 30 * * * Request timed out.
Trace complete.
C:\> Joe Marr (joe.marr@intektechnologies.com <mailto:joe.marr@intektechnologies.com> ) Sr. Technology Consultant Intek Technologies
Phone: 1.570.322.1915 www.intektechnologies.com <http://www.intektechnologies.com>
In a message written on Thu, Mar 18, 2004 at 12:15:19PM -0500, Joe Marr wrote:
I have a customer with a large citrix farm at hop 15. The customer has several remote offices, one as far away as Japan. One of their offices has been experiencing slow performance with their citrix connections. It's not
Your problem is right here:
14 urish-pitts-gw.centrepc.net (67.97.250.166) 100 msec * 100 msec
NT in many configs defaults to a TCP Window size of 8k, other NT and Server 2000 default to 16k. At one window per RTT that's 80k/sec or 160k/sec maximum throughput over a 100ms link. 80k/sec is not fast enough to make Citrix (which must send large portions of the screen) usable. HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\TCPWindowSize Note, bigger is not better, if you go make it 64k you're likely to just crash your machine. Google for "TCP Window Size" and read one of the 50,000 papers available before you tweek the value. More to the point: This is becoming an acute problem in ISP support (at least, if you offer more than dial speeds). We get several queries per week from customers who put boxes in a New York and LA colo on GigE, and then can't get more than a 200k/sec FTP and want to complain about the backbone. Reality is they can get the full 10000k/sec if they just fix their boxes. It's purely an end host issue. And yes, it is a Windows issue. Commercial and Free unix systems are generally all on 32k default windows, with a few at 64k default windows. These still need to be tuned for really high speed links, but don't make people complain. Microsoft waited too long to up these values (XP is 32k) so the masses of older servers show "slow" performance for no good reason. They really should update with a service pack. Cable/DSL providers who have software that tweeks users computer settings (which I have issues with, but if you do it...) should think about tweeking the value up for 98/NT/2000 users as part of the default software install. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ Read TMBG List - tmbg-list-request@tmbg.org, www.tmbg.org
participants (3)
-
Daniel Golding -
Joe Marr -
Leo Bicknell