Please see inline.
-----Original Message----- From: John Fraizer [mailto:nanog@enterzone.net] Sent: Monday, January 31, 2005 8:21 AM To: Charles Shen; nanog@merit.edu Subject: Re: Rapidly-variable routing on the time scale of seconds to minutes?
Charles Shen wrote:
An example traceroute record containing the fluttering is shown below (see the 5th hop)
Fri Apr 09 09:35:35 2004
1 cisfhfb.fh-friedberg.de (212.201.24.1) 1.095 ms 0.402 ms 0.321 ms 2 ar-frankfurt2.g-win.dfn.de (188.1.42.9) 120.105 ms 198.766 ms 200.040 ms 3 cr-frankfurt1-ge5-0.g-win.dfn.de (188.1.80.1) 2.093 ms 2.142 ms 2.087 ms 4 so-6-0-0.ar2.FRA2.gblx.net (208.48.23.141) 2.461 ms 2.349 ms 2.333 ms 5 pos5-0-2488M.cr2.FRA2.gblx.net (67.17.65.53) 2.448 ms pos6-0-2488M.cr1.FRA2.gblx.net (67.17.65.77) 2.368 ms 2.281 ms
That sure looks like ECM to me. Equal Cost Multi-Path. This is NOT anything new. What's the big deal?
From the responses, the answer to "the rapidly-variable routing on the time scale of seconds to minutes" seems to be:
1. It could be link layer load balancing, with the two interfaces belonging to the same router. 2. It could be per-flow load balancing where flows are defined via both L3 and L4 info, so traceroute probe could not reflect the truth. My question is then: would it be safe to argue that the above two causes explain all (or most of?) the observed "fluttering" routers? (some examples listed below) What we are concerned about is per-packet load balancing (packets in the same flow go through different paths), which will cause trouble to protocols that install state information in routers along the flow path. Example pairs: 144.223.27.146 sl-telia1-1-0.sprintlink.net 144.232.230.30 sl-telia1-4-0.sprintlink.net 216.140.0.66 s3-0-0.a1.hywr.broadwing.net 216.140.0.70 s4-0-0.a1.hywr.broadwing.net 67.17.65.53 pos5-0-2488M.cr2.FRA2.gblx.net 67.17.65.77 pos6-0-2488M.cr1.FRA2.gblx.net 67.17.65.54 so5-0-0-2488M.ar2.FRA2.gblx.net 67.17.65.78 so4-0-0-2488M.ar2.FRA2.gblx.net 67.17.65.57 pos11-0-2488M.cr2.FRA2.gblx.net 67.17.65.81 pos11-0-2488M.cr1.FRA2.gblx.net 67.17.65.58 so2-0-0-2488M.ar2.FRA3.gblx.net 67.17.65.82 so3-0-0-2488M.ar2.FRA3.gblx.net 67.17.64.66 pos6-0-2488M.cr2.SFO1.gblx.net 67.17.74.157 pos8-0-2488M.cr1.SFO1.gblx.net 129.250.2.183 p16-3-0-0.r01.snjsca04.us.bb.verio.net 129.250.5.136 p16-7-0-0.r00.snjsca04.us.bb.verio.net