Subject: IP over SONET considered harmful? Perhaps. I am concerned about the growing movement towards IP over SONET. Previously in my career I was a vocal advocate of IP over ATM for several reasons, primarily traffic engineering and statistical gathering ability (obvdisclaimer, this required an autonomous unshared network used only by the ip provider for interhub traffic). However, I am firmly rooted in the bandwagon advocating IP OVER SONET FOR EVERYONE. Firmly. Accordingly, I am concerned about the visible L3 hop inherent to packets transiting routers. An ATM core is, of course, invisible to L3; so the number of switches or hubs through which a packet travels is inconsequential to the TTL of the packet. When a backbone is constructed with a PACKET over SONET core, the packet is likely to decrement the TTL by 2 at every hop. The number 2 is assumed because you are likely to leave from a router different than the one you come in. Since I tend to think in formulas, I'll encourage you to do so as well. Variable Meaning -------------- --------------- ROUTER L3 device which decrements the ttl of an IP packet TRANSIT_HUBS The number of hubs which neither sources nor delivers the packet NONCOREROUTERS The number of routers which accept or deliver traffic to a peer or customer TRANSIT_ROUTER A router which transits the packet TTL_DECREMENTS The number of ttl counters which this network decrements Assuming an architecture with dual core routers and two layers of hierarchy (backbone v. customer aggregation/peering), I believe the following formulae dictate the TTL degredation expected: ATM NETWORK: ----------- TTL_DECREMENTS == (NONCOREROUTERS + TRANSIT_ROUTERS) * 2 IP NETWORK: ---------- TTL_DECREMENTS == (NONCOREROUTERS + TRANSIT_ROUTERS) * 2 + TRANSIT_HUBS * 2 Another assertion I would make is that a 'responsible' NSP should decrement no more than 1/4 of the TTLs in the least common denominator. This follows from a general assumption of 2 NSPs, and 2 Customers; hence 4 entities. I consider Windows 95 to be the least common denominator, which has a default IP TTL of 32. Yes, 32. So that implies that each NSP should decrement no less than 8 TTLs. Solving IP NETWORK for TTL_DECREMENTS=8 implies that a network can have a diameter of no more than 4 hubs. That's a pretty meshed network when you have more than a few hubs. Does anyone have any strong opinions or sources on this matter to alleve my fears? The only solution I see is to fix mswindows; but of course that is quite difficult. I'd hoped that MPLS would solve this problem, but from reviewing the drafts I believe that the LSRs _WILL_ decrement the TTL. Your comments appreciated. -alan