Subject: IP over SONET considered harmful? Perhaps. I am concerned about the growing movement towards IP over SONET. Previously in my career I was a vocal advocate of IP over ATM for several reasons, primarily traffic engineering and statistical gathering ability (obvdisclaimer, this required an autonomous unshared network used only by the ip provider for interhub traffic). However, I am firmly rooted in the bandwagon advocating IP OVER SONET FOR EVERYONE. Firmly. Accordingly, I am concerned about the visible L3 hop inherent to packets transiting routers. An ATM core is, of course, invisible to L3; so the number of switches or hubs through which a packet travels is inconsequential to the TTL of the packet. When a backbone is constructed with a PACKET over SONET core, the packet is likely to decrement the TTL by 2 at every hop. The number 2 is assumed because you are likely to leave from a router different than the one you come in. Since I tend to think in formulas, I'll encourage you to do so as well. Variable Meaning -------------- --------------- ROUTER L3 device which decrements the ttl of an IP packet TRANSIT_HUBS The number of hubs which neither sources nor delivers the packet NONCOREROUTERS The number of routers which accept or deliver traffic to a peer or customer TRANSIT_ROUTER A router which transits the packet TTL_DECREMENTS The number of ttl counters which this network decrements Assuming an architecture with dual core routers and two layers of hierarchy (backbone v. customer aggregation/peering), I believe the following formulae dictate the TTL degredation expected: ATM NETWORK: ----------- TTL_DECREMENTS == (NONCOREROUTERS + TRANSIT_ROUTERS) * 2 IP NETWORK: ---------- TTL_DECREMENTS == (NONCOREROUTERS + TRANSIT_ROUTERS) * 2 + TRANSIT_HUBS * 2 Another assertion I would make is that a 'responsible' NSP should decrement no more than 1/4 of the TTLs in the least common denominator. This follows from a general assumption of 2 NSPs, and 2 Customers; hence 4 entities. I consider Windows 95 to be the least common denominator, which has a default IP TTL of 32. Yes, 32. So that implies that each NSP should decrement no less than 8 TTLs. Solving IP NETWORK for TTL_DECREMENTS=8 implies that a network can have a diameter of no more than 4 hubs. That's a pretty meshed network when you have more than a few hubs. Does anyone have any strong opinions or sources on this matter to alleve my fears? The only solution I see is to fix mswindows; but of course that is quite difficult. I'd hoped that MPLS would solve this problem, but from reviewing the drafts I believe that the LSRs _WILL_ decrement the TTL. Your comments appreciated. -alan
Alan,
Subject: IP over SONET considered harmful?
[clipped...]
I'd hoped that MPLS would solve this problem, but from reviewing the drafts I believe that the LSRs _WILL_ decrement the TTL.
To be more precise, the issue is that an ingress LSR is required to copy IP TTL into Tag-TTL, *and* the egress LSR is *required* to copy Tag-TTL into IP TTL. The problem you mentioned in your message would be solved if the egress LSR would just decrement IP TTL by 1, rather than copying Tag-TTL into IP TTL. However, doing this introduces another problem - it breaks traceroute. And there are enough folks in the MPLS WG who think that the ability to traceroute through all the LSRs is an "unalienated right". In view of the above here are some of the possible avenues: (a) try to get "rough consensus" with the MPLS WG to allow decrement IP TTL by 1 on egress (rather than copy Tag TTL into IP TTL), or (b) talk to your favorite vendor(s), and ask the vendor(s) to put a "knob" that would decrement IP TTL by 1 on egress (rather than copying Tag TTL into IP TTL). Yakov.
Yakov Rekhter writes:
To be more precise, the issue is that an ingress LSR is required to copy IP TTL into Tag-TTL, *and* the egress LSR is *required* to copy Tag-TTL into IP TTL. The problem you mentioned in your message would be solved if the egress LSR would just decrement IP TTL by 1, rather than copying Tag-TTL into IP TTL. However, doing this introduces another problem - it breaks traceroute. And there are enough folks in the MPLS WG who think that the ability to traceroute through all the LSRs is an "unalienated right".
And somehow it is different that ATM and frame relay also "break" traceroute just as much, if by "break" it is meant that one cannot see the "physical" (not they they are really seeing that) topology?
In view of the above here are some of the possible avenues:
(a) try to get "rough consensus" with the MPLS WG to allow decrement IP TTL by 1 on egress (rather than copy Tag TTL into IP TTL), or
It would be nice were that an option at least.
(b) talk to your favorite vendor(s), and ask the vendor(s) to put a "knob" that would decrement IP TTL by 1 on egress (rather than copying Tag TTL into IP TTL).
Quite.
copying Tag-TTL into IP TTL. However, doing this introduces another problem - it breaks traceroute. And there are enough folks in the MPLS WG who think that the ability to traceroute through all the LSRs is an "unalienated right".
Without going to far into the traceroute source , would a solution to this problem be a modified traceroute that "understood" and could report Tag/MPLS "hops" as well as the normal method? There would need to be some way of identifying which devices were forwarding using Tag/Mpls and which were not, as well as getting and reporting the information. I think it falls into the same category of tracing cell forwarding in ATM without having access to the equipment doing the work.
In view of the above here are some of the possible avenues:
(a) try to get "rough consensus" with the MPLS WG to allow decrement IP TTL by 1 on egress (rather than copy Tag TTL into IP TTL), or
(b) talk to your favorite vendor(s), and ask the vendor(s) to put a "knob" that would decrement IP TTL by 1 on egress (rather than copying Tag TTL into IP TTL).
Alan and I discussed the "knob" solution before, it seems to solve the problem, but under certain circumstances a packet could end up being forwarded forever because of (mis)configuration. -pee
Paul E. Erkkila writes:
Without going to far into the traceroute source , would a solution to this problem be a modified traceroute that "understood" and could report Tag/MPLS "hops" as well as the normal method? There would need to be some way of identifying which devices were forwarding using Tag/Mpls and which were not, as well as getting and reporting the information. I think it falls into the same category of tracing cell forwarding in ATM without having access to the equipment doing the work.
Without getting into whether this is a good idea - I suspect many service providers would disable such a capability - one of the nice aspects of traceroute is that it uses the usual packet forwarding path until the last hop. In other words, it's a little hard to break the normal packet forwarding mechanism without breaking traceroute too. This charcteristic would be hard to preserve with such a solution.
Alan and I discussed the "knob" solution before, it seems to solve the problem, but under certain circumstances a packet could end up being forwarded forever because of (mis)configuration.
You do have to make sure that tunnels always move packets closer to their destination, yes.
At 09:33 AM 3/20/98 PST, Yakov Rekhter wrote:
To be more precise, the issue is that an ingress LSR is required to copy IP TTL into Tag-TTL, *and* the egress LSR is *required* to copy Tag-TTL into IP TTL. The problem you mentioned in your message would be solved if the egress LSR would just decrement IP TTL by 1, rather than copying Tag-TTL into IP TTL. However, doing this introduces another problem - it breaks traceroute. And there are enough folks in the MPLS WG who think that the ability to traceroute through all the LSRs is an "unalienated right".
I won't launch into a dissertation on this topic, but the issue of decrementing the IP TTL by 1 at the egress LSR/TSR has been discussed to death (and I argued in favor of a knob to both allow and disallow this on the MPLS list until I was blue in the face) with no clear consensus, IMO. I personally believe that this issue needs to be revisited within the MPLS wg. Not decrementing the IP TTL at each LSR hop no more breaks the IP TTL mechanisms than does frame-relay or ATM. $.02, - paul ps. In fact, there may even be some ISP's that would prefer that their internal L2 infrastructure remain invisible.
Not decrementing the IP TTL at each LSR hop no more breaks the IP TTL mechanisms than does frame-relay or ATM.
It should be the business of ISPs to hide or not to hide the lower layer infrastructures. Unless someone really wants to use 'traceroute' to see the DACs, ADMs, verilinks, DL3100s, etc. A more flexible 'knob' would be cool, something like combining the 'cos' bits in the labels with the decision to decremening or not. This will give the management station the power to 'trace' even the regular trace packet can't do.
- paul
ps. In fact, there may even be some ISP's that would prefer that their internal L2 infrastructure remain invisible.
------------------------------------------------------------------- - Naiming Shen MCI - MCI Internet Engineering 2100 Reston Parkway - +1 703-715-7056 fax:703-715-7066 v272-7056 Reston, VA 20191
Yes. For the very reason you have identified of TTL decrement on hop to hop links on IP over SONET we are in the process of building an optical Internet where we will use WDM cut through (and maybe SONET label switching) to provide the best performace possible. With WDM cut through there are absolutely NO switch or router latencies. For more information please see the CANARIE web site at http://www.canarie.ca Bill Bill St. Arnaud Director Network Projects CANARIE bill.st.arnaud@canarie.ca http://www.canarie.ca/bstarn +1 613 660-3497
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of Alan Hannan Sent: Friday, March 20, 1998 2:26 AM To: nanog@merit.edu Subject: IP over SONET considered harmful?
Subject: IP over SONET considered harmful?
Perhaps.
I am concerned about the growing movement towards IP over SONET.
Previously in my career I was a vocal advocate of IP over ATM for several reasons, primarily traffic engineering and statistical gathering ability (obvdisclaimer, this required an autonomous unshared network used only by the ip provider for interhub traffic).
However, I am firmly rooted in the bandwagon advocating IP OVER SONET FOR EVERYONE. Firmly.
Accordingly, I am concerned about the visible L3 hop inherent to packets transiting routers.
An ATM core is, of course, invisible to L3; so the number of switches or hubs through which a packet travels is inconsequential to the TTL of the packet.
When a backbone is constructed with a PACKET over SONET core, the packet is likely to decrement the TTL by 2 at every hop. The number 2 is assumed because you are likely to leave from a router different than the one you come in.
Since I tend to think in formulas, I'll encourage you to do so as well.
Variable Meaning -------------- --------------- ROUTER L3 device which decrements the ttl of an IP packet
TRANSIT_HUBS The number of hubs which neither sources nor delivers the packet
NONCOREROUTERS The number of routers which accept or deliver traffic to a peer or customer
TRANSIT_ROUTER A router which transits the packet
TTL_DECREMENTS The number of ttl counters which this network decrements
Assuming an architecture with dual core routers and two layers of hierarchy (backbone v. customer aggregation/peering), I believe the following formulae dictate the TTL degredation expected:
ATM NETWORK: -----------
TTL_DECREMENTS == (NONCOREROUTERS + TRANSIT_ROUTERS) * 2
IP NETWORK: ----------
TTL_DECREMENTS == (NONCOREROUTERS + TRANSIT_ROUTERS) * 2 + TRANSIT_HUBS * 2
Another assertion I would make is that a 'responsible' NSP should decrement no more than 1/4 of the TTLs in the least common denominator. This follows from a general assumption of 2 NSPs, and 2 Customers; hence 4 entities.
I consider Windows 95 to be the least common denominator, which has a default IP TTL of 32. Yes, 32. So that implies that each NSP should decrement no less than 8 TTLs.
Solving IP NETWORK for TTL_DECREMENTS=8 implies that a network can have a diameter of no more than 4 hubs. That's a pretty meshed network when you have more than a few hubs.
Does anyone have any strong opinions or sources on this matter to alleve my fears?
The only solution I see is to fix mswindows; but of course that is quite difficult.
I'd hoped that MPLS would solve this problem, but from reviewing the drafts I believe that the LSRs _WILL_ decrement the TTL.
Your comments appreciated.
-alan
participants (7)
-
Alan Hannan -
Bill St. Arnaud -
Joseph Malcolm -
Naiming Shen -
Paul E. Erkkila -
Paul Ferguson -
Yakov Rekhter