Here's something to chew on. As an end-node site who has recently become triply-homed I've been wondering whether it would be useful if routers had a way of making a route selection based on output queue levels.
Not. That will cause reordering of packets, and so trigger false TCP retransmits.
Ie., let's say I have 2 paths to a destination of equal as-path distance through neighbor A and neighbor B. Based on recent discussion it sounds like IOS will send the packet to the neighbor with the lowest IP address.
Not. This is a default tie-breaking rule in cisco's BGP implementation, and doesn't have anything to do with packet forwarding. --vadim
On Sat, 14 Dec 1996, Vadim Antonov wrote:
Here's something to chew on. As an end-node site who has recently become triply-homed I've been wondering whether it would be useful if routers had a way of making a route selection based on output queue levels.
Not. That will cause reordering of packets, and so trigger false TCP retransmits.
It could be stream based (address and port), and use a LRU cache with a planned capacity of 5 minutes, to decide on which interface to use. Sure, it is more complicated, but anything you add increases complexity. Has anybody collected stream duration data? What does the distribution look like? 98% of streams last how long? (I bet less than 5 minutes) I think something like this was brought up at the Ann Arbor NANOG. Mike. +------------------- H U R R I C A N E - E L E C T R I C -------------------+ | Mike Leber Direct Internet Connections Voice 408 282 1540 | | Hurricane Electric Web Hosting & Co-location Fax 408 971 3340 | | mleber@he.net http://www.he.net | +---------------------------------------------------------------------------+
On Sat, 14 Dec 1996, Mike Leber wrote: |} Has anybody collected stream duration data? What does the distribution |} look like? 98% of streams last how long? (I bet less than 5 minutes) |} I think something like this was brought up at the Ann Arbor NANOG. What are you calling a 'stream' ? People are starting to collect flow data in quite a few places with some interesting results. -jh-
It could be stream based (address and port), and use a LRU cache with a planned capacity of 5 minutes, to decide on which interface to use. Sure, it is more complicated, but anything you add increases complexity.
not so. right now our multihomed web gateways need a full routing table. that's pretty complicated, for my operator and for my customer's providers. we're shortly moving to an "interface default" such that all inbound streams will have their outbound route chosen as the incoming interface's default, and all outbound streams will get a "round robin default" outbound route that will then be rebound to the SYN-ACK's incoming route, in case the original source route is better-reachable by "this" destination via some other interface, a not uncommon case these days. this adds about 2 pages of code to the kernel. but it's a lot less complex than taking a full routing table from multiple providers.
Sounds like this would make everything be symmetrically routed. Has anyone researched whether this can scale? Will you handle the case where the interface a flow uses changes during the course of the flow? I assume this will lead to "round robin default" until an inbound packet for the flow is received from a new interface. Will the "round robin default" favor uncongested or highest speed links? I can't wait to hear the results of doing this! Chris On Sun, 15 Dec 1996, Paul A Vixie wrote:
not so. right now our multihomed web gateways need a full routing table. that's pretty complicated, for my operator and for my customer's providers. we're shortly moving to an "interface default" such that all inbound streams will have their outbound route chosen as the incoming interface's default, and all outbound streams will get a "round robin default" outbound route that will then be rebound to the SYN-ACK's incoming route, in case the original source route is better-reachable by "this" destination via some other interface, a not uncommon case these days.
this adds about 2 pages of code to the kernel. but it's a lot less complex than taking a full routing table from multiple providers.
Sounds like this would make everything be symmetrically routed.
No. It only means the end system's first hop is symmetrical. The intermediate systems might packet-stuff on multiple paths, or choose a different path based on private data, or whatever. My goal is not end to end path symmetry -- hot potato routing makes that impossible in any case. My goal is that an end system should be able to accept connections via multiple providers without needing a routing table from each of those providers. (Which can't be made symmetrical even in the average case, anyway.)
Has anyone researched whether this can scale?
I havn't. I know that right now, someone who wants to multihome has to do way too much work and they cannot reliably send packets back on the same first-hop as the last-hop they came in on, which makes it pretty much worthless to multihome unless you break your BGP listener to use round robin or pseudo random path selection -- and when we did this it made some things worse rather than better.
Will you handle the case where the interface a flow uses changes during the course of the flow?
If the link (or ARP) for the current next-hop dies, the route rehomes among the remaining interface defaults and then rehomes again next time a segment is received on some remaining-up interface.
I assume this will lead to "round robin default" until an inbound packet for the flow is received from a new interface.
That's what I remember saying, yes.
Will the "round robin default" favor uncongested or highest speed links?
Not initially. My test environment has a pair of 10baseT links and the congestion is all after the first hop, where interface statistcs can't help with local route selection. I won't entertain the above question seriously unless our initial results hint that it will be time well spent.
I can't wait to hear the results of doing this!
I expect to have an informal report for this mailing list by January 10th.
participants (5)
-
Chris Caputo -
Jonathan Heiliger -
Mike Leber -
Paul A Vixie -
Vadim Antonov