A few comments:
I don't think flow-caching is necessarily due to CEF.
CEF, afaik, is unaware of flows.
Even on dinky 2500 & 2600 series where you don't run CEF,
Many people run CEF on 2600's, it's about the only way to get to the cisco-advertised PPS on the box.
load balancing over multiple links uses a flow-hashed method. If you want per-packet load distribution you have to specifically enable it by saying "no ip route-cache" on each interface.
That is very deadly, please, don't anyone actually try that. CEF load balancing, IIRC, had two options, specifyable on a per-interface basis -- 'per-packet', and 'per-destination'. Both have obvious meanings. Newer IOS's seem to have a defaulting mechanism available in global config mode, but being a weirdo, I don't trust it. I still specify on the per-interface. We use this in several scenerios, specifically for load-balancing T1's, and it amazingly works well, with the links often being in balance to the tune of 1 to 3%. I've seen similar performance at DS3 rates.
Paul's statement about CEF is interesting. It's probably the first public statement I've ever heard where someone was praising CEF. Usually discussions about CEF are accompanied by liberal amounts of swearing...
I dunno; except for some silliness in 12.1(8a)E[1-4] on a MSFC2, we've seen general goodness from CEF from 2600, 3600, 4700, 5300, 7200, 7500. Then again, we're not UU or Sprint, and don't have the traffic loading they do.
Joe
On 4/8/02 9:03 AM, "Iljitsch van Beijnum" <iljitsch@muada.com> wrote:
On 8 Apr 2002, Paul Vixie wrote:
I seem to remember fast switching was per-destination, and CEF was round robin. But it seems CEF is now per-destination as well in IOS 12.2. Round robin is optional.
CEF is flow-hashed, and the hash seems to include both source and destination, and seems to include the port numbers. This is by observing the behaviour of flows hitting various members of the F.ROOT-SERVERS.NET set, each of whom sends F's address to several upstream routers using OSPF. CEF works like a charm -- the load is never split by more than 45-55 and that's damn good for wire speed hashing in my view.
We used CEF in 11.x and it behaved the same way. It was never round-robin in any way we could observe.
You're right. I was thinking of process switching.
According to: http://www.ils.unc.edu/dempsey/186s00/reorderingpaper.pdf
packet reordering at MAE East was extremely common a few years ago. Does anyone have information whether this is still happening?
-- Alex Rubenstein, AR97, K2AHR, alex@nac.net, latency, Al Reuben -- -- Net Access Corporation, 800-NET-ME-36, http://www.nac.net --