vixie@vix.com (Paul Vixie) (hey, that's me!) wrote:
as i said the other day, "all power tools can kill." if you turn on PPLB and it hurts, then turn it off until you can read the manual or take a class or talk to an expert. PPLB is a link bundling technology. if you turn it on in non-parallel-path situation, it will hurt you, so, "don't do that."
Iljitsch replied as follows:
Yes, per packet load balancing will cause reordering, and if that's an issue you shouldn't use it. But if with pplb packets end up at two different hosts, that's not the fault of the people who invented per packet load balancing or the people who turned it on, but the fault of the people giving the same address to two different hosts.
i then bypassed Iljitsch and went to the gallery:
i would be interested in hearing from anybody else who thinks that turning on pplb in a eyeball-centric isp that has multiple upstream paths is a reasonable thing to do, even if there were no anycast services deployed anywhere in the world.
so far, no takers. i've heard from rfc-writers who say pplb was never meant to be used the way Iljitsch is describing it, and i've heard from equipment vendors who say their customers don't do that and that if some customer did that and asked for support the response would be "don't do that!", and i've heard from network operators who say they would never do that, and i've heard from customers of network operators who did that with notable bad effects. but so far nobody has said "yes, what Iljitsch is describing should work." let me summarize. Iljitsch says that pplb is incompatible with anycast, since a pplb-using access router at the inner edge of an ISP could hear two different IGP routes to some destination, which ended up taking different exits from the ISP and thus different BGP paths. whereas pplb would normally only operate on equal-cost paths, the BGP->IGP path would hide the variance in BGP paths and make these "paths" eligible for pplb. i've said that pplb is only useful for turning two OC3's into an "OC6" (or similar circuit bundling where a pair of routers has multiple connections to eachother) and that even in this case, packet reordering is likely to occur, which will make tcp-flow performance suffer across this "link". i have also said that turning pplb on across non-parallel links, such as to multiple providers or through multiple tunnels or whatever, would pretty much guaranty that a word rhyming with "massive suckage" would occur. and i've made these claims independent of anycast -- that is, life will be bad if you use pplb outside its intended purpose, even if nobody anywhere was using anycast. loath though i am to treat a "preponderance of assertion" as equivilent to "proof", i see no alternative on this issue. noone is defending the use case Iljitsch is proposing. noone is even saying "i tried that and it was OK". lots of people are saying various things like "don't do that!" and "are you crazy?" it's important to point out a third time that it's indeed possible that Iljitsch's proposed use case for pplb would interact badly with anycast, and that i'm not arguing against that assertion. i'm saying that the pplb configuration proposed by Iljitsch would have really bad consequences even if noone, anywhere on the internet, was using anycast. and so we return to yesterday's statement:
at the moment i am completely certain that turning on pplb would be an irrational act, and would have a significant performance-dooming effect on a client population behind it, and that the times when pplb would actually be useful and helpful are very limited, and that anycast doesn't even enter into the reasons why doing as Iljitsch paints would be a bad idea.
and i'll repeat, again:
but my mind is open, if anyone can speak from experience on the matter.
and, "good luck storming the castle, boys." -- Paul Vixie