On Thu, Apr 12, 2001 at 10:16:00AM -0400, Craig Partridge wrote:
Having implemented distributed forwarding once, and consulted on it a couple of times, the answer is that it adds complexity but not where you'd think. [...] Now I just said "all you need" -- that's a lie :-). The major complexity is actually getting the forwarding tables out to the inbound interfaces. There are at least two issues: (a) bandwidth [a small forwarding table, times 32 or 64 ports rapidly becomes big, especially for the control data path which is typically short on bandwidth] and (b) finding a way to update the forwarding table in a way that is not too disruptive to forwarding [incremental updates are often mindbreakingly hard to do, esp. if the update changes the table size, or invalidates some entries -- you can have the forwarding table doing housecleaning instead of forwarding]. We used a bank swap -- we had an active forwarding table and an inactive one. Updates were sent to the inactive bank and when you had a fully updated bank, you send a message to the forwarding processor saying "swap banks." And, of course, the software to track which forwarding engine had which bank live and the current state of each bank's updates got a little exciting.
Yes, I had thought about this kind of consistency as well, within a single forwarding table. You have to provide a transactional system to update a complete table (or a self-contained subset) in an atomic operation. In practice, many of the forwarding tables for different interfaces will contain identical entries, correct? Most traffic for a given destination will leave through one interface, with the same next-hop. Is it possible (or useful) to attempt to send forwarding table entries (or chunks) across the bus to multiple interfaces at once, to approach the bandwidth issue? Thanks for sharing your experience with the list. -- - mdz