On Sat, May 18, 2013 at 11:39:55AM -0400, Nick Khamis wrote:
Hello Everyone,
We are running:
Gentoo Server on Dual Core Intel Xeon 3060, 2 Gb Ram Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06) Ethernet controller: Intel Corporation 82573E Gigabit Ethernet Controller (rev 03)
2 bgp links from different providers using quagga, iptables etc....
We are transmitting an average of 700Mbps with packet sizes upwards of 900-1000 bytes when the traffic graph begins to flatten. We also start experiencing some crashes at that point, and not have been able to pinpoint that either.
I was hoping to get some feedback on what else we can strip from the kernel. If you have a similar setup for a stable platform the .config would be great!
Also, what are your thoughts on migrating to OpenBSD and bgpd, not sure if there would be a performance increase, but the security would be even more stronger?
That hardware should be fine to do two gig ports upstream, with another two to go to your network? I'd check with "vmstat 1" to see what your interrupt rate is like, if it's above 40k/sec I'd check coalescing settings. I also prefer OpenBSD/OpenBGP myself. It's a simpler configuration, with less things to "fix". With Linux you have to disable reverse path filtering, screw around with iptables to do bypass on stateful filtering. Then Quagga itself can be buggy. (my original reason for shifting away from Linux was that Quagga didn't fix enough of Zebra's bugs.. although that was many years ago, things may have improved a little by then, but ime significantly buggy software tends to stay buggy even with fixing) With regards to security of OpenBSD versus Linux, you shouldn't be exposing any services to the world with either. And it's more stability/configuration that would push me to OpenBSD rather than performance. And with regards to crashing I'd try and figure out what was happening there quickly before making radical changes. Is it running out of memory, is Quagga dying? Is there a default route that works when Quagga crashes? One issue I had was I found Quagga crashing leaving a whole lot of routes lingering in the table, and I had a script that'd go through and purge them. I'm also a bit confused about your dual upstreams with two ethernet interfaces total, are they both sharing one pipe, or are there some Broadcom or such ethernet interfaces too. I've found Broadcom chipsets can be a bit problematic, and the only stability issue I've ever had with OpenBSD is a Broadcom interface wedging for minutes under DDOS attack, which was gigabit'ish speed DDOS with older hardware than you. oh, to check coalescing settings under linux use: "ethtool -c eth0; ethtool -c eth1" Ben.