On 5/19/13, Nikola Kolev <nikky@mnet.bg> wrote:
You might be maxing out your server's PCI bus throughput, so it might be a better idea if you can get Ethernet NICs that are sitting at least on PCIe x8 slots.
Nikola, thank you so much for your response! It kind of looks that way, and we do have another candidate machine that has a PCIe 3 x8. First thing, I never liked riser card and the candidate IBM x3250 M$ does use them. Not sure how much of a hit I will take for that. Secondly are there any proven intel 4 port cards in PCIe 3 preferably pro 1000.
Leaving that aside, I take it you've configured some sort of CPU/PCI affinity?
For interrupts we disabled CONFIG_HOTPLUG_CPU in the kernel, and assigned interrupts to the less used core using APIC. I am not sure if there is anything more we can do?
As for migration to another OS, I find FreeBSD better as a matter of network performance. The last time I checked OpenBSD was either lacking or was in the early stages of multiple cores support.
I know I mentioned migration, but gentoo has been really good to us, and we grew really fond of her :). Hope I can tune it further before retiring it as our OS of choice. Nick.