Re: Routers vs. PC's for routing - was list problems?
At 02:20 PM 5/23/2002 -0400, you wrote:
Vinny Abello wrote:
I would have to say for any Linux/BSD platform to be a viable routing solution, you have to eliminate all moving parts or as much as possible, ie. no hard drives because hard drives will fail. Not much you can do about the cooling fans in various parts of the machine though which routers also tend to have. Solid state storage would be the way to go as far as what the OS is installed on. You have to have something to imitate flash on the common router. Otherwise, if you can get the functionality out of a PC, I say go for it! The processing power of a modern PC is far beyond any router I can think of. I suppose it would just be a matter of how efficient your kernel, TCP/IP stack and routing daemon would be at that point. :)
I've several comments here.
First off, you're right about moving parts generally being a bad thing. However, it is not always necessary to eliminate the hard drive. Two drives in a RAID-0 configuration may be reliable enough. Especially if the failure of a single drive sets off sufficient alarms so that it can quickly be hot-swapped for a new drive.
I'm assuming you meant RAID-1. In RAID-0 if you 'swapped' any drive all your striped data is toast. ;)
The real problem with using PCs is bandwidth and hardware reliability.
PCs generally don't have many hot-swappable parts. You can get hot-swap hard drive assemblies without too much work, and redundant power supplies can be purchased. A motherboard that allows hot-swapping of PCI cards (usually by having multiple busses and a mechanism to turn them off individually) is very rare. You can get dual-CPU motherboards, but not with the processors being hot swappable. And I don't know if any will allow the system to transparently fail over from one CPU to another, should the primary fail.
I agree with you on that. Hot swapability for various interfaces is something routers obviously have over PC's.
Then there's the issue of the PCI bus. Standard PCI (32-bit 33MHz) has a theoretical maximum bandwidth of about 1Gbit/s. But you can never use all of a PCI bus's bandwidth, so actual limits will be less than this.
True... unless going for 64 bit PCI at 66MHz... still it's obvious that routers are designed for one simple purpose and generally have larger backplanes to handle that.
When you're doing software routing, every packet must cross the bus twice - once for the receive and once for the send. So your standard PCI bus (if used for nothing but packets) has a top speed of 500Mbit/s. Which is less than three ports of full-duplex fast Ethernet at line rate. Multiple busses and/or 64-bit 66MHz PCI can increase this limit, but now you're talking about much more expensive motherboards.
You can dramatically improve throughput if you can get line cards that have on-board forwarding chips. If cards can forward packets between each other without getting the CPU involved, then packets can cross the PCI bus only once. But these kinds of line cards are not cheap, if they can be found at all. And you will need some way of downloading your kernel routing table into the cards, which may require some serious OS hacking.
In other words, a PC equipped to be as reliable and capable as a decent router will likely end up costing as much as a router. And the reason has nothing to do with the CPU speed or the operating system.
I agree a router is probably more efficient in just routing packets, but in complex filtering or traffic manipulation/packet sniffing, a PC might have the edge. :) Don't get me wrong. I would never use a PC for a router, just as I wouldn't try to run my workstation as a 7206. ;) It's all just hypothesizing. Vinny Abello Network Engineer Server Management vinny@tellurian.com (973)300-9211 x 125 (973)940-6125 (Direct) Tellurian Networks - The Ultimate Internet Connection http://www.tellurian.com (888)TELLURIAN
I agree with you on that. Hot swapability for various interfaces is something routers obviously have over PC's.
Hot swap PCI is old news.
True... unless going for 64 bit PCI at 66MHz... still it's obvious that routers are designed for one simple purpose and generally have larger backplanes to handle that.
However, $ for $, even when buying used cisco gear at 80% off from dot-booms, a PC router will outperform any traditional router.
I agree a router is probably more efficient in just routing packets, but in complex filtering or traffic manipulation/packet sniffing, a PC might have the edge. :)
Yes, ipfw/dummy is very very cool. Like, inducing a few 100 msecs of latency to folks who don't pay on time :) -- Alex Rubenstein, AR97, K2AHR, alex@nac.net, latency, Al Reuben -- -- Net Access Corporation, 800-NET-ME-36, http://www.nac.net --
AR> Date: Thu, 23 May 2002 16:17:16 -0400 (Eastern Daylight Time) AR> From: Alex Rubenstein AR> Yes, ipfw/dummy is very very cool. Like, inducing a few 100 AR> msecs of latency to folks who don't pay on time :) 1. Oh, come on, I know you're more creative than _that_. How about 30% packet loss on their *:53 TCP/UDP? Or running them through stateful rules with uebershort timeouts? Or simply having all their traffic trigger a scan (help! my firewall is seeing attacks to port 113!) right back at them. 2. I dub thee "Alex 'BOFH' Rubenstein". ;-) -- Eddy Brotsman & Dreger, Inc. - EverQuick Internet Division Phone: +1 (316) 794-8922 Wichita/(Inter)national Phone: +1 (785) 865-5885 Lawrence ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature. These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
Vinny Abello wrote:
First off, you're right about moving parts generally being a bad thing. However, it is not always necessary to eliminate the hard drive. Two drives in a RAID-0 configuration may be reliable enough. Especially if the failure of a single drive sets off sufficient alarms so that it can quickly be hot-swapped for a new drive.
I'm assuming you meant RAID-1. In RAID-0 if you 'swapped' any drive all your striped data is toast. ;)
Oops. Yes. of course I meant RAID-1.
Then there's the issue of the PCI bus. Standard PCI (32-bit 33MHz) has a theoretical maximum bandwidth of about 1Gbit/s. But you can never use all of a PCI bus's bandwidth, so actual limits will be less than this.
True... unless going for 64 bit PCI at 66MHz...
64/66 PCI has 4 times as much bandwidth - about 4Gbit/s. Much better than standard PCI, but hard to find on a PC-compatible motherboard, and expensive when you do find it. Enough bandwidth for 10 line-rate 100M Ethernet ports or six line-rate OC-3 ports (in theory, anyway). But not really enough for anything faster (OC-12 or GigE) if you want line-rate forwarding. -- David
On Thu, May 23, 2002 at 05:47:40PM -0400, David Charlap wrote:
64/66 PCI has 4 times as much bandwidth - about 4Gbit/s. Much better than standard PCI, but hard to find on a PC-compatible motherboard, and expensive when you do find it. Enough bandwidth for 10 line-rate 100M Ethernet ports or six line-rate OC-3 ports (in theory, anyway). But not really enough for anything faster (OC-12 or GigE) if you want line-rate forwarding.
Why is this such a hard concept for people to grasp? If you just need to bat around a couple hundred Mbit, a PC based router could work beautifully for you. If you want to design a scalable but efficient system, you use dedicated hardware for the forwarding plane, cheap but powerful PC hardware for the control plane, and an ASIC to look at bytes in the header and come up with a destination interface. But Juniper has done this, so move on. I wish they would put a little more legitimacy on the Olive though, it could be a very useful product. Everything from very small guys who only need to move 100Mbit but who need more stability and policy power than a linsux box and zebra can provide, to the very big guys who could build a very beefy 2GHz box for computationally intensive tasks (like a route reflector). -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras PGP Key ID: 0x138EA177 (67 29 D7 BC E8 18 3E DA B2 46 B3 D8 14 36 FE B6)
True... unless going for 64 bit PCI at 66MHz...
64/66 PCI has 4 times as much bandwidth - about 4Gbit/s. Much better than standard PCI, but hard to find on a PC-compatible motherboard, and expensive when you do find it. Enough bandwidth for 10 line-rate 100M Ethernet ports or six line-rate OC-3 ports (in theory, anyway). But not really enough for anything faster (OC-12 or GigE) if you want line-rate forwarding.
Most reputable motherboards (high-end super micros, intel) support 64/66. -- Alex Rubenstein, AR97, K2AHR, alex@nac.net, latency, Al Reuben -- -- Net Access Corporation, 800-NET-ME-36, http://www.nac.net --
participants (5)
-
Alex Rubenstein -
David Charlap -
E.B. Dreger -
Richard A Steenbergen -
Vinny Abello