Routers vs. PC's for routing - was list problems?
I would have to say for any Linux/BSD platform to be a viable routing solution, you have to eliminate all moving parts or as much as possible, ie. no hard drives because hard drives will fail. Not much you can do about the cooling fans in various parts of the machine though which routers also tend to have. Solid state storage would be the way to go as far as what the OS is installed on. You have to have something to imitate flash on the common router. Otherwise, if you can get the functionality out of a PC, I say go for it! The processing power of a modern PC is far beyond any router I can think of. I suppose it would just be a matter of how efficient your kernel, TCP/IP stack and routing daemon would be at that point. :) At 10:48 PM 5/22/2002, you wrote:
On Wed, 22 May 2002, Andy Dills wrote:
From the number of personal replies I got about these topics, it seems like many people are interested in sharing information about how to do routing on a budget, or how to avoid getting shot in the foot with your Cisco box.
Routing on a budget? Dude, you can buy a 7200 for $2 grand. Why bother with a linux box? Heh, at least use FreeBSD :)
Before the dot com implosion, they weren't nearly that inexpensive. The average corporate user will also need smartnet (what's that on a 7200, a K or a few per year?) for support, warranty, and software updates. Some people just don't appreciate being nickled and dimed by cisco and forced to either buy much more router than they need, or risk ending up with another cisco boat anchor router when the platform they chose can no longer do the job in the limited memory config supported.
I have a consulting customer who, against my strong recommendation, bought a non-cisco router to multihome with. It's PC based, runs Linux, and with the exception of the gated BGP issue that bit everyone running gated a few months ago, has worked just fine. It's not as easy to work with in most cases, but there are some definite advantages, and some things that Linux actually makes easier. They'd initially bought a 2621 when multihoming was just a thought, and by the time it was a reality, 64mb on a 2621 couldn't handle full routes. The C&W/PSI depeering (which did affect this customer, as they were single homed to C&W at the time and did regular business with networks single homed to PSI) was proof that without full routes, you're not really multihomed.
-- ---------------------------------------------------------------------- Jon Lewis *jlewis@lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
Vinny Abello Network Engineer Server Management vinny@tellurian.com (973)300-9211 x 125 (973)940-6125 (Direct) Tellurian Networks - The Ultimate Internet Connection http://www.tellurian.com (888)TELLURIAN
VA> Date: Thu, 23 May 2002 09:26:41 -0400 VA> From: Vinny Abello VA> I would have to say for any Linux/BSD platform to be a viable I suppose it's been awhile since this thread has made the rounds, so I'll jump in for a moment... VA> routing solution, you have to eliminate all moving parts or VA> as much as possible, ie. no hard drives because hard drives EIDE-based flash drives have become very inexpensive. Some embedded systems use CompactFlash boards. VA> will fail. Not much you can do about the cooling fans in It's always nice if the CPU is happy with a "big enough" heatsink and no fans. VA> various parts of the machine though which routers also tend VA> to have. Solid state storage would be the way to go as far as VA> what the OS is installed on. You have to have something to I think that 128 MB CompactFlash boards are < $60 new now. I've not priced drives recently, but I'm sure they're similar. VA> imitate flash on the common router. Otherwise, if you can get VA> the functionality out of a PC, I say go for it! The VA> processing power of a modern PC is far beyond any router I Yes and no. The central CPU, yes. The line cards, no. VA> can think of. I suppose it would just be a matter of how VA> efficient your kernel, TCP/IP stack and routing daemon would VA> be at that point. :) You left out one critical thing: The bus/backplane. For DS1 service or a few DS3s, standard PCI will work fine. But once the bus is maxed out... you need something bigger (wider or faster bus) or better (cPSB ethernet midplane). Has anyone had the privilege of playing with cPSB gear? If so, I'd like to know what your experiences were... That said, I'm definitely a proponent of "roll your own" routers, although the great prices on used turnkey gear might just make RYO routing more expensive nowadays. (I assume that anyone clueful enough to build a router probably wouldn't need the bigger vendor service contracts.) Then again, if you need different behavior and can cut code, RYO is more flexible. -- Eddy Brotsman & Dreger, Inc. - EverQuick Internet Division Phone: +1 (316) 794-8922 Wichita/(Inter)national Phone: +1 (785) 865-5885 Lawrence ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature. These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
On Thu, 23 May 2002, E.B. Dreger wrote:
EIDE-based flash drives have become very inexpensive. Some embedded systems use CompactFlash boards.
Can you set flash drives to be write-only? Sorry if this is a basic question, but the only EIDE mass-storage devices I've used are more traditional drives. This would be a great solution for a Linux box I want to build as a bridge. -- Steve Sobol, CTO (Server Guru, Network Janitor and Head Geek) JustThe.net LLC, Mentor On The Lake, OH 888.480.4NET http://JustThe.net "In a 32-bit world, you're a 2-bit user/You've got your own newsgroup: alt.total.loser" - "Weird Al" Yankovic, "It's All About the Pentiums"
On Thu, 23 May 2002, Steven J. Sobol wrote:
On Thu, 23 May 2002, E.B. Dreger wrote:
EIDE-based flash drives have become very inexpensive. Some embedded systems use CompactFlash boards. Can you set flash drives to be write-only?
Why would you want to do this? -Dan -- [-] Omae no subete no kichi wa ore no mono da. [-]
On Thu, 23 May 2002, Jake Baillie wrote:
Why would you want to do this? Because flash has a limited number of writes. If you used it like a
At 02:28 PM 5/23/2002 -0700, Dan wrote: traditional file system, it would go kaput in no time.
And making it *write-only* as the original poster asked, would fix things how? -Dan -- [-] Omae no subete no kichi wa ore no mono da. [-]
Let me elaborate. I thought Steve was concerned about the limited writablity of flash. My thought was to build something like a Linux router, you'd have to load the OS into a RAMdisk (or something similar), and only write to flash when the config changed. Which means you'd need some sort of singular configuration file. But I was wrong. :) He meant "read-only" *back to lurk mode* -- jb At 02:49 PM 5/23/2002 -0700, Dan Hollis wrote:
And making it *write-only* as the original poster asked, would fix things how?
On Thu, 23 May 2002, Jake Baillie wrote:
the config changed. Which means you'd need some sort of singular configuration file.
But I was wrong. :) He meant "read-only"
I'm just throwing ideas out there. I could boot Linux off a floppy or a bootable CD and create a ramdisk upon bootup - Linux has always had this capability. I'm just a person who occasionally comes up with silly half-baked ideas and wonders if he can implement them. ;) And to be honest, I figured that having the OS boot off of some solid-state storage device would be useful... for something... -- Steve Sobol, CTO (Server Guru, Network Janitor and Head Geek) JustThe.net LLC, Mentor On The Lake, OH 888.480.4NET http://JustThe.net "In a 32-bit world, you're a 2-bit user/You've got your own newsgroup: alt.total.loser" - "Weird Al" Yankovic, "It's All About the Pentiums"
On Thu, May 23, 2002 at 06:04:09PM -0400, sjsobol@JustThe.net said: [snip]
I'm just throwing ideas out there. I could boot Linux off a floppy or a bootable CD and create a ramdisk upon bootup - Linux has always had this capability. I'm just a person who occasionally comes up with silly half-baked ideas and wonders if he can implement them. ;)
And to be honest, I figured that having the OS boot off of some solid-state storage device would be useful... for something...
This has come up a few times on misc@openbsd.org. You might want to check the archives, and maybe look at picobsd.org (among others) for more ideas in this vein. -- Scott Francis darkuncle@ [home:] d a r k u n c l e . n e t Systems/Network Manager sfrancis@ [work:] t o n o s . c o m GPG public key 0xCB33CCA7 illum oportet crescere me autem minui
Speaking of which: I have been looking for a reasonable priced hardware ramdisk. The ones I've seen (albeit expensive) are essentially a brick with DIMMs in them, and have either a IDE or SCSI interface. Some have a battery to back them up for a few hours. Anyone got some pointers? On Thu, 23 May 2002, Jake Baillie wrote:
Let me elaborate. I thought Steve was concerned about the limited writablity of flash.
My thought was to build something like a Linux router, you'd have to load the OS into a RAMdisk (or something similar), and only write to flash when the config changed. Which means you'd need some sort of singular configuration file.
But I was wrong. :) He meant "read-only"
*back to lurk mode*
-- jb
At 02:49 PM 5/23/2002 -0700, Dan Hollis wrote:
And making it *write-only* as the original poster asked, would fix things how?
-- Alex Rubenstein, AR97, K2AHR, alex@nac.net, latency, Al Reuben -- -- Net Access Corporation, 800-NET-ME-36, http://www.nac.net --
On Thu, 23 May 2002, Dan Hollis wrote:
On Thu, 23 May 2002, Steven J. Sobol wrote:
On Thu, 23 May 2002, E.B. Dreger wrote:
EIDE-based flash drives have become very inexpensive. Some embedded systems use CompactFlash boards. Can you set flash drives to be write-only?
Why would you want to do this?
Duh. Sorry about the brainfart. I was about to launch into a long explanation of what I want to do when I realized I wrote "write-only" instead of "read-only." I meant "read-only." Note to self: Engage brain *before* fingers. -- Steve Sobol, CTO (Server Guru, Network Janitor and Head Geek) JustThe.net LLC, Mentor On The Lake, OH 888.480.4NET http://JustThe.net "In a 32-bit world, you're a 2-bit user/You've got your own newsgroup: alt.total.loser" - "Weird Al" Yankovic, "It's All About the Pentiums"
On Thu, 23 May 2002, E.B. Dreger wrote:
EIDE-based flash drives have become very inexpensive. Some embedded systems use CompactFlash boards.
Can you set flash drives to be write-only? Sorry if this is a basic question, but the only EIDE mass-storage devices I've used are more traditional drives.
Write only? Sure, that's been around since at least 1972! http://www.ganssle.com/misc/wom1.jpg http://www.ganssle.com/misc/wom2.jpg </smartassmode> :) If you mean READ only, some of Sandisk's products (not their normal consumer grade Compact Flash disks) have a read only mode. Some of which even have a mode where you can blow a fuse inside the chip with a special instruction, and make it read-only forever. Someone else made a solid state flash based IDE compatible drive, too, that had an option for "write once" per sector (yet could be all blanked using another special command). I know they wrote a module for OS9 to support it using its native filesystem (most FS's don't like being unable to write to whatever they want, whenever they want). It may have been Atmel. -- Kevin
SJS> Date: Thu, 23 May 2002 17:23:43 -0400 (EDT) SJS> From: Steven J. Sobol SJS> Can you set flash drives to be write-only? Sorry if this is Depends on the drive, just like traditional HDDs. SJS> a basic question, but the only EIDE mass-storage devices SJS> I've used are more traditional drives. Why not partition wisely, then mount the desired partition as read-only? Or I guess one _could_ mount each partition as RO... But why? -- Eddy Brotsman & Dreger, Inc. - EverQuick Internet Division Phone: +1 (316) 794-8922 Wichita/(Inter)national Phone: +1 (785) 865-5885 Lawrence ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature. These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
On Thu, 23 May 2002, E.B. Dreger wrote:
SJS> a basic question, but the only EIDE mass-storage devices SJS> I've used are more traditional drives.
Why not partition wisely, then mount the desired partition as read-only? Or I guess one _could_ mount each partition as RO...
But why?
The box I want to build is passing packets between the rest of my network (and the public Internet) and one server that will hold sensitive data. It'll be a Linux box with the TCP/IP stack running in bridged mode, with two ethernet adapters installed. The box just needs to boot up and run. It doesn't need to log anything. -- Steve Sobol, CTO (Server Guru, Network Janitor and Head Geek) JustThe.net LLC, Mentor On The Lake, OH 888.480.4NET http://JustThe.net "In a 32-bit world, you're a 2-bit user/You've got your own newsgroup: alt.total.loser" - "Weird Al" Yankovic, "It's All About the Pentiums"
SJS> Date: Thu, 23 May 2002 18:01:03 -0400 (EDT) SJS> From: Steven J. Sobol SJS> The box I want to build is passing packets between the rest SJS> of my network (and the public Internet) and one server that SJS> will hold sensitive data. It'll be a Linux box with the SJS> TCP/IP stack running in bridged mode, with two ethernet SJS> adapters installed. The box just needs to boot up and SJS> run. It doesn't need to log anything. Might I suggest { ipfw | ipf | pf } on *BSD? Depending on the flavor you choose, you'd have some or all of: Stateful filtering... ISN proxying... firewall rules that can't be changed without a reboot... diverting packets to userland for custom munching... Not to turn NANOG into a BSD evangelism list, but many people who grok BSD and Linux seem to choose BSD. Try it. -- Eddy Brotsman & Dreger, Inc. - EverQuick Internet Division Phone: +1 (316) 794-8922 Wichita/(Inter)national Phone: +1 (785) 865-5885 Lawrence ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature. These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
On Thu, 23 May 2002 18:01:03 EDT, "Steven J. Sobol" said:
The box I want to build is passing packets between the rest of my network (and the public Internet) and one server that will hold sensitive data. It'll be a Linux box with the TCP/IP stack running in bridged mode, with two ethernet adapters installed. The box just needs to boot up and run. It doesn't need to log anything.
I've heard tell that a good way to secure a Linux box that's doing this is to have it boot, set up the interfaces, set up iptables, and then do a quick /sbin/halt - if you fail to 'ifconfig down' the interfaces on the way down, the kernel will happily forward the packets while being immune to exploits (since there's no processes running anymore). I haven't tried it, so I dont know if it works. Maybe there ARE cases where setting the default runlevel to 0 or 6 make sense. ;)
Date: Fri, 24 May 2002 00:52:14 -0400 From: Valdis.Kletnieks@vt.edu
I've heard tell that a good way to secure a Linux box that's doing this is to have it boot, set up the interfaces, set up iptables, and then do a quick /sbin/halt - if you fail to 'ifconfig down' the interfaces on the way down, the kernel will happily forward the packets while being immune to exploits
[ snip ] Hmmmm. A most interesting thought. Even if that doesn't work, one could modify /sbin/init to suit one's needs; several variants for embedded systems already exist. -- Eddy Brotsman & Dreger, Inc. - EverQuick Internet Division Phone: +1 (316) 794-8922 Wichita/(Inter)national Phone: +1 (785) 865-5885 Lawrence ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature. These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
## On Friday, May 24, 2002 12:52 AM -0400 ## Valdis.Kletnieks@vt.edu wrote:
I've heard tell that a good way to secure a Linux box that's doing this is to have it boot, set up the interfaces, set up iptables, and then do a quick /sbin/halt - if you fail to 'ifconfig down' the interfaces on the way down, the kernel will happily forward the packets while being immune to exploits (since there's no processes running anymore). I haven't tried it, so I dont know if it works. Maybe there ARE cases where setting the default runlevel to 0 or 6 make sense. ;)
This seems to be a rather dumb idea for at least a couple reasons. The increase in security is nothing compared to the headache you've created. a) How do you log? b) How do you update your rulesets? c) How do you figure out what went wrong when something DOES go wrong? A system with an out-of-band interface (dialup, serial, ethernet, IrDA, etc) can offer the same level of security without the trouble of a pseudo-halted system. It can log, it can update rulesets, the device can be configured to only allow management from that interface, etc... [as if you didn't know this] As to being immune to exploits I fail to see how. An exploit is an exploit -- it doesn't need to give you a root shell to accomplish a goal of crashing the packet filter. I'm more than happy to be proven wrong though, when is there a time when a pseudo-halted system is "more secure"? -davidu
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of David Ulevitch Sent: Friday, May 24, 2002 2:36 AM To: Valdis.Kletnieks@vt.edu Cc: nanog@merit.edu Subject: Re: Routers vs. PC's for routing - was list problems? [deleted] As to being immune to exploits I fail to see how. An exploit is an exploit -- it doesn't need to give you a root shell to accomplish a goal of crashing the packet filter. I'm more than happy to be proven wrong though, when is there a time when a pseudo-halted system is "more secure"? -davidu ---- EXACTLY! Vulnerabilities [especially in socket functions (you still *are* running a routing protocol right?)] can cause arbitrary code to execute irrespective of your current run level. Most people would agree that having to reboot the machine to change/check/edit anything is an unacceptable scenario. Further, how do you filter an attack in real-time? Deepak Jain AiNET
On Thu, 2002-05-23 at 09:26, Vinny Abello wrote: common router. Otherwise, if you can get the functionality out of a PC, I
say go for it! The processing power of a modern PC is far beyond any router I can think of. I suppose it would just be a matter of how efficient your kernel, TCP/IP stack and routing daemon would be at that point. :)
And that's MY real question. Who has actually done this in a production environment that can speak with some real experience on the topic? What can you replace with a linux box to route and run BGP for you in real life? A 7200? Bigger. I don't have the facilities to try these things out for real, and frankly would be worried about the uptime and finding the RIGHT PC hardware that isn't complete junk. So I guess it's really two questions: what is a PC capable of replacing as far as throughput goes, and just how reliable can a clone (or pick your manufacturer) be compared to a unit that was designed by electronic engineers to function as a 24x7 mission critical box? Daryl G. Jurbala Independent Consultant (read: looking for a job) daryl@introspect.net
And that's MY real question. Who has actually done this in a production environment that can speak with some real experience on the topic? What can you replace with a linux box to route and run BGP for you in real life? A 7200? Bigger.
I don't have the facilities to try these things out for real, and frankly would be worried about the uptime and finding the RIGHT PC hardware that isn't complete junk.
So I guess it's really two questions: what is a PC capable of replacing as far as throughput goes, and just how reliable can a clone (or pick your manufacturer) be compared to a unit that was designed by electronic engineers to function as a 24x7 mission critical box?
I've done it in a production environment and unless money was extremely tight I wouldn't consider doing it again. You will save on capital expediture but you need an army of resources to support it. When I did it, it was on NetBSD running GateD 3.x.x. And it supported in both cases two of the largest ISPs in Europe. There are more options now with Linux and Zebra etc but don't underestimate having to deal with PC issues and Unix issues. If your running LINUX you have to be subscribed to a million email lists to get an idea of issues etc and that takes up time. Anything above 200M-300Mbps then forget it, but as a cheap ethernet router its fine, and if it doesn't work you can always reuse the machines. I strongly recommend using an AWARD bios machine - everything else that I used had PCI bus timing issues. [ASUS motherboards were a good choice also]. Regards, Neil. -- Neil J. McRae - Alive and Kicking neil@DOMINO.ORG
On Thu, 23 May 2002, Neil J. McRae wrote:
I've done it in a production environment and unless money was extremely tight I wouldn't consider doing it again. You will save on capital expediture but you need an army of resources to support it. When I did it, it was on NetBSD running GateD 3.x.x. And it supported in both cases two of the largest ISPs in Europe.
Good point, I also did this for cash reasons and would just buy hardware on the used market today. As far as OS, I was using stripped down FreeBSD. I started with Linux, but at the time they did not support radix trees so routing tables killed the box. If I HAD to do it again I would still say away from Linux. -Nathan
Good point, I also did this for cash reasons and would just buy hardware on the used market today. As far as OS, I was using stripped down FreeBSD. I started with Linux, but at the time they did not support radix trees so routing tables killed the box. If I HAD to do it again I would still say away from Linux.
Yes I'd stick to NetBSD - when we first deployed it Linux didn't support VLSM! [Nor did SunOS/Solaris] Regards, Neil. -- Neil J. McRae - Alive and Kicking neil@DOMINO.ORG
On Thu, 23 May 2002, Daryl G. Jurbala wrote:
And that's MY real question. Who has actually done this in a production environment that can speak with some real experience on the topic? What can you replace with a linux box to route and run BGP for you in real life? A 7200? Bigger.
I ran a 100% PC router network for almost 2 years. I used them from everything from edge aggregation to core routers. You can make BGP do whatever you want in real life on a PC. I used modified GateD code and after some work became very happy with it.
I don't have the facilities to try these things out for real, and frankly would be worried about the uptime and finding the RIGHT PC hardware that isn't complete junk.
Yes, you need to build your own.
So I guess it's really two questions: what is a PC capable of replacing as far as throughput goes, and just how reliable can a clone (or pick your manufacturer) be compared to a unit that was designed by electronic engineers to function as a 24x7 mission critical box?
When you want to push over 30 meg you are better off looking at something other then a x86 to route packets.
<> Nathan Stratton CTO, Exario Networks, Inc. nathan at robotics.net nathan at exario.net http://www.robotics.net http://www.exario.net
Remember that a pc may have some certain functions that are "more powerful" than a router but a pc is a much more general computer. Routers are supposed to be and usually designed to do one thing only, route, not play quake, balance your check book, browse the net, etc etc. So although for example a gsr-12000 may hhave a slower cpu than the machine on your desk it probably will route and pass more traffic than your pc ever will because of its design. Not to say you can't route well with a linux or bsd system you can but at the high-end probably not as well. On Thu, 23 May 2002, Vinny Abello wrote:
I would have to say for any Linux/BSD platform to be a viable routing solution, you have to eliminate all moving parts or as much as possible, ie. no hard drives because hard drives will fail. Not much you can do about the cooling fans in various parts of the machine though which routers also tend to have. Solid state storage would be the way to go as far as what the OS is installed on. You have to have something to imitate flash on the common router. Otherwise, if you can get the functionality out of a PC, I say go for it! The processing power of a modern PC is far beyond any router I can think of. I suppose it would just be a matter of how efficient your kernel, TCP/IP stack and routing daemon would be at that point. :)
At 10:48 PM 5/22/2002, you wrote:
On Wed, 22 May 2002, Andy Dills wrote:
From the number of personal replies I got about these topics, it seems like many people are interested in sharing information about how to do routing on a budget, or how to avoid getting shot in the foot with your Cisco box.
Routing on a budget? Dude, you can buy a 7200 for $2 grand. Why bother with a linux box? Heh, at least use FreeBSD :)
Before the dot com implosion, they weren't nearly that inexpensive. The average corporate user will also need smartnet (what's that on a 7200, a K or a few per year?) for support, warranty, and software updates. Some people just don't appreciate being nickled and dimed by cisco and forced to either buy much more router than they need, or risk ending up with another cisco boat anchor router when the platform they chose can no longer do the job in the limited memory config supported.
I have a consulting customer who, against my strong recommendation, bought a non-cisco router to multihome with. It's PC based, runs Linux, and with the exception of the gated BGP issue that bit everyone running gated a few months ago, has worked just fine. It's not as easy to work with in most cases, but there are some definite advantages, and some things that Linux actually makes easier. They'd initially bought a 2621 when multihoming was just a thought, and by the time it was a reality, 64mb on a 2621 couldn't handle full routes. The C&W/PSI depeering (which did affect this customer, as they were single homed to C&W at the time and did regular business with networks single homed to PSI) was proof that without full routes, you're not really multihomed.
-- ---------------------------------------------------------------------- Jon Lewis *jlewis@lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
Vinny Abello Network Engineer Server Management vinny@tellurian.com (973)300-9211 x 125 (973)940-6125 (Direct)
Tellurian Networks - The Ultimate Internet Connection http://www.tellurian.com (888)TELLURIAN
"Not to say you can't route well with a linux or bsd system you can but at the high-end probably not as well." Tell that to Juniper. Scott Granados wrote:
Remember that a pc may have some certain functions that are "more powerful" than a router but a pc is a much more general computer. Routers are supposed to be and usually designed to do one thing only, route, not play quake, balance your check book, browse the net, etc etc. So although for example a gsr-12000 may hhave a slower cpu than the machine on your desk it probably will route and pass more traffic than your pc ever will because of its design. Not to say you can't route well with a linux or bsd system you can but at the high-end probably not as well.
On Thu, 23 May 2002, Vinny Abello wrote:
I would have to say for any Linux/BSD platform to be a viable routing solution, you have to eliminate all moving parts or as much as possible, ie. no hard drives because hard drives will fail. Not much you can do about the cooling fans in various parts of the machine though which routers also tend to have. Solid state storage would be the way to go as far as what the OS is installed on. You have to have something to imitate flash on the common router. Otherwise, if you can get the functionality out of a PC, I say go for it! The processing power of a modern PC is far beyond any router I can think of. I suppose it would just be a matter of how efficient your kernel, TCP/IP stack and routing daemon would be at that point. :)
At 10:48 PM 5/22/2002, you wrote:
On Wed, 22 May 2002, Andy Dills wrote:
From the number of personal replies I got about these topics, it seems like many people are interested in sharing information about how to do routing on a budget, or how to avoid getting shot in the foot with your Cisco box.
Routing on a budget? Dude, you can buy a 7200 for $2 grand. Why bother with a linux box? Heh, at least use FreeBSD :)
Before the dot com implosion, they weren't nearly that inexpensive. The average corporate user will also need smartnet (what's that on a 7200, a K or a few per year?) for support, warranty, and software updates. Some people just don't appreciate being nickled and dimed by cisco and forced to either buy much more router than they need, or risk ending up with another cisco boat anchor router when the platform they chose can no longer do the job in the limited memory config supported.
I have a consulting customer who, against my strong recommendation, bought a non-cisco router to multihome with. It's PC based, runs Linux, and with the exception of the gated BGP issue that bit everyone running gated a few months ago, has worked just fine. It's not as easy to work with in most cases, but there are some definite advantages, and some things that Linux actually makes easier. They'd initially bought a 2621 when multihoming was just a thought, and by the time it was a reality, 64mb on a 2621 couldn't handle full routes. The C&W/PSI depeering (which did affect this customer, as they were single homed to C&W at the time and did regular business with networks single homed to PSI) was proof that without full routes, you're not really multihomed.
-- ---------------------------------------------------------------------- Jon Lewis *jlewis@lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
Vinny Abello Network Engineer Server Management vinny@tellurian.com (973)300-9211 x 125 (973)940-6125 (Direct)
Tellurian Networks - The Ultimate Internet Connection http://www.tellurian.com (888)TELLURIAN
ADC> Date: Thu, 23 May 2002 14:30:16 -0400 ADC> From: Anthony D Cennami ADC> "Not to say you can't route well with a linux or bsd system ADC> you can but at the high-end probably not as well." ADC> ADC> Tell that to Juniper. Where can I buy their line cards for my PC? -- Eddy Brotsman & Dreger, Inc. - EverQuick Internet Division Phone: +1 (316) 794-8922 Wichita/(Inter)national Phone: +1 (785) 865-5885 Lawrence ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature. These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
bsd kernel eh? i believe netapp filers are based on that as well. Bri On Thu, 23 May 2002, Anthony D Cennami wrote:
"Not to say you can't route well with a linux or bsd system you can but at the high-end probably not as well."
Tell that to Juniper.
Scott Granados wrote:
Remember that a pc may have some certain functions that are "more powerful" than a router but a pc is a much more general computer. Routers are supposed to be and usually designed to do one thing only, route, not play quake, balance your check book, browse the net, etc etc. So although for example a gsr-12000 may hhave a slower cpu than the machine on your desk it probably will route and pass more traffic than your pc ever will because of its design. Not to say you can't route well with a linux or bsd system you can but at the high-end probably not as well.
On Thu, 23 May 2002, Vinny Abello wrote:
I would have to say for any Linux/BSD platform to be a viable routing solution, you have to eliminate all moving parts or as much as possible, ie. no hard drives because hard drives will fail. Not much you can do about the cooling fans in various parts of the machine though which routers also tend to have. Solid state storage would be the way to go as far as what the OS is installed on. You have to have something to imitate flash on the common router. Otherwise, if you can get the functionality out of a PC, I say go for it! The processing power of a modern PC is far beyond any router I can think of. I suppose it would just be a matter of how efficient your kernel, TCP/IP stack and routing daemon would be at that point. :)
At 10:48 PM 5/22/2002, you wrote:
On Wed, 22 May 2002, Andy Dills wrote:
>From the number of personal replies I got about these topics, it seems like many people are interested in sharing information about how to do routing on a budget, or how to avoid getting shot in the foot with your Cisco box.
Routing on a budget? Dude, you can buy a 7200 for $2 grand. Why bother with a linux box? Heh, at least use FreeBSD :)
Before the dot com implosion, they weren't nearly that inexpensive. The average corporate user will also need smartnet (what's that on a 7200, a K or a few per year?) for support, warranty, and software updates. Some people just don't appreciate being nickled and dimed by cisco and forced to either buy much more router than they need, or risk ending up with another cisco boat anchor router when the platform they chose can no longer do the job in the limited memory config supported.
I have a consulting customer who, against my strong recommendation, bought a non-cisco router to multihome with. It's PC based, runs Linux, and with the exception of the gated BGP issue that bit everyone running gated a few months ago, has worked just fine. It's not as easy to work with in most cases, but there are some definite advantages, and some things that Linux actually makes easier. They'd initially bought a 2621 when multihoming was just a thought, and by the time it was a reality, 64mb on a 2621 couldn't handle full routes. The C&W/PSI depeering (which did affect this customer, as they were single homed to C&W at the time and did regular business with networks single homed to PSI) was proof that without full routes, you're not really multihomed.
-- ---------------------------------------------------------------------- Jon Lewis *jlewis@lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
Vinny Abello Network Engineer Server Management vinny@tellurian.com (973)300-9211 x 125 (973)940-6125 (Direct)
Tellurian Networks - The Ultimate Internet Connection http://www.tellurian.com (888)TELLURIAN
As are f5 proeducts including bigip, 3dns and hmmm they make something else I forget:). On Thu, 23 May 2002, Brian wrote:
bsd kernel eh? i believe netapp filers are based on that as well.
Bri
On Thu, 23 May 2002, Anthony D Cennami wrote:
"Not to say you can't route well with a linux or bsd system you can but at the high-end probably not as well."
Tell that to Juniper.
Scott Granados wrote:
Remember that a pc may have some certain functions that are "more powerful" than a router but a pc is a much more general computer. Routers are supposed to be and usually designed to do one thing only, route, not play quake, balance your check book, browse the net, etc etc. So although for example a gsr-12000 may hhave a slower cpu than the machine on your desk it probably will route and pass more traffic than your pc ever will because of its design. Not to say you can't route well with a linux or bsd system you can but at the high-end probably not as well.
On Thu, 23 May 2002, Vinny Abello wrote:
I would have to say for any Linux/BSD platform to be a viable routing solution, you have to eliminate all moving parts or as much as possible, ie. no hard drives because hard drives will fail. Not much you can do about the cooling fans in various parts of the machine though which routers also tend to have. Solid state storage would be the way to go as far as what the OS is installed on. You have to have something to imitate flash on the common router. Otherwise, if you can get the functionality out of a PC, I say go for it! The processing power of a modern PC is far beyond any router I can think of. I suppose it would just be a matter of how efficient your kernel, TCP/IP stack and routing daemon would be at that point. :)
At 10:48 PM 5/22/2002, you wrote:
On Wed, 22 May 2002, Andy Dills wrote:
>>From the number of personal replies I got about these topics, it seems >like many people are interested in sharing information about how to do >routing on a budget, or how to avoid getting shot in the foot with your >Cisco box.
Routing on a budget? Dude, you can buy a 7200 for $2 grand. Why bother with a linux box? Heh, at least use FreeBSD :)
Before the dot com implosion, they weren't nearly that inexpensive. The average corporate user will also need smartnet (what's that on a 7200, a K or a few per year?) for support, warranty, and software updates. Some people just don't appreciate being nickled and dimed by cisco and forced to either buy much more router than they need, or risk ending up with another cisco boat anchor router when the platform they chose can no longer do the job in the limited memory config supported.
I have a consulting customer who, against my strong recommendation, bought a non-cisco router to multihome with. It's PC based, runs Linux, and with the exception of the gated BGP issue that bit everyone running gated a few months ago, has worked just fine. It's not as easy to work with in most cases, but there are some definite advantages, and some things that Linux actually makes easier. They'd initially bought a 2621 when multihoming was just a thought, and by the time it was a reality, 64mb on a 2621 couldn't handle full routes. The C&W/PSI depeering (which did affect this customer, as they were single homed to C&W at the time and did regular business with networks single homed to PSI) was proof that without full routes, you're not really multihomed.
-- ---------------------------------------------------------------------- Jon Lewis *jlewis@lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
Vinny Abello Network Engineer Server Management vinny@tellurian.com (973)300-9211 x 125 (973)940-6125 (Direct)
Tellurian Networks - The Ultimate Internet Connection http://www.tellurian.com (888)TELLURIAN
On Thu, May 23, 2002 at 12:54:57PM -0700, Scott Granados wrote:
As are f5 proeducts including bigip, 3dns and hmmm they make something else I forget:).
On Thu, 23 May 2002, Brian wrote:
bsd kernel eh? i believe netapp filers are based on that as well.
Indeed - bigIP is BSDI aka BSD/OS based, netapp uses NetBSD code. Greetz, Peter -- huk ~ kek
Though I might lend a comment here. I have had alot of experience with PC based routers, starting around 96, and getting majorly into it around 98 or so. To give you an idea. No moving parts except cooling fans. Main drive is an IDE style SanDisk flash drive. System goes through a multistage boot. System start, loads initial startup code into boot ramdisk. System mounts a partition on the flash read-only System creates soon to be / ramdisk and uncompresses final fs image to it System copies stored configs from flash to /etc on second ramdisk System unmounts flash and remounts rootfs to second ramdisk System frees first ramdisk System finishes boot This was of course a totally custom Linux distrib, with a set of config tools for manipulation of the boot config (The flash stores 2 operational config archives, 2 operational fs images and one recovery config and fs image.) The system would automagicly boot the primary config, on failure boot the secondary, on failure boot the recovery image. Boot image and config set selectable at boot via serial console. This allowed us to load a make config updates to the primary config, while saving the working configs to the secondary, and to handle fs image updates properly (can always drop back to last known working copy). Worst case the recovery image can reload from backup via the network in a matter of seconds. The base platform was a K6-3 450Mhz, giving us a 64k L1 and 256K L2 cache running at 450Mhz, and a 1M L3 at 100Mhz. Given 256M SDRAM for main memory (4 way interleave) and using 64MB for the rootfs with the distro specificly designed to run in a ram only environ everything worked well (especially without IDE bus interrupts screwing with things). The only time it touched flash was during boot, and when updating or backing up config or fs images. We used (and sold) many of these boxes as a 7200 replacement. A 7206VXR is at best a 300Mhz MIPS box with a 33Mhz PCI bus. Both the PC and the Linux box top out at just under 400Mbit over the main bus, but the Linux box had *alot* of CPU left over to run filters, logging, multiview BGP and CBQ. It was nice to have a box capable of BGP, OSPF, RSVP, filtering, CBQ, IP rewrites and NAT at 300Mbit+ with SSH and serial console access, costing < 10,000$USD with 2 x DS3 and 4 x 100Mbit-FDX ethernet in mid 1999, considering a 7200 cost 3 times that (with interfaces and memory), and was pretty weak as far as SSH, CBQ and NAT support went (As well as having issues with NWAY and FastEtherChannel trunking). If one is being used at the network core where filtering is not done there is some fastpath magic that can easily take the box up to about 800Mbit aggregate. Using multiport ether cards with 4 interfaces per on there own PCI sub bus it gets fun. Given the right card and driver and assuming you group your traffic it gets interesting. Only the IP headers cross the main bus, the payloads go direct card to card, if it is within the same iface group it never touches the main PCI bus. This was in late 1998. We also did some work with single and dual CPU 21264 as well as Ultra AXMP+ systems for the 64bit 66mhz PCI bus. We were very happy with the performance (1.5 - 2.0 Gbit/sec aggregate while running full filters and CBQ on a dual 21264 w/ 768 meg mem) but at the time was a bit high. These days a dual Athlon MB with 4 64bit 66Mhz PCI slots is < 350$USD... So, the easy rule? A 500Mhz *quality* PC booting from flash to ram can replace a 7206VXR. Up to quad DS3/Quad 100Mbit ether is fine. Your overall bandwidth limit is about the same, but at that bandwidth you can do a hell of alot more work (think stateful filters, CBQ, IP rewrites or IPSEC), as the limit is the PCI bus your have CPU and memory bandwidth to burn. Alot of this was R&D for product sales and ISP operations at a previous employer, and there are still boxes sitting around handling (for example) DS3 x 2 + 100Mbit x 4, 3 full views (each DS3 to seperate provider, 2 x 100Mbit-FDX EtherChannel link to a 7200 peer/backup, and 2 x 2 x 100Mbit-FDX EtherChannel link to a catalyst 2429XL for a server cluster and dialin hardware) Its 7200 peer dies now and again due to CPU overload from route flap/etc, never had any trouble with the LinuxRouter. Been in place since late 99 or so. At my current place I end up working with 2 port bandwidth controllers, and IPSEC VPN boxes. We have been known to produce a pretty slick 100Mbit full duplex bandwidth control box, as well as some neat VPN systems. These days if I want to do more than an OC3 or 2 we grab a Juniper, but if you want to do say IPSEC, a dual Athlon 2000 MP+ w/ 1G PC2100 ECC DDR and a Syskonnect 64bit/66Mhz GigE card is ~ 2,000$USD. It can do alot of work... Creating the initial distro, writing the CLI linking all the daemon config/etc and know what interrupt timers and packet timers to tweak takes skill. Just using one is easy. -- I route, therefore you are.
We had a lot of BSDI routers in past (in RELCOM, Russia); it was a good solution but there was always reliability problem: - you should use professional-grade PC which is not too chip (not brand name but something having good power supply, good and reliable fans, and so on...) - you should install everything on one PC and then _clone_ it to others. A good idea is to have a custom CD disk with everything preconfigured and collected. - moving parts such as disks is third problem. You should find some very reliable disks, even if they are not too big. We had a problem - we started from 200Mb disks, and when we began to think about replacement, we could not find anything less than 800Mb. Generally, PC based router cost much less than Cisco router, but need much more skilled people to serve it. So, it can be reasonable for the countrieas when people cost is less than in USA, and unreasonable for USA. ----- Original Message ----- From: "Scott Granados" <scott@graphidelix.net> To: "Vinny Abello" <vinny@tellurian.com> Cc: <nanog@merit.edu> Sent: Thursday, May 23, 2002 11:22 AM Subject: Re: Routers vs. PC's for routing - was list problems?
Remember that a pc may have some certain functions that are "more powerful" than a router but a pc is a much more general computer. Routers are supposed to be and usually designed to do one thing only, route, not play quake, balance your check book, browse the net, etc etc. So although for example a gsr-12000 may hhave a slower cpu than the machine on your desk it probably will route and pass more traffic than your pc ever will because of its design. Not to say you can't route well with a linux or bsd system you can but at the high-end probably not as well.
On Thu, 23 May 2002, Vinny Abello wrote:
I would have to say for any Linux/BSD platform to be a viable routing solution, you have to eliminate all moving parts or as much as possible, ie. no hard drives because hard drives will fail. Not much you can do about the cooling fans in various parts of the machine though which routers also tend to have. Solid state storage would be the way to go as far as what the OS is installed on. You have to have something to imitate flash on the common router. Otherwise, if you can get the functionality out of a PC, I say go for it! The processing power of a modern PC is far beyond any router I can think of. I suppose it would just be a matter of how efficient your kernel, TCP/IP stack and routing daemon would be at that point. :)
At 10:48 PM 5/22/2002, you wrote:
On Wed, 22 May 2002, Andy Dills wrote:
From the number of personal replies I got about these topics, it seems like many people are interested in sharing information about how to do routing on a budget, or how to avoid getting shot in the foot with your Cisco box.
Routing on a budget? Dude, you can buy a 7200 for $2 grand. Why bother with a linux box? Heh, at least use FreeBSD :)
Before the dot com implosion, they weren't nearly that inexpensive. The average corporate user will also need smartnet (what's that on a 7200, a K or a few per year?) for support, warranty, and software updates. Some people just don't appreciate being nickled and dimed by cisco and forced to either buy much more router than they need, or risk ending up with another cisco boat anchor router when the platform they chose can no longer do the job in the limited memory config supported.
I have a consulting customer who, against my strong recommendation, bought a non-cisco router to multihome with. It's PC based, runs Linux, and with the exception of the gated BGP issue that bit everyone running gated a few months ago, has worked just fine. It's not as easy to work with in most cases, but there are some definite advantages, and some things that Linux actually makes easier. They'd initially bought a 2621 when multihoming was just a thought, and by the time it was a reality, 64mb on a 2621 couldn't handle full routes. The C&W/PSI depeering (which did affect this customer, as they were single homed to C&W at the time and did regular business with networks single homed to PSI) was proof that without full routes, you're not really multihomed.
-- ---------------------------------------------------------------------- Jon Lewis *jlewis@lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
Vinny Abello Network Engineer Server Management vinny@tellurian.com (973)300-9211 x 125 (973)940-6125 (Direct)
Tellurian Networks - The Ultimate Internet Connection http://www.tellurian.com (888)TELLURIAN
participants (21)
-
Alex Rubenstein -
Alexei Roudnev -
Anthony D Cennami -
Brian -
Christopher E. Brown -
Dan Hollis -
Daryl G. Jurbala -
David Ulevitch -
Deepak Jain -
E.B. Dreger -
Jake Baillie -
Kevin Day -
Nathan Stratton -
neil＠DOMINO.ORG -
Peter van Dijk -
Randy Bush -
Scott Francis -
Scott Granados -
Steven J. Sobol -
Valdis.Kletnieks＠vt.edu -
Vinny Abello