Zebra/linux device production networking?
Greetings fellow nanogers, Long time lurker, first time poster (please, be gentle!). After looking at the archives, I didn't see this particular discussion, so here we go. First, a little background.. My CTO made my stomach curdle today when he announced that he wanted to do away with all our cisco [routers] and instead use Linux/zebra boxen. We are a small company, so naturally penny pinching is the primary motivation. That, and the sheer joy of watching me squirm. He has informed me that he has found "many people" who do this for their "core devices". I'm not so certain about this whole situation, so I humbly ask: How many of you have actually use(d) Zebra/Linux as a routing device (core and/or regional, I'd be interested in both) in a production (read: 99.999% required, hsrp, bgp, dot1q, other goodies) environment? And, if you care to spend this much time, what pitfalls/benefits did you find out about after implementation? Has there been any discussion (or musings) of moving towards such a solution? I've seen a lot of articles talking about it, but I've not actually seen many network operators chiming in. Here's the article that started it all (this was featured on /., so likely you've read it already). http://www.businessweek.com/technology/content/nov2004/tc20041129_5206_tc024... and another: http://www.networkworld.com/community/?q=node/5693 Feel free to respond off list. If anyone else is interested, I will of course summarize to list or to individuals. (ps, particulars are deliberately not included.. I'm not looking for advice, just if anyone has any solid experience with this..)
(ps, particulars are deliberately not included.. I'm not looking for advice, just if anyone has any solid experience with this..)
Unless you are absolutely certain of how routers need to work for your environment, and am willing to engineer your way out of problems, using this platform to achieve 99.x% uptime is quite not practical. Overall, this is a bad business decision, and if you quite had the clues to engineer most of the problems, you wouldn't be asking this question anyway ;) It's really a matter of lacking commercial support to route your traffic. If you can support yourself, then great, by all means go for it, and there are several operators running stable on cheap gears. If you can't support yourself, then you are opening up a can of worms. With that said, if you are looking to do one-router network for BGP, you may want to take a look at OpenBGPd, which is stable but currently lacks IGP support (though, openospfd is under works). Zebra is only stable when it's doing nothing or next to nothing. james
Linux routers are great for redundantly routing between your cable-modem and DSL at home. Using a linux router in production is a very very bad idea, although it may seem appealing to suits with no networking knowledge. I'm sure that other posters will provide you with many pages of reasons why linux routers suck, but I'll keep it short. 1. Mean Time Between Failures 2. OS exploits 3. Service/support Nick Burke wrote:
How many of you have actually use(d) Zebra/Linux as a routing device (core and/or regional, I'd be interested in both) in a production (read: 99.999% required, hsrp, bgp, dot1q, other goodies) environment?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Albert Meyer wrote:
2. OS exploits
One might argue that is an issue with any device. Cisco have their fair share of IOS updates due to security related bugs. Linux appears to have many, mostly due to the number of services that you can run. It's not like a Linux router is going to run Sendmail or Apache. David -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEhgQHTIgPQWnLowkRAt4bAKDWOP4MOu3tnxTGxZDqPY+nlmS9DgCfZ1qi M8eUX6BsNNePrtEfT88Z/Aw= =pGLM -----END PGP SIGNATURE-----
IMHO, it's a bad idea. A less intrusive alternative might be a FreeBSD based platform running Xorp/Quagga. Tiffany. On 6/6/06, Nick Burke <mrmud@mrmud.org> wrote:
Greetings fellow nanogers,
Long time lurker, first time poster (please, be gentle!).
After looking at the archives, I didn't see this particular discussion, so here we go.
First, a little background.. My CTO made my stomach curdle today when he announced that he wanted to do away with all our cisco [routers] and instead use Linux/zebra boxen. We are a small company, so naturally penny pinching is the primary motivation. That, and the sheer joy of watching me squirm. He has informed me that he has found "many people" who do this for their "core devices". I'm not so certain about this whole situation, so I humbly ask:
How many of you have actually use(d) Zebra/Linux as a routing device (core and/or regional, I'd be interested in both) in a production (read: 99.999% required, hsrp, bgp, dot1q, other goodies) environment?
And, if you care to spend this much time, what pitfalls/benefits did you find out about after implementation?
Has there been any discussion (or musings) of moving towards such a solution? I've seen a lot of articles talking about it, but I've not actually seen many network operators chiming in.
Here's the article that started it all (this was featured on /., so likely you've read it already).
http://www.businessweek.com/technology/content/nov2004/tc20041129_5206_tc024... and another: http://www.networkworld.com/community/?q=node/5693
Feel free to respond off list. If anyone else is interested, I will of course summarize to list or to individuals.
(ps, particulars are deliberately not included.. I'm not looking for advice, just if anyone has any solid experience with this..)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Nick Burke wrote:
How many of you have actually use(d) Zebra/Linux as a routing device (core and/or regional, I'd be interested in both) in a production (read: 99.999% required, hsrp, bgp, dot1q, other goodies) environment?
Sure - I've done this before. We ran 7200s on the border (DS-3 interfaces for Linux didn't make sense at the time) and Linux boxes running all these features (plus some others) on the core. Worked flawlessly and the only downtime encountered over the two years it was running was during failover which took <5sec. Of course, the time invested in building it totally offset any savings, but that particular employer considered your time to be 'free', even though you could be billing instead, but that's a whole other argument. However, if I've got a Cisco router, in my city I can easily find 20 people in half an hour who I'd trust to get into my gear and work on it. I'd find another 50 if I went out 200miles. Linux on the other hand - Maybe three, including me. State wide, probably not even 20. I'm not talking RHCE people - I'm talking about people who can really troubleshoot kernel networking issues, device driver problems and so forth. Not easily accessible (or cheap) resources. Right now I've got a pair of Linux boxes (Debian based, 2.6 kernels) running Quagga (Zebra fork - I'd recommend it over Zebra) for BGP and OSPF, pulling two full loads. HSRP is provided with LinuxVirtualServer (aka heartbeat) and I'm doing dot1q with STP. No PVST support on Linux though. It all just works. Had a memory problem on one box, which killed it, but I've had that on plenty of Cisco gear too. None of the problems have really been 'Linux' related. 99% of them are user related, in that, I set an IP wrong, or I screw up a netmask - Usual kind of junk. Basically, if you're not comfortable with the idea of it, you're not comfortable supporting it. It'll cost leaps and bounds more supporting the environment compared to Cisco hardware. I have specific Linux expertise and experience which makes me go "I can do that on Linux" and have it work without problems, but also coming from a Cisco background I know where the line between being able to prove a point and making something that is manageable comes into play. Right now we're looking at building out a small POP in another building. I'm seriously considering a pair of Linux boxes running Quagga rather than 7200s that we'd normally go with. I can easily dump 3+ full loads on them, plus I can get gig connections on PCIe without having to fork out 10 grand on a NPE-G1. Am I going to do it? No idea. Technically, there is no issue. If I drop dead the day after it's built and someone new has to maintain it, then that's a potential problem. David -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEhgdATIgPQWnLowkRAjPvAKDSoK/9kAZNjjQrix5aoMhM0v5fvACg7ilj 0fJYz8JLrH7iTjP49+XgmvE= =RAkO -----END PGP SIGNATURE-----
On Jun 6, 2006, at 4:42 PM, Nick Burke wrote:
How many of you have actually use(d) Zebra/Linux as a routing device (core and/or regional, I'd be interested in both) in a production (read: 99.999% required, hsrp, bgp, dot1q, other goodies) environment?
And, if you care to spend this much time, what pitfalls/benefits did you find out about after implementation?
We started out on a FreeBSD/Zebra routing solution for our company (content provider). While it did work acceptably for many years, it wasn't what I'd call robust. The "router" was a single P4 2.4GHz server. We had 4 GigE ports to 4 uplinks, each giving us a full BGP feed. Then two more GigE ports to our switches. We could route over 750mbps easily, without any packet loss or latency. The biggest issue we'd have was Zebra's single-threadedness. After a restart of bgpd, it would spend so much CPU time handling the BGP updates that it would get very very behind in processing BGP keepalives, and our sessions would time out before it had finished handling the initial burst. I'd have to shut down all sessions, then bring them up one at a time. It wasn't so much bgpd taking that much CPU, but bgpd not having very much left after the server was handling a few hundred mbps of traffic. Perhaps a dual CPU server would have worked better, but we never tried. There were also issues where you could get two zebra routers deadlocked - they'd both have many megabytes of BGP updates to send each other, and both would want to send a full update until completion before accepting any data in. Mucking with the kernel to allow TCP sockets to have a 16MB receive buffer helped, but still wasn't a cure. You're also giving up things like RIBs, fancy queuing/rate limiting, and any kind of hardware acceleration. Doing hundreds of megabits is easy, but software based routers seem to have trouble under DoS situations (lots of tiny packets) quicker. However, it was about as close to free as you could get. We re-used an old server, and only had to buy some 2 port ethernet cards. Support for Zebra is pretty iffy though. More often than not, I'd post a message to the Zebra mailing list to report a bug, and would get a "Yeah, known bug!" reply. The original author has all but abandoned development, leading to a fork called Quagga. Quagga is better (we still use it in a few places), but is still mostly a polished up Zebra. In the end, we needed to start pushing more traffic than we were able get our Zebra box to do. A couple 20+ minute outages during peak usage because of deadlocked bgpd processes helped my case that we needed to buy some Junipers instead. I know you're not giving specifics, but any kind of description of just how much traffic you're intending to push and how many ports you need would help in giving relevant advice. If you're talking about 1 BGP feed for 10mbps, I'd say go for it. If you're talking about a dozen sessions, and 2gbps of traffic... no way. Where you are between those is what really matters.
On Tue, 6 Jun 2006, Nick Burke wrote:
First, a little background.. My CTO made my stomach curdle today when he announced that he wanted to do away with all our cisco [routers] and instead use Linux/zebra boxen. We are a small company, so naturally penny pinching is the primary motivation. That, and the sheer joy of watching me squirm. He has informed me that he has found "many people" who do this for their "core devices". I'm not so certain about this whole situation, so I humbly ask:
How many of you have actually use(d) Zebra/Linux as a routing device (core and/or regional, I'd be interested in both) in a production (read: 99.999% required, hsrp, bgp, dot1q, other goodies) environment?
And, if you care to spend this much time, what pitfalls/benefits did you find out about after implementation? Having done exactly that previously, I wouldn't recommend it.
While it will work, most of the time, reaching 99.999% will be a challenge. Amount of engineering time you will spend in order to reach that point (and to maintain your setup) will dwarf the cost of leasing proper equipment. Issues encountered: *) Performance under ddos: Linux routing stack is route-cache-based. That means, performance is a function of flows per second, and even small random src/dst ddos will kill you. Even when this is fixed, performance will be limited by pps - and the "worst case" performance of PC router is not as impressive as "omg i can route 1gbit with p3/1ghz". In the end, "worst case" performance is what really matters, and it isn't all that awesome. *) Management: It takes certain amount of sysadmin time to manage each PC router (tools/etc). *) Integration: As it is not designed as a "complete system", you will have little wierdnesses, such as, quagga not seeing kernel-installed routes, or netlink not being able to keep up with route updates, etc. All of those are fairly small things, but there are more than enough of them. *) Troubleshooting/continuity of operations: It takes two orders of magnitude more clue to troubleshoot zebra network - there are simply *lots* more things that can possibly go wrong - you don't worry just about your links breaking, you have to worry about your software being buggy. While any CCIE will most likely be able to troubleshoot and run a cisco-based network, pool of engineers sufficiently clued in a myriad of things that relate to troubleshooting of a PC router (ie. both network engineer, system admin, protocol engineer, kernel hacker, and at times, zebra-source-code-hacker) is far smaller. *) Maturity: While it has been improving, things like Quagga have still have stability issues and "wierd issues that are resolved by killing ospfd". Because of a greater state of flux in such environment, you are likely to encounter things like "oh, this bug is fixed in latest release" - and then having to retest the new release which has completely different bugs. Yes, I know, you get that with proprietary vendors - but at least you get a benefit of *them* doing at least some amount of testing prior to release. *) Redundancy: Adding more redundancy to such a system is not likely to increase availability - in fact, it is likely to decrease availability because of added complexity and "more things to break". Your problems are not likely to be the PC losing power (complete failure). Your problem will be things like zebra's idea of routing table being different from kernel's idea, zebra being unhappy after a transit flaps sucking up CPU time, leading to other things timing out, etc. Redundancy will excarcerbate these issues, making troubleshooting *harder*. So, in conclusion, if you have a large number of clued linux hackers who have nothing better to do, it may be a good idea. Otherwise, you'll realize you are spending far more on sysadmin time than you are saving on equipment cost. -- Alex Pilosov | DSL, Colocation, Hosting Services President | alex@pilosoft.com 877-PILOSOFT x601 Pilosoft, Inc. | http://www.pilosoft.com
(resent after getting on nanog-post) On 6/6/06, Nick Burke <mrmud@mrmud.org> wrote:
How many of you have actually use(d) Zebra/Linux as a routing device (core and/or regional, I'd be interested in both) in a production (read: 99.999% required, hsrp, bgp, dot1q, other goodies) environment?
I work for a company putting together an open source router platform. (Vyatta.com) We have a linux distro that is built off of XORP, but has plenty of enhancements that make it more friendly for a typical router jockey. It has dot1q support, bgp, ospf, rip, vrrp and many other goodies. We're currently going through UNH testing of protocol conformance. We are always looking for folks to test the software out and see how it suits their needs. (or not) Caveats: 1. Keep in mind that current sever hardware won't push line rate GigE at 64-bytes, but I find it quite reasonable as a candidate for the access layer. (t1/t3 and possibly oc3 termination) So don't expect it to perform to the same level as dedicated hardware solutions. A few hundred Mbps of inet traffic (not 64 byte frames) is reasonable. 2. Keep in mind that cheap PC hardware will result in bad MTBF. Your PC router hardware should be quality gear with redundancy if you can't tolerate any downtime. We believe there's a place for open source routing platforms, but it'll take some testing from the router community to solidify and verify the stacks. Want to help? --joel
First, a little background.. My CTO made my stomach curdle today when he announced that he wanted to do away with all our cisco [routers] and instead use Linux/zebra boxen. We are a small company, so naturally penny pinching is the primary motivation.
It is primarily small companies that use zebra or Quagga or openbgpd or Xorp or the Click Modular Router project. There is more than one choice so do your research. The main drawback of all of these is that you cannot get PCI-bus cards that support some common circuit types and the PCI bus cannot handle switching high traffic volumes. Many people build and sell routers based on a PC server running UNIX. They work fine if they are no stretched beyond the role intended. Cisco routers are the same. Look at the limitations of the 2500/2600 series for instance. Some URLs of interest: http://www.read.cs.ucla.edu/click/ http://www.xorp.org/ http://www.openbgpd.org/ http://www.quagga.net/ http://www.zebra.org/
Has there been any discussion (or musings) of moving towards such a solution? I've seen a lot of articles talking about it, but I've not actually seen many network operators chiming in.
This tends to be a list focused on the cult of the BIG IRON, namely Cisco and Juniper. The people who use PC-based routers have their own hangouts. My main piece of advice is to seek out those hangouts and ask your questions there.
Here's the article that started it all (this was featured on /., so likely you've read it already).
Sorry, haven't seen these. --Michael Dillon
On Wed, 7 Jun 2006 Michael.Dillon@btradianz.com wrote:
First, a little background.. My CTO made my stomach curdle today when he announced that he wanted to do away with all our cisco [routers] and instead use Linux/zebra boxen. We are a small company, so naturally penny pinching is the primary motivation.
It is primarily small companies that use zebra or Quagga or openbgpd or Xorp or the Click Modular Router project. There is more than one choice so do your research. The main drawback of all of these is that you cannot get PCI-bus cards that support some common circuit types and the PCI bus cannot handle switching high traffic volumes.
I've talked to people using PC-based system on OC48 and analyzing that entire data. Sounded unbelievable to me but their numbers of how much data PCI(Express) can handle support that PC-based router would be able to do it. How reliable this is and if cost of supporting such router is worth going forward is another matter. Also both Linux and Freebsd are fairly equivalent as bases for such routers and if you have knowledgeable people (and you should if you're considering going with PC router), you should be able to set linux that is secure as freebsd. There are some differences in the routing code whereas Linux is designed with per-flow based switching in mind (which works very well when used as a server) and has extensive packet classification mechanism (which I strongly advise you test in the lab before trying in production). Freebsd has what I consider to be simpler code design for which many believe works better if you receive "unusual" packets, but personally I've used Linux as packet firewall at Gb rate and it handled DoS fine. Linux also supports multiple routing tables in the kernel, which I think latest quagga can take advantage of and it can make a difference whe selecting linux vs freebsd. Now do remember that biggest headache is going to be supporting this as such custom solution will require custom coding of tools and good engineer who really knows well both linux and networking and finding more such people to support your infrastructure if you grow maybe difficult. -- William Leibzon Elan Networks william@elan.net
Nick Burke wrote:
Greetings fellow nanogers,
How many of you have actually use(d) Zebra/Linux as a routing device (core and/or regional, I'd be interested in both) in a production (read: 99.999% required, hsrp, bgp, dot1q, other goodies) environment?
Just have a look for MTU. If you connect home - aDSL - someplace and your MTU is smaller than the aDSL packetsize then your connection is home - adsl - tunnel - someplace That tunnel consists of two routers, linux or whatever. Behind the tunnel you might find some 200 hosts. The speed is 2Meg through the tunnel. It used to connect one /18 and a handful of /24 The two linux boxes were maintained by a guru. They almost never gave problems. Mostly the hardware router behind that tunnel did. I dont know what kind of device it is. All I know is, it seems to know some 8 or more interfaces, hardware or virtual. The installation, a nuclear bunker, used to house some websites and services. (And an XTC-lab :) There are a lot of network bunkers arround. I guess half of them looks the same. Cheers Peter and Karin Dambier -- Peter and Karin Dambier Cesidian Root - Radice Cesidiana Graeffstrasse 14 D-64646 Heppenheim +49(6252)671-788 (Telekom) +49(179)108-3978 (O2 Genion) +49(6252)750-308 (VoIP: sipgate.de) mail: peter@peter-dambier.de mail: peter@echnaton.serveftp.com http://iason.site.voila.fr/ https://sourceforge.net/projects/iason/
On 6/7/06, Peter Dambier <peter@peter-dambier.de> wrote:
The installation, a nuclear bunker, used to house some websites and services. (And an XTC-lab :)
Ah, I sometimes wonder about how people get the idea of deploying alternate roots. Then I see that email from Peter and it all becomes blindingly clear. :) --srs -- Suresh Ramasubramanian (ops.lists@gmail.com)
On 6/7/06, Nick Burke <mrmud@mrmud.org> wrote:
First, a little background.. My CTO made my stomach curdle today when he announced that he wanted to do away with all our cisco [routers] and instead use Linux/zebra boxen.
This looks reasonable .. http://www.linux-vpn.de/lr101.php -- Suresh Ramasubramanian (ops.lists@gmail.com)
Thanks to all for all the feedback! It seems what a lot of people are saying is that it's almost acceptable (in that, you shouldn't if you can afford other devices), given the right time and engineering. The cost of supporting seems to be unanimously higher then going with a specific vendor. A number of people have noted that some of the support that the various packages of software for handling routing protocols may not play correctly with the os layer or even other packages. (IE: routing) I've seen confliction on if *bsd or linux is better, this (hopefully) isn't that surprising to anyone. The consensus is that when something breaks it takes longer to fix and requires greater technical aptitude. Finally, it appears as if, contrary to what the articles are saying, not many people are actively considering such a move. However, it is more common in smaller businesses starting new locations or building out. A lot of people seemed to of assumed the absolute worse case (which, might I add, is generally what I was looking for) scenario: a dusty box with interesting hardware out-of-the-box kernel no research a MSMD approach What about better case situations?* IE: toe cards custom kernel no moving parts (ie: hard drive, maybe fans if possible) up-to-date software packages with internal coders to fix ugly bugs, etc actual research into what packages & hardware would be best *This deviates from operational and gets into the more technical issues, so it's actually a not a question I'm looking for you kind folks to answer. But I feel I have to vindicate myself a little bit as my technical skills were called into question for even posting the original email... ;) Once again, thanks everyone!
On Wed, 7 Jun 2006, Nick Burke wrote:
What about better case situations?* IE:
toe cards custom kernel no moving parts (ie: hard drive, maybe fans if possible) up-to-date software packages with internal coders to fix ugly bugs, etc actual research into what packages & hardware would be best
I didn't notice anyone mention Imagestream, who sell Linux based routers using a custom distro and no moving parts other than fans. Storage is flash. I've helped a client manage several of them for several years. IMO, they're not bad as CPE, but I don't think we could use them if we wanted to on most of our network. Some of the features we need just aren't available. As others have mentioned, I wouldn't recommend it unless you have some people very comfortable with Linux and IP routing on Linux on staff. At one point, they had 4 full BGP feeds going into one Imagestream Gateway router, which is a P4, upgraded to 512MB RAM. With 2 full views now, they have 308MB free. It's an older installation, predating the addition of zebra/quagga to their distro, so it's still running gated_public, which works, but is fairly lacking in BGP knobs. ---------------------------------------------------------------------- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
I've seen confliction on if *bsd or linux is better, this (hopefully) isn't that surprising to anyone.
You should do a PPS throughput analysis of your own to see which OS works better on the hardware that you plan to use. Drivers, and the susceptibility of the kernel to livelock, are where there may be differences in performance.
Finally, it appears as if, contrary to what the articles are saying, not many people are actively considering such a move. However, it is more common in smaller businesses starting new locations or building out.
DEC's gateway to the Internet ran on host-based routers - DEC Alphas running Digital UNIX with turbochannel FDDI cards - from 1994 to sometime in 1999-ish (I stopped being responsible for it in 1998). I started with a pair and had suffered one all-night upgrade to eight when the PPS load of some AltaVista announcement pushed the pair over the edge into livelock.
What about better case situations?* IE:
toe cards
TOE won't help you, you aren't terminating TCP sessions on the box. At least you shouldn't be. Don't let anyone talk you into also running a web server.
custom kernel
This could be useful, if the kernel is able to handle all packet forwarding in the interrupt or polling input service routine.
no moving parts (ie: hard drive, maybe fans if possible)
That'll certainly help with reliability, as well as dual power supplies.
up-to-date software packages with internal coders to fix ugly bugs, etc actual research into what packages & hardware would be best
Both of those things, or a support agreement from one of the vendors that's trying to make the host-based open-source router business model work. Stephen
### On Wed, 07 Jun 2006 10:34:26 -0700, Nick Burke <mrmud@mrmud.org> ### casually decided to expound upon nanog@merit.edu the following thoughts ### about "Re: Zebra/linux device production networking? (summary)": NB> Finally, it appears as if, contrary to what the articles are saying, not NB> many people are actively considering such a move. However, it is more NB> common in smaller businesses starting new locations or building out. It all depends on the role in which you wish to deploy your equipment. What are your requirements? You can sit down and analyse each component for reliability, each piece of software, throw it all together and test it. Then go back and fix the issues, test, rinse, repeat. Harden what you can, improve the interface to make things more manageable, etc... Afterall, the T3 NSFNET ran on software based routers. |;^) But, at the end of the day, you need to sit back and ask yourself what the payoff is. Is building a router your business' core competency or even its core focus? Or do you simply buy one from one of the reputable companies out there who have already done the above and go about running your network? Now many people will tell you that doing a home-grown solution can save you a lot of money and for applications where you can get away with just interfaces that support T1s and metro-eth, you'll do fine by rolling your own software-based router. Afterall, most lower end Ciscos don't provide hardware accelerated forwarding anyways. However, when you weigh in the cost of time, material and labour of building your own T1 router against that of a comparable 2600/2800 class box, I'm not sure you're saving yourself a whole lot. Now FWIW, I've spent some time dealing with generic whitebox unix routers and scaling. One thing that's important is not only the number of routes but also the distribution of the prefixes. I've found that with a Pentium-III class machine with 512MB running a linux-2.4 kernel, Quagga/Zebra tends to deadlock around 80,000 routes of four full BGP peering sessions. The test was done by taking a full live Internet BGP feed from a route-collector, sending it into four linux whiteboxes running GateD where the aspaths were modified to make the routes distinct and then feeding it back towards the target. I performed the same test with GateD (commercial) on the same hardware and underlying OS and managed to scale to at least the full table (approx 150,000 routes) from each of the four peers. I've also done some more extensive testing with GateD. On a 1.8GHz Pentium-4 class machine with 2GB of memory, I was able to scale out to well over a million total routes (8 x full 150k BGP routing tables). Now bear in mind that many of these tests were performed without forwarding at line rates. That's a whole nother matter entirely and is very dependent on the OS (if software forwarding) or NP performance if accelerated hardware forwarding. Only low-rate pings were used to verify connectivity. However, I have been able to get up to DS3-level line-rate (over 100baseT) performance out of a Soekris Net4801 running FreeBSD-4.10 and commercial GateD. If you're looking to build a router on the cheap that's to be used in a relatively low-key role then those Soekris boxes are hard to beat. You can even get T1/E1 and T3/E3 cards for them in multiple configurations (singles, dual, quads, and even 8-port T1/E1). Whatever, you do, I would suggest you weigh not just the technical feasibilities but also do the financial due-diligence for the business case. You will also want to factour in things like support/maintenance, managability, skill-base of your staff, cost of development, cost of operation, etc... -- /*===================[ Jake Khuon <khuon@NEEBU.Net> ]======================+ | Packet Plumber, Network Engineers /| / [~ [~ |) | | --------------- | | for Effective Bandwidth Utilisation / |/ [_ [_ |) |_| N E T W O R K S | +=========================================================================*/
On Wed, Jun 07, 2006 at 09:31:51PM +0530, Suresh Ramasubramanian wrote:
On 6/7/06, Nick Burke <mrmud@mrmud.org> wrote:
First, a little background.. My CTO made my stomach curdle today when he announced that he wanted to do away with all our cisco [routers] and instead use Linux/zebra boxen.
This looks reasonable .. http://www.linux-vpn.de/lr101.php
LEAF <http://leaf.sourceforge.net/> and Coyote <http://www.coyotelinux.com/> are often cited live branches off the Linux Router Project. -- Joe Yao ----------------------------------------------------------------------- This message is not an official statement of OSIS Center policies.
On Tue, Jun 06, 2006 at 02:42:36PM -0700, Nick Burke <mrmud@mrmud.org> wrote a message of 39 lines which said:
How many of you have actually use(d) Zebra/Linux as a routing device
IMHO, the question is not perfectly phrased. You actually have several issues: * use a regular PC instead of big and expensive iron, * use Linux instead of FreeBSD or IOS or JunOS, * use Zebra instead of Quagga or Xorp. These questions are partly independent and should be addressed as such. For instance, Quagga + a free Unix can run on dedicated boxes like the Soekris, who have different characteristics than a regular PC (no moving parts, for instance). One last advice: be very careful when you read claims like "it may seem appealing to suits with no networking knowledge": many people never tried what they criticize, they just do not want their CEO to discover that the expensive network could have been done for much less. [I installed, in a former job, Debian + Linux + Zebra on PCs and they route fine.]
participants (17)
-
Albert Meyer -
alex＠pilosoft.com -
David Coulson -
Jake Khuon -
James -
Joel Krauska -
Jon Lewis -
Joseph S D Yao -
Kevin Day -
Michael.Dillon＠btradianz.com -
Nick Burke -
Peter Dambier -
Stephane Bortzmeyer -
Stephen Stuart -
Suresh Ramasubramanian -
Tiffany Snyder -
william(at)elan.net