The thing that surprises me is that there aren't any small vendors offering fairly generic routing boxes, i.e. Intel-based
imagestream does this, afaik. not too familiar with their offerings though.
I stand corrected. The following page comparing Cisco and Imagestream is quite interesting. http://www.imagestream.com/Cisco_Comparison.html How many of you would buy an Imagestream box to evaluate for your next network buildout? --Michael Dillon
I intend to give them a serious look: they sound like they could make good CPE for about 75% of my customers... (and of course, ssh v2 is a big plus :) -David Barak -Fully RFC 1925 Compliant- --- Michael.Dillon@radianz.com wrote:
http://www.imagestream.com/Cisco_Comparison.html
How many of you would buy an Imagestream box to evaluate for your next network buildout?
--Michael Dillon
===== David Barak -fully RFC 1925 compliant- __________________________________ Do you Yahoo!? Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes http://hotjobs.sweepstakes.yahoo.com/signingbonus
imagestream does this, afaik. not too familiar with their offerings though.
I stand corrected. The following page comparing Cisco and Imagestream is quite interesting.
http://www.imagestream.com/Cisco_Comparison.html
How many of you would buy an Imagestream box to evaluate for your next network buildout?
Imagestream uses Cisco list prices to sell their wares. No sane person that buys from Cisco pays list price. If one wants to complete with Cisco in a router business, they cannot claim that it costs $3411 to get 2x T1 router by Cisco or that 7507 chassis is $21,700. Alex
On Wed, Jan 14, 2004 at 09:23:35AM -0500, Alex Yuriev wrote:
Imagestream uses Cisco list prices to sell their wares. No sane person that buys from Cisco pays list price.
If one wants to complete with Cisco in a router business, they cannot claim that it costs $3411 to get 2x T1 router by Cisco or that 7507 chassis is $21,700.
That might be true, but have a look at the following article: they outperform a 26xx. (Ok, I admit, a 26xx is not a power-monster, but it's in the same price range as the small rebels)... http://www.nwfusion.com/reviews/2003/0714rev.html Kind Regards, Frank Louwers -- Openminds bvba www.openminds.be Tweebruggenstraat 16 - 9000 Gent - Belgium
In message <OF933FD648.FA437A02-ON80256E1B.003A114A-80256E1B.003A552C@radianz.c om>, Michael.Dillon@radianz.com writes:
The thing that surprises me is that there aren't any small vendors offering fairly generic routing boxes, i.e. Intel-based
imagestream does this, afaik. not too familiar with their offerings though.
I stand corrected. The following page comparing Cisco and Imagestream is quite interesting.
Hmm -- does anyone here have one? How good a job did they do locking down Linux? --Steve Bellovin, http://www.research.att.com/~smb
I stand corrected. The following page comparing Cisco and Imagestream is quite interesting.
Hmm -- does anyone here have one? How good a job did they do locking down Linux?
I have one. Two, actually. They have user-friendlied it up - you log in and you get a not-that-fancy menu interface. The router is based on gated, not zebra/quagga. It works alright, but there are a number of gaping inconsistencies with it. It's not necessarily that I have a problem with PC routers, it's that their implementation leaves some to be desired. For further information, inquiries to me off-list. Tim
On Wed, 14 Jan 2004 Michael.Dillon@radianz.com wrote:
I stand corrected. The following page comparing Cisco and Imagestream is quite interesting.
http://www.imagestream.com/Cisco_Comparison.html
How many of you would buy an Imagestream box to evaluate for your next network buildout?
I've been managing a couple of these for a customer for a couple of years. They work. The main problem I'd have with trying to use them on our network is a lack of certain features I'm either used to or totally dependent on in our ciscos. i.e. MPLSVPN (lack of it) would be a show stopper for us. The gated-public they come with lacks features...AFAIK there is no support for communities, prepending, etc. Their current software image does include zebra now, but last I looked it was not officially supported. For a relatively simple end-user BGP customer, it works fine. And the nice thing is it's PC-type hardware so if you need more RAM, just throw in another dimm. No worries about the global routing table growing and having to buy a bigger router because your year or two old one no longer supports enough memory to hold full routes. I suspect the CPUs are upgradable as well...but I've never actually touched the hardware...I've always worked on it remotely. OS-wise, it's a minimal Linux distribution with a menu interface (or you can drop to a shell) and there is a little space on the flash to add additional software if there something you want that they don't supply. ---------------------------------------------------------------------- Jon Lewis *jlewis@lewis.org*| I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On Wed, 14 Jan 2004, jlewis@lewis.org wrote:
http://www.imagestream.com/Cisco_Comparison.html
How many of you would buy an Imagestream box to evaluate for your next network buildout?
For a relatively simple end-user BGP customer, it works fine. And the nice thing is it's PC-type hardware so if you need more RAM, just throw in another dimm. No worries about the global routing table growing and having to buy a bigger router because your year or two old one no longer supports enough memory to hold full routes. I suspect the CPUs are upgradable as well...but I've never actually touched the hardware...I've always worked on it remotely.
Have been discussing PCs for a bit but as yet not deployed one, as I understand it a *nix based PC running Zebra will work pretty fine but has the constraints that: o) It has no features - not a problem for a lot of purposes This isnt necessarily a problem for what I have in mind o) On a standard PCI but your limit is about 350Mb, you can increase that to a couple of Gb using 64-bit fancy thingies For connecting to small IXPs, connecting customers, I dont need large amounts of throughput. o) This may be fixed but I found it slow to update the kernel routing table which isnt designed to take 120000 routes being added at once Icky, could perhaps cause issues if theres a major reconvergence due to an adjacent backbone router failing etc, might be okay tho o) As its entirely process based it will hurt badly in a DoS attack This is a show stopper. I need the box to stay up in an attack and be responsive to me whilst I attempt to find the source. I'm not an expert in PC hardware, so I do struggle to work out the architecture that I need and I'm sure its possible to build boxes that are optimised for this purpose however I'm still not convinced that the box can keep up with the demands of day to day packet switching - I'd like to hear otherwise tho.. has anyone deployed a PC with Zebra that could switch a few Gbs, didnt suffer from latency or jitter or fail under a DoS? Steve
: o) This may be fixed but I found it slow to update the kernel routing table : which isnt designed to take 120000 routes being added at once : : Icky, could perhaps cause issues if theres a major reconvergence due to an : adjacent backbone router failing etc, might be okay tho This is the general feeling on the Quagga list; that many limitations are not the routing daemon(s) themselves but the underlying OS and kernel. james
On Wed, 14 Jan 2004, Stephen J. Wilcox wrote:
Have been discussing PCs for a bit but as yet not deployed one, as I understand it a *nix based PC running Zebra will work pretty fine but has the constraints that:
o) It has no features - not a problem for a lot of purposes
Which "no features"? I haven't played with zebra yet, but my understanding is that it supports a large subset of the IOS BGP config language including application of route-maps to incoming/outgoing routes, and therefore things like prepending, setting metrics or preference, etc. Am I mistaken?
o) On a standard PCI but your limit is about 350Mb, you can increase that to a couple of Gb using 64-bit fancy thingies
The application where I'm caring for one of these is around a dozen T1's to several different transit providers on a Gateway router. According to Imagestream, this router can handle up to 1 OC3 at "wire speed". We're obviously not pushing anywhere near that through it. The same customer has a handful of Rebel routers used for T1s/ethernets within their network.
o) This may be fixed but I found it slow to update the kernel routing table which isnt designed to take 120000 routes being added at once
Icky, could perhaps cause issues if theres a major reconvergence due to an adjacent backbone router failing etc, might be okay tho
I've never timed it, but I haven't noticed it taking routes any slower than the ciscos I'm used to.
o) As its entirely process based it will hurt badly in a DoS attack
This is a show stopper. I need the box to stay up in an attack and be responsive to me whilst I attempt to find the source.
But it's got so much more CPU power than comparably priced ciscos...and most of the cisco gear I've worked on doesn't to terribly well under DoS...so I don't see a distinction here. Either way, getting DoS'd sucks, but I've never seen a DoS hit any of the Imagestreams, so I don't know how it copes.
I'm not an expert in PC hardware, so I do struggle to work out the architecture that I need and I'm sure its possible to build boxes that are optimised for this purpose however I'm still not convinced that the box can keep up with the demands of day to day packet switching - I'd
Their bigger routers, I'm pretty sure, have multiple PCI buses, so if you wanted to push lots of traffic, careful planning of which bus you put each card in may make a difference. Their tech support is pretty responsive, so they'd be the place to go with technical/architectural questions. Another nice feature is with iptables, they can now do stateful firewalling / connection tracking. ---------------------------------------------------------------------- Jon Lewis *jlewis@lewis.org*| I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On 14 Jan 2004, at 17:49, jlewis@lewis.org wrote:
On Wed, 14 Jan 2004, Stephen J. Wilcox wrote:
Have been discussing PCs for a bit but as yet not deployed one, as I understand it a *nix based PC running Zebra will work pretty fine but has the constraints that:
o) It has no features - not a problem for a lot of purposes
Which "no features"? I haven't played with zebra yet, but my understanding is that it supports a large subset of the IOS BGP config language including application of route-maps to incoming/outgoing routes, and therefore things like prepending, setting metrics or preference, etc. Am I mistaken?
It is my impression that Zebra is pretty feature-rich. There are some things that are difficult for Zebra to do since they relate to (absent) capabilities in the host kernel, though; RFC 2385 requires the host to support the TCP MD5 Signature option, for example, and most do not. Joe
There is one more interesting problem. Let's, say, you install PC with ZEBRA and have all 120,000 prefixes. Internet is _internet_, sometimes people make a crazy things, and create a bad (misconfigured, or very long, or very unusual) announces. Some announces are fatal for Cisco IOS, some for Zebra, some for WellFleet (do someone remember it? Very big competitor -:)). Now, say, announce A crash Cisco IOS. 99.9% Internet backbones are Ciscos, so this announce breaks few Ciscos around and die - so you never know about it (and will not have a chance to be happy that _this announce crash Ciscos but do not crash ZEBRA). Not bad, of course - you are alive, all Internet is alive. Now, say, announce B crash ZEBRA (and do not crash Cisco). It will spread until it reach first ZEBRA on it;'s road - _your_ ZEBRA. So all Zerbras in Internet crash at once (and you are unhappy). It is not a joke - we had such scenario few years ago (it was 'gated vs Cisco and WellFreet vs Cisco'). And such scenario make Juniper back-bone a little dangerous (but I believe that JUNIPER debugged such problems long ago, so it is not a case today).
: Which "no features"? I haven't played with zebra yet, but my : understanding is that it supports a large subset of the IOS BGP config : language including application of route-maps to incoming/outgoing routes, : and therefore things like prepending, setting metrics or preference, etc. : Am I mistaken? Yes, all of that is supported & more: zebra(config-router)# neighbor 1.1.1.1 advertisement-interval Minimum interval between sending BGP routing updates interface Interface peer-group Member of the peer-group port Neighbor's BGP port strict-capability-match Strict capability negotiation match timers BGP per neighbor timers transparent-as Do not append my AS number even peer is EBGP peer transparent-nexthop Do not change nexthop even peer is EBGP peer version Neighbor's BGP version activate Enable the Address Family for this Neighbor allowas-in Accept as-path with my AS present in it attribute-unchanged BGP attribute is propagated unchanged to this neighbor capability Advertise capability to the peer default-originate Originate default route to this neighbor description Neighbor specific description distribute-list Filter updates to/from this neighbor dont-capability-negotiate Do not perform capability negotiation ebgp-multihop Allow EBGP neighbors not on directly connected networks enforce-multihop Enforce EBGP neighbors perform multihop filter-list Establish BGP filters local-as Specify a local-as number maximum-prefix Maximum number of prefix accept from this peer next-hop-self Disable the next hop calculation for this neighbor override-capability Override capability negotiation result passive Don't send open messages to this neighbor prefix-list Filter updates to/from this neighbor remote-as Specify a BGP neighbor remove-private-AS Remove private AS number from outbound updates route-map Apply route map to neighbor route-reflector-client Configure a neighbor as Route Reflector client route-server-client Configure a neighbor as Route Server client send-community Send Community attribute to this neighbor shutdown Administratively shut down this neighbor soft-reconfiguration Per neighbor soft reconfiguration unsuppress-map Route-map to selectively unsuppress suppressed routes update-source Source of routing updates weight Set default weight for routes from this neighbor James Edwards Routing and Security jamesh@cybermesa.com At the Santa Fe Office: Internet at Cyber Mesa Store hours: 9-6 Monday through Friday 505-988-9200 SIP:1(747)669-1965
Have been discussing PCs for a bit but as yet not deployed one, as I understand it a *nix based PC running Zebra will work pretty fine but has the constraints that:
o) It has no features - not a problem for a lot of purposes
This isnt necessarily a problem for what I have in mind It depends. Zebra/Quagga has lots of features, it just may be that these aren't the features you want. At many cases, you can get a developer to implement the features you need for half the cost of a proprietary router.
I would add, more importantly for nanog audience: o) lack of unified tools to configure and manage: Your average PC router is configured at least by: * your distribution-based startup scripts * your routing protocol daemon (gated/zebra/etc) * linecard-specific tools (ethtool for linux) * protocol-specific tools (br2684 for rfc1483 encaps, for example) * eb/iptables to configure ACLs (or ipfw/ipf/pf) * tc to configure QoS (or ALTQ) Each of those tools has varied degrees of documentation, different configuration interface, vastly different 'status' interface, different support mailing lists, etc. It is much easier for a given organization to find a cisco/juniper/etc expert than to find someone who's experienced with Linux/FreeBSD network toolchain, and I don't foresee that changing anytime soon.
o) On a standard PCI but your limit is about 350Mb, you can increase that to a couple of Gb using 64-bit fancy thingies
For connecting to small IXPs, connecting customers, I dont need large amounts of throughput. 64/66 PCI hasn't been fancy for last 3 years.
On a single-processor P4/3ghz, I already can (and do:) route 400mbps of DoS-like traffic (one packet per flow, small packets, 400kpps). Of course, this is ridiculously low compared to current generation of high-end routers, however, it has its niche (and see below for possible scaling).
o) This may be fixed but I found it slow to update the kernel routing table which isnt designed to take 120000 routes being added at once
Icky, could perhaps cause issues if theres a major reconvergence due to an adjacent backbone router failing etc, might be okay tho Actually, considering the CPU on common desktop and CPU on a RE on common router (aka "you are reading this email on a machine with faster CPU than fastest RE in your network"), PCs can do BGP much faster than "hardware-based" routers (aka "forwarding ASICs don't run BGP"). As result, BGP Zebra/Linux can take 100k routes in <10 seconds (haven't measured exactly).
o) As its entirely process based it will hurt badly in a DoS attack
This is a show stopper. I need the box to stay up in an attack and be responsive to me whilst I attempt to find the source. Well, its not "process based", but it *is* "flow based". Yes, performance suffers as packets/flow rate decreases, however, it doesn't suffer as bad as other flow-based devices.
I'm not an expert in PC hardware, so I do struggle to work out the architecture that I need and I'm sure its possible to build boxes that are optimised for this purpose however I'm still not convinced that the box can keep up with the demands of day to day packet switching - I'd like to hear otherwise tho.. has anyone deployed a PC with Zebra that could switch a few Gbs, didnt suffer from latency or jitter or fail under a DoS? It is not gbps that kill you, it is the pps (and/or new-flows-per-second).
Getting to 1mpps on a single router today will probably be hard. However, I've been considering implementing a "clustered router" architecture, should scale pps more or less linearly based on number of "PCs" or "routing nodes" involved. I'm not sure if discussion of that is on-topic here, so maybe better to take it offline.
On Wed, 14 Jan 2004 alex@pilosoft.com wrote:
Getting to 1mpps on a single router today will probably be hard. However, I've been considering implementing a "clustered router" architecture, should scale pps more or less linearly based on number of "PCs" or "routing nodes" involved. I'm not sure if discussion of that is on-topic here, so maybe better to take it offline.
This is exactly what Pluris PC-based proof-of-concept prototype did in 97. PCs were single-board 133MHz P-IIs, running custom forwarding code on bare metal, yielding about 120kpps per board, or 1.9Mpps per cage. In the production box CPU-based forwarding was replaced with ASICs, 1Gbps hybrid optical/electrical butterfly/hypercube interconnect was replaced with 12Gbps optical hypercube interconnect, otherwise architecture was unchanged. That was a total overkill which was one of the reasons the company went down. --vadim
On 15-Jan-04 Unnamed Administration sources reported Vadim Antonov said :
On Wed, 14 Jan 2004 alex@pilosoft.com wrote:
Getting to 1mpps on a single router today will probably be hard. However, I've been considering implementing a "clustered router" architecture, should scale pps more or less linearly based on number of "PCs" or "routing nodes" involved. I'm not sure if discussion of that is on-topic here, so maybe better to take it offline.
This is exactly what Pluris PC-based proof-of-concept prototype did in 97. PCs were single-board 133MHz P-IIs, running custom forwarding code on bare metal, yielding about 120kpps per board, or 1.9Mpps per cage.
In the production box CPU-based forwarding was replaced with ASICs, 1Gbps hybrid optical/electrical butterfly/hypercube interconnect was replaced with 12Gbps optical hypercube interconnect, otherwise architecture was unchanged. That was a total overkill which was one of the reasons the company went down.
--vadim
I used to work with an Ascend GRF (goes real fast) Router that was nothing more than a hacked BSD os running on a hard drive at first then they moved to a flash card that controlled some custom switching hardware. But all the functions were via the BSD os and I think it just used Gated. Sounds very similiar. Nicole |\ __ /| (`\ | o_o |__ ) ) // \\ - nicole@daemontech.com - Powered by FreeBSD - ------------------------------------------------------ " Daemons" will now be known as "spiritual guides" -Politically Correct UNIX Page "Witchcraft is in essence the worship of the powers of this world, beautiful and terrible, but all in a circle under the turning sky that is the One." -C.A. Burland, "Echoes of Magic" "Connecting with energy is something humans have to be open to and talking about and expecting, otherwise the whole human race can go back to pretending that life is about power over others and exploiting the planet. If we go back to doing this, then we won't survive." -James Redfield, "The Celestine Prophecy"
Yep, that describes the old GRF400/800 to a T. It was gated. On Thu, 15 Jan 2004, Nicole wrote:
I used to work with an Ascend GRF (goes real fast) Router that was nothing more than a hacked BSD os running on a hard drive at first then they moved to a flash card that controlled some custom switching hardware. But all the functions were via the BSD os and I think it just used Gated.
Sounds very similiar.
Nicole
|\ __ /| (`\ | o_o |__ ) ) // \\ - nicole@daemontech.com - Powered by FreeBSD - ------------------------------------------------------ " Daemons" will now be known as "spiritual guides" -Politically Correct UNIX Page
"Witchcraft is in essence the worship of the powers of this world, beautiful and terrible, but all in a circle under the turning sky that is the One." -C.A. Burland, "Echoes of Magic"
"Connecting with energy is something humans have to be open to and talking about and expecting, otherwise the whole human race can go back to pretending that life is about power over others and exploiting the planet. If we go back to doing this, then we won't survive." -James Redfield, "The Celestine Prophecy"
-- /-------------------------------------------------> Marius Strom | Always carry a short length of fibre-optic cable. Professional Geek | If you get lost, then you can drop it on the System/Network Admin | ground, wait 10 minutes, and ask the backhoe http://www.marius.org/ | operator how to get back to civilization. \-------------| Mike Andrews |-------------------->
The GRFs started with gated, but throughout the time they were an Ascend product the code base moved farther and farther away from that. Unfortunately, the result wasn't ever quite ready for production use, though not through any lack of effort on the part of the Ascend GRF guys. Fortunately many have moved on to bigger and better router projects.
I used to work with an Ascend GRF (goes real fast) Router that was nothing more than a hacked BSD os running on a hard drive at first then they moved to a flash card that controlled some custom switching hardware.
yes, we tried those in beta. literally went up in flames, yes real flames. one of the more exciting routers made from washing machine parts i have ever seen. randy
yes, we tried those in beta. literally went up in flames, yes real flames. one of the more exciting routers made from washing machine parts i have ever seen.
We also used them but the number of issues in keeping the cards routeing tables in sync just made them too unreliable.
As I remember, it used commercial gated. ----- Original Message ----- From: "Nicole" <nmh@daemontech.com> To: "Vadim Antonov" <avg@kotovnik.com> Cc: <nanog@merit.edu>; <alex@pilosoft.com> Sent: Thursday, January 15, 2004 7:02 PM Subject: Re: PC Routers (was Re: /24s run amuck)
On 15-Jan-04 Unnamed Administration sources reported Vadim Antonov said :
On Wed, 14 Jan 2004 alex@pilosoft.com wrote:
Getting to 1mpps on a single router today will probably be hard.
I've been considering implementing a "clustered router" architecture, should scale pps more or less linearly based on number of "PCs" or "routing nodes" involved. I'm not sure if discussion of that is on-topic here, so maybe better to take it offline.
This is exactly what Pluris PC-based proof-of-concept prototype did in
However, 97.
PCs were single-board 133MHz P-IIs, running custom forwarding code on bare metal, yielding about 120kpps per board, or 1.9Mpps per cage.
In the production box CPU-based forwarding was replaced with ASICs, 1Gbps hybrid optical/electrical butterfly/hypercube interconnect was replaced with 12Gbps optical hypercube interconnect, otherwise architecture was unchanged. That was a total overkill which was one of the reasons the company went down.
--vadim
I used to work with an Ascend GRF (goes real fast) Router that was nothing more than a hacked BSD os running on a hard drive at first then they moved to a flash card that controlled some custom switching hardware. But all the functions were via the BSD os and I think it just used Gated.
Sounds very similiar.
Nicole
|\ __ /| (`\ | o_o |__ ) ) // \\ - nicole@daemontech.com - Powered by FreeBSD - ------------------------------------------------------ " Daemons" will now be known as "spiritual guides" -Politically Correct UNIX Page
"Witchcraft is in essence the worship of the powers of this world, beautiful and terrible, but all in a circle under the turning sky that is the One." -C.A. Burland, "Echoes of Magic"
"Connecting with energy is something humans have to be open to and talking about and expecting, otherwise the whole human race can go back to pretending that life is about power over others and exploiting the planet. If we go back to doing this, then we won't survive." -James Redfield, "The Celestine Prophecy"
Does any one in this group have a comment/view of the TippingPoint product line? Replies off list are encouraged. I can make a digest of the replies and post the consolidated replies so as to save clutter if anyone would like. Thanks in advance and Happy New Year Chris
On Wed, 14 Jan 2004, Stephen J. Wilcox wrote:
o) On a standard PCI but your limit is about 350Mb, you can increase that to a couple of Gb using 64-bit fancy thingies
The limit is not megabit/s, it's packet per second (or rather, interrupts per second). I-mix the average might be 350 megabit/s on a fairly good PC, but going PCI-X wont help you much there. -- Mikael Abrahamsson email: swmike@swm.pp.se
On Thu, Jan 15, 2004 at 04:34:00AM +0100, Mikael Abrahamsson wrote:
On Wed, 14 Jan 2004, Stephen J. Wilcox wrote:
o) On a standard PCI but your limit is about 350Mb, you can increase that to a couple of Gb using 64-bit fancy thingies
The limit is not megabit/s, it's packet per second (or rather, interrupts per second). I-mix the average might be 350 megabit/s on a fairly good PC, but going PCI-X wont help you much there.
I also think that it is extremely important to seperate "what you can do with a redhat cd and a dream" from "what someone can do with PC hardware". The bottom line is: You are only going to get so much performance when you forward packets through a box which is processing an interrupt per packet, doing a patricia tree lookup per packet, copying the packet in memory a couple times, and doing some sequential comparisons through a firewall ruleset on every packet. None of the above has anything to do with PC hardware, but rather the poor software that people currently making "PC routers" choose to run. If someone were to take *half* the software innovations which have been made over the past 15 years (a decent fib, interrupt coalescing, compiled packet matching rulesets, etc) and applied them as if they knew something about networking and coding, they could very easily produce a box using off the shelf PC hardware which woops up on a 7206vxr for somewhere less than $2000. If there is one thing PC hardware is good at, it is getting faster fast enough to keep up with the amount of bad code people keep churning out. :) Of course, then they would probably need to know a little bit more about routing protocols than just "how to compile zebra", but assuming they did that too... They would be bought by Cisco. :) Anything else is either a cute playtoy for your house, or an endless source of laughter for the people who know better as they watch you work away at it. The vast majority of this discussion falls into the latter category, but after a while even this gem of a subject turns from funny to just plain sad. :) -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
I also think that it is extremely important to seperate "what you can do with a redhat cd and a dream" from "what someone can do with PC hardware". Absolutely correct ;)
The bottom line is: You are only going to get so much performance when you forward packets through a box which is processing an interrupt per packet, doing a patricia tree lookup per packet, copying the packet in memory a couple times, and doing some sequential comparisons through a firewall ruleset on every packet. None of the above has anything to do with PC hardware, but rather the poor software that people currently making "PC routers" choose to run.
If someone were to take *half* the software innovations which have been made over the past 15 years (a decent fib, interrupt coalescing, compiled packet matching rulesets, etc) and applied them as if they knew something about networking and coding, they could very easily produce a box using off the shelf PC hardware which woops up on a 7206vxr for somewhere less than $2000. If there is one thing PC hardware is good at, it is getting faster fast enough to keep up with the amount of bad code people keep churning out. :) Of course, then they would probably need to know a little bit more about routing protocols than just "how to compile zebra", but assuming they did that too... They would be bought by Cisco. :) You may find it interesting that both Linux and FreeBSD now have interrupt coalescing, and www.hipac.org is building a compiled ruleset.
As far as broken-ness of linux rib/route lookup code: Yes, it is so very 1985, but there may be changes coming soon [Pilosoft may be sponsoring a rewrite].
Anything else is either a cute playtoy for your house, or an endless source of laughter for the people who know better as they watch you work away at it. The vast majority of this discussion falls into the latter category, but after a while even this gem of a subject turns from funny to just plain sad. :) ...Until they get bought by Cisco? ;)
Date: Wed, 14 Jan 2004 23:16:22 -0500 (EST) From: alex@...
You may find it interesting that both Linux and FreeBSD now have interrupt coalescing, and www.hipac.org is building a compiled ruleset.
grep usec_delay /sys/most/any/nic/driver/*.c Eddy -- Brotsman & Dreger, Inc. - EverQuick Internet Division Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita _________________________________________________________________ DO NOT send mail to the following addresses : blacklist@brics.com -or- alfra@intc.net -or- curbjmp@intc.net Sending mail to spambait addresses is a great way to get blocked.
This year is the 10 year aniversary of Demon using NetBSD/GateD to talk BGP4 to Sprint, Pipex, JANET and GBNet on Sparc IPX and i486/DX2/66 boxes, 20,000 routes at the time as I recall. [10,000 new routes a year ?] PC's as routers is a good way to save a few pounds [dollars!] only if you don't expect ever to need more than about 100M - 200M of traffic through the box and this number is highly variable depending on the packet size and number of packets. When PCs are pushing alot of traffic Gaming type applications suffer really badly. But for a small organisation who just wants a cheap way of talking BGP4 to an upstream its a great solution. The issues that you hit tend to be maintaining the boxes well. If you have a Unix team already supporting Linux or BSD then this shouldn't be a large amount of extra work - you also need a decent test rig to test new versions of things, but that is true of any platform. You still get hit with the usual PC issues, disk drive failures occur and wierdness around disks and filesystems happen. If your PC router crashes reboots and decides to delete the inodes for your serial ports that connect your box to the Internet during fsck its a major annoyance and it usually happens 2 bottles of beer into a Friday night. Yes there are issues with flash cards but these are much more manageable. If you don't have a good unix team don't even think about doing this.
o) It has no features - not a problem for a lot of purposes
I don't think thats true. What features do you need?
o) On a standard PCI but your limit is about 350Mb, you can increase that to a couple of Gb using 64-bit fancy thingies
If you stick to ethernet but I've found that you run into other issues when you use gige [dodgy motherboards and hardware slow ram etc]. One motherboard manufacturer that I've found that is very good is ASUS but they haven't done too much 64bit wise.
o) This may be fixed but I found it slow to update the kernel routing table which isnt designed to take 120000 routes being added at once
Not my experience but I'd say that this is true with other platforms.
Icky, could perhaps cause issues if theres a major reconvergence due to an adjacent backbone router failing etc, might be okay tho
Alot of people don't need the full routeing table. If you are smart you should ask your providers to announce their own internal routes and a default route. Use those routes so that traffic to Provider A goes via Provider A and the rest really doesn't matter in most cases.
o) As its entirely process based it will hurt badly in a DoS attack
That certainly isn't true and will depend on the OS and the way you have set it up. It is possible to compile PPP [etc] into the kernel and run them in kernel space, I found this to be a requirement on E1 serial drivers and I would expect the same to be true of higher bandwidth drivers.
This is a show stopper. I need the box to stay up in an attack and be responsive to me whilst I attempt to find the source.
I'm not an expert in PC hardware, so I do struggle to work out the architecture that I need and I'm sure its possible to build boxes that are optimised for this purpose however I'm still not convinced that the box can keep up with the demands of day to day packet switching - I'd like to hear otherwise tho.. has anyone deployed a PC with Zebra that could switch a few Gbs, didnt suffer from latency or jitter or fail under a DoS?
I doubt it, but the fact is the other major routeing vendors haven't solved this either! Regards, Neil.
participants (22)
-
Alex Yuriev -
alex＠pilosoft.com -
Alexei Roudnev -
Christopher Bird -
David Barak -
E.B. Dreger -
Frank Louwers -
james -
jlewis＠lewis.org -
jmalcolm＠uraeus.com -
Joe Abley -
Marius Strom -
Michael.Dillon＠radianz.com -
Mikael Abrahamsson -
neil＠DOMINO.ORG -
Nicole -
Randy Bush -
Richard A Steenbergen -
Stephen J. Wilcox -
Steven M. Bellovin -
Timothy Brown -
Vadim Antonov