Re: Vyatta as a BRAS
--- rdobbins@arbor.net wrote: When BCPs are followed, they don't tend to fall over the moment someone hits them with a few kpps of packets - which should be a key criteria for an edge device. ------------------------------------------------------- I'm guessing "a few kpps of packets" is tounge-in-cheek? Entry level script kiddies can get to a few hundred kpps easily. scott
On Jul 14, 2010, at 12:31 AM, Scott Weeks wrote:
I'm guessing "a few kpps of packets" is tounge-in-cheek? Entry level script kiddies can get to a few hundred kpps easily.
That's what I meant - even a very small botnet can easily overwhelm software-based edge routers. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
* Roland Dobbins:
That's what I meant - even a very small botnet can easily overwhelm software-based edge routers.
From or to your customers?
Stopping customer-sourced attacks is probably a good thing for the Internet at learge. And you can't combat attacks targeted at customers within your own network unless you've got very large WAN pipes, moving you into the realm of special-purpose hardware for other reasons. Previously, this was really a no-brainer because you couldn't get PCI cards with the required interfaces, but with Ethernet everywhere, the bandwidths you can handle on commodity hardware will keep increasing. Eventually, you'll need special-purpose hardware only for a smallish portion at the top of the router market, or if you can't get the software with the required protocol support on other devices. -- Florian Weimer <fweimer@bfk.de> BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99
On Jul 14, 2010, at 8:48 PM, Florian Weimer wrote:
From or to your customers?
Both.
Stopping customer-sourced attacks is probably a good thing for the Internet at learge.
Concur 100%.
And you can't combat attacks targeted at customers within your own network unless you've got very large WAN pipes, moving you into the realm of special-purpose hardware for other reasons.
Sure, you can, via S/RTBH, IDMS, et. al. While DNS reflection/amplification attacks are used to create crushing volumes of attack traffic, and even smallish botnets can create high-volume attacks, most packet-flooding attacks are predicated on throughput - i.e., pps - rather than bandwidth, and tend to use small packets. Of course, they can use *lots and lots* of small packets, and often do, but one can drop these packets via the various mechanisms one has available, then reach out to the global opsec community for filtering closer to the sources. The thing is, with many DDoS attacks, the pps/bps/cps/tps required to disrupt the targets can be quite small, due to the unpreparedness of the defenders. Many high-profile attacks discussed in the press such as the Mafiaboy attacks, the Estonian attacks, the Russian/Georgian/Azerbaijan attacks, the China DNS meltdown, and the RoK/USA DDoS attacks were all a) low-volume, b) low-throughput, c) exceedingly unsophisticated, and d) eminently avoidable via sound architecture, deployment of BCPs, and sound operational practices. In fact, many DDoS attacks are quite simplistic in nature and many are low in bandwidth/throughput; the miscreants only use the resources necessary to achieve their goals, and due to the unpreparedness of defenders, they don't have a need to make use of overwhelming and/or complex attack methodologies. This doesn't mean that high-bandwidth, high-throughput, and/or complex DDoS attacks don't occur, or that folks shouldn't be prepared to handle them; quite the opposite, we see a steady increase in attack volume, thoughput and sophistication at the high end. But the fact of the matter is that many DDoS targets - and associated network infrastructure, and services such as DNS - are surprisingly fragile, and thus are vulnerable to surprisingly simple/small attacks, or even inadvertent/accidental attacks.
Previously, this was really a no-brainer because you couldn't get PCI cards with the required interfaces, but with Ethernet everywhere, the bandwidths you can handle on commodity hardware will keep increasing.
Concur 100%.
Eventually, you'll need special-purpose hardware only for a smallish portion at the top of the router market, or if you can't get the software with the required protocol support on other devices.
I believe that the days of software-based routers are numbered, period, due to the factors you describe. Of course, the 'top of the router market' seems to keep moving upwards, despite many predictions to the contrary. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
On Wed, 14 Jul 2010 14:12:07 +0000 "Dobbins, Roland" <rdobbins@arbor.net> wrote:
On Jul 14, 2010, at 8:48 PM, Florian Weimer wrote:
From or to your customers?
Both.
Stopping customer-sourced attacks is probably a good thing for the Internet at learge.
Concur 100%.
And you can't combat attacks targeted at customers within your own network unless you've got very large WAN pipes, moving you into the realm of special-purpose hardware for other reasons.
Sure, you can, via S/RTBH, IDMS, et. al. While DNS reflection/amplification attacks are used to create crushing volumes of attack traffic, and even smallish botnets can create high-volume attacks, most packet-flooding attacks are predicated on throughput - i.e., pps - rather than bandwidth, and tend to use small packets. Of course, they can use *lots and lots* of small packets, and often do, but one can drop these packets via the various mechanisms one has available, then reach out to the global opsec community for filtering closer to the sources.
The thing is, with many DDoS attacks, the pps/bps/cps/tps required to disrupt the targets can be quite small, due to the unpreparedness of the defenders. Many high-profile attacks discussed in the press such as the Mafiaboy attacks, the Estonian attacks, the Russian/Georgian/Azerbaijan attacks, the China DNS meltdown, and the RoK/USA DDoS attacks were all a) low-volume, b) low-throughput, c) exceedingly unsophisticated, and d) eminently avoidable via sound architecture, deployment of BCPs, and sound operational practices.
In fact, many DDoS attacks are quite simplistic in nature and many are low in bandwidth/throughput; the miscreants only use the resources necessary to achieve their goals, and due to the unpreparedness of defenders, they don't have a need to make use of overwhelming and/or complex attack methodologies.
This doesn't mean that high-bandwidth, high-throughput, and/or complex DDoS attacks don't occur, or that folks shouldn't be prepared to handle them; quite the opposite, we see a steady increase in attack volume, thoughput and sophistication at the high end. But the fact of the matter is that many DDoS targets - and associated network infrastructure, and services such as DNS - are surprisingly fragile, and thus are vulnerable to surprisingly simple/small attacks, or even inadvertent/accidental attacks.
Previously, this was really a no-brainer because you couldn't get PCI cards with the required interfaces, but with Ethernet everywhere, the bandwidths you can handle on commodity hardware will keep increasing.
Concur 100%.
Eventually, you'll need special-purpose hardware only for a smallish portion at the top of the router market, or if you can't get the software with the required protocol support on other devices.
I believe that the days of software-based routers are numbered, period, due to the factors you describe. Of course, the 'top of the router market' seems to keep moving upwards, despite many predictions to the contrary.
Since specific routers have been mentioned, care to comment on the Cisco ASR? If the days of software-based routers are numbered, I'm sure Cisco would have recognised that, and not gone and developed it (or rather, bought the company that did). It seems to me that three key factors that haven't been discussed in this thread are the chances of failure, types of failure triggers and consequence of failure. It seems to have been a binary hardware = no failure, software = failure. If you put large amounts of traffic on a single router, you're likely to need a hardware router, driving up the cost, sacrificing flexibility and re-deployability, and impacting very large numbers of network users if it fails. You may not be vulnerable or as vulnerable to a DoS (software punt mentioned), but DoS's aren't the only type of failure you can suffer from. Software faults on these high end platforms can be a far more common issue within the first few years of release, because they're less widely deployed. Hardware forwarding doesn't protect you from protocol or protocol implementation vulnerabilities on the control plane, and since these are big boxes with a big consequence if they fail, they're a much larger target to aim for. OTOH, if you have options to divide the traffic load across a number of smaller routers, then you may gain the cost effectiveness of more commodity platforms (with the ultimate commodity platform being a PC acting as a router), more robustness because the platform is being used by far more people in far more environments, and less of a consequence when failures occur (DoS or not). I don't think the hardware/software augment is as simple as it is being made out to be. It is completely context dependent. Cost, availability, scalability and flexibility all need to be considered. I personally put a fair bit of weight on flexibility, because I can't tell the future, and therefore a software upgrade to an existing platform is a useful property compared to a fork lift replacement, and a box now sitting on the floor that can't be re-deployed anywhere else in the network. As long as you're aware of the associated limitations, you may be able to engineer around them, or around them enough, depending on the context.
----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
Injustice is relatively easy to bear; what stings is justice.
-- H.L. Mencken
On Jul 18, 2010, at 9:47 AM, Mark Smith wrote:
Since specific routers have been mentioned, care to comment on the Cisco ASR?
ASR1K, which is what I'm assuming you're referring to, is a hardware-based router. Same for ASR9K. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
On 18 Jul 2010, at 10:58, "Dobbins, Roland" <rdobbins@arbor.net> wrote:
ASR1K, which is what I'm assuming you're referring to, is a hardware-based router. Same for ASR9K.
My c* SE swears that the asr1k is a "software router". I didn't push him on it's architecture though. The asr9k is an npu based device - a completely different beast with different architecture / operating system / capabilities / etc. Nick
On Sun, Jul 18, 2010 at 06:12:29PM +0100, Nick Hilliard wrote:
On 18 Jul 2010, at 10:58, "Dobbins, Roland" <rdobbins@arbor.net> wrote:
ASR1K, which is what I'm assuming you're referring to, is a hardware-based router. Same for ASR9K.
My c* SE swears that the asr1k is a "software router". I didn't push him on it's architecture though.
All routers have hardware, and any but the most overwhelmingly simple "hardware" based devices are using ASICs running software to puah packets around. The line has been blurred for a long time, and the ASR1K makes it very, very blurry. It forwards packets in a relatively general-purpose (but not as general purpose as, say the Intel processors inside your servers) CPU that has 40 cores and is optimised (it's architecture, instruction set, etc.) for moving packets around. Is that hardware forwarding? Is that software forwarding? Depends in what you want to call it. Do video cards with high-end GPUs do things in "hardware" or "software"? There are now development kits to allow you to easily use those GPUs to do general purpose compute tasks. The processors in the ASR could do that, also, but Cisco hasn't written any code or released ay libraries to actually do that (at lease not publicly; I wouldn't be surprised to learn that some developer has hacked a 40-threaded SETI@Home or something like that onto it just to prove it could be done). So where do you draw the line? Is the ASR hardware forwarding? If so, would it still be hardware if, intead of the specialized processor, Cisco got Intel to develop a 40-core pentium and used that? What if Cisco instead used 10 off-the-shelf 4-core processors from Intel or AMD? Where along this continuoum do we cross the line from "software router" to "hardware router"? -- Brett
On Jul 19, 2010, at 1:55 AM, Brett Frankenberger wrote:
So where do you draw the line? Is the ASR hardware forwarding?
Yes - specialized muticore NPU plus TCAM. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
On Jul 19, 2010, at 1:12 AM, Nick Hilliard wrote:
My c* SE swears that the asr1k is a "software router". I didn't push him on it's architecture though.
Specialized multicore NPU + TCAM = hardware. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
On Sun, 18 Jul 2010 18:12:29 +0100 Nick Hilliard <nick@foobar.org> wrote:
On 18 Jul 2010, at 10:58, "Dobbins, Roland" <rdobbins@arbor.net> wrote:
ASR1K, which is what I'm assuming you're referring to, is a hardware-based router. Same for ASR9K.
My c* SE swears that the asr1k is a "software router". I didn't push him on it's architecture though.
This document supports that. If the definition of a software router is one that doesn't have a fixed at the factory forwarding function, then the ASR1K is one. http://www.cisco.com/en/US/prod/collateral/routers/ps9343/solution_overview_... The CRS-3 might be too, as they say they've added the quantumflow processor into those too.
The asr9k is an npu based device - a completely different beast with different architecture / operating system / capabilities / etc.
Nick
On Jul 19, 2010, at 5:43 AM, Mark Smith wrote:
This document supports that.
No, it doesn't. Specialized NPUs, TCAMs present in ASR1K. CRS-3 has specialized NPUs, ASICs, as well. Enough on this topic - it's obvious that both ASR1K and CRS-3 are hardware-based platforms. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
On Mon, Jul 19, 2010 at 07:13:46AM +0930, Mark Smith wrote:
This document supports that. If the definition of a software router is one that doesn't have a fixed at the factory forwarding function, then the ASR1K is one.
The code running in the ASICs on line cards in 6500-series chassis isn't fixed at the factory. Same with the code running on the PFCs in those boxes. There's not a tremendous amount of flexibility to make changes after the fact, because the code is so tightly integrated with the hardware, but there is some. (Not saying the 6500 is a software-based platform. It's pretty clearly a hardware-based platform under most peoples' definition. But: the line is blurry.) -- Brett
On Sun, Jul 18, 2010 at 8:01 PM, Brett Frankenberger <rbf+nanog@panix.com> wrote:
On Mon, Jul 19, 2010 at 07:13:46AM +0930, Mark Smith wrote:
This document supports that. If the definition of a software router is one that doesn't have a fixed at the factory forwarding function, then the ASR1K is one.
The code running in the ASICs on line cards in 6500-series chassis isn't fixed at the factory. Same with the code running on the PFCs in those boxes. There's not a tremendous amount of flexibility to make changes after the fact, because the code is so tightly integrated with the hardware, but there is some.
(Not saying the 6500 is a software-based platform. It's pretty clearly a hardware-based platform under most peoples' definition. But: the line is blurry.)
-- Brett
Surely the important point for most forwarding engines is that there is isolation between control, management and forwarding planes? If I'm looking for a box, I want line rate forwarding on all interfaces. I want stateless ACLs and policing functions on the forwarding plane. I want to use those functions to protect the control and management planes. I want the control plane to cope with the required amount of forwarding state and churn. I want the management plane to be somewhat as capable as the Linux tools I run to maintain the network. I don't honestly care whether it is a single cpu, multi-core multi-cpu, ASIC or NPU. That being said, for the networks I help maintain, the C6K meets most of those requirements. I think the N7K is movement in the right direction. I consider both to be L2/L3 switches :-) -- Tim:>
On Sun, 18 Jul 2010 21:07:36 -0400 Tim Durack <tdurack@gmail.com> wrote:
On Sun, Jul 18, 2010 at 8:01 PM, Brett Frankenberger <rbf+nanog@panix.com> wrote:
On Mon, Jul 19, 2010 at 07:13:46AM +0930, Mark Smith wrote:
This document supports that. If the definition of a software router is one that doesn't have a fixed at the factory forwarding function, then the ASR1K is one.
The code running in the ASICs on line cards in 6500-series chassis isn't fixed at the factory. Same with the code running on the PFCs in those boxes. There's not a tremendous amount of flexibility to make changes after the fact, because the code is so tightly integrated with the hardware, but there is some.
(Not saying the 6500 is a software-based platform. It's pretty clearly a hardware-based platform under most peoples' definition. But: the line is blurry.)
-- Brett
Surely the important point for most forwarding engines is that there is isolation between control, management and forwarding planes?
If I'm looking for a box, I want line rate forwarding on all interfaces. I want stateless ACLs and policing functions on the forwarding plane. I want to use those functions to protect the control and management planes. I want the control plane to cope with the required amount of forwarding state and churn. I want the management plane to be somewhat as capable as the Linux tools I run to maintain the network.
And that's the crux of the issue. Can the box survive if line rate maximum PPS is being aimed at it, either for forwarding or at the control plane? If the answer is yes, then whether it is a "software router" or "hardware router" is academic.
I don't honestly care whether it is a single cpu, multi-core multi-cpu, ASIC or NPU.
That being said, for the networks I help maintain, the C6K meets most of those requirements. I think the N7K is movement in the right direction. I consider both to be L2/L3 switches :-)
-- Tim:>
Except that the goal you set below is very very hard to do on a software router unless its CPU has packet classification properties implemented in HW. In some systems, just the act of receiving the packet in the ISR and classifying it into a bucket is enough to overwhelm the system without proper hardware assist. Bora -----Original Message----- From: Mark Smith [mailto:nanog@85d5b20a518b8f6864949bd940457dc124746ddc.nosense.org] Sent: Monday, July 19, 2010 2:39 AM To: Tim Durack Cc: NANOG list Subject: Re: Vyatta as a BRAS And that's the crux of the issue. Can the box survive if line rate maximum PPS is being aimed at it, either for forwarding or at the control plane? If the answer is yes, then whether it is a "software router" or "hardware router" is academic.
If there is sufficient CPU power (and I/O to the CPU) as compared to the bandwidth, then this is doable. Tony On Jul 19, 2010, at 11:40 PM, Akyol, Bora A wrote:
Except that the goal you set below is very very hard to do on a software router unless its CPU has packet classification properties implemented in HW.
In some systems, just the act of receiving the packet in the ISR and classifying it into a bucket is enough to overwhelm the system without proper hardware assist.
Bora
-----Original Message----- From: Mark Smith [mailto:nanog@85d5b20a518b8f6864949bd940457dc124746ddc.nosense.org] Sent: Monday, July 19, 2010 2:39 AM To: Tim Durack Cc: NANOG list Subject: Re: Vyatta as a BRAS And that's the crux of the issue. Can the box survive if line rate maximum PPS is being aimed at it, either for forwarding or at the control plane? If the answer is yes, then whether it is a "software router" or "hardware router" is academic.
On Monday, July 19, 2010 05:40:07 pm Akyol, Bora A wrote:
Except that the goal you set below is very very hard to do on a software router unless its CPU has packet classification properties implemented in HW.
And then there are Systems on a Chip (SoC) like the Realtek 8650 that really take it to another level. See http://www.csie.nctu.edu.tw/~cfliu/work/8650.htm for more information; by most definitions here this would be a SoC 'hardware' router. It's in many very low cost SoHo devices, such as NetGear FVS114.
participants (10)
-
Akyol, Bora A -
Brett Frankenberger -
Dobbins, Roland -
Florian Weimer -
Lamar Owen -
Mark Smith -
Nick Hilliard -
Scott Weeks -
Tim Durack -
Tony Li