Does anybody currently use vyatta as a bgp router for their company? If so have you ran into any problems with using that instead of a cisco or juniper router?
On Sep 13, 2011, at 1:42 AM, Ben Albee wrote:
Does anybody currently use vyatta as a bgp router for their company?
The days of public-facing software-based routers were over years ago - you need an ASIC-based edge router, else you'll end up getting zorched. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> The basis of optimism is sheer terror. -- Oscar Wilde
-----Original Message----- From: Dobbins, Roland [mailto:rdobbins@arbor.net] Sent: Monday, September 12, 2011 11:56 AM To: North American Network Operators' Group Subject: Re: vyatta for bgp
On Sep 13, 2011, at 1:42 AM, Ben Albee wrote:
Does anybody currently use vyatta as a bgp router for their company?
The days of public-facing software-based routers were over years ago - you need an ASIC-based edge router, else you'll end up getting zorched.
How do you come to this conclusion? I think a software-based router for enterprise level (let's say on the 1G per provider level) can handle a fair amount of zorching. I checked the Cisco and Juniper docs and neither vendor is anywhere near releasing their anit-zorching ASICs. Mike
On 12/09/2011 20:08, Michael K. Smith - Adhost wrote:
How do you come to this conclusion? I think a software-based router for enterprise level (let's say on the 1G per provider level) can handle a fair amount of zorching.
I presume by "a fair amount", I presume you mean "barely any"? At large packet sizes, an "enterprise level" router will just about handle a 1G DoS attack. Thing is, bandwidth DoS / DDoS is sufficiently easy to pull off on a large scale that a 1G DoS is pretty easy. Incidentally, most service providers use "enterprise level" as a by-word for mediocre quality kit, lacking in both stability and useful features. Nick
On Sep 12, 2011, at 12:35 PM, Nick Hilliard wrote:
On 12/09/2011 20:08, Michael K. Smith - Adhost wrote:
How do you come to this conclusion? I think a software-based router for enterprise level (let's say on the 1G per provider level) can handle a fair amount of zorching.
I presume by "a fair amount", I presume you mean "barely any"?
At large packet sizes, an "enterprise level" router will just about handle a 1G DoS attack. Thing is, bandwidth DoS / DDoS is sufficiently easy to pull off on a large scale that a 1G DoS is pretty easy.
Incidentally, most service providers use "enterprise level" as a by-word for mediocre quality kit, lacking in both stability and useful features.
Nick
In your typical enterprise environment, a 1G DoS will zorch the link long before it zorches the router at the enterprise side. I agree that software-based routers are not a good choice for a backbone provider, but, for an enterprise that is dealing with <1gbps links coming in from ≤3 providers, the difference in cost makes a software router an attractive option in many cases. Of course it is important to understand the limitations of the solution you choose, but, in such an environment, a USD100,000+ ASIC based router may be like trying to kill a mosquito with a sledge hammer. Owen
On Sep 13, 2011, at 2:45 AM, Owen DeLong wrote:
In your typical enterprise environment, a 1G DoS will zorch the link long before it zorches the router at the enterprise side.
This contradicts my experience - I've repeatedly witnessed only a few mb/sec of 64-byte packets making software-based routers fall over, including just last month. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> The basis of optimism is sheer terror. -- Oscar Wilde
On Mon, 12 Sep 2011 20:12:43 -0000, "Dobbins, Roland" said:
This contradicts my experience - I've repeatedly witnessed only a few mb/sec of 64-byte packets making software-based routers fall over, including just last month.
On the flip side, there's a *lot* of sites that have to make trade-offs, and the risk that their $10K software-based router may fall over doesn't justify adding another zero to the price tag, especially if their network includes a lot of branch offices that would all add another zero....
On Mon, Sep 12, 2011 at 5:12 PM, Dobbins, Roland <rdobbins@arbor.net> wrote:
On Sep 13, 2011, at 2:45 AM, Owen DeLong wrote:
In your typical enterprise environment, a 1G DoS will zorch the link long before it zorches the router at the enterprise side.
This contradicts my experience - I've repeatedly witnessed only a few mb/sec of 64-byte packets making software-based routers fall over, including just last month.
Would Cisco ISR G2 3925E classify as software-based router? Expected NDR performance is about 1845 pps (64-byte packets). That should deliver room for some 100s of Mbps. Do you expect it to bend itself down under a few Mbps of 64-byte packets? http://www.anticisco.ru/pubs/ISR_G2_Perfomance.pdf Everton
On Sep 13, 2011, at 3:43 AM, Everton Marques wrote:
Would Cisco ISR G2 3925E classify as software-based router?
Yes.
Do you expect it to bend itself down under a few Mbps of 64-byte packets?
Especially if they're directed at the router itself, at some point, sure - though the ISR2 certainly has more horsepower than the original ISRs, and I've personally yet to witness an ISR2 being DDoSed, so I've no feel for the specific numbers. Features also play a role. This isn't to say that the ISR2 isn't a fine router - but rather that one must be cognizant of performance envelopes prior to deployment in order to determine suitability to purpose. One can't reasonably expect vendors to exceed their design constraints in any type of equipment. ;> One can and should test the specific performance envelope of any prospective infrastructure purchase, of course. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> The basis of optimism is sheer terror. -- Oscar Wilde
On Mon, Sep 12, 2011 at 1:52 PM, Dobbins, Roland <rdobbins@arbor.net> wrote:
On Sep 13, 2011, at 3:43 AM, Everton Marques wrote:
Would Cisco ISR G2 3925E classify as software-based router?
Yes.
Do you expect it to bend itself down under a few Mbps of 64-byte packets?
Especially if they're directed at the router itself, at some point, sure - though the ISR2 certainly has more horsepower than the original ISRs, and I've personally yet to witness an ISR2 being DDoSed, so I've no feel for the specific numbers. Features also play a role.
This isn't to say that the ISR2 isn't a fine router - but rather that one must be cognizant of performance envelopes prior to deployment in order to determine suitability to purpose. One can't reasonably expect vendors to exceed their design constraints in any type of equipment.
;>
One can and should test the specific performance envelope of any prospective infrastructure purchase, of course.
----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
The basis of optimism is sheer terror.
-- Oscar Wilde
Lots of devices can have trouble if you direct high PPS to the control plane, and will exhibit performance degradation, leading up to a DoS eventually. That isn't limited to software based routers at all, it will impact dedicated ASICs. Vendors put together solutions for this, to protect the router itself/control plane, whether its a software based routed or ASICs. Now if this was a Microtik with an 1Ghz Intel Atom CPU, sure, lots of things could take that thing offline, even funny looks. But a modern, multi-core/multi-thread system with multi-queued NICs will handle hundreds of thousands of PPS directed to the router itself before having issues, of nearly any packet size. A high end ASIC can handle millions/tens of millions PPS, but directed to the control plane (which is often a general purpose CPU as well, Intel or PowerPC), probably not in most scenarios. I think its very fair for a small/medium sized organization to run software based routers, Vyatta included. -- Brent Jones brent@servuhome.net
On Sep 13, 2011, at 4:13 AM, Brent Jones wrote:
A high end ASIC can handle millions/tens of millions PPS, but directed to the control plane (which is often a general purpose CPU as well, Intel or PowerPC), probably not in most scenarios.
CoPP. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> The basis of optimism is sheer terror. -- Oscar Wilde
Brent, On Mon, Sep 12, 2011 at 11:13 PM, Brent Jones <brent@servuhome.net> wrote:
Lots of devices can have trouble if you direct high PPS to the control plane, and will exhibit performance degradation, leading up to a DoS eventually. That isn't limited to software based routers at all, it will impact dedicated ASICs. Vendors put together solutions for this, to protect the router itself/control plane, whether its a software based routed or ASICs. Now if this was a Microtik with an 1Ghz Intel Atom CPU, sure, lots of things could take that thing offline, even funny looks. But a modern, multi-core/multi-thread system with multi-queued NICs will handle hundreds of thousands of PPS directed to the router itself before having issues, of nearly any packet size. A high end ASIC can handle millions/tens of millions PPS, but directed to the control plane (which is often a general purpose CPU as well, Intel or PowerPC), probably not in most scenarios.
I think its very fair for a small/medium sized organization to run software based routers, Vyatta included.
Speaking of Mikrotik there, I recently pushed 350kpps small packets through an x86 routeros image running under kvm (using vt-d for nic) on my desktop machine (which is a number i seem to run into more than once when it comes to linux/linux-derivative forwarding on single queue & core). I saw a release note claiming their next sw release will do 15-20% more on both mips and x86. Unsurprisingly is open source software forwarding very far from 10G linerate of small pps through single cpu core still. 350kpps of 64B packets is of course merely 180 Mbps (notably, actually sufficient for handling incoming small packets on a 100 Mbps uplink). Re adversaries or random scum filling your uplinks with useless bits, I think I hear the largest DDoS'es now have filled 100G links, so.. don't make yourself a packeting target if you happen to run smaller links than that? :) Generally on staying alive through DDoS by anything else than some degree of luck, I guess having more bandwith between your network and your peers than what your peers all have to their peers is advised (the statement could possibly be improved upon using some minimum cut graph theory language). Best, Martin
On 9/12/2011 3:12 PM, Dobbins, Roland wrote:
On Sep 13, 2011, at 2:45 AM, Owen DeLong wrote:
In your typical enterprise environment, a 1G DoS will zorch the link long before it zorches the router at the enterprise side. This contradicts my experience - I've repeatedly witnessed only a few mb/sec of 64-byte packets making software-based routers fall over, including just last month.
----------------------------------------------------------------------- Roland Dobbins<rdobbins@arbor.net> //<http://www.arbornetworks.com>
The basis of optimism is sheer terror.
-- Oscar Wilde
+1 tv
On 12/09/2011 20:45, Owen DeLong wrote:
In your typical enterprise environment, a 1G DoS will zorch the link long before it zorches the router at the enterprise side.
It sure will, unless you have multiple 1G links into your router, in which case the ddos will effectively trash all the links.
I agree that software-based routers are not a good choice for a backbone provider, but, for an enterprise that is dealing with <1gbps links coming in from ≤3 providers, the difference in cost makes a software router an attractive option in many cases.
Of course it is important to understand the limitations of the solution you choose, but, in such an environment, a USD100,000+ ASIC based router may be like trying to kill a mosquito with a sledge hammer.
Indeed - as you implicitly point out, it's a cost / benefit thing. So then the question becomes this: for the set of organisations which are large enough to warrant multiple 1G upstreams, how long an outage can they sustain before the price difference becomes worth it? Let's throw some figures around (ridiculously simplified): a company has a choice between a pair of $10k software routers or something like a pair of MX80s for $25k each. So, one solution costs $20k; the other $50k. $30k cost difference works out as $625 per month depreciation (4 year). I.e. not going to affect the bottom line in any meaningful way. Now say that this company has a DoS attack for 24h, and the company effectively loses one day of revenue. On the basis that there are 260 office working days per year, the point at which spending an extra $30k for a hardware router would be of net benefit to the company would be 260*30k = $7.8m. I.e. if your annual revenue is higher than that, and if spending that cash would mitigate against your DoS problems, then it would be worth your while in terms of direct loss mitigation. Of course, this analysis is quite simplistic and excludes things like damage to reputation, online stores, the likelihood of DoS attacks happening in the first place, the cost of transit and many other points of reality. However, the point is that the break-even point for getting serious horsepower for your transit requirements is surprisingly low once you take into account the relationship between functional corporate internet connectivity and either or both of corporate revenue and corporate productivity. It's extraordinary how much attention senior management starts paying when everyone in the office starts twiddling their thumbs because connectivity has been down for the day. Nick
On Mon, 12 Sep 2011 22:38:57 BST, Nick Hilliard said:
Let's throw some figures around (ridiculously simplified): a company has a choice between a pair of $10k software routers or something like a pair of MX80s for $25k each. So, one solution costs $20k; the other $50k. $30k cost difference works out as $625 per month depreciation (4 year). I.e. not going to affect the bottom line in any meaningful way.
Now say that this company has a DoS attack for 24h, and the company effectively loses one day of revenue. On the basis that there are 260 office working days per year, the point at which spending an extra $30k for a hardware router would be of net benefit to the company would be 260*30k = $7.8m. I.e. if your annual revenue is higher than that, and if spending that cash would mitigate against your DoS problems, then it would be worth your while in terms of direct loss mitigation.
Of course, this analysis is quite simplistic and excludes things like damage to reputation, online stores, the likelihood of DoS attacks happening in the first place, the cost of transit and many other points of reality.
One important thing it overlooks is what percent of DDoS attackqs are simple "flood the pipe" attacks directed at a target behind the router. If you got a 100M or 1G pipe to the outside world and you're getting hammered by multiple G worth of packets, things are going to suck no matter what the router is. And let's face it, kicking that pipe to 10G is gonna cost a bit....
On Mon, Sep 12, 2011 at 2:35 PM, Nick Hilliard <nick@foobar.org> wrote:
I presume by "a fair amount", I presume you mean "barely any"? At large packet sizes, an "enterprise level" router will just about handle a 1G DoS attack. Thing is, bandwidth DoS / DDoS is sufficiently easy to [snip] How much "zorching" a software router can take depends on a lot of factors. If the hardware necessary to size appropriately for the link is economical and sufficient, zorching is not the largest concern. 1G link speed and 100M link speed offer very different worst-case scenarios; the link can be zorched long before the router is.
A software router running in a 32bit OS on an old Pentium 4 can take a lot less zorching than a router running on a server with 6-core 4Ghz CPUs, when interrupt coalescing is present and utilized efficiently. Hardware basic routers have a lower forwarding latency, which makes them more suitable for ISP/carrier networks, the "hop delay" penalty is lower, and jitter might be a concern on a router running a non real-time OS such as a vanilla Linux kernel or other OS not specially designed for the router task, but there's otherwise nothing wrong with appropriately specc'ed software forwarders. One thing.. the OP was asking about anyone using Vyatta for BGP. Using Vyatta for BGP doesn't necessarily mean the Vyatta unit is actually a device forwarding the packets... someone could be using it as a route server, or for otherwise populating forwarding tables of other devices with third-party next hops :-) -- -JH
On Mon, 12 Sep 2011 20:48:31 CDT, Jimmy Hess said:
One thing.. the OP was asking about anyone using Vyatta for BGP. Using Vyatta for BGP doesn't necessarily mean the Vyatta unit is actually a device forwarding the packets... someone could be using it as a route server, or for otherwise populating forwarding tables of other devices with third-party next hops :-)
I would expect a properly configured Vyatta running as a route server to be pretty darn near zortch-proof, no? (Barring BGP packet-o-death issues of course - but is there a router vendor who *hasn't* had at least 2 or 3 of those? ;)
On Sep 13, 2011, at 2:08 AM, Michael K. Smith - Adhost wrote:
How do you come to this conclusion?
Unhappy experiences. ;>
I think a software-based router for enterprise level (let's say on the 1G per provider level) can handle a fair amount of zorching.
My experiences indicates otherwise, FWIW. It's very easy to packet a software-based router over a relatively small transit link in the mb/sec range, much less gb/sec - it happens all the time, FYI. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> The basis of optimism is sheer terror. -- Oscar Wilde
----Original Message----- From: Dobbins, Roland [mailto:rdobbins@arbor.net] Sent: Monday, September 12, 2011 2:56 PM To: North American Network Operators' Group Subject: Re: vyatta for bgp
zorched.
----------------------------------------------------------------------- Zorch. I like that. Sounds like a Batman fight-scene bubble word. Is the concern over a DDOS aimed against the router itself, or just massive flows passing through? Chuck
On Sep 13, 2011, at 3:34 AM, Chuck Church wrote:
Is the concern over a DDOS aimed against the router itself, or just massive flows passing through?
Yes, but mainly the former. ;> ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> The basis of optimism is sheer terror. -- Oscar Wilde
In a message written on Mon, Sep 12, 2011 at 06:56:26PM +0000, Dobbins, Roland wrote:
The days of public-facing software-based routers were over years ago - you need an ASIC-based edge router, else you'll end up getting zorched.
Some enterprises get MPLS L3 VPN service from their providers, and need boxes that can route packets to it and speak BGP to inject their routes. They are not, per se, connected to the Internet, and thus won't be "zorched", at least in the sense you are using it. Also, many enterprises get DS-3, Cable Modem, or 100M Ethernet handoffs, and won't ever get a faster "zorch" due to link speed. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
In a message written on Mon, Sep 12, 2011 at 06:56:26PM +0000, Dobbins, Roland wrote:
The days of public-facing software-based routers were over years ago - you need an ASIC-based edge router, else you'll end up getting zorched.
Some enterprises get MPLS L3 VPN service from their providers, and need boxes that can route packets to it and speak BGP to inject their routes. They are not, per se, connected to the Internet, and thus won't be "zorched", at least in the sense you are using it. Also, many enterprises get DS-3, Cable Modem, or 100M Ethernet handoffs, and won't ever get a faster "zorch" due to link speed. --- Picking up on what Leo wrote: I think the OP stated he is using less than 10M (or a few T1s or something). The term Enterprise covers a lot of ground from SMEs to LBs. It's important to clarify that no router is perfect and all of them are sufficiently complex beasties to fully understand your problem/solution set. Software routers are simpler in that almost all of their complexities lie in their CPU/bus/interrupt limitations and provided you haven't hit those limits the software can do just about anything you ask of it. Hardware-assisted routers are promised to move lots and lots of pps and tolerate all kinds of bad behavior -- with all kinds of caveats, like control plane policing, understanding the minutiae of their ASIC design/layout and of course various oddities in their software configurations and releases (turn this on, but not with that, if you want this feature to work). Without rehashing 20+ years of collective knowledge & caveats on hardware-assisted routers, smaller guys who want to test their approach to purchasing need some kind of answer better than "it depends". Even though "it depends" (based on total uplink speeds), here are my suggestions: <200 mb/s a circa 2010+ software router, even talking to the internet as a whole, is probably fine, even to run BGP. You may have some weird edge cases where you can be attacked, but your pipe will probably limit you. At this level, you can also lean on your ISP to help if you get into a jam. 200mb/s to 2Gb/s , your software router may keep up, and you need to start considering hardware assisted routing and a stiff breeze could make your router fall over. More time will be required to tune your software router that could be better spent elsewhere. At the higher end of this range, your ISP is less able to help you (filter good traffic from bad) and you need to be able to do some of this in your router. Pipe speed is less of an issue and you can have badly behaved traffic that "zorches" you at far less than link speed. 2Gb/s +, your software solution is a dead duck or an accident waiting to happen. You will be victim to oddities related to inconsistent performance, jitter, and of course malicious attacks. You probably want more advanced traffic and profiling features a hardware platform allows you (at wire speed) too. Your ISP's hardware router will only do what you ask (nicely) for your ISP to do... and even that is limited. You are basically "big enough" to manage these connections on your own and should have equipment and staff available to do so. I just took a stab at the ranges and the concepts, only limited to the OP's context and directed at "Enterprise" customers. ISP's probably can't use these limits for their own router solution/sizing -- and we all know that ISPs vary in quality, especially at 4am when you are being DOS'd....so ymmv. HTH, Deepak Jain AiNET
On Sep 14, 2011, at 5:54 AM, Deepak Jain wrote:
Some enterprises get MPLS L3 VPN service from their providers, and need boxes that can route packets to it and speak BGP to inject their routes. They are not, per se, connected to the Internet, and thus won't be "zorched", at least in the sense you are using it.
Hence 'public-facing'. ;> ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> The basis of optimism is sheer terror. -- Oscar Wilde
Is Vyatta really not suited for the task? I keep checking up on it and holding off looking into it as they don't support multicast yet. Modern commodity sever hardware these days often out-powers big iron enough to make up for not using ASICs, though, at least on the lower end of the spectrum. Does anyone have any more details on Vyatta not scaling? Were you trying to run it as a VM? What were you using for NICs? etc. The hardware matters. Saying Vyatta doesn't cut it could mean anything... On Tue, Sep 13, 2011 at 7:36 PM, Dobbins, Roland <rdobbins@arbor.net> wrote:
On Sep 14, 2011, at 5:54 AM, Deepak Jain wrote:
Some enterprises get MPLS L3 VPN service from their providers, and need boxes that can route packets to it and speak BGP to inject their routes. They are not, per se, connected to the Internet, and thus won't be "zorched", at least in the sense you are using it.
Hence 'public-facing'.
;>
----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
The basis of optimism is sheer terror.
-- Oscar Wilde
-- Ray Soucy Epic Communications Specialist Phone: +1 (207) 561-3526 Networkmaine, a Unit of the University of Maine System http://www.networkmaine.net/
Hi, As usual this end-up in what people prefer. Vyatta is as good as the hardware it runs on, the backend they use and the people configuring/maintaining it. The nature of ASIC make it more reliable than a multi-purpose device (aka server) running an OS written for it. It end up being a choice between risk and cost and being that you can get your hand on second hand iron for cheap these days... Why risk it. ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 09/15/11 09:05, Ray Soucy wrote:
Is Vyatta really not suited for the task?
I keep checking up on it and holding off looking into it as they don't support multicast yet.
Modern commodity sever hardware these days often out-powers big iron enough to make up for not using ASICs, though, at least on the lower end of the spectrum.
Does anyone have any more details on Vyatta not scaling? Were you trying to run it as a VM? What were you using for NICs? etc.
The hardware matters. Saying Vyatta doesn't cut it could mean anything...
On Tue, Sep 13, 2011 at 7:36 PM, Dobbins, Roland<rdobbins@arbor.net> wrote:
On Sep 14, 2011, at 5:54 AM, Deepak Jain wrote:
Some enterprises get MPLS L3 VPN service from their providers, and need boxes that can route packets to it and speak BGP to inject their routes. They are not, per se, connected to the Internet, and thus won't be "zorched", at least in the sense you are using it. Hence 'public-facing'.
;>
----------------------------------------------------------------------- Roland Dobbins<rdobbins@arbor.net> //<http://www.arbornetworks.com>
The basis of optimism is sheer terror.
-- Oscar Wilde
I'll chime in, In an enterprise environment, I've worked with software routers as well as hardware beasts (ala Junipers, Cisco 6500s, ASAs, and more). Ultimately, the network is as reliable as you build it. With software, it's much cheaper to divide and scale horizontally. Hardware devices are expensive and usually horizontal scalability never happens. So in reality, an enterprise blows 100k on two routers, they both flop because of some "firmware bug", and you're down. The most reliable/cost effective solution is the cheap and redundant approach to architecture. Reliable hardware is incredibly inexpensive, and every year we get better CPUs and (recently) GPUs that are providing APIs and interfaces to their incredible parallel processing capability. btw, you guys might find PacketShader<http://shader.kaist.edu/packetshader/>a pretty interesting concept -Andreas On Thu, Sep 15, 2011 at 6:51 AM, Alain Hebert <ahebert@pubnix.net> wrote:
Hi,
As usual this end-up in what people prefer.
Vyatta is as good as the hardware it runs on, the backend they use and the people configuring/maintaining it.
The nature of ASIC make it more reliable than a multi-purpose device (aka server) running an OS written for it.
It end up being a choice between risk and cost and being that you can get your hand on second hand iron for cheap these days...
Why risk it.
----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443
On 09/15/11 09:05, Ray Soucy wrote:
Is Vyatta really not suited for the task?
I keep checking up on it and holding off looking into it as they don't support multicast yet.
Modern commodity sever hardware these days often out-powers big iron enough to make up for not using ASICs, though, at least on the lower end of the spectrum.
Does anyone have any more details on Vyatta not scaling? Were you trying to run it as a VM? What were you using for NICs? etc.
The hardware matters. Saying Vyatta doesn't cut it could mean anything...
On Tue, Sep 13, 2011 at 7:36 PM, Dobbins, Roland<rdobbins@arbor.net> wrote:
On Sep 14, 2011, at 5:54 AM, Deepak Jain wrote:
Some enterprises get MPLS L3 VPN service from their providers, and need
boxes that can route packets to it and speak BGP to inject their routes. They are not, per se, connected to the Internet, and thus won't be "zorched", at least in the sense you are using it.
Hence 'public-facing'.
;>
------------------------------**------------------------------** ----------- Roland Dobbins<rdobbins@arbor.net> //<http://www.arbornetworks.**com<http://www.arbornetworks.com>
The basis of optimism is sheer terror.
-- Oscar Wilde
On Wed, Sep 21, 2011 at 4:14 PM, Andreas Echavez <andreas@livejournalinc.com
wrote:
The most reliable/cost effective solution is the cheap and redundant approach to architecture.
Reliable hardware is incredibly inexpensive, and every year we get better CPUs and (recently) GPUs that are providing APIs and interfaces to their incredible parallel processing capability.
-Andreas
+1 Scaling Horizontally. Applies to your networking gear, your applications, etc. If you assume anything is going to break, just get more and scale/architect properly.
On Thu, Sep 15, 2011 at 6:51 AM, Alain Hebert <ahebert@pubnix.net> wrote:
Hi,
As usual this end-up in what people prefer.
Vyatta is as good as the hardware it runs on, the backend they use and the people configuring/maintaining it.
The nature of ASIC make it more reliable than a multi-purpose device (aka server) running an OS written for it.
It end up being a choice between risk and cost and being that you can get your hand on second hand iron for cheap these days...
Why risk it.
----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443
On 09/15/11 09:05, Ray Soucy wrote:
Is Vyatta really not suited for the task?
I keep checking up on it and holding off looking into it as they don't support multicast yet.
Modern commodity sever hardware these days often out-powers big iron enough to make up for not using ASICs, though, at least on the lower end of the spectrum.
Does anyone have any more details on Vyatta not scaling? Were you trying to run it as a VM? What were you using for NICs? etc.
The hardware matters. Saying Vyatta doesn't cut it could mean
anything...
On Tue, Sep 13, 2011 at 7:36 PM, Dobbins, Roland<rdobbins@arbor.net> wrote:
On Sep 14, 2011, at 5:54 AM, Deepak Jain wrote:
Some enterprises get MPLS L3 VPN service from their providers, and
need
boxes that can route packets to it and speak BGP to inject their routes. They are not, per se, connected to the Internet, and thus won't be "zorched", at least in the sense you are using it.
Hence 'public-facing'.
;>
------------------------------**------------------------------** ----------- Roland Dobbins<rdobbins@arbor.net> //<http://www.arbornetworks.**com<; http://www.arbornetworks.com>
The basis of optimism is sheer terror.
-- Oscar Wilde
-- Brandon Galbraith US Voice: 630.492.0464
On 09/21/2011 06:14 PM, Andreas Echavez wrote:
btw, you guys might find PacketShader<http://shader.kaist.edu/packetshader/>a pretty interesting concept
-Andreas
Excellent! I was wondering how far along this was. Good to see. Very exciting. I've got a couple parallel systems sitting around looking for packets to route... If anyone is doing research in this area, please let me know. Most of my research has been into accelerating IDS/IPS and fuzzing workloads with parallel systems. (Yes that's on top of starting an ISP). I've been looking into http://www.read.cs.ucla.edu/click/Click
Andreas Echavez [mailto:andreas@livejournalinc.com] originally wrote:
Ultimately, the network is as reliable as you build it. With software, it's much cheaper to divide and scale horizontally. Hardware devices are expensive and usually horizontal scalability never happens. So in reality, an enterprise blows 100k on two routers, they both flop because of some "firmware bug", and you're down.
With this in mind, I am keen to understand how many implementations of packages such as Quagga and Zebra that the group use. With the likes of Vyatta being discussed, I am keen to see if products such as Quagga as still regularly used as it used to be. Thoughts welcome! Kind regards, /P.
On 09/22/2011 05:37 AM, Pierce Lynch wrote:
Andreas Echavez [mailto:andreas@livejournalinc.com] originally wrote:
Ultimately, the network is as reliable as you build it. With software, it's much cheaper to divide and scale horizontally. Hardware devices are expensive and usually horizontal scalability never happens. So in reality, an enterprise blows 100k on two routers, they both flop because of some "firmware bug", and you're down. With this in mind, I am keen to understand how many implementations of packages such as Quagga and Zebra that the group use. With the likes of Vyatta being discussed, I am keen to see if products such as Quagga as still regularly used as it used to be.
I think that the original/upstream versions are out of date as compared to the one maintained by Vyatta. Or Google (for their MPLS processing needs). See http://www.nanog.org/meetings/nanog50/abstracts.php?pt=MTYzNSZuYW5vZzUw&nm=nanog50 <http://www.nanog.org/meetings/nanog50/abstracts.php?pt=MTYzNSZuYW5vZzUw&nm=nanog50>
Thoughts welcome!
Kind regards,
/P.
On 9/22/11 11:38 , Charles N Wyble wrote:
On 09/22/2011 05:37 AM, Pierce Lynch wrote:
Ultimately, the network is as reliable as you build it. With software, it's much cheaper to divide and scale horizontally. Hardware devices are expensive and usually horizontal scalability never happens. So in reality, an enterprise blows 100k on two routers, they both flop because of some "firmware bug", and you're down. With this in mind, I am keen to understand how many implementations of
Andreas Echavez [mailto:andreas@livejournalinc.com] originally wrote: packages such as Quagga and Zebra that the group use. With the likes of Vyatta being discussed, I am keen to see if products such as Quagga as still regularly used as it used to be.
I think that the original/upstream versions are out of date as compared to the one maintained by Vyatta. Or Google (for their MPLS processing needs). See http://www.nanog.org/meetings/nanog50/abstracts.php?pt=MTYzNSZuYW5vZzUw&nm=nanog50 <http://www.nanog.org/meetings/nanog50/abstracts.php?pt=MTYzNSZuYW5vZzUw&nm=nanog50>
We are actively supporting Quagga. We currently have a git repo at code.google.com with some BGP multipath updates, and are working with ISC to provide SQA on that branch. Hopefully more features will be forthcoming. Search quagga-dev if you're interested in more details. Vyatta has done a lot of great work on Quagga, as have many others. It would be nice to see all the various useful branches merged into a cherry-picked mainline that would simplify the Quagga development community's lives considerably. -Scott
We service most of the state's public schools and libraries (about 1000). Historically the CPE of choice was a small Cisco ISR (1600, 1700, 1800, and 1900 most recently). As bandwidth levels went up, and Ethernet-based transport services became available, we started looking and leveraging FOSS on commodity hardware to lower costs and move services to the edge. Right now we have about 100 of the bigger school districts being services by a Linux-based appliance running XORP for its routing engine (we would have tried Quagga, but they don't support multicast routing yet, nor does Vyatta). It's been a learning experience. Most of the problems we ran into have been resolved by tuning the kernel parameters to act more like a router than a desktop or server. XORP itself has had a rocky ride since we started, so the stability of the project has also been a concern. Thankfully it is seeing somewhat active development again. I will note that XORP is very touchy about how it's configured; if you have well tested configuration templates it's fine, but it's very easy to get it into a crashing state based on something as little the order of configuration directives. For the most part once it's running it's stable. Modest hardware (3.2GHz dual-core Xeon, 2GB RAM, with 1GB tied up as a RAM disk) seems to do the job well for 100 Mbps without much issue, and that's with stateful firewall, and web content filtering in place. Instead of doing it in-house we found a vendor in MA that was doing something similar to what we wanted and had them develop a modified version of their existing offering for us. The vendor is MECnet for those interested. On Thu, Sep 22, 2011 at 6:37 AM, Pierce Lynch <p.lynch@netappliant.com> wrote:
Andreas Echavez [mailto:andreas@livejournalinc.com] originally wrote:
Ultimately, the network is as reliable as you build it. With software, it's much cheaper to divide and scale horizontally. Hardware devices are expensive and usually horizontal scalability never happens. So in reality, an enterprise blows 100k on two routers, they both flop because of some "firmware bug", and you're down.
With this in mind, I am keen to understand how many implementations of packages such as Quagga and Zebra that the group use. With the likes of Vyatta being discussed, I am keen to see if products such as Quagga as still regularly used as it used to be.
Thoughts welcome!
Kind regards,
/P.
-- Ray Soucy Epic Communications Specialist Phone: +1 (207) 561-3526 Networkmaine, a Unit of the University of Maine System http://www.networkmaine.net/
Ray Download the Podcast "The Packet Pushers - Show 31" they talk a little about this topic... If nothing else it's a great listen Cheers! On Thu, Sep 15, 2011 at 11:05 PM, Ray Soucy <rps@maine.edu> wrote:
Is Vyatta really not suited for the task?
I keep checking up on it and holding off looking into it as they don't support multicast yet.
Modern commodity sever hardware these days often out-powers big iron enough to make up for not using ASICs, though, at least on the lower end of the spectrum.
Does anyone have any more details on Vyatta not scaling? Were you trying to run it as a VM? What were you using for NICs? etc.
The hardware matters. Saying Vyatta doesn't cut it could mean anything...
-- Regards, Jason Leschnik. Mob. 0432 35 4224 Uni mail. jml974@uow.edu.au
Thanks for the tip, first time I hear this podcast. On Thu, Sep 15, 2011 at 9:58 AM, Jason Leschnik <leschnik@gmail.com> wrote:
Ray
Download the Podcast "The Packet Pushers - Show 31" they talk a little about this topic... If nothing else it's a great listen
Cheers!
On Thu, Sep 15, 2011 at 11:05 PM, Ray Soucy <rps@maine.edu> wrote:
Is Vyatta really not suited for the task?
I keep checking up on it and holding off looking into it as they don't support multicast yet.
Modern commodity sever hardware these days often out-powers big iron enough to make up for not using ASICs, though, at least on the lower end of the spectrum.
Does anyone have any more details on Vyatta not scaling? Were you trying to run it as a VM? What were you using for NICs? etc.
The hardware matters. Saying Vyatta doesn't cut it could mean anything...
-- Regards, Jason Leschnik.
Mob. 0432 35 4224 Uni mail. jml974@uow.edu.au
-- Ray Soucy Epic Communications Specialist Phone: +1 (207) 561-3526 Networkmaine, a Unit of the University of Maine System http://www.networkmaine.net/
On Mon, Sep 12, 2011 at 2:42 PM, Ben Albee <balbee@orscheln.com> wrote:
Does anybody currently use vyatta as a bgp router for their company? If so have you ran into any problems with using that instead of a cisco or juniper router?
There was a bug where you couldn't use two IPv4 peers and then add IPv6. I haven't tested the newest versions yet to see if it still exists. Works great for two IPv4 peers.
On Mon, 2011-09-12 at 15:41 -0400, Jared Geiger wrote:
There was a bug where you couldn't use two IPv4 peers and then add IPv6. I haven't tested the newest versions yet to see if it still exists. Works great for two IPv4 peers.
Discussion between developers on bugfixes can often be seen in ##vyatta on Freenode. :) I find it interesting to idle/chime-in occasionally at least. Tom
Hi, In the past, I helped a few small ISP (sub 1Gbps) with software routers setup like Vyatta (Well FreeBSD/64 + Quagga really). Until recently the hardware required to run over 500Mbps + could be as pricey as a pair recycle Cisco 7206VXR since most MBs where coming with only 1 PCI busses which could kills the BW+PPS depending on the amount of interfaces you use. Now-a-days MBs with more than 1 PCI bus have become cheaper and shouldn't be a problem. Its all in the setup anyway: 2 servers ($3k each with 4 interface). Split your uplink on each router. 1 link for "client-reflector" | OSPF | <whatever you like> between the router. VRRP back-end. PS: Sub 10Gbps, any DDoS will kill the link before killing those routers, but there is solutions to this which is hella-easy to deal with in this situation. ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 09/12/11 14:42, Ben Albee wrote:
Does anybody currently use vyatta as a bgp router for their company? If so have you ran into any problems with using that instead of a cisco or juniper router?
On Sep 12, 2011, at 11:42, Ben Albee wrote:
Does anybody currently use vyatta as a bgp router for their company? If so have you ran into any problems with using that instead of a cisco or juniper router?
We're using Vyatta for a handful of fast ethernet links to the internet, with I think about three dozen BGP peers. (Mix of IPv4 and IPv6; about four full feeds on each protocol, the rest is peering). It's not as mature or polished as I understand some of the Cisco or Juniper platforms are; but on our small scale it's fine. We have a decent amount of of Linux expertise in the office (and virtually zero for Juniper/Cisco/...), so having more familiar tools on the routers is nice. As a small shop it's also convenient that the boxes are cheap (so we can have two hot ones with VRRP etc and cheaply a third cold spare) and that the spare parts etc are the same or similar to the rest of the boxes in the rack. - ask -- http://askask.com/
participants (26)
-
Alain Hebert -
Andreas Echavez -
Ask Bjørn Hansen -
Ben Albee -
Brandon Galbraith -
Brent Jones -
Charles N Wyble -
Chuck Church -
Deepak Jain -
Dobbins, Roland -
Everton Marques -
fredrik danerklint -
Jared Geiger -
Jason Leschnik -
Jimmy Hess -
Leo Bicknell -
Martin Millnert -
Michael K. Smith - Adhost -
Nick Hilliard -
Owen DeLong -
Pierce Lynch -
Ray Soucy -
Scott Whyte -
Tom Hill -
Tony Varriale -
Valdis.Kletnieks＠vt.edu