There has been a lot of conversation lately regarding 10Gbps+ routing without higher cost devices such as the junipers. I have been looking into a few options myself, below are my opinions so far. What are your recommendations, real life experiences and ideas? - Mikrotik Cloud Core Router The Mikrotik CCR might have 2 SFP+ ports but with any ACLs, etc fast path is disabled, this already limits the functionality a lot. The BGP calculations only happen on a single core which provides very slow performance for full routing tables. RouterOS is very unstable and had a large number of bugs even with version 6. I have had issues using them even on some small test environments, would not recommend this hardware for nearly any setup. - Linux Based Software Routing Quagga is great for BGP with the correct CPUs and configurations. Vyatta or VyOS provides a stable and simple configuration method for Quagga. The issues with all of the options currently available is forwarding plane performance, you are only looking at 1Gbps+ at line rate. Most providers will have to deal with DDOS attacks at one point or another and would not recommend taking the chance. If you are only looking at 1Gbps or less worth of traffic this is a great option. DDOS attacks information from just the Arbor Networks hardware. http://www.digitalattackmap.com/ Userspace processing of the forwarding plane will help a lot to overcome this issue. There are a few different solutions out there but the most common is Intel DPDK. Some of you would know about the Intel DPDK from the upcoming brocade vRouter 5600 which supports 10Gbps line rate per core. I can see Intel DPDK being used for other solutions such as DDOS filtering as currently you require specialised hardware such as Arbor Networks or NSFOCUS. It would be much cheaper if you could do some filtering from x86 hardware at line rate. http://blog.lukego.com/blog/2013/01/04/kernel-bypass-networking/ Brocade vRouter 5600 might be an option when it is released depending on price. As you still need to get all the hardware required and make sure you do your research regarding the chipsets, etc. Most Intel SFP+ NIC will handle around 9MPPS but has great support for drivers. Solarflare have some nice NICs that can handle 16MPPS but I can see a lot more reviews for different manufacturers coming out after the vRouter release. Hopefully VyOS or some other open source project can integrate Intel DPDK. - OpenFlow OpenFlow is a great method for really high PPS but the major limiting factor is the flow entries and flow mods. I personally like this architecture as it allows the control plane to run on X86 and the Data Plane to run on specialised hardware. For providers with 1 IP transit provider and a few peering IX most OpenFlow hardware will support enough flow entries. The issue is supporting providers with a reasonable number of full routing tables; I think summarization will help a decent amount to lower the flow entries required. NoviSwitch 1248 supports 1 million flow entries which is a reasonable number for smaller providers. I have only started to get my hand dirty with OpenFlow and would like to know if anyone is using it in production for routing? What OpenFlow controller are you using? E.g. RouteFlow https://sites.google.com/site/routeflow/ - Brocade CER The older model CER devices had a lot of issues/bugs but the newer models such as BR-CER-2024C-4X-RT-AC seem to be a lot more stable. There are reviews on webhostingtalk with people pushing more than 30Gbps on the newer models without issue. Based on other people’s comments such as Jon Sands the units should be around 10K each new which makes the units cost affective for a lot of implementations. If you are lucky enough to find one second hand you would only be looking around $5-6K. The 2024C-4X-RT has 4 SFP+ ports which is alright but would really like to see some larger options. Currently a lot of people just create a port channel with all 4 ports to a SFP+ switch which allow them to connect more ports up but need to be careful about overprovisioning. - Layer 3 SFP+ Switch Great for providers with only one uplink as they just use a default route but most providers require more than one uplink. There are lot of cheap options out there even the junipers are not that costly. Regards, Steven. Date: Fri, 27 Dec 2013 21:34:00 -0500 (EST) From: "Justin M. Streiner" <streiner@cluebyfour.org> To: William Waites <wwaites@tardis.ed.ac.uk> Cc: nanog@nanog.org Subject: Re: The Making of a Router Message-ID: <Pine.LNX.4.64.1312272133090.22688@whammy.cluebyfour.org> Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed On Sat, 28 Dec 2013, William Waites wrote:
On Fri, 27 Dec 2013 07:23:36 -0500 (EST), "Justin M. Streiner" < streiner@cluebyfour.org> said:
You end up combining some of the downsides of a hardware-based router with some of the downsides of a server (new attack vectors, another device that needs to be backed up, patched, and monitored...
Might be a good idea to back up, patch and monitor your routers too... Just sayin'
Yes, a given. jms