Re: Mikrotik Cloud Core Router and BGP real life experiences?
Exactly what Faisal Said. The BGP process appears to be single threaded at the moment. So taking on full BGP tables can be a bit slow compared to a decent X86 box. But in terms of raw forwarding power they are pretty monstrous. We replaced a few Maxxwave 6 port Atom's with the CCR. ~400Mb/s and ~40K pps aggregate across all ports. CPU load went from ~25% to ~0-2%. These are in a configuration where they have little or no firewall/nat/queue rules. And in most cases are running MPLS. We've not had any issues with stability so far either (Knock on wood). Nick Olsen Network Operations (855) FLSPEED x106 ---------------------------------------- From: "Faisal Imtiaz" <faisal@snappytelecom.net> Sent: Friday, December 27, 2013 10:33 AM To: "Geraint Jones" <geraint@koding.com> Cc: nanog@nanog.org, "Martin Hotze" <m.hotze@hotze.com> Subject: Re: Mikrotik Cloud Core Router and BGP real life experiences? FYI... Mikrotik Cloud Core routers are nice, however one has to keep something in mind when deploying them... Only One Core (of the CPU) is dedicated to each port / process. So this is good so as to contain what happens on a single port from taxing the whole CPU.. But not so good when you need more cpu power than a single core for that port. Also, BGP process will only use one core. While these units make for great 'customer facing' edge routers, with plenty of power and the ability to keep issues contained... The X-86 based (Core2Duo/i5/i7) Mikrotik are more suitable (Processing power wise) for running multiple full BGP tables peering. Regards & Good Luck. Faisal Imtiaz Snappy Internet & Telecom ----- Original Message -----
From: "Geraint Jones" <geraint@koding.com> To: "Martin Hotze" <m.hotze@hotze.com> Cc: nanog@nanog.org Sent: Friday, December 27, 2013 4:02:45 AM Subject: Re: Mikrotik Cloud Core Router and BGP real life experiences?
I am going to be deploying 4 as edge routers in the next few weeks, each will have 1 or 2 full tables plus partial IX tables. So I should have some empirical info soon.
They will be doing eBGP to upstreams and iBGP/OSPF internally. I went with the 16gb RAM models.
However these boxes are basically Linux running on top of tilera CPUs, in terms of throughput as long as everything stays on the fastpath they have no issues doing wire speed on all ports, however the moment you add a firewall rule or the like they drop to 1.5gbps.
On 27/12/2013, at 9:47 pm, Martin Hotze <m.hotze@hotze.com> wrote:
Hi,
looking at the specs of Mikrotik Cloud Core Routers it seems to be to good to be true [1] having so much bang for the bucks. So virtually all smaller ISPs would drop their CISCO gear for Mikrotik Routerboards.
We are using a handful of Mikrotik boxes, but on a much lower network level (splitting networks; low end router behind ADSL modem, ...). We're happy with them.
So I am asking for real life experience and not lab values with Mikrotik Cloud Core Routers and BGP. How good can they handle full tables and a bunch of peering sessions? How good does the box react when adding filters (during attacks)? Reloading the table? etc. etc.
I am looking for _real_ _life_ values compared to a CISCO NPE-G2. Please tell me/us from your first hand experience.
Thanks!
greetings, Martin
[1] If something sounds too good to be true, it probably is.
PPPoE Server is single thread too. 2013/12/27 Nick Olsen <nick@flhsi.com>
Exactly what Faisal Said. The BGP process appears to be single threaded at the moment. So taking on full BGP tables can be a bit slow compared to a decent X86 box. But in terms of raw forwarding power they are pretty monstrous.
We replaced a few Maxxwave 6 port Atom's with the CCR. ~400Mb/s and ~40K pps aggregate across all ports. CPU load went from ~25% to ~0-2%. These are in a configuration where they have little or no firewall/nat/queue rules. And in most cases are running MPLS.
We've not had any issues with stability so far either (Knock on wood).
Nick Olsen Network Operations (855) FLSPEED x106
---------------------------------------- From: "Faisal Imtiaz" <faisal@snappytelecom.net> Sent: Friday, December 27, 2013 10:33 AM To: "Geraint Jones" <geraint@koding.com> Cc: nanog@nanog.org, "Martin Hotze" <m.hotze@hotze.com> Subject: Re: Mikrotik Cloud Core Router and BGP real life experiences?
FYI... Mikrotik Cloud Core routers are nice, however one has to keep something in mind when deploying them...
Only One Core (of the CPU) is dedicated to each port / process. So this is good so as to contain what happens on a single port from taxing the whole CPU.. But not so good when you need more cpu power than a single core for that port.
Also, BGP process will only use one core.
While these units make for great 'customer facing' edge routers, with plenty of power and the ability to keep issues contained... The X-86 based (Core2Duo/i5/i7) Mikrotik are more suitable (Processing power wise) for running multiple full BGP tables peering.
Regards & Good Luck.
Faisal Imtiaz Snappy Internet & Telecom
----- Original Message -----
From: "Geraint Jones" <geraint@koding.com> To: "Martin Hotze" <m.hotze@hotze.com> Cc: nanog@nanog.org Sent: Friday, December 27, 2013 4:02:45 AM Subject: Re: Mikrotik Cloud Core Router and BGP real life experiences?
I am going to be deploying 4 as edge routers in the next few weeks, each will have 1 or 2 full tables plus partial IX tables. So I should have some empirical info soon.
They will be doing eBGP to upstreams and iBGP/OSPF internally. I went with the 16gb RAM models.
However these boxes are basically Linux running on top of tilera CPUs, in terms of throughput as long as everything stays on the fastpath they have no issues doing wire speed on all ports, however the moment you add a firewall rule or the like they drop to 1.5gbps.
On 27/12/2013, at 9:47 pm, Martin Hotze <m.hotze@hotze.com> wrote:
Hi,
looking at the specs of Mikrotik Cloud Core Routers it seems to be to good to be true [1] having so much bang for the bucks. So virtually all smaller ISPs would drop their CISCO gear for Mikrotik Routerboards.
We are using a handful of Mikrotik boxes, but on a much lower network level (splitting networks; low end router behind ADSL modem, ...). We're happy with them.
So I am asking for real life experience and not lab values with Mikrotik Cloud Core Routers and BGP. How good can they handle full tables and a bunch of peering sessions? How good does the box react when adding filters (during attacks)? Reloading the table? etc. etc.
I am looking for _real_ _life_ values compared to a CISCO NPE-G2. Please tell me/us from your first hand experience.
Thanks!
greetings, Martin
[1] If something sounds too good to be true, it probably is.
-- Eduardo Schoedler
Regards Alexander Alexander Neilson Neilson Productions Ltd Alexander@Neilson.net.nz 021 329 681
On 28/12/2013, at 5:06 am, Eduardo Schoedler <listas@esds.com.br> wrote:
PPPoE Server is single thread too.
PPP package is getting a multicore upgrade in 6.8 or 6.9 release. May introduce bugs but they are working to Multi core all the processes properly.
2013/12/27 Nick Olsen <nick@flhsi.com>
Exactly what Faisal Said. The BGP process appears to be single threaded at the moment. So taking on full BGP tables can be a bit slow compared to a decent X86 box. But in terms of raw forwarding power they are pretty monstrous.
We replaced a few Maxxwave 6 port Atom's with the CCR. ~400Mb/s and ~40K pps aggregate across all ports. CPU load went from ~25% to ~0-2%. These are in a configuration where they have little or no firewall/nat/queue rules. And in most cases are running MPLS.
We've not had any issues with stability so far either (Knock on wood).
Nick Olsen Network Operations (855) FLSPEED x106
---------------------------------------- From: "Faisal Imtiaz" <faisal@snappytelecom.net> Sent: Friday, December 27, 2013 10:33 AM To: "Geraint Jones" <geraint@koding.com> Cc: nanog@nanog.org, "Martin Hotze" <m.hotze@hotze.com> Subject: Re: Mikrotik Cloud Core Router and BGP real life experiences?
FYI... Mikrotik Cloud Core routers are nice, however one has to keep something in mind when deploying them...
Only One Core (of the CPU) is dedicated to each port / process. So this is good so as to contain what happens on a single port from taxing the whole CPU.. But not so good when you need more cpu power than a single core for that port.
Also, BGP process will only use one core.
While these units make for great 'customer facing' edge routers, with plenty of power and the ability to keep issues contained... The X-86 based (Core2Duo/i5/i7) Mikrotik are more suitable (Processing power wise) for running multiple full BGP tables peering.
Regards & Good Luck.
Faisal Imtiaz Snappy Internet & Telecom
----- Original Message -----
From: "Geraint Jones" <geraint@koding.com> To: "Martin Hotze" <m.hotze@hotze.com> Cc: nanog@nanog.org Sent: Friday, December 27, 2013 4:02:45 AM Subject: Re: Mikrotik Cloud Core Router and BGP real life experiences?
I am going to be deploying 4 as edge routers in the next few weeks, each will have 1 or 2 full tables plus partial IX tables. So I should have some empirical info soon.
They will be doing eBGP to upstreams and iBGP/OSPF internally. I went with the 16gb RAM models.
However these boxes are basically Linux running on top of tilera CPUs, in terms of throughput as long as everything stays on the fastpath they have no issues doing wire speed on all ports, however the moment you add a firewall rule or the like they drop to 1.5gbps.
On 27/12/2013, at 9:47 pm, Martin Hotze <m.hotze@hotze.com> wrote:
Hi,
looking at the specs of Mikrotik Cloud Core Routers it seems to be to good to be true [1] having so much bang for the bucks. So virtually all smaller ISPs would drop their CISCO gear for Mikrotik Routerboards.
We are using a handful of Mikrotik boxes, but on a much lower network level (splitting networks; low end router behind ADSL modem, ...). We're happy with them.
So I am asking for real life experience and not lab values with Mikrotik Cloud Core Routers and BGP. How good can they handle full tables and a bunch of peering sessions? How good does the box react when adding filters (during attacks)? Reloading the table? etc. etc.
I am looking for _real_ _life_ values compared to a CISCO NPE-G2. Please tell me/us from your first hand experience.
Thanks!
greetings, Martin
[1] If something sounds too good to be true, it probably is.
-- Eduardo Schoedler
participants (3)
-
Alexander Neilson -
Eduardo Schoedler -
Nick Olsen