Yeah the XR thing would be great but NCS540 would be too expensive and too much throughput meaning draws too much power, adam From: Jakub Horn (jakuhorn) <jakuhorn@cisco.com> Sent: Friday, October 16, 2020 10:08 PM To: adamv0025@netconsultings.com; nanog@nanog.org Subject: Re: cheap MPLS router recommendations If cisco, I wouldn’t consider 920… I think NCS540 is better option. XR based, feature rich and here are different models supporting different BW, different port density… I think price/performance ratio is superb! For better evaluation you should be more specific about scale, precise about required features….. Disclaimer: cisco employee….. Cheers -j From: NANOG <nanog-bounces+jakuhorn=cisco.com@nanog.org <mailto:nanog-bounces+jakuhorn=cisco.com@nanog.org> > on behalf of "adamv0025@netconsultings.com <mailto:adamv0025@netconsultings.com> " <adamv0025@netconsultings.com <mailto:adamv0025@netconsultings.com> > Date: Friday, 16 October 2020 at 22:59 To: "nanog@nanog.org <mailto:nanog@nanog.org> " <nanog@nanog.org <mailto:nanog@nanog.org> > Subject: cheap MPLS router recommendations Hi folks, I’m looking for recommendations on a cheap MPLS router (L3VPNs RSVP-TE and BFD). Around 60G throughput would do , heck even 30G. Few 1/10G ports. But netconf yang is almost a must. You know something like asr920 or juniper equivalent, but something that that is not EoS or EoL Oh and something not from China (you know, bad PR)… Any pointers much appreciated. adam

For this particular gig even the MX204 would be overkill in terms of price as well as performance. Ideally something like 204 but with only those 8 10/1G ports (i.e. without the 4x100G ports) adam From: Tony Wicks <tony@wicks.co.nz> Sent: Friday, October 16, 2020 10:36 PM To: adamv0025@netconsultings.com Cc: nanog@nanog.org Subject: RE: cheap MPLS router recommendations Juniper MX204, easy From: NANOG <nanog-bounces+tony=wicks.co.nz@nanog.org <mailto:nanog-bounces+tony=wicks.co.nz@nanog.org> > On Behalf Of adamv0025@netconsultings.com <mailto:adamv0025@netconsultings.com> Sent: Saturday, 17 October 2020 10:31 am To: 'Jakub Horn (jakuhorn)' <jakuhorn@cisco.com <mailto:jakuhorn@cisco.com> >; nanog@nanog.org <mailto:nanog@nanog.org> Subject: RE: cheap MPLS router recommendations Yeah the XR thing would be great but NCS540 would be too expensive and too much throughput meaning draws too much power, adam

Juniper ACX710. Yes it also has more ports, but you only pay for the capacity you need (100G minimum). So you could buy a license that would allow you to enable 10x 10G with the 100G ports dormant. On Fri, Oct 16, 2020 at 11:57 PM <adamv0025@netconsultings.com> wrote:

For this particular gig even the MX204 would be overkill in terms of price as well as performance.

Ideally something like 204 but with only those 8 10/1G ports (i.e. without the 4x100G ports)

adam

*From:* Tony Wicks <tony@wicks.co.nz> *Sent:* Friday, October 16, 2020 10:36 PM *To:* adamv0025@netconsultings.com *Cc:* nanog@nanog.org *Subject:* RE: cheap MPLS router recommendations

Juniper MX204, easy

*From:* NANOG <nanog-bounces+tony=wicks.co.nz@nanog.org> *On Behalf Of * adamv0025@netconsultings.com *Sent:* Saturday, 17 October 2020 10:31 am *To:* 'Jakub Horn (jakuhorn)' <jakuhorn@cisco.com>; nanog@nanog.org *Subject:* RE: cheap MPLS router recommendations

Yeah the XR thing would be great but NCS540 would be too expensive and too much throughput meaning draws too much power,

adam

It can handle a few full tables, but the performance of an MX80/MX104 is nearly the same as the EX4200 switch. Ryan On Oct 16 2020, at 4:41 pm, Tony Wicks <tony@wicks.co.nz> wrote:

Well, there is always the MX104 (if you want redundancy) or MX80 if you don’t. That will give you 80gig wire speed just don’t load it up with more than one full table.

From: adamv0025@netconsultings.com <adamv0025@netconsultings.com> Sent: Saturday, 17 October 2020 10:57 am To: 'Tony Wicks' <tony@wicks.co.nz> Cc: nanog@nanog.org Subject: RE: cheap MPLS router recommendations

For this particular gig even the MX204 would be overkill in terms of price as well as performance. Ideally something like 204 but with only those 8 10/1G ports (i.e. without the 4x100G ports)

adam From: Tony Wicks <tony@wicks.co.nz (mailto:tony@wicks.co.nz)> Sent: Friday, October 16, 2020 10:36 PM To: adamv0025@netconsultings.com (mailto:adamv0025@netconsultings.com) Cc: nanog@nanog.org (mailto:nanog@nanog.org) Subject: RE: cheap MPLS router recommendations

Juniper MX204, easy

On Saturday, 17 October, 2020 00:41, "Tony Wicks" <tony@wicks.co.nz> said:

Well, there is always the MX104 (if you want redundancy) or MX80 if you don’t. That will give you 80gig wire speed just don’t load it up with more than one full table.

Bear in mind that the MX80 is now in the EoL process, you have <4 years of support left. Depending on your expected life-time / depreciation rules, buying one new right now might be unwise. Do *not* throw a full table at it (or any of the PowerPC Junipers) unless you have a lot of patience for reconvergence, and black-holes while you wait. MX104 is a nice box for getting dual-RE in something relatively compact and cheap, and has environmental hardening if that matters to you, but is still not best pleased with full tables. OP could do with clarifying "cheap" :) Regards, Tim.

Just to clarify what cheap means, ideally -$2000 to $4000 new -new is preferred as buying used kit on second hand market one is at the mercy of the price fluctuations and availability. And the likes of the M2400 looks good 4x10G plus some 1G, unfortunately there are no details on the webpage (and the datasheet can’t be downloaded… ) Are there more folks out there bundling open NOS and white-box HW along with the support for the whole thing? adam From: NANOG <nanog-bounces+adamv0025=netconsultings.com@nanog.org> On Behalf Of Colton Conor Sent: Monday, October 19, 2020 4:51 PM To: tim@pelican.org Cc: NANOG <nanog@nanog.org> Subject: Re: cheap MPLS router recommendations I haven't tried one myself, but Dasan Zhone has the M2400 and M3000. Basically, a whitebox with IP Infusion code on it. New, I think the price point is sub $2000 to $4000 new. That's a ton of ports for that price point. Anyone tried these yet? https://dzsi.com/product-category/mobile-xhaul/ On Mon, Oct 19, 2020 at 3:38 AM tim@pelican.org <mailto:tim@pelican.org> <tim@pelican.org <mailto:tim@pelican.org> > wrote: On Saturday, 17 October, 2020 00:41, "Tony Wicks" <tony@wicks.co.nz <mailto:tony@wicks.co.nz> > said:

Well, there is always the MX104 (if you want redundancy) or MX80 if you don’t. That will give you 80gig wire speed just don’t load it up with more than one full table.

Bear in mind that the MX80 is now in the EoL process, you have <4 years of support left. Depending on your expected life-time / depreciation rules, buying one new right now might be unwise. Do *not* throw a full table at it (or any of the PowerPC Junipers) unless you have a lot of patience for reconvergence, and black-holes while you wait. MX104 is a nice box for getting dual-RE in something relatively compact and cheap, and has environmental hardening if that matters to you, but is still not best pleased with full tables. OP could do with clarifying "cheap" :) Regards, Tim.

https://www.multicominc.com/wp-content/uploads/DZS-M3000_M.pdf On Wed, Oct 21, 2020 at 4:08 PM Colton Conor <colton.conor@gmail.com> wrote:

Well then Adam I would say the Dasan Zhone fits the budget. The M3000 seems like a real beast for the price point with 100G ports.

Yes, other whitebox vendors are doing this, but they seem to want 2-4k for the whitebox, and even more for the operating system, making it more expensive that Juniper from what I have seen.

On Wed, Oct 21, 2020 at 3:27 PM <adamv0025@netconsultings.com> wrote:

...

Just to clarify what cheap means, ideally -$2000 to $4000 new

-new is preferred as buying used kit on second hand market one is at the mercy of the price fluctuations and availability.

And the likes of the M2400 looks good 4x10G plus some 1G, unfortunately there are no details on the webpage (and the datasheet can’t be downloaded… )

Are there more folks out there bundling open NOS and white-box HW along with the support for the whole thing?

adam

*From:* NANOG <nanog-bounces+adamv0025=netconsultings.com@nanog.org> *On Behalf Of *Colton Conor *Sent:* Monday, October 19, 2020 4:51 PM *To:* tim@pelican.org *Cc:* NANOG <nanog@nanog.org> *Subject:* Re: cheap MPLS router recommendations

I haven't tried one myself, but Dasan Zhone has the M2400 and M3000. Basically, a whitebox with IP Infusion code on it. New, I think the price point is sub $2000 to $4000 new. That's a ton of ports for that price point. Anyone tried these yet? https://dzsi.com/product-category/mobile-xhaul/

On Mon, Oct 19, 2020 at 3:38 AM tim@pelican.org <tim@pelican.org> wrote:

On Saturday, 17 October, 2020 00:41, "Tony Wicks" <tony@wicks.co.nz> said:

...

Well, there is always the MX104 (if you want redundancy) or MX80 if you don’t. That will give you 80gig wire speed just don’t load it up with more than one full table.

Bear in mind that the MX80 is now in the EoL process, you have <4 years of support left. Depending on your expected life-time / depreciation rules, buying one new right now might be unwise.

Do *not* throw a full table at it (or any of the PowerPC Junipers) unless you have a lot of patience for reconvergence, and black-holes while you wait.

MX104 is a nice box for getting dual-RE in something relatively compact and cheap, and has environmental hardening if that matters to you, but is still not best pleased with full tables.

OP could do with clarifying "cheap" :)

Regards, Tim.

Right, well in that price/performance range you either “roll your own” or this is your best option IMHO - https://mikrotik.com/product/CCR1072-1G-8Splus and I’d pick the Mikrotik every time. From: NANOG <nanog-bounces+tony=wicks.co.nz@nanog.org> On Behalf Of adamv0025@netconsultings.com Sent: Thursday, 22 October 2020 9:28 am To: 'Colton Conor' <colton.conor@gmail.com>; tim@pelican.org Cc: 'NANOG' <nanog@nanog.org> Subject: RE: cheap MPLS router recommendations Just to clarify what cheap means, ideally -$2000 to $4000 new -new is preferred as buying used kit on second hand market one is at the mercy of the price fluctuations and availability. And the likes of the M2400 looks good 4x10G plus some 1G, unfortunately there are no details on the webpage (and the datasheet can’t be downloaded… ) Are there more folks out there bundling open NOS and white-box HW along with the support for the whole thing? adam

I can second that. Mikrotik MPLS implementation is not good and lacking key features (speaking from personal experience also), and has not show any signs of improvement in quite some time. One unit to take a look at for a cheap 10G solution is this: https://www.zte.com.cn/global/products/bearer/Ethernet-Switch/5960-EN Many many years ago this was mentioned by some NANOG members (Baldur?) and I recall one had said they were using it with some success. -PC Tue, Oct 27, 2020 at 06:12:10AM +0000, Philip Loenneker wrote:

While I like MikroTik, I don’t recommend anyone uses it for MPLS.

There are problems with the way they handle labels that causes random connectivity issues, and can crash MPLS devices from other vendors. That’s from experience.

As an example, check out this post which was started back in 2013 and is still an issue in 2020:

https://forum.mikrotik.com/viewtopic.php?t=73820

Regards,

Philip

From: NANOG <nanog-bounces+philip.loenneker=tasmanet.com.au@nanog.org> On Behalf Of Tony Wicks Sent: Thursday, 22 October 2020 8:19 AM To: adamv0025@netconsultings.com Cc: 'NANOG' <nanog@nanog.org> Subject: RE: cheap MPLS router recommendations

Right, well in that price/performance range you either “roll your own” or this is your best option IMHO - https://mikrotik.com/product/CCR1072-1G-8Splus and I’d pick the Mikrotik every time.

From: NANOG <nanog-bounces+tony=wicks.co.nz@nanog.org> On Behalf Of adamv0025@netconsultings.com Sent: Thursday, 22 October 2020 9:28 am To: 'Colton Conor' <colton.conor@gmail.com>; tim@pelican.org Cc: 'NANOG' <nanog@nanog.org> Subject: RE: cheap MPLS router recommendations

Just to clarify what cheap means, ideally -$2000 to $4000 new

-new is preferred as buying used kit on second hand market one is at the mercy of the price fluctuations and availability.

And the likes of the M2400 looks good 4x10G plus some 1G, unfortunately there are no details on the webpage (and the datasheet can’t be downloaded… )

Are there more folks out there bundling open NOS and white-box HW along with the support for the whole thing?

adam

-- Patrick Cole <patrick.cole@spirit.com.au> Chief Engineer Spirit Technology Solutions 19-25 Raglan St, South Melbourne VIC 3205 Desk: 0385541391 Mobile: 0410626630

If we're talking about whitebox router and ipifusion, what we're really talking about is vyatta/vyOS and the linux foundation DANOS stuff on an ordinary x86-64 server that has a weird shape. https://www.ipinfusion.com/commercial-version-of-danos-product-page/ https://www.danosproject.org/ In which case it really comes down to how comfortable you are with the feature sets of the individual daemons contained within Vyatta/VyOS derived products (FRR, etc), and then your trust level in the hardware. Typically something such as a Taiwanese industrial/embedded platform manufacturer such as Lanner: http://www.lannerinc.com/products/network-appliances/x86-rackmount-network-a... If you look at the results of a linux kernel boot on a Lanner appliance running VyOS, or a lspci -v, they're not significantly different than taking a Dell or Supermicro rack server and sticking a whole bunch of Intel or Chelsio 2 or 4-port 10GbE cards into it. It's just a weird shaped motherboard, but ultimately derived from an Intel or AMD reference design, and shares a lot in common in a block diagram with a 1U dual socket server motherboard from a company like Tyan or Supermicro. You've got ethernet NICs attached to the PCI-E bus the same as if they were slotted into cards. Aside from the big names like Quanta, Compal and Clevo who will manufacture these things for you in a bespoke fashion if you're a big cloud scale operator, if you google "taiwan embedded industrial motherboard" you'll find the companies that make most of the x86-64 whitebox router hardware. I guess the point I'm trying to make above is that **if** you're confident in both the SW and HW, you can disaggregate your choice of software (vyatta/vyos/DANOS etc) from your own choice of hardware to best fit your needs, rather than purchasing it together as a package. On Wed, Oct 21, 2020 at 1:28 PM <adamv0025@netconsultings.com> wrote:

Just to clarify what cheap means, ideally -$2000 to $4000 new

-new is preferred as buying used kit on second hand market one is at the mercy of the price fluctuations and availability.

And the likes of the M2400 looks good 4x10G plus some 1G, unfortunately there are no details on the webpage (and the datasheet can’t be downloaded… )

Are there more folks out there bundling open NOS and white-box HW along with the support for the whole thing?

adam

*From:* NANOG <nanog-bounces+adamv0025=netconsultings.com@nanog.org> *On Behalf Of *Colton Conor *Sent:* Monday, October 19, 2020 4:51 PM *To:* tim@pelican.org *Cc:* NANOG <nanog@nanog.org> *Subject:* Re: cheap MPLS router recommendations

I haven't tried one myself, but Dasan Zhone has the M2400 and M3000. Basically, a whitebox with IP Infusion code on it. New, I think the price point is sub $2000 to $4000 new. That's a ton of ports for that price point. Anyone tried these yet? https://dzsi.com/product-category/mobile-xhaul/

On Mon, Oct 19, 2020 at 3:38 AM tim@pelican.org <tim@pelican.org> wrote:

On Saturday, 17 October, 2020 00:41, "Tony Wicks" <tony@wicks.co.nz> said:

...

Well, there is always the MX104 (if you want redundancy) or MX80 if you don’t. That will give you 80gig wire speed just don’t load it up with more than one full table.

Bear in mind that the MX80 is now in the EoL process, you have <4 years of support left. Depending on your expected life-time / depreciation rules, buying one new right now might be unwise.

Do *not* throw a full table at it (or any of the PowerPC Junipers) unless you have a lot of patience for reconvergence, and black-holes while you wait.

MX104 is a nice box for getting dual-RE in something relatively compact and cheap, and has environmental hardening if that matters to you, but is still not best pleased with full tables.

OP could do with clarifying "cheap" :)

Regards, Tim.

Eric Kuhnke <eric.kuhnke@gmail.com> Sent: Monday, October 26, 2020 6:52 PM

If we're talking about whitebox router and ipifusion, what we're really talking about is vyatta/vyOS and the linux foundation DANOS stuff on an ordinary x86- 64 server that has a weird shape.

https://www.ipinfusion.com/commercial-version-of-danos-product-page/

https://www.danosproject.org/

In which case it really comes down to how comfortable you are with the feature sets of the individual daemons contained within Vyatta/VyOS derived products (FRR, etc), and then your trust level in the hardware. Typically something such as a Taiwanese industrial/embedded platform manufacturer such as Lanner:

http://www.lannerinc.com/products/network-appliances/x86-rackmount- network-appliances

Yes I agree and I have been looking at Lanner (NCA-2510, NCA-2512) -and I'll be thankful if I could be pointed towards other companies "producing funny looking servers" for comparison. I've been also looking at the whole, a bit confusing, ecosystem of DANOS/Vytta/VyOS and their capabilities through my PE requirement optics (i.e. MPLS L3VPN, RSVP-TE/SR-TE & fast reroute, NETCONF-YANG) -which these NOS-es seem to have very little to no support for.

I guess the point I'm trying to make above is that **if** you're confident in both the SW and HW, you can disaggregate your choice of software (vyatta/vyos/DANOS etc) from your own choice of hardware to best fit your needs, rather than purchasing it together as a package.

I agree, however I need this whole SW to HW mapping and all associated pain of interworking and operational support to be absorbed and nicely packaged by a vendor. On that note, can Lanner fix me up with some DANOS/Vytta/VyOS on their NCA-2510 or NCA-2512 please? Or in more general terms, are there folks out there that specialize in providing support for NOS X, Y, Z with HW X, Y, Z -in any combination? -so that I could just pick NOS X with HW Y from their portfolio based on my project requirements and get similar support I'd get from a "standard" vendor? adam