NANOG, I know Arista is typically a switch manufacturer, but with their recently announced Arista 7500R Series and soon to be announced but already shipping 7280R Series Arista is officially getting into the routing game. The fixed 1U 7280R Series looks quite impressive. The 7500R series is your traditional chassis and line card based solution. Both of these products have the ability to hold the full internet routing table, and Arista is working on MPLS features. Both of these new products use the latest Broadcom Jerico chipsets. I would like to know how viable of a product NANOG thinks these Arista routers are compared to service provider grade routers from Cisco, Juniper, ALU, and Brocade? Cost wise, Arista seems to be much, much less per port. For example, the 1U Arista 7280R with 48x10GbE (SFP+) & 6x100GbE QSFP cost about the same as what Juniper sells a MX104 with only four 10G ports for (Under 20K). Can the Arista EOS software combine with their hardware based on the Broadcom Jericho chipset truly compete with the custom chipsets and accompanying software from the big guys?
Hey Colton, Comments inline: On Wed, Apr 20, 2016 at 3:37 PM, Colton Conor <colton.conor@gmail.com> wrote:
NANOG,
I know Arista is typically a switch manufacturer, but with their recently announced Arista 7500R Series and soon to be announced but already shipping 7280R Series Arista is officially getting into the routing game. The fixed 1U 7280R Series looks quite impressive. The 7500R series is your traditional chassis and line card based solution.
I must admit, i'm not usually excited by new hardware, but this announcement did catch my eye! Both of these products have the ability to hold the full internet routing
table, and Arista is working on MPLS features. Both of these new products use the latest Broadcom Jerico chipsets.
I would like to know how viable of a product NANOG thinks these Arista routers are compared to service provider grade routers from Cisco, Juniper, ALU, and Brocade?
Honestly? I think you need to look at what you actually need out of a box. At the end of the day, its a 1U switch. If you want to terminate a GRT a the edge of your network and do some basic path selection then it sounds like it would be an amazing and cheap fit. On the other hand, I don't think we can start throwing away core routers yet ;) Cost wise, Arista seems to be much, much less per port. For example, the
1U Arista 7280R with 48x10GbE (SFP+) & 6x100GbE QSFP cost about the same as what Juniper sells a MX104 with only four 10G ports for (Under 20K).
I'm consistently amazed at the density they are achieving for the $$ and I think it all comes down to what the actual application is here. Most basic BGP networks do not need all the bells and whistles of the MX104 and will really benefit from the extra port density. That being said, I wouldn't be replacing core PoPs in large ISPs with 1U switches!
Can the Arista EOS software combine with their hardware based on the Broadcom Jericho chipset truly compete with the custom chipsets and accompanying software from the big guys?
I've used Arista for a while now (Moving from Cisco / Extreme) and I truly believe that their software is excellent. They just seem to be doing it 'the right way', If you've not watched it, this video is worth a bit of your time! https://www.youtube.com/watch?v=VdJZq4dRjf4 Thats my $0.02 anyway Tom
On 20/04/16 15:37, Colton Conor wrote:
Can the Arista EOS software combine with their hardware based on the Broadcom Jericho chipset truly compete with the custom chipsets and accompanying software from the big guys?
In broad strokes: for your money you're either getting port density, or more features per port. The only difference here is that there's suddenly more TCAM on the device, and I still don't see the above changing too drastically. If it works* for you, use it. :) * Assuming that you've done your due diligence before purchasing, and not just skim-read the vendor PDF. -- Tom
On 23 April 2016 at 10:52, Tom Hill <tom@ninjabadger.net> wrote:
In broad strokes: for your money you're either getting port density, or more features per port. The only difference here is that there's suddenly more TCAM on the device, and I still don't see the above changing too drastically.
Yeah OP is comparing high touch chip (MX104) to low touch chip (Jericho) that is not fair comparison. And cost is what customer is willing to pay, regardless of sticker on the box. No one will pay significant mark-up for another sticker, I've never seen in RFP significant differences in comparable products. Fairer comparison would be QFX10k, instead of MX104. QFX10k is AFAIK only product in this segment which is not using Jericho. If this is competitive advantage or risk, jury is still out, I lean towards competitive advantage, mainly due to its memory design. -- ++ytti
Saku, Jericho is in no sense a low end chip, while there are some scale limitations (what can be done with SuperFEC, some bridging related stuff), from functionality prospective it is a very capable silicon. One has to: Understand how to program it properly (recursiveness, ECMP’s, etc) Know how to enhance SDK Have a rather rich control plane, which can be translated into rich forwarding functionality :-) I’m not familiar with Arista’s feature set NCS with XR would be a good proof Watch for Jericho updates from DNX Cheers, Jeff On 4/23/16, 11:20 AM, "NANOG on behalf of Saku Ytti" <nanog-bounces@nanog.org on behalf of saku@ytti.fi> wrote:
On 23 April 2016 at 10:52, Tom Hill <tom@ninjabadger.net> wrote:
In broad strokes: for your money you're either getting port density, or more features per port. The only difference here is that there's suddenly more TCAM on the device, and I still don't see the above changing too drastically.
Yeah OP is comparing high touch chip (MX104) to low touch chip (Jericho) that is not fair comparison. And cost is what customer is willing to pay, regardless of sticker on the box. No one will pay significant mark-up for another sticker, I've never seen in RFP significant differences in comparable products.
Fairer comparison would be QFX10k, instead of MX104. QFX10k is AFAIK only product in this segment which is not using Jericho. If this is competitive advantage or risk, jury is still out, I lean towards competitive advantage, mainly due to its memory design.
-- ++ytti
High Touch / Low Touch Is this a measure of the amount of fiddle diddling required to get the chip to work as documented, or is it some other kind of code? For example a "High Touch" chip needs lots of fiddle farting because it was designed by a moron and every possible thing that can be programmed incorrectly is programmed incorrectly, whereas in a "Low Touch" chip all the defaults are already set to the most useful and rational setting so that it can be used without touching it to fix all the defects? Perhaps it is a measure of the babysitting required while the chip is running. "High Touch" chips require constant attention, nappy changes, positive re-inforcement of the settings, etc., while operating because they are inherently unreliable and badly designed whereas "Low Touch" chips once set up just work and require little ongoing supervision unless you want to change something? Or is it just a strange translation for functionality (as in High End / Low End)?
-----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Saku Ytti Sent: Saturday, 23 April, 2016 14:21 To: Tom Hill Cc: nanog list Subject: Re: Arista Routing Solutions
On 23 April 2016 at 10:52, Tom Hill <tom@ninjabadger.net> wrote:
In broad strokes: for your money you're either getting port density, or more features per port. The only difference here is that there's suddenly more TCAM on the device, and I still don't see the above changing too drastically.
Yeah OP is comparing high touch chip (MX104) to low touch chip (Jericho) that is not fair comparison. And cost is what customer is willing to pay, regardless of sticker on the box. No one will pay significant mark-up for another sticker, I've never seen in RFP significant differences in comparable products.
Fairer comparison would be QFX10k, instead of MX104. QFX10k is AFAIK only product in this segment which is not using Jericho. If this is competitive advantage or risk, jury is still out, I lean towards competitive advantage, mainly due to its memory design.
-- ++ytti
On 24 April 2016 at 05:14, Keith Medcalf <kmedcalf@dessus.com> wrote:
High Touch / Low Touch
High touch means very general purpose NPU, with off-chip memory. Low touch means usually ASIC or otherwise simplified pipeline and on-chip memory. Granted Jericho can support off-chip memory too. L3 switches are canonical example of low touch. EZchip, Trio, Solar, FP3 etc are examples of canonical high touch NPUs. What low touch can do, it can do fast and economically. But like few terms, it's not exact, and borders are hazy and even subjective. -- ++ytti
Got it, thanks for the explanation!
-----Original Message----- From: Saku Ytti [mailto:saku@ytti.fi] Sent: Sunday, 24 April, 2016 11:03 To: Keith Medcalf Cc: nanog list Subject: Re: Arista Routing Solutions
On 24 April 2016 at 05:14, Keith Medcalf <kmedcalf@dessus.com> wrote:
High Touch / Low Touch
High touch means very general purpose NPU, with off-chip memory. Low touch means usually ASIC or otherwise simplified pipeline and on-chip memory. Granted Jericho can support off-chip memory too.
L3 switches are canonical example of low touch. EZchip, Trio, Solar, FP3 etc are examples of canonical high touch NPUs. What low touch can do, it can do fast and economically.
But like few terms, it's not exact, and borders are hazy and even subjective.
-- ++ytti
High Touch / Low Touch
High touch means very general purpose NPU, with off-chip memory. Low touch means usually ASIC or otherwise simplified pipeline and on-chip memory. Granted Jericho can support off-chip memory too.
L3 switches are canonical example of low touch. EZchip, Trio, Solar, FP3 etc are examples of canonical high touch NPUs. What low touch can do, it can do fast and economically.
Your analogy makes some sense, but what you classify as high-touch / low-touch is just one dimension and could do with a more modern update. I'd suggest a more modern analogy would be that historically the difference between a L3 switch and a router is the former has a fixed processing pipeline, limited buffering (most are just on-chip buffer) and limited table sizes. But more modern packet processors with fixed pipelines often have blocks or sections that are programmable or flexible. e.g. with a flexible packet parser, its possible to support new overlay or tunnel mechanisms, flexible key generation makes it possible to reuse different table resources in different ways, flexible rewrite engine means egress encap or tunnels or logic can be done. There's also often more capacity for recirc or additional stages as required. Specific to Jericho, the underlying silicon has all these characteristics. We [*] used the flexibility in all of the stages both now and in previous iterations (Arad) to add new features/functionality that wasn't natively there to start with. And it uses a combination of on-chip & off-chip buffering with VoQ Its also not only Arista that call it a router cisco do too (NCS5K5). Sure, using a NPU for packet processing essentially provided a 100% programmable packet forwarding pipeline, and maybe even a "run to completion" kind of packet pipeline where the pipeline could have a long tail of processing. However, engineering is a zero sum game, and to do that means you sacrifice power or density, or most often, both. I agree the lines have been blurred as to the characteristics, and we'd openly state that its not going to be useful in every use case of where a router is deployed, but for specific use cases, it fits the bill and has compelling density, performance and cost dynamics. To the OPs question, there are people running with this in EFT and others in production. My suggestion would be that if you think its of interest, reach out to your friendly Arista person [*] and try it out or talk through what it is you're after. We are generally a friendly bunch and often we can be quite creative in enabling things in different ways to old.
Yeah they are certainly much behind in features, but if you don't need those features, it's probably actually an advantage. For my use-cases Arista's MPLS stack is not there.
We've historically had the data-plane but not the control-plane. Thats a work in progress. Again, often there are creative solutions to ways of doing things that aren't necessarily the same as old ways but achieve the same end result. cheers, lincoln. [*] disclosure: i work on said products described ltd@arista.com.
Saku, I guess you are right the QFX10002-36Q is probably a better comparison. But let's be honest, Juniper is not going to sell a QFX10002-36Q for less than $20k like Arista will do for a semi- similar box. Even with a high discount (like 90 percent off list), the Juniper QFX10002-36Q at $360k list price comes nowhere close on the price point. Cisco, Juniper, ALU, etc are all not going to see a low cost high density fixed switch because that would cannibalize on their sales on the larger platforms. I really think Arista is kind of unique here as they don't have another routing platform to cannibalize, so they are competitively pricing their platform. So I guess the question becomes, what features are missing that Arista does not currently have? They seems to be adding more and more features, and taking more market share. Here is a list of features supported: https://www.arista.com/en/support/product-documentation/supported-features I have not personally used Arista myself, but I like what I am seeing as far as price point, company culture, and repruatation in the market place. I know their switching is solid, but I am not sure about their routing. Arista claims to have much, much faster BGP convergence time than all the other vendors. On Sat, Apr 23, 2016 at 1:20 PM, Saku Ytti <saku@ytti.fi> wrote:
On 23 April 2016 at 10:52, Tom Hill <tom@ninjabadger.net> wrote:
In broad strokes: for your money you're either getting port density, or more features per port. The only difference here is that there's suddenly more TCAM on the device, and I still don't see the above changing too drastically.
Yeah OP is comparing high touch chip (MX104) to low touch chip (Jericho) that is not fair comparison. And cost is what customer is willing to pay, regardless of sticker on the box. No one will pay significant mark-up for another sticker, I've never seen in RFP significant differences in comparable products.
Fairer comparison would be QFX10k, instead of MX104. QFX10k is AFAIK only product in this segment which is not using Jericho. If this is competitive advantage or risk, jury is still out, I lean towards competitive advantage, mainly due to its memory design.
-- ++ytti
On 24 April 2016 at 09:08, Colton Conor <colton.conor@gmail.com> wrote: Hey,
I guess you are right the QFX10002-36Q is probably a better comparison. But let's be honest, Juniper is not going to sell a QFX10002-36Q for less than $20k like Arista will do for a semi- similar box. Even with a high discount (like 90 percent off list), the Juniper QFX10002-36Q at $360k list price comes nowhere close on the price point. Cisco, Juniper, ALU, etc are all not going to see a low cost high density fixed switch because that would cannibalize on their sales on the larger platforms. I really think Arista is kind of unique here as they don't have another routing platform to cannibalize, so they are competitively pricing their platform.
20k seems a stretch, that's like 94.5% discount, it's not unheard off. If you have volume, I would imagine it being doable.
So I guess the question becomes, what features are missing that Arista does not currently have? They seems to be adding more and more features, and taking more market share. Here is a list of features supported: https://www.arista.com/en/support/product-documentation/supported-features I have not personally used Arista myself, but I like what I am seeing as far as price point, company culture, and repruatation in the market place. I know their switching is solid, but I am not sure about their routing.
Yeah they are ccertainly much behind in features, but if you don't need those features, it's probably actually an advantage. For my use-cases Arista's MPLS stack is not there.
Arista claims to have much, much faster BGP convergence time than all the other vendors.
I wouldn't be surprised, but honestly the competition does not set the bar high there. -- ++ytti
While the QFX in general is similar to Jericho-based platforms, I think the QFX10002 is perhaps not an ideal comparison. At 100G, there is a significant density penalty on that platform, as you can use all 36 ports at 40G, but only 12 ports at 100G. BGP convergence in the newer EOS releases is indeed very, very fast. On Sun, Apr 24, 2016 at 12:08 PM, Colton Conor <colton.conor@gmail.com> wrote:
Saku,
I guess you are right the QFX10002-36Q is probably a better comparison. But let's be honest, Juniper is not going to sell a QFX10002-36Q for less than $20k like Arista will do for a semi- similar box. Even with a high discount (like 90 percent off list), the Juniper QFX10002-36Q at $360k list price comes nowhere close on the price point. Cisco, Juniper, ALU, etc are all not going to see a low cost high density fixed switch because that would cannibalize on their sales on the larger platforms. I really think Arista is kind of unique here as they don't have another routing platform to cannibalize, so they are competitively pricing their platform.
So I guess the question becomes, what features are missing that Arista does not currently have? They seems to be adding more and more features, and taking more market share. Here is a list of features supported: https://www.arista.com/en/support/product-documentation/supported-features I have not personally used Arista myself, but I like what I am seeing as far as price point, company culture, and repruatation in the market place. I know their switching is solid, but I am not sure about their routing.
Arista claims to have much, much faster BGP convergence time than all the other vendors.
On Sat, Apr 23, 2016 at 1:20 PM, Saku Ytti <saku@ytti.fi> wrote:
On 23 April 2016 at 10:52, Tom Hill <tom@ninjabadger.net> wrote:
In broad strokes: for your money you're either getting port density, or more features per port. The only difference here is that there's suddenly more TCAM on the device, and I still don't see the above changing too drastically.
Yeah OP is comparing high touch chip (MX104) to low touch chip (Jericho) that is not fair comparison. And cost is what customer is willing to pay, regardless of sticker on the box. No one will pay significant mark-up for another sticker, I've never seen in RFP significant differences in comparable products.
Fairer comparison would be QFX10k, instead of MX104. QFX10k is AFAIK only product in this segment which is not using Jericho. If this is competitive advantage or risk, jury is still out, I lean towards competitive advantage, mainly due to its memory design.
-- ++ytti
Just wanted to interject, the port density of the Arista switches is quite impressive, especially considering the price point they're at. On Tue, Apr 26, 2016 at 12:46 PM, Ryan Woolley <rwoolleynanog@gmail.com> wrote:
While the QFX in general is similar to Jericho-based platforms, I think the QFX10002 is perhaps not an ideal comparison. At 100G, there is a significant density penalty on that platform, as you can use all 36 ports at 40G, but only 12 ports at 100G.
BGP convergence in the newer EOS releases is indeed very, very fast.
On Sun, Apr 24, 2016 at 12:08 PM, Colton Conor <colton.conor@gmail.com> wrote:
Saku,
I guess you are right the QFX10002-36Q is probably a better comparison. But let's be honest, Juniper is not going to sell a QFX10002-36Q for less than $20k like Arista will do for a semi- similar box. Even with a high discount (like 90 percent off list), the Juniper QFX10002-36Q at $360k list price comes nowhere close on the price point. Cisco, Juniper, ALU, etc are all not going to see a low cost high density fixed switch because that would cannibalize on their sales on the larger platforms. I really think Arista is kind of unique here as they don't have another routing platform to cannibalize, so they are competitively pricing their platform.
So I guess the question becomes, what features are missing that Arista does not currently have? They seems to be adding more and more features, and taking more market share. Here is a list of features supported:
I have not personally used Arista myself, but I like what I am seeing as far as price point, company culture, and repruatation in the market
https://www.arista.com/en/support/product-documentation/supported-features place.
I know their switching is solid, but I am not sure about their routing.
Arista claims to have much, much faster BGP convergence time than all the other vendors.
On Sat, Apr 23, 2016 at 1:20 PM, Saku Ytti <saku@ytti.fi> wrote:
On 23 April 2016 at 10:52, Tom Hill <tom@ninjabadger.net> wrote:
In broad strokes: for your money you're either getting port density, or more features per port. The only difference here is that there's suddenly more TCAM on the device, and I still don't see the above changing too drastically.
Yeah OP is comparing high touch chip (MX104) to low touch chip (Jericho) that is not fair comparison. And cost is what customer is willing to pay, regardless of sticker on the box. No one will pay significant mark-up for another sticker, I've never seen in RFP significant differences in comparable products.
Fairer comparison would be QFX10k, instead of MX104. QFX10k is AFAIK only product in this segment which is not using Jericho. If this is competitive advantage or risk, jury is still out, I lean towards competitive advantage, mainly due to its memory design.
-- ++ytti
-- Regards, Paras President ProTraf Solutions, LLC Enterprise DDoS Mitigation
Just wanted to interject, the port density of the Arista switches is quite impressive, especially considering the price point they're at.
Not in response to any point specifically, but the major issue which stopped us buying Arista a few months ago was the rather out-dated attitude to 3rd party transceiver support. I'm sure there are plenty of people running Arista on 3rd party optics, but all the noises that were being made by the sales and technical guys suggested that we could find ourselves abandoned by their support or a policy change in the future. I don't fundamentally have an issue with vendor optics, except when they are excessively priced. One or two vendors will actually sell their 10Gbps optics at a price that's pretty hard to refuse, given that it's all supported. The same couldn't be said in this case. Additionally, the insistence that we would have to buy a "small number" of Arista optics with each device for testing purposes gets old very quickly. Again, I could get on-board with this if it's just for troubleshooting, but not when these additional optics suddenly add 15% to the overall buy price of each switch. At that point, other vendors are firmly back on the table. YMMV of course, I suspect especially if you're buying 100s of boxes. T
On Thu, Apr 28, 2016 at 2:22 PM, Timothy Creswick <Timothy.Creswick@vorboss.com> wrote:
Not in response to any point specifically, but the major issue which stopped us buying Arista a few months ago was the rather out-dated attitude to 3rd party transceiver support.
I'm sure there are plenty of people running Arista on 3rd party optics, but all the noises that were being made by the sales and technical guys suggested that we could find ourselves abandoned by their support or a policy change in the future.
We're going to be getting some Arista gear soon and this issue came up. They made the same noises and vague overtures of "well, you *might* have problems with TAC if you go with 3rd party optics"... until I said "Oh really- well, that's a deal breaker, we can't really even consider that". And then they backpedaled at light speed and reassured me that 3rd party optics would be fine, they just "had to have the conversation". Also talked to a local Arista customer, much bigger than us and using a lot more of their gear. They have 0 Arista optics and 0 problems with 3rd party for a few years now. IMHO the whole thing was just sales guy FUD to try to squeeze a few extra bucks out.
We're going to be getting some Arista gear soon and this issue came up. They made the same noises and vague overtures of "well, you *might* have problems with TAC if you go with 3rd party optics"... until I said "Oh really- well, that's a deal breaker, we can't really even consider that". And then they backpedaled at light speed and reassured me that 3rd party optics would be fine, they just "had to have the conversation".
Similar experience here, but the conversation went on far too long and ultimately lost Arista the deal. There was a ridiculous amount of insistence that we would have to carry a "stock of Arista optics", but every attempt to clarify exactly what that meant (how many, what they would cost etc) failed to get a straight answer. It's 2016 and stupid conversations about vendor optics waste time and destroy deals. The slight difference here is that pretty much the first thing we said to Arista was that transceivers were out of the question unless they could price them reasonably** (they chose not to). On this particular deal we were probably only talking about 500 SR 10G transceivers. We've had similar conversations with Extreme, Brocade, Solarflare and Juniper, all of whom are quite happy with us running our own parts. Solarflare even certified our parts and put them on their website (http://solarflare.com/transceivers-and-cables).
Also talked to a local Arista customer, much bigger than us and using a lot more of their gear. They have 0 Arista optics and 0 problems with 3rd party for a few years now. IMHO the whole thing was just sales guy FUD to try to squeeze a few extra bucks out.
Doesn't surprise me, and I'm sure if we'd pushed for another week we could have got to this position. Unfortunately for Arista, there was another vendor quite happy to get the deal done faster and without all the BS so we voted with our feet. Clearly, mileage will vary on this one. T ** in this context, "reasonably" means no more than _double_ what I currently buy at.
participants (11)
-
Colton Conor -
Jeff Tantsura -
Keith Medcalf -
lincoln dale -
Paras Jha -
Rob van de Logt -
Ryan Woolley -
Saku Ytti -
Thomas Penrose -
Timothy Creswick -
Tom Hill