Recommended L2 switches for a new IXP
Dear Nanog community We are trying to build a new IXP in some US Metro areas where we have multiple POPs and I was wondering what do you recommend for L2 switches. I know that some IXPs use Nexus, Brocade, Force10 but I don't personally have experience with these switches. It would be great if you can share your experience and recommendations. There are so many options that I don't know if it makes sense to start with a modular switch (usually expensive because the backplane, dual dc, dual CPU, etc) or start with a 1RU high density switch that support new protocols like Trill and that supposedly allow you to create Ethernet Fabric/Clusters. The requirements are simple, 1G/10G ports for exchange participants, 40G/100G for uplinks between switches and flow support for statistics and traffic analysis. Thank you and have a great day. Regards
I look forward to this thread. I think one important thing is who is your addressable market size? I'm working with a startup IXP and there's only 20 carriers in the building. A chassis based switch would be silly as there would never be that many people present. 2x 1U switches would be more than plenty in their environment. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com ----- Original Message ----- From: "Manuel Marín" <mmg@transtelco.net> To: nanog@nanog.org Sent: Monday, January 12, 2015 12:35:15 AM Subject: Recommended L2 switches for a new IXP Dear Nanog community We are trying to build a new IXP in some US Metro areas where we have multiple POPs and I was wondering what do you recommend for L2 switches. I know that some IXPs use Nexus, Brocade, Force10 but I don't personally have experience with these switches. It would be great if you can share your experience and recommendations. There are so many options that I don't know if it makes sense to start with a modular switch (usually expensive because the backplane, dual dc, dual CPU, etc) or start with a 1RU high density switch that support new protocols like Trill and that supposedly allow you to create Ethernet Fabric/Clusters. The requirements are simple, 1G/10G ports for exchange participants, 40G/100G for uplinks between switches and flow support for statistics and traffic analysis. Thank you and have a great day. Regards
We used to use Brocade FastIrons until we needed more 10G port density. We moved to Brocade SX's. Originally, when it was 2 or 3 peers, we used an old Netgear switch. :) Aaron On 1/12/2015 7:07 AM, Mike Hammett wrote:
I look forward to this thread.
I think one important thing is who is your addressable market size? I'm working with a startup IXP and there's only 20 carriers in the building. A chassis based switch would be silly as there would never be that many people present. 2x 1U switches would be more than plenty in their environment.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
----- Original Message -----
From: "Manuel Marín" <mmg@transtelco.net> To: nanog@nanog.org Sent: Monday, January 12, 2015 12:35:15 AM Subject: Recommended L2 switches for a new IXP
Dear Nanog community
We are trying to build a new IXP in some US Metro areas where we have multiple POPs and I was wondering what do you recommend for L2 switches. I know that some IXPs use Nexus, Brocade, Force10 but I don't personally have experience with these switches. It would be great if you can share your experience and recommendations. There are so many options that I don't know if it makes sense to start with a modular switch (usually expensive because the backplane, dual dc, dual CPU, etc) or start with a 1RU high density switch that support new protocols like Trill and that supposedly allow you to create Ethernet Fabric/Clusters. The requirements are simple, 1G/10G ports for exchange participants, 40G/100G for uplinks between switches and flow support for statistics and traffic analysis.
Thank you and have a great day.
Regards
-- ================================================================ Aaron Wendel Chief Technical Officer Wholesale Internet, Inc. (AS 32097) (816)550-9030 http://www.wholesaleinternet.com ================================================================
Like Mike says, it depends on your market. Are these markets where there are existing exchanges? Cost per port is what we always look at. If we are going into a market where there won’t be much growth we look at Cisco and Force 10. Their cost per port is usually cheaper for smaller 10 Gig switches. You need something that is fairly robust. Reliability in an exchange is a key component. If you go with a non-chassis switch make sure you have redundancy in your design. We like Chassis based switches because they tend to be more robust. But thats just my take on it. Justin --- Justin Wilson j2sw@mtin.net http://www.mtin.net Managed Services – xISP Solutions – Data Centers http://www.thebrotherswisp.com Podcast about xISP topics http://www.midwest-ix.com Peering – Transit – Internet Exchange
On Jan 12, 2015, at 10:24 AM, Aaron <aaron@wholesaleinternet.net> wrote:
We used to use Brocade FastIrons until we needed more 10G port density. We moved to Brocade SX's.
Originally, when it was 2 or 3 peers, we used an old Netgear switch. :)
Aaron
On 1/12/2015 7:07 AM, Mike Hammett wrote:
I look forward to this thread.
I think one important thing is who is your addressable market size? I'm working with a startup IXP and there's only 20 carriers in the building. A chassis based switch would be silly as there would never be that many people present. 2x 1U switches would be more than plenty in their environment.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
----- Original Message -----
From: "Manuel Marín" <mmg@transtelco.net> To: nanog@nanog.org Sent: Monday, January 12, 2015 12:35:15 AM Subject: Recommended L2 switches for a new IXP
Dear Nanog community
We are trying to build a new IXP in some US Metro areas where we have multiple POPs and I was wondering what do you recommend for L2 switches. I know that some IXPs use Nexus, Brocade, Force10 but I don't personally have experience with these switches. It would be great if you can share your experience and recommendations. There are so many options that I don't know if it makes sense to start with a modular switch (usually expensive because the backplane, dual dc, dual CPU, etc) or start with a 1RU high density switch that support new protocols like Trill and that supposedly allow you to create Ethernet Fabric/Clusters. The requirements are simple, 1G/10G ports for exchange participants, 40G/100G for uplinks between switches and flow support for statistics and traffic analysis.
Thank you and have a great day.
Regards
-- ================================================================ Aaron Wendel Chief Technical Officer Wholesale Internet, Inc. (AS 32097) (816)550-9030 http://www.wholesaleinternet.com ================================================================
Substantial amounts of hive mind went into this topic in the formation of Open-IX and particularly around optimizing costs and maximizing traffic. See http://bit.ly/N-OIX1 for a reference. Best, -M< On Mon, Jan 12, 2015 at 10:34 AM, Justin Wilson - MTIN <lists@mtin.net> wrote:
Like Mike says, it depends on your market. Are these markets where there are existing exchanges?
Cost per port is what we always look at. If we are going into a market where there won't be much growth we look at Cisco and Force 10. Their cost per port is usually cheaper for smaller 10 Gig switches. You need something that is fairly robust.
Reliability in an exchange is a key component. If you go with a non-chassis switch make sure you have redundancy in your design. We like Chassis based switches because they tend to be more robust. But thats just my take on it.
Justin
--- Justin Wilson j2sw@mtin.net http://www.mtin.net Managed Services - xISP Solutions - Data Centers http://www.thebrotherswisp.com Podcast about xISP topics http://www.midwest-ix.com Peering - Transit - Internet Exchange
On Jan 12, 2015, at 10:24 AM, Aaron <aaron@wholesaleinternet.net> wrote:
We used to use Brocade FastIrons until we needed more 10G port density. We moved to Brocade SX's.
Originally, when it was 2 or 3 peers, we used an old Netgear switch. :)
Aaron
I look forward to this thread.
I think one important thing is who is your addressable market size? I'm working with a startup IXP and there's only 20 carriers in the building. A chassis based switch would be silly as there would never be that many
On 1/12/2015 7:07 AM, Mike Hammett wrote: people present. 2x 1U switches would be more than plenty in their environment.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
----- Original Message -----
From: "Manuel Marín" <mmg@transtelco.net> To: nanog@nanog.org Sent: Monday, January 12, 2015 12:35:15 AM Subject: Recommended L2 switches for a new IXP
Dear Nanog community
We are trying to build a new IXP in some US Metro areas where we have multiple POPs and I was wondering what do you recommend for L2
switches. I
know that some IXPs use Nexus, Brocade, Force10 but I don't personally have experience with these switches. It would be great if you can share your experience and recommendations. There are so many options that I don't know if it makes sense to start with a modular switch (usually expensive because the backplane, dual dc, dual CPU, etc) or start with a 1RU high density switch that support new protocols like Trill and that supposedly allow you to create Ethernet Fabric/Clusters. The requirements are simple, 1G/10G ports for exchange participants, 40G/100G for uplinks between switches and flow support for statistics and traffic analysis.
Thank you and have a great day.
Regards
-- ================================================================ Aaron Wendel Chief Technical Officer Wholesale Internet, Inc. (AS 32097) (816)550-9030 http://www.wholesaleinternet.com ================================================================
On Jan 12, 2015, at 10:34 AM, Justin Wilson - MTIN <lists@mtin.net> wrote: Cost per port is what we always look at. If we are going into a market where there won’t be much growth we look at Cisco and Force 10. Their cost per port is usually cheaper for smaller 10 Gig switches. You need something that is fairly robust.
We see a lot of IXPs being formed or upgrading with Cisco Nexus 3524 switches, which have 48 1G-10G SFP/SFP+ physical ports, license-limited to 24 active, upgradeable to 48 active. FWIW, 83% of IXPs have 48 or fewer participants, and 70% of IXPs have 24 or fewer participants. And the failure rate of chassis-based switches is _way_ higher than that of stand-alone switches. So we never recommend that an IXP buy a switch larger than necessary to accommodate 18 months reasonably-projectable growth. -Bill
On Monday, January 12, 2015 05:54:38 PM Bill Woodcock wrote:
We see a lot of IXPs being formed or upgrading with Cisco Nexus 3524 switches, which have 48 1G-10G SFP/SFP+ physical ports, license-limited to 24 active, upgradeable to 48 active.
FWIW, 83% of IXPs have 48 or fewer participants, and 70% of IXPs have 24 or fewer participants. And the failure rate of chassis-based switches is _way_ higher than that of stand-alone switches. So we never recommend that an IXP buy a switch larger than necessary to accommodate 18 months reasonably-projectable growth.
Would tend to agree with this approach, and the above. Multi-rate (i.e., 1Gbps/10Gbps SFP/SFP+) standalone 1U switches are reasonable these days. The issue you'll probably run into with them is limited support for features you find being implemented by larger exchange points (VPLS, Sflow, e.t.c.), and quirks with the hardware that could impact things like Layer 2 or Layer 3 filtering (especially if they are using off-the-self silicon), e.t.c. Test before you buy, in as far as you can anticipate your (growth) needs. Mark.
People seem to be avoiding recommending actual devices, well I would recommend the Juniper EX4600 - http://www.juniper.net/us/en/products-services/switching/ex-series/ex4600/ They are affordable, highly scalable, stackable and run JunOS. cheers
That's what I had recommended him directly ;) Mehmet
On Jan 12, 2015, at 1:41 PM, Tony Wicks <tony@wicks.co.nz> wrote:
People seem to be avoiding recommending actual devices, well I would recommend the Juniper EX4600 -
http://www.juniper.net/us/en/products-services/switching/ex-series/ex4600/
They are affordable, highly scalable, stackable and run JunOS.
cheers
On Mon, Jan 12, 2015 at 4:41 PM, Tony Wicks <tony@wicks.co.nz> wrote:
People seem to be avoiding recommending actual devices, well I would recommend the Juniper EX4600 -
http://www.juniper.net/us/en/products-services/switching/ex-series/ex4600/
They are affordable, highly scalable, stackable and run JunOS.
(and you can't do anything worthwhile for acls to protect that device from the world/ix-users)
On Monday, January 12, 2015 11:41:20 PM Tony Wicks wrote:
People seem to be avoiding recommending actual devices, well I would recommend the Juniper EX4600 -
http://www.juniper.net/us/en/products-services/switching/ ex-series/ex4600/
They are affordable, highly scalable, stackable and run JunOS.
We've been quite happy with the EX4550, but the EX4600 is good too, particularly if you're coming from its younger brother. Mark.
Is there any particular reason you prefer EX4600 over QFX5100 ? Not counting obvious differences like ports and upgrade options. It's the same chipset after all, and with all upgrades they have the same 10G density (with breakouts). Is that because you can have more 40G ports with EX4600 ? I'm still trying to find out if there are any noticeable software or feature differences. On 13.01.2015 09:01, Mark Tinka wrote:
On Monday, January 12, 2015 11:41:20 PM Tony Wicks wrote:
People seem to be avoiding recommending actual devices, well I would recommend the Juniper EX4600 -
http://www.juniper.net/us/en/products-services/switching/ ex-series/ex4600/
They are affordable, highly scalable, stackable and run JunOS.
We've been quite happy with the EX4550, but the EX4600 is good too, particularly if you're coming from its younger brother.
Mark.
QFX5100 is SDN ready. -- Eduardo Schoedler 2015-01-13 6:29 GMT-02:00 Stepan Kucherenko <twh@megagroup.ru>:
Is there any particular reason you prefer EX4600 over QFX5100 ? Not counting obvious differences like ports and upgrade options.
It's the same chipset after all, and with all upgrades they have the same 10G density (with breakouts). Is that because you can have more 40G ports with EX4600 ?
I'm still trying to find out if there are any noticeable software or feature differences.
On 13.01.2015 09:01, Mark Tinka wrote:
On Monday, January 12, 2015 11:41:20 PM Tony Wicks wrote:
People seem to be avoiding recommending actual devices, well I would recommend the Juniper EX4600 -
http://www.juniper.net/us/en/products-services/switching/ ex-series/ex4600/
They are affordable, highly scalable, stackable and run JunOS.
We've been quite happy with the EX4550, but the EX4600 is good too, particularly if you're coming from its younger brother.
Mark.
-- Eduardo Schoedler
What does it mean - to be SDN ready? Cheers, Jeff -----Original Message----- From: Eduardo Schoedler <listas@esds.com.br> Date: Tuesday, January 13, 2015 at 3:25 AM To: "nanog@nanog.org" <nanog@nanog.org> Subject: Re: Recommended L2 switches for a new IXP
QFX5100 is SDN ready.
-- Eduardo Schoedler
2015-01-13 6:29 GMT-02:00 Stepan Kucherenko <twh@megagroup.ru>:
Is there any particular reason you prefer EX4600 over QFX5100 ? Not counting obvious differences like ports and upgrade options.
It's the same chipset after all, and with all upgrades they have the same 10G density (with breakouts). Is that because you can have more 40G ports with EX4600 ?
I'm still trying to find out if there are any noticeable software or feature differences.
On 13.01.2015 09:01, Mark Tinka wrote:
On Monday, January 12, 2015 11:41:20 PM Tony Wicks wrote:
People seem to be avoiding recommending actual devices, well I would recommend the Juniper EX4600 -
http://www.juniper.net/us/en/products-services/switching/ ex-series/ex4600/
They are affordable, highly scalable, stackable and run JunOS.
We've been quite happy with the EX4550, but the EX4600 is good too, particularly if you're coming from its younger brother.
Mark.
-- Eduardo Schoedler
AhhhŠ vertically integrated horizontal API¹s Cheers, Jeff -----Original Message----- From: Nick Hilliard <nick@foobar.org> Date: Tuesday, January 13, 2015 at 2:23 PM To: Jeff Tantsura <jeff.tantsura@ericsson.com>, Eduardo Schoedler <listas@esds.com.br>, "nanog@nanog.org" <nanog@nanog.org> Subject: Re: Recommended L2 switches for a new IXP
On 13/01/2015 22:10, Jeff Tantsura wrote:
What does it mean - to be SDN ready?
it means "fully buzzword compliant".
Nick
My mistake, it's the OCX1100. http://www.networkworld.com/article/2855056/sdn/juniper-unbundles-switch-har... 2015-01-13 20:10 GMT-02:00 Jeff Tantsura <jeff.tantsura@ericsson.com>:
What does it mean - to be SDN ready?
Cheers, Jeff
-----Original Message----- From: Eduardo Schoedler <listas@esds.com.br> Date: Tuesday, January 13, 2015 at 3:25 AM To: "nanog@nanog.org" <nanog@nanog.org> Subject: Re: Recommended L2 switches for a new IXP
QFX5100 is SDN ready.
-- Eduardo Schoedler
2015-01-13 6:29 GMT-02:00 Stepan Kucherenko <twh@megagroup.ru>:
Is there any particular reason you prefer EX4600 over QFX5100 ? Not counting obvious differences like ports and upgrade options.
It's the same chipset after all, and with all upgrades they have the same 10G density (with breakouts). Is that because you can have more 40G ports with EX4600 ?
I'm still trying to find out if there are any noticeable software or feature differences.
On 13.01.2015 09:01, Mark Tinka wrote:
On Monday, January 12, 2015 11:41:20 PM Tony Wicks wrote:
People seem to be avoiding recommending actual devices, well I would recommend the Juniper EX4600 -
http://www.juniper.net/us/en/products-services/switching/ ex-series/ex4600/
They are affordable, highly scalable, stackable and run JunOS.
We've been quite happy with the EX4550, but the EX4600 is good too, particularly if you're coming from its younger brother.
Mark.
-- Eduardo Schoedler
-- Eduardo Schoedler
Either way, you can do "SDN" and automation with most Juniper kit. On purchase of JCare you get free access to Junos Space - great for provisioning and management of an IXP. Regards, Tim Raphael
On 14 Jan 2015, at 6:28 am, Eduardo Schoedler <listas@esds.com.br> wrote:
My mistake, it's the OCX1100. http://www.networkworld.com/article/2855056/sdn/juniper-unbundles-switch-har...
2015-01-13 20:10 GMT-02:00 Jeff Tantsura <jeff.tantsura@ericsson.com>:
What does it mean - to be SDN ready?
Cheers, Jeff
-----Original Message----- From: Eduardo Schoedler <listas@esds.com.br> Date: Tuesday, January 13, 2015 at 3:25 AM To: "nanog@nanog.org" <nanog@nanog.org> Subject: Re: Recommended L2 switches for a new IXP
QFX5100 is SDN ready.
-- Eduardo Schoedler
2015-01-13 6:29 GMT-02:00 Stepan Kucherenko <twh@megagroup.ru>:
Is there any particular reason you prefer EX4600 over QFX5100 ? Not counting obvious differences like ports and upgrade options.
It's the same chipset after all, and with all upgrades they have the same 10G density (with breakouts). Is that because you can have more 40G ports with EX4600 ?
I'm still trying to find out if there are any noticeable software or feature differences.
On 13.01.2015 09:01, Mark Tinka wrote:
On Monday, January 12, 2015 11:41:20 PM Tony Wicks wrote:
People seem to be avoiding recommending actual devices, well I would recommend the Juniper EX4600 -
http://www.juniper.net/us/en/products-services/switching/ ex-series/ex4600/
They are affordable, highly scalable, stackable and run JunOS.
We've been quite happy with the EX4550, but the EX4600 is good too, particularly if you're coming from its younger brother.
Mark.
-- Eduardo Schoedler
-- Eduardo Schoedler
Got you - artificially disabling 90% of the features otherwise supported by the OS and using half baked HAL makes product SDN ready! Sorry for the sarcasm, couldn¹t resist :) Cheers, Jeff -----Original Message----- From: Eduardo Schoedler <listas@esds.com.br> Date: Tuesday, January 13, 2015 at 2:28 PM To: "nanog@nanog.org" <nanog@nanog.org> Subject: Re: Recommended L2 switches for a new IXP
My mistake, it's the OCX1100. http://www.networkworld.com/article/2855056/sdn/juniper-unbundles-switch-h ardware-software.html
2015-01-13 20:10 GMT-02:00 Jeff Tantsura <jeff.tantsura@ericsson.com>:
What does it mean - to be SDN ready?
Cheers, Jeff
-----Original Message----- From: Eduardo Schoedler <listas@esds.com.br> Date: Tuesday, January 13, 2015 at 3:25 AM To: "nanog@nanog.org" <nanog@nanog.org> Subject: Re: Recommended L2 switches for a new IXP
QFX5100 is SDN ready.
-- Eduardo Schoedler
2015-01-13 6:29 GMT-02:00 Stepan Kucherenko <twh@megagroup.ru>:
Is there any particular reason you prefer EX4600 over QFX5100 ? Not counting obvious differences like ports and upgrade options.
It's the same chipset after all, and with all upgrades they have the same 10G density (with breakouts). Is that because you can have more 40G ports with EX4600 ?
I'm still trying to find out if there are any noticeable software or feature differences.
On 13.01.2015 09:01, Mark Tinka wrote:
On Monday, January 12, 2015 11:41:20 PM Tony Wicks wrote:
People seem to be avoiding recommending actual devices, well I would recommend the Juniper EX4600 -
http://www.juniper.net/us/en/products-services/switching/ ex-series/ex4600/
They are affordable, highly scalable, stackable and run JunOS.
We've been quite happy with the EX4550, but the EX4600 is good too, particularly if you're coming from its younger brother.
Mark.
-- Eduardo Schoedler
-- Eduardo Schoedler
On Wednesday, January 14, 2015 12:47:09 AM Jeff Tantsura wrote:
Got you - artificially disabling 90% of the features otherwise supported by the OS and using half baked HAL makes product SDN ready! Sorry for the sarcasm, couldn¹t resist :)
I once tested a Junos release with the X blah blah D blah blah letters in there on an EX4550. Couldn't even get LACP going, until I realized it was some kind of QFX'y release for the non-QFX EX boxes. Promptly got ride of that. Mark.
Software Defined Networking (SDN) features that QFX5100 supports: Automatic configuration of OVSDB-managed VXLANs with trunk interfaces 14.1X53-D15 OVSDB support 14.1X53-D10 OpenFlow v1.0 14.1X53-D10 OpenFlow v1.3.1 14.1X53-D10 VXLAN Gateway 14.1X53-D10 http://pathfinder.juniper.net/feature-explorer/select-software.html?swName=Junos+OS&typ=1#family=&platform=QFX5100&rel=14.1X53-D15&swName=Junos+OS On Tue, Jan 13, 2015 at 10:10:56PM +0000, Jeff Tantsura wrote:
What does it mean - to be SDN ready?
Cheers, Jeff
-----Original Message----- From: Eduardo Schoedler <listas@esds.com.br> Date: Tuesday, January 13, 2015 at 3:25 AM To: "nanog@nanog.org" <nanog@nanog.org> Subject: Re: Recommended L2 switches for a new IXP
QFX5100 is SDN ready.
-- Eduardo Schoedler
2015-01-13 6:29 GMT-02:00 Stepan Kucherenko <twh@megagroup.ru>:
Is there any particular reason you prefer EX4600 over QFX5100 ? Not counting obvious differences like ports and upgrade options.
It's the same chipset after all, and with all upgrades they have the same 10G density (with breakouts). Is that because you can have more 40G ports with EX4600 ?
I'm still trying to find out if there are any noticeable software or feature differences.
On 13.01.2015 09:01, Mark Tinka wrote:
On Monday, January 12, 2015 11:41:20 PM Tony Wicks wrote:
People seem to be avoiding recommending actual devices, well I would recommend the Juniper EX4600 -
http://www.juniper.net/us/en/products-services/switching/ ex-series/ex4600/
They are affordable, highly scalable, stackable and run JunOS.
We've been quite happy with the EX4550, but the EX4600 is good too, particularly if you're coming from its younger brother.
Mark.
-- Eduardo Schoedler
We love our 5100s here. I have 4 48S, and 2 24q¹s. Super fast, TISSU when it works is awesome as well... like, really awesome. Stephen Carter | IT Systems Administrator | Gun Lake Tribal Gaming Commission 1123 129th Avenue, Wayland, MI 49348 Phone 269.792.1773 On 1/13/15, 3:29 AM, "Stepan Kucherenko" <twh@megagroup.ru> wrote:
Is there any particular reason you prefer EX4600 over QFX5100 ? Not counting obvious differences like ports and upgrade options.
It's the same chipset after all, and with all upgrades they have the same 10G density (with breakouts). Is that because you can have more 40G ports with EX4600 ?
I'm still trying to find out if there are any noticeable software or feature differences.
On 13.01.2015 09:01, Mark Tinka wrote:
On Monday, January 12, 2015 11:41:20 PM Tony Wicks wrote:
People seem to be avoiding recommending actual devices, well I would recommend the Juniper EX4600 -
http://www.juniper.net/us/en/products-services/switching/ ex-series/ex4600/
They are affordable, highly scalable, stackable and run JunOS.
We've been quite happy with the EX4550, but the EX4600 is good too, particularly if you're coming from its younger brother.
Mark.
<br><hr><font face='Arial' color='Gray' size='1'>The information contained in this electronic transmission (email) is confidential information and may be subject to attorney/client privilege. It is intended only for the use of the individual or entity named above. ANY DISTRIBUTION OR COPYING OF THIS MESSAGE IS PROHIBITED, except by the intended recipient. Attempts to intercept this message are in violation of 18 U.S.C. 2511(1) of the Electronic Communications Privacy Act (ECPA), which subjects the interceptor to fines, imprisonment and/or civil damages.</font>
On Tue, Jan 13, 2015 at 4:45 PM, Stephen R. Carter <stephen.carter@gltgc.org> wrote:
We love our 5100s here.
Out of interest: Are you running 13.2 or 14.1? What features are you using? Our own experiences with a bunch of 48 & 96 port machines running 14.1 is painful to say the least. Richard
We always adhere to JTAC: http://kb.juniper.net/InfoCenter/index?page=content&id=KB21476&actp=SUBSCRI PTION unless otherwise required by their support to change. Currently it is Junos 13.2X51-D26. My advice to you is to not use 14.1 unless you have a reason, as that is more of a dev branch in terms of stability than anything. We use VRRP, OSPF, MC-LAG, and so forth. Nothing super fancy. Stephen Carter | IT Systems Administrator | Gun Lake Tribal Gaming Commission 1123 129th Avenue, Wayland, MI 49348 Phone 269.792.1773 On 1/15/15, 4:17 AM, "Richard Hartmann" <richih.mailinglist@gmail.com> wrote:
On Tue, Jan 13, 2015 at 4:45 PM, Stephen R. Carter <stephen.carter@gltgc.org> wrote:
We love our 5100s here.
Out of interest: Are you running 13.2 or 14.1?
What features are you using?
Our own experiences with a bunch of 48 & 96 port machines running 14.1 is painful to say the least.
Richard
<br><hr><font face='Arial' color='Gray' size='1'>The information contained in this electronic transmission (email) is confidential information and may be subject to attorney/client privilege. It is intended only for the use of the individual or entity named above. ANY DISTRIBUTION OR COPYING OF THIS MESSAGE IS PROHIBITED, except by the intended recipient. Attempts to intercept this message are in violation of 18 U.S.C. 2511(1) of the Electronic Communications Privacy Act (ECPA), which subjects the interceptor to fines, imprisonment and/or civil damages.</font>
On 12/01/2015 06:35, Manuel Marín wrote:
We are trying to build a new IXP in some US Metro areas where we have multiple POPs and I was wondering what do you recommend for L2 switches. I know that some IXPs use Nexus, Brocade, Force10 but I don't personally have experience with these switches. It would be great if you can share your experience and recommendations.
For a startup IXP, it would probably not be sensible to use chassis based kit due to cost / real estate issues. Some personal opinions: - I have a strong preference for using only open bridging protocols. This excludes out vendor proprietary fabrics (VDX, OTV, etc). This is important for when you do fabric upgrades on multi-site IXPs. - You will probably want a product which supports sflow, as peer-to-peer traffic graphs are massively useful. Most vendors support sflow on most of their products with the notable exception of Cisco where only the Nexus 3K team were enlightened enough to shim it in. I haven't yet come across a L2 netflow implementation which works well enough to be an adequate substitute, but ymmv. - VPLS based fabrics may be important if you have an interesting topology. If it is important to you, then you will need a VPLS implementation which will do proper load balancing over multiple links. Most don't and this is a very hard problem to handle on smaller kit. - There is no excuse for vendor transceiver locking or transceiver crippling (e.g. refusing to show DDM values) and vendors who do this need to be made aware that it's not an acceptable business proposition. - you need kit which will support Layer 2 ACLs and Layer 3 ACLs on layer 2 interfaces. - you should get in with the open-ix crowd and chat to people over pizza or peanuts. You will learn a lot from in an afternoon of immersion with peers. Nick
On Mon, Jan 12, 2015 at 10:43 AM, Nick Hilliard <nick@foobar.org> wrote: [ clip, good stuff ] - you should get in with the open-ix crowd and chat to people over pizza or
peanuts. You will learn a lot from in an afternoon of immersion with peers.
And you can find that crowd here http://mailman.open-ix.org/mailman/listinfo/public if interested. Best, -M<
Manuel Marín writes:
Dear Nanog community [...] There are so many options that I don't know if it makes sense to start with a modular switch (usually expensive because the backplane, dual dc, dual CPU, etc) or start with a 1RU high density switch that support new protocols like Trill and that supposedly allow you to create Ethernet Fabric/Clusters. The requirements are simple, 1G/10G ports for exchange participants, 40G/100G for uplinks between switches and flow support for statistics and traffic analysis.
Stupid thought from someone who has never built an IXP, but has been looking at recent trends in data center networks: There are these "white-box" switches mostly designed for top-of-rack or spine (as in leaf-spine/fat-tree datacenter networks) applications. They have all the necessary port speeds - well 100G seems to be a few months off. I'm thinking of brands such as Edge-Core, Quanta etc. You can get them as "bare-metal" versions with no switch OS on them, just a bootloader according to the "ONIE" standard. Equipment cost seems to be on the order of $100 per SFP+ port w/o optics for a second-to-last generation (Trident-based) 48*10GE+4*40GE ToR switch. Now, for the limited and somewhat special L2 needs of an IXP, couldn't "someone" hack together a suitable switch OS based on Open Network Linux (ONL) or something like that? You wouldn't even need MAC address learning or most types of flooding, because at an IXP this often hurts rather than helps. For building larger fabrics you might be using something other (waves hands) than TRILL; maybe you could get away without slightly complex "multi-chassis multi-channel" mechanisms, and so on. "Flow support" sounds somewhat tough, but full netflow support that would get Roland Dobbins' "usable telemetry" seal of approval is probably out of reach anyway - it's a high-end feature with classical gear. With white-box switches, you could try to use the given 5-tuple flow hardware capabilities - which might not scale that well -, or use packet sampling, or try to use the built-in flow and counter mechanisms in an application-specific way. (Except *that's* a lot of work on the software side, and a usably efficient implementation requires slightly sophisticated hardware/software interfaces.) Instead of a Linux-based switch OS, one could also build an IXP "application" using OpenFlow and some kind of central controller. (Not to be confused with "SDX: Software Defined Internet Exchange".) Has anybody looked into the feasibility of this? The software could be done as an open-source community project to make setting up regional IXPs easier/cheaper. Large IXPs could sponsor this so they get better scalability - although I'm not sure how well something like the leaf-spine/fat-tree design maps to these IXPs, which are typically distributed over several locations. Maybe they could use something like Facebook's new design, treating each IXP location as a "pod". -- Simon. [1] https://code.facebook.com/posts/360346274145943
participants (19)
-
Aaron -
Bill Woodcock -
Christopher Morrow -
Chuck Anderson -
Eduardo Schoedler -
Jeff Tantsura -
Justin Wilson - MTIN -
Manuel Marín -
Mark Tinka -
Martin Hannigan -
Mehmet Akcin -
Mike Hammett -
Nick Hilliard -
Richard Hartmann -
Simon Leinen -
Stepan Kucherenko -
Stephen R. Carter -
Tim Raphael -
Tony Wicks