FW: Force10 E Series at the edge?
Is anyone running an E300 Series Chassis at the internet edge with multiple Full BGP feeds? 95th percent would be about 300 meg of traffic. BGP session count would be between 2 and 4 Peers. 6k internal Prefix count as it stands right now. Alternative are welcome. Thought about the ASR1006 but I need some local switching as well. Full requirements include Full internet Peering over GigE Links. Fully Redundant Power Redundant "Supervisor/Route Processor" Would prefer a Small Chassis unit. (under 10u) Would also prefer a single unit as opposed to a two smaller units. ________________________________ This email and any attached files may contain confidential and/or privileged material and is intended solely for the use of the person to whom it is addressed. Any review, retransmission, dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender immediately and delete it and all attachments from your computer. Progressive Solutions is not liable for any errors or omissions in the content or transmission of this email.
Have you look at Juniper's MX stuff. On Tue, Mar 27, 2012 at 5:21 PM, Roberts, Brent < Brent.Roberts@progressive-solutions.com> wrote:
Is anyone running an E300 Series Chassis at the internet edge with multiple Full BGP feeds? 95th percent would be about 300 meg of traffic. BGP session count would be between 2 and 4 Peers. 6k internal Prefix count as it stands right now. Alternative are welcome. Thought about the ASR1006 but I need some local switching as well.
Full requirements include Full internet Peering over GigE Links. Fully Redundant Power Redundant "Supervisor/Route Processor" Would prefer a Small Chassis unit. (under 10u) Would also prefer a single unit as opposed to a two smaller units.
________________________________
This email and any attached files may contain confidential and/or privileged material and is intended solely for the use of the person to whom it is addressed. Any review, retransmission, dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender immediately and delete it and all attachments from your computer. Progressive Solutions is not liable for any errors or omissions in the content or transmission of this email.
I was very happy with the E300 as a data center core switch handling multiple full feeds (around 15) with about 10x the traffic you are talking about. The only problem I had was that Force10 didn't have a useful (basically forklift) upgrade to get more IPv4 prefixes, and the more I talked to them and the more I showed them the graphs demonstrating what we'd need for prefix space assuming even the most conservative assumptions at depletion, the more I realized they really Did Not Get It. In fact, their brand new architecture recently announced had only 500k prefixes allowed, at a time that the Juniper MX platform handled 2million easily. So I would be fine using Force10 again, given the following changes: 1. Large limits on IP prefixes allowed 2. Reallocation of useless memory from stupid things like MAC tables to prefixes (data centers have very few MACs, very many prefixes) 3. Command line logging The units worked great at failover, never had any problems gracefully failing over from one RP to another, but if you have to cold boot them for any reason it takes like 5 minutes :( On Mar 27, 2012, at 2:21 PM, Roberts, Brent wrote:
Is anyone running an E300 Series Chassis at the internet edge with multiple Full BGP feeds? 95th percent would be about 300 meg of traffic. BGP session count would be between 2 and 4 Peers. 6k internal Prefix count as it stands right now. Alternative are welcome. Thought about the ASR1006 but I need some local switching as well.
Full requirements include Full internet Peering over GigE Links. Fully Redundant Power Redundant "Supervisor/Route Processor" Would prefer a Small Chassis unit. (under 10u) Would also prefer a single unit as opposed to a two smaller units.
________________________________
This email and any attached files may contain confidential and/or privileged material and is intended solely for the use of the person to whom it is addressed. Any review, retransmission, dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender immediately and delete it and all attachments from your computer. Progressive Solutions is not liable for any errors or omissions in the content or transmission of this email.
-- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Brent, Your options include, for smaller boxes: - Brocade CER series, but make sure you the -RT versions due to RAM (haven't tried, though) - Juniper MX (MX80 is working well for us) - Cisco ASR1006 (heard a lot about BGP price issues) But for 300mb/sec, what not OpenBSD + Quagga? Tom ----- Original Message -----
I was very happy with the E300 as a data center core switch handling multiple full feeds (around 15) with about 10x the traffic you are talking about. The only problem I had was that Force10 didn't have a useful (basically forklift) upgrade to get more IPv4 prefixes, and the more I talked to them and the more I showed them the graphs demonstrating what we'd need for prefix space assuming even the most conservative assumptions at depletion, the more I realized they really Did Not Get It. In fact, their brand new architecture recently announced had only 500k prefixes allowed, at a time that the Juniper MX platform handled 2million easily.
So I would be fine using Force10 again, given the following changes: 1. Large limits on IP prefixes allowed 2. Reallocation of useless memory from stupid things like MAC tables to prefixes (data centers have very few MACs, very many prefixes) 3. Command line logging
The units worked great at failover, never had any problems gracefully failing over from one RP to another, but if you have to cold boot them for any reason it takes like 5 minutes :(
On Mar 27, 2012, at 2:21 PM, Roberts, Brent wrote:
Is anyone running an E300 Series Chassis at the internet edge with multiple Full BGP feeds? 95th percent would be about 300 meg of traffic. BGP session count would be between 2 and 4 Peers. 6k internal Prefix count as it stands right now. Alternative are welcome. Thought about the ASR1006 but I need some local switching as well.
Full requirements include Full internet Peering over GigE Links. Fully Redundant Power Redundant "Supervisor/Route Processor" Would prefer a Small Chassis unit. (under 10u) Would also prefer a single unit as opposed to a two smaller units.
________________________________
This email and any attached files may contain confidential and/or privileged material and is intended solely for the use of the person to whom it is addressed. Any review, retransmission, dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender immediately and delete it and all attachments from your computer. Progressive Solutions is not liable for any errors or omissions in the content or transmission of this email.
-- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
--- On Tue, 3/27/12, Tom Daly <tom@dyn.com> wrote:
From: Tom Daly <tom@dyn.com> Subject: Re: Force10 E Series at the edge? To: "Brent Roberts" <Brent.Roberts@progressive-solutions.com> Cc: "NANOG" <nanog@nanog.org> Date: Tuesday, March 27, 2012, 8:59 PM Brent, Your options include, for smaller boxes:
- Brocade CER series, but make sure you the -RT versions due to RAM (haven't tried, though) - Juniper MX (MX80 is working well for us) - Cisco ASR1006 (heard a lot about BGP price issues)
But for 300mb/sec, what not OpenBSD + Quagga?
Tom
I was very happy with the E300 as a data center core switch handling multiple full feeds (around 15) with about 10x the
talking about. The only problem I had was that Force10 didn't have a useful (basically forklift) upgrade to get more IPv4
the more I talked to them and the more I showed them
demonstrating what we'd need for prefix space assuming even the most conservative assumptions at depletion, the more I realized they really Did Not Get It. In fact, their brand new architecture recently announced had only 500k prefixes allowed, at a time that the Juniper MX platform handled 2million easily.
So I would be fine using Force10 again, given the following changes: 1. Large limits on IP prefixes allowed 2. Reallocation of useless memory from stupid things like MAC tables to prefixes (data centers have very few MACs, very many prefixes) 3. Command line logging
The units worked great at failover, never had any
----- Original Message ----- traffic you are prefixes, and the graphs problems gracefully
failing over from one RP to another, but if you have to
cold boot > them for any reason it takes like 5 minutes :( > > On Mar 27, 2012, at 2:21 PM, Roberts, Brent wrote: > > Is anyone running an E300 Series Chassis at the internet edge with > > multiple Full BGP feeds? 95th percent would be about 300 meg of > > traffic. BGP session count would be between 2 and 4 Peers. > > 6k internal Prefix count as it stands right now. Alternative are > > welcome. Thought about the ASR1006 but I need some local switching > > as well. > > > > Full requirements include > > Full internet Peering over GigE Links. > > Fully Redundant Power > > Redundant "Supervisor/Route Processor" > > Would prefer a Small Chassis unit. (under 10u) > > Would also prefer a single unit as opposed to a two smaller units. > >
I can't speak for forece10 which is DELL now. As Joe mentioned, the biggest problem is "their-support" of 680k prefixes with the QUAD-CAM linecards. DUAL-CAM line cards do 512K in theory. Regular ones don't work because thay support 320K prefifex and "die" around 300K They have other idiotic-implementations(when to set/NOT set ospf forwarding-address) buggy vrrp implemtation but I am told "it will be fixed in the next release of FTOS. So, NO! the 300i, 600 or 120 are good a good fit as edge/core layer devices. On a sepatare note.....their S50 switches; I have found to be "great" as long as your l2 environment doesn't require Rapid-PVST. They do PVST but 802.1W is a single instance. ./Randy
I can't speak for forece10 which is DELL now. As Joe mentioned, the biggest problem is "their-support" of 680k prefixes with the QUAD-CAM linecards. DUAL-CAM line cards do 512K in theory. Regular ones don't work because thay support 320K prefifex and "die" around 300K
If memory serves, it's worse than that. Those numbers can only be achieved if you turn off IPv6 altogether IIRC. Owen
-----Original Message----- From: Tom Daly Sent: Tuesday, March 27, 2012 8:59 PM To: Brent Roberts Cc: NANOG Subject: Re: Force10 E Series at the edge?
Brent, Your options include, for smaller boxes:
- Brocade CER series, but make sure you the -RT versions due to RAM (haven't tried, though) - Juniper MX (MX80 is working well for us) - Cisco ASR1006 (heard a lot about BGP price issues)
But for 300mb/sec, what not OpenBSD + Quagga?
Tom
I have been using a pair of CER (but not the -RT) at one location for a while now and so far have been flawless. These particular units aren't taking full tables so don't need the -RT but I wouldn't have any trouble using them. The -RT are basically a 1U XMR.
On 3/27/12 23:21 , Roberts, Brent wrote:
Is anyone running an E300 Series Chassis at the internet edge with multiple Full BGP feeds? 95th percent would be about 300 meg of traffic. BGP session count would be between 2 and 4 Peers. 6k internal Prefix count as it stands right now. Alternative are welcome. Thought about the ASR1006 but I need some local switching as well.
Doesn't support URPF which makes it unsuitable for RTBH and therefore customer or transit edge IMHO. I do use them and like them reasonably well for high density over-subscription 10Gb/s ports. Can get 280 2x oversubscribed 10Gb/s ports or 560 4x layer 3 out of an e1200i.
Full requirements include Full internet Peering over GigE Links. Fully Redundant Power Redundant "Supervisor/Route Processor" Would prefer a Small Chassis unit. (under 10u) Would also prefer a single unit as opposed to a two smaller units.
________________________________
This email and any attached files may contain confidential and/or privileged material and is intended solely for the use of the person to whom it is addressed. Any review, retransmission, dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender immediately and delete it and all attachments from your computer. Progressive Solutions is not liable for any errors or omissions in the content or transmission of this email.
On Mar 28, 2012, at 11:48 AM, Joel jaeggli wrote:
On 3/27/12 23:21 , Roberts, Brent wrote:
Is anyone running an E300 Series Chassis at the internet edge with multiple Full BGP feeds? 95th percent would be about 300 meg of traffic. BGP Doesn't support URPF which makes it unsuitable for RTBH and therefore
I was just about to pipe up and say "they do it fine!" and then I remembered that we built automatic filtering provisioning so that each edge customer got filters applied automatically based on their static assignments from us, or from IRR tables if a checkbox was marked. The boxes handled 1000x ports with ~6 filters per port no problem, but yeah, real uRPF would be nice. -- Jo Rhett Net Consonance : net philanthropy to improve open source and internet projects.
On 5/7/12 21:17 , Jo Rhett wrote:
On Mar 28, 2012, at 11:48 AM, Joel jaeggli wrote:
On 3/27/12 23:21 , Roberts, Brent wrote:
Is anyone running an E300 Series Chassis at the internet edge with multiple Full BGP feeds? 95th percent would be about 300 meg of traffic. BGP Doesn't support URPF which makes it unsuitable for RTBH and therefore
I was just about to pipe up and say "they do it fine!" and then I remembered that we built automatic filtering provisioning so that each edge customer got filters applied automatically based on their static assignments from us, or from IRR tables if a checkbox was marked. The boxes handled 1000x ports with ~6 filters per port no problem, but yeah, real uRPF would be nice.
Yeah, there's enough CAM on EJ linecards for the resultant rules either way but I know what I'd prefer. I do use and am reasonably happy with exascale, just not in that role.
-- Jo Rhett Net Consonance : net philanthropy to improve open source and internet projects.
FYI: The E300 is the TeraScale series. If you're looking at used, be sure to get dual-cam cards or else you'll top out at 256k routes. Dual-cam should give you 512K/32K (v4/v6). Next step up would be the E600i with EJ RPM(s) which is the ExaScale series and supports up to 688k/128k (v4/v6) routes (EH RPM's still being the Terascale platform so definitely look for EJ's if considering the E600i). 300Mbps is nothing for the E300. Even 300Gbps is still within spec. It's rated for switching up to 400 Gbps and forwarding capacity of 196 Mpps. This might be of use to you: http://i.dell.com/sites/content/shared-content/data-sheets/en/Documents/Dell... Although I don't work in the Force10 group (I do work for Dell), if you have any questions I can likely track down the right contacts to help you. Contact me off list. I've been learning the product line myself and playing with an E600i in just the past few months coming from familiarity with Cisco and Brocade. If you haven't used Force10 and FTOS before but are familiar with Cisco IOS, you'll pick it up fast. -Vinny -----Original Message----- From: Jo Rhett [mailto:jrhett@netconsonance.com] Sent: Monday, May 07, 2012 5:17 PM To: Joel jaeggli Cc: NANOG Subject: Re: Force10 E Series at the edge? On Mar 28, 2012, at 11:48 AM, Joel jaeggli wrote:
On 3/27/12 23:21 , Roberts, Brent wrote:
Is anyone running an E300 Series Chassis at the internet edge with multiple Full BGP feeds? 95th percent would be about 300 meg of traffic. BGP Doesn't support URPF which makes it unsuitable for RTBH and therefore
I was just about to pipe up and say "they do it fine!" and then I remembered that we built automatic filtering provisioning so that each edge customer got filters applied automatically based on their static assignments from us, or from IRR tables if a checkbox was marked. The boxes handled 1000x ports with ~6 filters per port no problem, but yeah, real uRPF would be nice. -- Jo Rhett Net Consonance : net philanthropy to improve open source and internet projects.
Sorry... small correction. EH/EJ are line cards with different CAM sizes. RPM's for Terascale vs Exascale are EF vs EH. I'm getting my letters mixed up. :) -Vinny -----Original Message----- From: Abello, Vinny Sent: Monday, May 07, 2012 6:46 PM To: jrhett@netconsonance.com; joelja@bogus.com Cc: nanog@nanog.org Subject: RE: Force10 E Series at the edge? FYI: The E300 is the TeraScale series. If you're looking at used, be sure to get dual-cam cards or else you'll top out at 256k routes. Dual-cam should give you 512K/32K (v4/v6). Next step up would be the E600i with EJ RPM(s) which is the ExaScale series and supports up to 688k/128k (v4/v6) routes (EH RPM's still being the Terascale platform so definitely look for EJ's if considering the E600i). 300Mbps is nothing for the E300. Even 300Gbps is still within spec. It's rated for switching up to 400 Gbps and forwarding capacity of 196 Mpps. This might be of use to you: http://i.dell.com/sites/content/shared-content/data-sheets/en/Documents/Dell... Although I don't work in the Force10 group (I do work for Dell), if you have any questions I can likely track down the right contacts to help you. Contact me off list. I've been learning the product line myself and playing with an E600i in just the past few months coming from familiarity with Cisco and Brocade. If you haven't used Force10 and FTOS before but are familiar with Cisco IOS, you'll pick it up fast. -Vinny -----Original Message----- From: Jo Rhett [mailto:jrhett@netconsonance.com] Sent: Monday, May 07, 2012 5:17 PM To: Joel jaeggli Cc: NANOG Subject: Re: Force10 E Series at the edge? On Mar 28, 2012, at 11:48 AM, Joel jaeggli wrote:
On 3/27/12 23:21 , Roberts, Brent wrote:
Is anyone running an E300 Series Chassis at the internet edge with multiple Full BGP feeds? 95th percent would be about 300 meg of traffic. BGP Doesn't support URPF which makes it unsuitable for RTBH and therefore
I was just about to pipe up and say "they do it fine!" and then I remembered that we built automatic filtering provisioning so that each edge customer got filters applied automatically based on their static assignments from us, or from IRR tables if a checkbox was marked. The boxes handled 1000x ports with ~6 filters per port no problem, but yeah, real uRPF would be nice. -- Jo Rhett Net Consonance : net philanthropy to improve open source and internet projects.
Brent, While the E300 can probably get your job done for more flexibility and growth I would personally steer you towards the E600 (or E600i now). It is slightly outside of your RU requirement coming in at 16 RU but it fits the bill otherwise. The main reasons I make this suggestion is due to the fact that the E600i chassis gives you numerous options. The "standard" LC memory config is 10M, however you can buy cards with an increased 40M cam as well. Also Force10 has redundant route processors but takes it a little farther. The RPM which is redundant and supports hitless failover has three CPU's. CP - Control Processor RP1 - Handles the majority of the Layer 3 protocols RP2 - Handles the majority of the Layer 2 protocols including sflow. I could have that swapped in my head but its one way or the other. On the linecards you can change your memory allocation provisioning as well if need be, granted its more useful when you have the 40M CAM cards. The E600i can also be configured two ways.. 1 as a TeraScale supporting 4x10G XFP linerate and 16x10G XFP OverSub as well as 1G, or an ExaScale supporting 10x10G linerate and 40x10G OverSub. As well as numerous 1G options as well, take a look at this chart: http://i.dell.com/sites/content/shared-content/data-sheets/en/Documents/Dell... Redundancy/Availability 1+1 redundant RPMs 4:1 redundant SFMs 1+1 redundant DC PEMs 2+2 redundant AC PSMs - 200/240 VAC 3+1 redundant AC PSMs - 100/120 VAC and 200/240 VAC FTOS is quite polished these days as well, and command accounting does work. Its just not captured in the switch log, but does record just fine on the TACACS side: 2012-03-28 23:12:29 -0700 xxx.xxx.xxx.xxx bbianchi vty0 xxx.xxx.xxx.xxx stop task_id=410 timezone=UTC service=shell priv-lvl=15 cmd=show interfaces description <cr> Id be happy to answer any specific questions you may have off list as well. -Brandon I have been supporting a large Force10 install base for a few years now and can attest to On Mar 27, 2012, at 2:21 PM, Roberts, Brent wrote: Is anyone running an E300 Series Chassis at the internet edge with multiple Full BGP feeds? 95th percent would be about 300 meg of traffic. BGP session count would be between 2 and 4 Peers. 6k internal Prefix count as it stands right now. Alternative are welcome. Thought about the ASR1006 but I need some local switching as well. Full requirements include Full internet Peering over GigE Links. Fully Redundant Power Redundant "Supervisor/Route Processor" Would prefer a Small Chassis unit. (under 10u) Would also prefer a single unit as opposed to a two smaller units. ________________________________ This email and any attached files may contain confidential and/or privileged material and is intended solely for the use of the person to whom it is addressed. Any review, retransmission, dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender immediately and delete it and all attachments from your computer. Progressive Solutions is not liable for any errors or omissions in the content or transmission of this email.
participants (10)
-
Brandon Bianchi -
George Bonser -
james jones -
Jo Rhett -
Joel jaeggli -
Owen DeLong -
Randy -
Roberts, Brent -
Tom Daly -
Vinny_Abello＠Dell.com