Any 1U - 2U Ethernet switches that can handle 4K VLANs?
Does anybody know of 1U - 2U form factor Ethernet switches that can handle 4K VLANs, or at a minimum 2000 VLANs? Note that we're specifically looking for the ability to handle this number of VLANs operating simultaneously, not only VLAN *IDs* in the full 4K range. (This rules out popular switches like the Cisco 3550 and 3750 series, which can only handle 1024 VLANs operating simultaneously.) The switches should have 12 - 24 Fast Ethernet ports. Some form of "Q in Q" or stackable VLANs, ie. the ability to handle more than one VLAN tag, is vital. Spanning tree is needed, but can be one common spanning tree for all VLANs (per-VLAN spanning tree is not needed). Other features that would be nice to have: - RSTP (802.w) and MST (802.1s). - A couple of GigE ports (GBIC or SFP based, presumably) for uplinks. - L3 (IP routing). - DC power. Steinar Haug, Nethelp consulting, sthaug@nethelp.no
On Sun, 25 Jan 2004 sthaug@nethelp.no wrote:
Does anybody know of 1U - 2U form factor Ethernet switches that can handle 4K VLANs, or at a minimum 2000 VLANs? Note that we're specifically looking for the ability to handle this number of VLANs operating simultaneously, not only VLAN *IDs* in the full 4K range.
Extreme Summit48si.
The switches should have 12 - 24 Fast Ethernet ports. Some form of "Q in Q" or stackable VLANs, ie. the ability to handle more than one VLAN tag, is vital.
You can do this by changing the ethertype of VLANs, Extreme calls this VMAN (9100 for vlans intead of 8100). This requires a network design to match. The switch has 48 ports and two SFP gig ports.
Spanning tree is needed, but can be one common spanning tree for all VLANs (per-VLAN spanning tree is not needed).
It does that.
Other features that would be nice to have:
- RSTP (802.w) and MST (802.1s). - A couple of GigE ports (GBIC or SFP based, presumably) for uplinks. - L3 (IP routing). - DC power.
I dont know about RSTP and MST, but it does the rest. It also has EAPS for subsecond L2 failover. -- Mikael Abrahamsson email: swmike@swm.pp.se
On Sun, Jan 25, 2004 at 08:13:45PM +0100, sthaug@nethelp.no wrote:
Does anybody know of 1U - 2U form factor Ethernet switches that can handle 4K VLANs, or at a minimum 2000 VLANs? Note that we're specifically looking for the ability to handle this number of VLANs operating simultaneously, not only VLAN *IDs* in the full 4K range.
I would check the Foundry Fastiron series - maybe the 4802. Everything I've read appears to indicate they support all 4096 vlans simultaneously, although you will of course want to verify this. Extreme also appear to support 4096 vlans - you'd be looking at the Summit 200 or Summit 48si for that. Even Cisco's new 3750 Metro only supports 1024 vlans - but both this and it's similarly-named predecessors are aimed as CPE; I suppose this is because they want you to fork out for 6500. W
On Sun, 2004-01-25 at 14:44, Will Hargrave wrote:
I would check the Foundry Fastiron series - maybe the 4802. Everything I've read appears to indicate they support all 4096 vlans simultaneously, although you will of course want to verify this.
I don't think this is true. Those of you with BigIron units know that (at least in m3 supervisors) they support only 512 vlans at most. I do not think the older, and generally less capable, FastIron switches are likely to support more. The command to check this on BigIron is `show default values`. -- Jeff
* jsw@five-elements.com (Jeff S Wheeler) [Sun 25 Jan 2004, 22:10 CET]:
I would check the Foundry Fastiron series - maybe the 4802. Everything I've read appears to indicate they support all 4096 vlans simultaneously, although you will of course want to verify this. I don't think this is true. Those of you with BigIron units know that (at least in m3 supervisors) they support only 512 vlans at most. I do not think the older, and generally less capable, FastIron switches are
On Sun, 2004-01-25 at 14:44, Will Hargrave wrote: likely to support more.
The command to check this on BigIron is `show default values`.
That indicates a maximum of 4095 on a recent switch here that runs the layer-2 only image. Older models appear to have a limit of 2048, but I can't tell for sure whether this is hardware or software related. Configurability is, of course, no guarantee for things to actually work. -- Niels.
try extreme... summit alpine and blackdiamond should all do that although only the summits fit in the form-factor you're thinking of. My experience with extremes as l3 boxes is neither recent nor pleasant, but that's not how we use them anyway. On Sun, 25 Jan 2004 sthaug@nethelp.no wrote:
Does anybody know of 1U - 2U form factor Ethernet switches that can handle 4K VLANs, or at a minimum 2000 VLANs? Note that we're specifically looking for the ability to handle this number of VLANs operating simultaneously, not only VLAN *IDs* in the full 4K range.
(This rules out popular switches like the Cisco 3550 and 3750 series, which can only handle 1024 VLANs operating simultaneously.)
The switches should have 12 - 24 Fast Ethernet ports. Some form of "Q in Q" or stackable VLANs, ie. the ability to handle more than one VLAN tag, is vital.
Spanning tree is needed, but can be one common spanning tree for all VLANs (per-VLAN spanning tree is not needed).
Other features that would be nice to have:
- RSTP (802.w) and MST (802.1s). - A couple of GigE ports (GBIC or SFP based, presumably) for uplinks. - L3 (IP routing). - DC power.
Steinar Haug, Nethelp consulting, sthaug@nethelp.no
-- -------------------------------------------------------------------------- Joel Jaeggli Unix Consulting joelja@darkwing.uoregon.edu GPG Key Fingerprint: 5C6E 0104 BAF0 40B0 5BD3 C38B F000 35AB B67F 56B2
1) Use Cisco 2924 or 3524 2) Redesign your network to fit into 1024 VLANs 3) Do not spend time with junk (non Cisco, for the switches). U1 switch have only 24 - 48 ports, so you never need to handle 2000 VLAN's on it. And I suspect, that the whole design is wrong. Do not build custom configuration (4000 VLANs), build standard configuration (20 - 40 VLANs) /except - if you want to became a QA for the whole vendor/. Did you paid attention to the private VLANs, dynamic VLANs, port authentication protocols, etc etc? In 99% cases, you can stay with 10 private VLANs instead of 4,000 static VLAN's. ----- Original Message ----- From: "Joel Jaeggli" <joelja@darkwing.uoregon.edu> To: <sthaug@nethelp.no> Cc: <nanog@merit.edu> Sent: Sunday, January 25, 2004 11:45 AM Subject: Re: Any 1U - 2U Ethernet switches that can handle 4K VLANs?
try extreme...
summit alpine and blackdiamond should all do that although only the summits fit in the form-factor you're thinking of.
My experience with extremes as l3 boxes is neither recent nor pleasant, but that's not how we use them anyway.
On Sun, 25 Jan 2004 sthaug@nethelp.no wrote:
Does anybody know of 1U - 2U form factor Ethernet switches that can handle 4K VLANs, or at a minimum 2000 VLANs? Note that we're specifically looking for the ability to handle this number of VLANs operating simultaneously, not only VLAN *IDs* in the full 4K range.
(This rules out popular switches like the Cisco 3550 and 3750 series, which can only handle 1024 VLANs operating simultaneously.)
The switches should have 12 - 24 Fast Ethernet ports. Some form of "Q in Q" or stackable VLANs, ie. the ability to handle more than one VLAN tag, is vital.
Spanning tree is needed, but can be one common spanning tree for all VLANs (per-VLAN spanning tree is not needed).
Other features that would be nice to have:
- RSTP (802.w) and MST (802.1s). - A couple of GigE ports (GBIC or SFP based, presumably) for uplinks. - L3 (IP routing). - DC power.
Steinar Haug, Nethelp consulting, sthaug@nethelp.no
-- -------------------------------------------------------------------------- Joel Jaeggli Unix Consulting joelja@darkwing.uoregon.edu GPG Key Fingerprint: 5C6E 0104 BAF0 40B0 5BD3 C38B F000 35AB B67F 56B2
Hmm; if they need to run 2000 VLAN's, they do not need L3 routing in every box... Of course, if they want L3 routing on every box (I do not like such idea, but it's possible), then 3550 (or what do they have now?) is the best choice. But I am very suspicious about such design... in 99% cases, 4,000 VLAn,s and 100 24-port switches means _bad network / solution schema_. If use other (non Cisco) switches - it may be very good choice in getting low price, but it requires long and careful testing. My experience is strictly asgainst non-cisco devices in such areas, as - VoIP, IP routing, L2 / L3 switches/routers (and almost the same in switches). ----- Original Message ----- From: "Mikael Abrahamsson" <swmike@swm.pp.se> To: <nanog@merit.edu> Sent: Sunday, January 25, 2004 12:40 PM Subject: Re: Any 1U - 2U Ethernet switches that can handle 4K VLANs?
On Sun, 25 Jan 2004, Alexei Roudnev wrote:
1) Use Cisco 2924 or 3524
Didnt you mean 2950 and 3550?
-- Mikael Abrahamsson email: swmike@swm.pp.se
On Sun, 25 Jan 2004, Alexei Roudnev wrote:
Of course, if they want L3 routing on every box (I do not like such idea, but it's possible), then 3550 (or what do they have now?) is the best choice.
Definately not. The 3550 is an overpriced outdated product with moderate performance with way too small table sizes. For instance: The Summit48si handles 128k MAC addresses. The 3550 handles something like 6-15k. The Summit48si can do buffering when doing QoS/shaping, the 3550 does only policing. If you want to deliver a 2meg service over ethernet to a customer, this is a big issue. There is only one product in the 3550 line that is pricewise worth getting is the 3550-12G if you need to do L2 gig aggregation to 1gig uplink and you do not have many VLANs. There are three issues I see where the 3550 actually has a selling point: VRFs (even though they are too few) Q-in-Q (limited by the small mac table size) CEF (if you have very small routing table size and no broadcasts) -- Mikael Abrahamsson email: swmike@swm.pp.se
3550 runs IOS. That's an answer. I never allow any non-IOS router in production environment (except high end devices, such as Juniper, when benefits are very high). And 3550 is not expansive (yes, it is not cheap). PS. How much ethernet ports do you have in the office? Do you have 100 K ports? If not, why do you need 128K MAC's? (I know only one case, when I need so much - some kind of DSL service... In most cases, you have 500 - 5,000 ports in one building. If you have more, it is unlikely that you use 3550 switches. So, it is enough for the tasks (just as performance - it have _enough_ performance). Btw, I believed that catalist swithes have not any limitations for the MAC tables (because they use memory _on demand_); where did you get this limitations? /I may be wrong here/ PPS. I do not know for sure, but 3550 should support traffic shaping, which makes bufferring. Technically, yes, CEF (with packet dropping) is not good to provide 2 Mbit by 100 Mbit link.
On Sun, 25 Jan 2004, Alexei Roudnev wrote:
Of course, if they want L3 routing on every box (I do not like such
idea,
but it's possible), then 3550 (or what do they have now?) is the best choice.
Definately not. The 3550 is an overpriced outdated product with moderate performance with way too small table sizes. For instance:
The Summit48si handles 128k MAC addresses. The 3550 handles something like 6-15k.
The Summit48si can do buffering when doing QoS/shaping, the 3550 does only policing. If you want to deliver a 2meg service over ethernet to a customer, this is a big issue.
There is only one product in the 3550 line that is pricewise worth getting is the 3550-12G if you need to do L2 gig aggregation to 1gig uplink and you do not have many VLANs.
There are three issues I see where the 3550 actually has a selling point:
VRFs (even though they are too few) Q-in-Q (limited by the small mac table size) CEF (if you have very small routing table size and no broadcasts)
-- Mikael Abrahamsson email: swmike@swm.pp.se
On Mon, 26 Jan 2004, Alexei Roudnev wrote:
PS. How much ethernet ports do you have in the office? Do you have 100 K ports? If not, why do you need 128K MAC's? (I know only one case, when I need so much - some kind of DSL service...
I guess you're not into metro networking.
(just as performance - it have _enough_ performance). Btw, I believed that catalist swithes have not any limitations for the MAC tables (because they use memory _on demand_); where did you get this limitations? /I may be wrong here/
<http://www.cisco.com/en/US/customer/products/hw/switches/ps646/products_tech_note09186a0080094bc6.shtml> You have something like 16-24.000 entries which are shared between routes, QoS, mac adress table size etc. Default is 5k mac adress size on the 3550-24/48. For metro applications, this is not enough.
PPS. I do not know for sure, but 3550 should support traffic shaping, which makes bufferring. Technically, yes, CEF (with packet dropping) is not good to provide 2 Mbit by 100 Mbit link.
The 3550 doesnt support shaping of any kind, only policing (dropping packets, never buffer them). How can you advocate a switch which you seem to know so little about? -- Mikael Abrahamsson email: swmike@swm.pp.se
PS. How much ethernet ports do you have in the office? Do you have 100 K ports? If not, why do you need 128K MAC's? (I know only one case, when I need so much - some kind of DSL service...
I guess you're not into metro networking.
This is one of my exceptions - you really need 128K MAC's for meto network. And, for metro network, it may be reasonable to spend time in QA'ing and configuration and select non-cisco solution - because it is a very big project. But it is exceptional case.
PPS. I do not know for sure, but 3550 should support traffic shaping, which makes bufferring. Technically, yes, CEF (with packet dropping) is not good to provide 2 Mbit by 100 Mbit link.
The 3550 doesnt support shaping of any kind, only policing (dropping packets, never buffer them). How can you advocate a switch which you seem to know so little about? I just never tried to configure 'traffic-shape' on it, so I do not know. It is great switch for it's niche. Metro LAN's is not standard switch niche, it is very special network. As I said above, non-cisco solution can paid off for this.
-- Mikael Abrahamsson email: swmike@swm.pp.se
Alexei Roudnev wrote:
1) Use Cisco 2924 or 3524 2) Redesign your network to fit into 1024 VLANs 3) Do not spend time with junk (non Cisco, for the switches).
U1 switch have only 24 - 48 ports, so you never need to handle 2000 VLAN's on it. And I suspect, that the whole design is wrong. Do not build custom configuration (4000 VLANs), build standard configuration (20 - 40 VLANs) /except - if you want to became a QA for the whole vendor/.
I agree, but you could still have 4000 VLANs with multiple VTP domains. Using 3550-48s you can have L3 links between VTP domains. Jeff
* jeff-kell@utc.edu (Jeff Kell) [Mon 26 Jan 2004, 00:35 CET]:
Using 3550-48s you can have L3 links between VTP domains.
The point of using VLANs is that you don't need to route. There's probably a good reason for switching instead of routing in the original poster's scenario. (Perhaps a FTTH-like project?) -- Niels.
On Mon, 26 Jan 2004, Niels Bakker wrote:
* jeff-kell@utc.edu (Jeff Kell) [Mon 26 Jan 2004, 00:35 CET]:
Using 3550-48s you can have L3 links between VTP domains.
The point of using VLANs is that you don't need to route. There's probably a good reason for switching instead of routing in the original poster's scenario. (Perhaps a FTTH-like project?)
Correct me if I'm wrong here, but at some point you will have to route all those VLAN's. To really answer the question about wether > 1000 VLAN's are necessary one would need to see the network design.
From my point of view I'd have to question the need to carry that many VLAN's over a large portion of the network. I would think that the network should be more partitioned so most of the VLAN's don't need to be seen outside a small area.
bye, ken emery
On Sun, 25 Jan 2004, Joel Jaeggli wrote:
My experience with extremes as l3 boxes is neither recent nor pleasant, but that's not how we use them anyway.
This is interesting, what problems did you run into? We have an extensive Extreme networks used both for L2 and L3, and apart from the fact that it always cpu routes ICMP, I see no major flaw in the L3 forwarding function (for access/distribution) for all normal purposes. My few experiences with the Cisco 3550 as L3 routers has been much worse, even with claimed CEF capability I have seen it melt and die where the equivalent Extreme box didnt experience the same problems (of course there are cases where it's the other way around). Overall I have more confidence in the Extreme access boxes for L3 than Ciscos equivlanent, and they definately kick ciscos ass when it comes to L2 (mac address table size and number of vlans for instance). -- Mikael Abrahamsson email: swmike@swm.pp.se
On Sun, Jan 25, 2004 at 09:39:05PM +0100, Mikael Abrahamsson wrote:
This is interesting, what problems did you run into?
We have an extensive Extreme networks used both for L2 and L3, and apart from the fact that it always cpu routes ICMP, I see no major flaw in the L3 forwarding function (for access/distribution) for all normal purposes.
ACLs are per-port and known to be buggy when operating on port numbers - in particular UDP ACLs match will match arbritary data when presented with a subsequent IP fragments (think NFS...) As pointed out in a similar thread recently, the 'flow-based' (well, destination IP based) ipfdb will crap out on the Extremes under heavy load - e.g. virus'd machines internal to your network doing heavy scanning. Symptom is very poor performance and the 'top' command will show heavy CPU usage as subsequent flows are CPU routed.
My few experiences with the Cisco 3550 as L3 routers has been much worse, even with claimed CEF capability I have seen it melt and die where the equivalent Extreme box didnt experience the same problems (of course there are cases where it's the other way around). Overall I have more confidence in the Extreme access boxes for L3 than Ciscos equivlanent, and they definately kick ciscos ass when it comes to L2 (mac address table size and number of vlans for instance).
The 'recommended max' number of SVIs for the 3550 is something low like 8. There is no limited stated in the datasheet for the 3750 - is anyone running more than 8 SVIs on a 3750? The ACL capability on the 3550 seems a lot more capable but the lack of unicast RPF is irritating. (More irritating, 'ip verify unicast reachable-via...' is accepted but silently does nothing) I'd be very interested to hear what conditions you've found cause problems for Cat3550s. We're planning to buy quite a few more of this range (probably 3750-24) to reduce L2 size in our network and for CPE-type uses.
Will Hargrave wrote:
The 'recommended max' number of SVIs for the 3550 is something low like 8. There is no limited stated in the datasheet for the 3750 - is anyone running more than 8 SVIs on a 3750?
We're running 30 SVIs on a 3550-12 (only 10 active at the moment, we're in a transition). It is an aggregation switch that feeds back via L3.
The ACL capability on the 3550 seems a lot more capable but the lack of unicast RPF is irritating. (More irritating, 'ip verify unicast reachable-via...' is accepted but silently does nothing)
Agreed - we had PSIRT look into it and the "solution" is probably going to be removing ip verify from the CLI parser :-( We had another 3550 replace a struggling 2621 and it blew it away.
I'd be very interested to hear what conditions you've found cause problems for Cat3550s. We're planning to buy quite a few more of this range (probably 3750-24) to reduce L2 size in our network and for CPE-type uses.
In a new building deployment we used 4500 Sup-IVs as MDF/IDF anchors and populated the distributions with 3550-48s. Most of the 4500s had one 48 port copper 10/100/1000 blade to supply gig-to-desktop where needed (their ASICs are overloaded 8-to-1 so be careful about placement). The 4500 not only doesn't do uRPF, it doesn't do flow either. The ACLs/MLS features are nice, supporting input ALCs, 'established' keyword, and logging (unlike, say MLS to a 5500 NFFC). It will not process switch these packets but rather "forks" a copy to the CPU to log if necessary. It is very annoying that neither 3550 nor 4500 support uRPF. Does anyone know if the 3750 does? Jeff
On Sun, 25 Jan 2004, Jeff Kell wrote:
We're running 30 SVIs on a 3550-12 (only 10 active at the moment, we're in a transition). It is an aggregation switch that feeds back via L3.
According to the documentation on the Cisco site: http://www.cisco.com/warp/public/473/145.html The 3550-12 is only capable of handling 16 SVIs in hardware (regardless of SDM template), after that you get into "resource exhaustion" which means it add the SVIs, but will go back to software/CPU-based routing. Does the 3550/3750 give any indication that it's in this state (software routing) other than melting under high traffic volumes? We're currently waiting on Cisco getting back to us on figures for the 3750, but given that it has a similar TCAM setup to the 3550-12, I'd doubt it would be different. Rich
On Mon, 26 Jan 2004, Will Hargrave wrote:
I'd be very interested to hear what conditions you've found cause problems for Cat3550s. We're planning to buy quite a few more of this range (probably 3750-24) to reduce L2 size in our network and for CPE-type uses.
Well, we're not really sure. We put it in front of a 7200 doing approx 50 megabits of data with 50% cpu load, to divert the internet traffic and make the 7200 handle only MPLS PE functionality. The 3550 had only 3 SVIs. It might be broadcast related, we had a lot of L2 broadcasts on that segment. -- Mikael Abrahamsson email: swmike@swm.pp.se
participants (11)
-
Alexei Roudnev -
Jeff Kell -
Jeff S Wheeler -
Joel Jaeggli -
ken emery -
Mikael Abrahamsson -
Niels Bakker -
Randy Bush -
sthaug＠nethelp.no -
variable＠ednet.co.uk -
Will Hargrave