Auto MDI/MDI-X + conference rooms + bored == loop
Anyone have suggestions on Ethernet LAN loop-prevention? With the advent of Auto MDI/MDI-X ports on switches, it seems way too easy to accidentally or maliciously create loops between network jacks. We have bored or inattentive people plugging in patch cords between adjacent network jacks. STP for loop-prevention isn't working so well for us. STP "edge" or "portfast" or "faststart" modes are required for end-station ports (with normal STP, DHCP often times out after 30+ seconds it takes to go into Forwarding state). Since the "edge" STP mode goes into Forwarding state immediately, there is a period when loops will form, causing havok with upstream gear until STP blocks the port (if it ever does see below). "Desktop" switches. You know, those 4 or 5 port Gigabit Ethernet switches. Apparently, many of them don't do any kind of STP at all. Recommendations on ones that do STP? RSTP: is it any better than traditional STP in regards to "edge" ports and blocking before a loop gets out of hand? Or perhaps blocking for 5-10 seconds before going into Forwarding state, hopefully preventing loops before they happen but also allowing DHCP clients to get an address without timeouts? Recommendations on "Desktop" switches that do RSTP? Thanks for your suggestions/discussion. -- - Chuck (354 Days until IPv4 depletion: http://ipv4depletion.com/)
Disable the jacks all together and go wireless? Have them put in a trouble ticket if they absolutely need a port activated in a conference room for a one-time meeting. -Mike On Fri, Mar 26, 2010 at 3:09 PM, Chuck Anderson <cra@wpi.edu> wrote:
Anyone have suggestions on Ethernet LAN loop-prevention? With the advent of Auto MDI/MDI-X ports on switches, it seems way too easy to accidentally or maliciously create loops between network jacks. We have bored or inattentive people plugging in patch cords between adjacent network jacks. STP for loop-prevention isn't working so well for us.
STP "edge" or "portfast" or "faststart" modes are required for end-station ports (with normal STP, DHCP often times out after 30+ seconds it takes to go into Forwarding state). Since the "edge" STP mode goes into Forwarding state immediately, there is a period when loops will form, causing havok with upstream gear until STP blocks the port (if it ever does see below).
"Desktop" switches. You know, those 4 or 5 port Gigabit Ethernet switches. Apparently, many of them don't do any kind of STP at all. Recommendations on ones that do STP?
RSTP: is it any better than traditional STP in regards to "edge" ports and blocking before a loop gets out of hand? Or perhaps blocking for 5-10 seconds before going into Forwarding state, hopefully preventing loops before they happen but also allowing DHCP clients to get an address without timeouts? Recommendations on "Desktop" switches that do RSTP?
Thanks for your suggestions/discussion.
-- - Chuck (354 Days until IPv4 depletion: http://ipv4depletion.com/)
Bpduguard if running cisco. set all the switch ports to bpduguard or enable it globally -----Original Message----- From: Chuck Anderson [mailto:cra@WPI.EDU] Sent: Friday, March 26, 2010 6:09 PM To: nanog@nanog.org Subject: Auto MDI/MDI-X + conference rooms + bored == loop Anyone have suggestions on Ethernet LAN loop-prevention? With the advent of Auto MDI/MDI-X ports on switches, it seems way too easy to accidentally or maliciously create loops between network jacks. We have bored or inattentive people plugging in patch cords between adjacent network jacks. STP for loop-prevention isn't working so well for us. STP "edge" or "portfast" or "faststart" modes are required for end-station ports (with normal STP, DHCP often times out after 30+ seconds it takes to go into Forwarding state). Since the "edge" STP mode goes into Forwarding state immediately, there is a period when loops will form, causing havok with upstream gear until STP blocks the port (if it ever does see below). "Desktop" switches. You know, those 4 or 5 port Gigabit Ethernet switches. Apparently, many of them don't do any kind of STP at all. Recommendations on ones that do STP? RSTP: is it any better than traditional STP in regards to "edge" ports and blocking before a loop gets out of hand? Or perhaps blocking for 5-10 seconds before going into Forwarding state, hopefully preventing loops before they happen but also allowing DHCP clients to get an address without timeouts? Recommendations on "Desktop" switches that do RSTP? Thanks for your suggestions/discussion. -- - Chuck (354 Days until IPv4 depletion: http://ipv4depletion.com/)
On Fri, Mar 26, 2010 at 5:21 PM, Matthew Huff <mhuff@ox.com> wrote:
Bpduguard if running cisco. set all the switch ports to bpduguard or enable it globally
Bpduguarding a cool idea, and not a bad protective measure, if running that vendor's equipment, but it still allows a possibly large disruption for the (small) duration until the first BPDU is received and relies on reasonable operation from the thing creating a loop -- the guard is a no-op if whatever crosses jacks does not pass STP PDUs. Whether it is enough, depends on your threshold of pain for looping, packet loss, and the frequency of conference room physical configuration errors. Most all switch manufacturers provide some type of port security feature that allows an end-user connection port to automatically be disabled and require admin intervention to re-activate, if the number of MAC addresses exceed a configurable number, e.g. allow 5 MAC addresses, which are remembered as that port's list of "secured" MAC addresses with an aging time of 5 minutes. Use that function, and use the functions of a switch that provide storm control for client ports. With a reasonable aging duration for expiring secured MAC addresses. If a client port emits packets with more than the expected number of MAC address sources within a short time, then that should be an early warning that traffic has taken an improper path. Keeping in mind a loop doesn't necessarily create an instant issue, until there is other broadcast traffic on the network, that crosses the loop, and starts messing up the CAM table on the upstream device, as well as possibly generating duplicate traffic. But with port security, the number of devices that lose packets due to the loop should be limited (the smaller you set the limit without impacting legitimate use of the port, the better). -- -J
On Fri, Mar 26, 2010 at 08:29:52PM -0500, James Hess wrote:
Most all switch manufacturers provide some type of port security feature that allows an end-user connection port to automatically be disabled and require admin intervention to re-activate, if the number of MAC addresses exceed a configurable number, e.g. allow 5 MAC addresses, which are remembered as that port's list of "secured" MAC addresses with an aging time of 5 minutes.
In fact, the last time this happened, I implemented exactly what you describe, mac-security with a limit of 5 MACs. The security kicked in and shutdown the port, but not before the core shutdown the edge switch's uplinks (see below).
Use that function, and use the functions of a switch that provide storm control for client ports. With a reasonable aging duration for expiring secured MAC addresses.
Have that.
If a client port emits packets with more than the expected number of MAC address sources within a short time, then that should be an early warning that traffic has taken an improper path.
Have that.
Keeping in mind a loop doesn't necessarily create an instant issue, until there is other broadcast traffic on the network, that crosses the loop, and starts messing up the CAM table on the upstream device, as well as possibly generating duplicate traffic.
Which pretty much means within milliseconds on my network.
But with port security, the number of devices that lose packets due to the loop should be limited (the smaller you set the limit without impacting legitimate use of the port, the better).
So basically, the problem is the core switches implement a proprietary loop-prevention protocol that sends "beacon" frames out every 500ms, and if a certain number of these special frames come back (exceeds threshold) it shuts down the port. Even with a 10:1 ratio of threshold settings on the two redundant links to the edge switch, it still trips both thresholds fast enough that both redundant links get shutdown in the short time before the edge switch's protection mechanism (mac-security, STP, bpduprotect, whatever) kicks in. I've now set the ratio to 100:1 (500:5 in actual packet counts) and the transmit interval to 1000ms in the hopes that at least one core link will survive long enough for the edge port to shutdown and break the loop first, but I'm beginning to think that this protocol is crap and I should just disable it and let the core ride out the loop in the hopes that it survives without taking down the entire core before the edge switch shutdown happens. The good news is that this core is being replaced soon, hopefully with gear that will be able to implement a service-provider-like design with per-port VLAN separation as was suggested in this thread. But it surprises me that low-end switch vendors (like NetGear) still put out crap that doesn't do STP, especially when the switch does Auto MDI/MDI-X, which is just asking for trouble. Anyone know if Auto MDI/MDI-X is inherent or required in 1000Base-T? It would be nice if I could shut it off. Thanks for all the ideas.
On Fri, Mar 26, 2010 at 9:29 PM, Chuck Anderson <cra@wpi.edu> wrote:
So basically, the problem is the core switches implement a proprietary loop-prevention protocol that sends "beacon" frames out every 500ms, and if a certain number of these special frames come back (exceeds --> loop first, but I'm beginning to think that this protocol is crap and I should just disable it and let the core ride out the loop in the
Ah, nasty.. it seems like you definitely should want to keep the beacon frames from getting injected then. Taking down core links ought to be harder than 1 user emitting a few frames. A malicious user, or a naive user with a malicious trojan on their computer could try to send fake beacons, to cause trouble. I for one might start thinking if the beacons can be sunk from end user ports by brute force, using a Layer 2 ACL. I wonder if RFC 5556, IETF TRILL specs, or 802.1aq/802.1Qbb / Datacenter Ethernet / Bridging standards and more robust standards-based loop avoidance standards will ever get finalized, considering they have been drafts for over 5 years, it seems like the standardization is very sluggish. A new protocol is probably the right solution, but it might not be ready until 2015 at this rate.
Anyone know if Auto MDI/MDI-X is inherent or required in 1000Base-T? It would be nice if I could shut it off.
Auto MDI/MDI-X is an optional feature in the 1000BaseT standard. Automatic negotiation of speeds and duplex, is mandatory due to 802.3ab, but not auto-crossover You can get that here http://standards.ieee.org/getieee802/802.3.html Clause 40.4.4 in IEEE 802.3-2008 -- Section Three states the following: "40.4.4 Automatic MDI/MDI-X Configuration Automatic MDI/MDI-X Configuration is intended to eliminate the need for crossover cables between simi lar devices. Implementation of an automatic MDI/MDI-X configuration is optional for 1000BASE-T devices. If an automatic configuration method is used, it shall comply with the following specifications. The assignment of pin-outs for a 1000BASE-T crossover function cable is shown in Table40–12 in 40.8. " -- -J
On Mar 26, 2010, at 7:29 PM, Chuck Anderson wrote:
On Fri, Mar 26, 2010 at 08:29:52PM -0500, James Hess wrote:
Most all switch manufacturers provide some type of port security feature that allows an end-user connection port to automatically be disabled and require admin intervention to re-activate, if the number of MAC addresses exceed a configurable number, e.g. allow 5 MAC addresses, which are remembered as that port's list of "secured" MAC addresses with an aging time of 5 minutes.
In fact, the last time this happened, I implemented exactly what you describe, mac-security with a limit of 5 MACs. The security kicked in and shutdown the port, but not before the core shutdown the edge switch's uplinks (see below).
Use that function, and use the functions of a switch that provide storm control for client ports. With a reasonable aging duration for expiring secured MAC addresses.
Have that.
If a client port emits packets with more than the expected number of MAC address sources within a short time, then that should be an early warning that traffic has taken an improper path.
Have that.
Keeping in mind a loop doesn't necessarily create an instant issue, until there is other broadcast traffic on the network, that crosses the loop, and starts messing up the CAM table on the upstream device, as well as possibly generating duplicate traffic.
Which pretty much means within milliseconds on my network.
Sounds like you forgot to configure the "Root is that-way ->" sanity check on your switches. Make sure that Root bridge can't be determined to be in a direction other than "upstream" will help a lot with this.
But with port security, the number of devices that lose packets due to the loop should be limited (the smaller you set the limit without impacting legitimate use of the port, the better).
So basically, the problem is the core switches implement a proprietary loop-prevention protocol that sends "beacon" frames out every 500ms, and if a certain number of these special frames come back (exceeds threshold) it shuts down the port. Even with a 10:1 ratio of
That's Icky... Can you replace that with traditional spanning tree? It's just too sensitive for a deployment of any real size.
threshold settings on the two redundant links to the edge switch, it still trips both thresholds fast enough that both redundant links get shutdown in the short time before the edge switch's protection mechanism (mac-security, STP, bpduprotect, whatever) kicks in. I've now set the ratio to 100:1 (500:5 in actual packet counts) and the transmit interval to 1000ms in the hopes that at least one core link will survive long enough for the edge port to shutdown and break the loop first, but I'm beginning to think that this protocol is crap and I should just disable it and let the core ride out the loop in the hopes that it survives without taking down the entire core before the edge switch shutdown happens.
Yes, I think that would be much better. (Of course, another alternative would be traditional spanning tree, or, some variant of STP with faster timers).
The good news is that this core is being replaced soon, hopefully with gear that will be able to implement a service-provider-like design with per-port VLAN separation as was suggested in this thread. But it surprises me that low-end switch vendors (like NetGear) still put out crap that doesn't do STP, especially when the switch does Auto MDI/MDI-X, which is just asking for trouble.
Usually people don't use Netgear cheap switches in environments with more than a desktop worth of topology.
Anyone know if Auto MDI/MDI-X is inherent or required in 1000Base-T? It would be nice if I could shut it off.
Yes, it is. (This is actually a good thing in everyone else's environment). Owen
On Sat, Mar 27, 2010 at 02:11:32AM -0700, Owen DeLong wrote:
Sounds like you forgot to configure the "Root is that-way ->" sanity check on your switches. Make sure that Root bridge can't be determined to be in a direction other than "upstream" will help a lot with this.
No STP in the core, only on the managed edges.
So basically, the problem is the core switches implement a proprietary loop-prevention protocol that sends "beacon" frames out every 500ms, and if a certain number of these special frames come back (exceeds threshold) it shuts down the port. Even with a 10:1 ratio of
That's Icky... Can you replace that with traditional spanning tree? It's just too sensitive for a deployment of any real size.
STP is eliminated by vendor's design recommendations. Active/active split LAG across two core boxes. But yes, I agree that this design is proving--lacking.
The good news is that this core is being replaced soon, hopefully with gear that will be able to implement a service-provider-like design with per-port VLAN separation as was suggested in this thread. But it surprises me that low-end switch vendors (like NetGear) still put out crap that doesn't do STP, especially when the switch does Auto MDI/MDI-X, which is just asking for trouble.
Usually people don't use Netgear cheap switches in environments with more than a desktop worth of topology.
We don't generally put them in, users do. There are a few cases where we have a dearth of cable or conduit space and needed something small and quiet to put there. Hence my question about better switches to use in those scenarios.
Anyone know if Auto MDI/MDI-X is inherent or required in 1000Base-T? It would be nice if I could shut it off.
Yes, it is. (This is actually a good thing in everyone else's environment).
It's easy to claim that no one else but me has this problem. Designing a "dekstop" switch that makes it easy to create accidental loops, but then has no loop-prevention mechanism seems irresponsible to me...
Along with bpduguard, Cisco switches also continue to look for loops with loopguard. They continuously look for the Keepalive packets that they send out each port. So as long as you have not turned off STP all together on the port, you will be fine. On 3/26/10 6:21 PM, Matthew Huff wrote:
Bpduguard if running cisco.
set all the switch ports to bpduguard or enable it globally
-----Original Message----- From: Chuck Anderson [mailto:cra@WPI.EDU] Sent: Friday, March 26, 2010 6:09 PM To: nanog@nanog.org Subject: Auto MDI/MDI-X + conference rooms + bored == loop
Anyone have suggestions on Ethernet LAN loop-prevention? With the advent of Auto MDI/MDI-X ports on switches, it seems way too easy to accidentally or maliciously create loops between network jacks. We have bored or inattentive people plugging in patch cords between adjacent network jacks. STP for loop-prevention isn't working so well for us.
STP "edge" or "portfast" or "faststart" modes are required for end-station ports (with normal STP, DHCP often times out after 30+ seconds it takes to go into Forwarding state). Since the "edge" STP mode goes into Forwarding state immediately, there is a period when loops will form, causing havok with upstream gear until STP blocks the port (if it ever does see below).
"Desktop" switches. You know, those 4 or 5 port Gigabit Ethernet switches. Apparently, many of them don't do any kind of STP at all. Recommendations on ones that do STP?
RSTP: is it any better than traditional STP in regards to "edge" ports and blocking before a loop gets out of hand? Or perhaps blocking for 5-10 seconds before going into Forwarding state, hopefully preventing loops before they happen but also allowing DHCP clients to get an address without timeouts? Recommendations on "Desktop" switches that do RSTP?
Thanks for your suggestions/discussion.
-- Steve KingSenior Senior Linux Engineer - Advance Internet, Inc. Cisco Certified Network Associate CompTIA Linux+ Certified Professional CompTIA A+ Certified Professional
Switches that support STP? There are switches that have STP protection such that they are portfast until they see an inbound BPDU and then revert to spanning tree on that port (it blocks, listens, learns, then forwards if appropriate). The only draw-back to such a configuration I am aware of is that you have the (very small) overhead of all such ports sending BPDUs. Owen On Mar 26, 2010, at 3:09 PM, Chuck Anderson wrote:
Anyone have suggestions on Ethernet LAN loop-prevention? With the advent of Auto MDI/MDI-X ports on switches, it seems way too easy to accidentally or maliciously create loops between network jacks. We have bored or inattentive people plugging in patch cords between adjacent network jacks. STP for loop-prevention isn't working so well for us.
STP "edge" or "portfast" or "faststart" modes are required for end-station ports (with normal STP, DHCP often times out after 30+ seconds it takes to go into Forwarding state). Since the "edge" STP mode goes into Forwarding state immediately, there is a period when loops will form, causing havok with upstream gear until STP blocks the port (if it ever does see below).
"Desktop" switches. You know, those 4 or 5 port Gigabit Ethernet switches. Apparently, many of them don't do any kind of STP at all. Recommendations on ones that do STP?
RSTP: is it any better than traditional STP in regards to "edge" ports and blocking before a loop gets out of hand? Or perhaps blocking for 5-10 seconds before going into Forwarding state, hopefully preventing loops before they happen but also allowing DHCP clients to get an address without timeouts? Recommendations on "Desktop" switches that do RSTP?
Thanks for your suggestions/discussion.
-- - Chuck (354 Days until IPv4 depletion: http://ipv4depletion.com/)
On Fri, Mar 26, 2010 at 03:33:56PM -0700, Owen DeLong wrote:
Switches that support STP?
Yes, "soho" or "desktop" switches I mean. Apparently Netgear GS105's do not do STP at all, at least they don't claim to.
There are switches that have STP protection such that they are portfast until they see an inbound BPDU and then revert to spanning tree on that port (it blocks, listens, learns, then forwards if appropriate).
Do the ports so configured also send BPDUs such that a loop on a "desktop" switch uplinked to that port will cause the port to see its own BPDUs and revert to STP Blocking? Even if that is the case, I think the detection of BPDU and subsequent transition to Blocking will happen too slowly. By then the damage is already done upstream in the collapsed L2/L3 core.
Hi Chuck,
Anyone have suggestions on Ethernet LAN loop-prevention? With the
In general, I avoid the potential for layer2 loops to any user-accesible layer2 ports in a manner that many edge network and broadband providers may find familiar -- vlan per user, tail, port, etc. -- aggregated in a hierarchical manner within the building, metro area, or city. This simple and logical layer2 isolation (real isolation, none of this pvlan-edge stuff) basically solves many problems by simply avoiding the preconditions necessary for loops/etc to pose a problem to the agg/border/etc of a network. Don't worry about users' being entire walled-off from each other -- unicast IP reachability is not broken with this model. Currently, the IOS implementation of unnumbered vlans/subints provides proxy-arp responses for all in-prefix (as defined by loopback/interface pointer-membership) addresses that your end-users may issue who-has's for. This is analogous to docsis and rfc1483 half-bridge as often used on xDSL network edges -- layer3's not broken, but layer2 is nicely walled off per-user. Perhaps you could think of this as dedicated layer2 resources per customer edge (CE), rather, "complaining entity." More here: http://www.cisco.com/en/US/docs/ios/12_3t/12_3t4/feature/guide/gtunvlan.html I use this feature on 3550/3560/3750, sup2/sup720 on 6500; several colleagues have verified this works on 4900m an 4500's in 12.2SG trains. Of course, terminating lower-speed subints or QinQ tag'd vlan bundles works on higher-end ports (sip/spa), as well as all cpu-based boxes... 7200/7301 will consume QinQ ip subints for breakfast, and even give populate option 82 info in DHCP transactions auto-magically, if given the chance (by using helder-addrs on subints). The user-facing port config is usually some per-site variation of this following flavor, configured expecting users to land at 10/fdx or hdx: interface FastEthernet0/1 description Unit 201 load-interval 30 speed 10 port security max-mac-count 10 port security aging time 5 port storm-control broadcast action shutdown port storm-control broadcast threshold rising 100 falling 50 port storm-control multicast action shutdown port storm-control multicast threshold rising 100 falling 50 port storm-control unicast action filter port storm-control unicast threshold rising 3000 falling 1000 switchport access vlan 201 spanning-tree portfast spanning-tree bpdufilter enable ...Errdisable autorecover (or having the user call the support desk) are some of the ways out of the down/down access port penalty box; mix and combine these elements to taste. If terminating fast or gig-e, scale things accordingly. After the access ports are setup and trunking per-port layer2 frames up to the l3 edge (could be 3550, 650x, mwr-1941, etc), we have pages of things like: [...] interface FastEthernet0/1.112 encapsulation dot1Q 112 ip unnumbered Loopback0 ! interface FastEthernet0/1.113 encapsulation dot1Q 113 ip unnumbered Loopback0 ! interface FastEthernet0/1.114 encapsulation dot1Q 114 ip unnumbered Loopback0 [...] In my mdu and mtu layer2 edge sites, this approach has saved our posteriors from real problems--year in, year out. A few words on this config: in what you see above, a user simply cannot introduce enough traffic to the network (unicast) to matter (i.e. perhaps they create an unknown unicast dest flood..), and will be shut down if they spew enough bcast/mcast frames (thresholds set appropriate given your expected end-user profiles). Further, only the first 10 mac addresses can ride this bus (sorry, no LAN parties without prior approval), mitigating concerns for CAM or vlan table exhaustion. Lastly, no funky l3/4 acl's are required to prevent users handing out DHCP addresses, leaking RA's, or fronting ARP as your routers MAC address to their vlan-sharin' neighbors--simply because they don't get to send layer2 frames to anyone but the upstream routers control plane. -Tk
On Fri, Mar 26, 2010 at 06:56:15PM -0400, Anton Kapela wrote:
In general, I avoid the potential for layer2 loops to any user-accesible layer2 ports in a manner that many edge network and broadband providers may find familiar -- vlan per user, tail, port, etc. -- aggregated in a hierarchical manner within the building, metro area, or city.
If you have 2 network jacks next to each other in a conference room, do they each get configured as a separate "user"? What happens if a user connects them together? What happens if a user plugs a desktop switch into one of them, then connects two ports on *that* switch together?
avoiding the preconditions necessary for loops/etc to pose a problem to the agg/border/etc of a network. Don't worry about users' being
Would this work in a collapsed L2/L3 core (no agg, no L3 at edge)?
After the access ports are setup and trunking per-port layer2 frames up to the l3 edge (could be 3550, 650x, mwr-1941, etc), we have pages of things like:
When doing 1:1 VLAN:Port mapping, can you do more than 4096 VLANs/ports? Or are you doing QinQ?
A few words on this config: in what you see above, a user simply cannot introduce enough traffic to the network (unicast) to matter (i.e. perhaps they create an unknown unicast dest flood..), and will be shut down if they spew enough bcast/mcast frames (thresholds set appropriate given your expected end-user profiles). Further, only the first 10 mac addresses can ride this bus (sorry, no LAN parties without prior approval), mitigating concerns for CAM or vlan table exhaustion. Lastly, no funky l3/4 acl's are required to prevent users handing out DHCP addresses, leaking RA's, or fronting ARP as your routers MAC address to their vlan-sharin' neighbors--simply because they don't get to send layer2 frames to anyone but the upstream routers control plane.
Cool, but I'm not sure this will work in my non-Cisco campus environment with 10,000 edge ports. Thanks.
On Mar 26, 2010, at 7:48 PM, Chuck Anderson wrote:
If you have 2 network jacks next to each other in a conference room, do they each get configured as a separate "user"?
Indeed, most of the buildings have a 'community room' like that -- but all the deployed ports (unless ordered differently) will get incrementing-vlan assignments, so indeed, they'd be different vlans back to l3 core.
What happens if a user connects them together?
Nothing, basically, as the network from edge port towards IP edge is (or should be) loop-free. The router will hear DHCP req's on 2x ints, but the client will (should) pick the first-heard response. Depending on the DHCP client implementation, it may wedge/break, but I haven't encountered one in testing. For higher-availability from edge towards IP core, LACP/PAGP provides link-independence, and UDLD/802 OAM provide something of a decent safety-net for breakage detection in metro-spans over other providers/resellers.
What happens if a user plugs a desktop switch into one of them, then connects two ports on *that* switch together?
In my example config, bcast or mcast over 100 pps shuts the port that's receiving the bcast or mcast's down -- but, that's a configurable action. It could discard them, police them, or just report a syslog/trap to the NMS... Of course, this is all switch-vendor specific, etc.
Would this work in a collapsed L2/L3 core (no agg, no L3 at edge)?
Oh, indeed -- and is. The UTOPIA network (http://www.utopianet.org/) in SLC, Utah, is doing basically this for it's ISP-reseller tiers. ISP's get customers on vlans or Q-stacked vlans, and do what they will with it. The ISP's I've talked with have tended to use Juni ERX for this, but there's nothing stopping one from using IOS, or another vendor that can do this trick. It just implies something to consider in the layer2 transport network (support for man l2 addrs in cam, QinQ, etc) at design-time.
When doing 1:1 VLAN:Port mapping, can you do more than 4096 VLANs/ports? Or are you doing QinQ?
Indeed -- q-stacking enables this. In most cases, I don't backhaul more than a few hundred vlans per building -- if it's over 200 to 250 ports/jacks, I generally drop local 3550/3560/3750 or cpu-based boxes on-site, routing towards the metro edge/backbone.
Cool, but I'm not sure this will work in my non-Cisco campus environment with 10,000 edge ports.
Ahh; a pickle. C and J do indeed enable this in many of the popular boxes, which is great. That's not to say other vendors don't have something like it--the concept is perhaps the most valuable bit to discuss here, imho; the vendor-particulars are less important. -Tk
"Desktop" switches. You know, those 4 or 5 port Gigabit Ethernet switches. Apparently, many of them don't do any kind of STP at all. Recommendations on ones that do STP?
If the network fabric you're on is important enough to cause you grief in the event of a STP event, you shouldn't be fielding 'dumb' switches. Even the 'dumbest' switch I would ever place into user-space is fully managable, layer 2 with VLAN's and STP support. That is, it's in a cabinet or TC and fed by infrastructure cabling, and the only folks who can get at it are the engineers and techs supporting the site. The other side of things is that if DHCP times out once during STP negotiation, it rarely times out twice. Users whos machines are 'dynamically connected' often enough to have STP related glitches in their DHCP grab should know enough to hit 'repair' or run ipconfig /renew - or should be told to reboot :-)
RSTP: is it any better than traditional STP in regards to "edge" ports and blocking before a loop gets out of hand? Or perhaps blocking for 5-10 seconds before going into Forwarding state, hopefully preventing loops before they happen but also allowing DHCP clients to get an address without timeouts? Recommendations on "Desktop" switches that do RSTP?
There's plenty of desktop switches out there which are close to 'fully featured' - but obviously there's money involved. If your uplink switch (at the very least) supports STP then at least you can isolate the problem if the switch itself can't handle, but I wouldn't recommend this. Havn't fielded any recently but there's a fanless version of the Cisco 2960 I was looking at a while ago for desktop use (fan noise is usualy an issue). Mark.
On Mar 26, 2010, at 4:33 PM, Mark Foster wrote:
"Desktop" switches. You know, those 4 or 5 port Gigabit Ethernet switches. Apparently, many of them don't do any kind of STP at all. Recommendations on ones that do STP?
If the network fabric you're on is important enough to cause you grief in the event of a STP event, you shouldn't be fielding 'dumb' switches.
Even the 'dumbest' switch I would ever place into user-space is fully managable, layer 2 with VLAN's and STP support. That is, it's in a cabinet or TC and fed by infrastructure cabling, and the only folks who can get at it are the engineers and techs supporting the site.
The other side of things is that if DHCP times out once during STP negotiation, it rarely times out twice. Users whos machines are 'dynamically connected' often enough to have STP related glitches in their DHCP grab should know enough to hit 'repair' or run ipconfig /renew - or should be told to reboot :-)
or reboot is problematic in many cases. Many systems drop link-state during reboot for a long-enough period that the bridge-port restarts its spanning tree process, making results across reboots consistently bad.
RSTP: is it any better than traditional STP in regards to "edge" ports and blocking before a loop gets out of hand? Or perhaps blocking for 5-10 seconds before going into Forwarding state, hopefully preventing loops before they happen but also allowing DHCP clients to get an address without timeouts? Recommendations on "Desktop" switches that do RSTP?
There's plenty of desktop switches out there which are close to 'fully featured' - but obviously there's money involved. If your uplink switch (at the very least) supports STP then at least you can isolate the problem if the switch itself can't handle, but I wouldn't recommend this.
With the additional advantage that the uplink switch link to the conference-room switch doesn't flap often enough to cause DHCP issues, but, will shut down the port if properly configured and the conference-room switch at least passes the BPDUs around the loop. Owen
or reboot is problematic in many cases. Many systems drop link-state during reboot for a long-enough period that the bridge-port restarts its spanning tree process, making results across reboots consistently bad.
Interesting; Windows tends to bring link up well-prior to the login dialogue and ive never seen a dhcp lease fail such that the user has had no lease by the time they try to login...
On Mar 26, 2010, at 9:24 PM, Mark Foster <blakjak@blakjak.net> wrote:
or reboot is problematic in many cases. Many systems drop link- state during reboot for a long-enough period that the bridge-port restarts its spanning tree process, making results across reboots consistently bad.
Interesting; Windows tends to bring link up well-prior to the login dialogue and ive never seen a dhcp lease fail such that the user has had no lease by the time they try to login...
Easy to make happen with 802.1X, default IOS timers and an unconfigured supplicant
On Fri, 26 Mar 2010 18:09:22 -0400 Chuck Anderson <cra@WPI.EDU> wrote:
Anyone have suggestions on Ethernet LAN loop-prevention? With the advent of Auto MDI/MDI-X ports on switches, it seems way too easy to accidentally or maliciously create loops between network jacks. We
Some time ago I did some work on implementing what cisco called 'port security'. The idea was to add some layer 2 protection from a security perspective. It turns out in practice, at least in the environment I was in, they never happen. However, it did offer protection for loops since if a secured port saw a source address show up another another port, it would block it and generate logs, which we monitored and could then go deal with while the network remained up. There are some potential gotchas depending on how you implement port security so you need consider carefully how you implement it if you do it. Its been awhile since I've done anything in this space, but this better captures my experience since my memory of it is beginning to fade: <http://www.ops.ietf.org/lists/opsec/opsec.2005/msg00033.html> John
We had a school district that had a large number of "dumb" switches in each class room hanging off real ones. These would get looped when a student or staff member plugged a patch cable into two ports on the end switch, taking down large portions of the network. It seems Cisco 3500's ignore a BPDU that comes in the same port it comes out. We switched them to 3750's as part of other upgrades, which eliminated the BPDU problem (3560's and 3550's also work correctly), RSTP, enabled port fast, root guard, loop back detection, and storm control. Then set the switches to automatically come back in service from err-disable after 60 seconds or so. In every single test we did (looping off a dumb switch, looping two ports on the 3750, looping between two 3750 in different stacks), there was immediate blocking occurring that prevented any non-sense from effecting the network. Of course the little switches get taken out along with anything connected, but that's really just an indicator of the need for more drops from real switches. Additionally, turning on only one of the features at a time still shut down the port within a second or so. I don't really like BPDUGuard when rootguard is available, as I think other devices should be able to participate in STP so long as they aren't trying to reconverge the network by grabbing root or becoming a transit between two building switches. As for RSTP, it's on for every switch we deploy unless there is some compelling reason not to do so. I have yet to find another switch that will not work even if it only supports "old" STP. -WT -----Original Message----- From: Chuck Anderson [mailto:cra@WPI.EDU] Sent: Friday, March 26, 2010 6:09 PM To: nanog@nanog.org Subject: Auto MDI/MDI-X + conference rooms + bored == loop Anyone have suggestions on Ethernet LAN loop-prevention? With the advent of Auto MDI/MDI-X ports on switches, it seems way too easy to accidentally or maliciously create loops between network jacks. We have bored or inattentive people plugging in patch cords between adjacent network jacks. STP for loop-prevention isn't working so well for us. STP "edge" or "portfast" or "faststart" modes are required for end-station ports (with normal STP, DHCP often times out after 30+ seconds it takes to go into Forwarding state). Since the "edge" STP mode goes into Forwarding state immediately, there is a period when loops will form, causing havok with upstream gear until STP blocks the port (if it ever does see below). "Desktop" switches. You know, those 4 or 5 port Gigabit Ethernet switches. Apparently, many of them don't do any kind of STP at all. Recommendations on ones that do STP? RSTP: is it any better than traditional STP in regards to "edge" ports and blocking before a loop gets out of hand? Or perhaps blocking for 5-10 seconds before going into Forwarding state, hopefully preventing loops before they happen but also allowing DHCP clients to get an address without timeouts? Recommendations on "Desktop" switches that do RSTP? Thanks for your suggestions/discussion. -- - Chuck (354 Days until IPv4 depletion: http://ipv4depletion.com/)
I had a similar issue in which someone had accidentally looked a Cisco VoIP phone back into the network. However, I found it strange how often this would occur and eventually came across this field notice that might apply to others on the list: http://www.cisco.com/en/US/ts/fn/610/fn61863.html Problem Description Disconnecting power from a locally powered Cisco IP Phone connected to a non-Power Over Ethernet (POE) Cisco switch may expose the customer's network to loop back storms that destabalize the virtual local area network (VLAN). This exposure can be mitigated by configuring the switches with automatic loop detection and port recovery. Notes indicate this normally applies onto to 10Mb connections, but that there have been reported cases on a 100Mbit network when the VoIP phone is connected to a "highly sensitive uplink switch". Trey
participants (12)
-
Anton Kapela -
Chuck Anderson -
James Hess -
John Kristoff -
John Payne -
Mark Foster -
Matthew Huff -
Mike Lyon -
Owen DeLong -
Steven King -
Trey Valenta -
William Mullaney