ARP Table Timeout and Mac-Address-Table Timeout
I am a network engineer for a large web hosting company. We are having an issue with our distribution routers flooding traffic in one of our VLANs. We have a customer with a routed mode ASA 5550. They have their own private VLAN that is a /23 This VLAN is 145. The outside interface of the firewall is in VLAN 132. We are routing all traffic for VLAN 145 to the IP of the outside interface of the firewall in VLAN 132. VLAN 132 is Layer3 routable and VLAN 145 is only Layer2 switchable. We have two distribution switches which are redundant with HSRP. Dist1 is the active forwarder in this case. Traffic coming into these two routers are load balanced between Dist1 and Dist2 with EIGRP routes with equal cost. We have found that traffic coming into Dist2 (the standby) is flooding traffic destined for the firewall outside interface. But Dist1 is not. We have tracked down the cause of this to the MAC-Address-Table timing out before the ARP table times out. We leave these values at the Cisco default. ARP = 4hr MAC = 5 minutes. Since Dist2 is not receiving any traffic from the firewall going out to the internet, it is not updating the MAC-Address-Table after it expires. Instead, it waits 4 hours for the ARP cache to expire for that IP, and then updates everything. But Dist2 ends up flooding traffic for that 4 hours causing latency. We have done some research on this problem and have found so far the best solution to be to make the ARP timeout less than the MAC-Address-Table aging-timer.We have set the ARP = 1hr and MAC = 2hrs in this case to correct the problem. So when the ARP entry times out before the MAC entry, the forced update of the ARP entry before the MAC timeout causes the MAC entry age to reset. Indeed this does correct the problem. Is this the best solution to the problem, or is there another preferred solution? Has anyone ran into this in their own Enterprise Networks? Please let me know if I didn't explain anything well enough. -- Steve King Network Engineer - Liquid Web, Inc. Cisco Certified Network Associate CompTIA Linux+ Certified Professional CompTIA Network+ Certified Professional CompTIA A+ Certified Professional
Steven: This was recently discussed on cisco-nsp: http://marc.info/?l=cisco-nsp&m=121316151010190&w=2 Frank -----Original Message----- From: Steven King [mailto:sking@kingrst.com] Sent: Sunday, September 14, 2008 7:27 PM To: nanog@nanog.org Subject: ARP Table Timeout and Mac-Address-Table Timeout I am a network engineer for a large web hosting company. We are having an issue with our distribution routers flooding traffic in one of our VLANs. We have a customer with a routed mode ASA 5550. They have their own private VLAN that is a /23 This VLAN is 145. The outside interface of the firewall is in VLAN 132. We are routing all traffic for VLAN 145 to the IP of the outside interface of the firewall in VLAN 132. VLAN 132 is Layer3 routable and VLAN 145 is only Layer2 switchable. We have two distribution switches which are redundant with HSRP. Dist1 is the active forwarder in this case. Traffic coming into these two routers are load balanced between Dist1 and Dist2 with EIGRP routes with equal cost. We have found that traffic coming into Dist2 (the standby) is flooding traffic destined for the firewall outside interface. But Dist1 is not. We have tracked down the cause of this to the MAC-Address-Table timing out before the ARP table times out. We leave these values at the Cisco default. ARP = 4hr MAC = 5 minutes. Since Dist2 is not receiving any traffic from the firewall going out to the internet, it is not updating the MAC-Address-Table after it expires. Instead, it waits 4 hours for the ARP cache to expire for that IP, and then updates everything. But Dist2 ends up flooding traffic for that 4 hours causing latency. We have done some research on this problem and have found so far the best solution to be to make the ARP timeout less than the MAC-Address-Table aging-timer.We have set the ARP = 1hr and MAC = 2hrs in this case to correct the problem. So when the ARP entry times out before the MAC entry, the forced update of the ARP entry before the MAC timeout causes the MAC entry age to reset. Indeed this does correct the problem. Is this the best solution to the problem, or is there another preferred solution? Has anyone ran into this in their own Enterprise Networks? Please let me know if I didn't explain anything well enough. -- Steve King Network Engineer - Liquid Web, Inc. Cisco Certified Network Associate CompTIA Linux+ Certified Professional CompTIA Network+ Certified Professional CompTIA A+ Certified Professional
I saw that one before. Thats what we based our current fix on. Frank Bulk wrote:
Steven:
This was recently discussed on cisco-nsp: http://marc.info/?l=cisco-nsp&m=121316151010190&w=2
Frank
-----Original Message----- From: Steven King [mailto:sking@kingrst.com] Sent: Sunday, September 14, 2008 7:27 PM To: nanog@nanog.org Subject: ARP Table Timeout and Mac-Address-Table Timeout
I am a network engineer for a large web hosting company. We are having an issue with our distribution routers flooding traffic in one of our VLANs.
We have a customer with a routed mode ASA 5550. They have their own private VLAN that is a /23 This VLAN is 145. The outside interface of the firewall is in VLAN 132. We are routing all traffic for VLAN 145 to the IP of the outside interface of the firewall in VLAN 132.
VLAN 132 is Layer3 routable and VLAN 145 is only Layer2 switchable.
We have two distribution switches which are redundant with HSRP. Dist1 is the active forwarder in this case. Traffic coming into these two routers are load balanced between Dist1 and Dist2 with EIGRP routes with equal cost.
We have found that traffic coming into Dist2 (the standby) is flooding traffic destined for the firewall outside interface. But Dist1 is not.
We have tracked down the cause of this to the MAC-Address-Table timing out before the ARP table times out. We leave these values at the Cisco default. ARP = 4hr MAC = 5 minutes. Since Dist2 is not receiving any traffic from the firewall going out to the internet, it is not updating the MAC-Address-Table after it expires. Instead, it waits 4 hours for the ARP cache to expire for that IP, and then updates everything. But Dist2 ends up flooding traffic for that 4 hours causing latency.
We have done some research on this problem and have found so far the best solution to be to make the ARP timeout less than the MAC-Address-Table aging-timer.We have set the ARP = 1hr and MAC = 2hrs in this case to correct the problem. So when the ARP entry times out before the MAC entry, the forced update of the ARP entry before the MAC timeout causes the MAC entry age to reset. Indeed this does correct the problem.
Is this the best solution to the problem, or is there another preferred solution? Has anyone ran into this in their own Enterprise Networks?
Please let me know if I didn't explain anything well enough.
-- Steve King
Network Engineer - Liquid Web, Inc. Cisco Certified Network Associate CompTIA Linux+ Certified Professional CompTIA Network+ Certified Professional CompTIA A+ Certified Professional
-- Steve King Network Engineer - Liquid Web, Inc. Cisco Certified Network Associate CompTIA Linux+ Certified Professional CompTIA Network+ Certified Professional CompTIA A+ Certified Professional
participants (2)
-
Frank Bulk -
Steven King