Thanks for everyone's great input. Here are answers to Justin's questions. #1 - 12.3.6a - 7204VXR (NPE400) 512MB - 200+ MB free #2 - 12.2.15T5 - cisco 7204VXR (NPE225) - 256MB (I have a NPE400 - 512MB I want to swap in) - 23MB Free (Issue?) Full Routes from all peers. No internal routing protocol as of yet, all static routes. Getting ready to implement OSPF. I have not rebooted the routers as a test. I have CEF on both routers. I have had some customers complaining about slowness. Mike Walter, MCP Systems Administrator 3z.net a PCD Company http://www.3z.net "When Success is the Only Solution think 3z.net" Voice (859) 331-9004 Fax (859) 578-3522 -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Justin M. Streiner Sent: Wednesday, September 06, 2006 9:42 AM To: nanog@merit.edu Subject: Re: Router / Protocol Problem On Wed, 6 Sep 2006, Mike Walter wrote:
I normally would not post to the group, but I am 100% stumped and have talked with peers with no luck.
I have (2) Cisco 7204 Routers running BGP with 3 peers and HSRP. I am not doing anything special with BGP, pretty much a default config that has not changed in years.
Please provide details on both your default config and the hardware you're using. You say you have two Cisco 7204s - are these straight '04s, or 7204VXRs? What NPE(s) are you using, and how much memory is on them? The BGP you're getting from your peers - are you getting full routes from any of them? Do you have CEF enabled on these routers? What IOS version(s) are running on these routers? What else are they doing besides slinging BGP routes? Does the problem go away for a while if you reboot one router or the other? Without knowing any of this, it sounds like you might have NPE-225, -300, or -400 with 256 MB of RAM and you are running into memory exhaustion issues from carrying full routes. That's been a pretty popular topic on this list and others like cisco-nsp in the last 12 months :) At a minimum, what do the output of "show mem summary" and "show ip bgp sum" from each router show you? Have you seen other performance problems lately, such as things getting mysteriously slower, beyond the rachability issues you mentioned above? If so, check if CEF is still running (if it was configured in the first place). When a 7200 gets dangerously low on free memory and CEF is running, it may cannibalize the IP CEF process to try to conserve memory. Earlier 12.0 releases did this - I don't know if newer ones still do it. jms
On Wed, 6 Sep 2006, Mike Walter wrote:
Thanks for everyone's great input. Here are answers to Justin's questions.
#1 - 12.3.6a - 7204VXR (NPE400) 512MB - 200+ MB free #2 - 12.2.15T5 - cisco 7204VXR (NPE225) - 256MB (I have a NPE400 - 512MB I want to swap in) - 23MB Free (Issue?)
Full Routes from all peers. No internal routing protocol as of yet, all static routes. Getting ready to implement OSPF. I have not rebooted the routers as a test. I have CEF on both routers. I have had some customers complaining about slowness.
That NPE-225 could be the culprit. 23 MB free is workable, but definitely low. I see that router is running 12.2.15T5. T train IOS releases are where features are normally introduced first and are often more buggy than mainline IOS releases. I wouldn't be surprised if that release has some memory leaks that is exacerbating your issue. If you graph memory utilization with MRTG or some other handy graphing tool, memory leaks often show up as a pronounced 'saw-tooth' pattern when you look at the utilization over time. Also, how badly is the CPU getting hit on that NPE-225? Since you're carrying full routes, I wouldn't be surprised to see the BGP Scanner process beating your CPU up every 60-ish seconds. What does the output of "show proc cpu hist" show you? If you have lots of spikes up to 100% peak utilization in the history graphs, then that can definitely be a source of latency. jms
On Wed, 6 Sep 2006, Mike Walter wrote:
Thanks for everyone's great input. Here are answers to Justin's questions.
#1 - 12.3.6a - 7204VXR (NPE400) 512MB - 200+ MB free #2 - 12.2.15T5 - cisco 7204VXR (NPE225) - 256MB (I have a NPE400 - 512MB I want to swap in) - 23MB Free (Issue?)
Full Routes from all peers. No internal routing protocol as of yet, all static routes. Getting ready to implement OSPF. I have not rebooted the routers as a test. I have CEF on both routers. I have had some customers complaining about slowness.
No internal routing protocol? Not even iBGP? How do the 2 routers exchange info? How do the internal systems know which router to exit from? Or are they both independent? I assume you are AS26241 and peer with 3356, 4323 and 6181. I also assume you should be announcing your 2 prefixes: 69.4.64.0/20 216.68.104.0/21 but you have deaggregated a single /24 - 69.4.71.0/24 which has sent 34 BGP updates in the past 24 hours (which might be ok). So, it is a bit hard to debug this with only partial info. Regards, Hank Nussbacher http://www.interall.co.il
participants (3)
-
Hank Nussbacher -
Justin M. Streiner -
Mike Walter