High Processor Rates on Routers.
Ladies & Gentleman. Was wondering if it is common for processor rates on higher end Crisco boxes to race from 1 or 2 % show on the 5 second processor rate interval, to anywhere from 70 to 90 %, instanteously, then drop back down to 1 or 2 % after 10 seconds. The above mentioned scenario would happen every 30 seconds. The higher end boxes are very lightly loaded, with total traffic throughput of maybe 100 Meg, in some cases. Think that if these boxes are loaded more, to the point where the aggragated throughtput approaches a Gig, this will cause the processors to run at high levels constantly, driving the CPU rates indicators on the box for 1 & 5 minute intervals up to the values mentioned above. ( 70 to 90 % ). Higher end devices would be something like 7200, 7500, 12xxx series routers. This scenerio would be in a large ( very large network, atleast a tier 2 OR tier 1 provider )... possible rebutals to the fact that processors could race like mentioned above on lightly loaded devices of a higher end nature would be that BGP scanner is causing the problem, Telnet to the devices, show config when on the devices... So what happens to the processor rates when the devices are more heavily loaded ( 1 G aggragated throughput opposed to 100 Meg throughtput )? Will the devices work worrectly AND would a large internet be stable ? Is it also common for BGP Routing tables to be changing version every 1/2 second ? Same high end routers, lightly loaded, large Autonomous System. Thank-you for your time on this query.
On Wed, Nov 06, 2002 at 08:21:39AM -0500, internetguy205@hotmail.com wrote:
Ladies & Gentleman.
Was wondering if it is common for processor rates on higher end Crisco boxes to race from 1 or 2 % show on the 5 second processor rate interval, to anywhere from 70 to 90 %, instanteously, then drop back down to 1 or 2 % after 10 seconds.
Check out for down interfaces, Serial/POS on the router. I've seen this cause these spikes (although not normal for 10 seconds, more normally about a second or so) every 30 seconds and make sure they're shut down. Also check out OSPF to see if there is an interface flapping somewhere in your network with the undocumented command 'sh ip ospf stat', which will tell you what OSPF is doing, and look for a correlation between the delta times and your high CPU utilisation. BGP can cause a lot of processor utilisation when updates are received, although this is not normally at accurate 30 second intervals, so I wouldn't suspect this particularly. Cheers, -- |=========----- -------=======| | Chris Roberts (croberts@bongle.co.uk) | |=======------- -----=========|
## On 2002-11-06 15:54 -0000 Chris Roberts typed: CR> CR> CR> BGP can cause a lot of processor utilisation when updates are received, CR> although this is not normally at accurate 30 second intervals, so I CR> wouldn't suspect this particularly. CR> I've seen this happen with OSPF routes flapping(due to a flapping interface) causing the BGP next-hop to flap - the OSPF didn't seem to be the culprit as it only had ~150 routes while BGP had ~110K routes (mostly iBGP) -- Rafi
I have seen similar behavior like this, on various platforms (3640, 7206 and 7513) when using PBR (Policy Based Routing) on an interface looking at source IP's to make a routing decision. The BGP SCANNER process jumps up to 99% for a second or 2 every 60 seconds. After taking out the POLICY MAP's on the appropriate interfaces, the BGP SCANNER processes never jump up again. I never really found out why this happened from Cisco, TAC stated that most routers do not have specialized ASIC's to handle PBR, and so it is handled like a generic CPU process, and if your BGP table is large, and it has to walk the BGP TABLE often (if you receive large amounts of updates etc...), and PBR is enabled, weird things happen... Hope this helps... -Aaron -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of Chris Roberts Sent: Wednesday, November 06, 2002 8:54 AM To: internetguy205@hotmail.com Cc: nanog@merit.edu; headguy@techies.com Subject: Re: High Processor Rates on Routers. On Wed, Nov 06, 2002 at 08:21:39AM -0500, internetguy205@hotmail.com wrote:
Ladies & Gentleman.
Was wondering if it is common for processor rates on higher end Crisco
boxes to race from 1 or 2 % show on the 5 second processor rate interval, to anywhere from 70 to 90 %, instanteously, then drop back down to 1 or 2 % after 10 seconds.
Check out for down interfaces, Serial/POS on the router. I've seen this cause these spikes (although not normal for 10 seconds, more normally about a second or so) every 30 seconds and make sure they're shut down. Also check out OSPF to see if there is an interface flapping somewhere in your network with the undocumented command 'sh ip ospf stat', which will tell you what OSPF is doing, and look for a correlation between the delta times and your high CPU utilisation. BGP can cause a lot of processor utilisation when updates are received, although this is not normally at accurate 30 second intervals, so I wouldn't suspect this particularly. Cheers, -- |=========----- -------=======| | Chris Roberts (croberts@bongle.co.uk) | |=======------- -----=========|
If you happen NOT to run CEF on your boxes which see large number of prefixes, you'll see CPU jump every 60 seconds when the box does cache cleaning. This is not an option 12xxx's. Pete internetguy205@hotmail.com wrote:
Ladies & Gentleman.
Was wondering if it is common for processor rates on higher end Crisco boxes to race from 1 or 2 % show on the 5 second processor rate interval, to anywhere from 70 to 90 %, instanteously, then drop back down to 1 or 2 % after 10 seconds.
The above mentioned scenario would happen every 30 seconds.
The higher end boxes are very lightly loaded, with total traffic throughput of maybe 100 Meg, in some cases.
Think that if these boxes are loaded more, to the point where the aggragated throughtput approaches a Gig, this will cause the processors to run at high levels constantly, driving the CPU rates indicators on the box for 1 & 5 minute intervals up to the values mentioned above. ( 70 to 90 % ).
Higher end devices would be something like 7200, 7500, 12xxx series routers.
This scenerio would be in a large ( very large network, atleast a tier 2 OR tier 1 provider )... possible rebutals to the fact that processors could race like mentioned above on lightly loaded devices of a higher end nature would be that BGP scanner is causing the problem, Telnet to the devices, show config when on the devices...
So what happens to the processor rates when the devices are more heavily loaded ( 1 G aggragated throughput opposed to 100 Meg throughtput )? Will the devices work worrectly AND would a large internet be stable ?
Is it also common for BGP Routing tables to be changing version every 1/2 second ? Same high end routers, lightly loaded, large Autonomous System.
Thank-you for your time on this query.
participants (5)
-
Aaron D. Britt -
Chris Roberts -
internetguy205ļ¼ hotmail.com -
Petri Helenius -
Rafi Sadowsky