When will 128M not be enough?
Is there any research out there that tries to estimate how long it will be before BGP routers with 128M of RAM start to choke on the routing table? I know there are a lot of factors - the number of BGP peers, how much "other stuff" the router is doing, etc. - I'm really looking for a ballpark estimate, if one exists. I'm trying to convince someone NOT to use a Cisco 2650 for BGP multihoming, and my argument will, IMO, be a bit more effective if I can say "you'll need to replace it within X months/years"... -C --------------------------- Christopher A. Woodfield rekoil@semihuman.com PGP Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xB887618B
It looks like some recent aggregation has been helping to slow down the growth. In any case, I've got 48MB (out of 128MB) free with 2 full views and very little else (a few ACLs, OSPF) on a 7204VXR. On Sat, 14 Jul 2001, Christopher A. Woodfield wrote:
Is there any research out there that tries to estimate how long it will be before BGP routers with 128M of RAM start to choke on the routing table? I know there are a lot of factors - the number of BGP peers, how much "other stuff" the router is doing, etc. - I'm really looking for a ballpark estimate, if one exists.
I'm trying to convince someone NOT to use a Cisco 2650 for BGP multihoming, and my argument will, IMO, be a bit more effective if I can say "you'll need to replace it within X months/years"...
-C
--------------------------- Christopher A. Woodfield rekoil@semihuman.com
PGP Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xB887618B
James Smallacombe PlantageNet, Inc. CEO and Janitor up@3.am http://3.am =========================================================================
up@3.am schrieb:
It looks like some recent aggregation has been helping to slow down the growth.
If you look at some figures (e.g. http://www.employees.org/~tbates/cidr-report.html) it's even pretty stable around 101K prefixes. 128M should do if you only have two or three upstreams, soft reconfiguration disabled and almost nothing else enabled. Does anyone have experience with zebra/mrt/... as a route server?. I need one and would like to you for the cheap server based solution instead of having to buy a fully fledged router. TIA -- Arnold
disabled and almost nothing else enabled. Does anyone have experience with zebra/mrt/... as a route server?. I need one and would like to you for the cheap server based solution instead of having to buy a fully fledged router.
I have used Zebra for about 2 years, with two and three uplinks in what I would call a very basic conservative configuration and good results. Currently using .89a YMMV: Linux running Zebra: shredder /root]# uptime 3:29pm up 69 days, 13:28, 4 users, load average: 0.00, 0.00, 0.00 2 full feeds: %CPU %MEM VSZ RSS Start 0.7 14.3 39376 36972 Jun18 272:26 /usr/local/sbin/bgpd -d It'd be up longer if I had not been playing with it.
At 7/15/01 02:03 AM, Nipper, Arnold wrote:
up@3.am schrieb:
It looks like some recent aggregation has been helping to slow down the growth.
If you look at some figures (e.g. http://www.employees.org/~tbates/cidr-report.html) it's even pretty stable around 101K prefixes.
The question is how long will this 'stability' persist. Two factors appear to be at play at present: 1 - there has been an effort by many AS's to reduce the number of address fragments advertised into the global routing tables. The 'noise' in the routing table is being reduced (slightly) 2 - there is some slowdown in overall Internet growth in terms of a) the number of new AS's being added into the global routing table and b) the total span of address space being announced into the global routing table. Either this is an effect of an economic condition (most likely) or a number of folk at the edge are increasing their use of NAT and are disappearing behind the NAT boxes (less likely,but also possible). factor 1 will only be temporary - at some stage the entries which add no additional policy into the routing table will be reduced to the point that no further reduction is possible. At this point int time factor 1 will disappear. factor 2 may or may not disappear - if the slowing down of consumption of AS numbers and the slowing down in the rate of growth of advertised address space is a side effect of a broader economic condition, then one can expect the growth levels to resume at about the same time as we see some level of improvement in the economic outlook. If the factors are based on NAT then this condition may persist for some time. My _guess_ is that the current hiatus in routing table growth is temporary. regards, Geoff (www.telstra.net/ops/bgp has tables relating to address advertisements and AS numbers if you are interested
Two factors appear to be at play at present:
1 - there has been an effort by many AS's to reduce the number of address fragments advertised into the global routing tables. The 'noise' in the routing table is being reduced (slightly)
2 - there is some slowdown in overall Internet growth in terms of a) the number of new AS's being added into the global routing table and b) the total span of address space being announced into the global routing table. Either this is an effect of an economic condition (most likely) or a number of folk at the edge are increasing their use of NAT and are disappearing behind the NAT boxes (less likely,but also possible).
There is a third force behind this -- one that I've rarely seen mentioned here. It's called entropy. Of course, actual new autonomous systems appear all the time, needing new AS numbers. Likewise, the number of actual hosts on the Internet is increasing, justifying the need for new IP addresses. But I'd venture to guess that most of the increase in AS numbers and routes is because of increasing entropy on the Internet. We are losing those resources to natural inefficiency. It should be noted that aggregation is only part of the story; while it can reduce the number of advertised prefixes (usually just once!), it can not turn back the passage of time. And time introduces routing table bloat -- by introducing situations that make aggregation impractical or impossible. Case in point: an ISP merger or acquisition, something very common in this quickly consolidating business. You end up with two AS numbers (at minimum) and two sets of netblocks. Because of practical reasons, those assets may be combined into one in a) a week b) a month c) a year d) never. The recent example of Exodus and GlobalCenter is, to my understanding, a good one, since they are renumbering. This seems to be an exception to the rule, though. I can't see anything else changing this than mandary, RIR-enforced renumbering. My company is a member of RIPE, and the spectre of mandatory renumbering *is* flashed in the RIPE documents -- I'm just enough of a novice to never have seen or heard it happen. Does it? --
Magenta Sites Marko Karppinen
> I'm trying to convince someone NOT to use a Cisco 2650 for BGP > multihoming, and my argument will, IMO, be a bit more effective if > I can say "you'll need to replace it within X months/years"... You're doing the wrong thing. The 2650 can hold 8-10 full views currently, and it's my guess that it will continue to be adequate for at least 24 months. It costs about $3K. The alternative is a 7206VXR. Let's posit that that would cost $30K and last for four years. The cost of the 7206 is thus five times higher than simply buying appropriate technology and upgrading as necessary. Plus if you upgrade over time, you get to have new technology in the future, rather than being stuck with what will seem like an old clunker for several years. -Bill
On Sat, 14 Jul 2001, Christopher A. Woodfield wrote:
I'm trying to convince someone NOT to use a Cisco 2650 for BGP multihoming, and my argument will, IMO, be a bit more effective if I can say "you'll need to replace it within X months/years"...
I was just recently trying to convince someone just the opposit. I lost and they went with a non-Cisco solution. If the 2650's 128mb of memory isn't enough, what are you recommending? The only Cisco options I'm aware of are the 3660 series or 7x00 class...both of which are overkill for a small network with 2-3 T1's and BGP peers. -- ---------------------------------------------------------------------- Jon Lewis *jlewis@lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On Sun, 15 Jul 2001 jlewis@lewis.org wrote:
If the 2650's 128mb of memory isn't enough, what are you recommending? The only Cisco options I'm aware of are the 3660 series or 7x00 class...both of which are overkill for a small network with 2-3 T1's and BGP peers.
Kinda a different twist on the topic, but I fail to see why a site of that size needs to take a full table anyway. If all you have are a couple of T1s multihomed to a couple of the larger networks, you can probably just take customer routes and will achieve as optimum routing as possible. For even better scaling and redundancy get two smaller routers (26xx or rs3000) and run IBGP between them for still a more ecomical solution. andy
On Sun, 15 Jul 2001, Andy Walden wrote:
Kinda a different twist on the topic, but I fail to see why a site of that size needs to take a full table anyway. If all you have are a couple of T1s multihomed to a couple of the larger networks, you can probably just
In most cases you don't. What pushed the client I'm thinking of to multihome though was the C&W/PSI peering issue and the threat of similar issues in the future. They only had connectivity to C&W and needed to reach networks only on PSI. Multiple BGP peers and full routes would have worked for them in this case. Customer routes and multiple defaults would likely not have worked. -- ---------------------------------------------------------------------- Jon Lewis *jlewis@lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On Sun, 15 Jul 2001 jlewis@lewis.org wrote:
In most cases you don't. What pushed the client I'm thinking of to multihome though was the C&W/PSI peering issue and the threat of similar issues in the future. They only had connectivity to C&W and needed to reach networks only on PSI. Multiple BGP peers and full routes would have worked for them in this case. Customer routes and multiple defaults would likely not have worked.
True. Of course there are multiple ways to take customer routes, one of which is filtering. A unique situation such as the CW/PSI mess would have called for a prefix filter tweak, which, albeit, not automatic, relatively painless on a small scale (i.e, you don't have to change hundreds of customer routers because they don't have the talent themselves). andy
On Sun, 15 Jul 2001 jlewis@lewis.org wrote:
If the 2650's 128mb of memory isn't enough, what are you recommending? The only Cisco options I'm aware of are the 3660 series or 7x00 class...both of which are overkill for a small network with 2-3 T1's and BGP peers.
Kinda a different twist on the topic, but I fail to see why a site of that size needs to take a full table anyway. If all you have are a couple of T1s multihomed to a couple of the larger networks, you can probably just take customer routes and will achieve as optimum routing as possible. For even better scaling and redundancy get two smaller routers (26xx or rs3000) and run IBGP between them for still a more ecomical solution.
andy
Then what happens when one of your two providers decides to cut off their peering with another major provider? Do you just send half of your traffic to that provider to nowhere? If you want fault tolerance against connectivity losses, you need full routes. DS
David Schwatz schrieb:
to that provider to nowhere? If you want fault tolerance against connectivity losses, you need full routes.
Not necessarily. You might get from your upstream their prefixes (or anything you want) and point default to all with different metrics. So if connectivity to one of your upstreams goes away, nothing breaks. Of course if connectivity to an AS behind your default goes away you don't get backup even if it were possible via some of the other upstreams.
DS
-- Arnold
peering with another major provider? Do you just send half of your traffic to that provider to nowhere? If you want fault tolerance against connectivity losses, you need full routes.
I tried partial... after changing which upstream provider was my 'default route' a few times I quickly realized this was stupid. Ram and CPU is cheap enough to make full routes, even on a Cisco, a VERY desirable thing.
peering with another major provider? Do you just send half of your
mike harrison schrieb: traffic
to that provider to nowhere? If you want fault tolerance against connectivity losses, you need full routes.
I tried partial... after changing which upstream provider was my 'default route' a few times I quickly realized this was stupid. Ram and CPU is cheap enough to make full routes, even on a Cisco, a VERY desirable thing.
Upgrading a 2650 to 128MB for USD 5,700 is not just cheap. The box itself is around USD 3,300 ... Arnold
Well, you don't _have_ to buy your DRAM from Crisco... On Mon, Jul 16, 2001 at 03:14:53AM +0200, Nipper, Arnold wrote:
mike harrison schrieb:
peering with another major provider? Do you just send half of your traffic to that provider to nowhere? If you want fault tolerance against connectivity losses, you need full routes.
I tried partial... after changing which upstream provider was my 'default route' a few times I quickly realized this was stupid. Ram and CPU is cheap enough to make full routes, even on a Cisco, a VERY desirable thing.
Upgrading a 2650 to 128MB for USD 5,700 is not just cheap. The box itself is around USD 3,300 ...
Arnold
-- Christian Kuhtz <ck@arch.bellsouth.net> -wk, <ck@gnu.org> -hm Sr. Architect, Engineering & Architecture, BellSouth.net, Atlanta, GA, U.S. "I speak for myself only.""
True, I prefer $30 after rebate from Circuit City... :) -C On Sun, Jul 15, 2001 at 09:19:16PM -0400, Christian Kuhtz wrote:
Well, you don't _have_ to buy your DRAM from Crisco...
Upgrading a 2650 to 128MB for USD 5,700 is not just cheap. The box itself is around USD 3,300 ...
Arnold
-- Christian Kuhtz <ck@arch.bellsouth.net> -wk, <ck@gnu.org> -hm Sr. Architect, Engineering & Architecture, BellSouth.net, Atlanta, GA, U.S. "I speak for myself only.""
-- --------------------------- Christopher A. Woodfield rekoil@semihuman.com PGP Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xB887618B
On Sun, Jul 15, 2001 at 09:34:52PM -0400, Vincent J. Bono wrote:
Upgrading a 2650 to 128MB for USD 5,700 is not just cheap. The box itself is around USD 3,300 ...
Arnold
Only Cisco's RAM (you know, the soothing-ointment not included kind) costs that much. Closer to $600 from Kingston.
Or try www.crucial.com (aka Micron). Just as reputable. -- Christian Kuhtz <ck@arch.bellsouth.net> -wk, <ck@gnu.org> -hm Sr. Architect, Engineering & Architecture, BellSouth.net, Atlanta, GA, U.S. "I speak for myself only.""
On Mon, Jul 16, 2001 at 03:14:53AM +0200, Nipper, Arnold wrote:
Upgrading a 2650 to 128MB for USD 5,700 is not just cheap. The box itself is around USD 3,300 ...
Thanks for the quotes. Winners are: 2.29% of list price: Kingston 7.85% of list price: MemoryX (Cisco approved) -- Arnold
participants (14)
-
Andy Walden -
Arnold Nipper -
Bill Woodcock -
Christian Kuhtz -
Christopher A. Woodfield -
David Schwartz -
Geoff Huston -
jlewis＠lewis.org -
Majdi S. Abbas -
Marko Karppinen -
mike harrison -
Nipper, Arnold
-
up＠3.am -
Vincent J. Bono