Hi, I have a couple of questions concerning BGP deployment. I would appreciate any feedback from folks knowledged in the deployment. Thanks very much. 1. What is the maximum no. of peers a core-BGP peers with externally? What is a good average or median number? How does this vary with Tier-1 BGP speakers vs. Tier-2 BGP speakers? Also, What is an average no. of peers a BGP border router multi-homes with? (Do not include Border routers with a single ISP peer - only the multi-homed border routers) 2. I understand, an AS by itself does not originate more than 10,000 (UUnet being the one with this many) subnets. But, I believe, when you peer with a tier-1 ISP BGP speaker, you will get AS Paths for the entire 90,000+ routes (or whatever the maximum core routing tabel size is) exchanged at BGP connection setup time. On the other hand, I believe, the number of routes exchanged to be much less when you peer with a tier-2 BGP. What is a resonable average size of routing entries you could expect from a tier-2 ISP (and even a Tier-1 ISP, for that matter)? 3. Do yo have an estimate of memory requirements for some of the core routers (peering with tier-1 ISPs or tier-2 ISPs)? Is there a relation with the number of BGP peers? Thanks. Regards, suresh ===== __________________________________________________ Do You Yahoo!? Get personalized email addresses from Yahoo! Mail - only $35 a year! http://personal.mail.yahoo.com/
On Tue, Feb 13, 2001 at 01:00:00PM -0800, Pyda Srisuresh wrote:
1. What is the maximum no. of peers a core-BGP peers with externally? What is a good average or median number? How does this vary with Tier-1 BGP speakers vs. Tier-2 BGP speakers? Also, What is an average no. of peers a BGP border router multi-homes with? (Do not include Border routers with a single ISP peer - only the multi-homed border routers)
I'm not sure what the maximum number supported by various OSes is, but most people seem to limit it to around 30-50 per router. Of course, the realisitic limit depends on router CPU and memory hardware more than anything else - a Cisco 3640 isn't going to be able to handle as much as a 12000GSR. An "average" is meaningless - there are many rotuers out there that are multihomed to only two or three ISPs and therefore only have a handful of BGP sessions.
2. I understand, an AS by itself does not originate more than 10,000 (UUnet being the one with this many) subnets. But, I believe, when you peer with a tier-1 ISP BGP speaker, you will get AS Paths for the entire 90,000+ routes (or whatever the maximum core routing tabel size is) exchanged at BGP connection setup time. On the other hand, I believe, the number of routes exchanged to be much less when you peer with a tier-2 BGP. What is a resonable average size of routing entries you could expect from a tier-2 ISP (and even a Tier-1 ISP, for that matter)?
Any ISP shuld give you the entire internet routing table - 90k+ prefixes. However, your router will only use a certain number of those as "best" routes. The number of "best" routes per ISP depends entirely on who the ISPs are, and doesn't (necessarily) have any relation to what Tier the ISPs are.
3. Do yo have an estimate of memory requirements for some of the core routers (peering with tier-1 ISPs or tier-2 ISPs)? Is there a relation with the number of BGP peers?
A Cisco 3640 with 64Mb will (Or at least did) just about handle a BGP feed from two or three peers. Memory requirements (And CPU requirements) increase with the number of peers, but 192Mb should be plenty for most applications. -- Ryan O'Connell - <ryan@complicity.co.uk> - http://www.complicity.co.uk I'm not losing my mind, no I'm not changing my lines, I'm just learning new things with the passage of time
See below... On Wed, 14 Feb 2001, Ryan O'Connell wrote:
On Tue, Feb 13, 2001 at 01:00:00PM -0800, Pyda Srisuresh wrote:
1. What is the maximum no. of peers a core-BGP peers with externally? What is a good average or median number? How does this vary with Tier-1 BGP speakers vs. Tier-2 BGP speakers? Also, What is an average no. of peers a BGP border router multi-homes with? (Do not include Border routers with a single ISP peer - only the multi-homed border routers)
I'm not sure what the maximum number supported by various OSes is, but most people seem to limit it to around 30-50 per router. Of course, the realisitic limit depends on router CPU and memory hardware more than anything else - a Cisco 3640 isn't going to be able to handle as much as a 12000GSR.
An "average" is meaningless - there are many rotuers out there that are multihomed to only two or three ISPs and therefore only have a handful of BGP sessions.
A GSR with 256MB will handle many full views. I haven't tested to the limits, but 50 would be a comfortable number. IOS based routers store additional views relatively efficiently. I'm sure someone with Cisco can give a better number, but from observation, if a full view take n memory, than each additional full view takes about .1n memory. On a Juniper, this relationship is much more linear. That's why most Junipers come with much more memory - 256mb will go south after a dozen full views. 768mb is recommended (and is what they ship with now, BTW).
2. I understand, an AS by itself does not originate more than 10,000 (UUnet being the one with this many) subnets. But, I believe, when you peer with a tier-1 ISP BGP speaker, you will get AS Paths for the entire 90,000+ routes (or whatever the maximum core routing tabel size is) exchanged at BGP connection setup time. On the other hand, I believe, the number of routes exchanged to be much less when you peer with a tier-2 BGP. What is a resonable average size of routing entries you could expect from a tier-2 ISP (and even a Tier-1 ISP, for that matter)?
Any ISP shuld give you the entire internet routing table - 90k+ prefixes. However, your router will only use a certain number of those as "best" routes. The number of "best" routes per ISP depends entirely on who the ISPs are, and doesn't (necessarily) have any relation to what Tier the ISPs are.
Routing policies (i.e. what you filter), have a much greater effect on routing table size as perceived by a downstream
3. Do yo have an estimate of memory requirements for some of the core routers (peering with tier-1 ISPs or tier-2 ISPs)? Is there a relation with the number of BGP peers?
A Cisco 3640 with 64Mb will (Or at least did) just about handle a BGP feed from two or three peers. Memory requirements (And CPU requirements) increase with the number of peers, but 192Mb should be plenty for most applications.
Not any more. We have had customer try this configuration and go to malloc hell. 128mb is the minimum for a full view these days. A Riverstone RS8000 or a Cisco 3640 will handle several full views nicely with 128mb. Remember the 3640 won't go above 128mb, though. A 3660 or a SSR makes a nice CPE box if you need 256mb.
-- Ryan O'Connell - <ryan@complicity.co.uk> - http://www.complicity.co.uk
I'm not losing my mind, no I'm not changing my lines, I'm just learning new things with the passage of time
Daniel Golding
On Wed, 14 Feb 2001, Daniel L. Golding wrote:
A GSR with 256MB will handle many full views. I haven't tested to the limits, but 50 would be a comfortable number. IOS based routers store additional views relatively efficiently. I'm sure someone with Cisco can give a better number, but from observation, if a full view take n memory, than each additional full view takes about .1n memory.
256MB is not much anymore. Today, we see ~1.6million paths and ~100,000 routes. with a full iBGP mesh in the core, and current memory leaks from ciscos IOS (know issues) the GSRs are running out of memory. too bad we cant replace our GSRs with 7200s as they can hold 512M of ram :) Christian
On Wed, Feb 14, 2001 at 11:00:55AM -0500, Daniel L. Golding wrote: [snip various points that I agree with]
On Wed, 14 Feb 2001, Ryan O'Connell wrote:
A Cisco 3640 with 64Mb will (Or at least did) just about handle a BGP feed from two or three peers. Memory requirements (And CPU requirements) increase with the number of peers, but 192Mb should be plenty for most applications.
Not any more. We have had customer try this configuration and go to malloc hell. 128mb is the minimum for a full view these days. A Riverstone RS8000 or a Cisco 3640 will handle several full views nicely with 128mb. Remember the 3640 won't go above 128mb, though. A 3660 or a SSR makes a nice CPE box if you need 256mb.
Unless Cisco have changed something, a 3640 can only take 64Mb of memory max. You'd need a 3660 for 128Mb. You can get away with a Cisco 3640 for full BGP, but it does depend on the cards you have installed. NM-1A-T3 is a memory hog for example, but if you've only got NM-2E2Ws you're probably OK. You might also need to tweak iomem, but later IOS releases do this for you. 11.2 might be a better choice for BGP-speaking 3640s are the memory requirements are lower. -- Ryan O'Connell - <ryan@complicity.co.uk> - http://www.complicity.co.uk I'm not losing my mind, no I'm not changing my lines, I'm just learning new things with the passage of time
On Tue, Feb 13, 2001 at 01:00:00PM -0800, Pyda Srisuresh wrote: Hi, I have a couple of questions concerning BGP deployment. I would appreciate any feedback from folks knowledged in the deployment. Thanks very much. warning do not consider myself knowledged in bgp deployment. however, caida's skitter project http://www.caida.org/tools/measurement/skitter has more data on topology/connectivity (by far) than anything i've encountered and some of our analyses may not be totally useless to you (andre did all the work below; i just translated it to english...) Pyda: 1. What is the maximum no. of peers a core-BGP peers with externally? What is a good average or median number? How does this vary with Tier-1 BGP speakers vs. Tier-2 BGP speakers? Also, What is an average no. of peers a BGP border router multi-homes with? (Do not include Border routers with a single ISP peer - only the multi-homed border routers) see: http://www.caida.org/~broido/bgp/asoutdeg.html 2. I understand, an AS by itself does not originate more than 10,000 (UUnet being the one with this many) subnets. But, I believe, when you peer with a tier-1 ISP BGP speaker, you will get AS Paths for the entire 90,000+ routes (or whatever the maximum core routing tabel size is) exchanged at BGP connection setup time. On the other hand, I believe, the number of routes exchanged to be much less when you peer with a tier-2 BGP. What is a resonable average size of routing entries you could expect from a tier-2 ISP (and even a Tier-1 ISP, for that matter)? http://www.caida.org/~broido/bgp/stubas.html (viz could be better on some of the graphs, but you'll get the point) btw wrt your UUcomment i think 701 originates about 2300 prefixes (at least on 29 november seen from routeviews, w another 74 announced by 701 and by other ASes... to routeviews, anyway) (if you count 702-705,7046,284,etc, it was another few K maybe. not more than 5K total even including uucanada.) another 1K or so might be announced by 701 from stubs behind 701etc, haven't done that analysis 3. Do you have an estimate of memory requirements for some of the core routers (peering with tier-1 ISPs or tier-2 ISPs)? Is there a relation with the number of BGP peers? think someone else got that one (good thing) k and andre // to the pure geometer the radius of curvature is an incidental characteristic - like the grin of the cheshire cat. to the physicist it is an indispensable characteristic. it would be going too far to say that to the physicist i the cat is merely incidental to the grin. physics is concerned with interrelatedness such as the interrelatedness of cats and grins. in this case the cat without a grin and the grin without a cat are equally set aside as purely mathematical phantasies. -- sir arthur eddington, the expanding universe //
participants (5)
-
Christian Nielsen -
Daniel L. Golding -
k claffy -
Pyda Srisuresh -
Ryan O'Connell