Tony Li previously wrote:
The original post was about making time at the Pittsburg NANOG meeting to talk about solutions to the problem of some exchange routers seeing tens of thousands (even 100x) paths. The size of a full-Internet route table is growing (albeit more slowly than it used to), the number of ASes meeting at the various NAPs and MAEs is increasing, etc. Will the number of paths heard by some routers at the XPs ever be so large that they don't fit in the 7000 or 4500m's 64MB of RAM? Possibly (IMHO, likely). What then? Use lots of filtering to keep the number of paths heard down and lose flexibility?
Then I ask you to sign a nondisclosure agreement. ;-)
Oh, then the whole argument is for naught. :)
We've actually been very good (some say too good) about adding features.
Yes, Cisco is very good about adding software features and even about allowing its customers to beta test software.
Tony
Nick
Then I ask you to sign a nondisclosure agreement. ;-)
Oh, then the whole argument is for naught. :)
Correct. Please fixate on the long term growth rate. That is key. Unless you want your PC to have Gigabytes of RAM.
Tony
Uh, I've seen PC's with Gigabytes of RAM, but have yet to hear of a router with such. CIDR is a bandaid. The problem is translating from from BGP to forwarding tables installed in routers. Current routers do not have enough BGP processing power to do the BGP filtering and processing power. The other problem is the mesh nature of BGP which makes any BGP peering site, with full mesh peering, an N^2 problem. By utilizing BGP "proxies", i.e. RS or Gated workstations, you can reduce the complexity of packet forwarding to a limit of the number of routes, and the number of updates (and RS or Gated workstations could impletement routing flap dampening). Seeing as memory is typical 1/2 the price for a workstation as for a Cisco router, you can almost aford to have 4 BILLION host routes in the workstation, compared to the cost of the replacing your entire backone with the Cisco of the Month Club. (Hm... thinking on this, a Cisco 7000 with just Ethernet, and 64Megs of memory, could work as a proxy BGP also.). -- ---------------------------------------------------------------------------- | Jeremy Porter (512)-339-6094 Freeside Communications, Inc. info@fc.net | | jerry@fc.net (512)-339-4466 (data) P.O. Box 530264 Austin, TX 78753 | ----------------------------------------------------------------------------
Uh, I've seen PC's with Gigabytes of RAM, but have yet to hear of a router with such. True. There's no market yet. CIDR is a bandaid. If you seriously believe that, then you better present some other mechanism for scaling routing. We know of only one: hiearachical routing. The problem is translating from from BGP to forwarding tables installed in routers. Sorry, no. There are a number of problems. This isn't one. Current routers do not have enough BGP processing power to do the BGP filtering and processing power. I have about 100 counterexamples. Obviously, you can configure arbitrarily complex filtering and if you do that, you need an arbitrarily large amount of compute power. The fact is that there's enough to work today. The other problem is the mesh nature of BGP which makes any BGP peering site, with full mesh peering, an N^2 problem. This is fixed. Seeing as memory is typical 1/2 the price for a workstation as for a Cisco router, you can almost aford to have 4 BILLION host routes in the workstation, compared to the cost of the replacing your entire backone with the Cisco of the Month Club. No one said that you had to buy the memory for your cisco from cisco. Tony
Uh, I've seen PC's with Gigabytes of RAM, but have yet to hear of a router with such.
True. There's no market yet.
Once the market is there, its usually too late for the poor fools running the backbone.
CIDR is a bandaid.
If you seriously believe that, then you better present some other mechanism for scaling routing. We know of only one: hiearachical routing.
I SERIOUSLY belive CIDR is a bandaid. And talk of not accepting routes smaller than X bits, and not allowing holes in in CIDR blocks breaks CIDR. CIDR is a good idea, if you do NOT impletment these practices, and is a good idea in general in that case. However, it does not solve the problem of "router table growth" which perhaps could more correctly be described as path growth on the gobal Internet. The more "general" peering sites, the more total paths you have to take, in addition to the problem of total peering sessions.
The problem is translating from from BGP to forwarding tables installed in routers.
Sorry, no. There are a number of problems. This isn't one.
Ok, I guess I was confused, I understood that route flapping which causes a cisco router to need to be reloaded can be caused by a flapping session, causing all of the forwarding tables/caches to be flushed, causing the CPU load in a Cisco 7000 to reach 115%? Have I been lied to?
Current routers do not have enough BGP processing power to do the BGP filtering and processing power.
I have about 100 counterexamples. Obviously, you can configure arbitrarily complex filtering and if you do that, you need an arbitrarily large amount of compute power. The fact is that there's enough to work today.
Again, see above comment about CPU utilization and its detrmental effects to CPU load on a Cisco causing BGP sessions to fail (which causes more route flapping, etc.)
The other problem is the mesh nature of BGP which makes any BGP peering site, with full mesh peering, an N^2 problem.
This is fixed.
How is this fixed? With N peers at a peering location you must have each route must have N peering sessions, which looks like N^2 to me. Maybe a clarification of terms in order here, because it seems fairly easy for people without formal backgrounds to confuse the issue. And with the multiple path problem it seems easy enough to add N proxies to reduce the total number of paths in any one box. NOTE: this could have serious impacts on stablity with several multihope paths in a configuration, each one being a point of failure.
Seeing as memory is typical 1/2 the price for a workstation as for a Cisco router, you can almost aford to have 4 BILLION host routes in the workstation, compared to the cost of the replacing your entire backone with the Cisco of the Month Club.
No one said that you had to buy the memory for your cisco from cisco.
No, but even certified memory from my PC/workstation vendor is cheaper than Cisco certified memory. Even with SIMMs being SIMMs, I know better than to just plug anyone's memory module into any box. A typical SIMM module may have as many as 70 different timing parameters any one of which may vary by enough nanoseconds to cause a heavily loaded machine to crash. (Maybe cisco's don't do this, but every other workstation I've worked in in the last 15 years had the same problem.) -- ---------------------------------------------------------------------------- | Jeremy Porter (512)-339-6094 Freeside Communications, Inc. info@fc.net | | jerry@fc.net (512)-339-4466 (data) P.O. Box 530264 Austin, TX 78753 | ----------------------------------------------------------------------------
CIDR is a bandaid.
If you seriously believe that, then you better present some other mechanism for scaling routing. We know of only one: hiearachical routing.
However, it does not solve the problem of "router table growth" which perhaps could more correctly be described as path growth on the gobal Internet. CIDR, if correctly deployed, allows aggregation to happen. This either slows or terminates the growth of the routing table. The more "general" peering sites, the more total paths you have to take, in addition to the problem of total peering sessions. Correct. So if you want to establish a peering, you must allocate memory accordingly. CIDR helps this again by reducing the growth in the number of prefixes.
The problem is translating from from BGP to forwarding tables installed in routers.
Sorry, no. There are a number of problems. This isn't one.
Ok, I guess I was confused, I understood that route flapping which causes a cisco router to need to be reloaded can be caused by a flapping session, causing all of the forwarding tables/caches to be flushed, causing the CPU load in a Cisco 7000 to reach 115%? Have I been lied to? Well, let's put it this way. It is impossible to get a cisco 7000 over 100%. There _isn't_ anything there. Yes, route flap is a problem. Yes, a flapping peering can cause forwarding tables and caches much indigestion. Yes, things can go to hell quickly. This is why we've implemented damping.
The other problem is the mesh nature of BGP which makes any BGP peering site, with full mesh peering, an N^2 problem.
This is fixed.
How is this fixed? We've implemented route reflectors, which allow you to partition your IBGP mesh. So it's no longer N^2. With N peers at a peering location you must have each route must have N peering sessions, which looks like N^2 to me. You only need a full mesh if you want information directly. If you can get information from another peer at the interconnect (and note that it does NOT imply that packets take an extra hop), then you don't need a direct peering. Yes, I know that many people aren't willing to do this. That's a political problem, not a technical one. What was supposed to happen at the NAPs was that the Route Server performs a centralized role and it hands routes to anyone.
No one said that you had to buy the memory for your cisco from cisco.
No, but even certified memory from my PC/workstation vendor is cheaper than Cisco certified memory. Granted. If your third party sources are too expensive, well, I guess you should use a PC instead. Seems to me this should be in the noise for any serious operation. Tony
participants (3)
-
Jeremy Porter -
Nicolas Williams -
Tony Li