"2M today, 10M with no change in technology"? An informal survey.
Hi, In another mailing list, someone has asserted that "noone believes router vendors who say [they can support 2M routes today and 10M with no change in technology]". Or perhaps more accurately, the router vendors claiming this are being a bit disingenuous in that while it is possible routers can handle this many static routes, they'll quickly fall down if they were subjected to real world dynamic conditions ISPs would see if you extrapolate routing flux in today's tables up to (say) 2M routes. My questions: Do you believe router vendors who state they today have "capacities on the order of 2 million ipv4 routes and they have no reason to expect that they couldn't deliver 10 million route FIB products in a few years given sufficient demand."? If you do not (or you believe the router vendors are being disingenuous) and routing system growth continues: - where do you believe existing routing technology will fall down? - what steps will you take/are you taking to limit your vulnerability? Feel free to respond privately if you don't feel comfortable discussing this in a public forum. I promise to hold any responses confidential, publishing only a summary of responses. Thanks, -drc
On 8/25/07, David Conrad <david.conrad@icann.org> wrote:
In another mailing list, someone has asserted that "noone believes router vendors who say [they can support 2M routes today and 10M with no change in technology]".
Do you believe router vendors who state they today have "capacities on the order of 2 million ipv4 routes and they have no reason to expect that they couldn't deliver 10 million route FIB products in a few years given sufficient demand."?
David, NNTP is similar to BGP in that every message must spread to every node. Usenet scaled up beyond what anyone thought it could. Sort of. Its not exactly fast and enough messages are lost that someone had to go invent "par2".
- where do you believe existing routing technology will fall down?
I guess you could say that I think BGP has an NNTP future. It never quite breaks completely, it just gets worse and worse at doing its job.
- what steps will you take/are you taking to limit your vulnerability?
As a multihomed endpoint network, I can sacrifice some reliability by introducing a default route and filtering longer prefixes if I really have to. I hope the folks upstream have a better answer. Regards, Bill Herrin -- William D. Herrin herrin@dirtside.com bill@herrin.us 3005 Crane Dr. Web: <http://bill.herrin.us/> Falls Church, VA 22042-3004
On Sat, 25 Aug 2007 20:44:45 -0400 "William Herrin" <herrin-nanog@dirtside.com> wrote:
On 8/25/07, David Conrad <david.conrad@icann.org> wrote:
In another mailing list, someone has asserted that "noone believes router vendors who say [they can support 2M routes today and 10M with no change in technology]".
Do you believe router vendors who state they today have "capacities on the order of 2 million ipv4 routes and they have no reason to expect that they couldn't deliver 10 million route FIB products in a few years given sufficient demand."?
David,
NNTP is similar to BGP in that every message must spread to every node. Usenet scaled up beyond what anyone thought it could. Sort of. Its not exactly fast and enough messages are lost that someone had to go invent "par2".
Netnews was originally designed for 300 bps dial-up modems with O(1) hubs. Fortunately, the technology evolved to meet the load. Will BGP evolve that way? Netnews didn't demand anything more in common than a file format, and the only major change in it was within 2-3 years after it was invented. BGP doesn't have that property. --Steve Bellovin, http://www.cs.columbia.edu/~smb
On Sat, Aug 25, 2007 at 08:44:45PM -0400, William Herrin wrote:
NNTP is similar to BGP in that every message must spread to every node. Usenet scaled up beyond what anyone thought it could. Sort of. Its not exactly fast and enough messages are lost that someone had to go invent "par2".
I think the context of (the other) David's question was wether or not there need to be any changes in technology. In that context, I don't think NNTP is a good analogy to prove the point that no changes in technology are necessary. NNTP acheived its ends in large part due to a protocol update for 'streaming' feeds - the CHECK and TAKETHIS commands to de-synchronize sender and receiver (supplanting 'IHAVE' and 'SENDME') allowed servers to fill the socket buffer and make full use of TCP large-window and selective-ACK. I do not think I overstate the importance of this change to call it an 'NNTP rewrite'; it literally reversed NNTP's core design. There was at least one company that sold commercial NNTP software - and provided a catalyst that caused most other software to reflect upon itself and redesign core processes. Virtually all software changed significantly (and there's some debate wether it was for the better). But the biggest part of NNTP's survival, I think, were the behind the scenes news mega hubs - expensive machines with a lot of memory bandwidth, solid state disks, and fat network connections, taking and giving feeds to anyone who would ask. Some (most I think) were operated at a loss - purely to support the network. -- Ash bugud-gul durbatuluk agh burzum-ishi krimpatul. -- David W. Hankins "If you don't do it right the first time, Software Engineer you'll just have to do it again." Internet Systems Consortium, Inc. -- Jack T. Hankins
On 8/25/07 4:30 PM, "David Conrad" <david.conrad@icann.org> wrote:
- where do you believe existing routing technology will fall down?
Well, to get specific, I think that it will be interesting to see what happens when the size of the route table exceeds the stock TCAM on the Cisco Catalyst platform. Before I got to my current employer Cisco sold then 7604s with Sup32s (I hope they weren't more expensive than 6504 chassis because all they did was change the paint). I'm going to hope that Cisco comes out with a Sup upgrade that includes the larger TCAM of the 3BXL without the switch fabric mojo - that's stuff's expensive. The whole thing really makes me wonder about the value of selling the Cat platform as a customer edge router... -- John A. Kilpatrick john@hypergeek.net Email| http://www.hypergeek.net/ john-page@hypergeek.net Text pages| ICQ: 19147504 remember: no obstacles/only challenges
On Sun, Aug 26, 2007, John A. Kilpatrick wrote:
without the switch fabric mojo - that's stuff's expensive. The whole thing really makes me wonder about the value of selling the Cat platform as a customer edge router...
Its a great sale; they suddenly have hard limits which "the internet exceeds", forcing the hardware upgrade cycle. Remember how long the Cisco 75xx persisted and note how many people are still running Cisco 720x's with NPE-225's or NPE-400's w/ full tables simply by adding RAM. Adrian
Adrian Chadd writes [on Cisco's TCAM-based 7600/Cat6500 routers]:
Its a great sale; they suddenly have hard limits which "the internet exceeds", forcing the hardware upgrade cycle. Remember how long the Cisco 75xx persisted and note how many people are still running Cisco 720x's with NPE-225's or NPE-400's w/ full tables simply by adding RAM.
"Simply adding RAM" may not be that easy/cheap, especially when you have to upgrade it on many linecards (VIP2s anyone?). On distributed platforms with hardware forwarding in the linecards (GSR) this is/was probably even worse, you have these "hard limits" in the linecards. Replacing centralized switching engines from time to time doesn't seem such a bad value proposition compared to replacing/memory-upgrading line cards. -- Simon. (Who doesn't care much because we're running at ~30'000 IPv4 routes.)
On Tue, Aug 28, 2007, Simon Leinen wrote:
Adrian Chadd writes [on Cisco's TCAM-based 7600/Cat6500 routers]:
Its a great sale; they suddenly have hard limits which "the internet exceeds", forcing the hardware upgrade cycle. Remember how long the Cisco 75xx persisted and note how many people are still running Cisco 720x's with NPE-225's or NPE-400's w/ full tables simply by adding RAM.
"Simply adding RAM" may not be that easy/cheap, especially when you have to upgrade it on many linecards (VIP2s anyone?). On distributed platforms with hardware forwarding in the linecards (GSR) this is/was probably even worse, you have these "hard limits" in the linecards.
Yes, but people -are- still acquiring VIP2-80's and such, maxing them out with RAM, and deploying them in the network. You might not see it in the US as much but, if c-nsp is anything to go by, they're quite popular in "internet developing" nations. People are "simply adding RAM" to older routers to squeeze the last few cents. Then you get people that'll quite happily throw on BGP filters to drop down the table/FIB size and use a default to get to the rest. Or people doing similar tricks on Cisco 3550 L3 switches. In any case, I was primarily referring to the staying power of the non-distributed Cisco forwarding platforms. Adrian
In another mailing list, someone has asserted that "noone believes router vendors who say [they can support 2M routes today and 10M with
no change in technology]".
Can you tell us who said this on which email list?
Or perhaps more accurately, the router vendors claiming this are being a bit disingenuous in that while it is possible routers can handle this many static routes, they'll quickly fall down if they were subjected to real world dynamic conditions ISPs would see if you extrapolate routing flux in today's tables up to (say) 2M routes.
I remember quite clearly at a much earlier meeting the statement that "obviously the Internet will keel over dead if the top level IP routing table reaches 10,000 entries" (this is not an exact quote, but the "10,000" limit was what was stated at the time). At the time no one challenged this observation, although at the time I did wonder why someone thought that this was obviously true. Of course this statement has been obviously false for quite a few years. Sadly, I don't recall whether this statement was made at a very early IETF, or at a GADS (gateway algorithms and data structures) meeting, which was the group that preceded the IETF ("gateway" was an early word for "router"). Of course, over the years there have been improvements in both implementations and protocols to make the growth from "less than 10,000" to "more than 200,000" work okay. I don't think that anyone is expecting that today's routers will work with 10,000,000 top level Internet routes without any change at all in any aspect of the implementation. Ross
Ross, On Aug 26, 2007, at 8:32 PM, Ross Callon wrote:
In another mailing list, someone has asserted that "noone believes router vendors who say [they can support 2M routes today and 10M with no change in technology]". Can you tell us who said this on which email list?
The discussion was occurring on the ARIN PPML list.
I remember quite clearly at a much earlier meeting the statement that "obviously the Internet will keel over dead if the top level IP routing table reaches 10,000 entries" (this is not an exact quote, but the "10,000" limit was what was stated at the time). At the time no one challenged this observation, although at the time I did wonder why someone thought that this was obviously true. Of course this statement has been obviously false for quite a few years.
Yes, and I remember when quite a few folks were doing "ISP code release of the day" to ISPs whose routers were having "issues" because of the routing load (ah, the good old days :-)).
Of course, over the years there have been improvements in both implementations and protocols to make the growth from "less than 10,000" to "more than 200,000" work okay. I don't think that anyone is expecting that today's routers will work with 10,000,000 top level Internet routes without any change at all in any aspect of the implementation.
Taken from http://www.nanog.org/mtg-0702/presentations/bof-report.pdf: "In the case of Cisco that means delivering switch routers with a capacity of about a million routes now. In the case of Foundry they are projecting that with some FIB aggregation techniques that switches capable of 512k fib entries will still be usable by 2014. Juniper is delivering new products (m120 mx960) with DRAM rather than TCAM/SRAM based FIB's with capacities on the order of 2 million ipv4 routes and they have no reason to expect that they couldn't deliver 10 million route FIB products in a few years given sufficient demand." The question I am asking is whether or not folks in the operational community believe these statements are accurate or realistic in the face of real world Internet dynamics. I know some people do not. I'm trying to get a feel from the wider community as I am in no position to judge. Regards, -drc
Ross,
In another mailing list, someone has asserted that "noone believes router vendors who say [they can support 2M routes today and 10M with no change in technology]".
Can you tell us who said this on which email list?
In terms of the original statement, we were also given this presentation during the routing working group of the most recent RIPE meeting: http://www.ripe.net/ripe/meetings/ripe-54/presentations/Router_Scaling_Trend... See slides nine and ten. Regards, Rob
dire predictions of the MSFC2 "topping out" have occured on this list. anyone want to SWAG the number of MSFC2's are in use today? It might be nice to also extrapolate that number as a percentage of total routing engines... but thats an even larger SWAG. if the number is roughly the same as the number of AGS+ boxen in use, i'm less worried --bill
participants (10)
-
Adrian Chadd -
bmanning＠vacation.karoshi.com -
David Conrad -
David W. Hankins -
John A. Kilpatrick -
Rob Evans -
Ross Callon -
Simon Leinen -
Steven M. Bellovin -
William Herrin