Getting pretty close to default IPv4 route maximum for 6500/7600 routers.
Hi all, I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark. We are at about 94/95% right now of 512K. For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service. Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does. In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources. Just something to think about before it becomes a story the community talks about for the next decade. -Drew
On 06/05/2014 16:39, Drew Weaver wrote:
In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources.
to fix the problem on sup720/rsp720:
Router(config)#mls cef maximum-routes ip 768
This requires a reload to take effect. If you don't recarve the TCAM and you accidentally hit the maximum number of prefixes, the entire chassis will go into software forwarding mode and will require a reboot to recover. IOW, there is no way of avoiding a reload, so best plan as soon as possible. More info here:
http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-swit...
This problem also affects ASR9000 boxes running typhoon line cards which by default will only set aside 500k slots for ipv4 prefixes. The fix for these boxes is:
0/RSP0/CPU0:router# admin hw-module profile scale l3
...followed by appropriate line card reloads. more information at:
http://www.cisco.com/c/en/us/support/docs/routers/asr-9000-series-aggregatio...
Nick
-----Original Message----- From: Nick Hilliard [mailto:nick@foobar.org] Sent: Tuesday, May 6, 2014 12:11 PM To: Drew Weaver; 'nanog@nanog.org' Subject: Re: Getting pretty close to default IPv4 route maximum for 6500/7600 routers. This problem also affects ASR9000 boxes running typhoon line cards which by default will only set aside 500k slots for ipv4 prefixes. The fix for these boxes is:
I believe you mean "This problem also affects ASR9000 boxes running ...trident... line cards". Please confirm? -Drew
On Tue, 6 May 2014, Drew Weaver wrote:
Hi all,
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.
We are at about 94/95% right now of 512K.
For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service.
Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does.
In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources.
Just something to think about before it becomes a story the community talks about for the next decade.
I've been configuring 6500/Sup7203bxl's with mls cef maximum-routes ip 768 The only gotcha is, you have to reload for that to be effective. Speaking of which, I've had WS-X6708-10GE cards "go bad on reload" in a couple of 6500s. I see cisco finally released some more info on their "bad memory" announcement from several months back: http://www.cisco.com/c/en/us/support/docs/field-notices/637/fn63743.html It seems our "gone bad" 6708s may be included in this issue. If you don't have enough spare ports or spare cards, this puts you in a somewhat precarious situation. You need to reload to affect the v4/v6 route storage change, but you might lose some blades in the process. ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On 5/6/2014 11:39 AM, Drew Weaver wrote:
Hi all,
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.
We are at about 94/95% right now of 512K.
For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service.
Yes, a Sup720/PFC3CXL defaults to 512K IPv4 routes, and reconfiguring the FIB requires a reload. So I've been quietly expecting a somewhat serious meltdown when we hit 512K :) Jeff
It would probably be a good time to upgrade the memory on my 7206 NPE-G1 as well (512MB). I was going to replace the router but am going to keep it around for the Fall Semester. Anyone know of any good 3rd party memory modules that are equivalent to the MEM-NPE-G1-1GB? I got a quote for the official Cisco ones last summer and it was around $5,000 lol On 5/6/2014 11:39 AM, Drew Weaver wrote:
Hi all,
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.
We are at about 94/95% right now of 512K.
For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service.
Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does.
In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources.
Just something to think about before it becomes a story the community talks about for the next decade.
-Drew
Vlad
I just recently got four sets off eBay. Purportedly genuine Cisco. A shade over $100. Raid the departmental beer fund. :) -r Vlade Ristevski <vristevs@ramapo.edu> writes:
It would probably be a good time to upgrade the memory on my 7206 NPE-G1 as well (512MB). I was going to replace the router but am going to keep it around for the Fall Semester. Anyone know of any good 3rd party memory modules that are equivalent to the MEM-NPE-G1-1GB? I got a quote for the official Cisco ones last summer and it was around $5,000 lol
On 5/6/2014 11:39 AM, Drew Weaver wrote:
Hi all,
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.
We are at about 94/95% right now of 512K.
For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service.
Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does.
In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources.
Just something to think about before it becomes a story the community talks about for the next decade.
-Drew
Vlad
I would like to see Cisco send something out... -----Original Message----- From: "Drew Weaver" <drew.weaver@thenap.com> Sent: 5/6/2014 11:42 AM To: "'nanog@nanog.org'" <nanog@nanog.org> Subject: Getting pretty close to default IPv4 route maximum for 6500/7600routers. Hi all, I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark. We are at about 94/95% right now of 512K. For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service. Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does. In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources. Just something to think about before it becomes a story the community talks about for the next decade. -Drew
There is currently a doc for the ASR9k. We're working on getting on for 6500 as well. http://www.cisco.com/c/en/us/support/docs/routers/asr-9000-series-aggregatio... On Tue, May 6, 2014 at 1:34 PM, <bedard.phil@gmail.com> wrote:
I would like to see Cisco send something out...
-----Original Message----- From: "Drew Weaver" <drew.weaver@thenap.com> Sent: 5/6/2014 11:42 AM To: "'nanog@nanog.org'" <nanog@nanog.org> Subject: Getting pretty close to default IPv4 route maximum for 6500/7600routers.
Hi all,
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.
We are at about 94/95% right now of 512K.
For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service.
Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does.
In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources.
Just something to think about before it becomes a story the community talks about for the next decade.
-Drew
The doc on how to adjust the 6500/7600 TCAM space was just published. http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-swit... On Tue, May 6, 2014 at 3:48 PM, Pete Lumbis <alumbis@gmail.com> wrote:
There is currently a doc for the ASR9k. We're working on getting on for 6500 as well.
http://www.cisco.com/c/en/us/support/docs/routers/asr-9000-series-aggregatio...
On Tue, May 6, 2014 at 1:34 PM, <bedard.phil@gmail.com> wrote:
I would like to see Cisco send something out...
-----Original Message----- From: "Drew Weaver" <drew.weaver@thenap.com> Sent: 5/6/2014 11:42 AM To: "'nanog@nanog.org'" <nanog@nanog.org> Subject: Getting pretty close to default IPv4 route maximum for 6500/7600routers.
Hi all,
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.
We are at about 94/95% right now of 512K.
For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service.
Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does.
In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources.
Just something to think about before it becomes a story the community talks about for the next decade.
-Drew
Just had to do this on my router last week. Came in a few mornings ago and we were software switching, yay! On Mon, Jun 9, 2014 at 12:30 PM, Pete Lumbis <alumbis@gmail.com> wrote:
The doc on how to adjust the 6500/7600 TCAM space was just published.
http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-swit...
On Tue, May 6, 2014 at 3:48 PM, Pete Lumbis <alumbis@gmail.com> wrote:
There is currently a doc for the ASR9k. We're working on getting on for 6500 as well.
http://www.cisco.com/c/en/us/support/docs/routers/asr-9000-series-aggregatio...
On Tue, May 6, 2014 at 1:34 PM, <bedard.phil@gmail.com> wrote:
I would like to see Cisco send something out...
-----Original Message----- From: "Drew Weaver" <drew.weaver@thenap.com> Sent: 5/6/2014 11:42 AM To: "'nanog@nanog.org'" <nanog@nanog.org> Subject: Getting pretty close to default IPv4 route maximum for 6500/7600routers.
Hi all,
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to
512K route mark.
We are at about 94/95% right now of 512K.
For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable
the public
service.
Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does.
In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources.
Just something to think about before it becomes a story the community talks about for the next decade.
-Drew
-- eSited LLC (701) 390-9638
Why, in your example, do you bias the split so heavily toward IPv4 that the router won't be able to handle a current full v6 table? I've been using mls cef maximum-routes ip 768 which is probably still a little too liberal for IPv6 FIB TCAM maximum routes : ======================= Current :- ------- IPv4 - 768k MPLS - 16k (default) IPv6 + IP Multicast - 120k (default) given that a full v6 table is around 17k routes today. A more important question though is how many 6500/7600 routers will fully survive the reload required to affect this change? I've lost a blade (presumably to the bad memory issue) each time I've rebooted a 6500 to apply this. On Mon, 9 Jun 2014, Pete Lumbis wrote:
The doc on how to adjust the 6500/7600 TCAM space was just published.
http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-swit...
On Tue, May 6, 2014 at 3:48 PM, Pete Lumbis <alumbis@gmail.com> wrote:
There is currently a doc for the ASR9k. We're working on getting on for 6500 as well.
http://www.cisco.com/c/en/us/support/docs/routers/asr-9000-series-aggregatio...
On Tue, May 6, 2014 at 1:34 PM, <bedard.phil@gmail.com> wrote:
I would like to see Cisco send something out...
-----Original Message----- From: "Drew Weaver" <drew.weaver@thenap.com> Sent: ÿÿ5/ÿÿ6/ÿÿ2014 11:42 AM To: "'nanog@nanog.org'" <nanog@nanog.org> Subject: Getting pretty close to default IPv4 route maximum for 6500/7600routers.
Hi all,
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.
We are at about 94/95% right now of 512K.
For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service.
Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does.
In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources.
Just something to think about before it becomes a story the community talks about for the next decade.
-Drew
---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
The IPv6 table will not be as big as the v4 table even after full acceptance. Given that most providers will be advertising a single /32 and then rest will be some /48 routes for multi-homed scenarios. My router looks like this FIB TCAM maximum routes : ======================= Current :- ------- IPv4 - 600k MPLS - 32k IPv6 - 160k IP multicast - 32k Probably a little heavy on MPLS considering we dont use it. With the current level of exhaustion I dont think IPv4 will make it past 600k. We are currently seeing 520,000 routes. There are currently 107M IPs left globally. If those all went to /21's that would require 26,255 prefixes. If those all went to /22's that would require 52,510 prefixes. If those all went to /24's that would require 105,021 prefixes. So even the most conservative maximum should be no more than 626K On Mon, Jun 9, 2014 at 1:09 PM, Jon Lewis <jlewis@lewis.org> wrote:
Why, in your example, do you bias the split so heavily toward IPv4 that the router won't be able to handle a current full v6 table? I've been using
mls cef maximum-routes ip 768
which is probably still a little too liberal for IPv6
FIB TCAM maximum routes : ======================= Current :- ------- IPv4 - 768k MPLS - 16k (default) IPv6 + IP Multicast - 120k (default)
given that a full v6 table is around 17k routes today.
A more important question though is how many 6500/7600 routers will fully survive the reload required to affect this change? I've lost a blade (presumably to the bad memory issue) each time I've rebooted a 6500 to apply this.
On Mon, 9 Jun 2014, Pete Lumbis wrote:
The doc on how to adjust the 6500/7600 TCAM space was just published.
http://www.cisco.com/c/en/us/support/docs/switches/ catalyst-6500-series-switches/117712-problemsolution-cat6500-00.html
On Tue, May 6, 2014 at 3:48 PM, Pete Lumbis <alumbis@gmail.com> wrote:
There is currently a doc for the ASR9k. We're working on getting on for
6500 as well.
http://www.cisco.com/c/en/us/support/docs/routers/asr-9000- series-aggregation-services-routers/116999-problem-line-card-00.html
On Tue, May 6, 2014 at 1:34 PM, <bedard.phil@gmail.com> wrote:
I would like to see Cisco send something out...
-----Original Message----- From: "Drew Weaver" <drew.weaver@thenap.com> Sent: яя5/яя6/яя2014 11:42 AM To: "'nanog@nanog.org'" <nanog@nanog.org> Subject: Getting pretty close to default IPv4 route maximum for 6500/7600routers.
Hi all,
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.
We are at about 94/95% right now of 512K.
For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service.
Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does.
In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources.
Just something to think about before it becomes a story the community talks about for the next decade.
-Drew
---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
-- eSited LLC (701) 390-9638
It is generally much better to do the following: mls cef maximum-routes ipv6 90 mls cef maximum-routes ip-multicast 1 This will leave v4 and mpls in one big pool, puts v6 to something useful for quite a while and steals all of the multicast space which is not really used on most deployments. This gives us the following (which is pretty great for IP backbone purposes in dual stack): #show mls cef maximum-routes FIB TCAM maximum routes : ======================= Current :- ------- IPv4 + MPLS - 832k (default) IPv6 - 90k IP multicast - 1k John -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Jon Lewis Sent: Monday, June 09, 2014 12:10 PM To: Pete Lumbis Cc: nanog@nanog.org Subject: Re: Getting pretty close to default IPv4 route maximum for 6500/7600routers. Why, in your example, do you bias the split so heavily toward IPv4 that the router won't be able to handle a current full v6 table? I've been using mls cef maximum-routes ip 768 which is probably still a little too liberal for IPv6 FIB TCAM maximum routes : ======================= Current :- ------- IPv4 - 768k MPLS - 16k (default) IPv6 + IP Multicast - 120k (default) given that a full v6 table is around 17k routes today. A more important question though is how many 6500/7600 routers will fully survive the reload required to affect this change? I've lost a blade (presumably to the bad memory issue) each time I've rebooted a 6500 to apply this. On Mon, 9 Jun 2014, Pete Lumbis wrote:
The doc on how to adjust the 6500/7600 TCAM space was just published.
http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-serie s-switches/117712-problemsolution-cat6500-00.html
On Tue, May 6, 2014 at 3:48 PM, Pete Lumbis <alumbis@gmail.com> wrote:
There is currently a doc for the ASR9k. We're working on getting on for 6500 as well.
http://www.cisco.com/c/en/us/support/docs/routers/asr-9000-series-agg regation-services-routers/116999-problem-line-card-00.html
On Tue, May 6, 2014 at 1:34 PM, <bedard.phil@gmail.com> wrote:
I would like to see Cisco send something out...
-----Original Message----- From: "Drew Weaver" <drew.weaver@thenap.com> Sent: ÿÿ5/ÿÿ6/ÿÿ2014 11:42 AM To: "'nanog@nanog.org'" <nanog@nanog.org> Subject: Getting pretty close to default IPv4 route maximum for 6500/7600routers.
Hi all,
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.
We are at about 94/95% right now of 512K.
For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service.
Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does.
In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources.
Just something to think about before it becomes a story the community talks about for the next decade.
-Drew
---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
John, great point! Regardless, shouldn't need more than 626K to make it to v6 and we wont need as many for v6. That was one of the problems that v6 was designed to address. On Mon, Jun 9, 2014 at 1:27 PM, John van Oppen <jvanoppen@spectrumnet.us> wrote:
It is generally much better to do the following:
mls cef maximum-routes ipv6 90 mls cef maximum-routes ip-multicast 1
This will leave v4 and mpls in one big pool, puts v6 to something useful for quite a while and steals all of the multicast space which is not really used on most deployments.
This gives us the following (which is pretty great for IP backbone purposes in dual stack):
#show mls cef maximum-routes FIB TCAM maximum routes : ======================= Current :- ------- IPv4 + MPLS - 832k (default) IPv6 - 90k IP multicast - 1k
John
-----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Jon Lewis Sent: Monday, June 09, 2014 12:10 PM To: Pete Lumbis Cc: nanog@nanog.org Subject: Re: Getting pretty close to default IPv4 route maximum for 6500/7600routers.
Why, in your example, do you bias the split so heavily toward IPv4 that the router won't be able to handle a current full v6 table? I've been using
mls cef maximum-routes ip 768
which is probably still a little too liberal for IPv6
FIB TCAM maximum routes : ======================= Current :- ------- IPv4 - 768k MPLS - 16k (default) IPv6 + IP Multicast - 120k (default)
given that a full v6 table is around 17k routes today.
A more important question though is how many 6500/7600 routers will fully survive the reload required to affect this change? I've lost a blade (presumably to the bad memory issue) each time I've rebooted a 6500 to apply this.
On Mon, 9 Jun 2014, Pete Lumbis wrote:
The doc on how to adjust the 6500/7600 TCAM space was just published.
http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-serie s-switches/117712-problemsolution-cat6500-00.html
On Tue, May 6, 2014 at 3:48 PM, Pete Lumbis <alumbis@gmail.com> wrote:
There is currently a doc for the ASR9k. We're working on getting on for 6500 as well.
http://www.cisco.com/c/en/us/support/docs/routers/asr-9000-series-agg regation-services-routers/116999-problem-line-card-00.html
On Tue, May 6, 2014 at 1:34 PM, <bedard.phil@gmail.com> wrote:
I would like to see Cisco send something out...
-----Original Message----- From: "Drew Weaver" <drew.weaver@thenap.com> Sent: ÿÿ5/ÿÿ6/ÿÿ2014 11:42 AM To: "'nanog@nanog.org'" <nanog@nanog.org> Subject: Getting pretty close to default IPv4 route maximum for 6500/7600routers.
Hi all,
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.
We are at about 94/95% right now of 512K.
For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service.
Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does.
In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources.
Just something to think about before it becomes a story the community talks about for the next decade.
-Drew
---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
-- eSited LLC (701) 390-9638
Yep, exactly… the problem is the carving suggested by most kills the fact that MPLS and v4 are pooled, which on a larger network is very nice, especially if using 6PE where each v6 route may need an MPLS route too. From: Bryan Tong [mailto:contact@nullivex.com] Sent: Monday, June 09, 2014 12:37 PM To: John van Oppen Cc: nanog@nanog.org Subject: Re: FW: Getting pretty close to default IPv4 route maximum for 6500/7600routers. John, great point! Regardless, shouldn't need more than 626K to make it to v6 and we wont need as many for v6. That was one of the problems that v6 was designed to address. On Mon, Jun 9, 2014 at 1:27 PM, John van Oppen <jvanoppen@spectrumnet.us<mailto:jvanoppen@spectrumnet.us>> wrote: It is generally much better to do the following: mls cef maximum-routes ipv6 90 mls cef maximum-routes ip-multicast 1 This will leave v4 and mpls in one big pool, puts v6 to something useful for quite a while and steals all of the multicast space which is not really used on most deployments. This gives us the following (which is pretty great for IP backbone purposes in dual stack): #show mls cef maximum-routes FIB TCAM maximum routes : ======================= Current :- ------- IPv4 + MPLS - 832k (default) IPv6 - 90k IP multicast - 1k John -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org<mailto:nanog-bounces@nanog.org>] On Behalf Of Jon Lewis Sent: Monday, June 09, 2014 12:10 PM To: Pete Lumbis Cc: nanog@nanog.org<mailto:nanog@nanog.org> Subject: Re: Getting pretty close to default IPv4 route maximum for 6500/7600routers. Why, in your example, do you bias the split so heavily toward IPv4 that the router won't be able to handle a current full v6 table? I've been using mls cef maximum-routes ip 768 which is probably still a little too liberal for IPv6 FIB TCAM maximum routes : ======================= Current :- ------- IPv4 - 768k MPLS - 16k (default) IPv6 + IP Multicast - 120k (default) given that a full v6 table is around 17k routes today. A more important question though is how many 6500/7600 routers will fully survive the reload required to affect this change? I've lost a blade (presumably to the bad memory issue) each time I've rebooted a 6500 to apply this. On Mon, 9 Jun 2014, Pete Lumbis wrote:
The doc on how to adjust the 6500/7600 TCAM space was just published.
http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-serie s-switches/117712-problemsolution-cat6500-00.html
On Tue, May 6, 2014 at 3:48 PM, Pete Lumbis <alumbis@gmail.com<mailto:alumbis@gmail.com>> wrote:
There is currently a doc for the ASR9k. We're working on getting on for 6500 as well.
http://www.cisco.com/c/en/us/support/docs/routers/asr-9000-series-agg regation-services-routers/116999-problem-line-card-00.html
On Tue, May 6, 2014 at 1:34 PM, <bedard.phil@gmail.com<mailto:bedard.phil@gmail.com>> wrote:
I would like to see Cisco send something out...
-----Original Message----- From: "Drew Weaver" <drew.weaver@thenap.com<mailto:drew.weaver@thenap.com>> Sent: ÿÿ5/ÿÿ6/ÿÿ2014 11:42 AM To: "'nanog@nanog.org<mailto:nanog@nanog.org>'" <nanog@nanog.org<mailto:nanog@nanog.org>> Subject: Getting pretty close to default IPv4 route maximum for 6500/7600routers.
Hi all,
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.
We are at about 94/95% right now of 512K.
For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service.
Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does.
In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources.
Just something to think about before it becomes a story the community talks about for the next decade.
-Drew
---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ -- eSited LLC (701) 390-9638
I botched those numbers. Let me fix. According to this countdown: http://inetcore.com/project/ipv4ec/index_en.html we have 6.41 /8's left. So that is 107,541,955 IPs. CIDR - Prefixes ----------------- /20 - 26,255.36 /21 - 52,510.72 /22 - 105,021.44 /23 - 210,042.88 /24 - 420,085.76 My apologies for my erroneous math, I was off one number making the table. Johns solution to combine MPLS and IPv4 is the best solution. I would implement it if I hadn't already rebooted a few days ago.
Even if the first numbers were correctly calculated, they don't allow for further deaggregation of already advertised prefixes, which shouldn't be underestimated as the commercial value of each address increases... On 10.06.2014 06:17, Bryan Tong wrote:
I botched those numbers.
Let me fix.
According to this countdown: http://inetcore.com/project/ipv4ec/index_en.html we have 6.41 /8's left. So that is 107,541,955 IPs.
CIDR - Prefixes ----------------- /20 - 26,255.36 /21 - 52,510.72 /22 - 105,021.44 /23 - 210,042.88 /24 - 420,085.76
My apologies for my erroneous math, I was off one number making the table.
Johns solution to combine MPLS and IPv4 is the best solution. I would implement it if I hadn't already rebooted a few days ago.
On 10 Jun 2014, at 05:48 , Andrew Jones <aj@jonesy.com.au> wrote:
Even if the first numbers were correctly calculated, they don’t allow for further deaggregation of already advertised prefixes, which shouldn't be underestimated as the commercial value of each address increases...
IPv4 addresses have little commercial value anymore and IPv6 is basically free. The only people who still haven’t realised don’t have enough money to spend on IPv4 to keep themselves alive for another decade. — Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983
On (2014-06-10 09:41 +0000), Bjoern A. Zeeb wrote:
IPv4 addresses have little commercial value anymore and IPv6 is basically free. The only people who still haven’t realised don’t have enough money to spend on IPv4 to keep themselves alive for another decade.
Wishing how markets should be and how markets are in this instance are divergent. -- ++ytti
On 10 Jun 2014, at 10:10 , Saku Ytti <saku@ytti.fi> wrote:
On (2014-06-10 09:41 +0000), Bjoern A. Zeeb wrote:
IPv4 addresses have little commercial value anymore and IPv6 is basically free. The only people who still haven’t realised don’t have enough money to spend on IPv4 to keep themselves alive for another decade.
Wishing how markets should be and how markets are in this instance are divergent.
You mean it’s more likely people acquire/merge with other companies for IP space then go through transfer? https://www.arin.net/knowledge/statistics/ — Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983
On (2014-06-10 10:28 +0000), Bjoern A. Zeeb wrote:
You mean it’s more likely people acquire/merge with other companies for IP space then go through transfer? https://www.arin.net/knowledge/statistics/
I mean that demand for IPv4 addresses will continue to foreseeable future, if you are offering content, and there is difference between reach for IPv6 and IPv4+IPv6, you're going to want to acquire some IPv4 addresses to avoid giving your competition an unfair advantage. How will this reflect to price, you imply price of IPv4 address is going to decrease, I expect it's not reached peak yet, due to still having RIR availability which sets natural limit to price. -- ++ytti
On 10/06/14 12:28, Bjoern A. Zeeb wrote:
You mean it’s more likely people acquire/merge with other companies for IP space then go through transfer? https://www.arin.net/knowledge/statistics/
You have to consider that most likely there will be an increase in de-aggregation due to: - The RIRs allocating smaller and smaller address space to every member, even smaller than a /24; - The RIRs allocating from smaller, fragmented allocations received from IANA. This means that it would also be possible - in theory - for example, to receive an allocation for a /22, sparse in two, non-contiguous, /23s (or a similar situation). - The holders of large parts of address space monetizing via transfers, thus fragmenting even more. Ciao! -- Max
On the RSP720-10GE at least, it seems that IPv4 and MPLS are not shared. Am I correct or am I missing something? FIB TCAM maximum routes : ======================= Current :- ------- IPv4 - 768k MPLS - 64k IPv6 + IP Multicast - 96k (default) Randy On 6/9/14, 3:27 PM, "John van Oppen" <jvanoppen@spectrumnet.us> wrote:
It is generally much better to do the following:
mls cef maximum-routes ipv6 90 mls cef maximum-routes ip-multicast 1
This will leave v4 and mpls in one big pool, puts v6 to something useful for quite a while and steals all of the multicast space which is not really used on most deployments.
This gives us the following (which is pretty great for IP backbone purposes in dual stack):
#show mls cef maximum-routes FIB TCAM maximum routes : ======================= Current :- ------- IPv4 + MPLS - 832k (default) IPv6 - 90k IP multicast - 1k
John
-----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Jon Lewis Sent: Monday, June 09, 2014 12:10 PM To: Pete Lumbis Cc: nanog@nanog.org Subject: Re: Getting pretty close to default IPv4 route maximum for 6500/7600routers.
Why, in your example, do you bias the split so heavily toward IPv4 that the router won't be able to handle a current full v6 table? I've been using
mls cef maximum-routes ip 768
which is probably still a little too liberal for IPv6
FIB TCAM maximum routes : ======================= Current :- ------- IPv4 - 768k MPLS - 16k (default) IPv6 + IP Multicast - 120k (default)
given that a full v6 table is around 17k routes today.
A more important question though is how many 6500/7600 routers will fully survive the reload required to affect this change? I've lost a blade (presumably to the bad memory issue) each time I've rebooted a 6500 to apply this.
On Mon, 9 Jun 2014, Pete Lumbis wrote:
The doc on how to adjust the 6500/7600 TCAM space was just published.
http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-serie s-switches/117712-problemsolution-cat6500-00.html
On Tue, May 6, 2014 at 3:48 PM, Pete Lumbis <alumbis@gmail.com> wrote:
There is currently a doc for the ASR9k. We're working on getting on for 6500 as well.
http://www.cisco.com/c/en/us/support/docs/routers/asr-9000-series-agg regation-services-routers/116999-problem-line-card-00.html
On Tue, May 6, 2014 at 1:34 PM, <bedard.phil@gmail.com> wrote:
I would like to see Cisco send something out...
-----Original Message----- From: "Drew Weaver" <drew.weaver@thenap.com> Sent: ÿÿ5/ÿÿ6/ÿÿ2014 11:42 AM To: "'nanog@nanog.org'" <nanog@nanog.org> Subject: Getting pretty close to default IPv4 route maximum for 6500/7600routers.
Hi all,
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.
We are at about 94/95% right now of 512K.
For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service.
Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does.
In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources.
Just something to think about before it becomes a story the community talks about for the next decade.
-Drew
---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On the sup 720 they become unshared if you carve v4 away from the default separately, that is why I carve the other two instead.
Ah. I had to ³no mls cef max ip² and ³no mls def max mpls² for it to share. They were previously adjusted separately. :) Thanks. On 6/10/14, 3:12 AM, "John van Oppen" <jvanoppen@spectrumnet.us> wrote:
On the sup 720 they become unshared if you carve v4 away from the default separately, that is why I carve the other two instead.
On Mon, 9 Jun 2014, John van Oppen wrote:
It is generally much better to do the following:
mls cef maximum-routes ipv6 90 mls cef maximum-routes ip-multicast 1
This will leave v4 and mpls in one big pool, puts v6 to something useful for quite a while and steals all of the multicast space which is not really used on most deployments.
This gives us the following (which is pretty great for IP backbone purposes in dual stack):
#show mls cef maximum-routes FIB TCAM maximum routes : ======================= Current :- ------- IPv4 + MPLS - 832k (default) IPv6 - 90k IP multicast - 1k
I was just looking at / thinking about this again, and though I don't disagree that doing the split your way is probably better, I think it's a moot point. I strongly suspect these boxes will run out of RAM before they're able to utilize another 256k routing slots with multiple full v4 tables. ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On 13/06/2014 15:54, Jon Lewis wrote:
I was just looking at / thinking about this again, and though I don't disagree that doing the split your way is probably better, I think it's a moot point. I strongly suspect these boxes will run out of RAM before they're able to utilize another 256k routing slots with multiple full v4 tables.
to a certain extent that depends on what software you're using. 12.x seems to be a good bit more memory efficient than 15.x on the sup720. Otherwise yeah, RP memory is the next critical limiting factor on these boxes, assuming if you can live with the crippling convergence times with large numbers of prefixes. Nick
as many people will be hitting the wall on all sorts of platforms, perhaps it's wiki time. or have i just missed it? randy
On 05/06/2014 05:39 PM, Drew Weaver wrote:
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.
Thanks for this e-mail with clear subject ;) Did anyone yet calculated roughly when the ipv4 routing table will hit 512K ?
On Tue, May 06, 2014 at 03:39:13PM +0000, Drew Weaver wrote:
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.
Closer to? Internap announces 507K prefixes to me today. Coupled with the prefixes I carry in iBGP internally, I've been sitting at 511K for quite some time, and at occasions, exceeded 512K in the last 2 weeks. L3 Forwarding Resources FIB TCAM usage: Total Used %Used 72 bits (IPv4, MPLS, EoM) 802816 511848 64% IP routing table maximum-paths is 32 Route Source Networks Subnets Overhead Memory (bytes) connected 0 31 3012 4464 static 1 78 17456 11376 ospf 1 1 310 22392 44784 Intra-area: 110 Inter-area: 160 External-1: 0 External-2: 41 NSSA External-1: 0 NSSA External-2: 0 bgp 23456 167707 343507 36808128 75466680 External: 507479 Internal: 3735 Local: 0 internal 6054 13246152 Total 173763 343926 36850988 88773456 -- Brandon Ewing (nicotine@warningg.com)
I haven't seen anyone bring up this point yet, but I feel like I'm missing something... I receive a full BGP table from several providers. They send me ~490k *prefixes* each. However, my router shows ~332k *subnets* in the routing table. As I understand it, the BGP table contains duplicate information (for example a supernet is announced as well as all subnets within that supernet) or excess information (prefix is announced as two /17's instead of a single /16) and can otherwise be summarized to save space in the RIB. It appears to me that the weekly CIDR report shows similar numbers: Recent Table History Date Prefixes CIDR Agg 30-05-14 502889 283047 31-05-14 502961 283069 01-06-14 502836 283134 02-06-14 502943 283080 03-06-14 502793 283382 04-06-14 503177 282897 05-06-14 503436 283062 06-06-14 503988 282999 In this case, does the 512k limit of the 6500/7600 refer to the RIB or the FIB? And does it even matter since the BGP prefix table can automatically be reduced to ~300k routes? Thanks, --Blake Drew Weaver wrote the following on 5/6/2014 10:39 AM:
Hi all,
I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.
We are at about 94/95% right now of 512K.
For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service.
Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does.
In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources.
Just something to think about before it becomes a story the community talks about for the next decade.
-Drew
Hi Blake, On 10 Jun 2014, at 19:04, Blake Hudson <blake@ispn.net> wrote:
In this case, does the 512k limit of the 6500/7600 refer to the RIB or the FIB? And does it even matter since the BGP prefix table can automatically be reduced to ~300k routes?
Te 512k limit refers to FIB in the B/C (base) versions of 6500/7600 Supervisors and DFCs (for line cards). BXL/CXL versions have FIB for 1M IPv4 prefixes. You can find more information here: http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-swit... And yes, you’re right - no matter how many neighbors you have, the FIB will only contain best paths, so it will be closer to 500k entries in total rather than N times number of neighbours. -- "There's no sense in being precise when | Łukasz Bromirski you don't know what you're talking | jid:lbromirski@jabber.org about." John von Neumann | http://lukasz.bromirski.net
On 6/10/14, 10:15 AM, Łukasz Bromirski wrote:
Hi Blake,
On 10 Jun 2014, at 19:04, Blake Hudson <blake@ispn.net> wrote:
In this case, does the 512k limit of the 6500/7600 refer to the RIB or the FIB? And does it even matter since the BGP prefix table can automatically be reduced to ~300k routes?
Te 512k limit refers to FIB in the B/C (base) versions of 6500/7600 Supervisors and DFCs (for line cards). BXL/CXL versions have FIB for 1M IPv4 prefixes.
You can find more information here:
http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-swit...
And yes, you’re right - no matter how many neighbors you have, the FIB will only contain best paths, so it will be closer to 500k entries in total rather than N times number of neighbours.
Until you add multi-as multipath and addpath and a couple VRFs and all of sudden FIB budget blows up.
Łukasz Bromirski wrote the following on 6/10/2014 12:15 PM:
Hi Blake,
On 10 Jun 2014, at 19:04, Blake Hudson <blake@ispn.net> wrote:
In this case, does the 512k limit of the 6500/7600 refer to the RIB or the FIB? And does it even matter since the BGP prefix table can automatically be reduced to ~300k routes? Te 512k limit refers to FIB in the B/C (base) versions of 6500/7600 Supervisors and DFCs (for line cards). BXL/CXL versions have FIB for 1M IPv4 prefixes.
You can find more information here:
http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-swit...
And yes, you’re right - no matter how many neighbors you have, the FIB will only contain best paths, so it will be closer to 500k entries in total rather than N times number of neighbours.
Please correct me if I'm wrong, but if the BGP table contains ~500k prefixes, which are then summarized into ~300k routes (RIB), and the FIB contains only the "best path" entries from the RIB, wouldn't the FIB be at or below 300k? --Blake
On 10 Jun 2014, at 19:39, Blake Hudson <blake@ispn.net> wrote:
And yes, you’re right - no matter how many neighbors you have, the FIB will only contain best paths, so it will be closer to 500k entries in total rather than N times number of neighbours. Please correct me if I'm wrong, but if the BGP table contains ~500k prefixes, which are then summarized into ~300k routes (RIB), and the FIB contains only the "best path" entries from the RIB, wouldn't the FIB be at or below 300k?
Because you need to do your own summarization or ask your upstreams to do it for you. Until then, most of transit accepts loosely prefixes in exact length but also longer (i.e. /24 but also both /25s). You’ll see more and more deaggregation with the rise of smaller entities fighting for chance to do some traffic engineering, so be prepared to constant rise of prefixes overall. -- "There's no sense in being precise when | Łukasz Bromirski you don't know what you're talking | jid:lbromirski@jabber.org about." John von Neumann | http://lukasz.bromirski.net
On Tuesday, June 10, 2014 08:07:35 PM Łukasz Bromirski wrote:
Because you need to do your own summarization or ask your upstreams to do it for you. Until then, most of transit accepts loosely prefixes in exact length but also longer (i.e. /24 but also both /25s).
A couple of major service providers today permit announcements of any length, provided they are registered in an IRR that they use to build filters. Of course, there are no guarantees that prefixes typically longer than general industry practice permits will see the light of day beyond their network. Mark.
On 6/10/14, 10:39 AM, Blake Hudson wrote:
Łukasz Bromirski wrote the following on 6/10/2014 12:15 PM:
Hi Blake,
On 10 Jun 2014, at 19:04, Blake Hudson <blake@ispn.net> wrote:
In this case, does the 512k limit of the 6500/7600 refer to the RIB or the FIB? And does it even matter since the BGP prefix table can automatically be reduced to ~300k routes? Te 512k limit refers to FIB in the B/C (base) versions of 6500/7600 Supervisors and DFCs (for line cards). BXL/CXL versions have FIB for 1M IPv4 prefixes.
You can find more information here:
http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-swit...
And yes, you’re right - no matter how many neighbors you have, the FIB will only contain best paths, so it will be closer to 500k entries in total rather than N times number of neighbours.
Please correct me if I'm wrong, but if the BGP table contains ~500k prefixes, which are then summarized into ~300k routes (RIB),
Unlikely, just because prefixes could be cidr aggregated doesn't mean they are. the more specifics exist for a reason, in the case of deaggrates with no covering anouncement, well not much you're doing with those. your rib should be the sum of all received routes that you did not filter.
and the FIB contains only the "best path" entries from the RIB, wouldn't the FIB be at or below 300k?
a live example of rib size from a router with two transit providers. bird> show route count 979842 of 979842 routes for 490932 networks a live example of rib size from a router with one ibgp peer with addpath and three upstream transit providers bird> show route count 1471242 of 1471242 routes for 491977 networks
--Blake
joel jaeggli wrote the following on 6/10/2014 1:10 PM:
Łukasz Bromirski wrote the following on 6/10/2014 12:15 PM:
Hi Blake,
On 10 Jun 2014, at 19:04, Blake Hudson <blake@ispn.net> wrote:
In this case, does the 512k limit of the 6500/7600 refer to the RIB or the FIB? And does it even matter since the BGP prefix table can automatically be reduced to ~300k routes? Te 512k limit refers to FIB in the B/C (base) versions of 6500/7600 Supervisors and DFCs (for line cards). BXL/CXL versions have FIB for 1M IPv4 prefixes.
You can find more information here:
http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-swit...
And yes, you’re right - no matter how many neighbors you have, the FIB will only contain best paths, so it will be closer to 500k entries in total rather than N times number of neighbours.
Please correct me if I'm wrong, but if the BGP table contains ~500k prefixes, which are then summarized into ~300k routes (RIB), Unlikely, just because prefixes could be cidr aggregated doesn't mean
On 6/10/14, 10:39 AM, Blake Hudson wrote: they are. the more specifics exist for a reason, in the case of deaggrates with no covering anouncement, well not much you're doing with those.
your rib should be the sum of all received routes that you did not filter. On the couple Cisco platforms I have available with full tables, Cisco summarizes BGP by default. Since this thread is talking about Cisco gear, I think it's more topical than results from BIRD.
One example from a non-transit AS: ASR#sh ip route sum IP routing table name is default (0x0) IP routing table maximum-paths is 32 Route Source Networks Subnets Replicates Overhead Memory (bytes) connected 0 10 0 600 1800 static 1 2 0 180 540 application 0 0 0 0 0 bgp xxxxx 164817 330796 0 29736780 89210340 External: 495613 Internal: 0 Local: 0 internal 5799 20123680 Total 170617 330808 0 29737560 109336360
and the FIB contains only the "best path" entries from the RIB, wouldn't the FIB be at or below 300k? a live example of rib size from a router with two transit providers.
bird> show route count 979842 of 979842 routes for 490932 networks
a live example of rib size from a router with one ibgp peer with addpath and three upstream transit providers
bird> show route count 1471242 of 1471242 routes for 491977 networks
The RIB counts and memory used by the RIB seem to be nearly identical between a Cisco router with 3 full BGP feeds and another with 1 BGP feed. The differences seem to lie in the memory used by BGP for prefix tracking. If your router has multiple routing tables, or your installing multiple routes for the same prefix in the RIB, then I can understand why your RIB will be larger. These are nice features, but certainly not a requirement for everyone. --Blake
On Tue, Jun 10, 2014 at 11:36 AM, Blake Hudson <blake@ispn.net> wrote:
joel jaeggli wrote the following on 6/10/2014 1:10 PM:
On 6/10/14, 10:39 AM, Blake Hudson wrote:
Łukasz Bromirski wrote the following on 6/10/2014 12:15 PM:
Hi Blake,
On 10 Jun 2014, at 19:04, Blake Hudson <blake@ispn.net> wrote:
In this case, does the 512k limit of the 6500/7600 refer to the RIB
or the FIB? And does it even matter since the BGP prefix table can automatically be reduced to ~300k routes?
Te 512k limit refers to FIB in the B/C (base) versions of 6500/7600 Supervisors and DFCs (for line cards). BXL/CXL versions have FIB for 1M IPv4 prefixes.
You can find more information here:
http://www.cisco.com/c/en/us/support/docs/switches/ catalyst-6500-series-switches/117712-problemsolution-cat6500-00.html
And yes, you’re right - no matter how many neighbors you have, the FIB will only contain best paths, so it will be closer to 500k entries in total rather than N times number of neighbours.
Please correct me if I'm wrong, but if the BGP table contains ~500k prefixes, which are then summarized into ~300k routes (RIB),
Unlikely, just because prefixes could be cidr aggregated doesn't mean they are. the more specifics exist for a reason, in the case of deaggrates with no covering anouncement, well not much you're doing with those.
your rib should be the sum of all received routes that you did not filter.
On the couple Cisco platforms I have available with full tables, Cisco summarizes BGP by default. Since this thread is talking about Cisco gear, I think it's more topical than results from BIRD.
One example from a non-transit AS: ASR#sh ip route sum IP routing table name is default (0x0)
IP routing table maximum-paths is 32 Route Source Networks Subnets Replicates Overhead Memory (bytes) connected 0 10 0 600 1800 static 1 2 0 180 540 application 0 0 0 0 0 bgp xxxxx 164817 330796 0 29736780 89210340 External: 495613 Internal: 0 Local: 0 internal 5799 20123680 Total 170617 330808 0 29737560 109336360
I'm not sure you're reading that correctly. 164817+330796 = 495613 That is, the BGP routing table size is the union of the "Networks" and the "Subnets"; it's not magically doing any summarization for you. Matt
Matthew Petach wrote the following on 6/10/2014 7:03 PM:
On the couple Cisco platforms I have available with full tables, Cisco summarizes BGP by default. Since this thread is talking about Cisco gear, I think it's more topical than results from BIRD.
One example from a non-transit AS: ASR#sh ip route sum IP routing table name is default (0x0)
IP routing table maximum-paths is 32 Route Source Networks Subnets Replicates Overhead Memory (bytes) connected 0 10 0 600 1800 static 1 2 0 180 540 application 0 0 0 0 0 bgp xxxxx 164817 330796 0 29736780 89210340 External: 495613 Internal: 0 Local: 0 internal 5799 20123680 Total 170617 330808 0 29737560 109336360
I'm not sure you're reading that correctly. 164817+330796 = 495613
That is, the BGP routing table size is the union of the "Networks" and the "Subnets"; it's not magically doing any summarization for you.
Matt
Thank you Matt for directly addressing my question. My interpretation, which seems likely incorrect, was that smaller announcements could be discarded if there was a covering prefix (that otherwise matched the same AS path and other BGP metrics) and that many smaller prefix announcements could be bundled (again, assuming that all BGP metrics were the same between the prefixes). The numbers I was seeing in my routers for subnets coincided closely with the cidr-report's summzarization numbers http://www.cidr-report.org/as2.0/aggr.html, and I assumed the two used the same logic (not magic) to calculate how to reduce routes without losing any routing functionality. Your explanation that I was simply interpreting the numbers incorrectly seems the most logical now that I look again. Thanks, --Blake
On (2014-06-10 12:39 -0500), Blake Hudson wrote:
Please correct me if I'm wrong, but if the BGP table contains ~500k prefixes, which are then summarized into ~300k routes (RIB), and the FIB contains only the "best path" entries from the RIB, wouldn't the FIB be at or below 300k?
There is nothing to summarize away from global BGP table, if you have number showing less, it's probably counter bug or misinterpretation. Global BGP table, single BGP feed, will take same amount of RIB and FIB. You can see your FIB use in 'show plat hard capa pfc': #show platform hardware capacity pfc Module FIB TCAM usage: Total Used %Used 2 72 bits (IPv4, MPLS, EoM) 884736 712819 81% 144 bits (IP mcast, IPv6) 81920 19235 23% You're probably bit better off than I am :) -- ++ytti
On Wed, Jun 11, 2014 at 1:28 AM, Saku Ytti <saku@ytti.fi> wrote:
On (2014-06-10 12:39 -0500), Blake Hudson wrote: There is nothing to summarize away from global BGP table, if you have number showing less, it's probably counter bug or misinterpretation. Global BGP table, single BGP feed, will take same amount of RIB and FIB. [snip]
That depends.... if by "summarize" they mean filter prefixes longer than say /22, or otherwise...: discard extraneous prefixes from networks that were allocated a /16 network but chose to deaggregate and advertise every /24 ------ choosing to accept only the /16 advertisement instead of installing these extra /24 routes in the FIB, then there are plenty of entries to "summarize" away. -- -JH
Hello, On 10.6.2014 19:04, Blake Hudson wrote:
I haven't seen anyone bring up this point yet, but I feel like I'm missing something... I receive a full BGP table from several providers. They send me ~490k *prefixes* each. However, my router shows ~332k *subnets* in the routing table. As I understand it, the BGP table contains duplicate information (for example a supernet is announced as well as all subnets within that supernet) or excess information (prefix is announced as two /17's instead of a single /16) and can otherwise be summarized to save space in the RIB.
many people deaggregate their address space purposely, including large networks like Google, Akamai, Netflix etc. Full list for analysis like "who does that" is available at http://www.cidr-report.org/as2.0/aggr.html These days also some people split their allocated aggregatable space (PA) with different routing policies for each subnet, substituting old PI addresses (at least in RIPE region). Technically nothing blocks this and politically - it's up to each, what accepts. But some unreachable subnet means problems with customers... There's no summarization in hardware/software from RIB to FIB. From the vendor perspective, they would like to sell you new hardware with larger TCAMs/etc, of course... :-) With regards, Daniel
participants (25)
-
Andrew Jones -
bedard.phil＠gmail.com -
Bjoern A. Zeeb -
Blake Hudson -
Brandon Ewing -
Bryan Tong -
chiel -
Daniel Suchy -
Drew Weaver -
Jeff Kell -
Jimmy Hess -
joel jaeggli -
John van Oppen -
Jon Lewis -
Mark Tinka -
Massimiliano Stucchi -
Matthew Petach -
Nick Hilliard -
Pete Lumbis -
Randy Bush -
Randy Epstein -
Rob Seastrom -
Saku Ytti -
Vlade Ristevski -
Łukasz Bromirski