Recent Table History Date Prefixes CIDR Agg 04-02-05 151613 103143 05-02-05 152142 103736 06-02-05 152231 103721 07-02-05 152353 103830 08-02-05 152514 103966 09-02-05 153855 104090 10-02-05 154283 104246 11-02-05 154341 104240
<...> ~ +3000 routes in one week? Anyone else frightened by this? Florian
~ +3000 routes in one week? Anyone else frightened by this?
Only people who have stock in vendors welcome it. Be prepared for another huge glut of unnecessary outages, hardware and memory upgrades soon folks! FYI - at $job [AS8220] we have just completed a program to fix the aggregation problem that we had. It took about 10 weeks to complete and I will present a paper on this at LINX 48 next week. Regards, Neil.
On Fri, 11 Feb 2005, Neil J. McRae wrote:
~ +3000 routes in one week? Anyone else frightened by this?
Only people who have stock in vendors welcome it. Be prepared for another huge glut of unnecessary outages, hardware and memory upgrades soon folks!
Actually, from a quick look at the following: http://www.cidr-report.org/as4637/aggrweek.html http://www.cidr-report.org/as4637/as-announce.txt http://www.cidr-report.org/as4637/as-wdl.txt either I'm misreading the data, or the script that generates the email is broken and giving wrong numbers of total routes. FWIW, my I'm not seeing any more than 152843 routes in the global table...but I reject anything longer than /24. According to the weekly agg in the first link, there are 151999 routes in the table today and were 151889 on 11Feb05, which is the date the emailed report goes up to. According to the weekly add/withdraw data, 1732 routes were added and 1964 routes were withdrawn, a decrease of 232 routes in the week, and looking at the aggrweek.html page gives a net decrease of 232 routes for the week (12Feb05 - 05Feb05). Time Warner apparently had some sort of deaggregation accident where they went from 962 nets on 07Feb05 to 2238 routes 08Feb05, 2602 routes 09Feb05, and finally back down to 1031 on 11Feb05. Anyone know what happened there? ---------------------------------------------------------------------- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On Fri, 11 Feb 2005, Frotzler, Florian wrote:
Recent Table History Date Prefixes CIDR Agg 04-02-05 151613 103143 05-02-05 152142 103736 06-02-05 152231 103721 07-02-05 152353 103830 08-02-05 152514 103966 09-02-05 153855 104090 10-02-05 154283 104246 11-02-05 154341 104240 <...>
~ +3000 routes in one week? Anyone else frightened by this?
Florian
any thoughts on how to fix it? my peers keep sending these to me and i'll even admit my customers do too. telling people its bad doesnt appear to have an effect, at the small end networks seem to collect /24s and announce them freely, at the large end i'm still without an explanation as to why large networks require so many prefixes - none of them seem to comment? if people arent self policing it seems the only other way is for the larger transit providers to stop accepting prefixes and telling their customers to fix their s**t. and i dont see them doing this. Steve
any thoughts on how to fix it? my peers keep sending these to me and i'll even admit my customers do too. telling people its bad doesnt appear to have an effect, at the small end networks seem to collect /24s and announce them freely, at the large end i'm still without an explanation as to why large networks require so many prefixes - none of them seem to comment?
if people arent self policing it seems the only other way is for the larger transit providers to stop accepting prefixes and telling their customers to fix their s**t. and i dont see them doing this.
proxy aggregation if they don't fix it. Neil.
On Fri, 11 Feb 2005, Stephen J. Wilcox wrote:
On Fri, 11 Feb 2005, Frotzler, Florian wrote:
Recent Table History Date Prefixes CIDR Agg 04-02-05 151613 103143 05-02-05 152142 103736 06-02-05 152231 103721 07-02-05 152353 103830 08-02-05 152514 103966 09-02-05 153855 104090 10-02-05 154283 104246 11-02-05 154341 104240 <...>
~ +3000 routes in one week? Anyone else frightened by this?
Florian
any thoughts on how to fix it? my peers keep sending these to me and i'll even admit my customers do too. telling people its bad doesnt appear to have an effect, at the small end networks seem to collect /24s and announce them freely, at the large end i'm still without an explanation as to why large networks require so many prefixes - none of them seem to comment?
if people arent self policing it seems the only other way is for the larger transit providers to stop accepting prefixes and telling their customers to fix their s**t. and i dont see them doing this.
It seems to me they get paid to carry prefixes by their customers. And their peers listen to the prefixes because they make money by using those prefixes. So, to the extent you make money listening to them, use the routes. And if they start to cause you problems you will have to take corrective action to stablize your network, as was done a long time ago (internet time): http://www.merit.edu/mail.archives/nanog/1995-09/msg00047.html (link grabbed at random from the archives, I'm sure there are better posts that actually list the full old school sprint filters.) However, if you are the one filtering and all your competitors figure out how to handle 154,000 routes then you will be at a competitive disadvantage. Coincidentally, the largest networks also spend the most with their vendors and get to tell the vendors what they want in the next generation of boxes they buy. Mike. +----------------- H U R R I C A N E - E L E C T R I C -----------------+ | Mike Leber Direct Internet Connections Voice 510 580 4100 | | Hurricane Electric Web Hosting Colocation Fax 510 580 4151 | | mleber@he.net http://www.he.net | +-----------------------------------------------------------------------+
On Fri, 11 Feb 2005, Mike Leber wrote:
On Fri, 11 Feb 2005, Stephen J. Wilcox wrote:
On Fri, 11 Feb 2005, Frotzler, Florian wrote:
Recent Table History Date Prefixes CIDR Agg 04-02-05 151613 103143 05-02-05 152142 103736 06-02-05 152231 103721 07-02-05 152353 103830 08-02-05 152514 103966 09-02-05 153855 104090 10-02-05 154283 104246 11-02-05 154341 104240 <...>
~ +3000 routes in one week? Anyone else frightened by this?
Florian
any thoughts on how to fix it? my peers keep sending these to me and i'll even admit my customers do too. telling people its bad doesnt appear to have an effect, at the small end networks seem to collect /24s and announce them freely, at the large end i'm still without an explanation as to why large networks require so many prefixes - none of them seem to comment?
if people arent self policing it seems the only other way is for the larger transit providers to stop accepting prefixes and telling their customers to fix their s**t. and i dont see them doing this.
It seems to me they get paid to carry prefixes by their customers.
the payment would be the same if it was a /19 or 32x/24 announced at source
And their peers listen to the prefixes because they make money by using those prefixes.
So, to the extent you make money listening to them, use the routes.
so the problem is noone wants to be the first to jump as it costs money? so whats the suggestion for how to not be first? ie is it possible for a small group of large operators to agree a consensus? you dont even have to actively filter to start this, if a script were run to advise customers daily when they were announcing routes incompliant to the transits 'routing policy' it would have some effect. one thing i've found from some of my customers is they're actually ignorant to the problems they cause, they think its cool to announce 10 prefixes and can be educated otherwise. Steve
And if they start to cause you problems you will have to take corrective action to stablize your network, as was done a long time ago (internet time):
http://www.merit.edu/mail.archives/nanog/1995-09/msg00047.html
(link grabbed at random from the archives, I'm sure there are better posts that actually list the full old school sprint filters.)
However, if you are the one filtering and all your competitors figure out how to handle 154,000 routes then you will be at a competitive disadvantage.
Coincidentally, the largest networks also spend the most with their vendors and get to tell the vendors what they want in the next generation of boxes they buy.
Mike.
+----------------- H U R R I C A N E - E L E C T R I C -----------------+ | Mike Leber Direct Internet Connections Voice 510 580 4100 | | Hurricane Electric Web Hosting Colocation Fax 510 580 4151 | | mleber@he.net http://www.he.net | +-----------------------------------------------------------------------+
Mike,
It seems to me they get paid to carry prefixes by their customers.
And their peers listen to the prefixes because they make money by using those prefixes.
I'm sure this type of statement helps drug dealers to sleep at night! :-) If the top 100 AS's de-aggregated and increased the routes they announce by 6000 each would we be so content?
However, if you are the one filtering and all your competitors figure out how to handle 154,000 routes then you will be at a competitive disadvantage.
But its not just 154,000 routes though is it?! If we all start doing this at a much increased rate [as we've seen in recent times] then it will be more like 1,540,000 routes. When we cleaned the issue from 8220 we found that a lot of the issues were config issues and ancient history transient workarounds for problems that were not fixed after the event. The issue we see is bad aggregation - the root cause is bad practise and processes that manifest into bad aggregation. I would argue that networks with poor aggregation are also networks that will tend to have more routeing issues and other outages although I have no data to back that claim up.
Coincidentally, the largest networks also spend the most with their vendors and get to tell the vendors what they want in the next generation of boxes they buy.
And look how well that's worked out not notably on this issue. Regards, Neil.
Neil J. McRae said the following on 12/02/2005 21:06:
The issue we see is bad aggregation - the root cause is bad practise and processes that manifest into bad aggregation. I would argue that networks with poor aggregation are also networks that will tend to have more routeing issues and other outages although I have no data to back that claim up.
I think it depends on which part of the world you look in. But I've visited enough ISPs in the last 7 years in my part of the world (outside US and Western Europe) to know that this is an accurate statement. Again, no data to back this experience up. Neil J. McRae said the following on 12/02/2005 21:07:
Commercial reasons? The traffic goes to the 32x/24 instead of the /19.
If that's the reason why the table is growing so much then we are all in deep deep trouble.
Quite often many service providers are de-aggregating without knowing it. They receive their /20 or whatever from the RIR, but they consider this to be 16 Class Cs - I'm not joking - and announce them as such to the Internet. I spend a lot of time getting these folks to announce aggregates, but it is hard work convincing people that this will even work. Even if the RIR recommends that they announce their address block, they still consider it as Class Cs - even Class Bs for some big allocations. :( Solution is education, but the industry is such in many parts of the world that education is not desired but considered a negative reflection on people's abilities, whether they have the abilities or not. And the lack of educators - there are several of us on the worldwide NOG trail but it's very clear we are not enough. Nor do the NOGs cover the ISP industry, but just those who are interested in participating. We are not enough due to lack of time, lack of supportive employers, more focus on making profit in these leaner times, etc... Where next I don't know... philip --
Hi Philip, On Sat, 12 Feb 2005, Philip Smith wrote:
Quite often many service providers are de-aggregating without knowing it. They receive their /20 or whatever from the RIR, but they consider this to be 16 Class Cs - I'm not joking - and announce them as such to the Internet. I spend a lot of time getting these folks to announce aggregates, but it is hard work convincing people that this will even work. Even if the RIR recommends that they announce their address block, they still consider it as Class Cs - even Class Bs for some big allocations. :(
this is getting into what i was implying earlier.. you have wider experience than me - would you agree that most of the poor deaggregating is not intentional ie that they're announcing their '16 class Cs' or historically had 2 /21s and dont even realise they could fix it.. that applies to medium and large providers too reading this list - how often do they actually check what prefixes they are sourcing, from my recent work at a couple of european IXes i had a number of folks email me offlist as they hadnt realised til I sent out an email they had deaggregation and once it was pointed out they just fixed it. so to repeat my earlier suggestion - if transit providers, particularly the larger ones setup scripts to notify their customers daily/weeks of routing deaggregation do you think we might gain some traction in educating and fixing this? Steve
On Sat, 12 Feb 2005, Stephen J. Wilcox wrote:
this is getting into what i was implying earlier.. you have wider experience than me - would you agree that most of the poor deaggregating is not intentional ie that they're announcing their '16 class Cs' or historically had 2 /21s and dont even realise they could fix it.. that applies to medium and large providers too reading this list - how often do they actually check what prefixes they are sourcing, from my recent work at a couple of european IXes i had a number of folks email me offlist as they hadnt realised til I sent out an email they had deaggregation and once it was pointed out they just fixed it.
so to repeat my earlier suggestion - if transit providers, particularly the larger ones setup scripts to notify their customers daily/weeks of routing deaggregation do you think we might gain some traction in educating and fixing this?
Why does it have to be transit providers (customer relationship, so it's not spam? :) Anyone could look at the global table (or just take the CIDR Report data) and automate emailing the ASN contacts for each ASN a list of what they're announcing and a list of the routes they probably should be announcing instead. I've personally dealt with a customer not too long ago who when we turned them up was announcing 2 /20s, a /21, a /22, and several /23s and /24s all deaggregated as /24s. Sprint and Qwest (their other upstreams at the time) apparently had no problem with this. I saw what they were doing and asked them why. "That's how our router consultant set it up." There was no technical reason for it. I helped them reaggregate their BGP announcements. I'll bet there are at least hundreds of similar AS's that just need to be prodded (or maybe even some hand holding or config help) in order to clean up their announcements. ---------------------------------------------------------------------- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On Sat, 12 Feb 2005, Jon Lewis wrote:
I've personally dealt with a customer not too long ago who when we turned them up was announcing 2 /20s, a /21, a /22, and several /23s and /24s all deaggregated as /24s. Sprint and Qwest (their other upstreams at the time) apparently had no problem with this. I saw what they were doing and asked them why. "That's how our router consultant set it up." There was no technical reason for it. I helped them reaggregate their BGP announcements.
I'll bet there are at least hundreds of similar AS's that just need to be prodded (or maybe even some hand holding or config help) in order to clean up their announcements.
It would appear that ARIN education is somehow lacking in regards to their paying membership. -Hank [see my previous post]
---------------------------------------------------------------------- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On Sat, 12 February 2005 14:58:42 +0000, Stephen J. Wilcox wrote:
From: "Stephen J. Wilcox" <steve@telecomplete.co.uk> [...] - would you agree that most of the poor deaggregating is not intentional ie that they're announcing their '16 class Cs' or historically had 2 /21s and
Think about someone putting in a Null0 route and re- exporting stuff unconditionally, now after he originates his /19 he is then adding a /24 here, and a /25 there. Lack of experience, when you suggest to them they should remove these announcements they are afraid to change it, not understanding the implications, etc. Not to mention ppl using cisco and prefix lists, it is way too easy with cisco to say '/19 le 24', and then they use outbound prefix lists to their transit supplier (different, but related as I see it). Some transit ISPs use that a lot, and encourage the table growth. Also I think a further problem is (from experience) some content hosters wanting to bleed (right word?) /24's to their providers, even though their ratio is 10:1 or more and inbound traffic is not exactly relevant. Too often it makes no sense, and I am speaking of the '32* /24 is a /19' version, mind you, sometimes not even announcing the supernet... Seeing the larger DSL prefixes in Europe then in Europe what do we conclude? See for 3352 or 3269 (sorry)... But when we try to measure (de-) aggregation by continent it gets tricky... (and I believe I know winner and loser) I am not sure doing it the Swisscom way (they filter a lot) is the way to go, yet I would be curious how many routes they currently carry for a full route set. Ah, here it is: -> route-views.oregon-ix.net>sh ip bg su | incl 3303 164.128.32.11 4 3303 3351176 140593 74037481 0 0 2w2d 69713 <- I have a hard time argueing this topic with customers, and I have the feeling I am not the only one. We are past that already, I feel.
so to repeat my earlier suggestion - if transit providers, particularly the larger ones setup scripts to notify their customers daily/weeks of routing deaggregation do you think we might gain some traction in educating and fixing this?
That may be a way to go actually. Like the mail for what prefixes are lacking a route: object that one might just send to customers, not to peers... (5430 peers surely remember) Alexander
Alexander Koch wrote:
I am not sure doing it the Swisscom way (they filter a lot) is the way to go, yet I would be curious how many routes they currently carry for a full route set. Ah, here it is:
-> route-views.oregon-ix.net>sh ip bg su | incl 3303 164.128.32.11 4 3303 3351176 140593 74037481 0 0 2w2d 69713 <-
Since you mentioned it: http://www.ip-plus.net/technical/route_filtering_policy.en.html Additionally you might want to see the slides of André Chapuis' presentation held at SwiNOG #7: http://www.swinog.ch/meetings/swinog7/BGP_filtering-swinog.ppt Pro's and con's, of course. But I guess Swisscom is still living with 128 Meg ;-) Fredy Kuenzler Init Seven AG / AS13030
At 02:52 PM 2/12/2005, Fredy Kuenzler wrote:
Alexander Koch wrote:
I am not sure doing it the Swisscom way (they filter a lot) is the way to go, yet I would be curious how many routes they currently carry for a full route set. Ah, here it is: -> route-views.oregon-ix.net>sh ip bg su | incl 3303 164.128.32.11 4 3303 3351176 140593 74037481 0 0 2w2d 69713 <-
Since you mentioned it: http://www.ip-plus.net/technical/route_filtering_policy.en.html
Additionally you might want to see the slides of André Chapuis' presentation held at SwiNOG #7: http://www.swinog.ch/meetings/swinog7/BGP_filtering-swinog.ppt
Pro's and con's, of course. But I guess Swisscom is still living with 128 Meg ;-)
If that list is current, they're also living without connectivity to many networks on the Internet (entire /8's missing). ;) Vinny Abello Network Engineer Server Management vinny@tellurian.com (973)300-9211 x 125 (973)940-6125 (Direct) PGP Key Fingerprint: 3BC5 9A48 FC78 03D3 82E0 E935 5325 FBCB 0100 977A Tellurian Networks - The Ultimate Internet Connection http://www.tellurian.com (888)TELLURIAN There are 10 kinds of people in the world. Those who understand binary and those that don't.
On Sat, 12 Feb 2005, Alexander Koch wrote:
On Sat, 12 February 2005 14:58:42 +0000, Stephen J. Wilcox wrote:
From: "Stephen J. Wilcox" <steve@telecomplete.co.uk> [...] - would you agree that most of the poor deaggregating is not intentional ie that they're announcing their '16 class Cs' or historically had 2 /21s and
Think about someone putting in a Null0 route and re- exporting stuff unconditionally, now after he originates his /19 he is then adding a /24 here, and a /25 there. Lack of experience, when you suggest to them they should remove these announcements they are afraid to change it, not understanding the implications, etc.
Not to mention ppl using cisco and prefix lists, it is way too easy with cisco to say '/19 le 24', and then they use outbound prefix lists to their transit supplier (different, but related as I see it). Some transit ISPs use that a lot, and encourage the table growth.
There are some business reasons to de-aggregate. Look at some outages caused by 'routing problems' (someone leaked my /24's to their peers, peers, peer and my traffic got blackholed, because the public net only knows me as a /20) There are multiple reasons for deaggregation aside from 'dumb operator', some are even 'valid' if you look at them from the protection standpoint. -Chris
On Sun, 13 February 2005 07:31:16 +0000, Christopher L. Morrow wrote: [..]
There are some business reasons to de-aggregate. Look at some outages caused by 'routing problems' (someone leaked my /24's to their peers, peers, peer and my traffic got blackholed, because the public net only knows me as a /20)
I am surprised you bring such an argument up. While we can surely agree on this happening on the net, I have yet to hear from someone saying this is happening more than once a month or so. Maybe Todd from Renesys has other examples besides the Yahoo incident.^
There are multiple reasons for deaggregation aside from 'dumb operator', some are even 'valid' if you look at them from the protection standpoint.
I won't argue that, but how many ISPs are using this line of argument? I have not heard anyone yet telling me this, not in years. And surely you can get notification for any more specific prefixes in your networks from Renesys and likely others already, if you do not write a script yourself that has a look on routeviews or other similar services and notifies you. Also things like conditional announcements and no-export are often not known, and if you have two POPs and lack the 'backbone' capacity between these there are still other ways than just announce more specifics all around the place. Alexander
On Sun, 13 Feb 2005, Alexander Koch wrote:
On Sun, 13 February 2005 07:31:16 +0000, Christopher L. Morrow wrote: [..]
There are some business reasons to de-aggregate. Look at some outages caused by 'routing problems' (someone leaked my /24's to their peers, peers, peer and my traffic got blackholed, because the public net only knows me as a /20)
I am surprised you bring such an argument up. While we can surely agree on this happening on the net, I have yet to hear from someone saying this is happening more than once a month or so. Maybe Todd from Renesys has other examples besides the Yahoo incident.^
if it happens once to you and lasts long enough... I'm not condoning it, nor saying it's even a valid reason to do it, just pointing out that it does happen :(
There are multiple reasons for deaggregation aside from 'dumb operator', some are even 'valid' if you look at them from the protection standpoint.
I won't argue that, but how many ISPs are using this line of argument? I have not heard anyone yet telling me this, not in years.
a few have... recently in fact.
I have recently heard companies saying their reasoning for de-aggregation was 1) to protect against outages to their customer base when a more specific of their aggregate was announced somewhere else and 2) if they are getting DDOS attacked on a given /24 they can just drop that advertisement and only affect part of their customer base. As technically savvy folks, we may not agree with this line of reasoning. However, keep in mind that the technically savvy folks are not always the ones making the decisions within a company. Just because someone has enable access and clue does not mean they have the authority to make certain decisions. Most of those people probably spend a large amount of their time arguing with the decision makers to try and do the right thing but at some point they lose those arguments. -- Justin Ryburn justin@ryburn.org -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Christopher L. Morrow Sent: Sunday, February 13, 2005 2:30 AM To: Alexander Koch Cc: nanog@merit.edu Subject: Re: The Cidr Report On Sun, 13 Feb 2005, Alexander Koch wrote:
On Sun, 13 February 2005 07:31:16 +0000, Christopher L. Morrow wrote: [..]
There are some business reasons to de-aggregate. Look at some outages caused by 'routing problems' (someone leaked my /24's to their peers, peers, peer and my traffic got blackholed, because the public net only knows me as a /20)
I am surprised you bring such an argument up. While we can surely agree on this happening on the net, I have yet to hear from someone saying this is happening more than once a month or so. Maybe Todd from
Renesys has other examples besides the Yahoo incident.^
if it happens once to you and lasts long enough... I'm not condoning it, nor saying it's even a valid reason to do it, just pointing out that it does happen :(
There are multiple reasons for deaggregation aside from 'dumb operator', some are even 'valid' if you look at them from the protection standpoint.
I won't argue that, but how many ISPs are using this line of argument? I have not heard anyone yet telling me this, not in years.
a few have... recently in fact.
On Sun, 13 Feb 2005, Justin Ryburn wrote:
I have recently heard companies saying their reasoning for de-aggregation was 1) to protect against outages to their customer base when a more specific of their aggregate was announced somewhere else and 2) if they are getting DDOS attacked on a given /24 they can just drop that advertisement and only affect part of their customer base.
1) this only provides partial protection, even if you announce a /24 i can still announce my own /24 and get some of your traffic 2) either they are operating networks that cant support their business and i dont see why we should bale them out or in the cases where certain hosts are accepted by us as targets (ircnets etc) you could argue to obtain a discrete /24 which is the better evil than taking a /16 and breaking it down to take out a /24 i'm not keen on this latter idea, what if i operate an anti-ddos specialist isp, hosting ircnets, gambling, security sites etc - do i put each host in a /24 and waste a whole /16 with a couple hundred customers? i strongly believe if you want to be an autonomous internet provider then you should be able to run your network by accepted means not thro cheap hacks
As technically savvy folks, we may not agree with this line of reasoning. However, keep in mind that the technically savvy folks are not always the ones making the decisions within a company. Just because someone has enable access and clue does not mean they have the authority to make certain decisions. Most of those people probably spend a large amount of their time arguing with the decision makers to try and do the right thing but at some point they lose those arguments.
if their suppliers/peers disagree strongly they would not be able to present these options in the first place.. lack of regulation has its downsides it would seem.. Steve
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 13, 2005, at 2:31 AM, Christopher L. Morrow wrote:
On Sat, 12 Feb 2005, Alexander Koch wrote:
On Sat, 12 February 2005 14:58:42 +0000, Stephen J. Wilcox wrote:
From: "Stephen J. Wilcox" <steve@telecomplete.co.uk> [...] - would you agree that most of the poor deaggregating is not intentional ie that they're announcing their '16 class Cs' or historically had 2 /21s and
Think about someone putting in a Null0 route and re- exporting stuff unconditionally, now after he originates his /19 he is then adding a /24 here, and a /25 there. Lack of experience, when you suggest to them they should remove these announcements they are afraid to change it, not understanding the implications, etc.
Not to mention ppl using cisco and prefix lists, it is way too easy with cisco to say '/19 le 24', and then they use outbound prefix lists to their transit supplier (different, but related as I see it). Some transit ISPs use that a lot, and encourage the table growth.
There are some business reasons to de-aggregate. Look at some outages caused by 'routing problems' (someone leaked my /24's to their peers, peers, peer and my traffic got blackholed, because the public net only knows me as a /20)
There are multiple reasons for deaggregation aside from 'dumb operator', some are even 'valid' if you look at them from the protection standpoint.
-Chris
That and the "I have 1 circuit to $good_provider and 1 circuit to $bad_provider and the only way I can make them balance is to split my space in half and announce more specifics out through each provider" argument. I have also often seen people do this without announcing the aggregate because <some undefined bad thing> will happen, usually justified with much hand-waving. The people who do this can usually not be reasoned with.... It happens all the time... Warren.
- -- "He who laughs last, thinks slowest." -- Anonymous -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFCEMBhHSkNr4ucEScRArsVAKD98l4rpQLmPh6PBuCqvaYHFWYPhwCg1+Ua KP85z1snGejdGB+D7klo+U8= =Mz3a -----END PGP SIGNATURE-----
On Mon, 14 Feb 2005, Warren Kumari, Ph.D, CCIE# 9190 wrote:
On Feb 13, 2005, at 2:31 AM, Christopher L. Morrow wrote:
There are multiple reasons for deaggregation aside from 'dumb operator', some are even 'valid' if you look at them from the protection standpoint.
That and the "I have 1 circuit to $good_provider and 1 circuit to $bad_provider and the only way I can make them balance is to split my space in half and announce more specifics out through each provider" argument. I have also often seen people do this without announcing the aggregate because <some undefined bad thing> will happen, usually justified with much hand-waving. The people who do this can usually not be reasoned with....
this just reinforces the argument that they are lacking in technical savvy. i have a transit provider who i dont want to carry much traffic and i dont want to prepend my announcements.. by looking at that providers supported customer communities i just get them to prepend as they export to other major networks thus moving the main volume of the traffic to the desired ingress paths no deaggregation, no prepending.. Steve
From: "Warren Kumari, Ph.D, CCIE# 9190" <warren@kumari.net> Date: Mon, 14 Feb 2005 10:14:38 -0500 To: <nanog@merit.edu> Subject: Re: The Cidr Report
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Feb 13, 2005, at 2:31 AM, Christopher L. Morrow wrote:
On Sat, 12 Feb 2005, Alexander Koch wrote:
On Sat, 12 February 2005 14:58:42 +0000, Stephen J. Wilcox wrote:
From: "Stephen J. Wilcox" <steve@telecomplete.co.uk> [...] - would you agree that most of the poor deaggregating is not intentional ie that they're announcing their '16 class Cs' or historically had 2 /21s and
Think about someone putting in a Null0 route and re- exporting stuff unconditionally, now after he originates his /19 he is then adding a /24 here, and a /25 there. Lack of experience, when you suggest to them they should remove these announcements they are afraid to change it, not understanding the implications, etc.
Not to mention ppl using cisco and prefix lists, it is way too easy with cisco to say '/19 le 24', and then they use outbound prefix lists to their transit supplier (different, but related as I see it). Some transit ISPs use that a lot, and encourage the table growth.
There are some business reasons to de-aggregate. Look at some outages caused by 'routing problems' (someone leaked my /24's to their peers, peers, peer and my traffic got blackholed, because the public net only knows me as a /20)
There are multiple reasons for deaggregation aside from 'dumb operator', some are even 'valid' if you look at them from the protection standpoint.
-Chris
That and the "I have 1 circuit to $good_provider and 1 circuit to $bad_provider and the only way I can make them balance is to split my space in half and announce more specifics out through each provider" argument. I have also often seen people do this without announcing the aggregate because <some undefined bad thing> will happen, usually justified with much hand-waving. The people who do this can usually not be reasoned with....
It happens all the time...
Warren.
So, say I'm a provider that has received a /22 from UUNet (just for example Chris :-) ) and I now get another transit provider and announce the /22 there. So, I call UUNet and ask them to announce the /22 as a more specific because I don't want a de-facto asymmetric configuration. I *want* to get a /20 from ARIN but my usage doesn't justify it yet, so I have to ride the /22 for some time. By the long string of anecdotal attacks in the string to date, listing most or all such providers as "bad" or "uninformed" how do you separate out those providers who are legitimately interested in routing redundancy and not clue impaired? Do we just say "too bad, routing table bloat is more important than your need for redundancy small guy!"? I find it interesting that the general theme is one of "we're smarter than they are because we aggregate more routes" as if clue were directly correlated to aggregated routing announcements. Mike -- Michael K. Smith NoaNet 206.219.7116 (work) 866.662.6380 (NOC) mksmith@noanet.net http://www.noanet.net
On Sun, 13 Feb 2005, Michael Smith wrote:
From: "Warren Kumari, Ph.D, CCIE# 9190" <warren@kumari.net> On Feb 13, 2005, at 2:31 AM, Christopher L. Morrow wrote:
That and the "I have 1 circuit to $good_provider and 1 circuit to $bad_provider and the only way I can make them balance is to split my space in half and announce more specifics out through each provider" argument. I have also often seen people do this without announcing the aggregate because <some undefined bad thing> will happen, usually justified with much hand-waving. The people who do this can usually not be reasoned with....
So, say I'm a provider that has received a /22 from UUNet (just for example Chris :-) ) and I now get another transit provider and announce the /22 there. So, I call UUNet and ask them to announce the /22 as a more specific
Meaning you have PA space from UUNET, and you have BGP so you can multi-home... I'd expect you to know how to deaggregate yourself. You MIGHT even know how to send no-export on deaggregated prefixes, or use the 1996 policies to influence preferences/prepends internal to 701, yes?
because I don't want a de-facto asymmetric configuration. I *want* to get a /20 from ARIN but my usage doesn't justify it yet, so I have to ride the /22 for some time.
I'm not clear as to how the /22 to /20 discussion goes, or how it's even relevant... but it's been a long day. Can you elaborate?
By the long string of anecdotal attacks in the string to date, listing most or all such providers as "bad" or "uninformed" how do you separate out those providers who are legitimately interested in routing redundancy and not clue
a /22 in both directions seems like safe 'redundancy'. Adding no-export /24's or /32's if you want (yuck) would get you more preference inside one provider or the other. I'm also fairly sure I didn't say: "bad" or "uniformed" the 'bad provider' is from Warren, not I.
impaired? Do we just say "too bad, routing table bloat is more important than your need for redundancy small guy!"?
I think that folks have been pushed toward multihoming with multiple providers (not just 'redundant T1' or 'shadow T1' services inside the same provider) over the last few years. That means some bloat is bound to occur. I'm not measuring it myself, but the renesys folks and LCS folks have been I think? Perhaps they can comment on that phenomenon?
I find it interesting that the general theme is one of "we're smarter than they are because we aggregate more routes" as if clue were directly correlated to aggregated routing announcements.
it's not? :) (joking of course) As I said before some folks feel they have a legitimate reason for deaggregating. If you can spend some time chatting them up about their reasons and either: 1) realizing they hav a point 2) re-purpose their thoughts toward 'better cidr management' (as pfs said) then good for you... and everyone else :) I have spent sometime on occasion doing this, sometimes it works out, othertimes it doesn't :( It's always an experience though. -Chris
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 13, 2005, at 6:19 PM, Christopher L. Morrow wrote:
On Sun, 13 Feb 2005, Michael Smith wrote:
From: "Warren Kumari, Ph.D, CCIE# 9190" <warren@kumari.net> On Feb 13, 2005, at 2:31 AM, Christopher L. Morrow wrote:
That and the "I have 1 circuit to $good_provider and 1 circuit to $bad_provider and the only way I can make them balance is to split my space in half and announce more specifics out through each provider" argument. I have also often seen people do this without announcing the aggregate because <some undefined bad thing> will happen, usually justified with much hand-waving. The people who do this can usually not be reasoned with....
So, say I'm a provider that has received a /22 from UUNet (just for example Chris :-) ) and I now get another transit provider and announce the /22 there. So, I call UUNet and ask them to announce the /22 as a more specific
Meaning you have PA space from UUNET, and you have BGP so you can multi-home... I'd expect you to know how to deaggregate yourself. You MIGHT even know how to send no-export on deaggregated prefixes, or use the 1996 policies to influence preferences/prepends internal to 701, yes?
because I don't want a de-facto asymmetric configuration. I *want* to get a /20 from ARIN but my usage doesn't justify it yet, so I have to ride the /22 for some time.
I'm not clear as to how the /22 to /20 discussion goes, or how it's even relevant... but it's been a long day. Can you elaborate?
By the long string of anecdotal attacks in the string to date, listing most or all such providers as "bad" or "uninformed" how do you separate out those providers who are legitimately interested in routing redundancy and not clue
a /22 in both directions seems like safe 'redundancy'. Adding no-export /24's or /32's if you want (yuck) would get you more preference inside one provider or the other.
I'm also fairly sure I didn't say: "bad" or "uniformed" the 'bad provider' is from Warren, not I.
Whoops, I guess I wasn't very clear. By $good_provider and $bad_provider I wasn't meaning to imply that $good_provider ran their network "better" or "cleaner" than $bad_provider, merely that (by default and without tuning) more traffic travels via $good_provider than via $bad_provider (e.g. $bad_provider buys transit from $good_provider). I guess I should have used big_provider and little_provider or something.
impaired? Do we just say "too bad, routing table bloat is more important than your need for redundancy small guy!"?
No, I don't think anybody was saying that, just that many people are needlessly de-aggregating space. I have seen someone with a single T3 (and obviously a single provider!) announcing his PA /19 as a bunch of /24s, redistributed into BGP from OSPF! Some consultant had come in, set it up and left. After a bit of help, said person turned off BGP and has been running fine ever since. No-one was trying to take away your redundancy, just limit the number of unnecessary announcements. See Chris's comments above on how to get redundancy without making others pay for it....
I think that folks have been pushed toward multihoming with multiple providers (not just 'redundant T1' or 'shadow T1' services inside the same provider) over the last few years. That means some bloat is bound to occur. I'm not measuring it myself, but the renesys folks and LCS folks have been I think? Perhaps they can comment on that phenomenon?
I find it interesting that the general theme is one of "we're smarter than they are because we aggregate more routes" as if clue were directly correlated to aggregated routing announcements.
Well, often lack of aggregation is directly caused by lacy of clue. Obviously there are legitimate reasons for de-aggregating a big block (otherwise we would all just carry 0/0 :-) ) but if there is no additional information in the more specifics, then there is no reason for them the be announced.
it's not? :) (joking of course) As I said before some folks feel they have a legitimate reason for deaggregating. If you can spend some time chatting them up about their reasons and either: 1) realizing they hav a point 2) re-purpose their thoughts toward 'better cidr management' (as pfs said)
then good for you... and everyone else :) I have spent sometime on occasion doing this, sometimes it works out, othertimes it doesn't :( It's always an experience though.
It certainly is...
-Chris
- -- Militant Agnostic--I don't know and you don't either! -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFCEAWZHSkNr4ucEScRAoz3AKD6qP+le+n38KEodea6WsoWB/av9gCdH/bu 4YG3VVrMNd/61Lr5ZZBgnRY= =/Ebs -----END PGP SIGNATURE-----
On Sun, 13 Feb 2005, Michael Smith wrote:
From: "Warren Kumari, Ph.D, CCIE# 9190" <warren@kumari.net> Date: Mon, 14 Feb 2005 10:14:38 -0500 To: <nanog@merit.edu> Subject: Re: The Cidr Report
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Feb 13, 2005, at 2:31 AM, Christopher L. Morrow wrote:
On Sat, 12 Feb 2005, Alexander Koch wrote:
On Sat, 12 February 2005 14:58:42 +0000, Stephen J. Wilcox wrote:
From: "Stephen J. Wilcox" <steve@telecomplete.co.uk> [...] - would you agree that most of the poor deaggregating is not intentional ie that they're announcing their '16 class Cs' or historically had 2 /21s and
Think about someone putting in a Null0 route and re- exporting stuff unconditionally, now after he originates his /19 he is then adding a /24 here, and a /25 there. Lack of experience, when you suggest to them they should remove these announcements they are afraid to change it, not understanding the implications, etc.
Not to mention ppl using cisco and prefix lists, it is way too easy with cisco to say '/19 le 24', and then they use outbound prefix lists to their transit supplier (different, but related as I see it). Some transit ISPs use that a lot, and encourage the table growth.
There are some business reasons to de-aggregate. Look at some outages caused by 'routing problems' (someone leaked my /24's to their peers, peers, peer and my traffic got blackholed, because the public net only knows me as a /20)
There are multiple reasons for deaggregation aside from 'dumb operator', some are even 'valid' if you look at them from the protection standpoint.
-Chris
That and the "I have 1 circuit to $good_provider and 1 circuit to $bad_provider and the only way I can make them balance is to split my space in half and announce more specifics out through each provider" argument. I have also often seen people do this without announcing the aggregate because <some undefined bad thing> will happen, usually justified with much hand-waving. The people who do this can usually not be reasoned with....
It happens all the time...
Warren.
So, say I'm a provider that has received a /22 from UUNet (just for example Chris :-) ) and I now get another transit provider and announce the /22 there. So, I call UUNet and ask them to announce the /22 as a more specific because I don't want a de-facto asymmetric configuration. I *want* to get a /20 from ARIN but my usage doesn't justify it yet, so I have to ride the /22 for some time.
Hi Mike, this isnt the scenario being discussed. The scenario of issue is where you get your /22 from UUNET and announce 4x /24 ie based on what ips you have you choose to announce more than the minimum to make them routable
By the long string of anecdotal attacks in the string to date, listing most or all such providers as "bad" or "uninformed" how do you separate out those providers who are legitimately interested in routing redundancy and not clue impaired? Do we just say "too bad, routing table bloat is more important than your need for redundancy small guy!"?
As I say this isnt the issue here, altho what you suggest would be an example of further aggregation that i personally think is extreme. Multihoming must be possible and a hierarchical structure to the internet is not appropriate.
I find it interesting that the general theme is one of "we're smarter than they are because we aggregate more routes" as if clue were directly correlated to aggregated routing announcements.
Well, there does seem to be some loose correlation it cant be denied... (counter examples not required, we know they exist) Steve
Hi Stephen, Stephen J. Wilcox said the following on 13/02/2005 00:58:
that applies to medium and large providers too reading this list - how often do they actually check what prefixes they are sourcing, from my recent work at a couple of european IXes i had a number of folks email me offlist as they hadnt realised til I sent out an email they had deaggregation and once it was pointed out they just fixed it.
I don't think many people have the time to check customers deaggregation - it doesn't obviously help with their company's business/profits therefore isn't on the radar. Or is a job done when they have time. Like you, Hank and I (amongst many others) have spent quite a bit of time pointing out possible leakages to providers - and most have been very polite and appreciative of the advice. I've had a few negative reactions, mostly questioning why this is any of my business. :(
so to repeat my earlier suggestion - if transit providers, particularly the larger ones setup scripts to notify their customers daily/weeks of routing deaggregation do you think we might gain some traction in educating and fixing this?
Interesting thought - yes that might help, but then how many providers would be willing to do this? The CIDR report website is available, anyone can pop their ASN into the tool there, and they can figure out what needs to be aggregated. Should the tool be packaged up as something anyone can run from their web page, or a cron job? What would customers think about this if their upstreams do it? Or should some third party do it, like Geoff and I post our respective regional reports and aggregation summaries...? How would this differ from spam - and anyway, how would be get in touch with every ISP on the Internet... etc... philip --
Until there's deep shame, or real financial incentive to not being listed as a member of the dirty 30, nothing is going to happen in terms of aggregation. Unfortunately, an automated email going out to each of the dirty 30 weekly from the Cidr Report saying that their network again made the list of top 30 most shameful examples of how to participate as an active member in the global routing table would probably have little effect. If they cared, they'd already be doing something about it. Nothing is going to happen unless enough people (ASNs) take a simultaneous, and UNITED stand, and make it painful for those that don't care about the routes they leak to the net. Here's an idea, it's probably not the best idea, and has a *lot* of potential problems, but it's just an idea: Pick the top 1 or two worst offenders every week, and automatically dump them into a route distribution server would work in the same way as the Team Cymru bogon server list. I bet THAT would get people to scramble aggregate! Want to make a clear business case for spending time to clean up routes? How about "global routability" ? Every week, the top of the list would be singled out, and they could be placed on the server, and anyone that wanted to null route them based on that information could do so. A level of automation would be required to quickly remove them from the blacklist as soon as they aggregated, and quickly re-add them without warning if they decide to deaggregate within a certain time frame of being on the blacklist. If the addition/removal was automated, it would be clear cut as to why the "victim" was placed on the list. No favoritism or politics would come in to play. It would get results. I'm not sure what those results would be, and the result might just be a bunch of really mad and aggravated people, and a slightly more broken internet, but there'd definitely be results. Or something. ;-) (I bet it would be a lot like the early days of DNS-RBL for mail servers) I'm sure someone on this list who is wiser than me, has a better idea. I'd love to hear it discussed. I'm going to repeat what I typed earlier: Nothing is going to happen unless enough people (ASNs) take a simultaneous, and UNITED stand, and make it painful for those that don't care about the routes they leak to the net. -Jerry
On Sun, 13 Feb 2005, Jerry Pasker wrote:
Until there's deep shame, or real financial incentive to not being listed as a member of the dirty 30, nothing is going to happen in terms of aggregation.
[...]
Nothing is going to happen unless enough people (ASNs) take a simultaneous, and UNITED stand, and make it painful for those that don't care about the routes they leak to the net.
Similarly, "enough people" won't get involved until there is real financial incentive to do so. Most people won't particularly care about the number of routes in a full table until their equipment becomes unable to handle it. Does anyone have any idea of the routing table size limits of old but still common equipment? I suspect we'll see a lot more interest in routing table size when things start to break. -- Aaron
On Feb 13, 2005, at 2:36 AM, Jerry Pasker wrote:
Pick the top 1 or two worst offenders every week, and automatically dump them into a route distribution server would work in the same way as the Team Cymru bogon server list. I bet THAT would get people to scramble aggregate! Want to make a clear business case for spending time to clean up routes? How about "global routability" ?
Great idea. Too bad the networks at the top of the list are exactly the networks we would need to implement the filtering for it to really hurt. I can't see them filtering themselves.... -- TTFN, patrick
Jerry Pasker wrote:
Until there's deep shame, or real financial incentive to not being listed as a member of the dirty 30, nothing is going to happen in terms of aggregation.
I sometimes wonder if this list is seen as some sort of hit parade of potential peers and if that is the case then perhaps another list acknowledging the largest players with the best aggregation might also be in order. Mark.
Hello Stephen,
any thoughts on how to fix it?
back to the "smallest allocation" per /8 that the RIRs have published and make it part of the MoU at the larger NAPs/exchange points.
at the large end i'm still without an explanation as to why large networks require so many prefixes - none of them seem to comment?
Commercial reasons? The traffic goes to the 32x/24 instead of the /19. Not to mention the BGP customer may go to another provider. You end up in discussions (if you get that chance at all) with your sales group and look like a dogmatic fool as "other companies do this". Saving the world works best if the pain is no one's fault - at least "not my fault" ;-) Regards, Marc -- Marc Binderberger <marc@sniff.de>
Frotzler, Florian said the following on 11/02/2005 21:31:
Recent Table History Date Prefixes CIDR Agg 04-02-05 151613 103143 05-02-05 152142 103736 06-02-05 152231 103721 07-02-05 152353 103830 08-02-05 152514 103966 09-02-05 153855 104090 10-02-05 154283 104246 11-02-05 154341 104240
~ +3000 routes in one week? Anyone else frightened by this?
Yup. From my own Routing Report (due out in a couple of hours), a quick glance shows that the vast majority of the increase comes from ASNs assigned by ARIN (the ASNs from the other three registry regions show minimal increase in announcements). Most seem to come from AS4323. Today they are announcing 2606 prefixes, a week ago they were announcing 844 prefixes. philip --
On Sat, 12 Feb 2005, Philip Smith wrote:
From my own Routing Report (due out in a couple of hours), a quick glance shows that the vast majority of the increase comes from ASNs assigned by ARIN (the ASNs from the other three registry regions show minimal increase in announcements).
Duh! No suprise there. ARIN just gives IP space and only offers some measly online training: http://www.arin.net/library/training/index.html RIPE on the other hand, has 3-6 course a month, throughout Europe: http://www.ripe.net/training/lir/index.html http://www.ripe.net/cgi-bin/courselist.pl.cgi APNIC also has a number of courses and goes out to where it is needed: http://www.apnic.net/training/schedule.html As long as ARIN just doles out IP space with no education, the routing table will continue to grow. -Hank
Most seem to come from AS4323. Today they are announcing 2606 prefixes, a week ago they were announcing 844 prefixes.
philip
I split my Routing Analysis based on registry region so that the constituents of each region know what is going on in their area. As you know registries offer training if their membership ask for it. APNIC and RIPE NCC membership seem to ask for training other than just how to be an LIR. But how to be an LIR doesn't cover every single ASN who is announcing address space. For example, APNIC has an extensive region wide programme, but there is still large deaggregation here in the AP region... So I really don't see what RIR training has to do with this. It helps, but it isn't the complete solution. If the industry is worried, the industry has to do something about it. philip -- Hank Nussbacher said the following on 13/02/2005 04:16:
Duh! No suprise there. ARIN just gives IP space and only offers some measly online training: http://www.arin.net/library/training/index.html
RIPE on the other hand, has 3-6 course a month, throughout Europe: http://www.ripe.net/training/lir/index.html http://www.ripe.net/cgi-bin/courselist.pl.cgi
APNIC also has a number of courses and goes out to where it is needed: http://www.apnic.net/training/schedule.html
As long as ARIN just doles out IP space with no education, the routing table will continue to grow.
-Hank
Most seem to come from AS4323. Today they are announcing 2606 prefixes, a week ago they were announcing 844 prefixes.
philip
hank@mail.iucc.ac.il (Hank Nussbacher) wrote:
Duh! No suprise there. ARIN just gives IP space and only offers some measly online training: http://www.arin.net/library/training/index.html
RIPE on the other hand, has 3-6 course a month, throughout Europe: http://www.ripe.net/training/lir/index.html http://www.ripe.net/cgi-bin/courselist.pl.cgi
You should read the course outline. RIPE teaches nothing whatsoever to do with routing. It's all registration stuff... But certainly, a routing course could be added, maybe to a somewhat more "techy" track like where the DNSSEC courses sit. Yours, Elmar. -- "Begehe nur nicht den Fehler, Meinung durch Sachverstand zu substituieren." (PLemken, <bu6o7e$e6v0p$2@ID-31.news.uni-berlin.de>) --------------------------------------------------------------[ ELMI-RIPE ]---
participants (20)
-
Aaron Hopkins -
Alexander Koch -
Christopher L. Morrow -
Elmar K. Bins -
Fredy Kuenzler -
Frotzler, Florian -
Hank Nussbacher -
Jerry Pasker -
Jon Lewis -
Justin Ryburn -
Marc Binderberger -
Mark Prior -
Michael Smith -
Mike Leber -
Neil J. McRae -
Patrick W Gilmore -
Philip Smith -
Stephen J. Wilcox -
Vinny Abello -
Warren Kumari, Ph.D, CCIE# 9190