Ok, this is in follow up to a discussion Bill Woodcock, John Hawkinson, Kim Claffy and I had on the IPNMoo (or CAIDAnce or whatever it's called) regarding the IRR. This discussion led me to ask a bunch of questions. I'd like to present these questions for discussion, and share a few of my comments. How accurately does the IRR represent the actual routing policy of the various participating networks and of the Internet as a whole? Well, according to the IRR Violations statistics at http://compute.merit.edu/problems.html, not very well. Since MAE-East is the "most important" (don't even start on that one) exchange on the Internet, let's look at the very few datapoints available. 10187 IRR violations at MAE-East on 11/13/96 9873 IRR violations at MAE-East on 11/12/96 There is no data available on the web page for any other days. Barring all of the obvious flames about the "if's" involved (number of data points, statistical methodology, what constitutes a violation, multiple violations, Merit stats don't mean anything, etc.) this doesn't look real good. If there are about 42000 paths in the "default-free" routing table (this part we know), and every last one of them is registered in the IRR (which isn't true), ~25% of the routes on the Internet do not jive with the IRR, according to these Merit stats. Obviously, this calculation isn't exactly statistically sound, but it's something to think about. The fact that there are so many if's is kind of the point, too. If you don't trust Merit stats, whose? Is there anything else available to base any sort of conclusions on? I know of at least one example of a reported "IRR violation" that isn't, so there could certainly be (many) more. Anyone from Merit care to explain how these stats are collected, and/or if they should be considered accurate? There's nothing on the web pages concerning IRR violation statistic collection methodology so far as I can tell. To quote the eloquent folks at Merit: "Maintaining accurate IRR information is important as several providers base their router configurations on IRR information. Incorrect IRR data may result in loss of connectivity. The IRR also provides a valuable source of information for analysis and debugging tools." How many providers are basing their policy solely on IRR information? Are these providers (if they exist) at all concerned with the notion that (if the RA statistics on IRR violations are at all useful) they may not be able to get to up to 25% of the Internet at any one time? Might the IRR be more harmful than goodful if it's actually used to generate router config's by any small group of providers, and not by everyone? (Unless you think segmentation is good). Is the IRR more valuable as a "source of information for analysis and debugging tools" than as a source of information to base routing policy on? I would say apparently so. Is the IRR supported? Well, as far as I can tell almost everyone that should be participating (in some form) is. Is it supported in the sense that everyone that is participating is commited to maintaining accurate information? I would say apparently not. Is it supported in the sense that it is actually used for anything other than "analysis and debugging"? (And what sort of useful analysis can be done with data that is 25% bad anyhow?) I don't know. Should the IRR be used for anything presumed to be "actually useful"? Well, I certainly wouldn't rely (solely) upon it. How about you? If the IRR were useless would it break the Internet? Nope. Could the IRR improve the Internet if it were used by everyone? Maybe. Should the IRR exist? I think yes. This whole "discussion" begs a lot of other questions. Adressing them is "beyond the scope of this document". :) To be Scott Bradner for a moment, here's a disclaimer. I don't have large sums of grant money to do statistics, and I have to rely on people who do, so don't complain about how terrible my stat is. I know many (all?) of these questions have been brought up before. I believe there is interest in discussion, and so I have brought them up again. If you don't want to talk about it, don't. These bits will not break your network (well, unless you're <insert anyones favorite congested network here>). Brian ___ Brian Merritt bmerritt@getnet.net ___ Principal Network Engineer GetNet International
reasonable questions. my general reaction to your message is that the irr could be used for more things than it is currently *if* the information were more accurate. it's logical to ask "do we *need* those 'more things?'" one argument is "apparently not, because the internet is working without them." but at the same time, as the net continues to grow and the routing continues to become more complex (e.g., with increasing layers of aggregation), having a database of routing information and tools which can analyze that routing information seems like A Good Thing i come from a provider which believes in the irr. but at this point i think we "believe in it" for what we use it for: route-filtering our bgp customers in order to provide a sanity check before passing those routes onto the global internet. we think that's important, and we think that by doing this we are being a responsible member of the global internet routing system (especially given our size) it *is* worth noting that we have local-use stuff in our registry to do two things. first is automatic config of static routes. second is to track all of our bgp peers (in a lot more practical detail than either inet-rtr objects or as-in/as-out/interas-in/interas-out attributes in aut-num objects). so we most definitely "believe in" those uses as well we are interested in some of the uses of the irr that others have proposed and are using (e.g., curtis has proposed some interesting uses, and the raconfig tool is actively being used by at least one provider), though we haven't yet chosen to dedicate resources to those uses ourselves so having said all of that, what's an irr violation? for us an "irr violation" would be that for some reason we're not accepting a route from a customer. in those cases that customer calls us and we work with them to fix the problem. the important point being that we think our use of the registry, even if considered a "violation" by some stats- collection system elsewhere on the net, results in our AS being a more stable part of the global routing system so working to get rid of "violations" is a good thing because it might usher in some other very helpful uses of the registry. but in the meantime, keep in mind that even though one person may call an irr entry a "violation" because of the way *they* use the irr, other people may be using that entry to help keep the routing system stable my US$0.02 /jws
Ok, this is in follow up to a discussion Bill Woodcock, John Hawkinson, Kim Claffy and I had on the IPNMoo (or CAIDAnce or whatever it's called) regarding the IRR. This discussion led me to ask a bunch of questions. I'd like to present these questions for discussion, and share a few of my comments.
How accurately does the IRR represent the actual routing policy of the various participating networks and of the Internet as a whole?
Well, according to the IRR Violations statistics at http://compute.merit.edu/problems.html, not very well.
Since MAE-East is the "most important" (don't even start on that one) exchange on the Internet, let's look at the very few datapoints available.
10187 IRR violations at MAE-East on 11/13/96 9873 IRR violations at MAE-East on 11/12/96
There is no data available on the web page for any other days.
Barring all of the obvious flames about the "if's" involved (number of data points, statistical methodology, what constitutes a violation, multiple violations, Merit stats don't mean anything, etc.) this doesn't look real good. If there are about 42000 paths in the "default-free" routing table (this part we know), and every last one of them is registered in the IRR (which isn't true), ~25% of the routes on the Internet do not jive with the IRR, according to these Merit stats.
Obviously, this calculation isn't exactly statistically sound, but it's something to think about. The fact that there are so many if's is kind of the point, too. If you don't trust Merit stats, whose? Is there anything else available to base any sort of conclusions on? I know of at least one example of a reported "IRR violation" that isn't, so there could certainly be (many) more. Anyone from Merit care to explain how these stats are collected, and/or if they should be considered accurate? There's nothing on the web pages concerning IRR violation statistic collection methodology so far as I can tell.
To quote the eloquent folks at Merit:
"Maintaining accurate IRR information is important as several providers base their router configurations on IRR information. Incorrect IRR data may result in loss of connectivity.
The IRR also provides a valuable source of information for analysis and debugging tools."
How many providers are basing their policy solely on IRR information? Are these providers (if they exist) at all concerned with the notion that (if the RA statistics on IRR violations are at all useful) they may not be able to get to up to 25% of the Internet at any one time? Might the IRR be more harmful than goodful if it's actually used to generate router config's by any small group of providers, and not by everyone? (Unless you think segmentation is good). Is the IRR more valuable as a "source of information for analysis and debugging tools" than as a source of information to base routing policy on? I would say apparently so. Is the IRR supported? Well, as far as I can tell almost everyone that should be participating (in some form) is. Is it supported in the sense that everyone that is participating is commited to maintaining accurate information? I would say apparently not. Is it supported in the sense that it is actually used for anything other than "analysis and debugging"? (And what sort of useful analysis can be done with data that is 25% bad anyhow?) I don't know. Should the IRR be used for anything presumed to be "actually useful"? Well, I certainly wouldn't rely (solely) upon it. How about you? If the IRR were useless would it break the Internet? Nope. Could the IRR improve the Internet if it were used by everyone? Maybe. Should the IRR exist? I think yes.
This whole "discussion" begs a lot of other questions. Adressing them is "beyond the scope of this document". :)
To be Scott Bradner for a moment, here's a disclaimer.
I don't have large sums of grant money to do statistics, and I have to rely on people who do, so don't complain about how terrible my stat is. I know many (all?) of these questions have been brought up before. I believe there is interest in discussion, and so I have brought them up again. If you don't want to talk about it, don't. These bits will not break your network (well, unless you're <insert anyones favorite congested network here>).
Brian ___ Brian Merritt bmerritt@getnet.net ___ Principal Network Engineer GetNet International
I build the router configuration tool that most providers use. I also build quite a few analysis and diagnosis tools using IRR. I will respond to this message from that perspective of a tool builder. Brian Merritt (bmerritt@asdf.getnet.com) on November 14:
How many providers are basing their policy solely on IRR information? Are these providers (if they exist) at all concerned with the notion that (if the RA statistics on IRR violations are at all useful) they may not be able to get to up to 25% of the Internet at any one time? Might the IRR be more harmful than goodful if it's actually used to generate router config's by any small group of providers, and not by everyone? (Unless you think segmentation is good). Is the IRR more valuable as a "source of information for analysis and debugging tools" than as a source of information to base routing policy on? I would say apparently so. Is the IRR supported? Well, as far as I can tell almost everyone that should be participating (in some form) is. Is it supported in the sense that everyone that is participating is commited to maintaining accurate information? I would say apparently not. Is it supported in the sense that it is actually used for anything other than "analysis and debugging"? (And what sort of useful analysis can be done with data that is 25% bad anyhow?) I don't know. Should the IRR be used for anything presumed to be "actually useful"? Well, I certainly wouldn't rely (solely) upon it. How about you? If the IRR were useless would it break the Internet? Nope. Could the IRR improve the Internet if it were used by everyone? Maybe. Should the IRR exist? I think yes.
There are quite a few providers who are using RtConfig to configure their routers. There are providers who have their own configuration tools as well. So far, everyone who has given me feedback on using RtConfig to configure their production routers have been positive and enthusiastic about it. They find it to be a lot simpler/more manageable to specify policies at the high level and generate low level router configurations from it than configuring each router individually and maintaining consistency across them. The data needed for these providers are all accurate. An in some case the data become even more accurate after they started using the tools. Is the data 100% accurate? No. Would it help if it were? Definitely. We have written a tool to analyze the policies and the Internet topology in IRR and aggressively suggest CIDR aggregations which do not break routing. This tool is an important one, because Internet is growing, the topology is becoming more complex, and more and more fraction of the engineers are not up to speed on cidrization. However, this tool needs close to 100% accurate policies to work effectively. Otherwise, the routing may break (with small probability since the tool is very pessimistic when the policies are missing). Recently, we focused on tools to make IRR more accurate. We built roe, route object editor. A provider by feeding a routing table dump, can fix most of its incorrect route objects by few mouse clicks. aoe, autonomous system object editor, will be released soon which tries to do the same for the aut-num objects. My few Liras. Cengiz -- Cengiz Alaettinoglu Information Sciences Institute (310) 822-1511 University of Southern California http://www.isi.edu/~cengiz
Hi Brian, at Thu, 14 Nov 1996 08:16:38 MST, you wrote:
Barring all of the obvious flames about the "if's" involved (number of data points, statistical methodology, what constitutes a violation, multiple violations, Merit stats don't mean anything, etc.) this doesn't look real good. If there are about 42000 paths in the "default-free" routing table (this part we know), and every last one of them is registered in the IRR (which isn't true), ~25% of the routes on the Internet do not jive with the IRR, according to these Merit stats.
The IRR AS origin discrepancy web page counts the number of announcements per provider that have an AS origin that differs from what is registered in the IRR. So, if two providers announce the same mis-registered prefix, this prefix will be counted twice. Though the number is still high, I suspect that percentage of incorrectly resgistered routes is much lower than 25%. We don't have numbers for the number of unique routing table entries that have IRR origin discrepancies, bit it might be useful to look at a chart of some of the larger providers. With the exception of Sprint, most providers seem to have ~10% error in their BGP announcements (of course, this is from a very small sampling). (Data from 11/18/96) Provider #RT entries #discrep % -------------------------------------------- ANS 2611 186 7.13 Advantis 183 21 11.47 BBN 8996 1129 12.55 Compuserve 1523 3 .20 MCI 15135 1578 10.43 SPRINT 14085 3584 25.45 UUNet (no information available)
Obviously, this calculation isn't exactly statistically sound, but it's something to think about. The fact that there are so many if's is kind of the point, too. If you don't trust Merit stats, whose? Is there anything else available to base any sort of conclusions on? I know of at least one example of a reported "IRR violation" that isn't, so there could certainly be (many) more. Anyone from Merit care to explain how these stats are collected, and/or if they should be considered accurate? There's nothing on the web pages concerning IRR violation statistic collection methodology so far as I can tell.
Sorry about the lack of documentation (we're working on it... :) ) The methodology is: For each entry in a RS routing table dump with an IGP origin, 1) compare the BGP aspath origin with the origin(s) returned from an exact route object lookup in the IRR. If a route object(s) exists and the origins match the BGP information, continue on and check another dump entry 2) else, find the next less specific route object in the IRR, and compare the origin(s) with the BGP aspath information. If there is not a match, flag the entry as an IRR discrepancy To address some performance/load problems, we recently began using a local, caching version of the IRR server for the discrepancy reports. Unfortunately, there are some known bugs with the new server in the way that it performs less specific lookups. So, until we get the bugs resolved, there will probably be a few errors in the discrepancy reports. Though, the bugs should result in an overestimatation (if anything) of the number of of IRR discrepancies. I agree that the quality of information in the IRR could use substantial improvements. Though, the quality of BGP information could also use improvement... To reiterate some of Cengiz's comments, I agree that real anaylsis and rational management of the Internet's topological (routing) growth requires some sort of prescriptive database. As IRR based tools and reports continue to develop and mature, I am hopeful that there will be growing incentive for ISPs to help maintain the IRR information. - Craig -- Craig Labovitz labovit@merit.edu Merit Network, Inc. (313) 764-0252 (office) 4251 Plymouth Road, Suite C. (313) 747-3745 (fax) Ann Arbor, MI 48105-2785
In message <199611181859.NAA09628@merit.edu>, Craig Labovitz writes:
We don't have numbers for the number of unique routing table entries that hav e IRR origin discrepancies, bit it might be useful to look at a chart of some o f the larger providers. With the exception of Sprint, most providers seem to have ~10% error in their BGP announcements (of course, this is from a very small sampling).
(Data from 11/18/96) Provider #RT entries #discrep % -------------------------------------------- ANS 2611 186 7.13 Advantis 183 21 11.47 BBN 8996 1129 12.55 Compuserve 1523 3 .20 MCI 15135 1578 10.43 SPRINT 14085 3584 25.45 UUNet (no information available)
I just looked through: http://compute.merit.edu/stats/mae-west/problems/IRR/1673.961119.html Of the 192, in 54 the registered origin AS was in the AS path, but was not the origin AS. This is common. The provider registers the route object using the provider's AS rather the true origin AS. In some of these cases the prefix is dual homed to two ANS AS and only registered in one with policy set to accept either. One was deleted from the IRR and replaced with an aggregate but our configs haven't picked upthe change and two were listed in the Merit report as AS0, but there were IRR entries for those with the correct origin. I spot checked a bunch of others (208.141.24.0/22, 208.141.16.0/21, 207.166.192.0/19, 207.70.114.0/24) and there were route objects corresponding to the origin AS so there is also a problem with the reporting where there are false errors being listed. Curtis
participants (5)
-
Brian Merritt -
Cengiz Alaettinoglu -
Craig Labovitz -
Curtis Villamizar -
John W. Stewart III