- Test - lists.nanog.org

CIDR FAQ
by Hank Nussbacher 28 Nov '95

28 Nov '95

------------------------------------------------------------------------ The CIDR FAQ Version 5 15 November 1995 ------------------------------------------------------------------------ The following document is a collection of Frequently Asked Questions about CIDR. This document is not meant to be a networking/routing guide and tutorial. Where appropriate pointers to other documents of a more general nature have been mentioned. Updates from a previous version are marked with a '|' in column 1. If you have any questions you would like added, please send them to the editor mentioned below: Hank Nussbacher (Tel Aviv University and IBM Israel) hank(a)vm.tau.ac.il or hank(a)ibm.net.il If you would like to "discuss" items from this FAQ please send your mail to cidrd(a)iepg.org This FAQ is being distributed to the following groups and lists: alt.internet.services alt.internet.access.wanted nanog(a)merit.edu inet-access(a)earth.com iap(a)vma.cc.nd.edu local-ir(a)ripe.net cidrd(a)iepg.org |local-ir(a)apnic.net To retrieve the most up-to-date version of this document: |- ftp://ftp.ibm.net.il/pub/docs/cidr.faq |- http://www.rain.net/faq/cidr.faq.html ------------------------------------------------------------------------ General questions ----------------- 1. What does CIDR stand for? CIDR stands for Classless Inter-Domain Routing and is documented in RFC1517/1518/1519/1520. CIDR is an effective method to stem the tide of IP address allocation as well as routing table overflow. Without CIDR having been implemented in 1994 & 1995, the Internet would not be functioning today. Basically, CIDR eliminates the concept of class A, B, and C networks and replaces this with a generalized "IP prefix". CIDR can be used to perform route aggregation in which a single route can cover the address space of several "old-style" network numbers and thus replace a lot of old routes. This lessens the local administrative burden of updating external routing, saves routing table space in all backbone routers and reduces route flapping (rapid changes in routes), and thus CPU load, in all backbone routers. CIDR will also allow delegation of pieces of what use d to be called "network numbers" to customers, and therefore make it possible to utilize the available address space more efficiently. See question #6 below for details on "IP prefix"s. 2. What is an ASN? ASN stands for Autonomous System Number and acts to merge many networks into a logical domain. 3. What is BGP? BGP stands for Border Gateway Protocol and is the de-facto standard for routing between Autonomous Systems in the Internet. All communications between Internet Service Providers (ISP) is handled via BGP4 which supports CIDR. 4. Why should I make the effort and convert my routing to be CIDRized? The routing tables in the Internet have been growing as fast as the Internet and the router technology specifically and computer technology in general has not been able to keep pace. In December 1990 there were 2190 routes and 2 years later there were over 8500 routes. In July 1995 there are now over 29,000 routes, which require approximately 10 MB in a router with a single peer. Routers at interconnection points (or multi-homed hosts doing full routing with many peers) receive these routes from several peers, and need several dozen megabytes of RAM (and the appropriate CPU horsepower) to handle this. A list of those routers that can handle this appears at the end of this question. Routers with 64MB of memory have the capacity for approximately 60,000 routes after which some routes will just have to be left out of the global routing tables, and the more likely ones to be left out are routes covering small pieces of address space. Without the CIDRization work that has gone on for the past 2 years the routing tables would be in excess of 65,000 routes. By CIDRizing you help the Internet reduce the routing overload as well as increasing the liklihood that in the future your routes will be carried by all ISPs. The major benefit of CIDR is to allow for continuous, uninterrupted growth of the Internet. For a significant percentage of sites connected to the Internet the value of the Internet increases with the total number of sites connected to the Internet. Therefore, taking steps needed to allow for continuous uninterrupted growth (like CIDRizing, or renumbering) is beneficial to such sites. The routers today that are available to handle the full routing table are: cisco 7000 w/ 64Mb cisco 4500 w/ 32Mb IBM ENSS and CNSS w/ 64Mb BayNetworks models AN/ASN/BLN/BCN w/ 32Mb 5. Can you give an example of a simple CIDR configuration for a cisco router? The following example creates 2 aggregates and suppresses any more specific addresses that may be contained within those aggregates. The access-list causes only those nets to be distributed as listed, and not any others that may exist in the BGP routing tables. router bgp 64100 no synchronization aggregate-address 172.16.0.0 255.248.0.0 summary-only aggregate-address 192.168.50.0 255.255.255.0 summary-only neighbor 192.168.54.2 remote-as 65000 neighbor 192.168.54.2 distribute-list 12 out default-metric 70 ! access-list 12 permit 192.168.50.0 0.0.0.255 access-list 12 permit 172.16.0.0 0.7.255.255 | Another method might be: | | router bgp 64100 | no synchronization | aggregate-address 172.16.0.0 255.248.0.0 | aggregate-address 192.168.50.0 255.255.255.0 | neighbor 192.168.54.2 remote-as 65000 | neighbor 192.168.54.2 distribute-list 102 out | default-metric 70 | ! | access-list 102 deny 172.16.0.0 0.7.255.255 255.252.0.0 0.3.255.255 | access-list 102 deny 192.168.50.0 0.0.0.255 255.255.255.128 0.0.0.127 An alternate method is via network and route statements: router bgp 64100 no synchronization network 172.16.0.0 mask 255.248.0.0 network 192.168.50.0 mask 255.255.255.0 neighbor 192.168.54.2 remote-as 65000 neighbor 192.168.54.2 default-metric 70 ip route 172.16.0.0 255.248.0.0 Null0 254 ip route 192.168.50.0 255.255.255.0 Null0 254 In this case, only those routes explicitly mentioned in "network" statements will be announced with BGP. For these routes to be announced, there has to be a corresponding route in the IP forwarding table, thus the need to create the static routes. The static routes will also serve as "pull-ups" for the route advertisements and thus prevent route flapping: these routes will always be announced with BGP by this router. Note that as long as more specific routes exist internally in your network, these will be used in preference to the static "less specific" route entries (longest prefix matching is being used). A good rule to follow is to never redistribute IGP learnt routes directly into BGP, but to rather use network or aggregate-address statements. And if you must redistribute dynamically learnt IGP routes into BGP, you MUST use filtering. The reasons for this advice are several, some of which are: 1) if your IGP is classful (e.g. RIP or IGRP) you will by default not do any route aggregation 2) if you have an internal stability problem (accidents do happen), this will be reflected as a "route flap" in the whole routing system, globally burning CPU cycles better spent on other things 3) if the IGP -> BGP transition is unrestricted, this can lead to false routing information escaping from your network (especially if you do not fully have administrative control over your IGP) 6. What do all these /16s and /24s mean in my BGP tables? This refers to the number of bits of the network part of the IP address. A former class B may appear as 172.50.0.0/16, which is the same as 256 class C's which can appear as 192.200.0.0/16. A single class C appears as 192.201.1.0/24. These "things" are often called an "IP prefix", which consists of an IP address and a mask length. The mask length specifies the number of leftmost contiguous significant bits in the corresponding IP address. Thus, an IP prefix with a prefix length of 15 (denoted /15) covers the address space of 128k IP addresses, and a /17 covers the address space of 32k IP addresses. Here is a table of the more popular CIDR blocks: # of former CIDR class C block nets ---- ---- /27 1/8 /26 1/4 /25 1/2 /24 1 /23 2 /22 4 /21 8 /20 16 /19 32 /18 64 /17 128 /16 256 = 1 former class B /15 512 /14 1024 /13 2048 In general, advertising a prefix covering less address space than a /24 prefix will probably not get into the global routing tables, and global Internet connectivity is less likely to happen. Note that for you as an administrator of an AS, it is a good idea to announce as few prefixes as possible and to utilize the address space as much as possible. | A more comprehensive table came out in October 1995 as: | | RFC1860: Variable Length Subnet Table For IPv4 | | This RFC is being revised and a new one will be out shortly. 7. Do I need to carry the full Internet routing table? When would it be necessary? What routers on the Internet carry full routing tables and how much memory is needed? No you do not need to carry the full Internet routing table. If you are single-homed, meaning you have a single connection to an ISP, then all you need to do is point a default route to the ISP and tell your ISP not to send you the full routing table. If you are multi-homed, you will want to know which nets to route via connection A and which nets to route via connection B. The easiest way to do this is to request a partial routing table from one ISP - with those nets that are closest to them, and default everything else to the other ISP. This way your routing tables need not contain the entire Internet universe but only a small subset. The closer you get to the hub or nexus of the Internet, the larger your routing tables need to be. For example, those connected to public exchange points (like the NAPs, CIX, STIX, LINX, dGIX) in general, carry full routing tables and run without a default route. 8. What is there in the Internet to stop me from making a mistake and announcing via BGP an aggregate that is larger than the nets I am in charge of? In principle there is nothing to stop you. The responsibility falls on both ends of the BGP link - you are responsible to filter what you announce and the receiving end - if it has its act together - filters also what it *thinks* it should be hearing from you so as prevent mistakes on your part. Those sites that do not work with access lists and filters and just readily accept what is sent to them are just waiting for a problem to happen. Filtering can either be done at the IP network level or at the BGP path (BGP orgin AS) level. See question 20 below for details on a tool to assist in filtering. 9. Who has to renumber with CIDR ? Sites that move from one ISP to another, and who had been allocated addresses from their original ISP's CIDR block, in all likelihood will have to return those addresses as part of the move. The reason is to keep the number of prefixes in the global routing system within the limits of current technology. | For further hints and procedures for renumbering, see the PIER | (Procedures for Internet/Enterprise Renumbering) homepage at: | | http://www.isi.edu:80/div7/pier Specific questions ------------------ 10. I have a /16 but have registered parts of it as /24s in the RADB. I now want to CIDRize. The problem is parts of the /16s are missing and are routed via a different ASN. Can you explain how more specific routes override more general ones and will I hurt my routing if I just advertise the /16 and not a bunch of /20s and /21s? There are two aspects to the answer: 1) Real (BGP) world: Given there are several AS's sharing addresses out of a /16 prefix, every AS should advertise exactly those prefixes which it is really originating. However, if there is one AS "originating" a significant majority of this address space, the concerned AS's might agree that this one and only advertises the /16 and all others their more specifics. The more specifics always take precedence over the less specific. 2) Routing registry: The registry DB, of course, should always reflect reality. If in the above example the AS's agree on the "big AS" announcing the /16, the "big AS" should document with the route-object that it's not really originating the whole aggregate by using "hole" attributes (see ripe.181, 5. The Route Object). 11. How can I redistribute our IGP routes (IGRP) so that they become aggregated when sent out via BGP? It is strongly discouraged to redistribute IGPs into BGP, because local IGP configuration errors might easily corrupt routing information of the whole Internet. If, however, you have to do it anyway, you MUST use strict distribute-lists with explicit permits (or route-maps) for redistribution. Here is an example for a Cisco configuration: router bgp 64100 aggregate-address 192.168.0.0 255.255.192.0 summary-only aggregate-address 172.16.0.0 255.254.0.0 summary-only redistribute igrp 64100 route-map origin-AS64100 ! ! or: ! redistribute igrp 64100 ! distribute-list 10 out igrp 64100 ! route-map origin-AS64100 permit 10 match ip address 10 ! access-list 10 permit 192.168.0.0 0.0.63.0 access-list 10 permit 172.16.0.0 access-list 10 permit 172.17.0.0 This example would generate one route 192.168/18 and one route 172.16/15 if any of the contained networks is in the IGP. 12. I am multihomed to three ISPs and can only CIDRize to two of them but to the third I need to still announce specific nets. What damage will this do to my AS? No damage can be done if the non-CIDR peer does not further announce your specifics to the global Internet. If your non-CIDR ISP DOES announce your specifics to the global Internet those specifics will have preference over the less specifics and therefore all traffic to you will get routed through the non-CIDR ISP. 13. I don't want to CIDRize. Can someone do proxy aggregation for me? Proxy aggregation should only be done with great care and should be avoided if you are not single-homed ! If you are single-homed ask your ISP. Others may proxy aggregate over your address range without your consent, and send your traffic over paths/links not of your choosing. Use of Routing Registries may help to identify and correct these problem areas. 14. What routers on the market today do support CIDR (classless routing)? Routers that are capable of handling CIDR are: - all Cisco routers running 10.0 or higher - all Bay Networks routers running 8.01 or higher - 3Com Netbuilder II and Netbuilder Remote Office - Telebit EMPB - Unix w/ BSD/OS 2.0 w/ gated 3.5alpha_11 + radix-fixes - IBM 2210 routers 15. How do I reach other parts of a subnetted old-style network when I have only partial routing information for that same old-style network?" There are actually three ways to solve this particular problem with Cisco's software. Which of them applies will depend on what software version is involved: o Preferred solution: turn on "ip classless" in your routers and use a default route inside your AS. The "ip classless" command prevents the existence of a single "subnet" route from blocking access via the default route to other subnets of the same old-style network. o Workaround for 9.1 or later software where the "ip classless" command is not available: install a "default network route" like this: "ip route 39.0.0.0 255.0.0.0 next-hop" along the axis your default route would normally take. o Workaround for 9.0 or older software: create a "default subnet route": "ip route 39.x.y.0 next-hop" combined with "ip default-network 39.x.y.0", otherwise as the 9.1 fix. Both of the latter solutions rely on static routes, and in the long run these will be impossible to maintain. In some topologies the use of static routes can be a problem (e.g. if you have more than one possible exit point from your AS to choose from). Supplemental information ------------------------ The following information is presented as supplemental information that is related to the CIDRization process. 16. What is the Internet Routing Registry? The IRR is a way for ASN's to publicize their own intended routing policies without having to request a change from a go-between. The RAdb which stands for the Routing Arbiter Data Base, which is part of the IRR, is part of a joint project between Merit and ISI. For full details contact: http://www.ra.net/routing.arbiter/RA/index.html. The Routing Arbiter is a project of the US National Science Foundation. As part of that project, it runs a routing registry database. That database (the RAdb) forms part of the IRR collection of databases. The RIPE database is not part of the RAdb but does participate in the IRR. At present, there are five entities that contribute to the IRR effort and more are expected. Today, all the contributing registries use the RIPE-181 database format. All IRR participants can be contacted via automail handlers that accept batch updates via email. An example of a routing update appears below: password: xxxxxxxx *rt: 138.134.0.0/16 *de: NET-IEC *or: AS378 *mb: AS378-MNT *ch: hank(a)aristo.tau.ac.il 950724 *so: RIPE The *rt: tag identifies the net and the routing policy is based on *or: tag. An example of a routing policy is presented below: aut-num: AS378 descr: ILAN descr: Israeli Academic and Research Network as-in: from AS1755 100 accept ANY as-in: from AS174 100 accept ANY as-in: from AS3339 100 accept AS3339 as-out: to AS1755 announce AS378 AS3339 as-out: to AS174 announce AS378 AS3339 as-out: to AS3339 announce ANY default: AS174 10 default: AS1755 20 default: AS3339 30 guardian: HANK(a)vm.biu.ac.il mnt-by: AS378-MNT admin-c: Hank Nussbacher tech-c: Hank Nussbacher changed: hank(a)vm.tau.ac.il 950627 source: RIPE For further details read over ripe-120.ps, ripe-121.ps and ripe-181.ps (via anonymous ftp from info.ripe.net/ripe/docs) |17. Are there any statistics available as to aggregation in the | Internet? | COUNTS OF RADB PREFIXES BY LENGTH - The number of Route objects | registered in the RADB, with active and withdrawn routes listed | by prefix length. Data from the current week is available from: | | ftp://ftp.ra.net/routing.arbiter/radb/reports/ | counts-by-prefix/Summarize_prefix.current | | Reports from previous weeks are available from: | | ftp://ftp.ra.net/routing.arbiter/radb/reports/ | counts-by-prefix/summarize_prefix.yymmdd | | IRR ROUTES SUMMARIZED BY PREFIX LENGTH WITH AGGREGATION - A | summary of unique prefixes registered in the IRR. Routes are | summarized by the first octet of the network number. If routes | within a prefix can be aggregated, a count is printed for each | prefix length that has a different count after aggregation. Data | from the current week is available from: | | ftp://ftp.ra.net/routing.arbiter/radb/reports/ | IRR_profile/IRR_Profile.current | | Reports from previous weeks are available from: | | ftp://ftp.ra.net/routing.arbiter/radb/reports/ | IRR_profile/IRR_Profile.yymmdd 18. How do I update the registered routing information for my ASN? You need to submit a "route" object update and perhaps an "aut-num" object update (see examples above). Route objects add new nets to your autonomous system (or you can remove nets from your autonomous system) and the Autonomous-system object describes the type of routing you wish to have. 19. Which Routing database takes precedence? RIPE? RADB? MCI? Do I have to update all of them? Each provider is allowed to select the preference order for authentic data. For example, ANS uses the following precedence: ANS, CANET, MCI, RIPE, RADB If there are two routes (with different origins) within one database, the changed date is used as a tiebreaker. Else, only database precedence is used. Thus, if the RADB entry has a more recent changed date than the RIPE, ANS will use the RIPE entry. You should only have to register in one of the IRR component databases. | There is a report which shows all routes in the RADB for a specified | AS and whether there are any duplicate routes in other IRR | registries or any aggregates which cover the route (in any of the | registries, including the RADB): | | http://www.ra.net/cgi-bin/ra/radb-duplicates.pl 20. How do I check what is registered in the IRR? The tool to use is whois. A few examples make the command self explanatory: whois -h whois.ra.net 128.228.0.0 whois -h whois.ripe.net as378 whois -h whois.canet.ca 142.77.0.0 | There is an easy Web interface to query and update the IRR: | | http://nap-roma.uni.net/cgi-bin/whois 21. Is there a tool to automatically create route filters based on IRR information? rlc is a route list compiler which is a subset of nlc/alc that allows the generation of route based filters (cisco access- lists) by extracting the nets belonging to an AS or AS MACRO from a routing database (i.e. Ripe Routing Database). In addition, it supports a limited set of functions to generate AS based filter lists. rlc is fully classless, and hence supports CIDR routes and subnets, as well as host routes. Source: ftp://dxcoms.cern.ch/pub/ripe-routing-wg Author: Jean-Michel Jouanigot, CERN <jimi(a)dxcoms.cern.ch> |22. What other tools are available in the CIDR world? | A web version of this information can be located at: | http://www.ra.net/~ra/tools | | ------------------------ | Online Tools and Reports | ------------------------ | | IRRWeb | | This graphical interface into the Internet Routing Registry makes | it possible to use the Web to query the IRR and update RADB AS | objects, Route objects, and Maintainer objects. Users can enter | any value that can be submitted through a whois query, such as an | AS number, network IP address, or maintainer. IRRWeb then displays | the corresponding AS objects, Route objects, or Maintainer objects | from the various registries in the IRR. Authorized maintainers can | edit the objects directly; IRRWeb performs a cursory pre-check and | mails the revised object to auto-dbm(a)ra.net. The user then | receives e-mail from auto-dbm confirming and displaying the revised | object, or explaining why the object was rejected. | | See http://www.ra.net/cgi-bin/ra/query-radb.pl | | Source code is available at | ftp://ftp.ra.net/routing.arbiter/tools/irrweb.tar.gz | | | Route History Server | | The Route History Server provides a mechanism for tracking | the announce/withdraw history of a given prefix for the last 24 | hours. The History Server peers with the route servers at | each NAP and records all BGP updates in History Server/Route | Server peering session. | | See http://www.ra.net/cgi-bin/ra/rshist.pl | | | ----------------------- | Tools Available for FTP | ----------------------- | | RSd | | The Route Server daemon (RSd) is an enhanced version of | the GateD routing software that provides multiples views of | routing information. | The software provides a mechanism for network service provides | to off-load the computational complexity of routing policy | calculations from their routers. RSd supports configuration and | transparent passing of BGP MEDs. The software also supports | BGP route flap dampening. | | Available at ftp://ftp.ra.net/routing.arbiter/tools/rsd.tar.gz | | | Peval | A policy evaluator that inputs a RIPE-181 policy | expression, performs essential background | calculations such as symbolic evaluations and | expansions, and outputs another RIPE-181 policy | expression that is used by other tools, such as | RTConfig. | | Available as part of the RAToolSet at | ftp://ftp.ra.net/routing.arbiter/tools/RAToolSet | | | RTConfig | A router configuration tool that can be used by | providers to generate router configs directly from | the RADB or other IRR registries. Currently in | production use for the RA Route Servers, ANSNET, and | CA*net, RTConfig is a front-end tool that uses peval | and radbserver transparently to users. | | Available as part of the RAToolSet at | ftp://ftp.ra.net/routing.arbiter/tools/RAToolSet | | | rrc2r/rrmerge | This Cisco-to-RIPE-181 conversion package makes it possible for | users to convert Cisco router configuration files to RIPE-181 | objects that can be submitted to the Internet Routing Registry | (IRR). The Cisco access-list can be converted into an explicit set | of nets embedded in an AS object or represented as a community. | The RADBserver is queried for any missing or previously registered | information. | | Available at | ftp://ftp.ra.net/routing.arbiter/tools/RAToolSet/rrc2r-0.2.tar.gz | | | CiscoBGP | IRR users have long recognized the need for tools that check the | IRR against routes actually propagated in the Internet. CiscoBGP | obtains and analyzes routing information from production Ciscos, | and compares the data with routes in the IRR. The software also | flags prefixes that are reserved by RFC 1597, Address Allocation | for Private Internets. | | Available as part of the MRT software distribution at | http://www.merit.edu/~mrt | | | BGPCheck | BGPCheck obtains and analyzes routing information from BGP4 | peering sessions, and compares the data with routes in the IRR. The | software also flags prefixes that are reserved by | RFC 1597, <em>Address Allocation for Private Internets</em>. | | Available as part of the MRT software distribution at | http://www.merit.edu/~mrt | | | PRtraceroute | The 'prtraceroute' tool is a powerful form of traceroute that | displays the Autonomous Systems that a packet traverses. | PRtraceroute was developed by the Pride project. | | A version of prtraceroute that queries the RADB is available at | ftp://ftp.ra.net/routing.arbiter/pride/prtraceroute-2.0beta2.shar.gz | | | ------------------ | Tool Announcements | ------------------ | PGP-Based Radb Authentication | Follow these steps to take advantage of the new RADB support for | PGP-based digital signatures: | | 1. Register your public key with the Routing Arbiter Service. | 2. Modify your Maintainer object to reflect your use of digital | signatures. | 3. Use PGP to sign your RADB transactions. | | For detailed instructions, see: | http://www.ra.net/routing.arbiter/RA/RADB.tools.docs/pgp.html Contributors: Christian Panigl - Vienna University, Austria Bill Manning - ISI Tony Li - Cisco Systems Havard Eidnes - SINTEF, Norway Yakov Rekhter - Cisco Systems | Craig Labovitz - Merit Network

1 0

99 and 44/100 % pure... [Re: outages, quality monitoring, trouble tickets, etc]
by sgoldste＠nsf.gov 28 Nov '95

28 Nov '95

At 3:03 PM 11/27/95, Jon Zeeff wrote: >Being in the web hosting business, we measure our own "availability" >and that of others. The top providers do 99.9%.... I recall having calculated several years back that ~4 hours of unavailability (to customers) per 30-day month implies 99.44% availability, an Ivory tower number to be sure (are there any of you out there old enough to remember the incessant ad's for Ivory Soap "99 and 44/100 percent pure"?). Sprint proposed such a number to NSF back in 1989 for the international services under the International Connections Management (ICM) agreement. One thing we both learned along the way was that there are factors beyond any carrier's abiolity to control, especially in international (read "multi-player") services. So, for example, when a cisco box needed re-powering in Stockholm or London, but the local staff was gone for the night and nothing would happen until morning, or when the physical circuit passed through Amsterdam or Germany, and a line break depended on PTT Netherlands or Deutsche Telekom to repair, could/should we hold Sprint responsible? We decided to ask Sprint to produce numbers that related to the portion of the end-to-end linnk under their direct control, e.g., ICM box in Washington to ICM box at London demarc. The numbers, of course, looked a lot better. (I cannot recall them anymore.) But even then, we did not impose quality measures like packet drop rates (and, if we had, how could we know if they were caused by traffic congestion or by poor line quality if the link was heavily loaded almost all day?). Just to mudy the waters a bot more... --SG ____________________________________________ Steve Goldstein, National Science Foundation +1(703)306-1949 Ext. 1119 "Let's not procrastinate until tomorrow!"

1 0

Re: outages, quality monitoring, trouble tickets, etc
by kwe＠6SigmaNets.com 28 Nov '95

28 Nov '95

At 5:34 PM 11/27/95, Dave Siegel wrote: >> >> It is because of these sorts of questions that I never know what >> it means when someone says that their network is available 99.9% > >Problems with CPE is usually not figured into network uptime. > I've been away from the NOC for a couple of years now, but one way that made sense for reporting a general availability number is to consider the backbone as one component and each leaf node as an identical independent component (leased line, router and CSU/DSU). Then you can define the end to end availability as the concatentation of two independent leaf nodes and the backbone. Highly redundant backbones always have a high availability (>99%). Leased lines are abominable (low 90s or less -- don't ask me about circuits to Italy) and two routers and two CSU/DSUs lead to an overall end to end of 95% or less. In my experience with leased lines, they always dominated this equation and overall availability is strictly limited by actual leased line availability. So keep that CPE in the equation and figure out how to get highly available bit pipes. I think actual networks like frame relay have a chance of being much more available since the network actually understands frames. Leased lines are simply electrical signalling conduits to the phone company -- not a network. I'm skeptical of any end-to-end availability figures over 97%. I don't think they reflect the reality of leased line circuits today, or else they don't include the leaf node circuits and only report backbone availability. For a highly redundant backbone, almost any definition of availability should result in a number like 99.mumble%. Remember 99.9% availability means less than 9 hours outage per year. Routing hiccups take that much. One or two leased lines outages is all you get for 9 hours. The real world is a lot less available than that. But since half the web servers I try to talk to refuse me half the time, I'm not sure that network availability per se (HWB's complaints duly acknowledged) is the tallest pole in the tent. --Kent

1 0

Re: Links on the blink - what will/should mci & sprint do?
by Tony Li 28 Nov '95

28 Nov '95

[Sorry for being late in responding, I've been on vacation...] Let me start by pointing out that the global problem here is to maintain and grow a routing infrastructure at exponential rates and at "interesting" flap rates. Using a level 2 switching infrastructure as a means of bandwidth control is clearly not a comprehensive solution as it does not provide the necessary level 3 routing [hopefully this is trivially obvious to anyone reading this, but...]. Is a level 2 infrastructure a cost effective means of doing bandwidth control? Possibly, but the exact economics vary widely depending on the level of intra-POP bandwidth necessary and the prices that each ISP is able to obtain. Using a level 2 architecture clearly does nothing to avoid the "brick wall". This becomes quite clear as we can easily demonstrate brick wall phenomena at any interconnect, regardless of the router (or make of router) and the fabric behind it. The question then focuses on the maintainability of the routing infrastructure. We know that we have two limits: perfectly stable routing, and completely unstable routing. Neither is interesting. Perfectly stable routing is simply utopian thinking. Completely unstable routing is unusable even given arbitrarily fast technology. In between, we have an enormous gray area, with no good way to quantify it: routes that flap, and packets that interact with that flap. Clearly if a route flap does not affect any packets, that should not perturb the infrastructure. [We're pleased to say that extensive testing of a 7000 and 7500 points out that in fact, it doesn't. ;-)] For the sake of brevity, let's call this "uninteresting flap." What then is the response of the routing system to intesting flap? We can start by characterizing the flap by T_{down}, the period of time that a route is withdrawn, measured from the "down edge" of the withdrawl of the route to the "up edge" of the installation of the replacement. Subsequent to the "up edge", we have a further T_{recover}, which is the time that it takes for the infrastructure to forward at peak rate again. Using this, we can begin to characterize the behaviors that we'd like. Clearly, during T_{down}, we have a problem: we have no route and (as the interesting part of the problem is in the default-free portion of the net), we have no other shorter prefix for it. One can then either drop packets or forward them along the old route anyhow. The Internet philosophy has always been to drop packets as quickly as possible, as close to the source as possible. The alternative is to forward the packet anyhow, hoping that the packet is not forwarded into a sustained forwarding loop. As the routing protocols aren't truly architected to protect against such a forwarding loop, the latter behavior seems risky. The implication is that during T_{down}, the infrastucture will drop packets from all sources to the destination. Next, consider T_{recover}. As no one has invented FTL hardware (yet ;-), T_{recover} is non-zero on all interesting architectures that I know of. Further, during T_{recover}, packet loss should also be expected. This includes the case of a router which fully distributes the routing table, as the distribution itself takes finite time. What then are interesting values for T_{recover}? Clearly, \infinity is unacceptable. If T_{recover} << T_{down}, it hardly seems relevant. What then is an acceptable upper bound? And what is the sensitivity of this value? I leave this as an open question for discussion... Selective packet discard (a new method we're putting in for prioritizing routing protocol packets over transit packets) will insure that T_{recover} is finite, and based on our lab tests, what I consider to be acceptable. I should note in fairness that credit for driving selective discard should be given to both Sprint and ANS. The former for demonstrating the need in the real world, the latter for focusing cisco's management on the subject. I'd also like to correct some misperceptions that have been distributed: - Distributing the full routing table is not the only way to achieve acceptable response to interesting flap. If someone has a technical argument suggesting that this is the only true way, then we have missed hearing it, and we'd really apprecaite it if you could tell us again. In detail. - Distributing the full routing table to the SSE is not a logical way to proceed. For both obvious and non-obvious reasons, spending significant resources developing software for the 7000 at this point makes little sense, and without some convincing argument that it would move T_{recover} from the unacceptable to the acceptable, is beyond the bounds of rationality. - The interesting flap rate introduced by end users can be controlled by deployment of CIDR. I trust no one on this list remains to be convinced. - Certain statements about communications of problems to cisco mentioned in this forum have been patently incorrect. Sending a mail message to your favorite engineer saying "when will you do XYZ" is not a reasonable way to insure that you're mission critical business feature is being implemented. Still, if you believe you've attempted to communicate with us and you don't think we got the message, then we can only ask that you have patience, increase your retry count, and retransmit. We can't neccessarily do what you might like, but we can hear you and acknowledge your concerns. Finally, I'd like to echo what Dennis said: we're all behind the eight ball, and we all know it. Recriminations are much less interesting than solutions. What should MCI & Sprint do? Well, let's just say that we've privately offered them our opinions, and they don't seem to object too strenuously. Assuming that things actually go that way, well, "You ain't seen nothing yet, baby." Tony p.s. I'm not on the mailing list, so please cc me if you wish me to respond.

4 6

Re: outages, quality monitoring, trouble tickets, etc
by Jonathan Heiliger 28 Nov '95

28 Nov '95

On Nov 27, 7:11pm, Jeff.Ogden(a)um.cc.umich.edu wrote: > It is because of these sorts of questions that I never know what > it means when someone says that their network is available 99.9% > of the time. Hell at least part of MichNet is available 100% of > the time, but that never seems to cut it with the folks who don't > have service right now. I hate to say it, but the relationship in my Email regarding carriers not disclosing network outages could potentially be to maintain the guise of a better than real-life network availability. If we want to enter the tangent discussion that seems inevitable at this point, perhaps we should at least change the subject. :-) -jh-

1 0

Operational declaration of route servers at PacBell NAP
by Jake Khuon 27 Nov '95

27 Nov '95

The Routing Arbiter Service is pleased to announce it is fully in production at the PacBell NAPs. Two route servers are installed at the NAP to provide redundancy, and the are reconfigured every 4 hours. The route server configurations are based on the routing policy information stored in the Internet Routing Registry. The IRR contains the full set of data from five databases: Routing Arbiter Database, RIPE Routing Registry, internetMCI Routing Registry, CA*net Routing Registry, and ANSnet Database. For more information about the RA Service, please refer to: http://www.ra.net/routing.arbiter/RA/ To use the RA Service at the PacBell NAP, send mail to: ra-peer(a)ra.net -- /*====================[ Jake Khuon <khuon(a)Merit.Net> ]=====================+ | Systems Research Programmer, IE Group /| /|[~|)|~|~ N E T W O R K | | Vox: (313) 763-4907 Fax: (313) 747-3185 / |/ |[_|\| | Incorporated | +====[ Suite C2122, Bldg. 1 4251 Plymouth Rd. Ann Arbor, MI 48105 ]====*/

1 0

subscribe
by Bill Price 27 Nov '95

27 Nov '95

please subscribe ------------------------------------- E-mail: bprice(a)bsan.com General Manager, Internet Services BellSouth Date: 11/27/95 Time: 15:56:05 This message was sent by Chameleon -------------------------------------

1 0

Re: outages, quality monitoring, trouble tickets, etc
by Scott Huddle 27 Nov '95

27 Nov '95

Jonathan Heiliger <loco(a)mfst.com> writes > On Nov 27, 9:17am, Sean Doran wrote: > > Jonathan> Everyone likes to portray > > Jonathan> the image of having a 99.98% uptime whenever > > Jonathan> possible, even though most folks realize > > Jonathan> that it just plain isn't possible > > > > Well, more importantly, what on earth does a number like > > that mean? > > Sorry, bad choice of words. Rather than uptime, availability would be the > proper word. Availability tends to be the amount of time the network is > "available" for the customer to receive their expected service (whether > guaranteed in writing or not), and for the customers expectation of how the > service will perform when it is considered "in-service" is met. This is reasonable expectation, but unfortunately it's pretty difficult to measure what "the network" is. Do you mean your NSPs backbone? Its connections to other NSPs? Connections to a specific site? Global connectivity? How do you factor when you or a target site is singly connected? -scott

1 0

Re: outages, quality monitoring, trouble tickets, etc
by Jamshid Mahdavi 27 Nov '95

27 Nov '95

In article <951123045818.17481(a)SDG.DRA.COM> Sean Donelan <SEAN(a)SDG.DRA.COM> writes: > Welcome to the new and improved Internet. More clueless people cal > NOCs these days (is it plugged in?) so more caller screening is done. > Likewise there more clueless people working in NOCs so more levels > before reaching someone who even understands what the problem is. I've had an idea in the back of my head for a long time about dealing with this, ever since I tried to call an 800 customer assistance number and got a nice message saying "the current average wait is 45 minutes"... I guess it could apply equally well to NOCs, so here it is. Provide a "secret" layer 2 NOC. When someone calls the NOC 800 number who is obviously more clueful than the NOC operator/screening person, pass them onto the next level of experts, and also give them a new "secret" 800 number. Then in the future, that caller will skip the first layer of screeners who handle the "is-it-plugged-in" problems. Of course, there is no reason to only have two levels. Callers with proven technical competence (over time) automatically get shunted to help at the right level for their know-how. Periodically, you probably want to recycle the 800- numbers down to the lowest level so that word of the secret numbers doesn't get too widespread. I'd sure like to see someone implement this (or something like it). A few companies we have worked with have had individual techies give us their direct number and say "from now on, just call me direct", but this is rare and probably would be a bad idea for a NOC. --Jamshid

1 0

Re: outages, quality monitoring, trouble tickets, etc
by Sean Donelan 27 Nov '95

27 Nov '95

Sorry, another long one. Relevance to NANOG, well, NOCs are customers too. And "remote" NOCs often report problems that effect your paying customers too. Customer service should be of interest to operations folks, at least to the extent the problems are getting reported to the right people to fix. I doubt I can change anyone's mind that providing explanations to customers and non-customers when the network has problems is good for business. In the future I will simply recommend to customers to buy services from NSPs which do provide explanations when their networks fail. Since I haven't found a perfect network yet, I suspect it includes everyone on this list. > Why should they talk to you? Do you pay them a service fee? > That's my base issue, there is a hierarchy, and you can't skip > rope to the other guy. It just doesn't work, there's nothing in > the system to encourage it. The hierarchy is dead. None of the old NSFnet regional have a monopoly on service in their regions any more. Outside of the US, there are still a few monopoly providers, but they are a rare breed. If you aren't providing the level of service I need, I'll go to someone who can. If XYZ's NOC gives me better service than ABC's NOC, I'll recommend XYZ to my customers. > Sean Donelan has a terribly good point, he's my customer, and his > words mean alot, but I can't agree w/ him that he should/could > demand the same thing from another ex-NSFnet regional, or from > Sprint. I certainly see no reason why I should do this work for > you. Because it is in their self-interest? You are correct I can't make anyone run their network how I would like it run, not even MIDNET (GI). But I can point out long-term problems and code of silence is costing such providers money, and has already cost them customers. For example, I really wish my direct providers would stop munging BGP announcements, or explain why they are doing it. If I have made a mistake, I would like to fix it. Otherwise I will come to the conclusion those provider's NOCs are not up to the job and find a different provider that can do the job. When someone (anyone) reports a problem effecting connectivity with your network, more than likely the reverse is also true for your paying customers. DRA has a bunch of customers connected through just about every major NSP in North America and a couple of other continents. The only time "I" call another NSP is when the process has become totally FUBARed. When I call another NSP, it is usually that NSP's last chance to keep a paying customer on their network. I might call BARRNET because the University of California-Davis has reported problems reaching DRA to DRA's help desk, and the problem hasn't been resolved. No, BARNET doesn't *have* to talk to me. And I will report the same back to the customer. However, I suspect it is in BARRNET's self-interest to work with me in resolving the problem to ensure UC-Davis has end-to-end reliability. I track network reliability by dollars (not packet loss, not latency). I measure network providers, good and bad, by how many of our customers have used their own dollars to buy private lines to St. Louis because they couldn't get the reliability they needed from the network provider. It is not a pretty picture. <http://dranet.dra.com/dranet.html> has a picture where our private line customers are located. If you are an NSP, every one of those green boxes (some boxes represent many paying customers) is an arrow through the heart of your (former) customers view of your network reliability. If you are an NSP in one of those areas, not dealing with these problems or providing coherent explanations has cost you cold-hard cash. Money is something I expect most upper managers to understand. DRA makes its profits elsewhere. DRAnet is simply a vertical market VPN used to sell access to other things. I'm happy to use the Internet and other NSP's to provide that VPN, when the quality exists. On the other hand, if I have to manage a not-so virtual VPN with private lines to achieve the required level of quality, I do. Maybe Adam Smith's invisible hand will correct this eventually. > There seems to be this large obsession with linking information to > action. If you get an update you think something's happening. > Perhaps it's needed, but stuff will happen whether your hand is > held or not. As I said before: Ideally I want a reliable network. If you can't provide a perfectly reliability network I want an explanation when I can't get through. And I want the problem fixed. The better the explanation, the longer I'm willing to give you to fix the problem. If I get no explanation, I expect the problem to already be fixed. The current situation is the customer gets neither the explanation nor action solving the problem. My proof is the DRAnet map. DRA's customers take a very, very long time to budget money. Those green boxes represent customers whose problems went unanswered, and unsolved for a long time before they gave up on their NSP and expended their own dollars for a private line to St. Louis. Since the technicians seem to be having a very difficult time fixing the network, I thought upper management could meet my other goal. Give the customer an explanation. I'm not pointing fingers at any particular NSP, because frankly I don't have enough fingers to point. Everyone had problems. Yes, even DRA's NOC has fallen down a few times. I'm not asking for perfection, but an explanation when things don't work, while you fix the problem. The Internet is a global cooperative network. If people don't cooperate, the global nature of the network fails. Since your customers may in fact want to use the Internet to communicate globally, problems effect customers globally. When I go to the US Post Office, sometimes there is a sign on the wall that postal service to Timbukto may be delayed because Timbukto's main post office was blown up. I have no idea how many postal customers in Olivette, Missouri send mail to Timbukto. Even though the US Post Service has no control over rebuilding Timbukto's main post office, the US Post Service has discovered it is good customer service to inform their customers why their mail to Timbukto may be delayed. Can't NSPs provide their customers an explanation at least as well as the US Post Office? -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

5 4