- Test - lists.nanog.org

seeing the trees in the forest of confusion
by alan＠mindvision.com 26 Apr '97

26 Apr '97

The source was isolated within 60 minutes of the problem. The routes propagated for greater than 120 minutes after that without the source contributing. (yes, the routes with an originating AS path of 7007) This is the interesting problem that noone seems to focus on. I suppose it is more fun to criticize policy and NSPs, but it may well be a hole in the BGP protocol, or more likely implementations in vendor's code [or user's implementation of twiddleable holddown timers]. As well, for your question: ] where were all the network people .... The people who could and did fix this problem were on a bridged (voice) conference call understanding what was going on, sharing information and working to resolve the issues. The 24 hour NOC functionality worked quite properly. There is a balance between informing people what is going on and working to fix the problem. Most people would prefer the problem be resolved and the postmortem take lengths of time than to prolong the incident to inform the masses of what actions are being taken. Having intelligent people answer the phone and explain what was going on wouldn't have helped solve the problem, just make people feel better. You could achieve the same feeling by taking a walk in the outdoors. This wasn't an Internet outage, this wasn't a catastrophe. It was a brownout that sporadically hit providers at various strengths. At least one NSP measured their backbone load go down by only 15% during the incident. The Internet infrastructure did not react as expected to this technical problem. It continued to propagate the bad information when withdrawls should have caught up and canceled the announcement. To make a security analogy, an entity designs a security policy inherent to risks. We openly acknowledged that this could happen and would hurt. The risk assesment was not accurate because the infrastructure did not behave as expected. We did not expect it to hurt for 3 hours. It should have stopped earlier. Why it didn't is the only interesting question left. -a ] There are already 2 articles on the net about it ] ] http://www.news.com/News/Item/0,4,10083,00.html?latest ] http://www.wired.com/news/technology/story/3442.html ] ] and I am sure there are more to come. It seems too easy for one person/company ] to bring down the net. Yes, we all agree it was some kind of accident, one ] that wasn't supposed to happen BUT I have to ask, where were all the network ] people that should have caught this before it hit the internet full force. ] ] Yes, there is talk about having the RAs setup and to prevent this type of ] thing, but not much else. We put our stuff in the RA when we remember to but I ] don't think that many people look much at them anyways. ] ] Hopefully next time this can be stopped before long, like maybe within 10 - 20 ] mins. Maybe it is time for all companies, even the smaller ones, or those that ] use BGP to have and really USE a 24 hour noc. Where there are people there ] that really know about routers and BGP. Maybe as the internet grows, those ] making it grow need a more active role. Sorry, Voice mail doesn't cut it, or a ] busy number when the internet comes down.... ]

7 7

RE: offtopic for NANOG - do not read
by Jim Fleming 26 Apr '97

26 Apr '97

On Saturday, April 26, 1997 3:22 PM, Dave Crocker[SMTP:dcrocker@brandenburg.com] wrote: @ At 12:40 PM -0700 4/26/97, Karl Denninger wrote: @ >For 90% of that database, the development and operation was paid for with @ >*private funds* as a *private, for-profit, revenue-funded* business. @ @ under a government agreement, with a government assignment of @ responsibility and government review and approval of the fees. @ @ the agreement, by the way, says that the data belong to the government. @ @ >NSI put up the risk capital to do this. They did so with no guarantee @ >of a profit, and in fact claim they have lost money. @ @ no, really they didn't, since the captial came from the revenue stream. @ The "capital" came much earlier than the charging.. March 1992 Original NSF Solicitation <http://rs.internic.net/nsf/solicitation.html> May 1992 Original NSI Proposal October 1992 Modified NSI Proposal <http://rs.internic.net/nsf/nis/proposal-toc.html> "Network Solutions believes NSF's objectives will be met most effectively by the award of the bulk of the services to a single contractor." <http://rs.internic.net/nsf/nis/sectionM.html> "Network Solutions proposes Mr. Jon Postel as the IANA Manager and Chairman of the Advisory Panel for the NREN NIS Manager project. He will provide services as an employee of USC's Information Sciences Institute (ISI), subcontractor to Network Solutions." AT&T Proposal <http://ds1.internic.net/internic.info/proposal/> January 1, 1993 InterNIC == (General Atomics + AT&T + NSI) Cooperative Agreements General Atomics (GA) ??? AT&T ??? Network Solutions, Inc. (NSI) <http://rs.internic.net/nsf/agreement/> Estimated Total Amount: $4,219,339 Effective Date: January 1, 1993 Expiration Date: September 30, 1998 <1993...period of "cooperation"..1994> General Atomics was supposed to be the NIC of NICs and to help coordinate the activities of all three contractors. More NICs were supposed to be formed. December 1994 Midterm Evaluation <http://www.rs.internic.net/nsf/review/review-toc.html> "The InterNIC awards set the precedent of requiring significant self-coordination among a team of awardees, and requiring outreach to other Network Information Centers. The panel suggests that the NSF critically consider whether it is viable to expect significant self-coordination among a team of awardees in future awards. The panel also notes that the NSF's program management was not able to correct GA's problems early on despite excellent efforts by the NSF staff, primarily because the NSF staff were overextended by monitoring at least two major projects (the InterNIC and the NSFNET backbone) at once. The panel recommends that for future large scale efforts in the rapidly changing Internet environment, the NSF should form an ongoing advisory panel of outside experts or employ some external consultants to help manage such cooperative agreements, rather than waiting two years to call for a review." December 1994 General Atomics Dismissed InterNIC == (AT&T + NSI) January 1995 NSF gives NSI more money... Amendment 1 <http://rs.internic.net/nsf/agreement/amendment1.html> This amendment increases the funds available under Cooperative Agreement No. NCR-9218742 by $1,258,457. NSI changes key personnel... Amendment 2 <http://rs.internic.net/nsf/agreement/amendment2.html> February-March 1995 (???estimated date???) SAIC purchases NSI http://www.saic.com http://www.netsol.com Circumstances, terms, previous owners....???? not known ???? May 1995 NSF gives NSI more money... AND reviews NSI's plan to start charging Amendment 3 <http://rs.internic.net/nsf/agreement/amendment3.html> "The National Science Foundation (NSF) hereby awards $1,948,632 to Network Solutions, Inc. for additional support..." September 13, 1995 NSF approves NSI's plan to charge for domain name registrations <http://rs.internic.net/nsf/agreement/amendment4.html> October 1995 Many people and companies debate the issues surrounding the commercialization of domain name registrations. The NSF and the IANA prevent commercial registries from having new Top Level Domains. November 1995 Debates December 1995 Debates January 1996 Debates February 1996 Debates and delays March 1996 Debates and delays April 1996 Debates and promises of plans May 1996 Debates and more promises of plans June 1996 ISOC Board of Trustees - ANNUAL GENERAL MEETING Montreal, Quebec, Canada <http://www.isoc.org/trustees/96-004.htm> "RESOLVED, that the Board of Trustees of the Internet Society endorse in principle the proposal "New Registries and the Delegation of International Top Level Domains", dated June 1996 by Jon Postel, and approve the role assigned to the Internet Society in this proposal. The Board authorises Postel, in his IANA role, to refine the proposal to include a business plan for review and approval by the Board." July 1996 <http://www.verisign.com/pr/net_sol.html> NETWORK SOLUTIONS AND VERISIGN PROVIDE "ONE-STOP REGISTRATION" FOR INTERNET NAMING AND SECURITY Network Solutions to Make VeriSign Digital IDs SM Available with Domain Name Registration Debates and refinement of plans 8 - Intermiediate Version of New Registries Plan - Jon Postel <http://www.iiia.org/lists/newdom/current/0518.html> 16 - Stop this silliness - Michael Dillon <http://www.iiia.org/lists/newdom/current/1049.html> 16 - Issues on the table - Michael Dillon <http://www.iiia.org/lists/newdom/current/1052.html> 16 - TLD registration with AlterNIC - Chris Ambler <http://www.iiia.org/lists/newdom/current/1060.html 16 - Re: Issues on the table - Bill Manning <http://www.iiia.org/lists/newdom/current/1118.html> 19 - Inside the IANA - Jim Fleming <http://www.iiia.org/lists/newdom/current/1336.html> 29 - naughy pine - Bill Manning <http://www.iiia.org/lists/newdom/current/2269.html> 30 - do I really want to do this? - Bill Manning <http://www.iiia.org/lists/newdom/current/2352.html> newdom mailing list closed at iiia.org - July 30th August 1996 Debates, meetings and claims of an October 1st start From CNET - IANA to take applications in October <http://www.ar.com/lists/newdom/0609.html> 8 - rolling over - Chris Ambler <http://www.ar.com/lists/newdom/0114.html> September 1996 Operational registries start to appear...the industry is ready... The IANA and the ISOC now claim an "ad hoc" committee is needed... ...more delays... October 1996 The balance of the Intellectual Infrastructure Fund is disclosed... <http://rs.internic.net/announcements/iif-update.html> "..through October 31, 1996, $8,542,200.00 has been deposited into the account." The Federal Networking Council Advisory Committee advises the NSF <http://www.fnc.gov/FNCAC_10_96_minutes.html> "The FNCAC reiterates and underscores the urgency of transferring responsibility for supporting U.S. commercial interests in ITLD administration from the NSF to an appropriate entity." November 1996 "...through November 30, 1996, $9,911,000.00 has been deposited into the account." Commercial TRUE Root Name Servers start to appear... the NSF does not allow commercial registries to have TLD entries in the Root Name Servers they control... The NSF awards USC/ISI $1.5 million <http://www.nsf.gov/ftp/awards96/awd9615/a9615927.txt> December 1996 "...through December 31, 1996: $12,685,450.00 has been desposited into the account." The ISOC's IAHC gets started to provide recommendations to IANA http://www.iahc.org NSI ends 1996 registering about 80,000 names per month which is an approximate gross revenue of $8,000,000 per month at $100 per registration. That covers the first 2 years. On an annual basis, this amounts to approximately $48,000,000 per year. January 1997 IAHC debates continue... Plans are announced by NSI to launch ARIN to charge for IP addresses <http://www.arin.net> NSF gives NSI more money... <http://rs.internic.net/nsf/agreement/amendment5.html> NSI and the IANA announce plans to deploy TRUE Root Name Servers February 1997 IAHC recommendations to the IANA finalized <http://www.iahc.org/draft-iahc-recommend-00.html> Commercial registries which are now fully operational are still restricted from having their names entered into the widely used Root Name Servers controlled by the National Science Foundation. Companies are now actively deploying commercial Root Name Servers to replace the NSF supported servers. February 27, 1997 I/O Design Announces Lawsuit <http://www.iahc.org/iahc-discuss/mail-archive/1815.html> <http://www.iodesign.com/complaint.html> OIG NSF InterNIC Report Complete - Awaiting Response February 28, 1997 Summary <http://www.iahc.org/iahc-discuss/mail-archive/1873.html> March 1997 First eDNS Meeting in Atlanta Georgia <http://www.edns.net> IAHC announces they are closing their mailing list IAHC and ITU announce supposed signers to MoU Some announce signers publicly withdraw April 1997 eDNS continues to grow and more TLD registries started Additional eDNS Root Servers being tested ISP/C announces they will not support the IAHC NSF announces it will not rebid the InterNIC contract < YOU ARE HERE> September 1998 Cooperative Agreement Ends... -- Jim Fleming Unir Corporation http://www.Unir.Corp Check out...http://www.Naperville.Mall

1 0

RE: NEWDOM: Re: offtopic for NANOG - do not read
by Jim Fleming 26 Apr '97

26 Apr '97

On Saturday, April 26, 1997 8:06 AM, Michael Dillon[SMTP:michael@memra.com] wrote: @ On Sat, 26 Apr 1997, Karl Denninger wrote: @ <snip> @ @ The real question is, which individuals and organizations take their @ positions of "public trust" seriously and which ones are attempting to @ leverage such a position into lining their own pockets? @ @ So far, Postel, Vixie, et al., are the guardians of the public trust @ and all those who are lined up in opposition to them appear to be more @ concerned with lining their own pockets than anything else. @ @ Michael Dillon - Internet & ISP Consulting @ Memra Software Inc. - Fax: +1-250-546-3049 @ http://www.memra.com - E-mail: michael(a)memra.com @ @ @ Who buys your routers Michael...???? @@@@@ http://www.nsf.gov/ftp/awards96/awd9615/a9615927.txt Title : Testbed Routers for Advanced Internet Lab (TRAIL) Type : Award NSF Org : NCR Latest Amendment Date : August 27, 1996 File : a9615927 Award Number: 9615927 Award Instr.: Standard Grant Prgm Manager: Darleen L. Fisher NCR DIV OF NETWORKING & COMMU RES & INFRASTR CSE DIRECT FOR COMPUTER & INFO SCIE & ENGINR Start Date : October 1, 1996 Expires : September 30, 1999 (Estimated) Expected Total Amt. : $1,499,999 (Estimated) Investigator: Herbert Schorr schorr(a)isi.edu Allison Mankin Sponsor : U of Southern California University Park Los Angeles, CA 900074363 213/743-2311 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ -- Jim Fleming Unir Corporation http://www.Unir.Corp Check out...http://www.Naperville.Mall

1 0

Outage and IPv6
by Jim Fleming 26 Apr '97

26 Apr '97

Has anyone analyzed what impact the recent outage had on the IPv6 community...6-bone ? Since IPv6 is largely tunneling via the IPv4 network, does a serious "brown-out" of the IPv4 network have any impact on the 6-bone network ? If enough of these sort of events occur, will this drive more people to IPv6 ? Assuming that the IPv6 network would not have seen any impact. -- Jim Fleming Unir Corporation http://www.Unir.Corp Check out...http://www.Naperville.Mall

2 1

Re: NEWDOM: Re: offtopic for NANOG - do not read
by Carl Oppedahl 26 Apr '97

26 Apr '97

At 01:46 PM 04/26/97 -0500, Karl Denninger wrote: >On Fri, Apr 25, 1997 at 11:16:39PM -0700, Paul A Vixie wrote: >> > There's really no reason any of us should be worried about NSI and the >> > IAHC competing for domain name registries. But NSI is also pushing for >> > control over the root domain (.) and if things get too heated up over that >> > dispute we really could see a partition of the Internet at the DNS level. >> >> This is not a concern. The DNS software vendors, and the root servers, all >> follow the IANA. When IANA says "go", we go. If NSI says "go somewhere else" >> then they become irrelevant, and they become a travesty, and their gravestone >> becomes another mile marker on the Internet's road to glory. >> >> This means "." moves to IANA-operated servers as soon as IANA says so. >> >> This means "COM" moves to IANA-approved servers as soon as IANA says so. >> >> This means NSI had best fire all of its lawyers and maybe its President/CEO. >> >> Internet B-I-G, NSI "little". >> >> The DNS namespace is a public trust, not a corporate asset. "Get over it." [ad hominem attack deleted] >I'd like you to point out the major corporations and public universities who >will do this. I'd also like you to immediately return that nice root server >that NSI has paid for in part or whole, if you really believe this. NSI has never paid for anything. NSI has doled out money collected by it from NSF and from domain name owners. NSF (i.e. the US taxpayers) and the COM, NET etc. domain name owners have paid for whatever it is you think NSI has paid for. >Anyone trying to take "COM" and point it somewhere else will find that they >have created a class-action lawsuit with 1,000,000 plaintiffs -- all the >people who you instantly disconnect that have COM domains. Wrong. Nothing is going to get disconnected at such time as the roots get redirected to NSI's successor. That successor will have available to it the COM (and NET, etc.) database and there will be no interruption in service. Someone is going to be the successor to NSI in administering COM when NSI's contract runs out. Whoever that is, NSI is obligated to hand over the database to them. NSI was granted a five-year contract with a known expiration date. NSI knew that when it signed the contract. Someone was administering COM before NSI's contract began, and whoever that was, they handed over the COM database to NSI when that contract began. Someone will be administering COM after NSI's contract ends, and NSI will (indeed, must) hand over the database to that successor. >Anyone trying to STEAL NSI's COM zone (to appropriate it as their own) will >likely find themselves on the wrong end of a monstrous lawsuit, not to >mention potential felony theft charges. NSI doesn't own that. It is owned by its authors, namely the domain name owners who filled in their domain name registration and update templates. [second ad hominem attack deleted]

2 1

Re: seeing the trees in the forest of confusion
by Sean Donelan 26 Apr '97

26 Apr '97

> The source was isolated within 60 minutes of the problem. > > The routes propagated for greater than 120 minutes after that > without the source contributing. (yes, the routes with an > originating AS path of 7007) > > This is the interesting problem that noone seems to focus on. This has happened before. I had a problem with routes incorrectly being advertised by another ISP(GI/MIDNET). The problem was quickly identified, and corrected. An old set of filters created by an engineer that had left the company was incorrectly configured. However this ISP had transit from Sprint. A Sprint router a couple of hops away kept advertising the bogus routes for two days before I could finally get someone in Sprint to believe a problem existed, and shoot the router. Why it took two days is a story in itself. > We did not expect it to hurt for 3 hours. It should have stopped > earlier. Why it didn't is the only interesting question left. Perhaps if you had shared the information more widely, someone who had seen a similar problem before could have helped. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

1 0

Also, re: 7007
by Avi Freedman 26 Apr '97

26 Apr '97

Net Access, AS 4969, is now announcing: mae-netaxs#sho ip bgp reg 6082 BGP table version is 8459301, local router ID is 207.106.124.6 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 205.147.160.0/19 192.41.177.74 1 0 6082 ? *> 207.152.191.0 192.41.177.74 1 0 6082 ? *> 209.50.192.0/19 192.41.177.74 1 0 6082 ? mae-netaxs#sho ip access 68 Standard IP access list 68 permit 205.147.160.0, wildcard bits 0.0.31.255 permit 209.50.192.0, wildcard bits 0.0.31.255 permit 207.152.191.0, wildcard bits 0.0.0.255 permit 199.233.96.0, wildcard bits 0.0.0.255 deny any For ASN 6082 (MAI). This is a subset of routes to take care of some of their customers at another location. We're filtering their inbound routes and not announcing anyone's routes to them, FYI. Please don't filter _6082_. We're only announcing routes through distribute-lists, and MAI promises to be very careful with what they announce through 6082. MAI is still working with Bay at their data center to reach the point that they're comfortable re-advertising their routes directly. Ciscos have been offered to MAI, but MAI is still working with Bay. Avi

2 2

RE: Read this...Re: offtopic for NANOG - do not read
by Carl Oppedahl 26 Apr '97

26 Apr '97

At 03:50 PM 04/26/97 -0500, Jim Fleming wrote: >On Saturday, April 26, 1997 12:54 PM, Carl Oppedahl[SMTP:carl@oppedahl.com] wrote: >@ Thus, it seems likely that (barring a shift in control over .COM) NSI would >@ continue forever, carrying out its awful policy and putting innocent domain >@ name owners out of business. >@ >@ The only way that innocent domain name owners will have the cloud of NSI's >@ awful domain name policy lifted from their heads is if indeed the control >@ over .COM shifts elsewhere. So far as I can see, the most likely way for >@ that to happen is if IAHC's plans move forward and are put into place. > >People and companies can now register in >other domains than .COM. > >Why would a company want a .COM domain >when they can have something more expressive ? I guess you don't get it. Once a company has invested years in its business position in reliance on having a particular domain name, has invested years and energy and money getting people to put the company's URL into their bookmarks and getting people to set up links to the company's URL, then switching to some other domain name is out of the question. Similarly, if a company is an ISP with thousands or millions of customers whose email address ends with the company's domain name, then the company simply cannot change domain names. It is out of the question. Some of the companies I am thinking of signed up for their domain name *prior* to the start of NSI's five-year contract. Many of the companies I am thinking of signed up for their domain name prior to the July 28, 1995 start date of NSI's awful policy. Yet they are stuck with NSI's wacko handling of trademark challenges. It is all fine and good to talk about the domain name one *might* choose if starting a new business a year from now. But I am talking about present members of the Internet community whose businesses are subject to NSI's whim with respect to domain names. And whose businesses are subject to the truly scary prospect of NSF scuttling under the furniture in April of 1998, leaving NSI answering to no one, charging whatever it pleases for continued use of a domain name, and developing new and even worse ways of handling trademark challenges. For all these reasons, control of .COM has to get shifted away from NSI. And it is interesting but irrelevant to talk of other TLDs that may some day exist.

1 0

RE: NEWDOM: Re: offtopic for NANOG - do not read
by Carl Oppedahl 26 Apr '97

26 Apr '97

At 02:59 PM 04/26/97 -0500, Jim Fleming wrote: >On Saturday, April 26, 1997 11:49 AM, Carl Oppedahl[SMTP:carl@oppedahl.com] wrote: >@ At 01:46 PM 04/26/97 -0500, Karl Denninger wrote: >@ ><snip> >@ >@ >I'd like you to point out the major corporations and public universities who >@ >will do this. I'd also like you to immediately return that nice root server >@ >that NSI has paid for in part or whole, if you really believe this. >@ NSI has never paid for anything. NSI has doled out money collected by it >@ from NSF and from domain name owners. NSF (i.e. the US taxpayers) and the >@ COM, NET etc. domain name owners have paid for whatever it is you think NSI >@ has paid for. >Carl, >This is not quite true. >Paul Vixie has stated on the NANOG list that NSI >pays for his equipment. I have no doubt the check was from NSI. I am talking about where the money actually came from, and the answer is either (1) NSF (i.e. the US taxpayers) or (2) the domain name owners paying their $100 or $50 fees. >Paul is part of the Internet >Software Consortium (ISC) <http://www.ispc.org> >an IRS approved 501(c) company that accepts >donations so that Paul can write software that >supports the domain name system. > >Their web site says they have a $700,000/year budget. > >According to the IRS, you can get full-disclosure >on all of this. So what? Nobody doubts that the check Paul Vixie received was written by NSI -- if that's what he says, I believe him. No amount of full disclosure under 501(c)(3) is necessary to learn whose name was on the check. What I am talking about is the source of the money. NSI got it from you and me when we paid our $100 and our $50, or when they collected it from NSF (and thus from you and me).

1 0

Re: seeing the trees in the forest of confusion
by Russ Haynal 26 Apr '97

26 Apr '97

At 09:22 AM 4/26/97 -0500, Alan Hannan wrote: > This wasn't an Internet outage, this wasn't a catastrophe. It was > a brownout that sporadically hit providers at various strengths. > At least one NSP measured their backbone load go down by only 15% > during the incident. To go along with that... The MAE-EAST statistics (graph) also shows only a modest drop in traffic for a little over an hour... http://www.mfsdatanet.com/MAE/east.giga.overlay.html Russ __________________________________________________________________ Russ Haynal - Internet Consultant, Instructor, Speaker "Helping organizations gain the most benefit from the Internet" russ(a)navigators.com http://www.navigators.com 703-729-1757 ------------------------------------------------------------------ Author:"Internet; A Knowledge Odyssey" (Top-rated CD-ROM Tutorial) Available from MindQ Publishing: http://www.mindq.com

1 0