- NANOG - lists.nanog.org

TCG Fiber Cut: San Diego <-> Orange & Anaheim
by Kit Knox 15 Jul '99

15 Jul '99

There is currently a TCG fiber cut affecting at least 45 DS3's between San Diego <-> Orange & Anaheim. We lost circuits at 9:30AM PDT. Anyone have any details? -- Kit Knox <kit(a)connectnet.com>

1 0

revoked .edu's: does it happen?
by steve rader 15 Jul '99

15 Jul '99

Isn't it Jeffery Case that sez "as we say down on the farm, reality diverges greatly from reality"? Has anyone ever heard of a .edu domain being delegated and subsequently revoked because the owner's organization was not a four-year degree granting institution?? [My apologies for injecting some non-operational noise.] later steve - - - systems guy wiscnet.net

2 1

Network Solutions loses domain data -- Follow-up
by Daniel Senie 15 Jul '99

15 Jul '99

This is a follow-up to a message I posted earlier today. After a 24 hour absence from the root name servers, the missing domain (senie.com) has once again appeared. The lesson to be learned, it would seem, is to avoid doing domain updates around the time Network Solutions generates the root zone files (around 5PM Eastern Time). If I had to wager a guess, I'd suspect they've got a database/programming access issue. I suspect my record was being updated by one program, and thus was locked, at the moment another program was trying to run a report which generates the zone files. I tried to find someone at Network Solutions to mention this to, but there wasn't a clueful person in the dozen or so I talked with. Indeed, most of them didn't understand the relationship between their databases and the root name servers (I think some of them didn't know what the root name servers were). Given a total lack of accountability to anyone, and software that's clearly got some interesting bugs, it's kind of surprising more folks don't get clobbered by this sort of thing (perhaps they do, but don't notice the occasional 24 hour dropout). The sad thing is that competitive registries will likely only make things worse. In that case, you'll yell at the folks you registered with, and they would then have to yell at Network Solutions, who're going to maintain the database with the same buggy software. Sounds like a mess. -- ----------------------------------------------------------------- Daniel Senie dts(a)senie.com Amaranth Networks Inc. http://www.amaranthnetworks.com

3 2

Network Solutions loses domain data
by Daniel Senie 14 Jul '99

14 Jul '99

Last night, I did a number of updates to domains, adding a few new name servers as secondaries, and removing one of the old primary servers. I got confirmation for all the changes, and everything looked OK. I went to bed. This morning, I got a number of phone calls from customers who are having all kinds of problems. It seems that the update for one domain, SENIE.COM, didn't get propagated into the root zone file. The old data is gone. The new data isn't there. My business isn't functioning well. It would appear there is something wrong with the program which creates the root zone files from the Network Solutions database. It would be nice if I could find someone inside NSI who understood this process and could or would do something about it. Instead, I talked with 8 or 9 people, all of whom tried to convince me it was "just a propagation problem." After explaining why it wasn't a propagation problem (the zone has been alive for many years, and so the old data SHOULD be there even if the new data hadn't gotten in), they agreed it seemed like a bug, but didn't know what to do about it. The domain in question is "senie.com" and while that domain has little in and of itself of importance, it's the domain where my name servers and mail server live, and a lot of customer email travels through there. I've updated all of my other zones to provide alternate paths for email, but some activity does rely on senie.com resolving. So, I've wasted a day of my life trying to get the problem solved. The best anyone at NSI could say is to see if it gets updated at 5PM today when they run the update. If it does, then that'll be the solution. If it doesn't, I'm basically screwed for yet another 24 hour period. The claim is there's no way to fix a screw-up mid-day. Without such a method, how's it even possible to know if the same screw-up will occur on the next update? The Internic operates with the same monopoly "we don't have to care" attitude as the telephone companies do, with the exception that in the case of the phone companies the public utilities commission is generally able to apply pressure and get a resolution. There appears to be no oversight of Network Solutions whatsoever, and no interest on their part in proactively fixing problems they've created. If anyone has any suggestions or ideas on getting access to someone with a clue at NSI, or has any method for helping cure this problem, I can be reached at: dts(a)amaranth.net (since dts(a)senie.com won't resolve presently) or 978-779-6813 Dan -- ----------------------------------------------------------------- Daniel Senie dts(a)senie.com Amaranth Networks Inc. http://www.amaranthnetworks.com

7 6

BGP Peering Info
by Mark Cobb 14 Jul '99

14 Jul '99

I am trying to find out if any upstream providers that offer BGP to downstream providers, use any type of "form" ? I am curious as to what type of questions are asked. For example; full route or partial route (if partial, how many), do they offer route reflectors, confederations, multi-homing, or route filters?? If anyone knows of an upstream provider that offers BGP peering and has a form that I could look over, I would greatly appreciate that. Thanx

6 5

Proposal for mitigating DoS attacks
by Alex.Bligh 14 Jul '99

14 Jul '99

------- Blind-Carbon-Copy X-Mailer: exmh version 2.0.2 2/24/98 From: Alex Bligh <amb(a)gxn.net> To: amb(a)Gxn.net Subject: Proposal for mitigating DoS attacks Reply-To: amb(a)gxn.net Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sat, 10 Jul 1999 17:50:06 +0200 Sender: amb(a)gxn.net Warning: This message contains operational content. I'd invite comments on an idea I had to mitigate the effect of Denial of Service attacks. I've outlined it below. Alex Bligh GX Networks (formerly Xara Networks) Using a Well Known Community to mitigate the effects of a Denial of Service Attack. ========================================================= Author: Alex Bligh (amb(a)gxn.net) Version: 0.01 Date: Sat 10 Jul 1999 Background - ---------- Most Denial of Service (DoS) attacks consist of a large number of packets directed at a single or small number of specific hosts. The constituent packets often have spoofed source IP addresses, and the attacks may come from multiple sources, or via third party reflectors. In order to trace the attacks, even where technically possible, multiple NOC's have to be called and, due to difficulties in human-to-human interaction, as well as technical difficulties, it is difficult, and in practice often impossible, to trace and stop the attack in a timely manner. Thus, whilst tracing the ultimate perpetrator is obviously preferable from a legal and moral standpoint, from an operational standpoint much benefit can be gained from mitigating the effects of the attack. Many attacks have effects on hosts other than the target host. As the traffic to the victim host is increased, not only will the performance of that host suffer, but also the victim's entire connection to the net. It is possible that parts of the downstream ISP's backbone have lower available capacity than the magnitude of the attack, and that the consequent congestion or extra switching load will cause degradation to other traffic, or even link failure. This effect is referred to below as 'collateral damage'. An entire customer, or even an ISP, can be severely affected or cut off from the net by a concerted attack on a single host unless preventative measures are taken. Various heuristic measures are possible - rate-limiting ICMP inbound at network borders is in general an effective way to limit the damage done by a SMURF attack, for instance. However, in general, the best way to reduce the collateral damage component is to block the attack as close to its sources as possible. This in general requires working through the same set of tracing procedures as finding the perpetrator, and is often impractical. Thus an oft-used response to an attack is to block traffic either to, or from, particular IP addresses. In the case of attacks involving forged source IP addresses, or reflected attacks such as SMURF, the only way to easilly block these attacks to prevent collateral damage, is to prevent all traffic from reaching the IP address concerned (filtering) until the attack has ceased (either as a consequence of a parallel act of tracing, or otherwise). However, it is often insufficient for the victim to block the attack at their ingress point, as the magnitude of the attack may be larger than their connection to their ISP. It is thus often useful for the ISP to block the attack instead. However, the ISP suffers the same problem, and to minimize the effect on their backbone, will want to block the attack at their network boundaries (i.e. at the points of interconnect with their upstreams and peers). A very large attack may require upstreams and peers to put in the same measures. Currently, the procedures for implementing these measures are, to say the least, manual, and rely on levels of interprovider communication which appear not to prevalent in the industry in general. This proposal describes a method to implement blocking of particular IP addresses across large portions of the internet using existing technlogy and protocols, and without the need for interprovider communication beyond the initial setup. Use of a Well Known BGP Community ================================= The core concept behind this proposal is the propagation of /32 routes (i.e. host routes) throughout large proportions of the BGP-speaking set of ISPs, through peering and transit relationships. Each of these routes will be tagged with a well-known community. For the purposes of this document, the community will be referred to as 65534:1. Each route so tagged will be referred to below as a "Victim Route". Throughout each provider's iBGP mesh, where a Victim Route is received, traffic to that destination is discarded (examples of configuration for a Cisco follow). These routes are propagated using the same rules as the propagation of the supernet to which they belong. By this technology, anyone receiving a supernet advertisement will also receive any relevant victim routes, and discard the traffic. A discussion on Route Filtering =============================== Responsible transit providers filter the routes from their customers. Most ISPs apply some form of filtering to their peers. Normally neither form of filtering allows for /32s (host routes) to be admitted. Indeed many ISPs specifically reject routes longer than a /24. This proposal does not invalidate the concept of route filtering. In fact it is vital that the same level of filtering is applied to Victim Routes as to the superblock in which they reside; elsewise they could themselves be used by irresponsible people as a Denial of Service attack. The same technology that currently ensures ISP's do not lose connectivity to their customers by accepting similar routes from their peers can be used to filter acceptance of Victim Routes. Filter lists should be deployed which admit /32s only if they are tagged with the relevant community, and only if they are subnets of a block which would otherwise be accepted by the BGP peer. Increase in size of the Routing Table ===================================== Whilst it is inevitable that this proposal would increase the size of the routing table, and the volume of advertisements, it has some gains in this respect too. The main reason to restrict routing table growth is to minimize CPU load on routers. The CPU load gains from not switching DoS traffic, and from the consequent saving of BGP advertisements from lines flapping under excess congestion, is almost certainly well worth a few extra transient advertisements. Example configurations for a Cisco Router ========================================= Whilst this proposal is no doubt applicable on other vendors, it has only thus-far been tested on Cisco IOS 11.1.26CC1 running CEF. The above configuration on a 7507 happilly sinks at least 10Mb/s of ICMP Denial of Service traffic without measurable CPU impact. Using an ASN of 5555, the configuration router bgp 5555 neighbor 5555-IBGP-PEER route-map IN-5555-IBGP-PEER in is added to the IBGP peer group. ip community-list 9 permit 65534:1 is used to detect the Well Known Community. route-map IN-5555-IBGP-PEER permit 10 match community 9 set ip next-hop 10.1.1.1 ! route-map IN-5555-IBGP-PEER permit 20 ! is used to set the next hop to an unused IP address when received from any other IBGP peer. A similar route-map can be used with external peers. ip route 10.1.1.1 255.255.255.255 loopback0 ensures packets with this next hop are discarded in a CPU efficient manner. Where connecting to a customer who wishes to control their own blackholing within your AS (we assume here they have a single network 200.100.0.0/16) a configuration like the following is used: ip prefix-list CUSTOMER seq 5 permit 200.100.0.0/16 ! ip prefix-list CUSTOMER-BLACKHOLE seq 5 permit 200.100.0.0/16 ge 32 ! route-map IN-5555-CUST-CUSTOMER permit 5 match ip address prefix-list CUSTOMER-BLACKHOLE match community 9 [only necessary if customer is sending communities] set ip next-hop 10.1.1.1 set community 65534:1 [only necessary if customer doesn't send communities] ! route-map IN-5555-CUST-CUSTOMER permit 10 match ip address prefix-list 191 CUSTOMER match as-path 123 set community <whatever> ! router bgp 5555 neighbor 1.2.3.4 remote-as 1234 neighbor 1.2.3.4 route-map IN-5555-CUST-CUSTOMER in This ensures that routes received from the customer are appropriately dealt with and tagged. Alex Bligh ------- End of Blind-Carbon-Copy

9 9

Re: BGP Peering Info
by Rich Sena 13 Jul '99

13 Jul '99

Actually with that in mind - I'm interested in the same and in a similar manner the same for cross-connect agreements and such - any and all input would be appreciated - please as always - hit me privately if there is any interest I'll summarise to the list... -- I am nothing if not net-Q! - ras(a)poppa.thick.net On Tue, 13 Jul 1999, Mark Cobb wrote: > that offers BGP peering and has a form that I could look over, I would > greatly appreciate that. Thanx

1 0

Direct peering vs. transit peering?
by Dustin Clampitt 13 Jul '99

13 Jul '99

Greetings: On the west coast we currently have a both a transit connection via SAVVIS and multiple peering relationships at the PACBELL NAP. It has been suggested that the PACBELL NAP connection may be superfluous since most of the people with whom we peer can be reached at the cost of one extra hop via SAVVIS. Recently on NANOG there has been discussion of ISPs expanding their peering relationships as a hedge against the possibility of Tier 1 providers eliminating non-profitable peering relationships and adversely impacting routing to sections of the Internet. Whether this enhanced peering strategy applies to us is debatable. I personally believe that we should maintain existing and foster new peering relationships. We purchase transit from multiple providers, but I think the direct peerings have both a marketing and strategic value. I would be interested in hearing arguments supporting both sides of this question. Should we continue to peer, or should we effectively sub-contract our peering relationships to our transit providers? Our business is primarily a dial access business. We don't host a significant number of web sites. As we deploy DSL I expect that will change. Sincerely, Dustin Clampitt Ziplink Inc. (I leave on vacation in 5 minutes. I won't be able to respond to discussion, suggestions, flames, etc. until next Monday. Have a quiet week. - Dustin)

1 0

CA electricity and DoD Y2K on July 13
by Sean Donelan 13 Jul '99

13 Jul '99

Events to be on the look out for Today, July 13, 1999. If nothing happens, this was not an operations related message. On the other hand, if something happens... California's Independent System Operator (the organization responsible for managing the CA high-voltage electric power distribution grid) is estimating today's power usage will be even greater than Monday's usage. More than likely, this will trigger another California Stage One Electrical Alert. This is the first of three levels of electrical alerts. Customers are asked to make voluntary reductions. Stage Two is mandatory load and Stage Three would be rolling blackouts and voltage reduction as happened on the east coast. The US Department of Defense is conducting their largest Year 2000 test to date, including a Y2K test for the press today. Be on the lookout for buggy DoD software. The test of the logistics and supply systems includes interfaces to outside vendors, such as FedEx. If FedEx accidently delivers a F-15 to your door, I'm sure DoD would appreciate if you returned it. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

1 0

Got clue?
by Alex Rubenstein 12 Jul '99

12 Jul '99

nitrogen:/usr/local/monitored-sites$ ping www.cnn.com ping: cannot resolve www.cnn.com: Unknown host CNN'S INTERACTIVE DEPT: "It's an east coast problem." ME: "Huh?" .....

1 0