June 2004 - Test - lists.nanog.org

Yahoo mail public notice of problems ?
by Mike Tancsa 17 Jun '04

17 Jun '04

Is there a notice I can point non Yahoo Mail customers to explaining why there are delivery delays? We are seeing a lot of stalled deliveries again, and it would be nice to point to an explanation by yahoo as to whats up Stalls are both at the banner not coming up marble% time telnet mx1.mail.yahoo.com smtp Trying 64.156.215.19... Connected to mx1.mail.yahoo.com. Escape character is '^]'. ^] telnet> close Connection closed. 0.006u 0.000s 12:07.77 0.0% 0+0k 0+0io 0pf+0w marble% Or at the application layer with temp failures mta103.mail.sc5.yahoo.com Resources temporarily unavailable. Please try again later [#4.16.3]. I checked from another network totally independent of mine and they too are seeing similar problems. ---Mike -------------------------------------------------------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike(a)sentex.net Providing Internet since 1994 www.sentex.net Cambridge, Ontario Canada www.sentex.net/mike

2 1

NANOG 32 Call for Presentations
by Susan Harris 17 Jun '04

17 Jun '04

* * * * * * * * * * * * * * * * * CALL FOR PRESENTATIONS NANOG 32 GENERAL SESSION TUTORIALS CASE STUDIES OPERATIONS RESEARCH POSTER SESSION October 17-19, 2004 * * * * * * * * * * * * * * * * * The North American Network Operators' Group (NANOG) will hold its 32nd meeting October 17-19, 2004, in Reston, Virginia. This will be NANOG's third joint meeting with ARIN, the American Registry for Internet Numbers (www.arin.net) NANOG will meet from Sunday to Tuesday, and ARIN from Wednesday to Friday, October 20-22. NANOG registration opens September 1. NANOG conferences provide a forum for the coordination and dissemination of technical information related to backbone/enterprise networking technologies and operational practices. Meetings are held three times each year, and include two days of short presentations, plus afternoon/evening tutorial sessions. The meetings are informal, with an emphasis on relevance to current backbone engineering practices. NANOG conferences draw over 500 participants, mainly consisting of engineering staff from national service providers, and members of the research and education community. For more information about NANOG meetings, schedules, and logistics, see: http://www.nanog.org ------------------------------------------------------------------------------ CALL FOR PRESENTATIONS NANOG invites presentations on backbone engineering, coordination, and research topics. Presentations should highlight issues relating to technology already deployed or soon to be deployed in core Internet backbones and exchange points. Abstracts should include a concise summary of findings to be presented in the talk. Network operators are invited to present case studies detailing their experiences with network planning and design, protocol implementation, provisioning, useful tools, traffic engineering, problems solved, DoS scenarios and mitigation techniques, and troubleshooting, including automation, approaches, and techniques, Vendors are encouraged to work with operators to present deployment experiences with the vendor's products and interoperabilty. Researchers are invited to present short (10-minute) summaries of their work for operator feedback. Topics include routing, network performance, statistical measurement and analysis, and protocol development and implementation. Studies presented may be works in progress. Researchers from academia, government, and industry are encouraged to present. The community is invited to present talks or tutorials on: - Backbone traffic engineering - Experience with large-scale VoIP deployments and peering - Experience deploying metro-optical infrastructure - Freely available configuration, management, and measurement tools - Enterprise network security, management, and route control - Innovative access technologies, including metro-optical and PON - Case studies on deploying "triple play" (voice, video, data) - Routing scalability in backbone infrastructures - Security issues for the Internet core - Experience deploying L2 VPN/PW and inter-provider L3 VPN - Impact of BGP dynamics on backbone traffic patterns - Route processor architecture - OS architectures and implementations - SSM deployment experience - Inter-provider security and routing protocol authentication - Innovative network deployments supporting GRID - Circuit2Packet (C2P) experience with legacy services Tutorials have covered topics such as: - Troubleshooting BGP - Configuring IPsec - Customer-Triggered Real-Time Blackholes - BGP/MPLS Layer 3 VPNs ------------------------------------------------------------------------------ HOW TO PRESENT Submit an abstract and draft slides for the presentation in email to nanog-support(a)nanog.org. The deadline for proposals is September 7, 2004. While the majority of speaking slots will be filled by September 7, a limited number of slots may be available after that date for topics that are exceptionally timely, important, or critical to the operations of the Internet. Submissions will be reviewed by the NANOG Program Committee, and presenters will be notified of acceptance by September 20. Final drafts of presentation slides are due by October 6, and final versions October 13. NANOG also welcomes suggestions/recommendations for tutorials, panels, and other presentation topics. ---------------------------------------------------------------------------

1 0

Re: akamaie dns issue
by Curtis Maurand 17 Jun '04

17 Jun '04

This was in my mail this am. This is why there was an akamai dns issue. Curtis -- Curtis Maurand mailto:curtis@maurand.com http://www.maurand.com ---------- Forwarded message ---------- Date: Thu, 17 Jun 2004 00:49:42 -0400 From: David Kennedy CISSP <david.kennedy(a)acm.org> To: nanog(a)merit.edu Subject: Computerworld: Q&A: Tom Leighton, chief scientist at Akamai Q&A: Tom Leighton, chief scientist at Akamai He talked about the nature of yesterday's apparent DDoS attack News Story by Jaikumar Vijayan JUNE 16, 2004 (COMPUTERWORLD) - Akamai Technologies Inc. said today that the Domain Name System problems it encountered yesterday were the result of a sophisticated and targeted distributed denial-of-service attack against the company. In an interview with Computerworld, Tom Leighton, the company's chief scientist, talked about what happened. What was the nature of the yesterday's attack? It was a name server attack against four of our customers for whom we carry their name servers. Our assumption was this was an attack against Akamai and it was perpetrated by attacking our customer name service infrastructure. It is not impossible that this was a coordinated attack against those four Web sites. Akamai has a lot of key customers, and it could just be a coincidence that the four happened to be Akamai customers. [But] we are assuming it was an attack against Akamai. <snip> http://www.computerworld.com/securitytopics/security/story/0,10801,93875,00. html?SKC=security-93875

2 1

RE: Even you can be hacked
by David Schwartz 17 Jun '04

17 Jun '04

> At 7:07 PM -0700 2004-06-10, David Schwartz wrote: > > Most of the people on this list see things from the ISP's > > perspective. > > However, step back a bit and see it from the user's perspective. Do you > > expect to pay for phone calls you didn't make or do you expect > > the person > > whose deliberate conscious action caused those calls to be made? Do you > > expect to be responsible for patrolling your electric lines to > > make sure > > someone hasn't plugged into your outside outlets? > If you had a PBX in your home that was misconfigured and allowed > people to dial-in and then dial back out and get free long distance, > and your telephone company warned you about this weakness, forgives > your first month overages due to your being hacked, and yet you still > refused to fix the system, then you're toast. > > Under those circumstances, if someone makes $10M worth of long > distance calls via your PBX, then you're going to have to pay up. Of course, except in this case, the phone company can't easily tell the legitimate calls from the illegitimate ones and block only the illegitimate ones. Every analogy will break down, so don't expect to be able to convince people with analogies that seem so obviously right to you. Nothing is exactly accurate except the actual situation itself. And, again, alomst every contract has some insurance elements to it. There will be unusual cases where it's actually possible for the utility to lose money if something unusual happens. My main point is that the understanding that seems so obviously right to you may not seem so obviously right to your customers. As for all the people who talk about turning off their DSL access when they're away from home, they're missing the point. Obviously a person could do that. We could shut off our electricity when we leave home. We could have our telephone service temporarily disabled when we go on vacation too. A person could do all of these things. My point is that it's also perfectly reasonable for a person not to do these things. Because in general an ISP has more ability to control these things and it makes very little sense for a home user to insure an ISP, it makes more sense for the ISP to insure the user. In any unfortunate situation, you can find a hundred things that anyone could have done differently that would have avoided the situation. But that is not how you establish responsibility, financial or moral. You look at people who failed to use reasonable prudence. And, of course, the ISP always (or very nearly always) insures the user against the costs of inbound attack traffic that exceeds his line rate. The more demands you make of your customers, the more you decrease the value of your very own product. Frankly, if I ruled the world, obtaining Internet access would require a serious cluefulness test and you'd take a lot more responsiblity for generated traffic. I know a lot of people on this list wish things were the same way and sometimes want it so much that they're able to convince themselves that this is the way things actually are in the real world today. But they're not, and you may find that outside your group of friends, your views are found to be very odd by the majority of 'normal' (but, admittedly, inferior) people. The arguments that seem so obviously right to you may be greeted by amusement and the analogies you think work will be found unconvincing. This is because this argument is largely about other people's expectations. DS

27 78

Computerworld: Q&A: Tom Leighton, chief scientist at Akamai
by David Kennedy CISSP 17 Jun '04

17 Jun '04

Q&A: Tom Leighton, chief scientist at Akamai He talked about the nature of yesterday's apparent DDoS attack News Story by Jaikumar Vijayan JUNE 16, 2004 (COMPUTERWORLD) - Akamai Technologies Inc. said today that the Domain Name System problems it encountered yesterday were the result of a sophisticated and targeted distributed denial-of-service attack against the company. In an interview with Computerworld, Tom Leighton, the company's chief scientist, talked about what happened. What was the nature of the yesterday's attack? It was a name server attack against four of our customers for whom we carry their name servers. Our assumption was this was an attack against Akamai and it was perpetrated by attacking our customer name service infrastructure. It is not impossible that this was a coordinated attack against those four Web sites. Akamai has a lot of key customers, and it could just be a coincidence that the four happened to be Akamai customers. [But] we are assuming it was an attack against Akamai. <snip> http://www.computerworld.com/securitytopics/security/story/0,10801,93875,00. html?SKC=security-93875

1 0

RE: Akamai DNS Issue?
by Drew Weaver 16 Jun '04

16 Jun '04

Similar issues with Yahoo on and off since about 8:30am (EST). -Drew -----Original Message----- From: Leo Bicknell [mailto:bicknell@ufp.org] Sent: Tuesday, June 15, 2004 9:09 AM To: nanog(a)merit.edu Subject: Akamai DNS Issue? >From here neither www.google.com, nor www.apple.com work. Both seem to return CNAMES to akadns.net addresses (eg, www.google.akadns.net, www.apple.com.akadns.net) and from here all of the akadns.net servers listed in whois are failing to respond. Can someone confirm from another location? Comments from Akamai? -- Leo Bicknell - bicknell(a)ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ Read TMBG List - tmbg-list-request(a)tmbg.org, www.tmbg.org

7 9

Re: Travelling the backway to Google
by Tony Rall 16 Jun '04

16 Jun '04

On Wednesday, 2004-06-16 at 20:02 AST, "support services" <support(a)smsonline.net> wrote: > > On Wed, Jun 16, 2004 at 04:28:37PM -0700, Tony Rall wrote: > > > http://216.239.57.104 > > Yes, the old servers are still there but, they are out of date and not > in-sync with any other servers. They can act as mirrors but, they don't > appear as mirror. > > http://216.239.57.104 No Froogle Yes, they do appear to be somewhat backlevel, but that address (among a few others) is what www.google.com (among a few others) resolves to right now - even when doing a manual lookup starting at the gtld servers. Right now they're coming out with a short 10 minute ttl, but my local nameservers still have www.google.com cached (to those "old" servers) with more than 24 hours left on the ttl. I don't think Google has fully left emergency mode yet. And even if they have, the recovery measures they took will not age out for hours. Tony Rall

1 0

Akamai an Inside Job?
by David Kennedy CISSP 16 Jun '04

16 Jun '04

http://www.overclockersclub.com/?read=8733819 The Akamai attacks started in the morning and it was detected by Keynote Systems, a web tracking company that is able to track the load and bandwidth on the Internet. According to Keynote they saw an "Internet performance issue" this morning ... They have tracked the attacker back to person that is at the Akamai Technologies ISP. No other information has been given to us at this time. We do not know if the FBI is working on this issue right now, but we expect them to do so. [DMK: Source, beyond overclockers, unknown, reliability and accuracy unknown.] Akamai spin: http://home.businesswire.com/portal/site/google/index.jsp?ndmViewId=news_vie w&newsId=20040616005428&newsLang=en -- David Kennedy CISSP Risk Analyst TruSecure Corp. http://www.trusecure.com

7 6

Re: Akamai an Inside Job?
by Brian Mulvaney 16 Jun '04

16 Jun '04

At 08:23 AM 6/16/2004, David Kennedy CISSP wrote: >http://www.overclockersclub.com/?read=8733819 > >The Akamai attacks started in the morning and it was detected by >Keynote Systems, a web tracking company that is able to track the load >and bandwidth on the Internet. According to Keynote they saw >an "Internet performance issue" this morning Keynote's primary business model is measuring the performance and availability of public web sites as seen from a distributed network of synthetic probes. They don't offer any services that "track the load and bandwidth on the Internet". Here's what their public/PR type email alert said on the matter yesterday: Keynote Internet Performance Alert Starting at approximately 5:30am PDT today, a major Internet performance issue was detected by Keynote systems. By 6:00am, the availability of the Keynote Business 40 Internet Performance Index had dropped from its usual near-100% availability to 81% availability: <http://keynote.lyris.net/t/4086/732513/23/0/>http://web507.keynote.com/mykeynote/Post/KB40data_061504_085844.asp Further analysis by Keynote indicated that the availability issues were limited to several large sites, all of whom outsource their DNS services to Akamai. These sites dropped to near-zero availability: <http://keynote.lyris.net/t/4086/732513/24/0/>http://web507.keynote.com/mykeynote/Post/KB40data_061504_090509.asp Availability was largely restored by approximately 7:45am PDT. >... >They have tracked the attacker back to person that is at the Akamai >Technologies ISP. No other information has been given to us at this >time. We do not know if the FBI is working on this issue right now, but >we expect them to do so. > >[DMK: Source, beyond overclockers, unknown, reliability and accuracy unknown.] That's nonsense David. Keynote measurements can distinguish between availability problems caused by DNS outages versus those caused by connectivity or site outages. They manifestly don't track attackers. Brian Mulvaney

2 1

Asia Pacific Peering Guidebook Reviewer Request
by William B. Norton 16 Jun '04

16 Jun '04

Hi all - If you have no interest in Internet Peering in Asia, read no further. Otherwise,... Over the last year I have been working with the Peering Coordinator Community, particularly those that have built into and throughout Asia, to document what is *different* about peering in Asia as compared to say the U.S. and Europe. In February of this year I pulled together a group of Peering Coordinators for the APRICOT Peering Track. I asked them to share Peering processes that they found different, interesting, counter intuitive, or unexpected. This white paper documents the lessons learned and insights gained from Peering Coordinators experienced with peering in Asia. The Asia Pacific Peering Guidebook follows the path set by "The Evolution of the US Peering Ecosystem" white paper that I presented at the last NANOG, but focuses on the common dynamics and experiences that folks shared with me over the last year and at the APRICOT Peering Track. First, the paper documents what I call the "Foreign Tier 1 Peering Dynamic" that casts a Tier 1 ISP in one Peering Ecosystem as a Tier 2 ISP in another. We focus on the motivations, the reasons why this occurs as ISPs expand globally. Next, Peering Coordinators shared five reasons that they expanded into/within Asia: 1) Incumbent Tier 1 ISPs to peer their routes outside their home market 2) Meet US Tier 1 ISP Peering Prerequisites 3) Customers want them in Asia 4) Global Marketing Benefits 5) Sell transit in a more attractive transit market using four methods of interconnection: 1) Extend into foreign country 2) reciprically peer in each others coountry 3) Half-Circuit peering 4) Buy an ISP with Tier 1 peering in the foreign ecosystem The Peering Coordinators shared nine lessons, things they wish they had known before expanding into Asia: 1) Tier 1 ISPs will not want to peer in their home market 2) There are several challenges peering in Asia 2a) Peering Process is different - peering@ does not uniformly work 2b) Many language zones 2c) Many time zones - logisitics issues here 2d) Oceans add cost, latency - transpacific transport prices estimated in the paper 2e) local loops still a major expense 2f) transit prices highly variable across countries in Asia - transit street prices quoted 3) Creative peering deals 4) Watch for Inadvertent Tromboning 5) Local Presence is required 6) Separation of Int'l Transit from Domestic Transit in some markets 7) Content that transcends the language barrier is not allowed to be hosted in country 8) No true regional content in Asia 9) Content Peering works in Asia 10) Understand the Internet Peering Ecosystem prior to building in On this last point, the paper enumerates several data points including naming the Tier 1's, the emerging broadband players, the peering inclinations, etc. for Hong Kong, Tokyo, Sydney and Singapore. Some of the Yahoo!Broadband activities for example in Japan show that some places in Asia are leap-frogging the US in terms of next-gen access bandwidth. How would you like 40Mbps DSL access for $50/mo or FTTH for $100/mo? I'm at the point where I'd like to do a dozen or so final paper walk-throughs, where I email the white paper and walk interested folks through the paper to solicit comments and suggestions. I've found that talking people through the paper is this is the most effective way for me to make sure the paper (and subsequent presentation) flow well, and also increases the likelihood of me getting some feedback. If you have any interest in the topic and can spare about 30 minutes for a walkthrough over the next few weeks, send me a note and let's schedule a time I can talk you through the paper. I'm also planning a group walk through (a concall basically) towards the end of the month. As with all the Peering White Papers, this one will be made freely available to the community when it is finished. Bill Here are the other Peering White Papers that are available (google for "William B. Norton" to find them on the web, or email wbn(a)equinix.com with the white paper title in the subject line and I'll send you the latest version of the white paper.) 1. Interconnection Strategies for ISPs documents two dominant methods ISPs use to interconnect their networks. Over 200 ISPs helped create this white paper to identify when Internet Exchange Points make sense and the Direct Circuit interconnect method makes sense. Financial Models included in the paper quantify the tradeoffs between these two methods. All relevant data points are footnoted as to source. 2. Internet Service Providers and Peering answers the questions: "What is Peering and Transit? What are the motivations for Peering? What is the ISP Peering Coordinators Process for obtaining peering? What are criteria for IX selection?" 3. A Business Case for Peering builds upon the previous white papers but focuses on the questions important to the Chief Financial Officer: "When does Peering make sense from a financial standpoint? When do all the costs of Peering get completely offset by the cost savings?" 4. The Art of Peering: The Peering Playbook builds on the previous white papers by asking the Peering Coordinators to share the "Tricks of the Trade", methods of getting peering where otherwise they might not be able to get peering. These 20 tactics range from the straight forward to the obscure, from the clever to the borderline unethical. Nonetheless, Peering Coordinators might be interested in field-proven effective ways of obtaining peering in this highly controversial white paper. 5. The Peering Simulation Game finishes up my half day Peering Tutorial by engaging the audience in the role of the Peering Coordinator. Each ISP in turn rolls the dice, expands their network, collects revenue for each square of customer traffic, and pays transit fees to their upstream ISP. They quickly learn that if they peer with each other, the costs of traffic exchange are much less, but they need to negotiate how to cover the costs of the interconnect. ISP Peering coordinators have commented on how close the peering simulation game is to reality in terms of the dialog that takes place. 6. Do ATM-based Internet Exchange Points Make Sense Anymore? Applies the "Business Case for Peering" financial models to ATM and Ethernet-based IXes using current market prices for transit, transport, and IX Peering Costs. 7. The Evolution of the U.S. Peering Ecosystem, introduces and focuses on several fundamental changes in the Peering Ecosystem spurred by several events following the telecom collapse of 1999/2000. 8. The Art of Peering: The IX Playbook follows the same tact as The Peering Playbook; we first introduce the framework theory of how and why IXes are valuable from an economic perspective. We then enumerate about a dozen tactics IXes use to get over the "Start Up Hump", to build a strong critical mass of participants, and finally, defense tactics to maintain that population. (To be released at a future date.)

1 0