- NANOG - lists.nanog.org

NANOG36-NOTES 2006.02.13 talk 3 NTT labs AAAA query explosion worries
by Matthew Petach 13 Feb '06

13 Feb '06

(Huge apologies in advance for any and all names I completely mangle! check http://nanog.multiply.com/ to see names/faces correctly handled by Ren. ^_^; ) Matt 2006.02.13, talk 3 NTT labs, (Steve Feldman apologizes for mangling the pronnounciation of their names). NTT information sharing platform labs (didn't get names/info from opening slide) Outline Expect increase in number of DNS queries this year Discussion effect on cache server load and user response time how can we decrease number of unnecessary queries? Today's topic we focus on increase in number of queries between users and cache servers caused by IPv6 support number of 4A queries same as that of A queries domain name completion function DN completion by OS DN completion by application IPv6 enabled OS increases 4A queries Vista will be v6 enbled by default IPv6 and OS resolver IPv6 enabled OS sends 4A queries for every name resolution BSD/Windows Sends both A and 4A queries for every name resolution currently no way to disable one or the other Domain Name Completion when a name resolution fails, both OS and APP automatically try different prefix/suffix completions. OS using these domains to complete: FreeBSD: specified by "search" in /etc/resolv.conf, distributed by DHCP Windows: configured in control panel, distributed by DHCP Applications: Mozilla: retries with www domain prefix IE searches domain using MSN search and then retries name resolutions for domains by adding .com, .org, .net, .edu. Convenient for user, perhaps, hard on nameservers. Combination in FreeBSD completions are different depending on OS FreeBSD tried domain completions for A and 4A for each case. Windows tries all 4A records first, THEN tries all A records. So IPv6 queries in Windows means even if there's an A record in v4 space, it exhausts ALL 4A possibilities FIRST, before going back to get A record. Longhorn/Vista IPv6 default enabled ALWAYS tries 4A queries first! IE7 plus Vista results in 12 DNS queries per user click, best case. Worst case, one user click results in 40 DNS queries!! Slide showing projected impact based on historical data plus projected Vista deployment. Right now, 4A queries only about 5% of queries. After Vista, size of increase could dwarf rest of DNS queries. Release of Windows Vista (IPv6 by default) doubles at least the number of user queries causes more queries in domain name completions and domain search sequences Operators cache servers should be prepared for those increases stop domain distribution to users by DHCP or PPPoE Developers of OS is current search order of resolvers appropriate? eg should "A" record be resolved before domain completion. Ed from Neustar, at microphone: before we consider this a problem, consider from point of application provider; when you need a name, you don't know what transport you may have underneath; if you wait for NXDomain, you increase latency, so app developers generally send all queries at once. What about changing DNS to allow asking for multiple questions at once? Changing application behaviour isn't likely to happen, and changing protocols isn't easy; so why not just beef up the infrastructure to handle it? Joel Yagli, UofOregon; do you know how many of those queries will need to fail over from UDP to TCP due to responses being too large to fit into a single UDP response? Most of the responses coming back don't have data, so they don't need to go to TCP. Tony Bates--what happens when v6 record is returned as valid; does the chain stop there? Also, if you flip to return A record first, we'll never to move to v6. We NEED to start resolving v6 records first, to help move the 'Net off IPv4. Applause, on to next talk.

2 1

NANOG36-NOTES 2006.02.13 talk4 DNS infrastructure distribution
by Matthew Petach 13 Feb '06

13 Feb '06

2006.02.13 Steve Gibbard DNS infrastructure Distribution Steve Gibbard Packet Clearing House http://www.pch.net/ scg at pch.net Introduction Previous talk on importance of keeping criticical infrastructure local Without local infastructure, local communications are subject to far away outages, costs, and performance Critical infrastructure includes DNS If a domain is critical, so is everything above it in the hierarchy Sri Lanka a case in point. Previous talk was in Seattle last spring, highlighted undersea cable being cut; even local DNS queries failed since TLD servers couldn't be reached, even though local connectivity still worked. The ship dragging anchor in harbor cut only undersea path out of the country; international calling was down, and all of the Internet. But unlike local telephone system, even local networks failed to work. Root server placement Currently 110 root servers(?) Number is a moving target Operated by 12 organizations 13 IP addresses at most 13 servers visible from any one place at any one time six are anycast four are anycasted in large numbers All remaining unicast roots are in the bay area, LA, or washington DC Distribution by continet 34 in NA 8 each in BA/DC/ 5 in LA Only non-coastal roots in US are Chicago and Atlanta canada, monterrey, mexico some others 34 in Europe clusters of 4 each in London, and amsterdam, Europe's biggest exchanges even throughout rest of europe for rest. Distribution by continent 26 in Asia (excluding middle east) 5 in japan (4 tok, 1 kyoto) 3 in india, korea, singapore 2 in hongkong, jakarta, and beijing south asia an area of rapid expansion 6 in australia/new zealand 2 in brisbane 1 each in auckland, perth, sydney, and wellington 5 in middle east 1 each ankara, tel aviv, doha, dubai, abu dhabi 3 in africa 2 in johannesburg 1 in nairobi, 1 more being shipped very little intercity onr intercountry connectivity 2 in SA sao palo santiago de chile other parts of world not really served at all. world map with blobs showing coverage. Huge areas not covered. overlaid fiber maps with dots to get ideas of coverage (redundant); everyone else is one fiber or satellite cut from being isolated and dark. Pretty much follows the areas with money. Root server expansion 4 of 12 root servers actively installing new roots 110 root servers big improvement over 13 from 3 years ago two operators (autonomica, ISC) (I and F) are installing wherever they can get funding funding sources typically RIRs, local governments, or ISP associations Limitations in currently unserved areas are generally due to lack of money Fs and Is In large portions of world, several closest roots are Is and Fs At most 2 root IP addresses visible; others far way Does this matter? gives poorly connected regions less ability to use BINDs failure and closest server detection mechanisms non-BIND implementations may default to far-away roots Should all 13 roots be anycasted evenly? CAIDA study from 2003 assumed a maximum of 13 locations; not really relevant anymore Big Clusters Lots of complaints about uneven distribution Only really a concern if resources are finite Large numbers in some places donesn't prevent growth in others Bay Area and DC clusters seem a bit much, but sort of match topology Western Europe's dense but relatively even distribution exactly right Two per city perhaps a good goal for everywhere TLD distribution Like the root, locally used TLDs need to be served locally Locally used TLDs: local ccTLD; any other TLDs commonly in use Regions don't need ALL TLDs. gTLD distribution: .com/.net .com/.net well connected to the "internet core" servers in the big cities of US, Europe, Asia non-core location: sydney. Map of world with .com/.net overlaid with fiber maps shows "well-served areas" again following the money, with even less coverage outside NA/Europe/Asia. gTLD dist: .org/.info/.coop share same servers considered confidential. data may be incomplete significantly fewer publically visible servers, almost all in internet core. only one public locatino in each of asia and europe Even worse coverage worldwide, though they do have south africa. Do have some caching boxes next to caching resolvers at the big ISPs; not sure if it increases coverage or not. Few other gTLDs, didn't map them. .gov is us-centric .edu is US, some eu, some asia .int is california, netherlands, UK (not very international!!) Where should gTLDs be? presumably depend on their markets if it's ok for large portiions of the world to not use those gTLDs, then it's OK for them to not be hosted there. ccTLD dist: answers to where ccTLDs should be more straightforward working in their own regions a must working in the "core" could be a plus just over 2/3 of ccTLDs are hosted in their own countries (but a lot of those aren't ... Green map shows those countries that host their own ccTLDs locally. Most islands are red, in danger of being cut off from their ccTLDs. ccTLDs not slaved in the core 18 ccTLDs aren't slaved in the global core if their regions are cut off, those ccTLDs won't be visible to the rest of the world is that really an issue, if you can't get to the end site anyhow? violation of RFC2182, unclear data results not so much matters if nobody from out .bb .bd .bh .cn .ec .gf .jm .kg .kw .mp .mq Example countries Kenya exchange point, root server, ccTLD server, all external connectivity by satellite Pakistan root server, no exchange point, no TLDs locally (so how much use is the local root server?) Kenya: local exchange in nairobi root server ccTLD server so even if external link goes down, country can stay mostly functional. Pakistan: local root server (for at least one ISP) no TLDs .pk hosted entirely in US no local exchange to share local root server single fiber connection; when it breaks, nothing works. Local peering caveat local traffic has to be kept local before keeping DNS local is of much benefit. Requires either strict monopoly, or local exchange points Examples here highlight that. Methodology Get name server addresses for TLDs Assume everything in a /24 is same place or set of places. (really down and dirty shell scripts) bad assumption for UUnet nameservers; didn't find others. 625 /24s contain name servers for TLDs 135 host multiple TLDs; over 60 in RIPEs case Figure out where those subnets are traceroutes/ask questions Subnets with 10+ TLDs--read it from the slides. :D 193.0.12/24 192.36.125/24 Other sources www.root-servers.org had root server data; assumed accurate. ultraDNS locations considers its locations confidential Got info from Afilias's .Net application. Told missed some. In general, most other TLD operators were very helpful. Thanks! http://www.pch.net/resources/papers/infrastructure-distribution/ Mark Kosters, Verisign; notes there's two other root server groups also going anycast wherever people will pay to host them. K with RIPE is now going outside region, and Verisign (J?) is also talking about serving in multiple regions. Dealing with local customs getting in country tends to be the biggest challenge; PCH has seen similar challenges getting into countries. OK, break time now.

1 0

NANOG36-NOTES 2006.02.13 talk 2 Duane Wessels, DNS cache poisoning
by Matthew Petach 13 Feb '06

13 Feb '06

2006.02.13 talk 2 DNS cache poisoners Lazy, Stupid, or Evil Duane Wessels Motivation During March/April 2005, SANS internet storm center reported a number of DNS cache poisoning "attacks" were occurring Poisoned nameservers have bogus NS records for the com zone SANS ISC theorizes it may have been a vector for spyware propagation Microsoft windows (most versions) and symantic firewall products are affected. Slides are on the website, BTW. The poisoning attack: an auth nameserver (where queries normally go) is configured to return bogus and out of baliwick NS auth records. caching resolver receives and trusts those bogus referrals future queries for names in poisoned zone go to the bogus NS dig +trace longislandauction.com will show the poisoned NS auth responses NS auth1.ns.sargasso.net. which has NS com. auth1.ns.sargasso.net. so any caching resolvers may consider auth1.ns.sargasso.net authoritative for any unknowns in com zone. Vulnerable implementations: Windows NT (by default, SP4, can tweak it via reg) Windows 2K, (by default, later fixed) Windows 2003 (not by default, but easy to unfix) Symantec gateway firewalls SYM04-010 and SYM05-010 to Yahoo search and find more. How to find poisoners? start with a large list of DNS names or zones discover set of auth servers for the zone by following referrals on down from root query each auth nameserver compare the NS RR set in each reply to the previously-learned referrals for parent zones this technique only finds parent-zone poisoning. February 2006 Survey input list is about 6 million names from nameservers they have access to. Found 284 "poisoning" nameservers; returns bogus NS entries for root or TLD. . has 217 com 49 net 29 org 24 au 3 cc 2 cn 1 to 1 default 1 some nameservers poison more than one zone. List of some poisoners on slide 12. dns.internic.ca ns1.afternic.com ns0.directnic.com ns1.domainsarefree.com etc. Never attribute to malice what can be adequately to be explained by stupidity Many of the nameservers that return bad referrals appear to be companies in the DNS business registrars resellers speculators typo profiteers others appear to be legitimate companies they should know better many of the names leading to poisoners are either expired or parked Is the sky falling? with so many poisoners out there, why don't we hear more about the problem? Most implementations don't allow root to be poisoned If you were surfing the web with poisoned DNS cache, would you know it? let's simulate it... for every bad referral found, we put the nameserver's IP address go to www.google.com go to www.microsoft.com see what you get. bbns01.secureserver.net, for example, happily pretends to be google.com. dns.domainsatcost.ca is amusing, because their ads are from google, even as they hijack it. a.ns.nameflux.com at least does an HTTP redirect dns2.nai.com doesn't return any A record, so you at least know *something*'s wrong. More examples follow... ns1.frakes.net ns.pairnic.com "smart people use pairnic for DNS"... Duane would beg to differ. 65.75.128.178.com returns an amusing message that is clearly wrong, blaming the clients for the traffic the DNS server itself is causing. Lazy, Stupid, or Evil Laziness: ns1.hi2000.com The admin is too lazy to put each domain delegated to them into separate zone files. Instead, they create a com zone and list A records for each delegation. Laziness such as this is probably the source of most of the poison out there. (includes guess at what their zone file looks like) Stupidity: ns1.frakes.net Typos, combined with laziness, create an interesting situation. Looks like Frakes.net is using the com zone technique, but forgot to make the nameservers fully qualified. Note that ns1.com etc are legitimate DNS names and have A records different than those returned by ns1.frakes.net. Just forgot the dot after the trailing name on the NS record. Evilness: Our definition of an evil poisoning nameserver is one where it answers queries with the wrong address, and maybe proxies web traffic sent there so you get what you (mostly) expect. To help find them, give each source of poison an evilness ranking from 1-5, with one point for each issue below: Returning bad referral poisoning a TLD Answering an A query for "important names" Answering query incorrectly Answering the query such that the web browser looks like it *might* be correct DNS A few fours, no fives. Miscellany Some of the poison sources that we find are actually vulnerable implementations that hve been previously poisoned by someone else. Remember: authoritative nameservers should NEVER accept recursive queries!! Some NS records have non-FQDN names. The name "ns" is a popular example. It's a good thing even the vulnerable implementations don't let the root zone become poisoned. Bottom Line: Several hundred misconfigured nameserves out there that return bad referrals that can poison DNS caches About 75% try to poison root zone, which usually has no effect Probably 90% of nameservers out there today are not vulnerable to this type of poisoning Most attempted poisonings http://dns.measurement-factory.com/surveys/poisoners.html wessels at measurement-factory.com Question from microphone: what do you do when you find these poisoned nameservers? And how often do you run the scripts to avoid false positives? Right now, he doesn't do anything other than put it in the database seen above. He tried contacting one, but never got any answer back. Contacting people is very hit or miss; hard to get responses from people. For second question, he's run the survey a few times; it takes about 10 days to go through the 6 million names, so he might try running it once a month or so. He doesn't think of any cases of "false" positives; they may fix their config, but if it was positive, it's not "false". Applause, and onto the next talk.

1 0

NANOG36-NOTES talk 1--steve feldman
by Matthew Petach 13 Feb '06

13 Feb '06

Based on generally positive feedback from many people, I'll be posting my notes from the conference. I'll preface the subject line with NANOG36-NOTES, so if you want to mass-skip the thread, it should be easy to do so. 2006.02.13 NANOG36 day 1 Opening/welcome to Dallas Steve Feldman starts off--many people are still trapped out east unfortunately. Steve Feldman, Program Chair, CNET networks. Texas Compact Car == SUV Our Host: Brokaw Brian Mike Todd Parker Raj Patel Brad Parker Thanks to NANOG program commmitted List of them went too quickly Agenda Changes--Tuesday 12:10--12:25...went too fast to see. Agenda Changes Wednesday 9:30-10 Hurricane Katrina: Telecom infrastructre, impacts, solutions, and opportunities... (more on slide) Reminders Network Security con't use cleartext passwordss do use end-to-end encryption (ssh, VPN) PGP Key signing see link off wwww.nanog.org for details Yahoo Reception Bear and Gear reception Interpreting Badges. Blue -- steering committee Yellow -- program committee green badge -- yellow plus blue, both committees green dot: peering black dot: security red dot: PGP signer RED badges will be for mailing list panel members Lighting Talks six 10-minute slots available Criterion: on-topic for mailing list Signups start now! http://www.nanogpc.org/lightning Random acceptance of submissions made before 2pm Monday Submission order after that (if slots remain) That's it for Steve Feldman, next up is Brokaw Price from Yahoo. Welcome to Dallas, on behalf of Yahoo. Not many hotels in Dallas that can host a group of this size with two ballrooms (one for general session, one for beer and gear; and the Hyatt was the only other one with space, and they're booked since Katrina wiped out other conference spaces in the south.). There's a trolley one block over that will take you to the downtown restaurant areas, it's free, feel free to take it and explore the area, note it stops running around 9:30pm. It does quiet down after dark in downtown, unfortunately. One thing really needed for a NANOG is a really good sized Internet link. The hotel had a pair of T1s to start with, and they were getting a second pair when Yahoo began pulling the fiber in for NANOG. Terminal room is now virtual; feel free to use the laptops and printers should you need to print documents, boarding passes, etc. The laptops are cabled down, but if you need to borrow one, just track Brokaw down, and he'll take care of you. He's been feeling like he's part of the garment industry in doing all the gear (two sets of tee shirts, and the fleeces for people who peer with Yahoo--if you don't already peer with Yahoo, jump in and send us email--peering(a)yahoo-inc.com) There will be an awesome party tonight at Eddie Deens, everyone should make sure to attend--it'll be fun, Texas style, and Texas sized. :) Many thanks to Mike Gallagher for doing the NANOG36 specific website with details on the local area and local options for attendees. Betty Burke asked him to say a few words about what it takes to put on a conference like this; in many ways, it's been like being a huge wedding planner, only weddings don't need large internet connectivity. Start planning early!! Brokaw thanks the Merit people for being so supportive; they've been complete animals, biting into the details with gusto; they're like true roadies, getting gear packaged and shipped, audio gear, video gear, cables, power, everything. Many thanks to Larry, Betty, Chris, Dave, Susan, Laurie, Dwayne, Steve, Tony, Tom, Greg, SC, PC, everyone else, it's all been completely worth it! It's definitely exciting times--we're building something huge; traffic levels are growing at near-exponential levels, datacenters are rolling out faster and faster. It's great being part of this community, and we all need to help keep it alive, to nurture it and helping it grow. If you haven't hosted a NANOG yet, definitely consider it; it's an interesting process. It starts off with "What's NANOG?" "What's the ROI on a NANOG?" Dan Golding joking about having a PBS-style thermometer graph showing how much new peering we get each day, to demonstrate the ROI for hosting a NANOG. But really, it's about sharing the support for the community--it's about stepping up to the plate, and saying "it's our turn to pitch in." There's 26 inches of snow in central park, which is keeping many, many of our colleagues away; and if you DO host, try to avoid Valentine's Day!! If you have any questions about hosting, feel free to call us. The good people at ATT have been great working with us. The fiber coming into the hotel terminates in the parking garage, but that was about 60 feet from where it needed to be. December 23rd, discovered the shortfall, and discovered how to pull innerduct in through a hotel on short notice. No matter what else happens for NANOG Huge thanks to Yahoo crew, especially Mike Gallagher, Brian Lacroix, Todd Parker, Raj Patel, Brad Parker, The whole ATT crew for their OptiMAN handoff Plan ahead! :) And make sure your brand people Jeannie Conduit(?), Jean Modie(?) are on board with your drawings and designs. There are bar rooms on either side of the bar, the Brassiere, and the Pyramid rooms can be used for hanging out if the bar gets too full, they have good connectivity. So, let's get it underway!! First up is Duane Wessels.

1 0

Lighting talks at NANOG 36
by Steve Feldman 13 Feb '06

13 Feb '06

NANOG 36 attendees may now submit lightning talk proposals at http://www.nanogpc.org/lightining See that page for details. Steve

1 1

Any interest in notes from the talks at NANOG?
by Matthew Petach 13 Feb '06

13 Feb '06

Since there are several attendees that are snowed in and won't be able to make it to Dallas for NANOG, I was thinking of posting my notes from each presentation to the nanog list, so those who are stranded can follow along from home. Would that be of interest to the list, or would it be just so much more useless spam-like fodder to be deleted? So far, the response to my notes from last night's community meeting has been positive--but that's only 2 people, so I don't know if I've helped 2 people, but alienated 8,036 more in the process. ^_^; Let me know if you'd be interested in having me post the notes. Thanks! Matt

1 0

Gmail multiple messages on mailing lists - resolved
by Mark Foster 13 Feb '06

13 Feb '06

>From the Sysadmin of the box: "Resolved, in theory - we were being hit by a bug in SpamAssassin 3.1.0." I am not sure what exactly about Gmail made SpamAssassin go nuts... but it is resolved for now. Thanks to everyone who assisted, including Google, who were pretty quick to at least acknowledge my report. Now, if only their details were on puck.nether.net :) Mark.

1 0

2006.02.12 Open Committee Meeting Notes
by Matthew Petach 13 Feb '06

13 Feb '06

I captured some notes during tonight's open mike committee meeting, in case they may be of interest to the list. Apologies in advance for typos, it was hard to keep up with the speakers. ^_^; Matt Steering Committee Report (steering(a)nanog.org) 2006.02.12 1700 hours Central Time. AGENDA Steering Comittee (Randy Bush) Program Committee (Steve Feldman) Financial Report (Betty Burke) Mailing List Report (Chris Malayter) Steering committee report Tryng to hear the membershiip responsible for ML, PC, Lostistics But trying not to micro-manage Establishing normal but minimal business practices Semi-weekly minutes on web site SC Tries to listen Transparency: SC Minutes, ML, Stats, ... Trying Mon-Wed Meeting (instead of Sun-Tues) Newcomers' Session (did it work?) No more Terminal Room (Laptops plugged to printers near registration) Change badge fonts (larger company name/person name) (really a question of size, according to WBN) Suggestion from a nice gentleman at the microphone who says "why not print the badge on both sides, so you don't need to flip badge around all the time? Did NOT change Number of meetings per year Many costs of support are fixed, ie not per-meeting Currently amortized over three meetings If over two meetings, fees would go up significantly Did NOT change Working Lunch Hotels have monopoly on food The charges for lunch are what you would expect from a monopoly But we will try to be more sensitive to ease of getting lunch near or at the meeting venue (not economical to have hotel provide food) Rights in Data NANOG trademark is held by Mertic Presos are copyright by the author Right to freely distribute, but not modify, granted to NANOG PC is drafting this formally copyright notices on slides are OK if small and unobtrusive What does it mean to be a member? Attendance at meetings, and participation in the mailing list is pretty much what defines membership. Program Comittee First change using new process seemed successful Why have you not submitted a talk? Wht do you want to hear? Mailing List Worked the process to fill vacancy left by Steve Gibbard Still working with ML panel to document their process Still working With MP to develop an appeals processs Statistics are published monthly on NANOG web site ML Panel Appointments No Terms, etc. in current charter Straw proposal charger change parallels SC and PC two year terms staggered two sequential terms max without a vacation Please comment, change, propose (Bill Norton, Equinix, Use of nanog-futures to discuss this very type of issue...Randy will get to it) ML panel proces cont. This would give members a light at the end of the tunnel Volunteers would know what they're signing up for Allwos chang without bad vibe of removal Normal organizational practice Chartger Change Octove is the end of the process so start now MLL Panel straw poproasal staring Need to get Steve's name adn other star-upisms removed No other proposals received for this year New Ideas BLOG--no progress Wiki, no progress SlashNOG, no interest Trial of new tech gear at NANOG, nothing exciting Video in hallways, Do you like it? It's back, same size, better location!! By cafe tables near registration area, near where food will be. Traded terminal room for informal breakout rooms, informal seating, allow for more mingling. Ren Provo--http://nanog.multiply.com/, about 100 pictures with names and affiliations, so you can match up faces with names to help newcomers. Mailing Lists Engineering and Ops dicsussion only (nanog(a)nanog.org) Discussion about NANOG itself (nanog-futures(a)nanog.org) Steering (steering(a)nanog.org) Program (nanogpc(a)nanog.org) ML <nanog-admin(a)nanog.org) Fruit supplied, yum! Discussion? How can we make NANOG more useful, fun, informative? Randy gripes the mailing list has gotten boring recently. Cut to Steve Feldman for Program Committee Report. Steve Feldman, CNET, PC Chair. All opinions, mistakes, his, all the good stuff is thanks to the PC. NANOG 36 program 26 submissions (down from 41!!) 22 accepted 1 cancelled 1 withdrawn 2 rejected 2 very late, both accepted Areas for improvement Speaker solicitation Tool improvements self-service submission web interface Reports Program Format Mon-Weds format morning plenaries afternoon BOF, Tutorials Evening social events Newbie meeting is there a better name Tracks? not without more content!! for tutorials, bofs, hopefully not too much overlap or need to be in two places at once. Party tomorrow courtesy of Yahoo! Tuesday, Beer and Gear with sponsors. For Tracks, need sufficient space as well as content. Lightning Talks Criterion: on-topic for mailing list Signups start Monday morning instructions during plenary Random acceptance of submissions made before 2pm Monday Submission order after that (if slots remain) (No personal insults! Stay technical, keep it below 10 minutes) Feedback Talk to us! PC members have yellow and green badges Send mail nanog-futures(a)nanog.org (public) nanogpc(a)nanog.org (private) Open Discussion What topics would you like to hear more (or less) about? How can we (PC and community) get speakers on those topics? Joe Wool, ATT, newbie--what is mission or purpose of this organization? Might help us develop content if we knew the purpose of the organization. We're a group of network operators who talk about network operations stuff. Open to anyone who operates large networks using networking technology. Bill Norton--what obstacles are there for people that prevent them from doing Question from the gallery--what talks would be relevant to the group? Security! IPv6 deployment new last-mile connectivity (DSL, DOCSIS 3, etc) peering? how to convert from OSPF to ISIS economics of networking? what macroeconimic events are shaping networking? Geoff Huston did great set of talks at the last meeting, for example. Randy notes potter.net is his monthly article, same subject Randy notes broadband deployment in Japan might be of interest--Cho-san could talk about it; you can get 100baseT in most places in Japan for $30/month. US is one of the last place developed countries in last mile deployments. Steve notes PC has assumptions about what we don't want to hear; no sales pitches from router vendors, no far future standards that may or may not happen, no talks of internet politics like ICANN politics, etc. If those aren't valid assumptions, should be challenged. Bill Norton notes that finding topics is sometimes easier than finding people to get up to the microphone and talk about it. For example, company spending millions on IPv6 gear testing could do a simple presentation on what they found. You might think it's horribly boring, but others might find it more useful. Tom Schull, ATT, hard to talk about deployments when vendor bashing isn't permitted. Maybe relax that restriction? Are vendors really that intertwined with NANOG? Vendors do provide financial support in terms of beer and gear; vendors don't sway the program comittee choices. Most vendor talks aren't vendors, but tutorials seem to be (they have budget for it?) Honesty isn't disallowed, but try to steer clear of simple hyperbole and invective. What about performance measurements? Those tend to be very cut-and-dried, factual details. Virender? RiverDomain?, what about advent of malware, events of denial of service, how do large networks use to mitigate those type of events? Lessons learned type talks. Steve notes the Katrina talk will focus on that type of "lessons learned" talk--see if those are of use to the group or not. Thank you all for the comments, and send email as well. Next up, Betty Burke for Merit Financial Report. NANOG 36 community meeting, Betty Burke, PM, Merit. Welcome!! Attendance Chart NANOG35 Report (financial report) NANOG36 predictions General Updates Acknowledgements Attendance: risen from about 100 in 94 to about 500 in LA slight downward dip for 34 down to 450. Each meeting has location as well as time of year that affects attendance NANOG35, LA Oct.05 Revenu 267,198.07 Expense 240,306,13 Academic Calendar: Jul1 to Jun31 Will give fiscal year update next meeting Equipment supplies 16,431.79 merit staff travel 9,255.32 Non merit staff travel 0.00 Speakers 0.00 Hosting 93,879.93 Admin Overhead 120,739.09 Technical overhead Per Meeting Balance: 26,891.94 No speakers who needed to be sponsored to attend in LA. 93K hosting is check to hotel for food, meeting room, power, etc. Admin Overhead is a rollup; July to October admin staff salararies, tech staff salaries, other GA from Merit that needs to be charged to NANOG for hosting it. 518 attendees, 26,891.94 surplus to carry forward. NANOG36 predictions Current registration 340 w/o cancellations due to bad weather Current Sponsors B&G - 8 (usually 10, down 2 from normal) Breaks - 0 (no vendors sponsoring breaks this time) Expenses Hotel about the same Travel +2 (2 additional staff members, ensure the tools and wireless network would be up to snuff) Fewer equipment expenses (spend in October beefing up, reuse in Feb) Revenue number will be lower than LA, with similar expenses to last time, so may Updates Accomplished Improved Budget Reporting Video Wiring (less big bulky RGB cables in white crates!) Statistics New meeting format (set-up, badges, agenda) Next quarter Merit CEO Search Improved Voting process Improved Registration Process New switch hardware Possibilities Presenters website NANOG list server update Carol Wadsworth has been working hard to make sure talent logistics work out Merit is looking for new CEO, Betty Burke will have a new manager in June. The improved voting and registration process will finally be merging this year. Develop a database, capture photos, etc. of people. Will be getting some new switch hardware as well, to help improve connectivity. Considering a separate presenter's website. Aiming to beef up servers to speed response time. Future meeting sites: Spring 06 --not yet confirmed Oct 06 w/ARIN --confirmed Feb 07 w/Internet2 --planning underway with internet2 joint techs--one day overlap with them. Acknowledgements SC, PC, List-admins Host and Sponsors Yahoo, ATT, Duane, Arbinet, Arbor Networks, Cariden, Cisco Systems, Foundry Networks, Juniper Networks, Renesys, Tellabs Merit Staff Attending Manish, Jon, Bert, Mary Merit Staff in Ann Arbor Sue, Bob, Charlie, Jeff, Brad Merit NANOG Staff on site David, Rick, Jason, Dale, Chris, Larry, Gayle, Carol, Susan, There's Merit staff both here and in Ann Arbor working hard to keep everything running smoothly. Questions? Bill Norton asks why it's in Merit's strategic interests to continue playing the host/coordinator role for NANOG, esp. in light of the evolution that happened last year. In Betty's opinion, after the meeting with Merit's board two weeks ago, when that question was asked, it is worth it for Merit to go through the shakeup. Merit's purpose is to run a first-rate, high-end research and educational network in the state of michigan. But beyond the regional network, it aims to showcase its network to the wider world, and from that perspective, it makes good sense to support NANOG. Merit board of directors generally agrees. Betty notes there was a question of what would happen if Merit didn't handle NANOG? If the community felt it was important, it would go on; but Merit felt it was in a good position, and the cost recovery was preventing it from being a drain, that the community good will benefit was worth the bumps along the way. Bill Norton--are things easier? Randy goes back to first question. Look to Europe and Asia, where internet registry takes a stronger role; the ops community is largely run by the registry, and fragmentation has started to creep in. With ARIN, LACNIC, etc., it's more like a partnership, so less fragmentation. Merit also brings educational background, eliminates any partisanship that might creep in from vendor support. Back to Bill--it's easier, because we're talking more as a community, opening up to more transparency, showing the benefits Merit brings with their platforms more clearly to the community. Slight tension still, working as a community rather than just being "merit making decisions over here off to the side by ourselves". Oh! Looks like BLUE badges for Merit staff?? Chris Malayter, Mailing list admin panel presenter up to talk about the mailing list. AGENDA: Status update on membership Process Flow Addition to AUP -- Anti Spam List stastics Questions Rob, Susan here, Rob Wilcox not able to be here. Membership Stve Gibbard -- resigned Steve Wilcox --added Present committ Rob Seastrom Susan Cris Steve Process Flow User sends an email to the lsit an MLC voluntter jduges email to be a violation of the AUP AUP per EMAIL list AUP The SC is authorized to make changes per the NANOG charter For one-off violations, an MLC volunteer sends an off-list note to the offending user, giving guidance about the list's AUP and explaining the MLC volunteers' concerns. For repeated violleations, the MLC discussion on the nanog-admin list may result in an official warning being sent to the user, outlining the specific violations the user has been responsible for, and specifyng that further violations might result in revoking of posting privileges For repeated violations following one or more official warnings, MLC discussion on the nanog-admin list may result in a decision to remove posting privileges for the user for 3 to 6 months For repeated violations following one or more official warnings and also following a temporary posting ban, MLC discussion on the nanog-admin list may result in a decision to permantly revoke posting privagletges for the user Bill Norton asks if a single MLC volunteer can act unilaterally Joel Yagli?, regardless of how process has been designed thus far, taking the light touch is probably the best thing that could have been done thus far. Bill Woodcock wants more moderation and heavy handedness; his views are amusing. Louis, from Equinix; what about appeals? Chris notes there is a process being discussed for an appeals process; Merit is the last line of appeal, after the and in conjuction with the steering committee. Legally, Merit owns the trademark "NANOG", so it is responsible for legal ownership of the term NANOG. However, it wants to work with the community and not be heavy handed. But Merit does reserve the final go-no-go decision. Bill Norton asks if the warning comes from a group address, or an individual account? Rob Seastrom replies--after Steve left, he's been sending out the warnings from his own account. But if there is a user who has procmail'd a given list admin, can they be considered to have been giving fair warning? If the first warning is ignored, the second warning would come from a different list admin, who would hopefully not be procmail'd. Mike Wallace?, ModWest. Recommendation that a role account be created, but sign each message with the name and signature of the person writing it, but, the "from" line would be from the role account, which would prevent individual treatment of messages. Also, removes the view of actions being singular user singling out vs a more "group oriented action. Randy Bush, IIJ, notorious procmail; if you send mail Cc'd to something he's responsible for, it will always get past his procmail filters. If you have a role account, you can make it part of the subscription process that the role account be whitelisted past any filters. Propopsed addition to AUP first proposed addition since committee started. 8. Challenge/response sender whitelisting software which requires interaction by any party to validate a post to the NANOG mailing list as non-spam shall be treated by the list administration team like any other condition that generates a bounce message. Subscribers with software (such as but not limited to TMDA) that is (mis)configured ...slide moves on, can't capture the rest of it. Question--Regarding AUP Addition: Should we treat vacation messages the same way as mis-configured whitelisting messages? Joel Yagli--in general, opposed to summary kicking off the list; but in a case like this, which is essentially DoS'ing the list, yes. Jared Mauch, NTT America, he hosts puck.nether.net, which has mailing lists on it; his solution is to use his best judgement on when to remove people; auto-whitelist tend to be quickly removed. People on 2 month sabbatical tend to be problematic. Pete Templin, TexLink--should they discriminate to just posts that kick back to the list, vs that kick back to the sender. The big damage is when it goes to the full list, clearly. Martin Hannigan, Renesys--if Reply-To goes to the sender, it goes once in a 30 day period, so is this really a problem? Joe Provo; if it's a one-on-one problem, they shouldn't be whining to the list admin. Joel Yagli--if it's one message back from a vacation, that's annoying; but if it's a challenge for every mailing to the list, that's worthy of blockage. So, agreed, if it goes to the list, absolutely remove them. But even if it goes back to the sender, it does generate complaints to the mailing list administrators. Rob Seastrom agrees that it's a grey area, esp. on the vacation subject. About 5% nanog-post@ is the shadow mailing list that never gets traffic, but that governs whether you can post to the list or not. That way, any auto-responder won't be allowed to post to the list. Randy asks Joel to clarify the difference between vacation and bozo turing test. The vacation message is one-way, notification only. The Turing test is an arms race; the more widely used they are, the closer we get to the point where we start failing them. Also, some poorly handled systems have no hysteris dampening. Bill Woodcock again; in this region, we seem to be dilligent workers; in other regions, postings to lists result in deluge of autoresponders. Rob Seastrom notes that there's no stigma with this type of removal; you can rejoin immediately when you return from vacation. All that's being removed is a bot, not a person. When the removal happens, the person is ALWAYS notified of the removal, so they know to rejoin upon return. Martin Hannigan notes TMDA is clue factor; the vacation is more likely corporate policy. The removal is NOT just posting privileges, you are removed as a recipient as well, Rob Seastrom clarifies. Mike Hughes, LINX--no harm if you get kicked off, just read the archives. It's a temporary removal! Steve Gibbard--challenge/response replies usually indicates they're not getting the list mail in the first place. Martin Hannigan--what about making first incident warning, second offense is misdemeanor with removal? What was the most recent case? 2 and a half weeks ago, went to the whole list, and the admins were deluged with forwarded copies. Randy notes this is a network operators list, not a network idiots list. if you can't operate your mail client, you don't really belong on the list. Michael Loftis, ?, Discussion seems to center around Exchange users--but UNIX clients have similar issues, but the users are more clueful. What about honoring precendence settings on mail being sent through? Patrick Gilmore calls for vote and moving on--we're starting to beat a dead horse. The steering committee makes the change to the AUP, so they'll get the vote. List stats: January Stats (thanks, Merit!!) Posts to list 8039 Warnings: 2 Threads requiring intevention 2 Members with posting privileges removed 2 Bill Norton--can any light be shed on those two people who were removed? No, no, no. Patrick Gilmore--if you say no, it means we DO need an appeals process, as it means the process is closed and not transparent. Randy Bush, IIJ, Steering Committee. Trials in secret where the result/victims are not known, and causes are not known, are done in this society. (other than the president, who is NOT related to Randy). Randy feels the REASONS for people being banned needs to be made public; don't NAME them, but note the cause and process. RS notes that they'll be happy to divulge what information is required of them. If the steering committee believes that providing some information is appropriate, they will provide it. But without the framework and guidelines, they won't do it. Joe Provo--asks if they were clear abuse of AUP, or grey area. They were clear AUP abusers. Chris notes that when it's clear and removal happens, even if you don't say the name, everyone else will know who it is. But Bill Norton says that if everyone knows who it is, they've already been shamed, what additional shame is there in having their AUP violation listed? Chris declares it dead, until the steering committee directs the list admins to change their policy. Bill Norton finishes off with a call for more transparency in agreement with Randy--it's the lack of transparency that led to the evolution of last year. Randy notes direction WILL be provided. Steve Gibbard--when are the stats for? January 5th to February 5th, actually. He notes 2 was the same number that were removed during the 8 months he was on the team; so it seems the curve is growing rapidly. Joel Yagli--if you haven't noticed, there's not a lot of shame on the list; if you can't shame them into not deaggregating, how can you shame them into not posting?!! 8039 subscribers to the list, NOT posts in the month!!! Only 692 posts in the month. *whew*. Martin Hannigan, founder, former chair of the mailing list admins. In the first mailing list incarnation, they did not publish the names, but they did agree to publish the reason so it was open and fair. The reason for not posting the names was that the general agreement was there could be negative legal connotations to list them by name from the owners of the list; but listing AUP violations does not single out individuals. RS and Chris note they're in agreement, but won't take action until they have a framework from the steering committee, to ensure it is NOT an ad-hoc decision made by individuals on the list-admin team. Once that framework is forthcoming (Randy), the team will move forward in accordance with it. Martin Hannigan--we got here because we wanted the list to be more fair, more transparent, more open. He agrees with TMDA, doesn't agree with vacation, hopes that there will be clear guidance on it coming soon. Randy--how is this safer than transparency? RS notes transparency vs obscurity is one axis, framework and policy vs adhoceracy is a different, orthogonal axis. Chris just says he's not comfortable doing it until Merit says it's ok, since they have legal feet in the fire. Bill Woodcock asks for the next slide--it just says Questions, so we're already here. Jared notes much of the mailing list details aren't formalized, we should take this to the nanog-futures list to discuss openly. We can ask for some of these details to more proscriptively be put in the charter moving forward. Merit put together new charter, they left a lot of vagueness around the mailing list charter, as it was a hot topic, and there was already quite a bit of change happening on the group administration. They specifically left the void in the charter, so that the community could fill it in based on what their views and opinions were. Merit wants to support the list. Betty notes there is a desire to NOT single people out, but if the vote is to increase transparency by making information documented, Merit will be happy to support that effort by documenting the framework. Luis from Equinix; were the posts removed from the archives? No, there's no removal from the archive, you may go back and view them. Joel Yagli--ghost of usenet tends to show people for who they are, in spite of how they may have posted inappropriately in the past? (not sure what he meant). Mike Loftis, ModWest--we all think the list admins are doing a great job--we're not angry with any of them, we're just trying to hash out how best to handle this hot topic. On that note, we move on to the next topic on the Agenda. There's nothing scheduled to be next. Vince Fuller beat him up about something. He doesn't like the schedule change, they should be scheduled before or after so people who don't need to be tutored can skip them before or after. And never again on Valentine's day!! And not on Halloween either!! (ARIN!). With that, Randy calls the Community meeting to a close with some comment about stepping on hogs (poor innocent porcines!!) at 1903 hours Central Time.

1 0

nanog.org website - 403s?
by Mark Foster 12 Feb '06

12 Feb '06

Anyone else seeing 403's when trying to pull anything other than the index page from www.nanog.org?

4 4

Re: nanog.org website - 403s?
by Gregory Hicks 11 Feb '06

11 Feb '06

> Date: Sat, 11 Feb 2006 10:29:43 -0700 > From: Michael Loftis <mloftis(a)wgops.com> > To: nanog(a)nanog.org > Subject: Re: nanog.org website - 403s? > > > > > --On February 11, 2006 10:09:42 PM +1300 Mark Foster <blakjak(a)blakjak.net> > wrote: > > > > > Anyone else seeing 403's when trying to pull anything other than the > > index page from www.nanog.org? > > Not everywhere but almost every single page yes. mailinglist.html still > works... > > someones FTP client or ssh/scp client set with wrong umask? :D All the ones that I tried I received "Forbidden You don't have permission to access ..." --------------------------------------------------------------------- Gregory Hicks | Principal Systems Engineer Cadence Design Systems | Direct: 408.576.3609 555 River Oaks Pkwy M/S 6B1 | San Jose, CA 95134 | I am perfectly capable of learning from my mistakes. I will surely learn a great deal today. "A democracy is a sheep and two wolves deciding on what to have for lunch. Freedom is a well armed sheep contesting the results of the decision." - Benjamin Franklin "The best we can hope for concerning the people at large is that they be properly armed." --Alexander Hamilton

1 0