-----Original Message----- From: Matthew Petach [mailto:mpetach@gmail.com] Sent: Monday, January 31, 2005 11:15 AM To: Huopio Kauto Cc: nanog@nanog.org Subject: Re: Summary/minutes from sunday evening? On Mon, 31 Jan 2005 11:00:55 +0200, Huopio Kauto <Kauto.Huopio@ficora.fi> wrote:

So..

Could someone please summarize the Sunday evening meeting? Minutes anyone?

I'm not allowed to post to NANOG from this account...but if you'd like to forward this message to the list, I've included my minutes from the Sunday night session at the end of this message. Thanks! Matt

--Kauto

Kauto Huopio - kauto.huopio@ficora.fi Information Security Adviser / CERT-FI -coordinator Finnish Communications Regulatory Authority / CERT-FI tel. +358-9-6966772, fax. +358-9-6966515 CERT-FI duty desk +358-9-6966510 / http://www.cert.fi

2005.01.30 Coordinating NANOG--input from community Paul Vixie, moderator Betty Burke, Merit Steve Feldman, CNET Martin Hannigan, Verisign Paul Vixie, ISC Dan Golding, the Burton Group Betty Burke starts off. NANOG needs to evolve. Thanks to all who helped this effort We all recognize the need for continous change Our goals this evening: Get your input Ensure ongoing community coordination My role at Merit--Special Projects Why Merit? NANOG grew out of of NSFNET backbone services re-engineered and managed by Merit 87 -95 Merit has been joining diverse technologies and organizations nationally/regionally since late 60s Regional role: MichNet, Michigans largest ISP 525 customers (affiliates) 13 members (Michigans' 4-yr public universities) 26 peers National role: neutral organization for NANOG coordination NANOG's early years to today 94 - 95 meetings planned by Elise Gerich 95 - 98 Bill Norton takes over, begins to get outside help with agenda in 97 convenes advisor group 98 program committee formalized under Craig Labovitz 2000 Susan Harris becomes coordinator PC today: selects talks for agenda chaired by Susan Harris target is 18 members, currently at 17 Members selected by: self-nomination (volunteer) or PC nominations PC holds internal discussion, issues invitation to join Thanks to the program committee k claffy, CAIDA Keven Epperson, LEvel3 Steve Feldmen, CNET Elise Gerich, Juniper Barry Greene, Cisco Sue Hares, NextHop Susan Harris, Merit Crag Labovitz, Arbor Bill Manning, EP.NET Dave Meyer, U of O/Cisco Bill Norton, Equinix Dave O'Leary, Juniper Ted Seely, Sprint Stephen Stuart, Google Rob Thomas, Cisco/Team Cyrmru Curtis Villamizer Bill woodcock Mailing list growth evolved from NSFNET Regional-Techs list became NANOG in 1994 today: 7,919 subs, 10,000 receive mail Mailing List Moderation Sample annual volume: in 2004, 10,500 unique messsages sent to the list Moderator sent ~460 private messages since 2000 (about 2/week) 135 dealt with spam management topics loosened up on this in mid-04 29 noop (moderation review would be helpful) 67 pseudonyms (many added .sig on request) 52 foul language 46 politics 44 personal attack Neither Betty nor Susan are operators, so target may have been off on the 29 non-op moderations. 29 addresses temporarily removed since 2000 two warnings, three you're removed for 3-12 months, depending on violation People can re-subscribe upon request. Nobody has ever been denied a request to resubscribe to the list. NANOG attendance slide. Less than 100 in 1994, to peak of over 600 in 2001, back down to 400 now. Only a few registration fee increases since inception. Given hotel logistics, that's surprising, and difficult; the goal is to try to keep the cost down so people don't find it hard to manage to attend. Get creative with sponsorships to help with costs. That's it for formal information from Betty. Steve Feldman will take over talking about the program committee, and changes for the future. NANOG talk selection process Steve Feldman NANOG PC CNET Networks One of 17 Program Committee members Ancient History Originally the NANOG Chair did it all Bill Norton started informal admisory committeee in 1997 Mailing list used to coordinate Modern History Advisory committee became Program Committee in 1998 Mailing iist discussion as proposals recieved Formalized PC's talk selection process in 2004 Continuing refinements and improvement Goals Talks reflect attendees' needs Audience ISPs (transit and end-user) Content Providers Large Enterprises Research/Education Vendors Critera Relevance: operational experience Technical content (vs marketing) Quality: Slides Language Speaker (good with audience?) Timeline Call for Presentations, few months ahead CFP Deadline 6 weeks before meeting Prgram Committee Review and Rating PC Conference Call: Initial Selections (blind until this) Initial notifications 4 weeks before meeting Late Submission Review (more later) Agenda This was a short period since last meeting, a bit rushed due to holidays as well. Call for Presentation send to list PC Review and Rating Review starts after submission deadline PC Review PC Rating each rated 1-5 comments blind to each other until call Conflict of Interest Policy Adopted May 2004 PC members must disclose all conflicts of interest during discusssions PC members recuse themselves when they deem it appropriate (some vendors have opted out) Web Based Rating System allows topic to be rated 1-5, or skip, and add comments Then a summary page is generated with comments and ratings; that's what goes into the conf call PC Conference Call Review ratings and comments Decisions Accept Reject Conditional request modifications possible PC member oversight Re-review after changes made Late Submissions Late submissions will be considered if: exceptional quality or relevance available agenda slot Less formal procedure PC mailing list discussion/conssneus Chair has some discretion Agenda schedule set by chair after talks are seleected subject to change until sent to printer minor changes still possible after that Las Vegas Statistics 19 proposals considered: Accepted 15 Rejected 4 Tentative: 1 (eventually accepted after revisions) 5 late subissions 5 accepted Talks plus BOFs for these numbers Questions at end Dan Golding with proposal NANOG-reform working group Why Reform NANOG? Continuing problems with mailing list administration uneven moderation standards lack of transparency or reasoned decision making breakdown in mutual respect and collegiality Also issues with NANOG meetings Program committee perceived to be out of touch with general operator community selection of topics is extremely uneven perception of powerlessness on the part of NANOG community Time to stop complaining A working group "NANOG-Reform" has been busy since the last NANOG meeting Genesis after a spontaneous community meeting with Betty Burke of Merit concerning mailing list issues Representatives of various constituencies- former speakers, frequent attendees, previous hosts. Participants work for vendors, operators, researchers, and others Goal: come up with a prosal to fix what's wrong so we can move forward Work in progress: www.nanog-reform.org The big proposal: Turn NANOG into membership org in partnership with Merit an elected board of directors/steering committee 7 members, staggered 2 year terms One member elected as Chair public face of NANOG org Board responsibilities selection of list moderator and program committee sets policy for mailing list and conferences responsible for organization goals and direction working with committees, Merit staff, hosts, sponsors, speakers and the larger community budgetary oversight who votes for the commmittee? anyone who has attended a conference in the last 12 months (#insert membership-criteria) Proposal specifics: List administration committee responsible for NANOG mailing list administrative issues will coordinate use of peer pressure and other techniques to keep the list "on-topic" (liberally defined) must be technically knowledgable and have an understanding of all issues that can be discussed on the list (collectively) will implement BoD/SC policy for rare omposted moderation actions, which will be performed in a transparent manner Also FAQ keeper Specifics: Program committee selected BoD/SC for fixed terms responsible for conference program and agenda performs speaker recruitment and selection coordinate with Merit on venue selection and preparation Merit Networks Role Hosting of NANOG web site and mailing list registration, fee collection, and financial management coordinate with local hosts for venue selection Ex-Officio representation on BoD The Future Where do we go from here? community buy-in: please endorse at www.nanog-reform.org futher work: by-laws and initial elections cooperation of Merit and partner organizations like ARIN and RIPE Undesirable but possible outcomes Merit "taking input from the community" perpetually --no action Continued erosion of NANOG mailing list value Split of community with rival ops conference in association with ARIN (like RIPE) List of those responsible for putting together the propsoal Jim Bacher Randy Bush Steve Gibbard Patrick Gilmore Daniel Grollding Mike Huges Rodney Joffe Simon Lockharte Chirs Malayter Ren Provo and many others Bill Norton, Equinix, first at mike. there are already a number of competitive forums that have grown out of the lack of peering coordinator forums at NANOG, for example (Equinix peering forums) Question about presentations missing the mark; but only 4 were set aside, so not as though there are a wealth of talks that are being passed over. Rob Seastrom next up, not sure which org. some presenters feel rebuffed from talking about some topics. does thank merit for sticking with NANOG even when it wasn't solid financially. Now, earning $350K/year, according to his back o envelope calculations. wants Merit to help keep nanog relevant to community he talks about CBC, group he founded in Cambridge, Cambridge Bandwidth Consortium; leadership and direction should come from within the community, not be imposed on it. Betty notes Merit is a 501c3 non-profit; there is enough in coffers to cover the logistics needed to put on the conference; essentially, the reserve is there to cover the next conference coming up in case membership attendance is too low to pay the hotel bills. Maybe would help to share more details of the budget with the membership, to show the financial liability Merit assumes in covering NANOG. RS is grateful for reserve, and feels it is a good thing, just to clarify. Paul Vixie steps up to mike; he has periodically spoken at NANOG when Sue asks; he figures it's a bottom-of-the-barrel case when he speaks. The program committee is a volunteer effort, they're not terribly empowered, they do their work and then get back to their real jobs. It takes some amount of arm twisting to get the presenters from the community up front to talk about the projects they've been working on. Paul thinks it would be better for there to be 50 talks they had to select among, instead of 19, so that there could be a wider variety of talks. Paul urges reform of program committee Steve Wilcox from Telecomplete; document on nanog-reform site is a work in progress, he's got some suggestions he's been working on tweaking; there's a main thrust, and then there's details; the details can be worked out once the major idea is decided on. First get board in! Second point, even if you don't have a criticism with the current system, recognize the new system is a community based system; that will help ensure that going forward, there will be more direct community input into the process. Matthew Ringel of Tufts university. talks that are relevant is his recommendation. Might be good to have a best current practices track. Most people here have some hand/foot in trying to improve processes, whether ops-eng, or arch. Could pair like minded people for the BCPs, help Chris Malyter, twtelecom, would like to see the financials Betty talked about. Would like to see either quarterly or annual charts of where the money goes. Also notes nanog-reform is a working group; it's a draft for consideration, a work in progress that people should help contribute to. Get on the mailing list, and keep working with them to make it the best proposal possible. If there's 8,000 people on the list, and only 400 show up to conferences, how are we failing to atract the other members, what is holding back their more active participation. Alex Rubenstein, NAC is up next. Many people are wondering where the money they pay goes, creates some concern. Would be nice to have a wrap-up summary after each nanog. program committee is doing public requests, but what about making direct invitations to specific people for programs. Barry Greene repeats Andrew Partan's point that the PC already tries to twist arms to get people to present. Barry then notes that what we are proposing is like APIA and APRICOT, should talk to the folks on those committees to make sure we don't make the same mistakes. Program committee, easy to say they should recruit people, hard to actually get it done. There's fresh talent coming into community, but PC when they recruit only go back to the people they already know. Security side can pull from NSP-SEC, for example. Good to refresh program committee constantly to keep them in touch with the talent base that's out there. Next question to Merit; is Merit in a position to let go? With APIA, APRICOT, and APNIC, if the parent doesn't let go or refuses to cooperate, will make any reform very difficult. Betty responds to program committee challenge. there may be a certain lack of operator input, but they really do try to go out and solicit people; the people on the committee really do try. Betty agrees that buget information sharing is definitely something that can be done, and is probably overdue. Would also be good to share survey information from each meeting. an independent compilation happens, but they haven't reported it back to the community. she commits to maintaining the dialog that is starting at this point. She doesn't see it as letting go, as much as evolving in partnership; wants to come up with a structure that works for the overall community. Some room for change on both sides of the table. Merit is definitly on board with the idea of change--this meeting came out of the feedback from NANOG in Reston. Steve Feldman agrees that going out and recruiting talks is tough, but they could do more legwork on it. Some of the best people for recruiting are perhaps not those who actively operate, but those who touch many different operators. Daniel Golding agrees that other orgs also pull from vendors to get talks from operators. he notes that some PC members are on the reform list. It's not a matter of kicking people out, it's just a matter of getting fresh blood into the scene. More former Merit people on the PC than current operators; certainly some new fresh blood would help get a different variety of talks. Josh Snowhorn, NOTA, hosted twice, took about $185,000 to host it, so budget does take some looking at. Josh notes that he gets better discussions at the bar vs listening to talk. Notes that in RIPE, they have socials scheduled to get people to talk; would be good to do that, rather than just the beer and gear where you try to grab as much gear as possible and then get out. So, look at other orgs that work. Multiple tracks, more social interaction. The social interaction helps establish a web of trust between operators that helps us do our jobs more effeciently and effectively. Joe Provo, relaying from EFnet remotely. Some suggestions about minutae. One observation from several people; smaller hosting and ISP networks not well served. Tabulating the surveys shows about attendees, not about why the rest aren't attending. That was from Dan Senie, a frequent contributor. Spencer ? from tektronix; speaking as a vendor, last on the list, been coming since 2000, thanks for the work done in the past. Things at many different levels of detail here; how many different reform proposals are still waiting to come forward. Dan golding replies that yes, it's not fully baked, it still needs a bunch of work. Spencer was worried this might have gone through similar insanity like the IETF at the time. Mike Hughes, LINX. They do good parties, by the way. Agrees with Josh, largely. The program committee member list; note that the PC is from huge networks, generally. they may have been small to start with, but they get bought and grow, and the tendency is towards growth into huge networks. fresh blood is needed to get the smaller folks flowing into the system. Notes RIPE steering committee got together for a day to talk face to face to hash out issues. also notes RIPE has several days of single track, then split into separate tracks, to avoid having to repeat some topics that need to go to whole group. One of the key factors that got us here was the issue of moderation; let's not necessarily turn this into a huge reform effort that bogs down; we can look at addressing some of the smaller points like the moderation issue before the rest starts happening. Chris Malayter from TDS again. Bunch of forums starting with other groups like equinix, terremark, S&D; what about having forums happen in near conjunction, so that people can get the best of both? Nobody can afford to go to all the separate forums, but if the meetings could be grouped, it would essentially be like a start at a multi-track system. There's also MPLS-con that is held at the same time as one of the NANOG meetings each year. Rather than competing against, perhaps they could partner together or coordinate better? Betty agrees that it would better to have good cordination, but there are definite venue logistics to be considered, to get enough bandwidth to support NANOG. Steve Feldman talks about hosting NANOG 2 years ago. got dates, tried to find ballrooms of at least 10,000 sq ft to host it; one hotel sold space out from under them at the last minute, had to start over with only four months left. deals sometimes fall apart, it's a lot of hard work to try to schedule one. Ren notes that RIPE uses amerstdam twice a year, and then rotates one; that way, plumbing is re-used and provides a constant that can be counted on for scheduling. Chris Malayter says partners have been rebuffed from hosting adjacent meetings with Tony Kapella, was CW, now Testament to how open we can be, very very good. The types of talks we've been attracting seem to be self-selecting for similarities. He notes that other communities have activism internally that becomes visible, and that doesn't seem to be happening at NANOG. Schools for example may not know that NANOG exists, but that could talk about research efforts underway, or BCP's they implement. Dan Golding asks aboug VOIP, Tony agrees it's relevant but currently underrepresented. Steve Gibbard, PCH, editor of reform proposal. Want to balance spencer's comment; balance minutae with broad framework people could agree on; more effort towards the broad view, not getting too tied up on specific details at this stage. Vijay Gill, AOL. Easy to complain about lack of presentations; more relevant, talk to people you know who have something you'd like to hear to get talks going. Don't just leave it up to the PC. You want different talks, get off your ass and get your friends to present. Many relevant talks from the past isn't really relevant anymore. Much of our work is cookie cutter now; the application layer is where more of the work is happening. How is Skype affecting your network, for example KC Claffy from CAIDA, been on PC for years. Backs up Vijay's comment to get your friends to bring interesting talks. Is it time to revisit the agenda part of the system? Can we really fix lack of operational content by *adding* more tracks? Look at research community; public archives of work out there, SIGCOMM gets hundres of propsals, accepts very few; we're actually not in bad shape for rejections at all! Just get a wider net cast for presentations! KC claffy talked in 2003 about the top problems in the internet; not one of the problems has gotten better, have all gotten worse. should she talk to NANOG about these problems? Is that a good topic for her to talk about? What is the best use of the agenda time? Dan responds, is price too low to get gripping content? If it cost $50 more to get kick-ass content via incentives, would we do it? Mike Hughes notes that expense is shared equally; if you start comping people based on travel, etc, you end up with a two-tier system, which is bad. RIPE has same challenge. Difficult to get compelling speakers. People hold back due to concerns that their work isn't up to par. People should just go ahead an contribute their work anyhow. Steve Wilcox is back. Other forums have similar issues, similar people attend peering forums to talk about similar issues, why not start finding areas of commonality and combine to optimize time use? Proposal isn't really about changing meeting, it's about the overall structure. Spencer from tektronix, VOIP was mentioned, dave meyers did a talk on IMS, a VOIP technology on Tuesday morning. The PC has been asking for VOIP presentations for the past 4 NANOGs, and a PC member is giving this talk. What can be done to bring things forward a bit? What about poster sessions? currently 3 or 4 'dot' meetings, what about adding some more for VOIP BOF meetings. Miles? at mike, in past hasn't gotten impression that application layer wasn't really welcome at NANOG previously, it was network layer only. are things changing now? What about video, audio, collaborative environments, etc. What about overstepping bounds by expanding. Rob Seastrom notes question and money often go hand in hand. money isn't a free flow; many issues center around what does it take to get more money flowing? Hacks/kludges as well... Also, perception of unapproachability previously about the program committee. This meeting is a huge step forward to open up and start sharing information; too many assumptions made otherwise. Work towards a perception of approachability by the program committee. Joel Kraska from Cisco systems. Two different sets of people. Philip Smith gives the same BGP talk every NANOG, and there are always people watching; either there's a bunch of newbies that are coming in, or we are a bunch of slow learners. Elise Gerich, Juniper networks. Nobody other than Rob has addressed the difference between the reform proposal and the numbers Steve presented. Have the numbers that Betty and Steve presented answered some of the concerns that the reform proposal wanted to fix? Membership of program committee changes over time. 4 new members over the past year. it's tough to recruit new people, others have dropped off. Perhaps that needs to be shared. Moderation of the list was another issue mentioned. If you look at the overall number, 257 messages were censored out of 10,000. Is that just an overreaction? Dan Golding responds; the list moderation issue is more of concern to the community right now. The manner in which it has occurred is what started the ball rolling. The PC isn't involved in the moderation decision; it is even more of a black box. Sylvie Pla? from Teleglobe; we are all operators here, we all have operational issues. Do we have clearance to open the kimono from management? David Barak, SBC, disagrees with having tracks. he likes being exposed to new things that he wouldn't have otherwise seen; one of the few times tracks were tried, the more interesting talks all happened at the same time. Notes same clearance is needed to talk to a dozen people as 10,000 Concludes by thanking everyone Tony Hillard, broadwing--now that the proposal has been put out, what are the next steps? Closing remarks, from Betty. Definitely next steps that need to be taken. Dialog we've had here tonight has been outstanding, there are positive points in the reform proposal. Important comments online about current issues facing the organizational structure of NANOG. Critical to take next evolutionary steps for change. She will work with PC to outline next steps with Merit to address some of the concerns raised tonight. We're in this together, let's avoid an us/them scenario; we all want to see NANOG evolve to serve the community and stay whole. Want to be careful to not fragment the community as we move towards the future. Doesn't have all the answer, but will work with the PC and the reform proposal authors to come up with a timeline of changes. list moderation is a hot topic, there have been some changes at merit on how the moderation is handled; you have her committment to deal with those changes ASAP. She again can't promise she has all the answers; the dialog has been excellent, and should continue into the future. Martin Hannigan adjourns meeting at 2143 hours Pacific Time