Hi all - For those of you who could not attend the BOF, here are my notes from the Peering BOF. Comments welcome - Peering BOF VII - NANOG 30 - Miami 2/10/2004 7PM Moderator: William B. Norton We were at capacity in the room and started right at 7PM. The Great Debate -------------------------- The Peering BOF started with a refereed debate on "Restrictive Peering Policy: Makes Business Sense vs. Counter Productive." The debate at the Peering BOF was a first miniature stab at a Internet Operations debate focusing on an emotional charged topic, one area perceived as being concealed by NDAs, hidden agendas, greedy corporate interests, ill will: Restrictive Peering Policies. People attending the BOF heard in about 20 minutes some pretty good arguments why such a policy made business sense, and why it was also counter productive. I opened sharing some of the controversy, sharing the definitions of peering, transit, restrictive, selective and open peerign policies, etc. These should be on-line at http://www.nanog.org/mtg-0402/norton.html Vijay Gill (AOL) agreed to present the Affirmative Case, that Restrictive Peering Policies make business sense. He argued that Peering Policies are not emotional, not personal, but black & white decisions based on economics, a calculation on a spreadsheet. Peering is not free and there are scaling issues and associated operational costs that must be justified. Peering is also somewhat of a threat to revenue production as well, since peering between customers bypasses the ISP. Avi Freedman (Akamai) agreed to present the Counter Case, that Restrictive Peering Policies are counter-productive. He pointed to the overhead associated with peering as being a rare exception; peering sessions tends to stay up. Avi pointed to the counter productive aspects of a Restrictive Peering Policy: 1) It inspires the wrong thing. Sprint will never peer with anyone that ever bought transit from them, and that in fact drives the "End-run, peer with Sprint's customers" behavior that lowers revenue for Sprint. 2) Many large networks lose revenue by not peering more openly, as it turns off customers, and 3) Performance improvements from peering widely *increase* revenue. Avi eluded to his AboveNet experience where he witnessed a) customer behavior: customers spending more time on line because of a better experience, b) lower latency and lower packet loss lead to the TCP window opening up faster means more data is exchanged, thus leading to more $ for those that charge on a per-Mbps basis. Better performance drives more revenue. Vijay countered with the "reducto ad adsurdium" argument; the case of peering with each of 200 million laptops computers running zebra. The overhead of peering with 200M laptops would certainly fail the cost benefit analysis. As far as the performance argument, he claimed that the top visited sites from AOL show little performance difference ("imperceptible") between paths reached across peering versus transit links. Finally, he dismissed the selection of an ISP based on Peering Policy, claiming it is today all driven by price. The peering decision is business and mathematical, black & white, a pure economic decision that does what is best for the company. Avi finished up by dismissing the peer-with-all-laptops as not reasonable, not what anyone is advocating, not what is being debated. He argued that the debate is really speaking to the reasonable middle ground case, where both parties have infrastructure deployed. Around 50% of customer traffic will go around you if you do not peer more openly. Restrictive Peering Policies analysis must include the opportunity cost of lost business and lost revenue, a more difficult calculation to make but one that ultimately shows Restrictive Peering policies as counter productive. VERDICT: The audience voted on "which side presented the more compelling case". The winner: Restrictive Peering Policies are Counter Productive (35-43). (Editor's note: this was much closer than I think most people expected; the audience at the Peering BOFs are generally open or selective peers, and have or expect to be stubbed if trying to peer with a Restrictive Peering Policy peer. Both sides presented a good case.) Cable & Wireless: A Tier 1 Peering Policy evolution & C&W migration to Savvis ------------------------------------------------------------------------------------------------------------------ After the debate, Peter Jansen (Peering Coordinator for C&W) volunteered to share with the audience a bit about the evolution of the C&W Peering Policy and what led to their current Peering Policy. He positioned a restrictive policy as a natural outcome of commercial interests (peering costs money) and but said that they will in fact peer with those that meet the peering criteria. There is a committee that evaluates and verifies the requests and there are no personalities involved here. There was a good exchange in the group with a handful of Q&A. Peter did a great job - he put a face to what was otherwise seen as cold corporate interests. He also spoke about the transition C&W is going through, migrating its network to Savvis. The audience was definitely engaged and interactive during these two sessions. The correlation was brought up between Restrictive Peering Policies and Bankruptcy (Wcom, C&W, Genuity, TeleGlobe, ...). Someone made the point that many of the ISPs with Restrictive Peering Policies that had argued for making Peering decisions based on economic analysis had in fact gone bankrupt... How good can those spreadsheets be? The debate generated a good sized crowd at the Peering BOF. Everyone likes a good fight I suppose, and the Peering BOF is one of the few places where people understand and care about this stuff. Peering Personals --------------------------- We spent the second half of the Peering BOF with Peering Personals, a chance for Peering Coordinators to introduce themselves to each other in a 2 minute talk. On the screen behind them were Contact details, AS#, Peering Policies, and a US Map with Peering locations mapped out. They spoke about their network, their peering policy, what they look for in a peer and why others should want to peer with them. The following people participated in the Peering Personals: (I note below a couple remarkable points brought up during their 2 minutes...) Company AS Number First Name Last Name -------------- ----------------- ---------------- ---------------- Akamai 12222 Patrick Gilmore Broadwing 6395 Allison Feese Carpathia Hosting, Inc. 29748 Scott Bethke CENIC 2150++ Dave Reese CET Networks 22934 Dennis Nugent - 4Gbps of traffic, little peering (yet) FLAG Telecom, Ltd. 15412 Nigel Titley Global Crossing 3549 Maurice Dean Japan Telecom America 4725 Amir Arif - 3.5Gbps to the U.S. JENS Co. 2915, 4682 Daizo Tomigahara New Edge Networks 19029 Cathy Chen - DSL in many tier 2 cities Packet Clearing House 3856, 42 Steve Gibbard ServePath 26228 Steve Gibbard SingAREN 7610 Steve Gibbard TDS Telecom 4181 Chris Malayter TeleGlobe 6453, 8297 Sylvie LaPerriere - very strict peering policy, 300Mbps, US, Asia, Europe, 24x7 NOC, <2:1 ratio UPC 6830 Frank Hellemink - 10 countries, 14Gbps traffic, 65% peering today Los Nettos 226 Celeste Anderson Walmart.com 17374 Will Campbell Yahoo 5779, 10310 Brokaw Price - Brokaw contested the operational load argument from the debate; - they have over 600 peering sessions and the load is mostly virtual - they are saving "in the seven figures" annually by peering Microsoft 8068, 8075 Doug Wilson - Doug had a good sense of humor (I flashed the BSOD 25 seconds in ;-) ) Aleron 4200 Ryan McClune - requires multiple locations across the U.S. AARNet 7575+ Mark Prior China Telecom 4134 xiaoyi liu - 11 Gbps between US & China - 80% traffic in China is China Telecom Cable & Wireless 1273 Christian Kaufmann - (PLC) in U.S. will have a more open peering policy Adelphia Communications Corporation 18756, 19548 Dean Deback - 10 Gbps in, 6Gbps out - ~ 40% peer2peer (see slides) SoftbankBB 17676 Masato YAMANISHI - not peering in U.S. yet, 48Gbps ingress in Japan, 24Gbps outbound - 10G peering Google, Inc. 15169 Paul Nguyen UltraDNS 12008 Rodney Joffe - 12Gbps of traffic There were a lot of people on this list with "Pulse Peering Policies"; if you have a pulse, we will peer with you. (The previous name for this policy was replaced.) I produced a matrix showing those with open policies that were colocated at the same IXes, and there were a lot of green squares indicating that peering could/should/maybe already does occur between these companies. All in all I thought it was a useful and interesting Peering BOF. We ran over by about 20 minutes due to the late addition of Peter's C&W talk, and a couple Peering Personals that were past the cut off date but too important for this group to pass up. Both of these however represent significant changes in the U.S. Peering Ecosystem that they were worth the time. Comments welcome - Bill // // William B. Norton <wbn@equinix.com> +1.650.315.8635 // Co-Founder and Chief Technical Liaison Equinix, Inc. //