Yay! Last one! Cheers to Bill for another successful peering BOF. Matt 2008.02.19 Peering BOF XVII notes A handful of new people, first-time NANOGers Bill Norton starts off by explaining a bit about what a peering BOF is. Who attends it, and for the new folks, what should you expect. Bill Norton shows slide show of what peering coordinators do. Over last 8 or 9 years, peering bof has grown pretty large. Aim is to have community seating, let everyone see who's speaking as much as possible. Goal is to have some folks you'll want to peer with by the end of the meeting. Inspire some dialog, and get people to exchange information. Most important part is to get the right people in the room; get the right people talking to each other; the technical details follow on from that. Bill's job is to facilitate that process. Very full peering agenda peering survey efficient technique for enforcing internet peering policies summary proxy socialogical alaysis of peering coordinator interaction asr transit playbook peering personals, batch 1 ken florance, netflix, video and peering jeffrey payne, p2p, topology awareness and peering great debate--does peering still make sense with transit pricing dropping peering personals part 2, people come to front, mingle some more. Tom Scholl starts off with Ren and Greg, goal is to get some communication going between peering coordinators. Take survey online, see results at end of march, early april. http://tinyurl.com/3xoa6g Sample questions--do you use IPv6 unicast BGP peering? do you use BFD? do you use 4 byte ASNs? what is largest frame size you use on peering links? what is your biggest concern when deploying a new feature? This will be first survey that covers lots of technical issues. Josh will put link at the bottom of the global peering forum page to make it easy to find. Thanks to Ren, Greg, Tom, and don't forget to participate! David Smith--Cisco--efficient enforcement of enforcing policy between peers. Peer should only have IP reachability to your customer prefixes. Should NOT be able to use your AS to reach one of your other peers. Most ISPs rely on BGP to enforce policies between peers. But if you use routing tricks, like pointing a default, there's nothing that stops the packets, they're only enforced in control plane. Alternate options--only carry partial routes on your peering routers; doesn't stop local peers using you as transit. Assumes your peering router is for peering only, no transit, no core. Static interface based ACLs; headache to maintain accurately, hardware issues, etc. proposed technique ISP tags peer prefixes uniquely within BGP and FIB tables ISP tags external packets that ingress peering interconnects based upon longest match prefix match within FIB ISP forwards or discards packets that ingress peering interconnects based upon associated packet tag value Not a futurist talk technique available today supported on high-end routers; not supported on the 7600s use table-map command to do prefix tagging use bgp-policy command to do QPPB for packet tagging use service-policy command via MQC to do packet classification QPBB provides glue between control plane and forwarding plane. IOS configuration sample shown route-map set-prefix-type permit 10 match comm 1 set ip qos-group 66 ip comm-list 1 permit ASN:66 int p3/1 bgp-policy dest ip-qos-map service-policy input peer-in provides much more scaleable manner to enforce peering policy, prevents people from using you as transit against your will. Use of QPBB as glue between control and data. Changes in BGP are automatically reflected in data plane control based on the tagging from BGP. Complements many of the other BGP control plane techniques in use today. All hardware based on the platforms mentioned above on engine 5 and above. You can do this on a Juniper with DCU too. :) arguably more elegant as it can be done in the firewall filter as a global policy. With DCU, instead of matching on QoS value, you can give the forwarding class a name value to make it more readable. Not a new technique, but with the interactions between BGP and MQC, it's not widely understood and used. RAS notes that one note about using it on Juniper; he and Tom talked about it a few years ago, you can build a RIB with just your customer routes, and then send default down an LSP to a traffic sniffing box. Dave notes you can look at snmp counters on DCU and get alerted when it crosses a threshold level of traffic as well. You'd also see discards in the class of service via the MQC MIB. Do people think this is a widescale problem? Do people think bandwidth stealing is happening? Ren notes they found someone who was stealing hundreds of megs of bandwidth in AMSIX; they're no longer a member there. Steve Wilcox has seen this happen at LINX as well; this is a theft issue, so it's odd that we think of it as a technical issue, rather than a criminal issue. Do people see this as an issue on a private peer, or only on the public sessions? Chris Malayter, social network around peering, helps run the peering IRC network many of us hang out on. Less activity on the channel once NANOG started, guess we're all talking face to face, not online. shows you when users are generally online. irc.terahertz.net, if you were here, you know the channel. :) Peering personals, first batch now. Hopefully people here are looking for peering at the exchange points. VideoBox, AS36472, peer with them, get free membership 365Main, PAIX, San Jose Equinix, SFMix, Heavy transit load from them; 98% is transit. No clusters out east yet. If you have eyeballs, they've got the content your eyeballs want to see. AS36472.net tens of gigs of traffic. They do ACL on their side, see things like leaked OSPF traffic, and tier1s that are on the public fabric, with high bar, starting with X, they leak OSPF and may send packets towards you. Simon Ferret Veoh AS40415, another video provider, no peering yet; content delivery through partners at the moment; thinking about insourcing at some point. P2P as well as streaming; if people use p2p, saves a lot of CDN volume; most is still over CDN right now; video on website is CDN based, download to keep is p2p. Aim is to get a good TV experience. Bryan Berg, Imeem, AS36119, social networking site; old school napster meets youtube, ad supported music streaming site. Looking for eyeball networks, 15G of traffic. Ash, SJC, public, private in CHI, LAX, PAIX PAO Looking for south america, philippines, malaysia, portugal About 30% peering so far; wasn't as hard as he'd heard, but it's not a walk in the park. Jeffrey Payne, peer-to-peer QA Used to be at real networks, used his gear to broadcast nanog in the early days; very knowledgeable about the current peer to peer arena. Q: What techniques do peer 2 peer networks use to determine which host to pull a fragment of a file from? Well, let's go back to what p2p is; early model of p2p used central sites and servers; then RIAA got involved, gnutella had horrible search but was decentralized; then Kazaa came along, got search better. Now, bittorrent is a huge explosion of micronetworks. The simplest model in use has no intelligence; don't mind the local transfers, but the remote ones hurt--but the system has no knowledge about near vs far peers. Why does remote vs local matter? Is it due to asymmetric last mile? It's more the bit-miles issue; how much are you moving over how many network miles? commercial systems are trying to introduce intelligence; aimed at one-to-many data delivery, with central command and control that you expect from commercial delivery system. So, for commercial interests, need to work to optimize flows and reduce costs in one-to-many or few to many model. The P4P model has ISP intelligence in the Itracker to give some sense of topology to the torrent tracker. with the old system, it couldn't tell local peer from remote peer. Newer system has to 'infer' from hop counts and performance indices to figure out who is better/closer than others. Frank, talks about IPTV on your computer, p2p supported CDN, can take content from multiple sources. Goal is to add more control into tracker to optimize the assignment of clients to seeds. Pando networks, P4P networks speaks up, come talk to him, he can get you all the information you could want on it. But will networks really participate out of the goodness of their hearts? Not so much that, as about reducing the operational expenses. Is there some level of buyin from the people here around that architecture? Verizon notes that you don't need to provide a huge amount of data to get a big win. who would be willing to contribute data? a good chunk of the room. consumer adoption is the challenge; now that flash supports sockets, can start doing it in the browser. traffic shaping devices are targetting more and more protocols, like http, when they used to just target p2p traffic. The commercial side is agnostic; just don't want to be traffic shaped. Moving from mp3s to mpeg4, traffic volume is really skyrocketing, we need to start thinking more intelligently about the problem. Many of the major p2p efforts are behind it as well. Can we do more optimization in CMTS systems, DSLAMs, and try to get network layer efficiencies. Any last minute thoughts? Both trying to create these commercial content delivery systems; may not be seeing it quite as holistically, but both trying to improve the situation. Peering News Ren Provo moved from SBC/ATT to Comcast now, she can now watch premium channels while saying "No" Peter Cohen moved from uh...well...to Sprint as peering coordinator. The great debate--does peering make sense anymore? Pro--Patrick Gilmore--peering still makes sense Con--Guy Tal--peering doesn't make sense anymore 2 minutes per side, referreed match. Patrick still saves money to peer in the right place Also provides multiple vectors, provides better resiliency in case of issues. Also, provides better scaling as traffic grows, allows for scaling as needed to specific areas. Provides shorter paths to talk to people in an area. Helps getter better transit deals when you go to good central peering locations. Guy Peering is hard; as was pointed out earlier, our transit prices are decreasing, it's getting close to transport cost; why spend your resources and time doing an exercise that's someone else's core competency; focus on your core competency instead. Let the carriers do the heavy lifting for the most expensive part of the work; they have multiple vectors already, they know how to scale. Why try to map who the right peers are to go after, and how to protect yourself from your peers--let your transit handle that headache. Patrick Peering is indeed hard, it's hard to set up a network that's resilient; but there's many things that are hard but are still useful. You'll have to have some level of clue already to set up your network; it's not that much harder to add BGP to the portfolio of skills needed. Transport is getting more expensive relative to transit, but they haven't crossed yet; you can still get ring-based paths for less than transit for now; so you might as well go to the peering point, get access to your transit provider as well as your peering partners at the same facility. Guy While transport hasn't crossed transit, you also have many other costs; engineering resources, a NOC, pay for rack/power at peering sites, router costs to go into peering points, etc. There's many fixed costs, like buying a peering port; per-port cost may be more than you pay for transit initially. Transit gives you SLAs, peers don't. The more traffic you give to your transit, the better your cost. And whatever you may think, peering isn't free. You'll always have to have some transit relationship; if you're not already SFI, you'll never be able to be SFI, that club is closed; so, since you're not going to be able to meet everyone's peering requirements, you'll need to keep transit, might as well let them handle it all for you. Patrick Scalabiility, reliability, performance need same skillsets you already need to run your network to do BGP; doesn't greatly increase your skillset, should be a minor increase over skills you already have; will allow you to go to places where you can get better pricing, and will allow you weather internet daily issues that come up. Guy Having a higher percentage of peering will raise your costs initially; until you do a large amount of it, it won't save you money. And it's not true that it's a minor skillset above running a basic network; let someone with the real skillset handle it; you don't have to worry about someone patenting peering and suing you over it, or being accosted by Patrick asking whether or not you peer yet. So, who made the more compelling case--people get to vote once for each side. The people will be counted. "Patrick believes peering is hard, but is also useful" 62 and 45 in favour of Patrick? 14 and 14 in favour of Guy 107 to 28 in favour of Patrick. Now, feedback from audience. Port costs on public peering fabrics have not tracked in relation to transit costs dropping. Peering to someone can help get additional ports in place faster to provide capacity, rather than getting two different networks to upgrade their transit links. Prices in Europe are so different, it makes more sense in Europe than in US; prices in US need to get more in line with EU. Brokaw agrees that without visibility on links, it may be very hard to tell how well your transit provider will be able to handle surge loads or increases in traffic over time. Patrick notes that with most of the networks, it's the transit port to the other end of the conversation that has capacity challenges you can't fix. Over to Ken at netflix to talk about challenges with handling video traffic. They used to do DVD rentals only, now doing bits over the wire as well. In January, went from money capped, to model where service isn't capped. Unlimited PC viewing, not MAC viewing yet, DRM issue. Future looks like people consuming things on PCs, TVs, etc. Have services embedded in DVRs, HDplayers, game consoles, things hooked to both TV networks as well as IP networks. Volume can end up growing drastically. Want to keep traffic off people's backbones if at all possible; optimize p2p delivery, cache, etc. People are consuming their content through IP networks more and more. Goal is to get the bits into the access networks; can peer at centralized locations, or they can put equipment in their pops if that makes sense. If you have ideas on where they can stick their equipment, come let him know. Cheol Hee Yun AS3786 LG Datacom Largest IDC in Korea 4Gb in Korea, 10Gb in US, they are present in PAIX Palo Alto, and Equinix San Jose They don't have peering in Korea, unfortunately at this time. Again, don't forget to fill in the survey!