February 2008 - NANOG - lists.nanog.org

2008.02.20 NANOG 42 Graceful Restart, NSF and NSR
by Matthew Petach 21 Feb '08

21 Feb '08

Last set of three notes...was going to send them from the ballroom, but they started tearing down as soon as the closing finished, even though it was 30 minutes early. ^_^;; Matt 2008.02.20 Graceful restart, non-stop routing, spiteful switchover, and non-stop forwarding. Ken Weissner, kweissne(a)cisco.com systems and technology architect introduction to HA technologies Lots of moving parts, we start with some definitions: HA -- high availability general term SSO -- Stateful Switchover two processors/processes, transfers information from one processor so second can pick up from first one. NSF -- NonStop Forwarding forwarding is key portion; router continues moving packets as control plane recovers/restarts GR -- Graceful Restart IETF specificed mechanism, allows peers to give time for peer to come back before routes are flushed; both sides have to agree on it first. NSR -- NonStop Routing maintain session completely without losing state as it shifts from one processor to the other, without having to alert peer via graceful restart Thos four all work together to allow unplanned switchover to occur with minimal interruption of service (not quite "none"). ISSU -- In Service Software Upgrade use above items to allow upgrading of software while packets continue to move. Why do people care about SSO/NSF? Concerns about single point of failure; customer aggregation or customer connect point which would otherwise impact many, many customers when issues arise. They can also do it on non-distributed platforms; very little packet loss on them. Diagram showing what impacts happen with adjacency failures prior to SSO/NSF. Another slide showing nonstop forwarding due to graceful restart mechanisms. supported for LDP, OSPF, BGP, ISIS Graceful restart for EIGRP, and two different draft mechanisms for OSPF. Two modes; "aware" and "capable". If you're a device that is "capable", your peers need to be "aware" that you are capable of doing it. the capable device tells its 'aware' peers that it will come back within a timeout interval. "aware" and "helper" are generally synonymous. configuration for capability needs to be turned on; not on by default. awareness is turned on by default in Cisco code. TCP based protocols have to have it configured on both sides for graceful restart to be able to function. graceful restart concerns voiced at nanog 40 If I'm an aware peer, how do I tell if my neighbor really went away, and I should reroute quickly, or if it's going to be back shortly, and I should continue to move packets towards it. Need to know if NSF is active or not; but NSF isn't something you configure. so, graceful restart concerns addressed: For BGP, there's a restart timer which limits amount of time before peer comes back, which limits the amount of blackhole time; default is 120 seconds, but can be set shorter to limit the duration of blackhole events. Other conditions also apply; if link is POS point to point, linkdown will abort GR, and will tear down the session and flush routes. Once open message with restart bit set comes across, routes are put into stale bucket; still used, but stale counter begins to count down until they get flushed. Also, added "end of RIB" update message; at that point, it can build new table based on updates, and can act upon changes in the stale table; clear truly stale entries, stop stale timer, and process is done. For OSPF, two ways to handle it: RFC3623, vs draft-nguyen-ospf-restart-06 slight differences; does a new route update cause an abort out of graceful restart or not? Cisco supports Nonstop routing for ISIS and BGP in IOS inbox solution, no other communication needed. BGP configured on neighbor basis; ISIS is box-wide. Hybrid BGP NSR run GR on route reflectors Per peer GSR/NSR config for BGP currently BGP GR is globally enabled for all peers in the routing process Where BGP NSR is available, configured on a per peer basis. GR gets used over NSR if peer supports GR Need mechansism to protect interfaces/L2 state and forwarding; protect both data plane and control plane. Features work together to provide protection and redundancy; should really use them all together as designed. eg, enabling SSO also enables NSF (FIB checkpointing). Each routing protocol on the box needs to be aware and configured to get full benefit of NSR. routing protocol timers often cranked down for faster detection of failures. When switchover happens, takes a little while for new processor to catch up, if dead timer is set too low, graceful restart may not kick in before the sessions are torn down. First packet can be pushed in 10 seconds for BGP; but setting timers down to 1/5 or anything below 10 seconds can result in oscillation or other problematic interactions.

1 0

A couple or advanced references...
by Paul Ferguson 20 Feb '08

20 Feb '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Apologies for the noise, but I'd like to go ahead and provide references for a couple of data points which I plan to mention tomorrow during my brief presentation -- they are not referenced in my presentation slides, but they do highlight the issues I'm trying to address. Each are very recently announced studies, papers, or announced statistics. The first one is a study conducted by the fine folks at Google, wherein they "...investigated billions of URLs and found more than three million unique URLs on over 180,000 web sites automatically installing malware". The paper is located here: "All Your iFrame Are Point to Us" http://research.google.com/archive/provos-2008a.pdf ...and associated blog entry here: http://googleonlinesecurity.blogspot.com/2008/02/all-your-iframe-are-point- to-us.html This study reinforces what we are seeing -- literally hundreds of thousands of compromises on the web and server -side. Second, is a paper recently jointly released/presented by Ga. Tech and Google on the the rampant escalation of rogue/malicious DNS resolution paths: http://www.citi.umich.edu/u/provos/papers/ndss08_dns.pdf The numbers are somewhat... staggering. The two issues above contribute directly, and overlap, more than most people are aware. And thirdly is a figure that some folks may already be aware of; the fact that identity theft was the number one source of consumer fraud complaints submitted to the U.S. Federal Trade Commission in 2007. According to the agency's yearly report on fraud complaints for 2007, of 813,899 total complaints received in 2007, 258,427, or 32 percent, were related to identity theft: http://www.ftc.gov/opa/2008/02/fraud.pdf According to the FTC, total consumer fraud losses totaled $1.2 billion, with the average monetary loss for an individual at $349. Credit card fraud was the most common form of reported identity theft at 23 percent, followed by utilities fraud at 18 percent, employment fraud at 14 percent, and bank fraud at 13 percent. Now, there is a certain "fudge factor" in these numbers, of course, but I only mention these issues as a preface for the topics that I plan to solicit the NANOG community's assistance in addressing. Thanks, and see you tomorrow! :-) - - ferg -----BEGIN PGP SIGNATURE----- Version: PGP Desktop 9.6.3 (Build 3017) wj8DBQFHunbjq1pz9mNUZTMRAri9AKD8wY2qH07AMhpDc2dZpJkdFAHVFQCdEa+t uI1Cwhy1TlHjI6DlQHy5SCM= =V9Dm -----END PGP SIGNATURE----- -- "Fergie", a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/

5 4

2008.02.20 NANOG 42 Lightning talk notes, round 3
by Matthew Petach 20 Feb '08

20 Feb '08

I went ahead and grouped the notes from all three lightning talks together into one set of notes, as the speakers were powering through so quickly there was no time to even save one file and start a new one. ^_^; Again, apologies in advance for the horrific typos and misspellings, I think we're getting a higher and higher prepondeance of fast talkers presenting these days. :) Matt 2008.02.20 Lightning talks Three lightning talks, 10 minutes each. (notes from all 3 in same file for me.) Martin Hannigan from start to transport: submarine cable deployment hannigan(a)gmail.com, Keflavik Iceland deployed multiple submarine cables, has a background as a captain, done submarine cable deployment. iceland surrounded by water lowcap cable <5G hicap cable 720G lowcap cable is to EOL soon hicap subject to backhoe issue Farice1 Drop a berrilium steel splitter in middle of atlantic to peel off fibers to islands. Security benefits--communication to other countries, rights to access, freedom of information. Performance concerns (speed of light, etc.) .75ms per 150km ROI--will you get return on your investment parameters wet/dry plant manufacturer (tyco) capacity desktop study (DTS) tends to be vendor locked in for full system. biggest cable is 8pair cable. undersea routing--where will you drop it Risks length--too long may be too muc delay repeater distance and sparing; closer, the more you can pump out and handle upgrades to future tech, but costs more. shipping lanes, fisheries, geological considerations fisherman have associations; you've got to talk to the commercial fishers anchors, backhoes--more than 50% cause of outages. permitting to cross international boundries, and pay taxes. sign intent to proceed; gets materials going commision desktop study realeastate for landing spots complete engineering ocean survey, map topography A to Z manufacturer cable in segments, then join it at repeaters. A/Z ends dropped to the beach, cable rolled out Suprise, Greenland connect! 3 routes out of country now 41 active underwater US east and west coast 50 active cable ships that lay/repair ships are shared out, contracted to each other if another ship is closer to where a cut is; they all use a common jointing technique. Cables powered from landing stations, 48vDC powered into the cable. More denser, the cheaper. east coast to london, cables cross all over the place. Telegeography and Cable Operators are your friends. Use armor on cables, bury offshore; costs more, but worth it. prefix mapping to sub cables might be interesting. Be wary of FUD. Fishing conflicts analysis--reroute around fisheries. all nautical charts are in meters; 2150 meters down. Bathymetry charts showing currents, can cause friction landing station approach, very shallow. Credits. wow. ======================================================== Duane Wessels, the measaurement factory day in the life of the internet measurement project. will have a measurement day in March. our data will help researchers You don't have to send data to them; just need it to be collected, indexed, into database so researchers can get to it. motivated by 2001 report challenging reseachers to collect a solid day's worth of data. CAIDA and OARC did one in 2002, 48 hour DNS root server collection expanded in 2007 inspired to further expand for 2008 2007 5 root servers (CEFKM) NaMeX internet exchange, AS 112 Open root servers, etc. what did they learn? 5% of clients responsible for 95% of load at root servers 1-2% of queries are legitimate; rest is junk. query rates doubled from 2006 anycast very effective shows where clients are vs where servers are longitude chart vs bar colours showing where clients connect to servers. what is all the pollution at the roots? lots of repeated queries, identical queries invalid TLDs seem to hit F.root what learned? you always need more disk space than you think collecting is easy; indexing is hard. questions to pursue with DITL data: DNS source of garbage trends in v6, DNSsec, etc. traffic/performance questions botnets? voice/video? is v6 different than v4? RND vs commercial traffic patterns? need to accomodate privacy/legal concerns Make data selectively available, anonymize it. index it into CAIDA's DatCat Please participate! http://www.caida.org/projects/ditl/ http://imdc.datcat.org/ ===================================================== Peter Loesher slide decks are all on the website, btw, so feel free to read through thm on your own. Joao_Damas(a)isc.org BIND, AAAA and the root servers finally, AAAA records for the root servers in the root-servers.net zone initially 6 out of 13 are providing v6 root service about 80-100qps of v6 right now. F root uses v6 in same way as v4; all local nodes are advertising /48, covering /47 advertised on the global nodes to avoid blackholding 13 of 43 nodes worldwide serving v6. Most traffic going to paris and amsterdam, rest goes to global nodes in bay area. very little in japan, possibly due to peering culture in japan. peering with f root over v6 we are ready almost everywhere, let them know, they'll peer with you via v6 BIND changes BIND needs no code changes on its own. ISC will provide updated hints file in 9.5 in the mean time you can fetch your own copy from: ftp://rs.internic.net/domain *whew* PGP signing party in Garden room, quick break. Final talks will start at 11.

1 0

2008.02.20 NANOG 42 current state of 10gig pluggable technologies
by Matthew Petach 20 Feb '08

20 Feb '08

Continuing on with the notes from this morning's talks, pushing out the notes while we take a bit of a break. :) Matt 2008.02.20 10 gig pluggable technology Richard A Steenbergen from nlayer ras(a)nlayer.net Many thanks to Todd for donation of the slideset! Title is 10gig, but it applies to other technologies as well; fiberchannel, etc. What are pluggables? transcievers you plug into your router or switch, they can drive multiple media types (copper, fiber, short reach, long reach, etc) There's standardization among the parts, so that vendors can use common parts. technical benefits, you can mix-and-match media types on the same router/switch card; easier replacement in the event of optical failure. financial benefit--pay as you populate. reusable, easily re-deployable elsewhere cards are reusable as technology evolves; can migrate from fiber to copper easily. standardization lowers costs. standardized, allows for interoperability standardized via vendor MSA (multiple source agreement) specs published under SFF, which defines the strict specs to be followed. First gen pluggable technology; started in ethernet land with GBIC, first came from the fiberchannel world; commonly in use from 1998 to 2002. >From 2002 onward, started shifting to sfp form factor; multirate, so you can use same pluggable for 1g, 2g, 4g fiberchannel, can use for gig or OC48 SONET First introduced digital optical monitoring (DOM) in 2004. better density. 1g to 2.5g, had GBIC and SFP. For 10G, many, many choices (7) 300-pin XENPAK XPAK X2 XFP ... Problem in operator community now is that with so many options, sparing is hard. slide with overall size comparisons. First 10G was 300-pin MSA; not pluggable, but a snap-on connector; 8-14W, 16x622Mbps interface. Same thing used for 40G XENPAK MSA, march 2001, second most popular at this point. power 6-10W XAUI (4x3.125G) PHY framer onboard 80km+ ZR, DWDM tuned, other exotic optics X2 MSA, July 2002 same electrical as Xenpak 4-5W, rest is same as xenpak, just smaller. Cisco and HP are only users of it; lukewarm support for it. very few exotic optics choices for it. XPAK MSA similar to X2, XAUI interface uses LC connector, different form factor, for use on PCI cards no exotic optics not used in network world. XFP MSA, march 2002 XFI (9.995-11.1G serial interface) variable speed serial interface PHY framer is offloaded; can be 10G, OC192, transport, etc. One of most commonly deployed optics exotic: ZR/DWDM, limited CX4, no LX4 very, very popular eliminating serdes for 10GBASE-RW is big power saver power 1.5-3.5W SFP+ MSA draft dec 2007 interface SFI (8.5 - 11.1G) same dimensions as SFP, same connector extend down to do 8G fiberchannel PHY framer offloaded no exotic optics smaller than XFP, offloads CDR function SFI similar to XFI, adds 8G FC speed limited power use power 1-1.5W 10gig pluggable component technology PHY, physical layer component PCS --physical coding sublayer PMA physical medium attachment sublayer serdes CDR clocking data recovery chip 10gig PHY comes in 3 variants 10GBASE-R LAN 10GBASE-W WAN similar to R, but wrapped in OC192 SONET compatible frame 10GBASE-X Lan phy 4x2.5G parallel signal LX4 is 4 optical channels, mux built in CX4 is 4 copper channels same choices for interconnect; XAUI is 4x3.125G parallel channels (8B/10B encoding) XFI/SFI, single 10gig serial channel uses 64B/66B, much more efficient encoding each time you convert, it increases heat, power, consumes port space, etc. Ultimate goal is to match technologies; if you have 4 parallel lanes on each side better XAUI to 10GBASE-X XFI/SFI to 10GBASE-R/W So XAUI to 10GBASE-R, inefficient XENPAK has room for serdes stages, fortunately Future directions for technology? new technogies replace old 10GBASE-LRM long reach multimode replaces LX4 10GBASE-T to replace old CX4 for copper uses standard cat6/cat7, 8p8c (RJ45) connectors Advantages of offloading PHY framer can use same pluggable for multiple protocols allows better component reuse, lowers costs can use in transport gear with forward error correction 10GE WAN PHY much bette when done on host can use any PMD without having to buy special pluggables improves sparing, lowers costs, expands PMD options improves WAN layer control signalling and alarms provides access to SONET alarms, path trace, etc. vastly improves troubleshooting with OC192 carriers Advantages of offloading the CDR make pluggable smaller, uses less power disadvantages mostly zero sum gain; no components saved, just moved to board not all CDRs are created equal may want higher quality on longer runs can't upgrade pluggable to get better EDC (electronic disperson compensation) technology Evolution of pluggable power use (watts) power consumption going down over time; different classes of power use. Not only dropping power, but removing entire power classes. comparison chart of optics slide. Software side of pluggables talk to host via low speed control bus; power down, alarm signalling, etc. Digital Optical Monitoring lets you see power levels on the link, speeds up troubleshooting. TDR functionality for copper hosts can also read EEPROM data allows vendors to lock users into particular pluggable. Vendors don't make their own pluggables, they're made by JDS Uniphase, finisar, hitachi, etc. pluggable has EEPROM, set vendor string $1.4B in profit at Cisco from reselling pluggable optics. how keep customers locked in: FUD (won't support it, might blow up, support contract is void, etc.) financial component; many sellers, few buyers; if Cisco controls 70% of market, they can set the prices, and prevent them from being resold outside their channel. Vendor locking--if it's not ours, router won't support it. Big customers are pushing back on that. counterfiting cisco is now the new way to make money, as you make profit by charging for name. new model is doing feature impairment; certain features like DOM are set so that it's only usable with their own pluggable. What format is right for you? XENPAK has best selection, well stocked, most exotic optics options downside; very large, draws a lot of power, not protocol agnostic, not friendly with WAN PHY X2--smaller than xenpak, lower power allows for higher density full support for LX4/CX4, pushed by cisco very little deployed base, few sources, few exotic optics same as XENPAK, not protocol agnostic XFP--advantages--large deployed base, cheap, easy to find, technologies still being upgraded on it. power/density pretty good for most users. protocol agnostic, easy spare, good WAN PHY cheapest option for long reach/DWDM SFP+ -- very high density (48 per blade) physically same as SFP may allow for extremely low cost device (GoogleSwitch) downside; hard cutoff on power budget, hard to do longer reach optics; no DWDM, no LR, no ZR; won't ever be able to be full replacement for XFP because of that.

1 0

RE: IPV4 as a Commodity for Profit
by Paul Ferguson 20 Feb '08

20 Feb '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 >1. Where is the current demand for IPv4 coming from? Plenty of analysis here. > I never thought I'd be doing this but: Can we please move this thread elsewhere? - - ferg -----BEGIN PGP SIGNATURE----- Version: PGP Desktop 9.6.3 (Build 3017) wj8DBQFHu9n1q1pz9mNUZTMRAh93AKDSBuNQZqcrXlNOhdytdEYVJDGLRACfSN7k dhZMaUnjtOofL5gsOJ2Db/E= =cMU1 -----END PGP SIGNATURE----- -- "Fergie", a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/

4 3

2008.02.19 NANOG 42 Taiwan Earthquake Aftermath notes
by Matthew Petach 20 Feb '08

20 Feb '08

Sorry, quick flurry of notes all at once now that things are wrapping up. ^_^;; Matt 2008.02.19 Aftershocks from Taiwan Earthquake Two presentations, and the IPv6 hour is starting now... Randy Bush has some things to say about the IPv6 hour. The IPv4 LANs have been turned off; you will note that you don't have good v6 connectivity even if you're a v6 expert. Failure is as good as success for this. Thunderbird and Firefox have v6 DNS resolution turned off by default. Macintosh--if you put in v6 DNS server IP address, if you have capital A in it, it drops it! ISC DHCPv6 has issues Cisco NAT-PT has issues Linux based NAT-PT substituted in isn't scaling. So, we've learned a LOT already! The experiment has already been an excellent success as far as Randy is concerned. So, on to the talk. Martin Brown from Renesys will talk about the Taiwan earthquake analysis. With contributions from Alin, Todd, and Earl, all from Renesys. Will look at shape of aftereffects, and then will look at fallout, the shift in transit patterns. Large earthquake hit Luzon Strait, south of Taiwan on 26 Dec 2006 7 of 9 cables were severed in strait reviewed at APRICOT in Bali in 2007 2 not cut: Asia Netcom's EAC, and Guam-Philippines All cables reported reported on Feb 14, 2007. Renesys is like route views, but they do way more processing on the data. Adjacent or 1 hop away from 65% of all internet transit providers Focus on prefixes geo-located in Asia region. Defines what a network outage is, what unreachable means, and what unstable/flapping networks are. Pattern of taiwan earthquakes; shape of impact. ramping up of outages and spikes in instabilities. smaller quake on dec 27th Recovery pattern is typically noisy Outages/immediate aftermath of the quake, 10 days. 4 or 5 big quakes on the 26th, but outage ramps up slowly; the 27th quake has huge spike after that, much like last stick in Jenga. Almost 4000 networks suffered an outage due to the quakes. China, Indonesia, India hit very hard by it. Instabilities, same basic shape, more noisyness to it. same countries hit for outages and instability. Looked at the severity of impact; factor out baseline instability and outage for each country; compare that median to the peak; china/hong kong hit worst. About 70 times more outages in peak at hong kong as a result of the outage, 55 times more in taiwan For instability, china showed 1300 times more instability in 10 days after quake as in 2 weeks before the quake. what did it look like after the event, who went to new providers? Looked at transit relationships, mapped them into market regions, and ranked them based on size. So, score first, then rank. they geolocate all prefixes first of all, give it a location give score to prefix based on length pre-cidr are discounted, probably less well utilized also look at transit patterns for that prefix. ignore any more specifics that share same transit pattern. Now looking at all AS-to-AS relationship; they track all adjacencies on net. Will categorize the nature of the edge. computationally expensive, but lets them track all the relationships. gives a way to sum a score to a geo location or market. relationship between scores is important for ranking, the raw score doesn't matter. Don't show traffic volumes, profit, customer satisfaction, etc. If you have a retail score, if you show up adjacent in a market. probably in the region. Sprint is biggest transit for Sprint; but not much of a retail edge there. Can look at trend, see who gains or loses market share. coloured countries on map are most affected by quake. Look at the changes in the region since quake. compare by size, by deltas, and who gains, and who lost? India gained, Vietnam more than doubled in size four of 22 countries affected, look at the breakdown of who serves them. Can see which edges are interesting, and see who is growing. CW, tiscali, seabone/TI Fairly clear the transit patterns shifted as result of quake. Chunghua provided transit during the aftermath until the restoration. ATT lost Hanaro at quake, then VSNL dropped during time of repair, lost a bit of Asia Netcom, did well with Bharti. SingTel did very well; picked up China Netcom towards end of year; Vietnam Telecom chose singtel PCCW jumped 10 points, picked up starhub out of singapore close to cable repair point. telecom italia jumped 15 places in ranking; they got singtel, but no sharp jump in prefixes. simple metric of prefixes over time doesn't show whole story. Need to also see how *many* people chose the prefix over time. So new edge score is PPT (prefix, peers, time) sum the amount of time the peer saw the prefix routed on the edge during a time interval all prefixes have same weight cannot distinguish between an edge with a lot of prefixes seen by a few, and an edge with a few prefixes seen by many. In the aftermath of the quake, world prefers to use telecom italia to reach singtel. CW gains big in India, Bharti, TM net, jumps huge in the chart. Tiscali gains from Asia Netcom, 6000 prefixes, and wins providers in Hong Kong, Philippines Chinese providers grew, but didn't grow relative to other providers, so they dropped ranks. Still living with fragile internet, in asia and middle east; the cuts in mediterranean region from yesterday highlight that. Need more diversity in the region, both east and west. Q: CNET networks--were you able to detect partitioning, where japan could get to china, but not US, for example? Most likely, but they didn't focus on looking for that in this analysis. Q: Alin, Renesys, Telecom Italia and Singtel were peering before quake; after quake, relationship changed to transit, the prefixes shifted, and were picked up by rest of net. Q: Paul Ferguson, can you draw 30,000 foot view of impact of quake to the middle east cuts, to show relative impact of outages? They can can do that. Taiwan outages were 2.5x as bad as Alexandria, but press impact was higher. After taiwan quake, huge drop in spam and botnet attacks. Todd asks if that means he doesn't think the middle east is as good at spamming as Asia?

2 1

IPV4 as a Commodity for Profit
by Raymond Macharia 20 Feb '08

20 Feb '08

Hello the article here http://www.networkworld.com/news/2008/021308-ipv6-delay.html is an interesting read given the current state of IPv4 depletion/IPv6 conversion operational climate. As it is indicated, it's a proposal and there are considerations as to whether it makes things better or worse. Regards Raymond Macharia

20 48

newspeering
by Alex Rubenstein 20 Feb '08

20 Feb '08

Please, before you read on, put your [coke can|coffee mug|drinking apparatus] down, so that when you laugh, you don't get any liquids on your keyboard. I am overhauling our news transport network the past couple days. Phasing out some P2-300's and such. Anyway, we have tons of news peers out there, of which I have no idea how to contact anymore. Some of these peers go back to 2001. The issue is that our DNS will change for both inbound and outbound feeds. Whoever news peers with us will need to change to newspeer1.nac.net for both in and out. Email me and let me know who you are so I can actually have this information in the future. You know who you are, there are at least 40 to 50 of you on this list, I estimate. If you don't peer with our news box, and want to, email me your pertinent info. Back to your regularly scheduled hub-bub.

1 0

NANOG42 streaming down?
by William Petrisko 20 Feb '08

20 Feb '08

Anyone else not able to connect to any of the streams of the general session? http://winmedia.internet2.edu/nanog Servername incorrect or server busy. Try again. http://www.nanog.org/qtstream.mov 404: Not Found rtsp://quicktime.internet2.edu/nanog-qt.sdp 404: Not Found http://www.nanog.org/nanog2.ram rstp://198.108.1.99/broadcast/nanog2.rm Requested file not found bill

3 3

2008.02.19 NANOG 42 Peering BOF XVII notes
by Matthew Petach 20 Feb '08

20 Feb '08

Yay! Last one! Cheers to Bill for another successful peering BOF. Matt 2008.02.19 Peering BOF XVII notes A handful of new people, first-time NANOGers Bill Norton starts off by explaining a bit about what a peering BOF is. Who attends it, and for the new folks, what should you expect. Bill Norton shows slide show of what peering coordinators do. Over last 8 or 9 years, peering bof has grown pretty large. Aim is to have community seating, let everyone see who's speaking as much as possible. Goal is to have some folks you'll want to peer with by the end of the meeting. Inspire some dialog, and get people to exchange information. Most important part is to get the right people in the room; get the right people talking to each other; the technical details follow on from that. Bill's job is to facilitate that process. Very full peering agenda peering survey efficient technique for enforcing internet peering policies summary proxy socialogical alaysis of peering coordinator interaction asr transit playbook peering personals, batch 1 ken florance, netflix, video and peering jeffrey payne, p2p, topology awareness and peering great debate--does peering still make sense with transit pricing dropping peering personals part 2, people come to front, mingle some more. Tom Scholl starts off with Ren and Greg, goal is to get some communication going between peering coordinators. Take survey online, see results at end of march, early april. http://tinyurl.com/3xoa6g Sample questions--do you use IPv6 unicast BGP peering? do you use BFD? do you use 4 byte ASNs? what is largest frame size you use on peering links? what is your biggest concern when deploying a new feature? This will be first survey that covers lots of technical issues. Josh will put link at the bottom of the global peering forum page to make it easy to find. Thanks to Ren, Greg, Tom, and don't forget to participate! David Smith--Cisco--efficient enforcement of enforcing policy between peers. Peer should only have IP reachability to your customer prefixes. Should NOT be able to use your AS to reach one of your other peers. Most ISPs rely on BGP to enforce policies between peers. But if you use routing tricks, like pointing a default, there's nothing that stops the packets, they're only enforced in control plane. Alternate options--only carry partial routes on your peering routers; doesn't stop local peers using you as transit. Assumes your peering router is for peering only, no transit, no core. Static interface based ACLs; headache to maintain accurately, hardware issues, etc. proposed technique ISP tags peer prefixes uniquely within BGP and FIB tables ISP tags external packets that ingress peering interconnects based upon longest match prefix match within FIB ISP forwards or discards packets that ingress peering interconnects based upon associated packet tag value Not a futurist talk technique available today supported on high-end routers; not supported on the 7600s use table-map command to do prefix tagging use bgp-policy command to do QPPB for packet tagging use service-policy command via MQC to do packet classification QPBB provides glue between control plane and forwarding plane. IOS configuration sample shown route-map set-prefix-type permit 10 match comm 1 set ip qos-group 66 ip comm-list 1 permit ASN:66 int p3/1 bgp-policy dest ip-qos-map service-policy input peer-in provides much more scaleable manner to enforce peering policy, prevents people from using you as transit against your will. Use of QPBB as glue between control and data. Changes in BGP are automatically reflected in data plane control based on the tagging from BGP. Complements many of the other BGP control plane techniques in use today. All hardware based on the platforms mentioned above on engine 5 and above. You can do this on a Juniper with DCU too. :) arguably more elegant as it can be done in the firewall filter as a global policy. With DCU, instead of matching on QoS value, you can give the forwarding class a name value to make it more readable. Not a new technique, but with the interactions between BGP and MQC, it's not widely understood and used. RAS notes that one note about using it on Juniper; he and Tom talked about it a few years ago, you can build a RIB with just your customer routes, and then send default down an LSP to a traffic sniffing box. Dave notes you can look at snmp counters on DCU and get alerted when it crosses a threshold level of traffic as well. You'd also see discards in the class of service via the MQC MIB. Do people think this is a widescale problem? Do people think bandwidth stealing is happening? Ren notes they found someone who was stealing hundreds of megs of bandwidth in AMSIX; they're no longer a member there. Steve Wilcox has seen this happen at LINX as well; this is a theft issue, so it's odd that we think of it as a technical issue, rather than a criminal issue. Do people see this as an issue on a private peer, or only on the public sessions? Chris Malayter, social network around peering, helps run the peering IRC network many of us hang out on. Less activity on the channel once NANOG started, guess we're all talking face to face, not online. shows you when users are generally online. irc.terahertz.net, if you were here, you know the channel. :) Peering personals, first batch now. Hopefully people here are looking for peering at the exchange points. VideoBox, AS36472, peer with them, get free membership 365Main, PAIX, San Jose Equinix, SFMix, Heavy transit load from them; 98% is transit. No clusters out east yet. If you have eyeballs, they've got the content your eyeballs want to see. AS36472.net tens of gigs of traffic. They do ACL on their side, see things like leaked OSPF traffic, and tier1s that are on the public fabric, with high bar, starting with X, they leak OSPF and may send packets towards you. Simon Ferret Veoh AS40415, another video provider, no peering yet; content delivery through partners at the moment; thinking about insourcing at some point. P2P as well as streaming; if people use p2p, saves a lot of CDN volume; most is still over CDN right now; video on website is CDN based, download to keep is p2p. Aim is to get a good TV experience. Bryan Berg, Imeem, AS36119, social networking site; old school napster meets youtube, ad supported music streaming site. Looking for eyeball networks, 15G of traffic. Ash, SJC, public, private in CHI, LAX, PAIX PAO Looking for south america, philippines, malaysia, portugal About 30% peering so far; wasn't as hard as he'd heard, but it's not a walk in the park. Jeffrey Payne, peer-to-peer QA Used to be at real networks, used his gear to broadcast nanog in the early days; very knowledgeable about the current peer to peer arena. Q: What techniques do peer 2 peer networks use to determine which host to pull a fragment of a file from? Well, let's go back to what p2p is; early model of p2p used central sites and servers; then RIAA got involved, gnutella had horrible search but was decentralized; then Kazaa came along, got search better. Now, bittorrent is a huge explosion of micronetworks. The simplest model in use has no intelligence; don't mind the local transfers, but the remote ones hurt--but the system has no knowledge about near vs far peers. Why does remote vs local matter? Is it due to asymmetric last mile? It's more the bit-miles issue; how much are you moving over how many network miles? commercial systems are trying to introduce intelligence; aimed at one-to-many data delivery, with central command and control that you expect from commercial delivery system. So, for commercial interests, need to work to optimize flows and reduce costs in one-to-many or few to many model. The P4P model has ISP intelligence in the Itracker to give some sense of topology to the torrent tracker. with the old system, it couldn't tell local peer from remote peer. Newer system has to 'infer' from hop counts and performance indices to figure out who is better/closer than others. Frank, talks about IPTV on your computer, p2p supported CDN, can take content from multiple sources. Goal is to add more control into tracker to optimize the assignment of clients to seeds. Pando networks, P4P networks speaks up, come talk to him, he can get you all the information you could want on it. But will networks really participate out of the goodness of their hearts? Not so much that, as about reducing the operational expenses. Is there some level of buyin from the people here around that architecture? Verizon notes that you don't need to provide a huge amount of data to get a big win. who would be willing to contribute data? a good chunk of the room. consumer adoption is the challenge; now that flash supports sockets, can start doing it in the browser. traffic shaping devices are targetting more and more protocols, like http, when they used to just target p2p traffic. The commercial side is agnostic; just don't want to be traffic shaped. Moving from mp3s to mpeg4, traffic volume is really skyrocketing, we need to start thinking more intelligently about the problem. Many of the major p2p efforts are behind it as well. Can we do more optimization in CMTS systems, DSLAMs, and try to get network layer efficiencies. Any last minute thoughts? Both trying to create these commercial content delivery systems; may not be seeing it quite as holistically, but both trying to improve the situation. Peering News Ren Provo moved from SBC/ATT to Comcast now, she can now watch premium channels while saying "No" Peter Cohen moved from uh...well...to Sprint as peering coordinator. The great debate--does peering make sense anymore? Pro--Patrick Gilmore--peering still makes sense Con--Guy Tal--peering doesn't make sense anymore 2 minutes per side, referreed match. Patrick still saves money to peer in the right place Also provides multiple vectors, provides better resiliency in case of issues. Also, provides better scaling as traffic grows, allows for scaling as needed to specific areas. Provides shorter paths to talk to people in an area. Helps getter better transit deals when you go to good central peering locations. Guy Peering is hard; as was pointed out earlier, our transit prices are decreasing, it's getting close to transport cost; why spend your resources and time doing an exercise that's someone else's core competency; focus on your core competency instead. Let the carriers do the heavy lifting for the most expensive part of the work; they have multiple vectors already, they know how to scale. Why try to map who the right peers are to go after, and how to protect yourself from your peers--let your transit handle that headache. Patrick Peering is indeed hard, it's hard to set up a network that's resilient; but there's many things that are hard but are still useful. You'll have to have some level of clue already to set up your network; it's not that much harder to add BGP to the portfolio of skills needed. Transport is getting more expensive relative to transit, but they haven't crossed yet; you can still get ring-based paths for less than transit for now; so you might as well go to the peering point, get access to your transit provider as well as your peering partners at the same facility. Guy While transport hasn't crossed transit, you also have many other costs; engineering resources, a NOC, pay for rack/power at peering sites, router costs to go into peering points, etc. There's many fixed costs, like buying a peering port; per-port cost may be more than you pay for transit initially. Transit gives you SLAs, peers don't. The more traffic you give to your transit, the better your cost. And whatever you may think, peering isn't free. You'll always have to have some transit relationship; if you're not already SFI, you'll never be able to be SFI, that club is closed; so, since you're not going to be able to meet everyone's peering requirements, you'll need to keep transit, might as well let them handle it all for you. Patrick Scalabiility, reliability, performance need same skillsets you already need to run your network to do BGP; doesn't greatly increase your skillset, should be a minor increase over skills you already have; will allow you to go to places where you can get better pricing, and will allow you weather internet daily issues that come up. Guy Having a higher percentage of peering will raise your costs initially; until you do a large amount of it, it won't save you money. And it's not true that it's a minor skillset above running a basic network; let someone with the real skillset handle it; you don't have to worry about someone patenting peering and suing you over it, or being accosted by Patrick asking whether or not you peer yet. So, who made the more compelling case--people get to vote once for each side. The people will be counted. "Patrick believes peering is hard, but is also useful" 62 and 45 in favour of Patrick? 14 and 14 in favour of Guy 107 to 28 in favour of Patrick. Now, feedback from audience. Port costs on public peering fabrics have not tracked in relation to transit costs dropping. Peering to someone can help get additional ports in place faster to provide capacity, rather than getting two different networks to upgrade their transit links. Prices in Europe are so different, it makes more sense in Europe than in US; prices in US need to get more in line with EU. Brokaw agrees that without visibility on links, it may be very hard to tell how well your transit provider will be able to handle surge loads or increases in traffic over time. Patrick notes that with most of the networks, it's the transit port to the other end of the conversation that has capacity challenges you can't fix. Over to Ken at netflix to talk about challenges with handling video traffic. They used to do DVD rentals only, now doing bits over the wire as well. In January, went from money capped, to model where service isn't capped. Unlimited PC viewing, not MAC viewing yet, DRM issue. Future looks like people consuming things on PCs, TVs, etc. Have services embedded in DVRs, HDplayers, game consoles, things hooked to both TV networks as well as IP networks. Volume can end up growing drastically. Want to keep traffic off people's backbones if at all possible; optimize p2p delivery, cache, etc. People are consuming their content through IP networks more and more. Goal is to get the bits into the access networks; can peer at centralized locations, or they can put equipment in their pops if that makes sense. If you have ideas on where they can stick their equipment, come let him know. Cheol Hee Yun AS3786 LG Datacom Largest IDC in Korea 4Gb in Korea, 10Gb in US, they are present in PAIX Palo Alto, and Equinix San Jose They don't have peering in Korea, unfortunately at this time. Again, don't forget to fill in the survey!

1 0