- NANOG - lists.nanog.org

2006.06.06 NANOG-NOTES network-level spam behaviour
by Matthew Petach 07 Jun '06

07 Jun '06

2006.06.06 Nick Feamster, Network-level spam behaviour [slides are at: http://www.nanog.org/mtg-0606/pdf/nick-feamster.pdf Spam unsolicited commercial email feb 2005, 90% of all email is spam common filtering techniques are content based DNS balcklist queries are significant fraction of DNS traffic today. (DNSbls) Using IP address based spam black lists isn't so useful. How spammers evade blacklists will be discussed as well. Problems with content-based filters ...uh oh, some technical glitches... Content-based properties are malleable low cost to evasion altering content based on scripts is too easy customized emails are easy to generate content based filters need fuzzy hashes over content, etc. high cost to filter maintainers as content changes, filters need to be updated. constantly tweaking spamassasain rules is a pain. false positives are always an issue. Content-based filters are applied at the destination too little, too late -- wasted network bandwidth, storage, etc. ; many users recieve and store the same spam content. Network level spam filtering is robust (hypothesis) network-level propeerties are more fixed hosting or upstream ISP (as number) botnet membership location in the network IP address block country? are there common ISPs that host the spammers, for example? Avoid receiving mail from machines that are part of botnets. Challenge--which properties are most useful for distinguishing spam traffic from legitimate email? very little if anything is known about these characteristics yet! Randy gave a lightning talk last NANOG about some of this. Some properties listed. Spamming techniques mostly botnets, of course other techniques too we're trying to quantify this coordination characteristics how we're doing this correlations with Bobax victims from georgia tech botnet sinkhole other possilities: heuristics distance of client IP from the MX record coordinated, low-bandwidth sending looked at pcaps coming in from hijacked command and control station from bots trying to talk to it; spamming bots, Bobax drone botnet, exclusively used to send spam. Collection two domains instrumented with MailAvenger (both on the same network) sinkhole domain 1 continuous spam collection since aug 2004 no real email addresses--sink everything 10 million + pieces of spam sinkhole domain #2 recently registered Nov 2005 "clean control" domain posted at a few places not much spam yet--perhaps being too conservative contact page with random email contact, look at who crawls, and then who spams the unique email addresses Monitoring BGP route advertisments from same network Also capturing traceroutes, DNSBL results, passive TCP host fingerprinting, simultaneous with spam arrival (results in this talk focus on BGP+ spam only) Mail Avenger, not an MTA, it forks to sendmail or postfix, it sits in front of MTA, does things like do DNSBL lookups, add headers, passive OS fingerprinting, as the spam is arriving. Also logged BGP routes from same network that got the spam; see connectivity to the spamming machine at the time. Picture of collection up at MIT network. Mail Collection: MailAvenger X-Avenger header. best guess at operating system, POF, DNSBL lookups, traceroutes back to mail relay at the time the mail was sent (used for debugging BGP) distribution across IP space plot /24 prefix vs how much spam coming from it. steeper lines mean more spam from that part of the IP space; you can see where spam is coming from. bunch comes from apnic, cable modem space, etc. few interesting things to note; still redoing legitimate mail characteristics. from georgia tech mail machines, it's legit plus spam, need to split out better. between 90.* and 180.*, legitimate mail mainly. Is IP-based blacklisting enough? Probably not: more than half of spamming client IPs appear less than twice. Roughly 50% of the IPs showed up less than twice; but that's a single sinkhole domain, would help more across multiple domains. emphasizes need to collaborate across multiple domains to build blacklists; any one domain won't see repeated patterns of IPs. Distribution across ASes 40% of spam coming from the US BGP spectrum agility Log IP addresses of SMTP relays Join with BGP route advertisements seen at network where spam trap is co-located. A small club of persistent players appears to be using this technique 61.0.0.0/8 AS4678 66.0.0.0/8 AS21562 82.0.0.0/8 AS8717 somewhere between 1-10% of all spam (some clearly intentional, others might be flapping) about 10 minute announcement time of the /8 while spam is flooded out. Might be interesting to couple this with route hijacking alerting to filter out if this is really a hijacking vs a flapping legitimate route. A slightly different pattern; announce-spam-withdraw on a minute-by-minute basis. really really egregious! Why such big prefixes? flexibility: client IPs can be scattered throughout dark space within a large /8 same sender usually returns with different IP addresses visibility: route typically won't be filtered (nice and short prefix length) Characteristics of IP-agile senders IP addresses are widely distributed across the /8 spce IP addresses typically appear only once at the sinkhole Depending on which /8, 60-80% of these IP addresses were not reachable by traceroute when we spot-checked some IP addresses were in allocated, albeit unannounced space Some AS paths associated with the routes contained reserved AS numbers Odd AS numbers injected, usually well-known to make it look more legitimate. Length of short-lived BGP epochs 10% of spam coming from short-lived BGP events Spam from Botnets Example: Bobax approximate size: 100k bots one sinkhole domain--this is ONLY stuff that is verifiable as coming from bots via command and control hijacked IPs, intersect the single sinkhole domain, so much smaller data subset, but well correlated and verified. Proportionally less spam from bots in 61-90 range; that tends to be where BGP route hijacks happen instead. Most Bot IP addresses do not return 65% of bots only send mail to a domain once over 18 months. Some hang around for a *long* time. About 20% stick around for several months. collaborative spam filtering seems to be helping track bot IP addresses. Most bots send low volumes of spam most bot IP addresses send very little spam regardless of how long they have been spamming Effectiveness of blacklisting: only about half of the IPs spamming from short-lived BGP are listed in any blacklist spam from IP-agile senders tend to be listed in fewer blacklists Looking at 8 different spam blacklists, checking when the spam arrives at the sinkhole. Known Bobax drones listed in more DNSbls than the BGP agile senders. About 90-95% of the Bobax bot drones are listed in one or more DNSBLs. Suggests some of the spamming bots are listed more than other techniques--that is, bots are easier to identify than BGP-agile spammers or spammers using other techniques. Harvesting tracking web-based harvesting register domain, set up MX record post, link to page with randomly generated email addresses Example Phish: a flood of email for a phishing attack for paypal.com all to: addresses harvested in a single crawl on January 16th 2006 emails received from IPs different from those who crawl. X-mailer headers totally diffrent. Lessons for better spam filters: effective spam filtering requires a btter notion of end-host identity distribution of spamming IP addresses is highly skewed detection based on network-wide, aggregate behavioru may be more fruitful than focusing on individual IPs large, emergent properties. two critical pieces of the puzzle botnet detection securing the internet's routing infrastructure compare distributions of spam to legitimate mail, see if certain spaces are more likely to send spam than legitimate mail. Questions: Q: Steve Bellovin, columbia university bots from strange ASes, is tunnelling taking place from bots to BGP speakers? A: Not sure if there's evidence or not; some data from TORS?? but TORS latency may be too high. Q: Fingerprinting to try to identify who is doing things, see how many hosts are actually doing this? Many addresses being used, how many hosts does it actually represent? A: Not sure, haven't checked that. Haven't checked on aliasing, since not much was seen from a single IP. NAT'ing? What about hosts hopping? (same host using multiple IPs?) Not sure, they didn't do that correlation. Q: Randy Bush, IIJ, they did do OS fingerprinting, so some of that are in the paper. didn't do anything with the traceroutes, though. Q: Matt asks what the difference between the two domains was; was one of them a recognizable word or name, or were they both random character strings? A: they were both random character strings, but one of them had been used to host a real website for a while, which might explain why it gets such a huge volume of spam compared to the other. Q: Matt points out that for some networks, receiving spam is actually a good thing, as it helps balance out traffic ratios, which helps during peering negotiations. Q: Randy Bush, IIJ, responding to Matt about traffic ratios: only those backbones who are on ADSL should they care which way traffic goes. :P Curious to work with large networks, see if filters could be installed to detect it, and possibly take action.

1 0

RE: Zebra/linux device production networking?
by Mark D. Kaye 07 Jun '06

07 Jun '06

Hi, I am also newbie poster so likewise plz be kind. I tend to agree with the comments made so far, however depending upon the business, budgets are not always available that might match the requirements and hence I can to some degree understand the use of such boxes for small organisations. I would be interested to know how many "software" (for want of a better description) routers are in live production in this kind of environment i.e. the 99.9999% Uptime variety, from speaking to people albeit randomly in data centres it would seem to be more common than one might expect. Also does anyone have any peering policies which would exclude peers with "software" routers specifically, most have a requirement for the ability to support stable BGP peering but I have not seen any specific exclusions for such "devices"? Mark ________________________________________ From: owner-nanog(a)merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Tiffany Snyder Sent: 06 June 2006 23:29 To: Nick Burke Cc: nanog(a)merit.edu Subject: Re: Zebra/linux device production networking? IMHO, it's a bad idea. A less intrusive alternative might be a FreeBSD based platform running Xorp/Quagga. Tiffany. On 6/6/06, Nick Burke <mrmud(a)mrmud.org> wrote: Greetings fellow nanogers, Long time lurker, first time poster (please, be gentle!). After looking at the archives, I didn't see this particular discussion, so here we go. First, a little background.. My CTO made my stomach curdle today when he announced that he wanted to do away with all our cisco [routers] and instead use Linux/zebra boxen. We are a small company, so naturally penny pinching is the primary motivation. That, and the sheer joy of watching me squirm. He has informed me that he has found "many people" who do this for their "core devices". I'm not so certain about this whole situation, so I humbly ask: How many of you have actually use(d) Zebra/Linux as a routing device (core and/or regional, I'd be interested in both) in a production (read: 99.999% required, hsrp, bgp, dot1q, other goodies) environment? And, if you care to spend this much time, what pitfalls/benefits did you find out about after implementation? Has there been any discussion (or musings) of moving towards such a solution? I've seen a lot of articles talking about it, but I've not actually seen many network operators chiming in. Here's the article that started it all (this was featured on /., so likely you've read it already). http://www.businessweek.com/technology/content/nov2004/tc20041129_5206_t c024.htm and another: http://www.networkworld.com/community/?q=node/5693 Feel free to respond off list. If anyone else is interested, I will of course summarize to list or to individuals. (ps, particulars are deliberately not included.. I'm not looking for advice, just if anyone has any solid experience with this..)

3 2

2006.06.06 NANOG-NOTES DNS reflector attacks
by Matthew Petach 07 Jun '06

07 Jun '06

(I was going to try to get all the notes from today's panels out before going to bed, but I fell asleep on my keyboard finishing up these notes, so I think I'm going to wait and send the batch of Tuesday and Wednesday notes out after things wrap up on Wednesday. Sorry about the delay, but I need a bit more sleep I think. ^_^;; --Matt) 2006.06.06 Morning welcome, and introduction of Chris Morrow, panelist Please fill out survey today if you're going to be leaving! Frank Scalzo, Verisign Recent DNS reflector attacks. Attacker breaks into innocent authoritative DNS server, publishes large text record; then does queries from zombie army against that record, with sources spoofed with victim IP. 5 gig attack, 2.2G made it, 3gig didn't. E.TN.CO.ZA DNS attack, 64 byte query, 63:1 amplification, 4028 byte answer 34,668 reflectors. Victim sees 5G of traffic, 144,142bps per reflector, 13.5packets per second 4.5DNS answers per second. reflectors won't see this as anomolous for the most part; top talker only sent 8.5 answers per second. No visibility into the attacker at all, but best guess was 79Mb of source generated 5GB of responses. Record was maliciously installed; 2 auth servers, 1 compromised; 65% response, 35% name error. Answer comes in 3 fragments, larger than normal MTU. Attack came in 3 phases. first port 666, then port 53 and 666, then all 53. Port shifts are nearly instant, so fast command and control system in place for it. Filter out open recursive DNS servers; you can't put ACL in for 500,000 DNS servers. What about limiting DNS packets to 512 bytes? will break things. What about blocking 53 outside of your network hierarchy, force people to use your resolvers? What about discarding fragments? Challenge is getting your upstream to implement it, unless you have hardware and pipes to handle the flood coming at you to start with. Some ISPs won't do it unless they see live attack traffic, and a 24 minute attack is too short lived for ISPs to see and react to. data from Jan 11 - Feb 27 this year. Attack queries/second consistent with avg reflector qps. one reflector sent 1.9M DNS answers to 1593 victims, 605 different queries to generate answers. 180TB of attack traffic on Feb 1st. after feb 15th, ramped down. Assume 4KB response packet, see attacks between 3G and 7G, the scary part is that it only took 130Mb to generate the 7G attack, and the 3 gig attacks are all from less than a fastE connected compromised web server. 500,000 reflectors with 2G source could generate a 120GB DoS attack. Top victim got over 130Tb of attack traffic, top bunch are all over 100Tb 65,461 ports used, Top port is less than 10% of traffic though top 20 domains used, mostly innocent bystanders. Internet root . was second highest domain used; certainly can't filter *that* out. Fundamental challenge; UDP lacks 3 way handshake, easy to spoof DNS is easy target, so many unsecured DNS servers Other UDP servers need to be evaluated as well DNS closing 500,000 open recursive DNS servers will be very, very painful. poor separation between authoritative and recursive DNS servers. BIND allow-query ACLs, recursive DNS servers should not accept queries from outside. What if it's an embedded system like a wireless gateway? We depend on large records for DNSSec, etc. Beyond open recursive DNS servers root domain "." was used most authoritative name servers will answer with an upward referral doesn't include actual IPs, but it's still 438 bytes, and pretty much every DNS server responds to it. Source validation IETF BCP 38 How do you manage 70,000 ACLs on 500 routers? what about people who are multi-homed with static routes? what about legacy stuff that works but shouldn't? strict RPF breaks with traffic asymmetry; loose RPF doesn't help with this. ISPs see the problem as long, hard, expensive to overcome, and they're right. If we never start trying, we'll never fix it! Close open recursive DNS servers DNS servers should include filtering SOHO router vendors should fix their DNS proxy code, don't listen on outside interface BCP 38 otherwise we'll be jumping from protocol to protocol. Questions? Q: What does verisign do to protect their DNS servers? A: Anycast, massive peering and transit capacity Q: Jared Mauch, NTT/America; he turned on unicast rpf on the NANOG upstream link. 372,000 packets that people here have sent failed the RPF check. BCP 38 is hard Paul Quinn asked what percentage of the traffic that is. Bora Akyul, Broadcomm--any data on source ranges on the packets being seen? He could look at the 1 in 10,000 netflow sampling to see, but the individual link is a /30, looks like a normal customer link. The Merit router isn't RPF'ing either. Q: Ren Provo asks when they will peer; A: not yet, next few months, Miami Terremark, and other sites domestically and internationally in next year and a half.

1 0

Fwd: At Dig BIG, Nothing Runs Like a Deere (Really Backhoes and Fiber together)
by Robert Boyle 07 Jun '06

07 Jun '06

All, Just in case any of you want to see how the other half lives... and destroy some infrastructure after learning more about how to build it this week. :) > Work with >JOHN DEERE Equipment > to build REAL utilities in a REAL work setting. > >Dig BIG takes place at the best place on the planet for underground >working and learning - The Planet Underground. > >For two BIG days, The Planet Underground becomes a huge work site >with acres of REAL underground utilities. > >You won't want to miss this REAL BIG opportunity to use the John >Deere equipment at Dig BIG. Work side by side with colleagues with >the best equipment available! > >Free registration available for a limited >time.<http://underspace.com/mailer/redir.php?id=239&st_id=164> I subscribe to Underground Focus magazine and I just received this in my inbox. Register at http://www.underspace.com/TPU/register_digbig.php in case you really do want to attend. Now back to the regularly scheduled noise with some content due to NANOG 37. -Robert Tellurian Networks - The Ultimate Internet Connection http://www.tellurian.com | 888-TELLURIAN | 973-300-9211 "Well done is better than well said." - Benjamin Franklin

1 0

Lightning talk selections
by Steve Feldman 07 Jun '06

07 Jun '06

The Program Committee has selected these lightning talks for presentation during the general session tomorrow morning: Analysis of DNS Root Server Location, Martin Hannigan Metro WDM in provider networks, Alex Pilosov Fashonably Late - What Your Networks RTT Says About Itself, Anton Kapela Thepiratebay busted - network impact, Mikael Abrahamsson Alerting prefix owners of hijacks in near real time, Mohit Lad Reigning in the botnets operating on your network, Rick Wesson Thanks to everyone who submitted a proposal, unfortunately we couldn't fit them all into the available time. Steve Feldman PC chair

1 0

2006.06.05 NANOG-NOTES Peering BOF notes
by Matthew Petach 06 Jun '06

06 Jun '06

(This time around I opted to go to the peering BOF and take some notes. It's the one downside to parallel tracks--wish I could be in two places at once. ^_^;; --MNP) 2006.06.05 Peering BOF Bill Norton introduces the Agenda; unfortunately, my laptop took so long to boot, I missed the Agenda slide. Doug Toy?, Transit Cost Survey, data collected at NANOG 36; he's just here to present the collected info, not really representing anyone. Recap: At NANOG 36, people indicated their cost per Mb and commit level. length of contract was usually 1-2 years. 42 data samples collected avg $25/Mb $95/$10 were the extremes. Avg commit level 1440 Mbps Other observations as expected, cost per Mbs tends to decrease as the commit level increases. Tier1's are more expensive Cost tends to vary more with Tier 1 providers than with others. between 0-500Mb commit level, prices are all over; at higher commits, prices level out at the bottom. Question: Mbps, is that the cap, the usage, inbound plus outbound? A: That's the general 95th percentile higher of the two inbound and outbound. Committed amount. Graphs tend to approach a hard bottom; as commit increases, doesn't change all that much. Bottom is around $10/Mb, even though commit levels increase. Of samples collected, 2/3 were from Tier1 providers. 90% of contracts are 1-2 year in length, so didn't cause much variance. Tier 1 definition is based on Wikipedia definition. Questions from audience? Q: Data looked pretty clean; were there samples pulled out to make it look cleaner? A: No, other than people who left fields blank on the survey. Q: was there a timestamp of when contract started? A: much of it wasn't complete. Mostly within the 1-2 year range for length as well as start date, so nothing really ancient in there. BillNorton; people had some concerns about violating NDA or contract details when filling out the survey. Where do we draw the line in doing these types of surveys? SteveGibbard; NDA is agreement between transit provider and customer, and this was anonymized and voluntary. Data is interesting, both for purchasers and sellers of transit. Q: 42 samples graphed, there were 80-100 people in the room at the time; so the real comments from the rest of people weren't counted? A: No, there were less than 50 submissions total, of which 42 were complete. Q: Patrick; many people put more than one transit provider on their form; how were those other transit providers handled? A: no clue, he just got a spreadsheet with data. Back to Bill Norton Peering Lists Issue -- make available to customer prospects? 15 mins. Peering disclosure dilemma: customers often asked for peering list, sometimes peerings restricted under NDAs Metric for determining connectedness, capacity, resiliancy. Is there a better metric for customers? IX capacity in/out Peering pipe size? ISPs are getting commonly asked about this, based on hands raised in the room. How many people lose business because the customer doesn't get an answer? Sylvie, VSNL notes they provide the info when they're under an RFP; they won't give capacity, they give an aggregate, they won't go peer by peer, that would be a violation of NDA. BillN: are the NDAs written to allow total numbers like that? Sylvie: you should not disclose capacity per location or per circuit, but they don't forbid aggregate total numbers. BillN: is there something else that could be given to the customer that would satisfy their question without revealing what Chuck: A lot of ISPs lie about their peerings; he runs AMSIX, people claim to have multiple gig to the peering exchange, he knows they don't really have that much. Patrick: but he can look at the peering stats on AMSIX--Chuck notes only members can. Patrick: customers ask how many gigs they can send to a provider; it's available headroom, so they ask their upstreams how much available headroom is left. Most providers are having a lot of trouble getting the right capacities to the right networks. The reason many don't answer is they don't like the answer they have to give. Ted Seely, Sprint, how do you solve the problem? There's lots of traffic that needs to be exchanged, how do you fix it? Patrick: how about everyone upgrade to 10gigE in many places? If you can't afford it, stop selling bandwidth. But most people can't go to all the different providers, they have to buy from a small subset of providers. RAS: No technology problem with doing it; it's the money. Not charging enough to cover the costs of the technology you need to install to cover the bandwidth. Ted notes you can't just link at one spot, you have to connect at six places, and you need to have links in and out of the site to support the volume, etc. Patrick: can you tell us how much they exchange? 40Gb times 6 providers in six locations is probably more traffic that Sprint has in total. Ted Seeley; it's a time scale issue--yes, it can be solved, but in what timeframe? BillN feels it's reasonable information for a customer to request. Is there a proxy that customers could use that would give similar information? Tristan Horn, Q: has anyone use skitter to try to figure out the answer on their own? Ted: last skitter pass was March, 2005--data isn't current enough to be useful. Michael Abrams, tcp test, consumers can run to see how much traffic they can pass along; it becomes very quickly apparent who has problems, and consumers tend to reach quickly to that, go elsewhere. Shows packet loss and latency at different throughputs. Sweden, size of CA, they have 4 exchange points, and want more. Still gamers are most vocal about any problematic interconnects. Video Peering Video Stats and distribution models -- all YouTube Peering Personal -- Colin They're getting into the peering fray, and only a year old. This is gigs and gigs, has potential to dwarf current peering traffic. Current issues could be tiny compared to the flood of potential issues when hundreds of Gbs comes flooding towards customers. Voxley? on the web; 320x240 pixel image, want to do a one hour episode, will be about 210MB file. currently, show is watched by 10,000,000 households. So, that's 2.1PB to transfer all those episodes. If you try to stream it, via say iTunes, over 3 days, would be about 64Gbps solid for 3 days. That's a pretty big number. That's one episode, from one show. American Idol is even bigger. He claims a better way is using peer to peer. 2.5% of the agents could be propagation nodes, or 256,000 PCs as transfer agents; that would fit on existing broadband connections. What about multicast? problem at core, end distribution; pushing to 256,000 is much more doable. Ted Seeley notes you still have to pass the unicast streams eventually, so you still haven't solved the issue. If you don't do multicast or caching at the edge, you've just shuffled the problem. Swedish police, 100Gb of peer-to-peer traffic at peak, AMSIX lost 10gig, LINX lost 5gigs, probably lost about 40Gb weds/thurs last week when the swedish police shut it down. Which site? Pirate Bay torrent tracker. Peer to peer has similar replication pattern to multicast? But it's not localized properly, unlike multicast. RickW, the pirate bay site, that site was only distributing meta information. peer to peer traffic can be 12-20% higher because of the duplicated traffic. Peer to peer networks aren't as efficient as unicast streams. Matt notes 64Gb, 256,000 subscribers isn't that big a stretch anymore. Peer to peer is unlike multicast, because it doesn't have any localization to the best of his understanding. Do peer to peer systems use latency to control who they share with? Somewhat; faster peers are selected over slower peers. Comment from audience is that live events are still going to be the challenge; HDTV is getting gb/sec from cameras, needs to feed it out, no chance to cache, so multicast ends up being the only viable option for it. Question about is the assumption that getting it over the internet is the right model, vs using settop boxes? How do you get a whole country to change their viewing habits? Korea Telecom uses peer to peer to pass content around; it is limited to a certain DSLAM or POP, which answers Matt's localization/topology issue. Ted Seeley; yes, people are watching material online vs set top boxes, the paradigm is shifting already. RAS, works through numbers; a DVD of MPEG2 will be around 4GB, if you drop out the special bits, audio tracks, etc; heavily compressed, 600MB, to around 1GB mpeg4 for DVD quality, 2-4GB compressed for HD quality. if we take 1000MB filesize, cost per movie to deliver this. Assume $10/Mb, you get 1Mbit/sec/month for $1, Assume normal traffic curve, 150GB/month for your per Mb $10 point. Assume it's mpeg4 compressed to 1GB, that's 150 movies per $10/month, that's less than the cost of postage. How does it best make sense to distribute these files? Compression reduces file sizes about 15% per year. interstreamtv.com, has article about it. 1Mb/sec will do 320GB/month. So number of movies would vary. Peer to peer want immediately, and want quality, so the model might be problematic. Caching at the pop edge would help. And live events, multicast would help, use unicast to fix lost packets. How do you compensate for lossy networks? Forward error correction, which means you lose the benefit of compression you were hoping to get. Larger buffer won't help for retransmits unless it's TCP. Curtis points out live streaming vs Richard's downloaded movies. RAS was replicating Netflix model, you order in advance, get it later that day. If you want to stream live stuff, it needs to be multicast, provider needs to do the replication. How much of it is unavailable in the country where the provider is located, that's why you can't cache or work with the provider to distribute it. Brokaw notes more streaming is coming; more video stuff will be coming in the months and years to come. For the access networks, how many are multicast enabled, and are ready for helping this scale? 3 hands. Does the community agree we have a bit of a challenge here? Matt Peterson, encryption to make sure the file is different for each user, to make sure it's legal. Bittorrent with RSS to get metadata, weather channel model, ultraDNS, Akamai model multicast model What about skycache? One-to-many distribution? Colin from YouTube will start off the peering personals. They've recieved an AS, they push about 20Gb sustained, their first datacenter went live in the middle of March. It is definitely growing. Not multicast enabled, no demand for it. The average show they have is 4 minutes, hundreds of millions of different videos, not a good multicast target. 20% growth month to month, 50 million videos viewed per day. 20gbit/sec outbound? "Ask a ninja net neutrality" video. Product launched in December, that's when they saw real traffic; been moving more and more in house in March and April. They're pushing 20gig in house, not from CDN network. This is is all short form video content, hopefully no licensed content sneaking in. San Jose Lundy on equinix fabric PAIX on paix fabric Los Angeles on equinix fabric Linking all 3 locations together. Metro, long-haul providers take a long time. Lundy is poorly connected. Getting connectivity at 10 gig (Fiber swaps, metro links, cross datacenter crossconnects) Other items Starting to see traffic from v6-v4 gateways most hardware providers do v6 support in software Seeing v6 traffic from v6-to-v4 gateways from Asia. Are people doing much successful v6 on modern devices? Force10 s50, 4500 from cisco, all seem to be software based v6. Should they have a native v6 presence, go dual stack, what hardware would be good to use? sup720 will be the cheapest hardware to do hardware v6 at line rate 10 gig. Can't do it on the lower end. Person from puerto rico notes that hardware switches can do 200Gb/sec of IPv6 switching, why can't routers do it? Frustrated audience reactions from all over about differences between switching and routing. Robert Seastrom--should you do v6 at all? Should you be a pioneer, and make the v6 people happy, sure, do it; if you want to make money, no. Q: how do they make money, and will that model support their bandwidth? it's advertising supported, hopefully it will support their needs. Q: how do you compete with Google? No idea, he's not a business person. Q: Content providers often don't get peers at public sites, much harder to get peers than access providers; are they running into that? A: Right now, peers are still coming to him; he won't know about resistance until he goes out asking for it. Summary: Open Peering Policy peering(a)youtube.com colin(a)youtube.com AS# 36561 How much of the 20gig is going across peering links vs transit links? It's all transit right now--he's just now asking about peering. Jeffrey notes that if there are people who pay transit to get your content, they will peer regardless of ratios. DanGolding notes that the tier1's are stuck on a treadmill; they can't peer with you even if they desire it, for fear of messing with their own ratios. Josh asks if they will go to other locations than just the west coast of US. Colin notes that they're doing a datacenter month. Chicago is next. Dan Golding, network neutrality on the peering front. One year old company, video content, already doing 20gig/sec. This is a buttload of traffic, and they're already getting into the peering mode; will this traffic ultimately dwarf the rest of our traffic? Network Neutrality, an open discussion. Tier1research. What it means to peer. network neutrality: an introduction what is network neutrality simply put, it's the idea that internet service providers should treat all legitimate traffic the same way regardless of origin and destination. Why should I care? a significant portion of the internet is based off mutual... The Bell perspective An unfair load the bell companies-att and verizon-aren't big fans of neutrality. they feels they're carrying traffic for others and are uncompensated or not compensated sufficiently Google, Yahoo, making big profits from the sweat of their brows. The Bell Answer $$$ needs to flow from content carriers to the Bells. How would Bells make this work again? everything old is new again. in the PSTN world, the bells charge reciprocal compensation for voice calls terminating on their networks. Talk of prioritization... [he's really whizzing through his slides!] Content perspective The internet works because of neutrality non-neutrality would stifle innovation The bells have a bad business model, they're asking content to help prop it up. Prioritization is a protection model; The bells are in a power position their control of the internet is at an all time high. the conglomerate controls a huge amount of access, they are tier 1 internet peers, SFI peering with each others. Others are tempted to jump on. MSOs see non-neutrality as a way to keep other's VoIP off their networks On the other hand, the Bells will squeeze them; the cable companies lack peering. His take; leans towards content. Would be fair, except their networks were built under monopoly auspices. Consumers have little choice in many locations some sympathy for bells; it's their network, after all. How will this end? possibility #1; regulation of internet to enforce neutrality. regulation makes the internet less flexible probably vague law with lots of FCC regulations Regulatory Affairs officer has to vet ACLs Difficult to do filtering quickly ability to provide tiered services will be more difficult. Possibility #2 bells get their way VZ and ATT become tier 0 internet carriers depeer everyone else as soon as current restrictions expire charging reciprocal compensation to all others where are the MSOs? other access providers? why buy transit from anyone other than a bell? potentially disastrous scenario internet growth is stifled forces you to always buy transit from the bell companies. Possiblity #3 threat of regulation causes a continuation of status quo. best solution loss of neutrality could destroy the internet internet regulation could stifle it; and the bells are VERY good at regulation. perhaps the evolution of tiered consumer services could come about by volume, not quality consumers warming to this model content providers sponsor some access models. Patrick Gilmore: Tier 0, how does that work when VZ and ATT don't make up the bulk of the internet anymore. What about the rest of the world? The bells get regulations changed when the world doesn't move in the direction they like; reciprocal compensation got removed when the ISPs became CLECs and got lots of compensation from the Bells. Now, regulations modified. Sean Donelan; it's net neutrality, not internet neutrality, so what about other players? broadcasters, radio, etc. For voice calls, not such a big issue. broadcast networks don't have much of an idea of what this is about; not sure they see the applicability of this yet. When they do, they have huge lobbying power as well. Bells have very sophisticated lobbying efforts, make them very smart. The newbies like Google and Yahoo aren't that savvy in washington yet. 60% liklihood that the end result is regulation; is *not* a happy outcome. Is there some pivot point that may make people change their models? Patrick Gilmore asks for non-US peering coordinators; who would care if ATT/VZ depeered you? Not many respond Curtis--not just a US problem; similar discussions in European countries. At what layer does the competition come into play? In European, there's no text on the table yet, but there's discussions taking place at least. Somewhat of a reverse problem, it's the new players looking at tax monies. AMSIX, much better competition on the last mile and the infrastructure, the big players can't play the game the same way. Dan Golding notes that if we had many different ways of getting local loop to your house, it would be less of an issue. Incent development of alternate methods; wifi, 3G/4G/5G networks, etc. Mikhail Abramson, with high speed cellular, mobile is making network neutrality less of an issue, since you do have more options. The GSM providers are happy with the internet bandwidth usage on mobile data, it's making them really good money. Matt Peterson? Last comment about cell phones; but people are used to packet munging; most people are used to a clampdown at this point; they may not even notice anymore. Until Grandma using YouTube understands network neutrality, it'll be this community only fighting/caring. If we all went to a common $10 provider, we could create a new tier0 and bypass the bellheads. A few peering personals. Jeffrey Papen, MySpace, AS 33739, well over 20gigs, in LA and by august, SJ, CHI, ASH, all Equinix locations. Needs to be in peeringdb.org!! People are now talking about putting peeringdb.org in their peering contracts. Stafford Rau, CLEC in Portland, ops, in WA, northwest, currently acquiring ELI from frontier, will be about twice as big. SIX westin, PAIX, AS12003, 7845, will roll all of it including ELI into 7385, they don't get 5650 unfortunately. 2-3gig transit now. Philip, MSO in Canada 5769, 13571, about 15gigs, videotron.ca Matt Peterson, SIX seattle, copy it in SF, neutral, non-profit, 365 Main, mostly content providers, CNET, craigslist, others. load up your blogs, SFMIX, 365 Main, he or Tim Pozar from UnitedLayer. MetaInterfaces, video, 5gigs, makes money, tells you what kind of video it is. ^_^; Vish from Netflix, looking for potential peers in SJC, CHI, ASH, vish(a)netflix.com really aiming for eyeballs. Ann claiborne, prolexic technologies, also in peeringDB, NOTA, LINX, peering(a)prolexic.com hoping to come to west coast. Dash to restrooms and then to the next BOF.

6 5

AW: Phantom packet loss is being shown when using pathping in connection with asynchronous routing - although there is no real loss.
by Gunther Stammwitz 06 Jun '06

06 Jun '06

Hello Alan, Thanks for your reply. I know that icmp is being handled by the router's cpu and that the forwarding is being done in hardware. The interesting part here is that the loss starts at the router where the asynchronous routing begins and is even being seen at the final destination host - but this only happens with pathping and mtr. An ordinary ping doesn't show this problem. The problems in mtr/pathping disappear when a synchronous routing is being established. What's the case for this? What are the programmers of mtr and pathping doing wrong or is there another problem mayb in the implementation of icmp? Gunther > -----Ursprüngliche Nachricht----- > Von: Alan Clegg [mailto:alan@clegg.com] > Gesendet: Dienstag, 6. Juni 2006 17:27 > An: Gunther Stammwitz > Betreff: Re: Phantom packet loss is being shown when using > pathping in connection with asynchronous routing - although > there is no real loss. [..] > > I'd guess that the routing devices are just droping/ignoring > ICMP echo request packets. They are the least interesting > packet, and are quite often, responses to them are given > extremely low priority. > > Just my guess, > AlanC

1 0

2006.06.05 NANOG-NOTES Network Neutrality panel notes
by Matthew Petach 06 Jun '06

06 Jun '06

(since there's no slides for these online anywhere, and the slides were going past pretty quickly, I have to apologize for the gaps in the notes ahead of time. --MNP) 2006.06.05 Network Neutrality Panel [slides are not yet online] next up is the controversial subject of network neutrality; Bill Woodcock will be chairing the panel, so Randy Bush can go be a member of the audience again. Bill Woodcock: network neutrality has been in front of press and legislatures for the past several months, and has been in the works behind the scenes for almost a year. Brokaw Price, peering at Yahoo! Sean Doran, free agent, "rooting" for Sprint back in the day Sean Donelan, now at cisco, Gene Lew, at neustar now, has done cable operations before. Sean can pretend to be Vint Cerf for this panel. :D Network Neutrality--what does it mean to operations people? History: Michael Powell, Feb, 2004, defined four internet consumer rights (chairman FCC) freedom to access content freedom to use applications freedom to connect personal devices freedom to obtain service plan information History of net neutrality concept: feb 2005, madison river telephone company consent decree "Madison River shall not block ports used for VoIP..." August 2005, FCC policy statement access lawful internet content run applications and services subject to the needs of law enforcement connect legal devices that do no harm to the networks competition amongst... All of these principles are subject to reasonable network management. That last bullet is what gives telcos ability to quote QoS as a mandatory network management requirement. March 2006, internet non-discrimination act senate bill 2360 only 2% of americans have a choice in last mile. shall not interfere with block, degrade, alter, modify, impair, or change bits or content May take measures to protect customers from attacks may protect their own network infrastructu May 2006 Internet freedom and non-discrimination modifies clayton anti-trust act. Passed based on party lines. Turns over to Sean to talk about his thoughts. Sean Donelan Doesn't represent anyone right now. The Huck Finn approach. And no, you can't configure this on your router. What are we talking about? Rep John Conyers (D-MI) "internet as we know it is at risk" Same guy noted in 2003 about cable operators being smart enough to not poison their customer pool Not really a new issue: ANS CO+RE in 1991 unapproved networks filtered from R&E gateways ANS and CIX, June 1992 ANS agreed to "provisionally" interconnect CIX proposed filtering resellers (194) NSF NAP/NREN solicitation (1993) required NSPs connect to all priority Network Access Points (NAPs) uncertainty created opportunities for new service providers pizzahut.com debate--couldn't reach it from university networks, but you could reach it from UUnet. Two-tiered internet even back then. Regulations chasing change TitleI -- General FCC Authority (pre 1984) TitleII common carrier voice/phone calls/later data TitleIII spectrum licensing broadcast TV and radio TitleVI Cable Television (post 1984) FCC moved DSL (but not UNE-L and cable modems) back into TitleI again. VoIP is still unknown. Telecom act of 1996 was another biggie. before 1996, enhanced services transmitted over a common carrier. After 1996, info services were defined by the telecom act; they run over telecommunications provisioned by anyone, not just common carriers. That's why cable companies can offer telecommunications even though they're not common carriers. Also why Radio and TV can put IP in subcarrier without being common carriers. So, we have a really odd blend, they're not mutually exclusive. Now you have the potential for new entrants from any direction. Most home services now come over cable companies; multiple resellers of same product wasn't enough to satisfy customers. Customers are very fickle Interfering with a customer's use of the Internet would hurt the provider's business. No-one can predict what the next "Killer App" will be. and *everyone* can complain Both sides need each other to succeed. Predicting in advance what customers want and will consider improvement vs interference is hard to the point of being nearly impossible. And what customers consider an improvement one year may become interference, or what was once interference will now be considered improvement. If you start writing regulations, you imply investigative and reporting requirements along with it. Who would enforce these regulations? And regulations seldom prevent people from being evil, the government simply sets the price for being evil. Broadcast fairness doctrine--equal time; nobody can buy public advocacy on national networks, except for politicians, who get the best rates. Kingsbury committment--ATT had to interconnect with everyone--but that meant you didn't need a second long distance company, so ended up supporting monopoly expansion. Universal service backfired by giving a monopoly to those giving out the free phones to... Back to Bill Woodcock: Arguments in favour preference will go to dominant players instead of technological innovators If market isn't free, consumers must be protected from predatory practices. Nondescrimination is part of common carrier status Many access networks were built under monopoly protection are they thus the moral property of the consumers who were forced to pay to create them? Arguments against descrimination required to protect customers from "bad" traffic. service bundling needed to encourage future investment in infrastructure Nondescrimination is compelled speech in the first amendment sense. QandA for the panel. Q: Free speech: can carriers claim "common carrier" protection from responsibility for customer content, and simultaneously claim a seemingly-contradictory 1st amendment protection from compelled speech? Sean Doran--Canadian, working in EU on similar issues; can carriers degrade non-customer or competitor's traffic as part of deregulatory process? Unbundling local loops, etc; if people were given access to these components on an equal cost basis, would incumbent be able to sustain a biased offering around degraded service? Risk was taken that rather than regulate a monopoly, the regulators would create an artificial basis where the competitive playing field was forced to be level; seems to have worked, they've seen 100fold to 100,000fold price decreases since 1997; double-digit mbit connectivity four double-digit prices in US dollars are available in major cities; can get triple digit BW for triple digits in stockholm for example. Mutually competitive providers, incumbent PTT is forced to sell access to all comers at same price level. Regulating on other levels tends to enhance the power of the incumbent, rather than limit it, since they generally have the stronger lobbying power. Bill Norton at mike--In US, given the large amount of video traffic on the net, do cable companies have enough infrastructure to handle forseeable traffic levels? Since he can't be compelled to pay for speech that isn't paid for, he'll just replace your ads with his own as they pass along the wire. What about mandatory revenue sharing? Bill Woodcock passes the question to Brokaw Price from Yahoo! Well, Brokaw notes that's a push model, vs the reality of the internet, which is a pull environment. If incumbents did that, the model for the internet would start to collapse, as customer wouldn't be able to be sure where their traffic was coming from; more likely the incumbent would be likely to compete directly, rather than trying to circumvent. But that's likely to create more of a walled garden environment, and we've seen those walled gardens collapse in the past. Sean notes AOL could do it since they have proxies (ad replacement, that is). More likely that telcos would be able to show that a free-for-a-month link is better than your other providers, so you should pay them to get that better connectivity--and since you can't see into their routers, you have no way of knowing *why* their direct paid-for link is better than the alternative transit provider's link. The congestion issue is all this just an excuse for backbones to not upgrade, and just pass the pain and costs around to different players? Gene notes that Neustar is not taking any official position on network neutrality. But then back to earlier question; the ads on baseball games are often not what exist in the stadium, they're what the broadcaster decides their audience should see. Being a former carrier, it's debatable about skipping upgrades and forcing QoS payments instead to deal with congestion issues. Where he lives, he has 5 broadband providers he can chose from; in major metro markets, that's often true, and with municipal wifi coming into the play, could be a dark horse that will force the issue. The competition issue: do network neutrality and local loop unbundling matter only in the absence of competition? Brokaw: well, competition is generally good, gives customer a chance to decide which provider is better, assuming the customer has the wherewithall to move. In areas where the customer can't change, yes, it's likely to be more of a problem. In the content space, if he only has one option to reach a huge consumer base, price elasticity raises his ugly head, and it's likely the user experience will be adjusted to reflect that increased cost; and that's an ugly outlook for the internet if we follow it out to its conclusion. And if the big players are forced to do that, it's likely the smaller players will be under even more pressure. 3 choices: do we have sufficient competition now? Do we need more regulation? Or do we need more competition? Brokaw: consolidation is reducing the number of competitors in the market; it's still a free market, so we're not yet down to monopolies yet. It's a tough call. We're struggling with what the end vision will be. Sean Doran: US needs more competition; there's already too much regulation in all the wrong areas. Congress is introducing bills all over the place, likely to do for network neutrality the same thing they did with spam. not likely to turn out well. Sean Donelan: regulation tends to freeze things as a status quo; just adds more regulation, rather than adding more real competition. Regulation just gives more brand names without real competition. He feels regulation is useful as long as it goes in the right direction; tries to make everything look alike, rather than rewarding innovation. Greg Lew: Is there enough competition, or more needed? Well, yes for his area, but large areas (20%) don't have access to broadband, so those areas need more competition, definitely. Q: Michael Sinatri?, UC Berkeley. Service Provider (TV station) pays the content provider for the content, and negotiates the ad space placed on that. If the content is being provided for free, what right does the SP have to modify the content mid-stream? Content Accelerators--shrink images before being sent; if customer pays for that, does that infringe on content provider's original data? Q: Ted Seeley, Sprint. Incumbent ILECs spend millions to build out infrastructure to support that video distribution. Why pay SBC when you can buy it from Sony? That is, the ILEC spends the dollars to expand their network, hoping to recoup the cost from the customers over a 10 year period; But there's no guarantee that customer will be there for the full 10 year amortized time for paying back the buildout; if they jump ship to someone else, how do those upgrades get paid for? Q: Patrick Gilmore: if telco wants to charge content providers for bits they're sending, what are they going to charge each other for the high volume of illegal content they're sending back and forth to each other, which according to their own stats is the bulk of their traffic? Survey forms available on the web! If you're sticking around, fill it out. If you're leaving, fill it out! Maps with food places; will start up again at 2:15 for afternoon bofs. Lunchtime now!

1 0

2006.06.05 NANOG-NOTES IPv6 deployment at Comcast
by Matthew Petach 06 Jun '06

06 Jun '06

Randy Bush, moderator of the next section He begged to do the introduction for a specific reason; deployment of IPv6 that is beneficial to this companies' P&L; possibly the only one in existence thus far. He did a very studied and purposeful view of using IPv6 to benefit his company! IPv6 @ comcast Managing 100+ million IP Addresses [slides are at: http://www.nanog.org/mtg-0606/pdf/alain-durand.pdf Alain Durand Office of the CTO Director IPv6 Architecture Alain_Durand(a)cable.comcast.com Agenda Comcast needs for IPv6 Comcast plans for IPv6 Challenges simplistic view of comcast IP problem 20 million subscribers in video 2.5 set-top boxes per subscriber 2 IP per set top-box per DOCSIS std. total 100 millions IP addresses needed that's not including high speed data, nor comcast digital voice, nor mergers/acquisition Used to use RFC1918 for cable modems. that space was exhausted in 2005 Comcast recently was allocated the largest part of net 73 and has renumbered cable modems in that space. In the control plane, all devices need to be remotely managed, so NAT isn't going to help us IPv6 is the clear solution for us However, even we are starting now, the move to IPv6 isn't going to happen overnight. Triple play effect on the use of IP addresses 2005 HSD only 2006 T+ Cable Modem 1 1 Home computer/router 1 1 eMTA (voice adapter) 0 1-2 Set top box (STB) 0 2 total num of IP addresss 1-2 8-9 (assume 2.5 STB per household IP Addresses: Natural Growth vs New Services nice graph--based on trends, not real data Contingency plans: use public address space use "dark" space (pre-RFC1918 space) federalization (split into separate domains) (trying to avoid that) IPv6 strategy start early deployment plans started back in 2005 deploy v6 initially on the control plane for the management and operation of the edge devices they manage DOCSIS CM, set top boxes, packetCable MTA (voice) be ready to offer customers new services that use IPv6 LATER, not now--first step is to just be able to manage their own gear. migration to v6 must be minimally disruptive. deploying v6 must be in roadmap for all vendors ops, infrastructure, systems must be ready to support v6 devices. over time, IPv6 will penetrate Comcast "DNA" Deploy v6 for IP addrs of the CM and STB architecture: dual-stack at the core, v6 only at the edges deployment approach: from the core to the edges backbone->regional networks->CMTS->devices this is an incremental deloyment; existing deployments will be untouched in the beginning Follow same operational model as with IPv4; lots of DHCP! News Flash: All routers on Comcast IP backbone are IPv6 eanbled first ping on 10GE production backbone TTLs aren't quite working properly, still checking on that. [so, even mainstream vendors still don't have v6 working quite properly yet] New CM will be v6 ready (dual-stack capable) On an IPv4 only CMTS, CM will have v4 address only On v6 enabled CMTS, CM will only have v6 address No CM boxes will have both; if they could support v4 on all, wouldn't have this issue to start with! Provisioning, Monitoring, Back-Office mostly software upgrade problem not unlike the Y2K issue fields need to be bigger in database and web scripts Should system "X" be upgraded for v6? does it communicate with devices that are v6 only? payload Q: does sstem "x" manipulate IP data that could be v6 (store, input, display) Comcast inventory analysis About 100 systems 10 need major upgrades for transport 30 need minor upgrades just for display/storage Back office management of cable modems. network transport will still be v4 however, back office systems may need to be modified to display/input/store v6 related data (CM v6 addr) Payload can be v6 while transport is v4. IPv6 certification Basic IPv4 compliance taken for granted today IP level component testing is limited IPv6 is still new technology maturity level of vendor implementations vary greatly some have v6 for 10 years even those have features not fully baked others have nothing, will rush to buy 3rd party stack. Bar for v6 product acceptance has to be higher than what we typically accept now for IPv4 Formal v6 requirement list before purchasing v6 conformance testing/certification to accept product v6 training most engineers have heard about it, don't know much fear factor can expect new hires to have 2-4 years of v4, but 0 v6 initial and continuous training process is critical! v6 vendors CM (cable modems) (DOCSIS 3.0/2.0b) CMTS Router Provisioning system OSS Video/Voice back-end systems Retail Market (Consumer electronics) Home Gateways Video (eg TV with embedded cable modem) Right now, provisioning system is most challenging. v6 protocols MIBS: some OSS vendors implement RFC2465 (deprecated) some router vendor implement partial RFC4293 (new combined v4+v6 MIB, but only v6 part) IGP comcast run OSPF v2 for IPv4 looking at OSPF v3 and IS-IS for IPv6 Integrating v4 and v6 security Integrating v4 and v6 QoS OSPF v2 and v3 share acronym, that's about it. QoS code points will be challenging; mark, and then trust QoS to deal with markings independent of the type of packets the markings are on. That's pretty much it. Not such a difficult process, mainly making sure the vendors do the right thing, beating on them with a big stick. Q: Dave Huberman, ARIN--without going into specifics of comcast--thank you, that's a very big landmark! You have upgrade so many items, spend all this time retraining staff, training new staff; how do you sell this gigantic cost to the 44th floor? A: The cost is non-negligible, but it pertains to business continuance, so in general executives aren't going to say no to it. Q: Lane?--similar question, it's an upper management support question. It costs money to do this! A: Extremely strong business motivation for doing it in the case of comcast; a good object lesson for others in how to get management on board with v6 migrations. More pushback from midlevel managers than from the top levels has been seen, actually. Q: Steve Schultz, NASA; any plans to implement IPv6 multicast, interdomain? At some point, will need to bring multicast to the set top boxes? Will they need both v4 and v6 multicast streams in core? A: They only want one multicast stream, from bandwidth size requirements, so will translate at the edges for now from v4 to v6. This effort is all for device management, though, so multicast won't be an issue for a while. Q: Doug Montgomery?; need for certification and profiles--do you see LOGO program for v6, as sufficient? A: Has been looking at v6 LOGO phase 1 and phase 2, they go deeply into neighbor discovery, but doesn't cover MIBs, or routing protocols, or transports other than ethernet. It's a minimum entrance requirement (phase 1 now, phase 2 in a few months), but that's not sufficient on its own; they have requirements above and beyond LOGO phase 1 and phase 2. Q: Merike, global security; some vendors have limitations on their v6 security features; what has vendor response been to them, since they're a big company. A: They have had to raise pressure on a few, yes; but since nobody wants to lose Comcast account, they have been responsive. Q: Tony Hain, Cisco. Doing this with management, not seeing any demand from customers; customers shouldn't know underlying protocols, so they'll never demand. Is there a plan to eventually get it to customers? A: it's really based on services they can offer; if there's a service that's v6 only that they can sell to customers, they'll roll out v6 to the home. Q: someone from Juniper: why comcast needs 100 million IP addresses. They use v4 right now, what do they use? A: they don't have 100 million yet, that's what they're growing to; using 10/8 and 73/8 for now, may use some dark space during the migration period. They're still using public space for now. Q: Bora Akyul?, Broadcom; you mentioned this in the certification. Plenty of RFCs that obsolete each other. IPv6 is richly optioned protocol compared to v4. How do you decide which options you want to use? Will you publicize those to your vendors? A: DHCP v6 vs stateless autoconfiguration, may be conflicting methods. They decided they wanted to use DHCP v6, NOT stateless autoconfiguration, for example. Q: Exactly! will they use a lot of extension headers? Q: No, keep it simple! Q: Randy Bush, IIJ? Two things he knows; a) what is the simplest way to get from here to there b) don't try to convert the users--they don't need to know what's happening at the protocol layer Q: Matt, Yahoo, asks about whether they will be planning to give net 73 back when the IPv6 conversion is done? A: Well, they have a bunch of other systems that need to communicate using v4, until they have a solution that can do v6, will keep the v4 addresses.

1 0

2006.06.05 NANOG-NOTES interdomain routng via Wiser, Ratul Mahjan
by Matthew Petach 06 Jun '06

06 Jun '06

2006.06.05 A simple coordination mechasims for interdomain routing [slides are at: http://www.nanog.org/mtg-0606/pdf/ratul-mahajan.pdf Ratul Mahjan David Wetherall Tom Anderson University of Washington now @ Microsoft Research the nature of internet routing within a contractual framework, ISPs select routes that are best for themselves. Potential downsides higher BW provisioning requires manual tweaking to fix egregious problems inefficient end-to-end paths An alternative approach: coordinated routing ISPs have joint control path selection is based on the preferences of both ISPs Potential benefits lower BW provisioning no egregious cases that need manual tweaking efficient end-to-end paths basis for interdomain QoS Existing mechanisms cannot implement coordinated routing route optimization boxes help (stub) ISPs pick better routs from those available MEDS implement receiver's preferences. Cannot create better paths that don't already show up in the routing table. What makes for a good coordination mechanism? MEDS have some nice properties ISPs can express their own metrics ISPs are not required to disclose sensitive info lightweight requires only pairwise contracts Provides joint control and benefits all ISPs. Our solution: Wiser operates in a lowest-cost routing framework downstream ISPs advertise their cost upstream ISPs select paths based on both their own and received costs. Problems with vanilla lowest-cost routing ISP costs are incomparable Can be easily gamed When you bring incomparable costs together, the ISPs that use higher costs win out. Cost normalization Normalize costs such that both ISPs have "equal say" Normalize such that sum of costs is the same. Makes the system harder to game. Bounds on cost usage Downstreams log cost usage of the upstream ISPs Compute the ratio of avg. cost of paths used and announced Contractually stipulate a bound on the ratio. Similar to existing ratio requirements. Wiser in action Announce costs in routing messages. normalization occurs between ISP pairs. Example results using major ISP topologies for experiments Wiser provides better control under link failure. Wiser produces shorter path lengths Implementation XORP prototype Simple, backward-compatible extensions to BGP embed costs in non-transitive BGP communities border routers jointly compute normalization factors and log cost usage a slightly modified BGP decision process Benefits even the first two ISPs that deploy it. Summary Wiser is a simple mechanism to coordinate interdomain routing may lower provisioning, reduce manual tweaking, produce more efficient paths and help with interdomain QoS Feedback: ratul(a)cs.washington.edu http://www.cs.washington.edu/research/networking/negotiation/ Danny McPhereson: Q: how do you normalize across multiple ISPs? A: routing advertisements happen on the sum of the costs announced from me to you, and from you to me. He derived it from different values in his experimentation; utilization and latency in general. Q: Randy Bush: Whatever metrics are, you just normalize by summing them up. But Danny notes if you have multiple ISPs, how do you normalize them together? Q: Danny: where was the 20ms of control plane savings seen that he claimed in slide 11? A: That was based on default ISP policy, prefer customers over peers, etc. So the delay was control plane plus data plane; it wasn't control plane alone. He based it on the old rockefuel equation. Randy Bush: vendors--this is cool stuff, open your ears. Break time now.

1 0