January 2007 - NANOG - lists.nanog.org

Re: Internet Video: The Next Wave of Massive Disruption to the US Peer ing Ecosystem (v1.2)
by Fergie 09 Jan '07

09 Jan '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Bill, Just as an observation, it appears to me (at least) that the most popular method of video distribution today is via GooTube. :-) I think it remains to be seen that that model will actually change dramatically to more of a "semi- real-time" model, regardless of the desires (or fears) of various vendors or operators. $.02, - - ferg - -- "William B. Norton" <bill.norton(a)gmail.com> wrote: Hi all - Over the last year or so I have been working with Internet video companies who asked essentially the same question - "What is the most effective way of distributing massive quantities of Internet (video) traffic?" This has become a significant issue NOW because a few of the largest US ISPs are turning away these n*10G Internet video transit customers ! Thanks to all of you that shared your insights, or let me walk you through what this community has found to date, and especially those of you who shared their data points and allowed me to cite you as a source. I'm at the point now where I'd like to share the current draft (v1.2) of this discussion paper with a broader audience, epsecially those who will allow me to schedule a time to talk through the draft with you. (I have found this is the most effective way to get feedback next to face-to-face walkthroughs over lunch). Here's the Abstract: Video Internet: The Next Wave of Massive Disruption to the U.S. Peering Ecosystem (v1.2) In previous research we documented three significant disruptions to the U.S. Peering Ecosystem as the Cable Companies, Large Scale Network Savvy Content Companies, and Tier 2 ISPs started peering openly. By peering with directly each other they effectively bypassed the Tier 1 ISPs resulting in improved performance, greater control over the end-user experience, and overall lower operating costs. This paper predicts a new wave of disruption that potentially dwarfs currently peered Internet traffic. Some of this emerging wave of Video Traffic is demonstrating viral properties, so the more popular videos are generating massive "Flash Crowd" effects. Viral Amplifiers (sites that do not host but rather highlight the most popular videos) amplify any viral properties a video may have. If we combine this flash crowd effect and the increased size of the video files downloaded, we see the crest of the first wave of significant incremental load on the Internet. The majority of this paper details four models for Internet Video Distribution (Transit, Content Delivery Networks, Transit/Peering/DIY CDN, Peer2Peer) across three load models. The cost models include network and server equipment along with pricing models for various distribution methods. Dozens of walkthroughs of this paper have led to stepwise refinement of the models and insights into why one would prefer or not prefer one model over the other. The summary is a comparison in cost-per-video across small, medium, and large distributions. The models (spreadsheets) can be made available to those interested. Bill - - -- //------------------------------------------------ // William B. Norton <wbn(a)equinix.com> // Co-Founder and Chief Technical Liaison, Equinix // GSM Mobile: 650-315-8635 // Skype, Y!IM: williambnorton -----BEGIN PGP SIGNATURE----- Version: PGP Desktop 9.5.2 (Build 4075) wj8DBQFFo9Lzq1pz9mNUZTMRAmonAKChhR7OS2yFlvnv7sVXqnShgPgrJACgsOHm QOf1iGUMwD2ktDO/8+1FJhI= =s8q2 -----END PGP SIGNATURE----- -- "Fergie", a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/

3 2

RE: Internet Video: The Next Wave of Massive Disruption to the US Peer ing Ecosystem (v1.2)
by Fergie 09 Jan '07

09 Jan '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - -- "Bora Akyol" <bora(a)broadcom.com> wrote: >I think this discussion is going towards the content that one would >**actually** like to see. On this, we agree. :-) >I understand there are people that don't watch >TV at all. I am not one of them. I have had a Tivo since when they first >came out. The problem that I see is that the product pipeline for how TV >content should be distributed and watched got constipated mostly due to >the pressure from the content owners (possibly justified). > Question: How is your content delivered to your TV/TiVo today? How will it be tomorrow? - - ferg -----BEGIN PGP SIGNATURE----- Version: PGP Desktop 9.5.2 (Build 4075) wj8DBQFFo+Auq1pz9mNUZTMRAiLQAKCvvUIdnN6Gows01Zb7kdilvIFHsACcC5Fx 00iSBH3YpOXtmjJ4Cbdr1DE= =bhtZ -----END PGP SIGNATURE----- -- "Fergie", a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/

1 0

RE: Network end users to pull down 2 gigabytes a day, continuously?
by Brandon Butterworth 09 Jan '07

09 Jan '07

> Given that the broadcast model for streaming content > is so successful, why would you want to use the > Internet for it? We now have to pay for spectrum, when you have to pay you look for the cheapest delivery path. Until we switch off analogue there is a shortage of spectrum so we have limited channels and no room for much HD All spectrum is for sale so someone can come in and buy it out from underneath existing uses. Hence spectrum freed by analogue switch off is less likely to be available for TV. As price is determined by the highest bidder you can be priced out of carrying on doing what was fine before. People have content they want to deliver where the business model requires cheaper delivery than traditional broadcast TV brandon

1 0

Re: Network end users to pull down 2 gigabytes a day, continuously?
by imipak 08 Jan '07

08 Jan '07

Brandon Butterworth wrote: >>> If this application takes off, I have to presume that everyone's >>> baseline network usage metrics can be tossed out the window... > > That'll happen anyway, what used to be considered high volume > content is becoming the norm with lots of start ups and old > school broadcasters getting involved. > Indeed. http://news.bbc.co.uk/1/hi/technology/6239975.stm : "Microsoft's work in developing IPTV (internet protocol TV), which allows programmes to be delivered live or on demand over an internet connection, would soon come to Xbox 360 games consoles. By the end of 2007 partner companies will be offering IPTV services to Xbox 360 owners, he said. " Hmmm... http://www.wired.com/wired/archive/6.04/mstv.html \a -- Andrew Simmons

1 0

Re: Network end users to pull down 2 gigabytes a day, continuously?
by Fergie 07 Jan '07

07 Jan '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - -- Joe Abley <jabley(a)ca.afilias.info> wrote: >If I acquire content the same time as many other people, since what I'm watching is some coordinated, streaming event, then it seems far more likely that the popularity of the content will lead to network congestion, or push up a peak on an interface somewhere which will lead to a requirement for a circuit upgrade, or affect a 95%ile transit cost, or something. > >If asynchronous delivery of content is as free as I think it is, and synchronous delivery of content is as expensive as I suspect it might be, it follows that there ought to be more of the former than the latter going on. > Completely agree here. $.02, - - ferg -----BEGIN PGP SIGNATURE----- Version: PGP Desktop 9.5.2 (Build 4075) wj8DBQFFoVX6q1pz9mNUZTMRArMxAKC1HcQzuRVtw7RizPH9Sxubpd4CyACfe9Mp IVrcy6mKMtdNdzu6qMMdpOs= =ehDE -----END PGP SIGNATURE----- -- "Fergie", a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/

1 0

Anyone have details on MCI outage yesterday
by Owen DeLong 07 Jan '07

07 Jan '07

Yesterday, around 10:00 AM Pacific Time 1/5/07, Kwajalein Atoll lost all connectivity to the mainland. We were told this was because MCI "lost 40 DS-3s due to someone shooting up a telephone pole in California" This affected Internet, Telephones (although inbound phone calls to the islands were possible), and television. Kwajalein access to the mainlaind is via Satellite to Washington connected to terrestrial link to Georgia. The outage lasted more than 12 hours. It seems odd to me in this day and age that: 1. There wasn't a redundant path for these circuits. 2. It took 12 hours to restore or reroute these circuits. Any details would be appreciated. Thanks, Owen P.S. For those wondering "Where?" There are excellent resources in Wikipedia, but, the short answer is Kwajalein Atoll is part of the Marshall Islands and is about the midpoint on a line from Honolulu to Sydney. About 9N by 165E.

1 0

FON Router allows anonymous web access (fwd)
by Gadi Evron 07 Jan '07

07 Jan '07

---------- Forwarded message ---------- Date: 6 Jan 2007 19:49:56 -0000 From: l.friedrichs(a)gbs.nitag.de To: bugtraq(a)securityfocus.com Subject: FON Router allows anonymous web access Description: "La Fonera" routers distributed by FON allow web access to unauthenticated users via DNS tunneling. Explanation: The router gives a client an DHCP answer but does not forward ip traffic until the client authenticates via the captive portal. The given DNS server address is the router itself so there is a DNS forwarder running on the router. Even an unauthorized client is allowed to surf certain sites as of obvious reasons (google, skype and the accesspoint's owner's site) but instead of filtering the dns requests for these few domains, it resolves all domains. This is where an DNS tunnel comes handy... Environment: Tested with router's standard config on 01/04/07. Runs smoothly with NSTX (http://nstx.dereference.de/nstx/ Version 1.1-beta6) and an ssh-session for connection testing without any authentication via the FON captive portal. Impact: Unauthorized ressource usage (internet bandwidth) Solution: New firmware from FON, workaround can be rate limiting DNS traffic on the real router (if possible...)

2 1

Re: Network end users to pull down 2 gigabytes a day, continuously?
by odlyzko＠dtc.umn.edu 07 Jan '07

07 Jan '07

Responding to postings by Colm MacCarthaigh and Marshall Eubanks: 1. There is practically no live television (at least in the United States). After the Janet Jackson episode, networks are inserting a 5-second (or perhaps it is a 10-second, I don't recall) pause, in order to stop anything untoward from showing up on the screen. Admittedly, there are live events (videoconferencing, or sports events that some people get a thrill out of watching in real-time), but that is a small fraction of total traffic. 2. Business models (such as advertising-financed TV) are certainly slow to change, as both businesses and consumers do not alter their habits on Internet time. But neither business models nor consumer habits need to change when you move from streaming to file downloads. As long as the transmission does not have to be absolutely real-time (as it does with videoconferencing), you gain a lot. Say you have a 3 Mbps download link, and the transmission speed of your video is 1 Mbps, start shooting it down at 3 Mbps (possibly allowing the customer to start watching it right away), and after 5 seconds you will have the first 15 seconds in the buffer on the customer's device. Even if that person has been watching from the beginning, you now have a 10-second grace period when you can tolerate a complete network outage without disturbing your customer. Just think of how much simpler that makes the network! And if you do worry about long videos not being viewed to the end, shoot them down to the customers in 10-second increments. This solves concerns about advertising and everything else. And of course you can encrypt the files, and do whatever else you want. Andrew P.S. I have been puzzled by the fixation on streaming for over a decade. A couple of years ago I wrote about it in "Telecom dogmas and spectrum allocations," http://www.dtc.umn.edu/~odlyzko/doc/telecom.dogmas.spectrum.pdf At my networking lectures, I often do a poll, asking how many people in the audience see any advantage (for consumer, or service providers, very vague requirement) in faster-than-real-time download of video. The response rate has ranged from 0 to 20%, with the 20% rate at two networking seminars at Stanford and CMU, full of networking graduate students, professors, VCs, and the like. There is a (small) fraction of people who see buffering and file downloads as the obvious thing, and others mostly have never even imagined such a thing. What's strangest is that the two camps seem to coexist without ever trying to debate the issue. -------------------------------------------------------------------------- On Sat, Jan 06, 2007 at 09:09:19AM -0600, Andrew Odlyzko wrote: > 2. The question I don't understand is, why stream? There are other good reasons, but fundamentally; because of live telivision. > In these days, when a terabyte disk for consumer PCs is about to be > introduced, why bother with streaming? It is so much simpler to > download (at faster than real-time rates, if possible), and play it > back. That might be worse for download operators, because people may download an hour of video, and only watch 5 minutes :/ -- Colm MacCárthaigh Public Key: colm+pgp(a)stdlib.net -------------------------------------------------------------------------- Dear Andrew; On Jan 6, 2007, at 10:09 AM, Andrew Odlyzko wrote: > > A remark and a question: > > <snip> > 2. The question I don't understand is, why stream? In these days, > when a > terabyte disk for consumer PCs is about to be introduced, why > bother with > streaming? It is so much simpler to download (at faster than real- > time rates, > if possible), and play it back. > I can answer that very simply for myself : We are now making a profit with streaming from advertising. To answer what I suspect is your deeper question : Broadcast is a push model, and will not go away. If fact, I think that the Internet will revitalize the "long tail" in video content, and broadcast will be a crucial part of that. It, after all, has been making more for over a century now. Download appears to be very similar, but is really not the same business model at all IMHO. Doesn't mean it's bad or worse, it may even be better, but it's different. And as long as you can make a profit from broadcasting / streaming... > Andrew > > Regards Marshall -------------------------------------------------------------------------- On Jan 6, 2007, at 10:19 AM, Colm MacCarthaigh wrote: > > On Sat, Jan 06, 2007 at 09:09:19AM -0600, Andrew Odlyzko wrote: >> 2. The question I don't understand is, why stream? > > There are other good reasons, but fundamentally; because of live > telivision. > >> In these days, when a terabyte disk for consumer PCs is about to be >> introduced, why bother with streaming? It is so much simpler to >> download (at faster than real-time rates, if possible), and play it >> back. > > That might be worse for download operators, because people may =20 > download > an hour of video, and only watch 5 minutes :/ > Our logs show that, for every 100 people who start to watch a stream, =20= only 2 or 5 % watch over 30 minutes in one sitting, even for VOD where they presumably have =20 some interest in the movie up front, and more more than 9% will watch all of VOD movie, even over multiple =20 viewings. This is also very consistent with time, but I don't have any pretty plots handy. (Our cumulative =20 audience in 2006 was 2.74 million people, I have lots of statistics.) So, from that standpoint, making a video file available for download =20 is wasting order of 90% of the bandwidth used to download it. Regards Marshall > --=20 > Colm MacC=E1rthaigh Public Key: colm=20 > +pgp(a)stdlib.net

2 1

Re: Network end users to pull down 2 gigabytes a day, continuously?
by Brandon Butterworth 06 Jan '07

06 Jan '07

> > If this application takes off, I have to presume that everyone's > > baseline network usage metrics can be tossed out the window... That'll happen anyway, what used to be considered high volume content is becoming the norm with lots of start ups and old school broadcasters getting involved. > P2P is a bandwidth sharing mechanism And a way to put off addressing the problem. We've been using it for VOD but we really do want people to do multicast for live TV (http://support.bbc.co.uk/multicast) It also exposes home PCs to more exploits by 3rd parties. Previously it was only our own servers we put at risk > As streaming video takes off, it will > use more or less the same amounts of bandwidth, P2P or no, as long as > the underlying transport is unicast ISPs might dislike peering with content providers, creating massive uncontrolled P2P gets round that at their expense. We've avoided trying to make unicast too big but we have content that many millions use (concurrently) that won't stay away >> > It downloads 320 megabytes (MB) per hour from users' computers, >> > meaning that users could reach their monthly download limits in >> > hours and that it could be unusable for bandwidth-capped users. Our multicast streams are 360K for legacy users but 1.2M, 3-5M (full DTV quality same as off air) and 18M (HD, the encoders aren't mature yet, may be down to 8M this summer. Less is possible but 2Mbit isn't HD in any codec yet if you want it to be good) At the higher rates P2P over current UK ADSL (gain < 1) is going to need a lot of benevolent users willing to donate b/w when they're not active. brandon

1 0

cold-potato at L3/Chi into UU?
by Bradley Urberg Carlson 06 Jan '07

06 Jan '07

In Chicago, L3 appears to be routing some traffic to UU directly, but for some other UU destinations, L3 is handing-off to UU via New York: trace from L3/Chi to 209.98.0.2 (AS8015 via 701) is carried to UU via New York (edited): > 1 ae-1-53.bbr1.Chicago1.Level3.net (4.68.101.65) 8 msec > 2 as-4-0.bbr2.NewYork1.Level3.net (64.159.0.238) 20 msec > 4 mci-level3-te-newyork1.Level3.net (4.68.110.234) 20 msec > 8 visi-gw.customer.ALTER.NET (157.130.98.2) [AS701 {ALTERNET}] 32 > msec Oddly, a L3/Chi trace to the last hop above within AS701 (157.130.98.2) enters UU immediately in Chicago, taking the shorter path to Minneapolis: > 1 ge-7-0-0-56.edge1.Chicago1.Level3.net (4.68.101.173) 0 msec > 2 0.so-0-3-0.BR2.CHI13.ALTER.NET (204.255.168.117) [AS701 {ALTERNET}] > 0 msec > 6 visi-gw.customer.ALTER.NET (157.130.98.2) [AS701 {ALTERNET}] 8 msec UU support has verified that AS8015 prefixes are advertised at UU's L3/Chi peering point; and L3 support reports this is "routing as designed". I would guess that UU requires its peers to use hot-potato routing, so I'm at a loss as to what kind of traffic engineering is being used. -- Bradley Urberg-Carlson VISI Network Operations; pgp keys avaialble at http://www.keyserver.net/

1 0