Hello, for approx. last 14 days we're seeing problems with video playing from youtube (page loads without problems, but player shows error), and also other applications like maps are having problems. As these problems were only for some of prefixes announced out of AS 8251, we recognised that as problem with Google's CDN and reported it to Google. As workaround, filtering affected prefixes from peering in Prague partially helped. We already tried communicate problem with Google from the beginning (in our network, around 50000 end users are directly afected), and they're claiming, that problem is related only to our network and there's no global issue. But similar issues we're seeing from some other networks, which are peering with Google and have nothing common with our AS8251. We sent emails to Google NOC/NST, also tried to phone them about the problems, but according to end-user claims, problem persist. Problem is isolated only to peak hours, when something seems to be saturated. If I recall past optional IPv6 from Google, they said: "It is very important to us that our users always receive the best possible experience". But majority of end users still uses IPv4 and we're seeing problems here - and response is minimal. At least information about cause of the problem and expected time for problem resolution. Is anyone else seeing similar problems with Google/Youtube? With regards, Daniel
On (2012-11-18 23:47 +0100), Daniel Suchy wrote:
Is anyone else seeing similar problems with Google/Youtube?
My advice is, host the content locally. Certain Finnish domestic SPs had issues with youtube during peak hours for years, when content came via Stockholm, if content came from mainland europe or locally you were set. Perhaps Google was (maybe still is) regularly congested in Stockholm, and there might not be much incentive for google to add capacity everywhere sufficiently, as they can just pressure to people to host them for free. I'm bit curious about market position youtube has. GOOG claims youtube is making profit, but I think this is because network is considered other BUs cost and youtube rides on it for free (remember pre-youtube, how GOOG micro-optimized google front-page to save on network cost, post-youtube they rightly stopped caring and added predictive input etc.) I can't see how anyone could compete against youtube, I don't believe the service is anywhere near profitable (it's maybe 10% of Internet, and I can't see revenue being 10% of Internet), if it would have to pay for the network itself. Consequently you probably can't compete with them, as you need to cover the costs from the profits. It is just so ubiquitous service, that if it does not work your eyeballs will switch to network where it does, so you will give google free capacity, which you wouldn't probably do for others web streaming shops. -- ++ytti
On Nov 19, 2012, at 03:05 , Saku Ytti <saku@ytti.fi> wrote:
On (2012-11-18 23:47 +0100), Daniel Suchy wrote:
Is anyone else seeing similar problems with Google/Youtube?
My advice is, host the content locally.
Sound advice, IMHO.
I'm bit curious about market position youtube has. GOOG claims youtube is making profit, but I think this is because network is considered other BUs cost and youtube rides on it for free (remember pre-youtube, how GOOG micro-optimized google front-page to save on network cost, post-youtube they rightly stopped caring and added predictive input etc.)
I do not work for Google, nor have I asked anyone in Google how they do their accounting. However, I would be rather surprised to find the vast majority of their capacity is charged to the BU using a tiny fraction of that capacity, while the BU using the lion's share gets a "free ride".
I can't see how anyone could compete against youtube, I don't believe the service is anywhere near profitable (it's maybe 10% of Internet, and I can't see revenue being 10% of Internet), if it would have to pay for the network itself. Consequently you probably can't compete with them, as you need to cover the costs from the profits. It is just so ubiquitous service, that if it does not work your eyeballs will switch to network where it does, so you will give google free capacity, which you wouldn't probably do for others web streaming shops.
First, I believe YouTube is > 10% of the Internet. Second, I see no reason why that requires anything close - not even within a couple orders of magnitude - of 10% of the Internet's revenue to be profitable. Why would you assume such a thing? -- TTFN, patrick
On (2012-11-19 08:27 -0500), Patrick W. Gilmore wrote:
Second, I see no reason why that requires anything close - not even within a couple orders of magnitude - of 10% of the Internet's revenue to be profitable. Why would you assume such a thing?
Agreed, 10% of Internet's revenue would be exaggeration. What I'm trying to say, I can't see youtube generating anywhere nearly enough revenue who shift 10% (or more) of Internet. And to explain this conundrum to myself, I've speculated accounting magic (which I'd frown upon) and leveraging market position to get free capacity (which is ok, I'd do the same, had I the leverage) -- ++ytti
In a message written on Mon, Nov 19, 2012 at 03:59:22PM +0200, Saku Ytti wrote:
What I'm trying to say, I can't see youtube generating anywhere nearly enough revenue who shift 10% (or more) of Internet. And to explain this conundrum to myself, I've speculated accounting magic (which I'd frown upon) and leveraging market position to get free capacity (which is ok, I'd do the same, had I the leverage)
I suspect you're thinking about revenue in terms of say, the advertisements they run with the videos. I beleive you're right, that would never pay the bills. Consider a different model. Google checks out your gmail account, and discovers you really like Red Bull and from your YouTube profile knows you watch a lot of Ke$ha videos. It also discovers there are a lot more folks with the same profile. They can now sell that data to a marketing firm, that there is a strong link between energy drinks and Ke$ha videos. GOOG-411 - building a corpus of voice data for Android's voice recognition. ReCaptcha - improving visual recognition for their book scanning process. Most of the "free" services are simply the cheapest way to get the data needed for some other service that can make much more money. It may seem weird to write off all the costs of YouTube as data aquisition costs, but there's far more money to be made selling marketing data than ads against streaming videos... -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
From the latest csco prime presentation it appears it offers similar functionality in one of the modules that one can buy to it so that providers can have a sneak peak on these type of data in order to sell them to third parties Though I wouldn't even know whom to sell such information Nor have I been hit by a targeted advertisement, yet
adam -----Original Message----- From: Leo Bicknell [mailto:bicknell@ufp.org] Sent: Monday, November 19, 2012 3:30 PM To: nanog@nanog.org Subject: Re: Google/Youtube problems In a message written on Mon, Nov 19, 2012 at 03:59:22PM +0200, Saku Ytti wrote:
What I'm trying to say, I can't see youtube generating anywhere nearly enough revenue who shift 10% (or more) of Internet. And to explain this conundrum to myself, I've speculated accounting magic (which I'd frown upon) and leveraging market position to get free capacity (which is ok, I'd do the same, had I the leverage)
I suspect you're thinking about revenue in terms of say, the advertisements they run with the videos. I beleive you're right, that would never pay the bills. Consider a different model. Google checks out your gmail account, and discovers you really like Red Bull and from your YouTube profile knows you watch a lot of Ke$ha videos. It also discovers there are a lot more folks with the same profile. They can now sell that data to a marketing firm, that there is a strong link between energy drinks and Ke$ha videos. GOOG-411 - building a corpus of voice data for Android's voice recognition. ReCaptcha - improving visual recognition for their book scanning process. Most of the "free" services are simply the cheapest way to get the data needed for some other service that can make much more money. It may seem weird to write off all the costs of YouTube as data aquisition costs, but there's far more money to be made selling marketing data than ads against streaming videos... -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
From the latest csco prime presentation it appears it offers similar functionality in one of the modules that one can buy to it so that providers can have a sneak peak on these type of data in order to sell them to third
For some providers this might be an interesting revenue stream in these days where we need to build ever faster backbones to carry more and more video traffic for users that want to pay less and less for high-speed internet connectivity adam -----Original Message----- From: Adam Vitkovsky [mailto:adam.vitkovsky@swan.sk] Sent: Monday, November 19, 2012 4:17 PM To: 'Leo Bicknell'; 'nanog@nanog.org' Subject: RE: Google/Youtube problems parties Though I wouldn't even know whom to sell such information Nor have I been hit by a targeted advertisement, yet adam -----Original Message----- From: Leo Bicknell [mailto:bicknell@ufp.org] Sent: Monday, November 19, 2012 3:30 PM To: nanog@nanog.org Subject: Re: Google/Youtube problems In a message written on Mon, Nov 19, 2012 at 03:59:22PM +0200, Saku Ytti wrote:
What I'm trying to say, I can't see youtube generating anywhere nearly enough revenue who shift 10% (or more) of Internet. And to explain this conundrum to myself, I've speculated accounting magic (which I'd frown upon) and leveraging market position to get free capacity (which is ok, I'd do the same, had I the leverage)
I suspect you're thinking about revenue in terms of say, the advertisements they run with the videos. I beleive you're right, that would never pay the bills. Consider a different model. Google checks out your gmail account, and discovers you really like Red Bull and from your YouTube profile knows you watch a lot of Ke$ha videos. It also discovers there are a lot more folks with the same profile. They can now sell that data to a marketing firm, that there is a strong link between energy drinks and Ke$ha videos. GOOG-411 - building a corpus of voice data for Android's voice recognition. ReCaptcha - improving visual recognition for their book scanning process. Most of the "free" services are simply the cheapest way to get the data needed for some other service that can make much more money. It may seem weird to write off all the costs of YouTube as data aquisition costs, but there's far more money to be made selling marketing data than ads against streaming videos... -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
On (2012-11-19 06:30 -0800), Leo Bicknell wrote:
Consider a different model. Google checks out your gmail account, and discovers you really like Red Bull and from your YouTube profile knows you watch a lot of Ke$ha videos. It also discovers there are a lot more
Sure. I have no doubt the main reasons to keep youtube are. a) data mining b) contingency B) is essentially having the most popular platform, in case if video platform becomes viable marketing platform on itself. Data mining aspect might make it less dubious to sink network cost to different BU than to the BU which actually uses the network, as that network is also benefitting from the data. -- ++ytti
On Mon, Nov 19, 2012 at 6:30 AM, Leo Bicknell <bicknell@ufp.org> wrote:
In a message written on Mon, Nov 19, 2012 at 03:59:22PM +0200, Saku Ytti wrote:
What I'm trying to say, I can't see youtube generating anywhere nearly enough revenue who shift 10% (or more) of Internet. And to explain this conundrum to myself, I've speculated accounting magic (which I'd frown upon) and leveraging market position to get free capacity (which is ok, I'd do the same, had I the leverage)
I suspect you're thinking about revenue in terms of say, the advertisements they run with the videos. I beleive you're right, that would never pay the bills.
Consider a different model. Google checks out your gmail account, and discovers you really like Red Bull and from your YouTube profile knows you watch a lot of Ke$ha videos. It also discovers there are a lot more folks with the same profile. They can now sell that data to a marketing firm, that there is a strong link between energy drinks and Ke$ha videos.
Actually GOOG doesn't allow this as policy. Different BUs are rather quite restricted on how they can obtain other BUs data. In general "if you can't do it as XYZ corp, you can't do it from inside of GOOG either" -- there's a sort of privacy/policy watchdog group inside of the puzzle palace with at least a few people who are *very* concerned with privacy and data protection. I know this just because I've met a handful of them over the years. The ones in the group charged with making sure your data isn't opened up to everyone and their brother, even inside of google, to this sort of thing are pretty fanatical too. Ads can't use any data in any other way than anyone else could from GMail. Same goes with search. They can (and clearly do) share technology, software, infrastructure, and methodologies, but, the actual data is a pretty touchy subject between BUs due to their own policy. Even if they disband the group, everyone I've ever met with any responsibility towards user data shared the attitude that doing something many of us would consider "icky" would be somethign they'd block against internally. (such as just opening up the gmail to any advertiser that came along, aggregating data between BUs to sell individual preferences, etc) Will this be the case forever? Dunno. The ethos/culture is what keeps all this in check right now and culture is known to change. All that said, they're quite profitable now, and so I don't know that there's a pressure from profit motive to improve that revenue stream by doing dirty pool. Especially if the world governments decide they're playing dirty pool and go looking.
GOOG-411 - building a corpus of voice data for Android's voice recognition.
ReCaptcha - improving visual recognition for their book scanning process.
Most of the "free" services are simply the cheapest way to get the data needed for some other service that can make much more money. It may seem weird to write off all the costs of YouTube as data aquisition costs, but there's far more money to be made selling marketing data than ads against streaming videos...
What I'm trying to say, I can't see youtube generating anywhere nearly enough revenue who shift 10% (or more) of Internet. And to explain this conundrum to myself, I've speculated accounting magic (which I'd frown upon) and leveraging market position to get free capacity (which is ok, I'd do the same, had I the leverage) Or there's a simpler explanation. Which is that it makes money either
On 11/19/12 5:59 AM, Saku Ytti wrote: directly or as part of a salubrious interaction with other google properties. They had about 2.5Billion left over for their trouble in the quarter ending 9/30 which isn't too shabby on a gross of 14 billion.
WIth my limited understanding of such topics I've long been confused by something I read a couple of years back - in an Arbor report perhaps - to the effect that by being the originator of so much traffic, and as they built out their own network, Google were making money on transit. Can anyone elaborate or refute? On Mon, Nov 19, 2012 at 11:55 AM, joel jaeggli <joelja@bogus.com> wrote:
On 11/19/12 5:59 AM, Saku Ytti wrote:
What I'm trying to say, I can't see youtube generating anywhere nearly enough revenue who shift 10% (or more) of Internet. And to explain this conundrum to myself, I've speculated accounting magic (which I'd frown upon) and leveraging market position to get free capacity (which is ok, I'd do the same, had I the leverage)
Or there's a simpler explanation. Which is that it makes money either directly or as part of a salubrious interaction with other google properties.
They had about 2.5Billion left over for their trouble in the quarter ending 9/30 which isn't too shabby on a gross of 14 billion.
-- --------------------------------------------------------------- Joly MacFie 218 565 9365 Skype:punkcast WWWhatsup NYC - http://wwwhatsup.com http://pinstand.com - http://punkcast.com VP (Admin) - ISOC-NY - http://isoc-ny.org -------------------------------------------------------------- -
participants (8)
-
Adam Vitkovsky -
Daniel Suchy -
joel jaeggli -
Joly MacFie -
Leo Bicknell -
Michael Loftis -
Patrick W. Gilmore -
Saku Ytti