Re: Internet Video: The Next Wave of Massive Disruption to the US Peering Ecosystem (v1.2)
Then that wouldn't be enough since the other Tier 1's would need to upgrade their peering infrastructure to handle the larger peering links (n*10G), having to argue to their CFO that they need to do it so that their competitors can support the massive BW customers.
Someone will take the business so that traffic is coming regardless, they can either be that peer or be the source with the cash. If they can't do either then they're not in business, I hope they wouldn't ignore it congesting their existing peers (I know...)
Then even if the peers all upgraded the peering gear at the same time, the backbones would have to be upgraded as well to get that traffic out of the IXes and out to the eyeball networks.
The Internet doesn't scale, turn it off brandon
Why are folks turning away 10G orders? I forgot to mention a couple other issues that folks brought up: 4) the 100G equipment won't be standardized for a few years yet, so folks will continue to trunk which presents its own challenges over time. 5) the last mile infrastructure may not be able to/willing to accept the competing video traffic . There was some disagreement among the group I discussed this point with however. A few of the cable operations guys said there is BW and the biz guys don't want to 'give it away' when there is a potential to charge or block (or rather mitigate the traffic as they do now). My favorite data point was from Geoff Huston who said that the cable companies are clinging to their 1998 business model as if it were relevent in the world where peer-2-peer for distribution of large objects has already won. He believes that the sophisticated peer-2-peer is encrypting and running over ports noone will shut off, the secure shell ports that are required for VPNs. So give up, be the best dumb pipes you can be I guess. Bill On 1/10/07, Brandon Butterworth <brandon@rd.bbc.co.uk> wrote:
Then that wouldn't be enough since the other Tier 1's would need to upgrade their peering infrastructure to handle the larger peering links (n*10G), having to argue to their CFO that they need to do it so that their competitors can support the massive BW customers.
Someone will take the business so that traffic is coming regardless, they can either be that peer or be the source with the cash. If they can't do either then they're not in business, I hope they wouldn't ignore it congesting their existing peers (I know...)
Then even if the peers all upgraded the peering gear at the same time, the backbones would have to be upgraded as well to get that traffic out of the IXes and out to the eyeball networks.
The Internet doesn't scale, turn it off
brandon
-- //------------------------------------------------ // William B. Norton <wbn@equinix.com> // Co-Founder and Chief Technical Liaison, Equinix // GSM Mobile: 650-315-8635 // Skype, Y!IM: williambnorton
On Wed, Jan 10, 2007 at 09:33:41AM -0800, William B. Norton wrote:
Why are folks turning away 10G orders?
I forgot to mention a couple other issues that folks brought up: 4) the 100G equipment won't be standardized for a few years yet, so folks will continue to trunk which presents its own challenges over time.
well, there's a few important issues here: currently the "state-of-the-art" is to bundle/balance n*10G. While it's possible to do 40G/n*40G in some places, this is not entirely universal. Given the above constraint, in delivering 10G/n*10G to "customers" requires some investment in your infrastructure to be able to carry that traffic on your network. The cost difference between sonet/sdh ports compared to 10GE is significant here and continues to be a driving force, imho. Typically in the past, the "tier-1" isps have had a larger circuit than the customer edge. eg: I have my OC3, but my provider network is OC12/OC48. Now with everyone having 10G since it is "cheap enough", this drives multihoming, routing table size, fib/tcam and other memory consumption, including the corresponding CPU "cost".
5) the last mile infrastructure may not be able to/willing to accept the competing video traffic . There was some disagreement among the group I discussed this point with however. A few of the cable operations guys said there is BW and the biz guys don't want to 'give it away' when there is a potential to charge or block (or rather mitigate the traffic as they do now).
I suspect this in varies depending on how it's done. Most of the "cable" folks are dealing with short enough distances as long as the fiber quality is high enough, they could do 10/40G to the neighborhood. The issue becomes the coax side as well as the bandwidth consumption of those "analog" users. Folks don't upgrade their TV or set-top-box as quickly as they upgrade their computers. There's also a significant cost associated with any change and dealing with those grumpy users if they don't want a STB either.
My favorite data point was from Geoff Huston who said that the cable companies are clinging to their 1998 business model as if it were relevent in the world where peer-2-peer for distribution of large objects has already won. He believes that the sophisticated peer-2-peer is encrypting and running over ports noone will shut off, the secure shell ports that are required for VPNs.
So give up, be the best dumb pipes you can be I guess.
I suspect there's going to be continued seperation "at the top" as folks see it. Those that can take on these new 10G and n*10G customers and deliver the traffic and those who run into peering and their own network issues in being able to deliver the bits. While 100G will ease some of this, there's still those pesky colo/power issues to deal with. unless you own your own facility, and even if you do, you may have months if not years of slowly evolving upgrades to face. Perhaps there will be some technology that will help us through this, but at the same time, perhaps not, and we'll be getting out the huge rolls of duct tape. It may not be politics that drives partial-transit/paid peering deals, it may just be plain technology. - Jared
On 1/10/07, Brandon Butterworth <brandon@rd.bbc.co.uk> wrote:
Then that wouldn't be enough since the other Tier 1's would need to upgrade their peering infrastructure to handle the larger peering links (n*10G), having to argue to their CFO that they need to do it so that their competitors can support the massive BW customers.
Someone will take the business so that traffic is coming regardless, they can either be that peer or be the source with the cash. If they can't do either then they're not in business, I hope they wouldn't ignore it congesting their existing peers (I know...)
Then even if the peers all upgraded the peering gear at the same time, the backbones would have to be upgraded as well to get that traffic out of the IXes and out to the eyeball networks.
The Internet doesn't scale, turn it off
brandon
-- //------------------------------------------------ // William B. Norton <wbn@equinix.com> // Co-Founder and Chief Technical Liaison, Equinix // GSM Mobile: 650-315-8635 // Skype, Y!IM: williambnorton
-- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.
On Wed, Jan 10, 2007 at 09:33:41AM -0800, William B. Norton wrote:
Why are folks turning away 10G orders?
Quite simple... We've found a fair number of networks with no 10GE equipment and no budget to add it. Doubtless, some of these don't have OC192 capacity either. Others have 10G in the offing but are still putting it through "acceptance" testing and won't sell it for several more months. Others will happily sell you a 10GE circuit but then limit you to some fraction of that circuit because of internal limitations within their nodes. (I've seen this on more than one network.) And in any case, some of these don't have the egress capacity either from the local node to their backbone or to their peers/customers to be able to swallow that kind of traffic anyway. Truth be told, there really are not that many networks out there at present that are capable of accepting a 10G customer who actually intends to USE 10G. And believe it or not, there are still those out there that believe that customers aren't going to be able to pass a full 10G and therefore see no need to offer it at the edge. Currently, for all but the most intensive users, NxGE or NxOC48 still seems to be preferred termination. (Often this is also partly a factor of minimum commits and varying methods of billing.) This is changing but it's happening more slowly than I would like to see. My $0.37 (inflation's a pain) -Wayne --- Wayne Bouchard web@typo.org Network Dude http://www.typo.org/~web/
On Wed, 10 Jan 2007, William B. Norton wrote:
Why are folks turning away 10G orders?
In Hollywood, San Francisco and a few other cities with large concentration of movie/entertainment industries 10G network connections have been sold for at least a year, not necessarily connected to the "Internet." Don't we go through this every other network generation? I seem to recall the NSFNET only being able to handle 22Mbps on its T3 (45Mbps) connections for a while. A few years later, another backbone was out of capacity and had to stop selling new connections for several months while it caught up. Later we had the OC12, OC48 bottleneck on switches being used at some exchange points and getting GigE connections was a problem. And so on and so on. We'll have wailing and gnashing of teeth for many months but somehow things will balance out again as technology, equipment and revenue catch up with each other. And then it will start over again with someone wanting 100GE connections, and then 1000GE connections, etc, etc.
On Jan 10, 2007, at 12:33 PM, William B. Norton wrote:
Why are folks turning away 10G orders?
Some of this depends on how much you are willing to pay. The issue is as much 10G orders at today's transit prices as it is the capacity. We're used to paying less per unit for greater capacity, but when we're asking networks to sell capacity in chunks as large as the ones they use to build their backbones, that may simply not work. One other issue is that willingness to sell 10G is one vital competitive distinguisher in an otherwise largely commodity transit market. There have been rumors that older legacy carriers wish to punish more agile competitors for daring to "steal" 10G customers away from them, in spite of the fact that those older carriers have lots of trouble delivering 10G. - Dan
One other issue is that willingness to sell 10G is one vital competitive distinguisher in an otherwise largely commodity transit market. There have been rumors that older legacy carriers wish to punish more agile competitors for daring to "steal" 10G customers away from them, in spite of the fact that those older carriers have lots of trouble delivering 10G.
There isn't anything new to that either. I can think of (similar to Sean's story) a time when a backbone had all NxDS3 links in their network and used to be very upset at the idea of selling OC3s instead of NxDS3 ("If its good enough for *our* backbone..."). Then this network went ATM on Fore systems boxes with handoffs to Cisco routers to leapfrog their competitors... which um, didn't work... Then they reluctantly went POS and "better" systems since... They are still around today after about 6 name changes. I have no present-day-knowledge of their network or, really, their performance, so I don't want to mention any names. Deepak
participants (7)
-
Brandon Butterworth -
Daniel Golding -
Deepak Jain -
Jared Mauch -
Sean Donelan -
Wayne E. Bouchard -
William B. Norton