may I make just a passing observation? From a technology point of view the best effort internet certainly "works." Not surprisingly the comments here are primarily debating the finer points of the technology. The point I am making in my report is NOT that the best effort network has technology problems but rather that it has ECONOMIC PROBLEMS. That it might support 2 or 3 players not 2 or 3 HUNDRED. That until companies begin to go chapter seven and vanish, the best effort net will be a black hole that burns up capital because, for many players, the OPERATIONAL expense is more than they get for bandwidth never mind cap-ex. best effort won't go away. many best effort players will. for the time being, best effort bandwidth prices as an absolute commodity cannot sustain networks over the long haul. A network that can deliver QoS the report hypothesizes may be able to attract enough revenue to become profitable. How to to this my group is still discussing. We don't pretend that QoS is easy or any kind of mature collection of technologies, but increasingly it looks as though the industry, if it is ever going to be self sustaining, really needs to look at QoS services and solutions.
On Sat, 29 May 2004, Edward B. Dreger wrote:
Nitpicking: Latency isn't that important with unidirectional communication. However, VoIP users seem reasonably happy with current latency and jitter -- and the Internet still is _largely_ xxTP, anyway... particularly if one ignores peer-to-peer file- swapping programs.
Latency is fine for VOIP as long as you dont interact with the PSTN network, if you want to interact with PSTN then you need echo cancellation if you have high latency on the IP part.
Most VOIP applications can handle 40ms jitter, so that's normally no problem unless your local access is full. Packet loss is normally no problem for VOIP if you use a proper (non-telco developed) codec.
VOIP is actually better off with high packet loss and low jitter than the other way around (throwing off the old truth that core equipment should have lots of buffers).
-- Mikael Abrahamsson email: swmike@swm.pp.se
-- ============================================================= The COOK Report on Internet Protocol, 431 Greenway Ave, Ewing, NJ 08618 USA 609 882-2572 (PSTN) 703 738-6031 (Vonage) Subscription info & prices at http://cookreport.com/subscriptions.shtml Report on economic black hole of best effort networks at: http://cookreport.com/13.04.shtml E-mail cook@cookreport.com =============================================================
On Sat, 29 May 2004, Gordon Cook wrote:
discussing. We don't pretend that QoS is easy or any kind of mature collection of technologies, but increasingly it looks as though the
Tier 1 operators do not do "best effort" really, at least not in their cores (and they have the SLAs to back it up). They buy hugely expensive top notch gear (Cisco 12000 (and now CRS:s) and Junipers) to get the big packet buffers, the fast reroutes and the full routing table lookups for each packet to avoid the pitfalls of flow forwarding the cheaper platforms have. With the advent of 10GE WAN PHY (Force10, Foundry, Riverstone, Extreme Networks, Cisco 7600) and full L3 lookup for each packet on their newer platforms, we'll see very much cheaper L2/L3 equipment being able to take advantage of existing OC192 infrastructure and that's where I think you'll start to see the real "best effort" networks operating at. At least the L2/L3 equipment will be much cheaper for the operators choosing this equipment, at approx 1/5 the initial investment of similar capacity 12400 and Juniper equipment. Now, how will this translate in cost compared to DWDM equipment and OPEX part of the whole equation? Well, the bubble effect is still doing fine, so I think we wont see any stability for yet another 2-3 years, I'll definately give you that in your analysis. As long as there is equipment and unused installed capacity left from 2000-2001 out there, the price equation will be skewed compared to what it actually costs to replentish the capacity when you've sold it. -- Mikael Abrahamsson email: swmike@swm.pp.se
Tier 1 operators do not do "best effort" really, at least not in their cores (and they have the SLAs to back it up). They buy hugely expensive top notch gear (Cisco 12000 (and now CRS:s) and Junipers) to get the big packet buffers, the fast reroutes and the full routing table lookups for each packet to avoid the pitfalls of flow forwarding the cheaper platforms have.
When 12016s are on ebay for $12,000, even a low budget "tier 3" can afford proper routing gear... It's not as if the Internet is still powered by 7507s! (Well, a large part still is. :-)
Now, how will this translate in cost compared to DWDM equipment and OPEX part of the whole equation?
I am starting to see some interesting long-distance 2.5Gbps CWDM gear offered by European manufacturers, with 70km and 100km distance ratings. This stuff sells for a fraction of the price of equivalent Nortel/Ciena/Cisco ONS gear. Lots of optics companies are making 70km rated SFPs in 8 or 16 wavelengths now. So far it only runs at OC-48 speeds, but 10Gbps will be coming soon.
On Sat, 29 May 2004, Eric Kuhnke wrote:
When 12016s are on ebay for $12,000, even a low budget "tier 3" can afford proper routing gear... It's not as if the Internet is still powered by 7507s! (Well, a large part still is. :-)
12016 will only do OC48 speeds and the OC48 cards that used to be had at $3500-$5000 on eBay now is $10k+ and quickly drying up. Then there is the whole legal and support issue, if you need Cisco support and want to be legal, you can effectively double the above prices.
I am starting to see some interesting long-distance 2.5Gbps CWDM gear offered by European manufacturers, with 70km and 100km distance ratings. This stuff sells for a fraction of the price of equivalent Nortel/Ciena/Cisco ONS gear. Lots of optics companies are making 70km rated SFPs in 8 or 16 wavelengths now. So far it only runs at OC-48 speeds, but 10Gbps will be coming soon.
At least in Sweden there is still plenty of unlit dark fiber than can be had cheaply. It's not the fiber that'll cost you, it's the rent of floorspace and power in the amplifier stations, plus the manpower OPEX of keeping it running. Still, at prices of $25-40 per meg per month, it's hard to produce new bandwidth even by buying gear on ebay and doing everything the cheap way. Let's say you average 1 gig of paying traffic per month on your OC48 links, that'll only give you let's say $40k revenue per month. Not much to build an operation on. -- Mikael Abrahamsson email: swmike@swm.pp.se
Folks, This is a great discussion. I'm interested in understanding these types of limitations in the context of HFC cable networks. In my opinion, HDTV channel bandwidth (30mhz?) , increased demand for voip, and growing demand for IP connectivity is going to stress the cable network model as well, forcing cable operators to convert everything to IP before going out across the wire. Any input is appreciated. Regards, Christopher
Christopher J. Wolff wrote:
Folks,
This is a great discussion. I'm interested in understanding these types of limitations in the context of HFC cable networks. In my opinion, HDTV channel bandwidth (30mhz?) , increased demand for voip, and growing demand for IP connectivity is going to stress the cable network model as well, forcing cable operators to convert everything to IP before going out across the wire. Any input is appreciated.
The bandwidth you quote refers to the unencoded signal bandwidth. The actual transmission has the same channel spacing than traditional television, be that NTSC or PAL or SECAM. Terrestial applications typically carry 20-25 megabits and cable & satellite applications typically 40-50 megabits per channel/transponder. HDTV channel usually consumes 19.8Mbps for video plus some extra for audio + multiplexing information. So a channel that used to carry one NTSC analog channel over the air can be used to carry one HDTV channel or in the case of cable/satellite networks, two. I´m all for encapsulating all transmission to IP packets, because it would make a lot of things easier, but I would suspect some places need to freeze first :-) Pete
Thus spake "Christopher J. Wolff" <chris@bblabs.com>
This is a great discussion. I'm interested in understanding these types of limitations in the context of HFC cable networks. In my opinion, HDTV channel bandwidth (30mhz?) ,
Broadcast ATSC (aka HDTV) uses the same bandwidth as broadcast NTSC: 6MHz. The problem is that many (if not all) cable operators use high compression ratios to squeeze multiple channels into a single 6MHz slot. While this doesn't degrade quality noticeably with SDTV, it ruins HDTV, and many cable operators actually downconvert HDTV programming to SDTV to maintain constant quality (lowest common denominator).
increased demand for voip, and growing demand for IP connectivity is going to stress the cable network model as well, forcing cable operators to convert everything to IP before going out across the wire.
Well, there's technical challenges with DOCSIS, but the bigger question is whether broadcast content delivery is going to remain viable... Consumers are getting used to time-shifting thanks to TiVo et al, and IP PPV is a reality in some places. Verizon is already talking about delivering "cable" TV over IP once their FTTH is rolled out, and you can expect other telcos to pick up the idea as broadband penetration increases. Broadband VoIP is already here; while Vonage leads the way, I doubt the telcos are going to cede marketshare now that VoIP appears profitable (and inevitable). There's an old saying: "I'd rather get my cable service from the phone company than my phone service from the cable company." We're rapidly approaching a day consumers can get both from their broadband company. We may even see competition from third parties iff inter-provider reliability and capacity is improved. S PS- This dovetails nicely with my recent post on how unprofitable basic transport can be sustained through subsidies from add-on services. Thanks Christopher! Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
That's how time-warner is doing it here in Maine. However, they've assigned voip to its own channel while digital channels occupy another and hdtv signals are compressed and on another, etc. Its a 1Ghz system. Everything on the system is IP including the control boxes. They also have certain video and PPV on demand as well. Very interesting system that serves as a model for the rest of the country. Whatever they do they roll it out here first and this system totally rocks. I regularly get downloads at 3Mbps. Curtis -- Curtis Maurand mailto:curtis@maurand.com http://www.maurand.com On Sun, 30 May 2004, Christopher J. Wolff wrote:
Folks,
This is a great discussion. I'm interested in understanding these types of limitations in the context of HFC cable networks. In my opinion, HDTV channel bandwidth (30mhz?) , increased demand for voip, and growing demand for IP connectivity is going to stress the cable network model as well, forcing cable operators to convert everything to IP before going out across the wire. Any input is appreciated.
Regards, Christopher
Mikael Abrahamsson writes:
Tier 1 operators do not do "best effort" really, at least not in their cores (and they have the SLAs to back it up). They buy hugely expensive top notch gear (Cisco 12000 (and now CRS:s) and Junipers) to get the big packet buffers, the fast reroutes and the full routing table lookups for each packet to avoid the pitfalls of flow forwarding the cheaper platforms have.
With the advent of 10GE WAN PHY (Force10, Foundry, Riverstone, Extreme Networks, Cisco 7600)
I don't think there's 10 GE WAN PHY for the Cisco 7600 yet. It has very cost-effective 10 GE *LAN* PHY (10.0 Gb/s, not SONET-compatible) interfaces though, which I find even more interesting (see below).
and full L3 lookup for each packet on their newer platforms, we'll see very much cheaper L2/L3 equipment being able to take advantage of existing OC192 infrastructure and that's where I think you'll start to see the real "best effort" networks operating at. At least the L2/L3 equipment will be much cheaper for the operators choosing this equipment, at approx 1/5 the initial investment of similar capacity 12400 and Juniper equipment.
We find that the L1 equipment is getting much cheaper too, especially in the 10 GE LAN PHY space. Think DWDM XENPAKs (or XFPs), which go 70-100 kms and which can be multiplexed and amplified with pretty affordable optical equipment. If you're not interested in carrier-class boxes, "traditional" WDM equipment can sometimes be replaced with active parts that mostly look like GBICs, and passive parts that look like funny cables...
Now, how will this translate in cost compared to DWDM equipment and OPEX part of the whole equation? [...] -- Simon.
On Mon, 31 May 2004, Simon Leinen wrote:
I don't think there's 10 GE WAN PHY for the Cisco 7600 yet. It has very cost-effective 10 GE *LAN* PHY (10.0 Gb/s, not SONET-compatible) interfaces though, which I find even more interesting (see below).
Cisco won't release a WAN PHY for a long time and it'll likely be quite expensive since it competes with their other (even more expensive) OC192 stuff. (Yeah, there is most likely a technical reason also, they want to do extensive testing). Also, Cisco is as far as I know now the only player in the market which code SFPs and Xenpaks to avoid impacting their very nice business case of 500+ % markup on optics. I have beta units of WAN PHY Xenpaks directly from the manufacturer, they work nicely in Extreme equipment, unfortunately when putting them into a 7600 it wont even activate them and Cisco doesn't seem very keen on supplying an IOS that doesn't have this limitation so I can test it at all (they've had 8 working days now).
We find that the L1 equipment is getting much cheaper too, especially in the 10 GE LAN PHY space. Think DWDM XENPAKs (or XFPs), which go 70-100 kms and which can be multiplexed and amplified with pretty affordable optical equipment. If you're not interested in carrier-class boxes, "traditional" WDM equipment can sometimes be replaced with active parts that mostly look like GBICs, and passive parts that look like funny cables...
I know of DWDM GBICs, they've been around for quite some time. Just a matter of time until we get DWDM Xenpaks as well. I've also tried the CWDM OADMs which come on some patch cables. Nice if you want to do it in a small scale point to point. The thing speaking against cheaper DWDM stuff is that the transmission people aren't very happy about letting in "uncontrolled" equipment directly into their combiners/OADMs, especially when it comes to controlling power levels etc. -- Mikael Abrahamsson email: swmike@swm.pp.se
GC> Date: Sat, 29 May 2004 16:53:17 -0400 GC> From: Gordon Cook GC> The point I am making in my report is NOT that the best GC> effort network has technology problems but rather that it has GC> ECONOMIC PROBLEMS. That it might support 2 or 3 players not GC> 2 or 3 HUNDRED. Best effort is cheaper to provide. Cheaper sells. Is there enough of a market to sustain premium services? IP-based VPNs haven't replaced FR and PtP WAN links, but FR and PtP haven't thwarted IP-based VPNs. GC> That until companies begin to go chapter seven and vanish, GC> the best effort net will be a black hole that burns up GC> capital because, for many players, the OPERATIONAL expense is GC> more than they get for bandwidth never mind cap-ex. Definitely true about opex and capex... but I'm not convinced that QoS is the magic bullet that will make the marketplace big enough and profitable enough. I don't see service offerings fixing the woes of screwball pricing. GC> best effort won't go away. many best effort players will. If all best effort players provided QoS/guaranteed services, would the survival rate be significantly higher as a result? GC> for the time being, best effort bandwidth prices as an GC> absolute commodity cannot sustain networks over the long GC> haul. A network that can deliver QoS the report hypothesizes GC> may be able to attract enough revenue to become profitable. That's where I'm not convinced. Current IP delineates the lower reliability boundary and a benchmark price point. Premium services won't have a lower cost than best-effort, so they must sell for more. Would the incremental service improvements be high enough to draw customers away from cheap BE _and_ support "sufficient" margins? First class hasn't stopped the cycle of airline bankruptcies and government bailouts. I don't see "first class data" as much different. GC> How to to this my group is still discussing. We don't GC> pretend that QoS is easy or any kind of mature collection of GC> technologies, but increasingly it looks as though the GC> industry, if it is ever going to be self sustaining, really GC> needs to look at QoS services and solutions. Perhaps, but only if the price is right. DSL sells better than Internet T1 lines, which sell better than end-to-end private lines and packet clouds. There's a reason for that. Eddy -- EverQuick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita _________________________________________________________________ DO NOT send mail to the following addresses : blacklist@brics.com -or- alfra@intc.net -or- curbjmp@intc.net Sending mail to spambait addresses is a great way to get blocked.
interesting reading.... http://mail.internet2.edu:8080/guest/archives/qbone-arch-dt/log200205/msg000... regards, /vicky Edward B. Dreger wrote:
GC> Date: Sat, 29 May 2004 16:53:17 -0400 GC> From: Gordon Cook
GC> The point I am making in my report is NOT that the best GC> effort network has technology problems but rather that it has GC> ECONOMIC PROBLEMS. That it might support 2 or 3 players not GC> 2 or 3 HUNDRED.
Best effort is cheaper to provide. Cheaper sells. Is there enough of a market to sustain premium services? IP-based VPNs haven't replaced FR and PtP WAN links, but FR and PtP haven't thwarted IP-based VPNs.
GC> That until companies begin to go chapter seven and vanish, GC> the best effort net will be a black hole that burns up GC> capital because, for many players, the OPERATIONAL expense is GC> more than they get for bandwidth never mind cap-ex.
Definitely true about opex and capex... but I'm not convinced that QoS is the magic bullet that will make the marketplace big enough and profitable enough. I don't see service offerings fixing the woes of screwball pricing.
GC> best effort won't go away. many best effort players will.
If all best effort players provided QoS/guaranteed services, would the survival rate be significantly higher as a result?
GC> for the time being, best effort bandwidth prices as an GC> absolute commodity cannot sustain networks over the long GC> haul. A network that can deliver QoS the report hypothesizes GC> may be able to attract enough revenue to become profitable.
That's where I'm not convinced. Current IP delineates the lower reliability boundary and a benchmark price point. Premium services won't have a lower cost than best-effort, so they must sell for more. Would the incremental service improvements be high enough to draw customers away from cheap BE _and_ support "sufficient" margins?
First class hasn't stopped the cycle of airline bankruptcies and government bailouts. I don't see "first class data" as much different.
GC> How to to this my group is still discussing. We don't GC> pretend that QoS is easy or any kind of mature collection of GC> technologies, but increasingly it looks as though the GC> industry, if it is ever going to be self sustaining, really GC> needs to look at QoS services and solutions.
Perhaps, but only if the price is right. DSL sells better than Internet T1 lines, which sell better than end-to-end private lines and packet clouds. There's a reason for that.
Eddy -- EverQuick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita _________________________________________________________________ DO NOT send mail to the following addresses : blacklist@brics.com -or- alfra@intc.net -or- curbjmp@intc.net Sending mail to spambait addresses is a great way to get blocked.
Thus spake "Gordon Cook" <cook@cookreport.com>
The point I am making in my report is NOT that the best effort network has technology problems but rather that it has ECONOMIC PROBLEMS. That it might support 2 or 3 players not 2 or 3 HUNDRED. That until companies begin to go chapter seven and vanish, the best effort net will be a black hole that burns up capital because, for many players, the OPERATIONAL expense is more than they get for bandwidth never mind cap-ex.
best effort won't go away. many best effort players will.
for the time being, best effort bandwidth prices as an absolute commodity cannot sustain networks over the long haul. A network that can deliver QoS the report hypothesizes may be able to attract enough revenue to become profitable. How to to this my group is still discussing. We don't pretend that QoS is easy or any kind of mature collection of technologies, but increasingly it looks as though the industry, if it is ever going to be self sustaining, really needs to look at QoS services and solutions.
This problem has little to do with BE vs. QoS. It's a temporary market imbalance caused by providers willing to sell service for less than cost; in the absence of external factors, eventually enough providers will go under for prices to rise back above cost. Possible external factors mainly consist of providers finding other add-on services sufficiently profitable to offset losses from basic transport. QoS is one possible add-on, but I haven't seen any convincing evidence customers would buy it today if it were available. There are many other services that providers could offer on top of the basic transport that could have the same effect without the substantial technical and economic challenges that QoS presents. This adds up to a market where it's possible for IP transport (BE or otherwise) to be unprofitable yet exist indefinitely. S Stephen Sprunk "Stupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them." --Aaron Sorkin
On Sun, 30 May 2004, Stephen Sprunk wrote:
This problem has little to do with BE vs. QoS. It's a temporary market imbalance caused by providers willing to sell service for less than cost; in the absence of external factors, eventually enough providers will go under for prices to rise back above cost.
I am actually not sure that someone actually have to go under. When the spare capacity sold under cost is all used up, it doesn't make sense to build out to sell, so nobody will want to sell at these under cost prices any more. Either demand will make the prices go up and then people will start building again, or companies will start to cancel contracts that gives them the smalles amount of money, and sell that at a slightly higher price. Or... technology will catch up and the cost of producing the capacity needed will match the current prices. What will happen is probably a combination of all the above. -- Mikael Abrahamsson email: swmike@swm.pp.se
This problem has little to do with BE vs. QoS. It's a temporary market imbalance caused by providers willing to sell service for less than cost; in the absence of external factors, eventually enough providers will go under for prices to rise back above cost.
I've seen compelling evidence over the past two years that clearly shows some carriers who have sold well below cost who then also went into chapter 11. Unfortunately these zombies don't see to want to die! :-) Selling below cost is one of many issues. Also Internet access is one area of most organisations communications needs. I also then have discussions, with some of the people listed on the acknowledgements page of this "article", that have done things because quote "it costs too much" when the reality is that its cost them nothing.
Neil J. McRae wrote:
I've seen compelling evidence over the past two years that clearly shows some carriers who have sold well below cost who then also went into chapter 11.
Fascinating discovery, that. What on earth will happen to us if _that_ word leaks out?!??! -- Requiescas in pace o email Ex turpi causa non oritur actio http://members.cox.net/larrysheldon/
participants (12)
-
Christopher J. Wolff -
Curtis Maurand -
Edward B. Dreger -
Eric Kuhnke -
Gordon Cook -
Laurence F. Sheldon, Jr. -
Mikael Abrahamsson -
Neil J. McRae -
Petri Helenius -
Simon Leinen -
Stephen Sprunk -
Vicky Rode