You are right the 100Mbps is pure network dynamics. right now we are adapting a wait and see but your added war story means we have to do more watching as well Raymond Macharia On Wed, Aug 5, 2009 at 4:52 PM, Fred Baker <fred@cisco.com> wrote:

That is very much to be expected, if nothing else due to pent-up demand. The existing vsat infrastructure tends to be pretty saturated, meaning that users experience a lot of loss as well as delay. What if they stop losing traffic?

War story: in 1995 I found myself sharing a podium with Phill Gross, who was then a VP with MCI. He was more or less apologizing for the behavior of his network. They had recently upgraded to a then-new-and-gee-whiz OC-3 infrastructure, in many places using parallel bandwidth, and were dropping a lot - he reported one link to be dropping 20%. So they then upgraded the whole thing to OC-12 - and fiber that was OC-3 was replaced with an OC-12. They presumed that this would give them 75% overprovisioning at worst. What they actually saw was that those links that had been dropping 20% were now dropping 4%. TCP observed that it was not getting crunched into the ground, and started opening its windows.

The other big issue with satcom is of course delay, and with conversion to fiber the delay plummets. That means that where you might have had <mumble> connections running at <low> average speed due to RTT, the average connection speed for the same session is instantly multiplied by <old-RTT>/<new-RTT>. That gives the user time and incentive to click again during that same time window - new load.

Price is very important, of course, but I suspect your 100 MBPS is explainable by simple network dynamics.

On Aug 5, 2009, at 6:13 AM, Raymond Macharia wrote:

Hello all,in the last two weeks or so providers in East Africa,

...

particularly in Kenya where I am, have been moving from Satellite to Fibre for the internet Back bone connectivity. From where I am I have seen an upsurge of about 100Mbps in the last two days from my users. It would be interesting to know if anyone out there has seen an increase in traffic from this region and by how much. There is more to come as providers are cutting prices and adding bandwidth to their networks.

Best Regards

Raymond Macharia

Just out of curriousity, what type of equipment is used to terminate circuits of this capacity? My experience stops at the 10GB mark. Thanks, Mike -----Original Message----- From: Rod Beck [mailto:Rod.Beck@hiberniaatlantic.com] Sent: Thursday, August 13, 2009 1:34 PM To: nanog@nanog.org Subject: TransAtlantic 40 Gig Waves http://www.hiberniaatlantic.com/documents/Hibernia40GAcrossAtlanticPR-JSA2-F... Roderick S. Beck Director of European Sales Hibernia Atlantic Budapest, New York, and Paris

Congrats Rod. Southern Cross and Nortel have been trialing 40Gbps waves on the 8000km segment from Hawaii to New Zealand. http://www.itnews.com.au/News/152866,southern-cross-trials-40gbps-nortel-kit... The 8000km segment is a LONG way - a very long way but it should mean stability for any cable system (I'm not sure there are segments that are much longer on any other system) - the bandwidth limit hasn't been hit yet! MMC Rod Beck wrote:

http://www.hiberniaatlantic.com/documents/Hibernia40GAcrossAtlanticPR-JSA2-F...

Roderick S. Beck Director of European Sales Hibernia Atlantic Budapest, New York, and Paris

On Fri, Aug 14, 2009 at 10:14:59AM +0100, Rod Beck wrote:

Obviously using 40 gig waves as the foundation blocks of one's network provides some economies of scale and per unit capex cost savings.

I would be curious if anyone knows how to convert this SONET/SDH 40 gig waves into a 40 gig Ethernet handoff?

Afterall, OC768 route cards are a tad expensive ...

I'm not aware of any solution that isn't going to be a lot more expensive than just using the native OC768 card (which isn't "that" expensive in crazy bankrupt carrier dollars, it's just not in the same ballpark as 10GE solutions). This in turn is going to be a lot more expensive than just running 4x10GE, for the moment. Of course native 40GE is in the works, so eventually this will make technical and financial sense, just not yet. But this is one of the major reasons I've been a proponent of 40GE standardization instead of focusing solely on 100GE, 40 maps directly to the next-generation optical technology and allows you to efficiently and affordably transport ethernet over long distances, whereas 100 is largely just a fancy cable management solution for hiding multiple parallel links (i.e. 10x10G) within a datacenter. Rod, do you know if the 40G waves increased the spectrum efficiency of your fiber? On land systems they pretty much break even, i.e. you can have a 100GHz 40G channels or 4x25GHz 10G channels but at the end of the day you still get the same amount of signal out of the fiber. I don't know whats being done on undersea cables though. Eventually this will get better too, and 40G will become the "native" wavelength standard with 10G being muxed onto them, similar to what we saw with the transition from 2.5G->10G 10 years ago. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)

Rod, do you know if the 40G waves increased the spectrum efficiency of your fiber? On land systems they pretty much break even, i.e. you can have a 100GHz 40G channels or 4x25GHz 10G channels but at the end of the day you still get the same amount of signal out of the fiber. I don't know whats being done on undersea cables though. Eventually this will get better too, and 40G will become the "native" wavelength standard with 10G being muxed onto them, similar to what we saw with the transition from 2.5G->10G 10 years ago. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC) "Rod. This is direct from engineering: The number of wavelengths or channels Hibernia have on their DWDM infrastructure remains the same, however now each wave can be at a rate of 40Gb/s instead of only 10Gb/s. In the extreme case, we get 4 times the capacity, but in reality, because of the existing installed 10G's, we would not necessarily swap out all existing cards. We could say the overall increase in capacity is "up-to" 4 times. The enabling technology is based on advanced encoding techniques allowing a greater rate of symbol transfer."

On Mon, 17 Aug 2009, Rod Beck wrote:

Rod, do you know if the 40G waves increased the spectrum efficiency of your fiber? On land systems they pretty much break even, i.e. you can have a 100GHz 40G channels or 4x25GHz 10G channels but at the end of the day you still get the same amount of signal out of the fiber. I don't know whats being done on undersea cables though. Eventually this will get better too, and 40G will become the "native" wavelength standard with 10G being muxed onto them, similar to what we saw with the transition from 2.5G->10G 10 years ago.

Think of this as the natural progression equivalent to the evolution of speed over phone lines, from 300bps to 56k, ie use of DSPs to get more out of the available spectral capacity. 10G is generally pretty simple NRZ, 40/100G is much more complex using multiple bit per "baud" and/or polarization. <http://www.ripe.net/ripe/meetings/ripe-55/presentations/lothberg-40-gb-to-my-mother.pdf> (around page 34 is where the stuff you're asking about comes in) My guess is that 100G will be available and being rolled out before 40G is widely used, thus we'll wont see much 40G in DWDM systems, at least not with the current encoding standards (because they don't fit directly into 10G systems as they have higher requirements). -- Mikael Abrahamsson email: swmike@swm.pp.se

On Fri, 14 Aug 2009 19:55:36 BST, Rod Beck said:

Well, the funny thing is that when I approached bandwidth buyers at some well known publicly traded carriers, they told me that 40 gig waves across the Atlantic were impossible.

Theoretically impossible, or just "impossible on the fiber that's already underwater"? Big difference there.

Indeed, when we decided to launch LAN PHY 10 GigE, the builder of our cable system told us it wasn't possible.

Again, was this "impossible on a cable the builder was about to build", or "impossible on the cable that the builder put under the water already"?

...

Well, the funny thing is that when I approached bandwidth buyers at some well known publicly traded carriers, they told me that 40 gig waves across the Atlantic were impossible.

Theoretically impossible, or just "impossible on the fiber that's already underwater"? Big difference there.

...

Indeed, when we decided to launch LAN PHY 10 GigE, the builder of our cable system told us it wasn't possible.

Again, was this "impossible on a cable the builder was about to build", or "impossible on the cable that the builder put under the water already"?

1) 40G across large bodies of water is a nice achievement, kudos to everyone involved everywhere. 2) Even on cables that are already deployed where impossible things couldn't happen, have. It just takes a little longer for the technology involved. Case in point, I point to DSL over legacy (old) copper plants. Even super old fiber in the ground can do things that were originally considered impossible. For existing undersea systems (whomever owns them) inline amplifiers as well as cable issues need to be ironed out. Now that folks know the cost of rolling out a new cable vs the cost of engineering specialized solutions for those sorts of spans, they have decisions to make... but I have (and it has been proven) faith in technology to the impossible works... it just takes some time. :) 25Ghz spacing wasn't possible 5 years ago either. And there is hfDWDM spacing in labs somewhere too. So other than for the folks that are provisioning 40G or who are considering deploying their own systems... what's the big deal? Deepak