Howdy, Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it? c = 186,282 miles/second 2742 miles from Seattle to Washington DC mainly driving I-90 2742/186282 ~= 0.015 seconds Thanks, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/
On Sat, Jun 20, 2020 at 09:24:11AM -0700, William Herrin wrote:
Howdy,
Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it?
c = 186,282 miles/second 2742 miles from Seattle to Washington DC mainly driving I-90
2742/186282 ~= 0.015 seconds
Speed of light in a fiber is more like 124K miles per second. It depends on the refractive index. And of course amplifiers and stuff. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net "The strain of anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that 'my ignorance is just as good as your knowledge.'"-Asimov
Besides the refractive index of glass that makes like go about 2/3rds it can in a vacuum, "Stuff" also includes many other things like modulation/demodulation, buffers, etc. I did a quora answer on this you can find at: https://www.quora.com/How-can-one-describe-the-delay-characteristics-of-ping... On 6/20/20 9:29 AM, Joe Greco wrote:
On Sat, Jun 20, 2020 at 09:24:11AM -0700, William Herrin wrote:
Howdy,
Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it?
c = 186,282 miles/second 2742 miles from Seattle to Washington DC mainly driving I-90
2742/186282 ~= 0.015 seconds
Speed of light in a fiber is more like 124K miles per second. It depends on the refractive index. And of course amplifiers and stuff.
... JG
Speed of light in glass ~200 km/s 100 km rtt = 1ms Coast-to-coast ~6000 km ~60ms Tim:> On Sat, Jun 20, 2020 at 12:27 PM William Herrin <bill@herrin.us> wrote:
Howdy,
Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it?
c = 186,282 miles/second 2742 miles from Seattle to Washington DC mainly driving I-90
2742/186282 ~= 0.015 seconds
Thanks, Bill Herrin
-- William Herrin bill@herrin.us https://bill.herrin.us/
-- Tim:>
And of course in your more realistic example: 2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path On Sat, Jun 20, 2020 at 12:36 PM Tim Durack <tdurack@gmail.com> wrote:
Speed of light in glass ~200 km/s
100 km rtt = 1ms
Coast-to-coast ~6000 km ~60ms
Tim:>
On Sat, Jun 20, 2020 at 12:27 PM William Herrin <bill@herrin.us> wrote:
Howdy,
Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it?
c = 186,282 miles/second 2742 miles from Seattle to Washington DC mainly driving I-90
2742/186282 ~= 0.015 seconds
Thanks, Bill Herrin
-- William Herrin bill@herrin.us https://bill.herrin.us/
-- Tim:>
-- Tim:>
Doing some rough back of the napkin math, an ultra low-latency path from, say, the Westin to 1275 K in Seattle will be in the 59 ms range. This is considerably longer than the I-90 driving distance would suggest because: - Best case optical distance is more like 5500 km, in part because the path actually will go Chicago-NJ-WDC and in part because a distance of 5000 km by right-of-way will be more like 5500 km when you account for things like maintenance coils, in-building wiring, etc. - You’ll need (at least) three OEO regens on that distance, since there’s no value in spending 5x to deploy an optical system that wouldn’t need to (like the ones that would manage that distance subsea). This is in addition to ~60 in-line amplification nodes, although that adds significantly less latency even in aggregate Some of that is simply due to cost savings. In theory, you could probably spend a boatload of money to build a route that cuts off some of the distance inefficiency and gets you closer to 4500 km optical distance with minimal slack coil, and maybe no regens, so you get a real-world performance of 46 ms. But there are no algo trading sites of importance in DC, and for everybody else there’s not enough money in the difference between 46 and 59 ms for someone to go invest in that type of deployment. Dave Cohen craetdave@gmail.com
On Jun 20, 2020, at 12:44 PM, Tim Durack <tdurack@gmail.com> wrote:
And of course in your more realistic example:
2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path
On Sat, Jun 20, 2020 at 12:36 PM Tim Durack <tdurack@gmail.com> wrote: Speed of light in glass ~200 km/s
100 km rtt = 1ms
Coast-to-coast ~6000 km ~60ms
Tim:>
On Sat, Jun 20, 2020 at 12:27 PM William Herrin <bill@herrin.us> wrote: Howdy,
Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it?
c = 186,282 miles/second 2742 miles from Seattle to Washington DC mainly driving I-90
2742/186282 ~= 0.015 seconds
Thanks, Bill Herrin
-- William Herrin bill@herrin.us https://bill.herrin.us/
-- Tim:>
-- Tim:>
An intriguing development in fiber optic media is hollow core optical fiber, which achieves 99.7% of the speed of light in a vacuum. https://www.extremetech.com/computing/151498-researchers-create-fiber-networ... -mel On Jun 20, 2020, at 10:14 AM, Dave Cohen <craetdave@gmail.com> wrote: Doing some rough back of the napkin math, an ultra low-latency path from, say, the Westin to 1275 K in Seattle will be in the 59 ms range. This is considerably longer than the I-90 driving distance would suggest because: - Best case optical distance is more like 5500 km, in part because the path actually will go Chicago-NJ-WDC and in part because a distance of 5000 km by right-of-way will be more like 5500 km when you account for things like maintenance coils, in-building wiring, etc. - You’ll need (at least) three OEO regens on that distance, since there’s no value in spending 5x to deploy an optical system that wouldn’t need to (like the ones that would manage that distance subsea). This is in addition to ~60 in-line amplification nodes, although that adds significantly less latency even in aggregate Some of that is simply due to cost savings. In theory, you could probably spend a boatload of money to build a route that cuts off some of the distance inefficiency and gets you closer to 4500 km optical distance with minimal slack coil, and maybe no regens, so you get a real-world performance of 46 ms. But there are no algo trading sites of importance in DC, and for everybody else there’s not enough money in the difference between 46 and 59 ms for someone to go invest in that type of deployment. Dave Cohen craetdave@gmail.com On Jun 20, 2020, at 12:44 PM, Tim Durack <tdurack@gmail.com> wrote: And of course in your more realistic example: 2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path On Sat, Jun 20, 2020 at 12:36 PM Tim Durack <tdurack@gmail.com<mailto:tdurack@gmail.com>> wrote: Speed of light in glass ~200 km/s 100 km rtt = 1ms Coast-to-coast ~6000 km ~60ms Tim:> On Sat, Jun 20, 2020 at 12:27 PM William Herrin <bill@herrin.us<mailto:bill@herrin.us>> wrote: Howdy, Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it? c = 186,282 miles/second 2742 miles from Seattle to Washington DC mainly driving I-90 2742/186282 ~= 0.015 seconds Thanks, Bill Herrin -- William Herrin bill@herrin.us<mailto:bill@herrin.us> https://bill.herrin.us/ -- Tim:> -- Tim:>
And thus far, no one has mentioned switching speed and other electronic overhead such as the transceivers (that's the big one, IIRC.) I also don't recall if anyone mentioned that the 30ms is as the photon flies, not fiber distance. -Wayne On Sat, Jun 20, 2020 at 05:32:30PM +0000, Mel Beckman wrote:
An intriguing development in fiber optic media is hollow core optical fiber, which achieves 99.7% of the speed of light in a vacuum.
https://www.extremetech.com/computing/151498-researchers-create-fiber-networ...
-mel
On Jun 20, 2020, at 10:14 AM, Dave Cohen <craetdave@gmail.com> wrote:
??? Doing some rough back of the napkin math, an ultra low-latency path from, say, the Westin to 1275 K in Seattle will be in the 59 ms range. This is considerably longer than the I-90 driving distance would suggest because: - Best case optical distance is more like 5500 km, in part because the path actually will go Chicago-NJ-WDC and in part because a distance of 5000 km by right-of-way will be more like 5500 km when you account for things like maintenance coils, in-building wiring, etc. - You???ll need (at least) three OEO regens on that distance, since there???s no value in spending 5x to deploy an optical system that wouldn???t need to (like the ones that would manage that distance subsea). This is in addition to ~60 in-line amplification nodes, although that adds significantly less latency even in aggregate
Some of that is simply due to cost savings. In theory, you could probably spend a boatload of money to build a route that cuts off some of the distance inefficiency and gets you closer to 4500 km optical distance with minimal slack coil, and maybe no regens, so you get a real-world performance of 46 ms. But there are no algo trading sites of importance in DC, and for everybody else there???s not enough money in the difference between 46 and 59 ms for someone to go invest in that type of deployment.
Dave Cohen craetdave@gmail.com
On Jun 20, 2020, at 12:44 PM, Tim Durack <tdurack@gmail.com> wrote:
??? And of course in your more realistic example:
2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path
On Sat, Jun 20, 2020 at 12:36 PM Tim Durack <tdurack@gmail.com<mailto:tdurack@gmail.com>> wrote: Speed of light in glass ~200 km/s
100 km rtt = 1ms
Coast-to-coast ~6000 km ~60ms
Tim:>
On Sat, Jun 20, 2020 at 12:27 PM William Herrin <bill@herrin.us<mailto:bill@herrin.us>> wrote: Howdy,
Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it?
c = 186,282 miles/second 2742 miles from Seattle to Washington DC mainly driving I-90
2742/186282 ~= 0.015 seconds
Thanks, Bill Herrin
-- William Herrin bill@herrin.us<mailto:bill@herrin.us> https://bill.herrin.us/
-- Tim:>
-- Tim:>
--- Wayne Bouchard web@typo.org Network Dude http://www.typo.org/~web/
On Sat, 20 Jun 2020 at 20:52, Wayne Bouchard <web@typo.org> wrote:
And thus far, no one has mentioned switching speed and other electronic overhead such as the transceivers (that's the big one, IIRC.)
This will be something from tens of meters (low lat swich), to few hundred meters (typical pipeline), to 2km delay (NPU+FAB+NPU) per active IP device. If that is a big one, I guess it depends, cross atlantic, no, inside rack, maybe. -- ++ytti
On 6/20/20 1:56 PM, Saku Ytti wrote:
On Sat, 20 Jun 2020 at 20:52, Wayne Bouchard <web@typo.org> wrote:
And thus far, no one has mentioned switching speed and other electronic overhead such as the transceivers (that's the big one, IIRC.) This will be something from tens of meters (low lat swich), to few hundred meters (typical pipeline), to 2km delay (NPU+FAB+NPU) per active IP device. If that is a big one, I guess it depends, cross atlantic, no, inside rack, maybe.
I think he might be referring to the newer modulation types (QAM) on long haul transport. There's quite a bit of time in uS that the encoding takes into QAM and adding FEC. You typically won't see this at the plug-able level between switches and stuff. 60ms is nothing really, and I'm happy I don't need to play in the HFT space anymore. I do wish my home connection wasn't 60 ms across town as spectrum wants takes TPA-ATL-DCA-DEN-NY to get to my rack. :-) -- Bryan Fields 727-409-1194 - Voice http://bryanfields.net
On Sat, Jun 20, 2020 at 16:14 Bryan Fields <Bryan@bryanfields.net> wrote:
On 6/20/20 1:56 PM, Saku Ytti wrote:
On Sat, 20 Jun 2020 at 20:52, Wayne Bouchard <web@typo.org> wrote:
And thus far, no one has mentioned switching speed and other electronic overhead such as the transceivers (that's the big one, IIRC.) This will be something from tens of meters (low lat swich), to few hundred meters (typical pipeline), to 2km delay (NPU+FAB+NPU) per active IP device. If that is a big one, I guess it depends, cross atlantic, no, inside rack, maybe.
I think he might be referring to the newer modulation types (QAM) on long haul transport. There's quite a bit of time in uS that the encoding takes into QAM and adding FEC. You typically won't see this at the plug-able level between switches and stuff.
60ms is nothing really, and I'm happy I don't need to play in the HFT space anymore. I do wish my home connection wasn't 60 ms across town as spectrum wants takes TPA-ATL-DCA-DEN-NY to get to my rack. :-)
working on that ...... :-)
-- Bryan Fields
727-409-1194 - Voice http://bryanfields.net
On Sat, 20 Jun 2020 at 23:14, Bryan Fields <Bryan@bryanfields.net> wrote:
I think he might be referring to the newer modulation types (QAM) on long haul transport. There's quite a bit of time in uS that the encoding takes into QAM and adding FEC. You typically won't see this at the plug-able level between switches and stuff.
FEC is low tens of meters (i.e. low tens of nanoseconds), QAM is less. Won't impact the pipeline or NPU scenarios meaningfully, will impact the low latency scenario. -- ++ytti
Did you not read my posting on Quora? Tim On 6/20/20 10:49 AM, Wayne Bouchard wrote:
And thus far, no one has mentioned switching speed and other electronic overhead such as the transceivers (that's the big one, IIRC.)
I also don't recall if anyone mentioned that the 30ms is as the photon flies, not fiber distance.
-Wayne
On Sat, Jun 20, 2020 at 05:32:30PM +0000, Mel Beckman wrote:
An intriguing development in fiber optic media is hollow core optical fiber, which achieves 99.7% of the speed of light in a vacuum.
https://www.extremetech.com/computing/151498-researchers-create-fiber-networ...
-mel
On Jun 20, 2020, at 10:14 AM, Dave Cohen <craetdave@gmail.com> wrote:
??? Doing some rough back of the napkin math, an ultra low-latency path from, say, the Westin to 1275 K in Seattle will be in the 59 ms range. This is considerably longer than the I-90 driving distance would suggest because: - Best case optical distance is more like 5500 km, in part because the path actually will go Chicago-NJ-WDC and in part because a distance of 5000 km by right-of-way will be more like 5500 km when you account for things like maintenance coils, in-building wiring, etc. - You???ll need (at least) three OEO regens on that distance, since there???s no value in spending 5x to deploy an optical system that wouldn???t need to (like the ones that would manage that distance subsea). This is in addition to ~60 in-line amplification nodes, although that adds significantly less latency even in aggregate
Some of that is simply due to cost savings. In theory, you could probably spend a boatload of money to build a route that cuts off some of the distance inefficiency and gets you closer to 4500 km optical distance with minimal slack coil, and maybe no regens, so you get a real-world performance of 46 ms. But there are no algo trading sites of importance in DC, and for everybody else there???s not enough money in the difference between 46 and 59 ms for someone to go invest in that type of deployment.
Dave Cohen craetdave@gmail.com
On Jun 20, 2020, at 12:44 PM, Tim Durack <tdurack@gmail.com> wrote:
??? And of course in your more realistic example:
2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path
On Sat, Jun 20, 2020 at 12:36 PM Tim Durack <tdurack@gmail.com<mailto:tdurack@gmail.com>> wrote: Speed of light in glass ~200 km/s
100 km rtt = 1ms
Coast-to-coast ~6000 km ~60ms
Tim:>
On Sat, Jun 20, 2020 at 12:27 PM William Herrin <bill@herrin.us<mailto:bill@herrin.us>> wrote: Howdy,
Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it?
c = 186,282 miles/second 2742 miles from Seattle to Washington DC mainly driving I-90
2742/186282 ~= 0.015 seconds
Thanks, Bill Herrin
-- William Herrin bill@herrin.us<mailto:bill@herrin.us> https://bill.herrin.us/
-- Tim:>
-- Tim:>
--- Wayne Bouchard web@typo.org Network Dude http://www.typo.org/~web/
Mel Beckman <mel@beckman.org> wrote:
An intriguing development in fiber optic media is hollow core optical fiber, which achieves 99.7% of the speed of light in a vacuum.
https://www.extremetech.com/computing/151498-researchers-create-fiber-networ...
Here's an update from 7 years after that article which hints at the downside of hollow core fibre: https://phys.org/news/2020-03-hollow-core-fiber-technology-mainstream-optica... It sounds like attenuation was a big problem: "in the space of 18 months the attenuation in data-transmitting hollow-core fibers has been reduced by over a factor of 10, from 3.5dB/km to only 0.28 dB/km within a factor of two of the attenuation of conventional all-glass fiber technology." Tony. -- f.anthony.n.finch <dot@dotat.at> http://dotat.at/ Shetland Isles: Southeasterly 5 or 6, veering southerly or southwesterly 3 or 4, then backing southeasterly 5 later in southwest. Slight or moderate, occasionally rough later in far west. Occasional rain then mainly fair, but showers far in east. Good, occasionally moderate.
Hello, Taking advantage of this thread may I ask something?. I have heard of "wireless fiber optic", something like an antenna with a laser pointing from one building to the other, having said this I can assume this link with have lower RTT than a laser thru a fiber optic made of glass? Thanks, Alejandro, On 6/20/20 1:11 PM, Dave Cohen wrote:
Doing some rough back of the napkin math, an ultra low-latency path from, say, the Westin to 1275 K in Seattle will be in the 59 ms range. This is considerably longer than the I-90 driving distance would suggest because: - Best case optical distance is more like 5500 km, in part because the path actually will go Chicago-NJ-WDC and in part because a distance of 5000 km by right-of-way will be more like 5500 km when you account for things like maintenance coils, in-building wiring, etc. - You’ll need (at least) three OEO regens on that distance, since there’s no value in spending 5x to deploy an optical system that wouldn’t need to (like the ones that would manage that distance subsea). This is in addition to ~60 in-line amplification nodes, although that adds significantly less latency even in aggregate
Some of that is simply due to cost savings. In theory, you could probably spend a boatload of money to build a route that cuts off some of the distance inefficiency and gets you closer to 4500 km optical distance with minimal slack coil, and maybe no regens, so you get a real-world performance of 46 ms. But there are no algo trading sites of importance in DC, and for everybody else there’s not enough money in the difference between 46 and 59 ms for someone to go invest in that type of deployment.
Dave Cohen craetdave@gmail.com
On Jun 20, 2020, at 12:44 PM, Tim Durack <tdurack@gmail.com> wrote:
And of course in your more realistic example:
2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path
On Sat, Jun 20, 2020 at 12:36 PM Tim Durack <tdurack@gmail.com <mailto:tdurack@gmail.com>> wrote:
Speed of light in glass ~200 km/s
100 km rtt = 1ms
Coast-to-coast ~6000 km ~60ms
Tim:>
On Sat, Jun 20, 2020 at 12:27 PM William Herrin <bill@herrin.us <mailto:bill@herrin.us>> wrote:
Howdy,
Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it?
c = 186,282 miles/second 2742 miles from Seattle to Washington DC mainly driving I-90
2742/186282 ~= 0.015 seconds
Thanks, Bill Herrin
-- William Herrin bill@herrin.us <mailto:bill@herrin.us> https://bill.herrin.us/
-- Tim:>
-- Tim:>
On Sat, Jun 20, 2020 at 12:56 PM Alejandro Acosta < alejandroacostaalamo@gmail.com> wrote:
Hello,
Taking advantage of this thread may I ask something?. I have heard of "wireless fiber optic", something like an antenna with a laser pointing from one building to the other, having said this I can assume this link with have lower RTT than a laser thru a fiber optic made of glass?
See: Terrabeam from about the year 2000. -- Joe Hamelin, W7COM, Tulalip, WA, +1 (360) 474-7474
Microwave is used for long haul wireless transmission for the ultra-latency crowd. Free space laser has more bandwidth, but is sensitive to fog and at least until the last few years much less range. I sell ULL routes to financial players. A 10 meg microwave circuit CME/Secaucus Equinix ranges from $185K per month to $20K a month. ________________________________ From: NANOG <nanog-bounces+rod.beck=unitedcablecompany.com@nanog.org> on behalf of Joe Hamelin <joe@nethead.com> Sent: Saturday, June 20, 2020 10:19 PM To: Alejandro Acosta <alejandroacostaalamo@gmail.com> Cc: NANOG list <nanog@nanog.org> Subject: Re: 60 ms cross-continent On Sat, Jun 20, 2020 at 12:56 PM Alejandro Acosta <alejandroacostaalamo@gmail.com<mailto:alejandroacostaalamo@gmail.com>> wrote: Hello, Taking advantage of this thread may I ask something?. I have heard of "wireless fiber optic", something like an antenna with a laser pointing from one building to the other, having said this I can assume this link with have lower RTT than a laser thru a fiber optic made of glass? See: Terrabeam from about the year 2000. -- Joe Hamelin, W7COM, Tulalip, WA, +1 (360) 474-7474
The speed of light in fiber is only about 2/3 the speed of light in a vacuum, so that 15 ms is really about 22.5 ms. That brings the total to about 45 ms. Some would come from how many miles of extra glass in that 2,742 miles in the form of slack loops. Some would come from fiber routes not being straight lines. Allied Fiber's formerly planned route from the Westin Building to Equinix Ashburn was about 4,464 miles. That's about 38% longer than your 2,742 miles. Add that 38% to the previous 45 ms and you're at 62.1 ms. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "William Herrin" <bill@herrin.us> To: nanog@nanog.org Sent: Saturday, June 20, 2020 11:24:11 AM Subject: 60 ms cross-continent Howdy, Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it? c = 186,282 miles/second 2742 miles from Seattle to Washington DC mainly driving I-90 2742/186282 ~= 0.015 seconds Thanks, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/
Sent from my iPhone
On Jun 20, 2020, at 9:27 AM, William Herrin <bill@herrin.us> wrote:
Howdy,
Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it?
c = 186,282 miles/second
This is c in a vacuum. Light transmission through a medium is slower. In the case of an optical fiber about 31% slower. My lowest latency transit paths Palo Alto to the ashburn area are around 58ms. the great circle route for the two dcs involved is a distance 2408 miles which gives you a 39.6ms Lower bound. The path isn’t quite a straight as that, but if you eliminate the 6 routers in the path and count up the oeo regens I’m sure you can account most of the extra in the form of distance.
2742 miles from Seattle to Washington DC mainly driving I-90
2742/186282 ~= 0.015 seconds
Thanks, Bill Herrin
-- William Herrin bill@herrin.us https://bill.herrin.us/
On 2020-06-20, at 19:07, Joel Jaeggli <joelja@bogus.com> wrote:
This is c in a vacuum. Light transmission through a medium is slower.
Ob-movie: https://en.wikipedia.org/wiki/The_Hummingbird_Project Grüße, Carsten
This was also pitched as one of the killer-apps for the SpaceX Starlink satellite array, particularly for cross-Atlantic and cross-Pacific trading. https://blogs.cfainstitute.org/marketintegrity/2019/06/25/fspacex-is-opening... "Several commentators quickly caught onto the fact that an extremely expensive network whose main selling point is long-distance, low-latency coverage has a unique chance to fund its growth by addressing the needs of a wealthy market that has a high willingness to pay — high-frequency traders." Regards Marshall On Sat, Jun 20, 2020 at 2:01 PM Carsten Bormann <cabo@tzi.org> wrote:
On 2020-06-20, at 19:07, Joel Jaeggli <joelja@bogus.com> wrote:
This is c in a vacuum. Light transmission through a medium is slower.
Ob-movie: https://en.wikipedia.org/wiki/The_Hummingbird_Project
Grüße, Carsten
On Sat, Jun 20, 2020 at 5:05 PM Marshall Eubanks <marshall.eubanks@gmail.com> wrote:
This was also pitched as one of the killer-apps for the SpaceX Starlink satellite array, particularly for cross-Atlantic and cross-Pacific trading.
https://blogs.cfainstitute.org/marketintegrity/2019/06/25/fspacex-is-opening...
"Several commentators quickly caught onto the fact that an extremely expensive network whose main selling point is long-distance, low-latency coverage has a unique chance to fund its growth by addressing the needs of a wealthy market that has a high willingness to pay — high-frequency traders."
This is a nice plot for a movie, but not how HFT is really done. It's so much easier to colocate on the same datacenter of the exchange and run algorithms from there; while those algorithms need humans to guide their strategy, the human thought process takes a couple of seconds anyways. So the real HFTs keep using the defined strategy while the human controller doesn't tell it otherwise. And in order to preserve equality among traders, each exchange already adds physically (loops of fiber or copper cable) some ns to closer racks so everyone gets at the system at the same time. And then comes a really high added latency of the trade risk controller, which limits what a trader is allowed to expose itself to what is deposited or agreed with the exchange. And this comes with both latency and jitter due to its implementation, making even the faster HFT only faster on average, not faster at every transaction. Rubens
On Sun, Jun 21, 2020 at 02:17:08PM -0300, Rubens Kuhl wrote:
On Sat, Jun 20, 2020 at 5:05 PM Marshall Eubanks <marshall.eubanks@gmail.com> wrote:
This was also pitched as one of the killer-apps for the SpaceX Starlink satellite array, particularly for cross-Atlantic and cross-Pacific trading.
https://blogs.cfainstitute.org/marketintegrity/2019/06/25/fspacex-is-opening...
"Several commentators quickly caught onto the fact that an extremely expensive network whose main selling point is long-distance, low-latency coverage has a unique chance to fund its growth by addressing the needs of a wealthy market that has a high willingness to pay — high-frequency traders."
This is a nice plot for a movie, but not how HFT is really done. It's so much easier to colocate on the same datacenter of the exchange and run algorithms from there; while those algorithms need humans to guide their strategy, the human thought process takes a couple of seconds anyways. So the real HFTs keep using the defined strategy while the human controller doesn't tell it otherwise.
For faster access to one exchange, yes, absolutely, colocate at the exchange. But there's more then one exchange. As one example, many index futures trade in Chicago. The stocks that make up those indices mostly trade in New York. There's money to be made on the arbitrage, if your Chicago algorithms get faster information from New York (and vice versa) than everyone else's algorithms. More expensive but shorter fiber routes have been build between NYC and Chicago for this reason, as have a microwave paths (to get speed-of-light in air rather than in glass). There's competition to have the microwave towers as close as possible to the data centers, because the last mile is fiber so the longer your last mile, the less valuable your network. https://www.bloomberg.com/news/features/2019-03-08/the-gazillion-dollar-stan... -- Brett
On 6/21/20 1:53 PM, Brett Frankenberger wrote:
On Sun, Jun 21, 2020 at 02:17:08PM -0300, Rubens Kuhl wrote:
On Sat, Jun 20, 2020 at 5:05 PM Marshall Eubanks <marshall.eubanks@gmail.com> wrote:
This was also pitched as one of the killer-apps for the SpaceX Starlink satellite array, particularly for cross-Atlantic and cross-Pacific trading.
https://blogs.cfainstitute.org/marketintegrity/2019/06/25/fspacex-is-opening...
"Several commentators quickly caught onto the fact that an extremely expensive network whose main selling point is long-distance, low-latency coverage has a unique chance to fund its growth by addressing the needs of a wealthy market that has a high willingness to pay — high-frequency traders."
This is a nice plot for a movie, but not how HFT is really done. It's so much easier to colocate on the same datacenter of the exchange and run algorithms from there; while those algorithms need humans to guide their strategy, the human thought process takes a couple of seconds anyways. So the real HFTs keep using the defined strategy while the human controller doesn't tell it otherwise. For faster access to one exchange, yes, absolutely, colocate at the exchange. But there's more then one exchange.
As one example, many index futures trade in Chicago. The stocks that make up those indices mostly trade in New York. There's money to be made on the arbitrage, if your Chicago algorithms get faster information from New York (and vice versa) than everyone else's algorithms.
More expensive but shorter fiber routes have been build between NYC and Chicago for this reason, as have a microwave paths (to get speed-of-light in air rather than in glass). There's competition to have the microwave towers as close as possible to the data centers, because the last mile is fiber so the longer your last mile, the less valuable your network.
https://www.bloomberg.com/news/features/2019-03-08/the-gazillion-dollar-stan...
... and similar to this: https://www.extremetech.com/extreme/122989-1-5-billion-the-cost-of-cutting-l...
-- Brett
This is a nice plot for a movie, but not how HFT is really done. It's so much easier to colocate on the same datacenter of the exchange and run algorithms from there; while those algorithms need humans to guide their strategy, the human thought process takes a couple of seconds anyways. So the real HFTs keep using the defined strategy while the human controller doesn't tell it otherwise.
For faster access to one exchange, yes, absolutely, colocate at the exchange. But there's more then one exchange.
Yes, but to do real HFT you will need to colocate at each exchange. Otherwise your competitors have a head start on you.
As one example, many index futures trade in Chicago. The stocks that make up those indices mostly trade in New York. There's money to be made on the arbitrage, if your Chicago algorithms get faster information from New York (and vice versa) than everyone else's algorithms.
Most traded index futures are longer than just that day closing, usually months to a year in advance. They are influenced mostly by traders perception on economic futures, and the current stocks valuation is a poor proxy for it. There is more chance in reading the news feeds and speculating its impact on perception than stocks. Rubens
Serious HFT moved to shortwave years ago. The chicago-NYC routes by microwave still exist, but are only for things that need higher data rates (as measured in kbps). It's hard to hide a giant log-periodic or yagi-uda antenna. The sites near Chicago that are aimed at London are well known to those in the industry. On Sun, Jun 21, 2020 at 10:53 AM Brett Frankenberger <rbf+nanog@panix.com> wrote:
On Sun, Jun 21, 2020 at 02:17:08PM -0300, Rubens Kuhl wrote:
On Sat, Jun 20, 2020 at 5:05 PM Marshall Eubanks < marshall.eubanks@gmail.com> wrote:
This was also pitched as one of the killer-apps for the SpaceX Starlink satellite array, particularly for cross-Atlantic and cross-Pacific trading.
https://blogs.cfainstitute.org/marketintegrity/2019/06/25/fspacex-is-opening...
"Several commentators quickly caught onto the fact that an extremely expensive network whose main selling point is long-distance, low-latency coverage has a unique chance to fund its growth by addressing the needs of a wealthy market that has a high willingness to pay — high-frequency traders."
This is a nice plot for a movie, but not how HFT is really done. It's so much easier to colocate on the same datacenter of the exchange and run algorithms from there; while those algorithms need humans to guide their strategy, the human thought process takes a couple of seconds anyways. So the real HFTs keep using the defined strategy while the human controller doesn't tell it otherwise.
For faster access to one exchange, yes, absolutely, colocate at the exchange. But there's more then one exchange.
As one example, many index futures trade in Chicago. The stocks that make up those indices mostly trade in New York. There's money to be made on the arbitrage, if your Chicago algorithms get faster information from New York (and vice versa) than everyone else's algorithms.
More expensive but shorter fiber routes have been build between NYC and Chicago for this reason, as have a microwave paths (to get speed-of-light in air rather than in glass). There's competition to have the microwave towers as close as possible to the data centers, because the last mile is fiber so the longer your last mile, the less valuable your network.
https://www.bloomberg.com/news/features/2019-03-08/the-gazillion-dollar-stan...
-- Brett
William Herrin
Howdy,
Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it?
Wallstreet did :) https://www.wired.com/2012/08/ff_wallstreet_trading/ adam
On 6/22/20 12:59 AM, adamv0025@netconsultings.com wrote:
William Herrin
Howdy,
Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it?
Wallstreet did :) https://www.wired.com/2012/08/ff_wallstreet_trading/
“Of course, you’d need a particle accelerator to make it work.” So THAT'S why CERN wants to build an even bigger accelerator than the LHC!
Have you accounted for glass as opposed to vacuum? And the fact that fiber optic networks can't be straight lines if their purpose is to aggregate traffic along the way and they also need to follow some less-than-straight right of way. Regards, Roderick. ________________________________ From: NANOG <nanog-bounces+rod.beck=unitedcablecompany.com@nanog.org> on behalf of Stephen Satchell via NANOG <nanog@nanog.org> Sent: Monday, June 22, 2020 9:37 PM To: nanog@nanog.org <nanog@nanog.org> Subject: Re: 60 ms cross-continent On 6/22/20 12:59 AM, adamv0025@netconsultings.com wrote:
William Herrin
Howdy,
Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it?
Wallstreet did :) https://www.wired.com/2012/08/ff_wallstreet_trading/
“Of course, you’d need a particle accelerator to make it work.” So THAT'S why CERN wants to build an even bigger accelerator than the LHC!
Stephen Satchell via NANOG Sent: Monday, June 22, 2020 8:37 PM
On 6/22/20 12:59 AM, adamv0025@netconsultings.com wrote:
William Herrin
Howdy,
Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it?
Wallstreet did :) https://www.wired.com/2012/08/ff_wallstreet_trading/
“Of course, you’d need a particle accelerator to make it work.”
So THAT'S why CERN wants to build an even bigger accelerator than the LHC!
Yep, why to go around the planet chasing a perfect geodesic with as few relay towers or drones if you can go through (shortest distance is always a straight line as opposed to an arc). While maintaining the speed of light in vacuum since neutrinos don't seem interact with regular matter, that's why they are so darn hard to detect. All you need is an extremely powerful neutrino detector to get you above the 51:49 success ratio. (49% packet loss is not what we're accustomed to, but for these guys it's low enough to start making money). It's quite a fascinating networking world these guys live in, working for a HFT company would be my dream job, always pushing the envelope, racing to the bottom, it's like F1 of the networking world just without the safety and fairness BS to slow you down. adam
Many of the traders have set up their short wave radio transmitters for use across the Atlantic. Bandwidth is only 4 kliobits, but that is enough to send a message saying "buy the SPY Option contracts". It is quite a bit faster than fiber. Regards, Roderick. ________________________________ From: NANOG <nanog-bounces+rod.beck=unitedcablecompany.com@nanog.org> on behalf of adamv0025@netconsultings.com <adamv0025@netconsultings.com> Sent: Tuesday, June 23, 2020 10:20 AM To: nanog@nanog.org <nanog@nanog.org>; list@satchell.net <list@satchell.net> Subject: RE: 60 ms cross-continent
Stephen Satchell via NANOG Sent: Monday, June 22, 2020 8:37 PM
On 6/22/20 12:59 AM, adamv0025@netconsultings.com wrote:
William Herrin
Howdy,
Why is latency between the east and west coasts so bad? Speed of light accounts for about 15ms each direction for a 30ms round trip. Where does the other 30ms come from and why haven't we gotten rid of it?
Wallstreet did :) https://www.wired.com/2012/08/ff_wallstreet_trading/
“Of course, you’d need a particle accelerator to make it work.”
So THAT'S why CERN wants to build an even bigger accelerator than the LHC!
Yep, why to go around the planet chasing a perfect geodesic with as few relay towers or drones if you can go through (shortest distance is always a straight line as opposed to an arc). While maintaining the speed of light in vacuum since neutrinos don't seem interact with regular matter, that's why they are so darn hard to detect. All you need is an extremely powerful neutrino detector to get you above the 51:49 success ratio. (49% packet loss is not what we're accustomed to, but for these guys it's low enough to start making money). It's quite a fascinating networking world these guys live in, working for a HFT company would be my dream job, always pushing the envelope, racing to the bottom, it's like F1 of the networking world just without the safety and fairness BS to slow you down. adam
participants (23)
-
adamv0025＠netconsultings.com -
Alejandro Acosta -
Brett Frankenberger -
Bryan Fields -
Carsten Bormann -
Dave Cohen -
Eric Kuhnke -
Joe Greco -
Joe Hamelin -
Joel Jaeggli -
Marshall Eubanks -
Martin Hannigan -
Mel Beckman -
Mike Hammett -
Rod Beck -
Rubens Kuhl -
Saku Ytti -
Stephen Satchell -
Tim Durack -
Tim Požár -
Tony Finch -
Wayne Bouchard -
William Herrin