Current diameter of the Internet?
What is the current estimated diameter of the Internet? Maximum (worst-case) RTT edge-to-edge? Most public latency data is now edge-to-cloud, not edge-to-edge. Cloud engineers have done a great job, and edge-to-cloud less than 1-sec RTT. Where have the long-slow pipes gone? https://www.cloudping.co/grid https://learn.microsoft.com/en-us/azure/networking/azure-network-latency?tab... https://www.verizon.com/business/terms/latency/
On 7/19/24 15:07, Sean Donelan wrote:
What is the current estimated diameter of the Internet?
Maximum (worst-case) RTT edge-to-edge?
Most public latency data is now edge-to-cloud, not edge-to-edge. Cloud engineers have done a great job, and edge-to-cloud less than 1-sec RTT.
Where have the long-slow pipes gone?
https://learn.microsoft.com/en-us/azure/networking/azure-network-latency?tab...
The RTT across geostationary satellites has remained stubbornly constant we just don't use them unless there's no other alternative. A colleague sent me a trace the other day while flying from hawaii to los vegas that was otherwise performant but still incurs the overhead.
Pity we can’t ping Voyagers. S.
On Jul 19, 2024, at 19:45, joel jaeggli <joelja@bogus.com> wrote:
On 7/19/24 15:07, Sean Donelan wrote:
What is the current estimated diameter of the Internet?
Maximum (worst-case) RTT edge-to-edge?
Most public latency data is now edge-to-cloud, not edge-to-edge. Cloud engineers have done a great job, and edge-to-cloud less than 1-sec RTT.
Where have the long-slow pipes gone?
https://learn.microsoft.com/en-us/azure/networking/azure-network-latency?tab...
The RTT across geostationary satellites has remained stubbornly constant we just don't use them unless there's no other alternative. A colleague sent me a trace the other day while flying from hawaii to los vegas that was otherwise performant but still incurs the overhead.
Where do you get 3 days? Voyager 1 is about 15.2B miles or 22.665707 hours at the speed of light. On Sat, Jul 20, 2024 at 7:12 PM Nathan Angelacos <nangel@tetrasec.net> wrote:
On Sat, 2024-07-20 at 00:58 -0500, Stas Bilder wrote:
Pity we can’t ping Voyagers.
S.
ROTFL, you actually had me pull out Star Trek - The Movie... Wow... what a blast from 1979.
So yeah ... According to our media outlets, RTT of the internet is ... um 3 days.
See https://voyager.jpl.nasa.gov/mission/status/ Thanks, Donald =============================== Donald E. Eastlake 3rd d3e3e3@gmail.com On Sun, Jul 21, 2024 at 10:46 AM Josh Luthman <josh@imaginenetworksllc.com> wrote:
Where do you get 3 days?
Voyager 1 is about 15.2B miles or 22.665707 hours at the speed of light.
On Sat, Jul 20, 2024 at 7:12 PM Nathan Angelacos <nangel@tetrasec.net> wrote:
On Sat, 2024-07-20 at 00:58 -0500, Stas Bilder wrote:
Pity we can’t ping Voyagers.
S.
ROTFL, you actually had me pull out Star Trek - The Movie... Wow... what a blast from 1979.
So yeah ... According to our media outlets, RTT of the internet is ... um 3 days.
Josh, Because the speed of light is different in different mediums. It depends on the index of refraction. Most of the Internet is on fiber optics, and the speed of light in glass fiber is dramatically slower than in a vacuum. Long distance single-mode communication fiber typically has a core index of refraction of 1.4682 at 1550nm (mid-C-band). So the speed of light in this type of fiber is the speed of light in a vacuum 299,792,458 m/s divided by 1.4682 = 204,190,477 m/s. You have to add to that the latency of any optical to electrical transformations, which happens in most every router or switch. Three days is probably an underestimate. -mel beckman On Jul 21, 2024, at 7:47 AM, Josh Luthman <josh@imaginenetworksllc.com> wrote: Where do you get 3 days? Voyager 1 is about 15.2B miles or 22.665707 hours at the speed of light. On Sat, Jul 20, 2024 at 7:12 PM Nathan Angelacos <nangel@tetrasec.net<mailto:nangel@tetrasec.net>> wrote: On Sat, 2024-07-20 at 00:58 -0500, Stas Bilder wrote: Pity we can’t ping Voyagers. S. ROTFL, you actually had me pull out Star Trek - The Movie... Wow... what a blast from 1979. So yeah ... According to our media outlets, RTT of the internet is ... um 3 days.
Once upon a time, Mel Beckman <mel@beckman.org> said:
Because the speed of light is different in different mediums. It depends on the index of refraction. Most of the Internet is on fiber optics, and the speed of light in glass fiber is dramatically slower than in a vacuum. Long distance single-mode communication fiber typically has a core index of refraction of 1.4682 at 1550nm (mid-C-band). So the speed of light in this type of fiber is the speed of light in a vacuum 299,792,458 m/s divided by 1.4682 = 204,190,477 m/s. You have to add to that the latency of any optical to electrical transformations, which happens in most every router or switch. Three days is probably an underestimate.
Uh, you do know that Voyager isn't unspooling fiber as it goes, right? -- Chris Adams <cma@cmadams.net>
Chris, Of course I do. -mel
On Jul 21, 2024, at 8:55 AM, Chris Adams <cma@cmadams.net> wrote:
Once upon a time, Mel Beckman <mel@beckman.org> said:
Because the speed of light is different in different mediums. It depends on the index of refraction. Most of the Internet is on fiber optics, and the speed of light in glass fiber is dramatically slower than in a vacuum. Long distance single-mode communication fiber typically has a core index of refraction of 1.4682 at 1550nm (mid-C-band). So the speed of light in this type of fiber is the speed of light in a vacuum 299,792,458 m/s divided by 1.4682 = 204,190,477 m/s. You have to add to that the latency of any optical to electrical transformations, which happens in most every router or switch. Three days is probably an underestimate.
Uh, you do know that Voyager isn't unspooling fiber as it goes, right?
-- Chris Adams <cma@cmadams.net>
Mel, Voyager is using radio waves, which travel faster than the speed of light (in a vacuum, too!). But my point is more Earth to outside the solar system is ~24 hours so where did circumnavigating the globe get three days of latency? On Sun, Jul 21, 2024 at 2:29 PM Mel Beckman <mel@beckman.org> wrote:
Chris,
Of course I do.
-mel
On Jul 21, 2024, at 8:55 AM, Chris Adams <cma@cmadams.net> wrote:
Because the speed of light is different in different mediums. It depends on the index of refraction. Most of the Internet is on fiber optics, and the speed of light in glass fiber is dramatically slower than in a vacuum. Long distance single-mode communication fiber typically has a core index of refraction of 1.4682 at 1550nm (mid-C-band). So the speed of
Once upon a time, Mel Beckman <mel@beckman.org> said: light in this type of fiber is the speed of light in a vacuum 299,792,458 m/s divided by 1.4682 = 204,190,477 m/s. You have to add to that the latency of any optical to electrical transformations, which happens in most every router or switch. Three days is probably an underestimate.
Uh, you do know that Voyager isn't unspooling fiber as it goes, right?
-- Chris Adams <cma@cmadams.net>
On 7/21/24 4:05 PM, Josh Luthman wrote:
Mel,
Voyager is using radio waves, which travel faster than the speed of light (in a vacuum, too!). But my point is more Earth to outside the solar system is ~24 hours so where did circumnavigating the globe get three days of latency?
::Albert Einstein has entered the chat:: Mike
On Sun, Jul 21, 2024 at 2:29 PM Mel Beckman <mel@beckman.org> wrote:
Chris,
Of course I do.
-mel
> On Jul 21, 2024, at 8:55 AM, Chris Adams <cma@cmadams.net> wrote: > > Once upon a time, Mel Beckman <mel@beckman.org> said: >> Because the speed of light is different in different mediums. It depends on the index of refraction. Most of the Internet is on fiber optics, and the speed of light in glass fiber is dramatically slower than in a vacuum. Long distance single-mode communication fiber typically has a core index of refraction of 1.4682 at 1550nm (mid-C-band). So the speed of light in this type of fiber is the speed of light in a vacuum 299,792,458 m/s divided by 1.4682 = 204,190,477 m/s. You have to add to that the latency of any optical to electrical transformations, which happens in most every router or switch. Three days is probably an underestimate. > > Uh, you do know that Voyager isn't unspooling fiber as it goes, right? > > -- > Chris Adams <cma@cmadams.net>
On Sun, 2024-07-21 at 16:10 -0700, Michael Thomas wrote:
On 7/21/24 4:05 PM, Josh Luthman wrote:
Mel,
Voyager is using radio waves, which travel faster than the speed of light (in a vacuum, too!). But my point is more Earth to outside the solar system is ~24 hours so where did circumnavigating the globe get three days of latency?
I'm the one who said 3 days. I was wrong. Can we go with 1.833 days RTT (22 hrs out, 22 hrs back)? Sorry folks.
Easy. Bridge loop. -mel via cell On Jul 21, 2024, at 4:06 PM, Josh Luthman <josh@imaginenetworksllc.com> wrote: Mel, Voyager is using radio waves, which travel faster than the speed of light (in a vacuum, too!). But my point is more Earth to outside the solar system is ~24 hours so where did circumnavigating the globe get three days of latency? On Sun, Jul 21, 2024 at 2:29 PM Mel Beckman <mel@beckman.org<mailto:mel@beckman.org>> wrote: Chris, Of course I do. -mel
On Jul 21, 2024, at 8:55 AM, Chris Adams <cma@cmadams.net<mailto:cma@cmadams.net>> wrote:
Once upon a time, Mel Beckman <mel@beckman.org<mailto:mel@beckman.org>> said:
Because the speed of light is different in different mediums. It depends on the index of refraction. Most of the Internet is on fiber optics, and the speed of light in glass fiber is dramatically slower than in a vacuum. Long distance single-mode communication fiber typically has a core index of refraction of 1.4682 at 1550nm (mid-C-band). So the speed of light in this type of fiber is the speed of light in a vacuum 299,792,458 m/s divided by 1.4682 = 204,190,477 m/s. You have to add to that the latency of any optical to electrical transformations, which happens in most every router or switch. Three days is probably an underestimate.
Uh, you do know that Voyager isn't unspooling fiber as it goes, right?
-- Chris Adams <cma@cmadams.net<mailto:cma@cmadams.net>>
Whoops, that should have said radio waves travel faster than fiber (more so in a vacuum). On Sun, Jul 21, 2024 at 8:07 PM Chris Adams <cma@cmadams.net> wrote:
Once upon a time, Josh Luthman <josh@imaginenetworksllc.com> said:
Voyager is using radio waves, which travel faster than the speed of light (in a vacuum, too!).
No... -- Chris Adams <cma@cmadams.net>
Voyager is using radio waves, which travel faster than the speed of
Well...it gets complicated :) https://www.youtube.com/watch?v=pTn6Ewhb27k On Sunday, 21/07/2024 at 20:15 Josh Luthman wrote: Whoops, that should have said radio waves travel faster than fiber (more so in a vacuum). On Sun, Jul 21, 2024 at 8:07 PM Chris Adams wrote: Once upon a time, Josh Luthman said: light
(in a vacuum, too!).
No... -- Chris Adams
For a more academic treatment: “The crucial problem of how to synchronize clocks and measure the one-way speed of light was originally discussed by Poincaré and Einstein. After being neglected for many decades, the Poincaré-Einstein problem of synchronization revived in 1977 with the work of Mansouri and Sexl, by which the one-way speed remains undetermined, allowing for unequal values of the speed of light in opposite directions.” <https://link.springer.com/article/10.1140/epjd/e2012-20524-8> [10053.png] On measuring the one-way speed of light - The European Physical Journal D<https://link.springer.com/article/10.1140/epjd/e2012-20524-8> link.springer.com<https://link.springer.com/article/10.1140/epjd/e2012-20524-8> So it’s completely reasonable to assume the speed of light is 1/2 c in one direction and infinite in the other. It’s just an optional convention that we consider the speed to be the same in both directions. Einstein said that light’s one-way speed “is in reality neither a supposition nor a hypothesis about the physical nature of light, but a stipulation which I can make of my own freewill in order to arrive at a definition of simultaneity.”* *A. Einstein, Relativity: The Special and General Theory, authorized translation by R. W. Lawson (New York: Crown Publishers, 1961), p 23. Perhaps an RFC should be written to address this :) -mel On Jul 21, 2024, at 6:38 PM, Scott Q. <qmail@top-consulting.net> wrote: Well...it gets complicated :) https://www.youtube.com/watch?v=pTn6Ewhb27k On Sunday, 21/07/2024 at 20:15 Josh Luthman wrote: Whoops, that should have said radio waves travel faster than fiber (more so in a vacuum). On Sun, Jul 21, 2024 at 8:07 PM Chris Adams <cma@cmadams.net<mailto:cma@cmadams.net>> wrote: Once upon a time, Josh Luthman <josh@imaginenetworksllc.com<mailto:josh@imaginenetworksllc.com>> said:
Voyager is using radio waves, which travel faster than the speed of light (in a vacuum, too!).
No... -- Chris Adams <cma@cmadams.net<mailto:cma@cmadams.net>>
OMG, Not trying to solve Einstein's General Theory of Relativity. Just trying to choose reasonable timeouts for my TCP packets :-) Middleware boxes suck -- its IETF week. On Mon, 22 Jul 2024, Mel Beckman wrote:
*A. Einstein, Relativity: The Special and General Theory, authorized translation by R. W. Lawson (New York: Crown Publishers, 1961), p 23.
Perhaps an RFC should be written to address this :)
Right, that's why I asked where the 3 days come from. I found an India website and I'm located in Ohio. That's pretty close to the opposite side of the world. I'm assuming it's a terrestrial service. My results are comparable to others in this thread, 200-280 ms on the higher end. If you start going GSO satellite or airplanes, that likely changes things. On Mon, Jul 22, 2024 at 5:05 PM Sean Donelan <sean@donelan.com> wrote:
OMG, Not trying to solve Einstein's General Theory of Relativity.
Just trying to choose reasonable timeouts for my TCP packets :-)
Middleware boxes suck -- its IETF week.
On Mon, 22 Jul 2024, Mel Beckman wrote:
*A. Einstein, Relativity: The Special and General Theory, authorized translation by R. W. Lawson (New York: Crown Publishers, 1961), p 23.
Perhaps an RFC should be written to address this :)
On Mon, 2024-07-22 at 17:57 -0400, Josh Luthman wrote:
Right, that's why I asked where the 3 days come from.
I found an India website and I'm located in Ohio. That's pretty close to the opposite side of the world. I'm assuming it's a terrestrial service. My results are comparable to others in this thread, 200-280 ms on the higher end.
To be serious, from my experience Comcast consumer internet in Monterey CA to Eritrea / Burkina Faso (which is pretty crazy to get to) was within that range.
Sean, Your just trying to choose reasonable timeouts for your TCP packets? Well, why didn't you say so? (Or perhaps I missed it). Fortunately, Einstein's General Theory of Relativity was solved long ago, so that's take care of. 🙂 If you're trying to choose reasonable timeouts, a wrong question to ask is "What is the current diameter of the Internet?" No matter what the diameter is, whether measured in RTT or TTL hops, a more useful question is, "How long am I willing to wait before considering a packet lost and retransmitting?" That time doesn't really matter so much on modern networks, so just choose a nice value you can live with, such as an hour (which happens to be the default). The timeout field itself can record values up to one day, so someone apparently thought that might be useful sometime (see RFC2549). Keep in mind that TCP rarely is configured to acknowledge each packet before sending the next. Instead it expects an acknowledgement every so-many packet. Consider the deminimus thought experiment. Your packet timeout is set to 2000 ms (two seconds). In the midst of an existing TCP session, you fire a slew of TCP packets – enough to fill your Send Window, and then wait for an ACK. Before you know it, 2000 ms has elapsed, and under TCP rules, you must consider those packets "lost". So, you retransmit them. But then, at 2035 ms, you receive the presumed missing ACK, which indicates all your packets were received at the far end after all. No harm done, because the far end will just treat the retransmissions as duplicates and throw them away. The amount of delay a packet experiences has a lot less to do with the distance, diameter, or latency, and a lot more to do with the processing going on along the way. For example, a router becomes congested, and hangs on to packets longer than usual. The congestion spreads along the path your packets are taking, so the delay adds up. In the end, unless you're using some really unusual technology (e.g. RFC2549), just set the timeout to the default 3600 seconds (1 hour). For a detailed explanation, refer to Richard Stevens TCP/IP Illustrated, Chapter 14 .TCP Timeout and Retransmission. -mel ________________________________ From: Sean Donelan <sean@donelan.com> Sent: Monday, July 22, 2024 2:05 PM To: Mel Beckman <mel@beckman.org> Cc: nanog@nanog.org <nanog@nanog.org> Subject: Re: Current diameter of the Internet? OMG, Not trying to solve Einstein's General Theory of Relativity. Just trying to choose reasonable timeouts for my TCP packets :-) Middleware boxes suck -- its IETF week. On Mon, 22 Jul 2024, Mel Beckman wrote:
*A. Einstein, Relativity: The Special and General Theory, authorized translation by R. W. Lawson (New York: Crown Publishers, 1961), p 23.
Perhaps an RFC should be written to address this :)
Somewhere I have a first edition of Steven's TCP/IP Illustrated, in some box of books I can't find. Strangely, "3600 seconds" is one timeout value which doesn't exist in that chapter (or TCP). Yep, ISPs and middleware boxes use that value, often incorrectly. On Mon, 22 Jul 2024, Mel Beckman wrote:
In the end, unless you're using some really unusual technology (e.g. RFC2549), just set the timeout to the default 3600 seconds (1 hour). For a detailed explanation, refer to Richard Stevens TCP/IP Illustrated, Chapter 14 .TCP Timeout and Retransmission.
On 7/19/24 8:44 PM, joel jaeggli wrote:
On 7/19/24 15:07, Sean Donelan wrote:
What is the current estimated diameter of the Internet?
Maximum (worst-case) RTT edge-to-edge?
Most public latency data is now edge-to-cloud, not edge-to-edge. Cloud engineers have done a great job, and edge-to-cloud less than 1-sec RTT.
Where have the long-slow pipes gone?
https://learn.microsoft.com/en-us/azure/networking/azure-network-latency?tab...
The RTT across geostationary satellites has remained stubbornly constant we just don't use them unless there's no other alternative. A colleague sent me a trace the other day while flying from hawaii to los vegas that was otherwise performant but still incurs the overhead.
Geosync is probably the highest unloaded latency, but add in a little bufferbloat and you can easily get way more latency...I've definitely seen a minute+ pings on a plane, which is almost all queuing. I assume airplanes are not the only place with such horrendous queuing, even if most networks have done a good job at reducing that nonsense over the last decade. Matt
On 7/20/24 14:08, Matt Corallo wrote:
Geosync is probably the highest unloaded latency, but add in a little bufferbloat and you can easily get way more latency...I've definitely seen a minute+ pings on a plane, which is almost all queuing. I assume airplanes are not the only place with such horrendous queuing, even if most networks have done a good job at reducing that nonsense over the last decade.
I think if you eliminate packet loss and intermittency as a source of increased latency, geostationary satellite is probably as bad as it gets from a transport medium that is stable but just physically limited. 600ms is just about what you would see here. I have seen subsea deliver as high as 450ms, but this purely a function of distance traveled, e.g., Cape Town to Auckland via Hong Kong, for example. Mark.
If worst-case is an option, there are some interesting routing policies between certain places. One example is a Australia to China--take Perth to Chongqing as an example. They're at about the same longitude, but RTT is routinely greater than 500 ms. Packets travel to Singapore, then cross the entire Pacific ocean to the west coast of the US, and back, before making it back to China. Try it for yourself from this Perth-based looking glass[1]. PING 117.151.152.239 (117.151.152.239) 56(84) bytes of data. 64 bytes from 117.151.152.239: icmp_seq=1 ttl=236 time=511 ms 64 bytes from 117.151.152.239: icmp_seq=2 ttl=236 time=522 ms traceroute to 117.151.152.239 (117.151.152.239), 30 hops max, 60 byte packets 1 172.18.0.1 (172.18.0.1) 0.021 ms 0.004 ms 0.004 ms 2 45.248.78.65 (45.248.78.65) 0.611 ms 0.696 ms 0.777 ms 3 45.248.78.139 (45.248.78.139) 0.678 ms 0.818 ms 0.959 ms 4 core.p1.wa.hostuniversal.com.au (103.216.222.7) 0.145 ms 0.165 ms 0.195 ms 5 be6745.201.ccr51.per01.atlas.cogentco.com (154.18.100.65) 0.468 ms 0.418 ms 0.496 ms 6 be2428.ccr31.sin01.atlas.cogentco.com (154.54.88.138) 46.625 ms 46.659 ms 46.691 ms 7 be2913.ccr41.lax04.atlas.cogentco.com (154.54.27.54) 218.213 ms 237.901 ms 237.959 ms [...] 1: https://perth-lg.ransomit.com.au/ Sean Donelan <sean@donelan.com> writes:
What is the current estimated diameter of the Internet?
Maximum (worst-case) RTT edge-to-edge?
Most public latency data is now edge-to-cloud, not edge-to-edge. Cloud engineers have done a great job, and edge-to-cloud less than 1-sec RTT.
Where have the long-slow pipes gone?
https://learn.microsoft.com/en-us/azure/networking/azure-network-latency?tab...
Keeping this within the realm of TCP/IP, i.e. within Earth terresterial links and the sphere of Earth geostationary orbits (maybe Lagrange points, I don't know what communication protocols far-earth satellites use). I'm not including inter-planetary or inter-stellar communication protocols. What is the actual, real-world worst case edge-to-edge, network RTT seen in the current Internet? What latency numbers are ISPs putting on their FCC Broadband Labels? https://www.fcc.gov/broadbandlabels I remember in the "old days" some pacific islands with triple geostationary satellite hops. On Fri, 19 Jul 2024, Sean Donelan wrote:
What is the current estimated diameter of the Internet?
Maximum (worst-case) RTT edge-to-edge?
Most public latency data is now edge-to-cloud, not edge-to-edge. Cloud engineers have done a great job, and edge-to-cloud less than 1-sec RTT.
Where have the long-slow pipes gone?
https://learn.microsoft.com/en-us/azure/networking/azure-network-latency?tab...
According to Cloudflare <https://radar.cloudflare.com/quality/> Latency under average utilization by continent North America 41 ms South America 46 ms Europe 34 ms Africa 108 ms Asia 84 ms Oceania 35 ms Not clear what is included in Cloudflare "continents." Oceania is listed with the 2nd highest bandwidth (avg 26 Mbps), and lowest latency (avg 35 ms).
Hmm, made me curious,... What is the bisection bandwidth of the Internet today? -- //Shrikumar ---Original Message---
From: Sean Donelan <sean@donelan.com> Date: Mon, 22 Jul 2024 11:46:00 -0400 (EDT) To: nanog@nanog.org Subject: Re: Current diameter of the Internet?
According to Cloudflare <https://radar.cloudflare.com/quality/>
Latency under average utilization by continent
North America 41 ms South America 46 ms Europe 34 ms Africa 108 ms Asia 84 ms Oceania 35 ms
Not clear what is included in Cloudflare "continents." Oceania is listed with the 2nd highest bandwidth (avg 26 Mbps), and lowest latency (avg 35 ms).
According to speedtest.net (Akamai has apparently stop publishing its data) Fixed Broadband (Global median): 93 Mbps (download) / 47 Mbps (upload) Mobile Broadband (Global median): 56 Mbps (download) / 11 Mbps (upload) By Country (fixed broadband) 1. Singapore 285 Mbps .... 159. Cuba 2.7 Mbps According to Cloudflare (download speed under average utilization) Europe 28 Mbps Oceania 26 Mbps North America 21 Mbps South America 19 Mbps Asia 12 Mbps Africa 7 Mbps On Mon, 22 Jul 2024, Shrikumar.H wrote:
Hmm, made me curious,...
What is the bisection bandwidth of the Internet today?
participants (14)
-
Aaron Groom -
Chris Adams -
Donald Eastlake -
joel jaeggli -
Josh Luthman -
Mark Tinka -
Matt Corallo -
Mel Beckman -
Michael Thomas -
Nathan Angelacos -
Scott Q. -
Sean Donelan -
Shrikumar.H -
Stas Bilder