FYI, We are in the process of starting a new Working Group at IETF, Timer Variant Routing or TVR. https://datatracker.ietf.org/group/tvr/about/ Some of the uses cases are for space applications where you can predict or schedule the availability and capacity of "links" (radio, optical) This gets sort of merged with DTN (Delay/Disruption Tolerant Networking.) NASA GRC has developed a High Speed version of DTN aka HDTN that is being tested in terrestrial setups but soon to be tested in space. https://www1.grc.nasa.gov/space/scan/acs/tech-studies/dtn/ For now all this is experimental. Plus there are several commercial entities also working in this realm, one is https://www.aalyria.com/, spin-off of Google's Loon project. -J On Sun, Jan 22, 2023 at 5:17 PM Michael Thomas <mike@mtcc.com> wrote:

On 1/22/23 3:05 PM, Matthew Petach wrote:

On Sun, Jan 22, 2023 at 2:45 PM Michael Thomas <mike@mtcc.com> wrote:

...

I read in the Economist that the gen of starlink satellites will have the ability to route messages between each satellite. Would conventional routing protocols be up to such a challenge? Or would it have to be custom made for that problem? And since a lot of companies and countries are getting on that action, it seems like fertile ground for (bad) wheel reinvention?

Mike

Unlike most terrestrial links, the distances between satellites are not fixed, and thus the latency between nodes is variable, making the concept of "Shortest Path First" calculation a much more dynamic and challenging one to keep current, as the latency along a path may be constantly changing as the satellite nodes move relative to each other, without any link state actually changing to trigger a new SPF calculation.

One thing that is in their favor is that while they are moving, they are moving in a predictable manner. It seems that each router could, essentially, locally update routes until they are told otherwise?

I suspect a form of OLSR might be more advantageous in a dynamic partial mesh between satellites, but I haven't given it as much deep thought as would be necessary to form an informed opinion.

So, yes--it's likely the routing protocol used will not be entirely "off-the-shelf" but will instead incorporate continuous latency information in the LSDB, and path selection will be time-bound based on the rate of increase in latency along currently-selected edges in the graph.

Has IETF looked at this, do you know? Even if the routers can't interoperate with other systems, it would be good to have some routing clue with a lot of eyeballs on it to not make rookie mistakes. Mike

...

This gets sort of merged with DTN (Delay/Disruption Tolerant Networking.)

I have been saying that DTN is a reinvention of UUNET.

Hmmm, nope not even close.

As such, it should be noted that, in UUNET, availability of phone links between computers was scheduled.

You must be talking about UCCP, UUNET was a company. Availability of links was declared not scheduled, so pathalias was able to figure the best UUCP path from a given UUCP node. -J

You, seemingly, do not have much knowledge on UUNET.

Of course I don't :-)

#N atina #S Everex 386 Step 33; SCO Xenix System V 2.3.3 #O Ministerio de Relaciones Exteriores y Culto #C Jorge Marcelo Amodio #E atina!postmaster #T +54 1 315 4804, Fax: +54 1 315 4824 #P Reconquista 1088, Buenos Aires (1003) - ARGENTINA #L 34 34 S / 58 03 W city #R #W atina!pete ( Jorge M. Amodio ); Fri Mar 16 14:43:03 ARG 1990 atina .ar atina .mrec.ar atina = atina.ar, atina.mrec.ar atina agomar(DAILY), antar(DAILY), biotlp(DAILY), cab(HOURLY), cedro(EVENING), cenep(DAILY), cneaint(DAILY), cnea(EVENING), cnielf(DAILY), colimpo(DAILY), confein(DAILY), criba(EVENING), curbre(EVENING), dacfyb(DEMAND), dcfcen(DEMAND), ecord(DEMAND), enace(DAILY), epfrn(EVENING), fb1(DAILY), fcys(DAILY), fecic(DAILY), gagcha(EVENING), getinfo(DAILY), hasar(DAILY), iaros(DAILY), intiar(DAILY), invapba(DAILY/2), invapqq(DAILY/2), isoft(DAILY), itcgi(DAILY), labdig(DAILY), lasbe(DAILY), licmdp(EVENING), lis(EVENING), ludo(DAILY), maap(DAILY), meyosp(DAILY), minerva(DAILY), minjus(DAILY), mlearn(DAILY), occam(EVENING), oceanar(DAILY), onba(DAILY), opsarg(DEMAND), pnud009(EVENING), sadio(DAILY), saravia(DAILY), sdinam(DAILY), secyt(DEMAND), spok(DAILY), sykes(DAILY), tandil(DAILY), tsgfred(WEEKLY), ulatar(EVENING), unisel(EVENING), uunet(DEMAND) -J

On 1/22/23 16:05, Matthew Petach wrote:

On Sun, Jan 22, 2023 at 2:45 PM Michael Thomas <mike@mtcc.com> wrote:

I read in the Economist that the gen of starlink satellites will have the ability to route messages between each satellite. Would conventional routing protocols be up to such a challenge? Or would it have to be custom made for that problem? And since a lot of companies and countries are getting on that action, it seems like fertile ground for (bad) wheel reinvention?

Unlike most terrestrial links, the distances between satellites are not fixed, and thus the latency between nodes is variable, making the concept of "Shortest Path First" calculation a much more dynamic and challenging one to keep current, as the latency along a path may be constantly changing as the satellite nodes move relative to each other, without any link state actually changing to trigger a new SPF calculation.

I suspect a form of OLSR might be more advantageous in a dynamic partial mesh between satellites, but I haven't given it as much deep thought as would be necessary to form an informed opinion.

So, yes--it's likely the routing protocol used will not be entirely "off-the-shelf" but will instead incorporate continuous latency information in the LSDB, and path selection will be time-bound based on the rate of increase in latency along currently-selected edges in the graph.

Satellites move constantly relative to each other and to ground stations. There is a database available which contains the parameters for calculating a satellite's location at any instant in time. To maintain minimal link disruption, the idea is to calculate these relative relationships, and using some graph and network flow algorithms, you pre-calculate the links and then insert/remove those links and routes into the routing information base at the appropriate times. Then based upon latency, signal quality, and link availability, routing information is inserted/deleted into the forwarding information base. There are other contributors such as link saturation and overall end-to-end delays which could be applied based upon ground station state management. It becomes a multi-parameter link selection algorithm in a dynamic environment. Pretty much an interesting 'sdn' like scenario.

Yes re: Iridium. Contrary to what the Chief Huckster may say, inter-sat comms are not some revolutionary thing that he invented. It’s also not likely to function anything like they show in marketing promos, with data magically zipping around the constellation between nodes in different inclinations. Unless they have managed to solve for the Doppler effect in a way nobody has thought of yet. On Sun, Jan 22, 2023 at 18:25 Crist Clark <cjc+nanog@pumpky.net> wrote:

I suspect, although I have no references, that satellite to ground connectivity is probably more “circuit-based” than per-packet or frame.

Iridium has done inter satellite communication for decades. I wonder if it wouldn’t be something very similar. Although it would be totally on-brand for them to do it some “revolutionary” new way whether it actually makes any sense or not.

On Sun, Jan 22, 2023 at 3:06 PM Matthew Petach <mpetach@netflight.com> wrote:

...

On Sun, Jan 22, 2023 at 2:45 PM Michael Thomas <mike@mtcc.com> wrote:

...

I read in the Economist that the gen of starlink satellites will have the ability to route messages between each satellite. Would conventional routing protocols be up to such a challenge? Or would it have to be custom made for that problem? And since a lot of companies and countries are getting on that action, it seems like fertile ground for (bad) wheel reinvention?

Mike

Unlike most terrestrial links, the distances between satellites are not fixed, and thus the latency between nodes is variable, making the concept of "Shortest Path First" calculation a much more dynamic and challenging one to keep current, as the latency along a path may be constantly changing as the satellite nodes move relative to each other, without any link state actually changing to trigger a new SPF calculation.

I suspect a form of OLSR might be more advantageous in a dynamic partial mesh between satellites, but I haven't given it as much deep thought as would be necessary to form an informed opinion.

So, yes--it's likely the routing protocol used will not be entirely "off-the-shelf" but will instead incorporate continuous latency information in the LSDB, and path selection will be time-bound based on the rate of increase in latency along currently-selected edges in the graph.

An interesting problem to dive into, certainly. :)

Thanks!

Matt

Solved years ago … https://ieeexplore.ieee.org/ielaam/92/8502886/8412572-aam.pdf -Jorge

On Jan 23, 2023, at 1:30 AM, Raymond Burkholder <ray@oneunified.net> wrote:



...

On 1/22/23 21:54, Tom Beecher wrote: Yes re: Iridium. Contrary to what the Chief Huckster may say, inter-sat comms are not some revolutionary thing that he invented. It’s also not likely to function anything like they show in marketing promos, with data magically zipping around the constellation between nodes in different inclinations. Unless they have managed to solve for the Doppler effect in a way nobody has thought of yet.

I know of a group of satellite FPGA/RF guys who have worked on this doppler thingy. It is a solved problem.

On 2023-01-23 17:27, Tom Beecher wrote:

What I didn't think was adequately solved was what Starlink shows in marketing snippets, that is birds in completely different orbital inclinations (sometimes close to 90 degrees off) shooting messages to each other. Last I had read the dopplar effects there were so much larger due to relative speed deltas it just couldn't currently be done. If there is more out there on that solution, be glad to read up on what info anyone may have on that if they can share.

Worst case would be if the satellites are moving directly towards or directly away from each other. Each satellite will be moving at a speed of slighly under 8 km/s, and they will thus approach or depart from each other with a relative speed of somewhat less than 16 km/s. I get that for 1310 nm light, the doppler shift would be just under 0.07 nm, or 12.2 GHz: l0 = 1310 nm f0 = c / l0 f = f0 / sqrt((1 + 16 km/s / c) / (1 - 16 km/s / c)) l = c / f ≈ 1310.0699 nm f0 - f ≈ 12.2 GHz In the ITU C band, I get the doppler shift to be about 10.5 GHz (at channel 72, 197200 GHz or 1520.25 nm). (Formula from https://en.wikipedia.org/wiki/Relativistic_Doppler_effect first entry in the table under "Summary of major results".) These shifts are noticably less than typical grid widths used for DWDM (±50 GHz for the standard spacing), so it seems unlikely to me that the doppler shift would be a problem. /Bellman

Appreciate that. Definitely becoming clear to me that a lot of my knowledge here was rusty. Lots of papers on this specifically (Doppler effects on optical ISL) that I need to call in some favors to get access to. Thanks! On Mon, Jan 23, 2023 at 1:08 PM Thomas Bellman <bellman@nsc.liu.se> wrote:

On 2023-01-23 17:27, Tom Beecher wrote:

...

What I didn't think was adequately solved was what Starlink shows in marketing snippets, that is birds in completely different orbital inclinations (sometimes close to 90 degrees off) shooting messages to each other. Last I had read the dopplar effects there were so much larger due to relative speed deltas it just couldn't currently be done. If there is more out there on that solution, be glad to read up on what info anyone may have on that if they can share.

Worst case would be if the satellites are moving directly towards or directly away from each other. Each satellite will be moving at a speed of slighly under 8 km/s, and they will thus approach or depart from each other with a relative speed of somewhat less than 16 km/s.

I get that for 1310 nm light, the doppler shift would be just under 0.07 nm, or 12.2 GHz:

l0 = 1310 nm f0 = c / l0 f = f0 / sqrt((1 + 16 km/s / c) / (1 - 16 km/s / c)) l = c / f ≈ 1310.0699 nm f0 - f ≈ 12.2 GHz

In the ITU C band, I get the doppler shift to be about 10.5 GHz (at channel 72, 197200 GHz or 1520.25 nm).

(Formula from https://en.wikipedia.org/wiki/Relativistic_Doppler_effect first entry in the table under "Summary of major results".)

These shifts are noticably less than typical grid widths used for DWDM (±50 GHz for the standard spacing), so it seems unlikely to me that the doppler shift would be a problem.

/Bellman

My present understanding is that starlink satellites with lasers are not designed to communicate inter-plane. Each launch of starlink satellites is put into exactly the same orbital inclination (53.2 degrees or the more rare near polar orbits now launched from Vandenberg). In the weeks and months following their launch they spread out into an extended line all following each other in the same plane. Plane change maneuvers are extremely expensive in delta-v for any satellite and are generally avoided unless absolutely necessary. Best conjecture is that starlink satellites' on board propellant for hall effect or ion thrusters (or whatever they're using that has an ISP above 3000) is used almost exclusively for thrusting prograde to maintain altitude. If you view a launch of 45 or 50 starlink satellites in a live animated satellite tracking application, based on their TLE orbital data, they all follow each other in a line. Satellites in the same line may be using inter-satellite lasers to speak to the unit immediately in front of it, and immediately behind it, forming a conga-line like network of linked satellites until they get to one that is generally above a starlink earth station/terrestrial network facility. At which point the traffic is transferred. Starlink has recently made service available for purchase in Nunavut and all of the other high-latitude areas of northern Canada, which means that they clearly think they have sufficient (82 degree plus) inclination sets of satellites *and* inter-satellite links working to provide service in an area that definitely has no terrestrial fiber or starlink earth stations. On Mon, Jan 23, 2023 at 11:29 AM Thomas Bellman <bellman@nsc.liu.se> wrote:

On 2023-01-23 19:08, I wrote:

...

I get that for 1310 nm light, the doppler shift would be just under 0.07 nm, or 12.2 GHz: [...] In the ITU C band, I get the doppler shift to be about 10.5 GHz (at channel 72, 197200 GHz or 1520.25 nm). [...] These shifts are noticably less than typical grid widths used for DWDM (±50 GHz for the standard spacing), so it seems unlikely to me that the doppler shift would be a problem.

And as I was bicycling home, I of course thought of another aspect of the doppler shift: the timing between the symbols in the signal, or in other words the baud rate. There will be something like a phase-locked loop (PLL) in the receiver in order to know when one symbol ends and the next one starts, and that PLL can only deal with a certain amount of baud rate shift.

But we can use the same formula. And in general, the doppler shift for 16 km/s is about 53 parts per million. So e.g. a 112 Gbaud signal would be received as 6 Mbaud faster or slower than it was sent at. And here I have to confess that I don't know how generous typical receiver PLL:s in network equipment are.

Another potential aspect might be the decoding of phase-shift keying, i.e. when phase modulation is used for the signal. My *very*vague* understanding is that the typical way to decode a phase-modulated signal, is to mix the incoming signal with a reference carrier wave, generated locally by the receiver, and the interference between the two gives you the actual signal. But to do that, the reference must have the same frequency as the received wave, and, I guess, must match very closely. Can they adapt to an incoming wave that is 53 ppm offset from what it should be?

Or have I misunderstood this? Analogue signals is very much *NOT* my forte...

/Bellman

Musk didn't do anything revolutionary, besides launching a shload of LEO satellites. NASA and DoD have been working for long time on optical space communications, last year LCRD was launched and preliminary tests using it as a relay showed 622Mpbs, this year NASA will include on one of the cargo missions to ISS ILLUMA-T that will be installed at ISS and it is expected to provide 1.24Gpbs or more using LCRD as a relay with the two ground stations, one in HI, and one in CA. DoD/NRO have been working on this for some time now, but any information is in the top secret blackhole. -J -Jorge On Jan 23, 2023, at 1:54 AM, Forrest Christian (List Account) < lists@packetflux.com> wrote:  I think the thing they're calling revolutionary is the idea of those links being directional lasers. It makes some sense... if you can basically emit the same signal you'd shoot down a strand of single mode but aim it through the mostly vacuum of space in the exact direction of your neighbor then you've got something... Essentially the equivalent of a fiber optic network in space. For fun I tried plugging in some frequencies of light into a doppler calculator. Unfortunately that's where my "would the relative speed that mere mortals could attain make enough of a difference to affect a typical optical receiver" investigation ended as I'm mobile right now and can't do the rest of the work very easily. I'd be curious if the relative speed would be enough to cause enough shift to move it out of the pass band if a typical dwdm channel. And, I agree that little of what musk takes credit for is revolutionary. But what I do think he deserves credit for is being insane enough to try things everyone says is unworkable and failed in the past and somehow making at least some of them work. Having more money than God helps too. On Sun, Jan 22, 2023, 8:55 PM Tom Beecher <beecher@beecher.cc> wrote:

Yes re: Iridium. Contrary to what the Chief Huckster may say, inter-sat comms are not some revolutionary thing that he invented.

It’s also not likely to function anything like they show in marketing promos, with data magically zipping around the constellation between nodes in different inclinations. Unless they have managed to solve for the Doppler effect in a way nobody has thought of yet.

On Sun, Jan 22, 2023 at 18:25 Crist Clark <cjc+nanog@pumpky.net> wrote:

...

I suspect, although I have no references, that satellite to ground connectivity is probably more “circuit-based” than per-packet or frame.

Iridium has done inter satellite communication for decades. I wonder if it wouldn’t be something very similar. Although it would be totally on-brand for them to do it some “revolutionary” new way whether it actually makes any sense or not.

On Sun, Jan 22, 2023 at 3:06 PM Matthew Petach <mpetach@netflight.com> wrote:

...

On Sun, Jan 22, 2023 at 2:45 PM Michael Thomas <mike@mtcc.com> wrote:

...

I read in the Economist that the gen of starlink satellites will have the ability to route messages between each satellite. Would conventional routing protocols be up to such a challenge? Or would it have to be custom made for that problem? And since a lot of companies and countries are getting on that action, it seems like fertile ground for (bad) wheel reinvention?

Mike

Unlike most terrestrial links, the distances between satellites are not fixed, and thus the latency between nodes is variable, making the concept of "Shortest Path First" calculation a much more dynamic and challenging one to keep current, as the latency along a path may be constantly changing as the satellite nodes move relative to each other, without any link state actually changing to trigger a new SPF calculation.

I suspect a form of OLSR might be more advantageous in a dynamic partial mesh between satellites, but I haven't given it as much deep thought as would be necessary to form an informed opinion.

So, yes--it's likely the routing protocol used will not be entirely "off-the-shelf" but will instead incorporate continuous latency information in the LSDB, and path selection will be time-bound based on the rate of increase in latency along currently-selected edges in the graph.

An interesting problem to dive into, certainly. :)

Thanks!

Matt

Elon for whatever reason is insane enough to dump a lot of cash in industries which everyone said was a dead end and then has been lucky enough to prove the old guard wrong.

- Nobody had 'given up' on reusable launch vehicles. SpaceX (to their credit) just made it a core requirement in Falcon9 design from the outset, and was able to execute it. - Nobody had 'given up' on electric cars before Musk pushed the original founders of Tesla out. - Musk took Solarcity in the opposite direction (down) as the rest of the US solar industry grew. - Starlink still hasn't proven any of the 'old guard wrong'. Is Starlink operational? Yes. Has he proven it to be a viable business? No. (In fact, if you basic math on the numbers they espouse, it can't be.) Same for pretty much everything musk does, including starlink. So if

there is anything at all "revolutionary" here it's the insistence on ignoring conventional wisdom. I think it might be borderline insanity, but it seems to work for him.

It 'seems to work for him' because : 1. He is a showman, and good at it. 2. When something is delivered, it's only because of him. When something isn't, it's always because of someone/something else. On Mon, Jan 23, 2023 at 11:02 AM Forrest Christian (List Account) < lists@packetflux.com> wrote:

Like I said, they're calling it revolutionary. Didn't say it was.

However the idea that you can build spaceships which are fully reusable was certainly around the industry, but the consensus was largely "we tried, it costs too much, so we're sticking with one use rockets". Elon for whatever reason is insane enough to dump a lot of cash in industries which everyone said was a dead end and then has been lucky enough to prove the old guard wrong.

Same for pretty much everything musk does, including starlink. So if there is anything at all "revolutionary" here it's the insistence on ignoring conventional wisdom. I think it might be borderline insanity, but it seems to work for him.

On Mon, Jan 23, 2023, 3:46 AM Jorge Amodio <jmamodio@gmail.com> wrote:

...

Musk didn't do anything revolutionary, besides launching a shload of LEO satellites.

NASA and DoD have been working for long time on optical space communications, last year LCRD was launched and preliminary tests using it as a relay showed 622Mpbs, this year NASA will include on one of the cargo missions to ISS ILLUMA-T that will be installed at ISS and it is expected to provide 1.24Gpbs or more using LCRD as a relay with the two ground stations, one in HI, and one in CA.

DoD/NRO have been working on this for some time now, but any information is in the top secret blackhole.

-J

-Jorge

On Jan 23, 2023, at 1:54 AM, Forrest Christian (List Account) < lists@packetflux.com> wrote:

 I think the thing they're calling revolutionary is the idea of those links being directional lasers.

It makes some sense... if you can basically emit the same signal you'd shoot down a strand of single mode but aim it through the mostly vacuum of space in the exact direction of your neighbor then you've got something... Essentially the equivalent of a fiber optic network in space.

For fun I tried plugging in some frequencies of light into a doppler calculator. Unfortunately that's where my "would the relative speed that mere mortals could attain make enough of a difference to affect a typical optical receiver" investigation ended as I'm mobile right now and can't do the rest of the work very easily. I'd be curious if the relative speed would be enough to cause enough shift to move it out of the pass band if a typical dwdm channel.

And, I agree that little of what musk takes credit for is revolutionary. But what I do think he deserves credit for is being insane enough to try things everyone says is unworkable and failed in the past and somehow making at least some of them work. Having more money than God helps too.

On Sun, Jan 22, 2023, 8:55 PM Tom Beecher <beecher@beecher.cc> wrote:

...

Yes re: Iridium. Contrary to what the Chief Huckster may say, inter-sat comms are not some revolutionary thing that he invented.

It’s also not likely to function anything like they show in marketing promos, with data magically zipping around the constellation between nodes in different inclinations. Unless they have managed to solve for the Doppler effect in a way nobody has thought of yet.

On Sun, Jan 22, 2023 at 18:25 Crist Clark <cjc+nanog@pumpky.net> wrote:

...

I suspect, although I have no references, that satellite to ground connectivity is probably more “circuit-based” than per-packet or frame.

Iridium has done inter satellite communication for decades. I wonder if it wouldn’t be something very similar. Although it would be totally on-brand for them to do it some “revolutionary” new way whether it actually makes any sense or not.

On Sun, Jan 22, 2023 at 3:06 PM Matthew Petach <mpetach@netflight.com> wrote:

...

On Sun, Jan 22, 2023 at 2:45 PM Michael Thomas <mike@mtcc.com> wrote:

...

I read in the Economist that the gen of starlink satellites will have the ability to route messages between each satellite. Would conventional routing protocols be up to such a challenge? Or would it have to be custom made for that problem? And since a lot of companies and countries are getting on that action, it seems like fertile ground for (bad) wheel reinvention?

Mike

Unlike most terrestrial links, the distances between satellites are not fixed, and thus the latency between nodes is variable, making the concept of "Shortest Path First" calculation a much more dynamic and challenging one to keep current, as the latency along a path may be constantly changing as the satellite nodes move relative to each other, without any link state actually changing to trigger a new SPF calculation.

I suspect a form of OLSR might be more advantageous in a dynamic partial mesh between satellites, but I haven't given it as much deep thought as would be necessary to form an informed opinion.

So, yes--it's likely the routing protocol used will not be entirely "off-the-shelf" but will instead incorporate continuous latency information in the LSDB, and path selection will be time-bound based on the rate of increase in latency along currently-selected edges in the graph.

An interesting problem to dive into, certainly. :)

Thanks!

Matt

Matthew Petach wrote:

Unlike most terrestrial links, the distances between satellites are not fixed, and thus the latency between nodes is variable, making the concept of "Shortest Path First" calculation a much more dynamic and challenging one to keep current, as the latency along a path may be constantly changing as the satellite nodes move relative to each other, without any link state actually changing to trigger a new SPF calculation.

As LEO satellites should be leafs to a network of MEO satellites, 1 minutes of update period between MEO satellites should be enough, which is not so dynamic. Physical layer of MEO communications must (to save power and to prevent broadcast storms) be point to point with known orbital elements and link layer should be some point to point protocol perhaps with ARQ. As only meaningful metric between satellites is physical distance, 16bit metric of OSPF should be enough. The most annoying part is to have multiple ground stations, which, as usual, makes the MEO network DFZ with more than 1M routing table entries. Masataka Ohta

For the people who have seen their US48 state earth station setups in person it is pretty normal on the network level. Being colocated with major inter-city long haul dark fiber DWDM regen sites (Level3 dark fiber path Seattle to Boise, ID which has a regen hut site in Prosser, WA is a perfect example) gives them the ability to buy a transport circuit to the nearest major city/IX point and haul traffic there. I believe they're buying single 100 Gbps waves. On Mon, Jan 23, 2023 at 2:18 PM Chris J. Ruschmann <chris@scsalaska.net> wrote:

Don’t quote me on this, but I wouldn’t say they are doing anything different than you or I can do and have access to on the routing layer. It's probably just Nokia and Arista and whatever those systems provide. Stuff like Tunneling, ECMP, BFD and VxLan... Think spatially coordinated Zerotier and not based on latency. They also have a pretty good team of experts that have experience with large scale networking and automation they've plucked from various places.

How the Satellites talk to the end users is where all the magic is. But my understanding is that it's all custom developed networking as code that handles all the frequency coordination and hand offs with the ground.

-----Original Message----- From: NANOG <nanog-bounces+chris=scsalaska.net@nanog.org> On Behalf Of Michael Thomas Sent: Sunday, January 22, 2023 1:43 PM To: nanog@nanog.org Subject: Starlink routing

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

I read in the Economist that the gen of starlink satellites will have the ability to route messages between each satellite. Would conventional routing protocols be up to such a challenge? Or would it have to be custom made for that problem? And since a lot of companies and countries are getting on that action, it seems like fertile ground for (bad) wheel reinvention?

Mike

The original and traditional high-cost way of how this is done for MEO/LEO is exemplified by an o3b terminal, which has two active motorized tracking antennas. The antenna presently in use for the satellite that is overhead follows it until it's descending towards the horizon, while at the same time the second antenna aims itself at where the next 'rising' satellite is predicted to appear at the opposite horizon, and forms a link to it. Make-before-break. If anyone has seen photographs in their marketing material/videos of the Oneweb beta test earth stations in Alaska they are operating using the same general concept. Oneweb has clearly positioned their market focus for telecoms and ISPs and large enterprise end users, because their CPE equipment is considerably larger, expensive and more power hungry. The beta test sites I've seen installed on top of a telecom equipment shelter occupy an area approximately 8 feet long x 4 feet wide including radomes and mounting. On Mon, Jan 23, 2023 at 2:49 PM Kevin McCormick <kmccormick@mdtc.net> wrote:

My original thought was this would be more like Client Optimized Roaming with WiFi access points.

Communication between the client dish or base station and satellites to transparently move client dish and base station from satellites moving out of view to a satellite in view.

Kevin McCormick

-----Original Message----- From: NANOG <nanog-bounces+kmccormick=mdtc.net@nanog.org> On Behalf Of Michael Thomas Sent: Sunday, January 22, 2023 4:43 PM To: nanog@nanog.org Subject: Starlink routing

CAUTION: This email originated from outside your organization. Exercise caution when opening attachments or clicking links, especially from unknown senders.

I read in the Economist that the gen of starlink satellites will have the ability to route messages between each satellite. Would conventional routing protocols be up to such a challenge? Or would it have to be custom made for that problem? And since a lot of companies and countries are getting on that action, it seems like fertile ground for (bad) wheel reinvention?

Mike

I think it's useful to clarify terminology - the starlink antenna unit itself is the CPE. With my v1 starlink terminal you can plug literally anything into the PoE injector that is a 1500 MTU 1000BaseT DHCP client and it'll get an address and a default route out to the internet. All of the smarts happen in the antenna unit/phased array unit which also has its own fairly capable embedded CPU/RAM and routing capability. The starlink *indoor* CPE, the home wifi router itself ,is a very basic thing that looks like something derived from a Taiwan ODM 802.11ac home router OpenWRT reference design with a custom firmware load.Or similar. If you've seen a teardown of one they're very simple. With the v2 rectangular terminals it's similar but you need a cable adapter to go from the proprietary starlink cable to indoor unit, and additionally the indoor CPE unit also serves as the PoE injector. In some other ISP type environments you might be expecting the indoor unit to be the CPE, such as what you'd get with a Comcast DOCSIS3.0 all-in-one modem+coax interface+router+wifi device attached to some coax coming in through a wall. On Mon, Jan 23, 2023 at 3:36 PM Michael Thomas <mike@mtcc.com> wrote:

On 1/23/23 3:14 PM, Eric Kuhnke wrote:

...

The original and traditional high-cost way of how this is done for MEO/LEO is exemplified by an o3b terminal, which has two active motorized tracking antennas. The antenna presently in use for the satellite that is overhead follows it until it's descending towards the horizon, while at the same time the second antenna aims itself at where the next 'rising' satellite is predicted to appear at the opposite horizon, and forms a link to it. Make-before-break. If anyone has seen photographs in their marketing material/videos of the Oneweb beta test earth stations in Alaska they are operating using the same general concept.

Oneweb has clearly positioned their market focus for telecoms and ISPs and large enterprise end users, because their CPE equipment is considerably larger, expensive and more power hungry. The beta test sites I've seen installed on top of a telecom equipment shelter occupy an area approximately 8 feet long x 4 feet wide including radomes and mounting.

I'm trying to understand this so sorry if this comes off dumb. So does the base station mediate all handoffs where the CPE is told when/what to handoff? Or does the CPE have some part in it (other than receiving the handoff)? Does the CPE accept control traffic (L2?) from any bird? Are there cases where the CPE needs to de-dup packets due to handoffs?

This is pretty fascinating stuff.

Mike