On Jan 23, 2020, at 10:16 AM, Kaiser, Erich <erich@gotfusion.net> wrote:

Yeah we saw that as well. Must be a game release or something.

Yes, that’s my understanding as well. - Jared

Just found the size of the updates, 48 GB on PC, 13 GB on PS4, and 18 GB on Xbox One. Makes sense why the college residence halls we supply internet to were pulling so much data. All those systems if left on would pull a lot of data. Thank you, Kevin McCormick From: NANOG <nanog-bounces@nanog.org> On Behalf Of Todd Baumgartner Sent: Thursday, January 23, 2020 9:30 AM To: Jared Mauch <jared@puck.nether.net> Cc: NANOG list <Nanog@nanog.org> Subject: Re: akamai yesterday - what in the world was that Call of Duty Modern Warfare Update came out yesterday. https://dotesports.com/call-of-duty/news/cod-mw-update-version-1-13 On Thu, 23 Jan 2020 at 10:21, Jared Mauch <jared@puck.nether.net<mailto:jared@puck.nether.net>> wrote:

On Jan 23, 2020, at 10:16 AM, Kaiser, Erich <erich@gotfusion.net<mailto:erich@gotfusion.net>> wrote:

Yeah we saw that as well. Must be a game release or something.

Yes, that’s my understanding as well. - Jared

Modern Warfare update is what I'm being told. I did around 4Gpbs from the Xbox network and 1.5Gbps via PS. -----Original Message----- From: NANOG <nanog-bounces@nanog.org> On Behalf Of Jared Mauch Sent: Thursday, January 23, 2020 9:21 AM To: Kaiser, Erich <erich@gotfusion.net> Cc: NANOG list <Nanog@nanog.org> Subject: Re: akamai yesterday - what in the world was that *External Email: Use Caution*

On Jan 23, 2020, at 10:16 AM, Kaiser, Erich <erich@gotfusion.net> wrote:

Yeah we saw that as well. Must be a game release or something.

Yes, that’s my understanding as well. - Jared

Fornite update? On Thu, Jan 23, 2020 at 10:22 AM Jared Mauch <jared@puck.nether.net> wrote:

...

On Jan 23, 2020, at 10:16 AM, Kaiser, Erich <erich@gotfusion.net> wrote:

Yeah we saw that as well. Must be a game release or something.

Yes, that’s my understanding as well.

- Jared

This echoed events a month or so ago, and I'm curious as to what is making these releases more, uh, network-impacting. Game releases are hardly a new thing, but these last two events seem to be almost an order of magnitude higher than what we're used to (at least on our predominantly eyeball network.) Any thoughts from the community? We're taking steps to accommodate, but from a capacity-planning perspective, this seems non-linear to me. Or is it just in my head? On 1/23/20 9:20 AM, Jared Mauch wrote:

...

On Jan 23, 2020, at 10:16 AM, Kaiser, Erich <erich@gotfusion.net> wrote:

Yeah we saw that as well. Must be a game release or something.

Yes, that’s my understanding as well.

- Jared

People have faster connections these days? On Thu, Jan 23, 2020 at 11:14 AM Bryan Holloway <bryan@shout.net> wrote:

This echoed events a month or so ago, and I'm curious as to what is making these releases more, uh, network-impacting.

Game releases are hardly a new thing, but these last two events seem to be almost an order of magnitude higher than what we're used to (at least on our predominantly eyeball network.)

Any thoughts from the community? We're taking steps to accommodate, but from a capacity-planning perspective, this seems non-linear to me.

Or is it just in my head?

On 1/23/20 9:20 AM, Jared Mauch wrote:

...

...

On Jan 23, 2020, at 10:16 AM, Kaiser, Erich <erich@gotfusion.net>

wrote:

...

...

Yeah we saw that as well. Must be a game release or something.

Yes, that’s my understanding as well.

- Jared

If true (not arguing), that's really dumb. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Brandon Martin" <lists.nanog@monmotha.net> To: nanog@nanog.org Sent: Thursday, January 23, 2020 10:23:24 AM Subject: Re: akamai yesterday - what in the world was that On 1/23/20 11:13 AM, Bryan Holloway wrote:

This echoed events a month or so ago, and I'm curious as to what is making these releases more, uh, network-impacting.

My understanding is that, in addition to factors others have mentioned (games are larger, more network based delivery, etc.), that there's a move AWAY from differential patching, to the extent it was previously being used, toward simply delivering an entire new copy of the game, including assets that completely duplicate those that someone may already have. Apparently the rationale is that this is easier on the publisher and those preparing the release, which allows them to get things out sooner, since they don't have to come up with a decent differential patcher and can just make use of the delivery mechanisms already present on the content platform the user is already using. When you've got 100GB games with huge market penetration and each "patch" is an entirely new copy of said 100GB game, that's a lot of traffic. -- Brandon Martin

On 24/Jan/20 01:35, Jared Mauch wrote:

Apple did this with the original iPhone. Turned out even in their ecosystem they didn't get it right. The full restore images have always been there and diffs didn't reappear until you could "OTA" the device (WiFi)

Still the case today. Diffs are only OTA. If you update via iTunes, it's the full whack. This is why I stopped updates via iTunes 3 years ago.

I can't imagine how hard a console would be with every random app writing data wherever. 

Sandboxes and jails have been escaped as long as they have been around as well so they can help but are far from perfect

Isn't the PS4 running FreeBSD :-)? Mark.

On Jan 23, 2020, at 11:52 AM, Valdis Klētnieks <valdis.kletnieks@vt.edu> wrote:

On Thu, 23 Jan 2020 17:13:15 +0100, Bryan Holloway said:

...

Game releases are hardly a new thing, but these last two events seem to be almost an order of magnitude higher than what we're used to (at least on our predominantly eyeball network.)

Any thoughts from the community? We're taking steps to accommodate, but from a capacity-planning perspective, this seems non-linear to me.

Be prepared for an entire new world of hurt this holiday season. Sony has already confirmed that PS5 releases will ship on 100Gbyte blu-ray disks. Which means that download sizes will be comparable…

There’s also the “we will stream you all the data things” I keep hearing about like the Consoles without discs or some other thing I can’t remember the name of. I think this is a tribute to how we’ve built and upgraded networks for capacity and speed. - Jared

Love it Love it Love it I have been telling people that the IEEE 802.3 Ethernet Working Group needs to start looking beyond 400 Gb/s Ethernet. It’s only a matter of time where we will need it! From: NANOG <nanog-bounces@nanog.org> On Behalf Of Tom Beecher Sent: Thursday, January 23, 2020 6:39 PM To: Jared Mauch <jared@puck.nether.net> Cc: NANOG list <Nanog@nanog.org> Subject: Re: akamai yesterday - what in the world was that I think this is a tribute to how we’ve built and upgraded networks for capacity and speed. I think it's spot on. In years past it made more sense to distribute smaller , incremental patches. More work on the software side, but it was likely a better option than getting blasted on Twitter because "OMG I WANT TO PLAY AND MY DOWNLOAD IS TAKING 8 HOURS". This just follows the same rules as networks have always seemed to; If you build it, they will come, and you'll have to build more. :) On Thu, Jan 23, 2020 at 11:57 AM Jared Mauch <jared@puck.nether.net <mailto:jared@puck.nether.net> > wrote:

On Jan 23, 2020, at 11:52 AM, Valdis Klētnieks <valdis.kletnieks@vt.edu <mailto:valdis.kletnieks@vt.edu> > wrote:

On Thu, 23 Jan 2020 17:13:15 +0100, Bryan Holloway said:

...

Game releases are hardly a new thing, but these last two events seem to be almost an order of magnitude higher than what we're used to (at least on our predominantly eyeball network.)

Any thoughts from the community? We're taking steps to accommodate, but from a capacity-planning perspective, this seems non-linear to me.

Be prepared for an entire new world of hurt this holiday season. Sony has already confirmed that PS5 releases will ship on 100Gbyte blu-ray disks. Which means that download sizes will be comparable…

There’s also the “we will stream you all the data things” I keep hearing about like the Consoles without discs or some other thing I can’t remember the name of. I think this is a tribute to how we’ve built and upgraded networks for capacity and speed. - Jared

This just follows the same rules as networks have always seemed to; If you build it, they will come, and you'll have to build more. :)

https://en.m.wikipedia.org/wiki/Induced_demand :-) On Thu., Jan. 23, 2020, 09:40 Tom Beecher <beecher@beecher.cc> wrote:

I think this is a tribute to how we’ve built and upgraded networks for

...

capacity and speed.

I think it's spot on.

In years past it made more sense to distribute smaller , incremental patches. More work on the software side, but it was likely a better option than getting blasted on Twitter because "OMG I WANT TO PLAY AND MY DOWNLOAD IS TAKING 8 HOURS".

This just follows the same rules as networks have always seemed to; If you build it, they will come, and you'll have to build more. :)

On Thu, Jan 23, 2020 at 11:57 AM Jared Mauch <jared@puck.nether.net> wrote:

...

...

On Jan 23, 2020, at 11:52 AM, Valdis Klētnieks <valdis.kletnieks@vt.edu> wrote:

On Thu, 23 Jan 2020 17:13:15 +0100, Bryan Holloway said:

...

Game releases are hardly a new thing, but these last two events seem to be almost an order of magnitude higher than what we're used to (at least on our predominantly eyeball network.)

Any thoughts from the community? We're taking steps to accommodate, but from a capacity-planning perspective, this seems non-linear to me.

Be prepared for an entire new world of hurt this holiday season. Sony has already confirmed that PS5 releases will ship on 100Gbyte blu-ray disks. Which means that download sizes will be comparable…

There’s also the “we will stream you all the data things” I keep hearing about like the Consoles without discs or some other thing I can’t remember the name of.

I think this is a tribute to how we’ve built and upgraded networks for capacity and speed.

- Jared

On Jan 23, 2020, at 1:25 PM, Warren Kumari <warren@kumari.net> wrote:

On Thu, Jan 23, 2020 at 12:45 PM Hugo Slabbert <hugo@slabnet.com> wrote:

...

...

This just follows the same rules as networks have always seemed to; If you build it, they will come, and you'll have to build more. :)

https://en.m.wikipedia.org/wiki/Induced_demand

Yup, there is also (in networking at least) suppressed demand -- I'm sure we've all seen capacity planning discussions along the lines of: "My 1GE is running at 95% capacity - I'm replacing it with a 10GE and it will be around 10% used... wait?! What?! It's now at 7Gbps?! How the hell did that happen?!" scenarios. They usedto be funny, but these days I just find it depressing...

I find it both happy and disturbing. I remember the first 2.4/2.5g links I turned up as well as the first 10g and (eventually) the first 100g links. I was leaving the house earlier this week thinking about how it used to be Mbps of traffic that was a lot and now it’s Gbps and how that’s shifted to Tbps. While it makes me feel old, it’s also something that I marvel about periodically. - jared

On January 23, 2020 at 19:52 paul@nashnetworks.ca (Paul Nash) wrote:

...

While it makes me feel old, it’s also something that I marvel about periodically.

A bit of perspective on bandwidth and feeling old. The first non-academic connection from Africa (Usenet and Email, pre-Internet) ran at about 9600 bps over a Telebit Trailblazer in my living room.

The first non-academic IP connection was a satellite connection (64Kbps IIRC, not in my living room :-)).

Someone asked about Antarctica recently. I remember the day in the 80s when they, I'm pretty sure McMurdo Station, got its first "internet" connection. It was announced on lists like this one. It was a satellite which was good for only so many minutes per day as it flew in and out of sight and exchanged batched email etc via Kermit at probably around 9600bps if that, probably variable depending on conditions. If I may...which also reminds me of a project in Africa which used some sort of wireless link (probably packet-radio) on top of buses. People with the right equipment could get a batch exchange as a bus drove by. I'm pretty certain that really was implemented and used. No idea what the bandwidth was, I think packet-radio in that era generally was glad to achieve around 1200bps. Moral: Really, really, bad connectivity is a zillion times better than no connectivity. -- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

Paul Nash writes:

A bit of perspective on bandwidth and feeling old. The first non-academic connection from Africa (Usenet and Email, pre-Internet) ran at about 9600 bps over a Telebit Trailblazer in my living room.

For your amusement, this latest e-bloodbath, erm -sports update, at 48GB ("PC" version), would take about 463 days (~15 months) to complete at 9600 bps (not counting overhead like packet headers etc.) At 64kbps (ISDN/Antarctica) you could do it in 69 days, maybe even finishing before the next - undoubtedly bigger - release comes out. -- Simon. [I conservatively used decimal Gigabytes, not "Gibibytes" - at 48GiB the numbers would be 497 or 74.5 days respectively.]

* ximaera@gmail.com (Töma Gavrichenkov) [Fri 24 Jan 2020, 11:49 CET]:

And now for our amusement Akamai can do it *accidentally*.

What do you mean? The CDNs don't publish the games nor do they buy the games. The people downloading aren't even their customers. The publishers generally decide the % of traffic share each CDN serves if they contract with multiple CDNs for a project. -- Niels.

On 23/Jan/20 21:52, Paul Nash wrote:

A bit of perspective on bandwidth and feeling old. The first non-academic connection from Africa (Usenet and Email, pre-Internet) ran at about 9600 bps over a Telebit Trailblazer in my living room.

Where in Africa? It's a small place you know :-)...

Now we have a bajillion Gbps over submarine fibre landing pretty much everywhere, and my guess is that it is not enough bandwidth.

It probably won't be, but judging by where things are going, the submarine, cross-continental links will be carrying most of the traffic for a handful of networks. Hint: they aren't the telco's.

All this to bring such vital resources as Facebook and Netflix :-)

I used to say back in my Malaysia days, "Content is not King, Connectivity is King". This was the days of Hi5 and MySpace. As for Netflix, we're just tired of having to leave the house, but I suppose Amazon is the grand daddy of that :-). Mark.

On Fri, Jan 24, 2020 at 9:55 AM Aaron Gould <aaron1@gvtc.com> wrote:

Thanks Jared, When I reminisce with my boss he reminds me that this telco/ISP here initially started with a 56kbps internet uplink , lol

Oh, gods, what have you done?! This comment will bring everyone out of the woodwork, reminiscing about the good ol' days.... So, I grew up in South Africa, and one of the more fascinating / cooler things I saw was a modem which would get you ~50bps (bps, not Kbps) over a single strand of barbed wire -- you'd hammer a largish nail into the ground, and clip one alligator[0] clip onto that, and another alligator clip onto the barbed wire. Repeat the process on the other side (up to ~5km away), plug the modems in, and bits would flow... I only saw these used a few times, but always thought they were cool.... One of the first ISPs I worked at had an AGS+ as the "core" router (and a pile of IGS's) -- the AGS had metal plate in the front to access the "line cards", and a monster big squirrel fan - the squirrel fan was strong enough that the vacuum / airflow would keep the metal plate sucked on if you didn't do up the screw.. anyway, the device also had a watchdog timer, which would reboot the box if IOS[1] locked up, and the fan would slow down while this occurred -- so, for a long time, our fastest / most reliable monitoring was an empty PC case placed on the floor under the rack -- when the router locked up, the fan would slow down, the front would fall off[2] and bounce on the PC case, making an unholy racket - and alerting the NOC that something bad was happening.... Anyway, so I tied an onion to my belt, which was the style at the time. Now, to take the ferry cost a nickel, and in those days, nickels had pictures of bumblebees on 'em. Give me five bees for a quarter, you'd say. Now where were we? Oh yeah: the important thing was I had an onion on my belt, which was the style at the time. They didn't have white onions because of the war. The only thing you could get was those big yellow ones... W [0]: In .za we called them crocodile clips -- true story. [1]: For all you young whippersnappers, that's the Cisco IOS, not the Apple iOS.. :-/ [2]: https://www.youtube.com/watch?v=3m5qxZm_JqM -- Unlike the rest of this email, this is off-topic, but still great....

-Aaron

-- I don't think the execution is relevant when it was obviously a bad idea in the first place. This is like putting rabid weasels in your pants, and later expressing regret at having chosen those particular rabid weasels and that pair of pants. ---maf

So, I grew up in South Africa, and one of the more fascinating / cooler things I saw was a modem which would get you ~50bps (bps, not Kbps) over a single strand of barbed wire -- you'd hammer a largish nail into the ground, and clip one alligator[0] clip onto that, and another alligator clip onto the barbed wire. Repeat the process on the other side (up to ~5km away), plug the modems in, and bits would flow... I only saw these used a few times, but always thought they were cool….

Do you remember anything about the actual type of modem? Or where you deployed them? In the days before the Internet came to SA, I ran a dial-up email link between the US and Pretoria, polled by various people locally (including CSIR, SAIMR). I also carried mail for the UNHCR in Northern Mozambique. Mail came via Karl Deninger (DDSW1) in Chicago, IIRC. They were missing several kilometres of phone wire, so connected the link to the fence on each side of the road. We get about 1200bps on a good day IIRC, and would loose carrier whenever someone moved cattle from one field to another and opened a gate in the fence. paul

The Civil Engineering version of this is SWER electrical distribution. Single-Wire, Earth-Return. And it’s as crazy in implementation as it sounds now. -Ben Cannon CEO 6x7 Networks & 6x7 Telecom, LLC ben@6by7.net <mailto:ben@6by7.net>

On Jan 25, 2020, at 8:24 AM, Allen McKinley Kitchen (gmail) <allenmckinleykitchen@gmail.com> wrote:

On Jan 25, 2020, at 08:52, Paul Nash <paul@nashnetworks.ca> wrote:

...



...

So, I grew up in South Africa, and one of the more fascinating / cooler things I saw was a modem which would get you ~50bps (bps, not Kbps) over a single strand of barbed wire -- you'd hammer a largish nail into the ground, and clip one alligator[0] clip onto that, and another alligator clip onto the barbed wire. Repeat the process on the other side (up to ~5km away), plug the modems in, and bits would flow... I only saw these used a few times, but always thought they were cool….

Do you remember anything about the actual type of modem? Or where you deployed them?

Decades ago, I cobbled together a 20mA current loop interface that may have been an early version of this .. ran a set of Baudot machines (pre-ASCII, upper case & figs only) mostly just to have fun with a set of old ASR 32 teletypes. I used a couple of 500’ spools of zip cord lying on the ground from end to end. Never mind backhoes - it was lawn-mower vulnerable. (However, being flat on the ground seemingly made it less vulnerable to lightning strikes.)

Of course, this was hardly critical infrastructure!

Blessings..

Allen

Valdis Klētnieks wrote on 24/01/2020 21:20:

I remember when a "gateway" was a Microvax II with an ethernet card and a bisync card

I remember the day when the microvax II and all the other vaxes on campus were upgraded from CMU-TEK to the Multinet TCP/IP stack. Gone were the days of maxing out at 14kbit/s on the high speed 10 megabit co-ax campus backbone - we could now get hundreds of kilobits per second. The joy of it. That decision to upgrade happened the day that all CMU-TEK installations in the world went offline due to some bug or other. Nick

I started what became 6x7 with a 64k ISDN line. And 9600 baud modems… in ’93 or so. (I was a child, in Jr High…) -Ben. -Ben Cannon CEO 6x7 Networks & 6x7 Telecom, LLC ben@6by7.net <mailto:ben@6by7.net>

On Jan 24, 2020, at 3:21 PM, bzs@theworld.com wrote:

On January 24, 2020 at 08:55 aaron1@gvtc.com (Aaron Gould) wrote:

...

Thanks Jared, When I reminisce with my boss he reminds me that this telco/ISP here initially started with a 56kbps internet uplink , lol

Point of History:

When we, The World, first began allowing the general public onto the internet in October 1989 we actually had a (mildly shared*) T1 (1.544mbps) UUNET link. So not so bad for the time. Dial-up customers shared a handful of 2400bps modems, we still have them.

* It was also fanned out of our office to a handful of Boston-area customers who had 56kbps or 9600bps leased lines, not many.

-- -Barry Shein

Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

On Sat, 2020-01-25 at 22:29 -0600, Aaron Gould wrote:

From: Ben Cannon [mailto:ben@6by7.net]  I started what became 6x7 with a 64k ISDN line.   And 9600 baud modems…   

Pah! Luxury! When *I* were a lad we had to touch the wires with our tongues to tell one from zero, no job for a sissy lemme tell you. And don't talk to me about bandwidth. You could increase it easily enough by wiring up other body parts, but it was hard to keep the staff. My ancestors - well, up was light and down was dark, that's all *they* knew, swimmin' around in the primordial seas. The phrase "next generation technology" *meant* something back then. You tell that to young folk these days and they don't believe you... -K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@biplane.com.au) http://www.biplane.com.au/kauer http://twitter.com/kauer389 GPG fingerprint: 8D08 9CAA 649A AFEF E862 062A 2E97 42D4 A2A0 616D Old fingerprint: A0CD 28F0 10BE FC21 C57C 67C1 19A6 83A4 9B0B 1D75

On January 26, 2020 at 15:59 kauer@biplane.com.au (Karl Auer) wrote:

On Sat, 2020-01-25 at 22:29 -0600, Aaron Gould wrote:

...

From: Ben Cannon [mailto:ben@6by7.net]  I started what became 6x7 with a 64k ISDN line.   And 9600 baud modems…   

Pah! Luxury!

When *I* were a lad we had to touch the wires with our tongues to tell one from zero, no job for a sissy lemme tell you. And don't talk to me about bandwidth. You could increase it easily enough by wiring up other body parts, but it was hard to keep the staff.

You had ones?! We couldn't afford them, we had to guess from the time delays between zeros. -- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

On 26/Jan/20 06:59, Karl Auer wrote:

You tell that to young folk these days and they don't believe you...

No use. They couldn't describe a cassette if it came with the manual; nor a Nokia 5110 phone for that matter:-). Mark.

I was a student worker at a computer lab at USC in the 70s and a buddy had a system operator job at ISI in Marina Del Rey. One day he connected to his office from my lab via a 300baud acoustic modem and then got on the ARPA-NET. From there he connected to a system called ATLAS in the UK. I had no idea what to do at the prompt so I typed

?

to get list of commands. My global eyes were opened when the response was Pardon? instead of the usual rude or cryptic error message that I was used to. There was a big world out there and we were definitely not in Kansas anymore! Gene On 2/16/20 1:25 PM, bzs@theworld.com wrote:

Ok it's Sunday...

The first time I got on the internet was around 1977.

A friend dropped by the lab I worked in at Harvard and wondered if I had an MIT ITS account and I said no wasn't even sure what it was other than a time sharing system at MIT.

So we had a modem and dumb terminal and dialed-in and one could create an account from the login prompt which I guess today seems mundane but really was totally unintuitive, getting logins on time shared systems generally required paper work and proof one should have access.

And I became BARRYS@MIT-AI (no stinkin' dots back then.)

He showed me some ARPAnet things and I was suitably amazed and explored more from home where I had my own dumb tty and modem.

TBH I didn't really have much use for it at the time other than joining mailing lists or similar.

Occasionally if someone was in the room I'd say "watch this!" and get to a login prompt at Stanford or UCL (London.) They were usually impressed.

I did use the local area network to access MIT-MC to use MacSyma (a symbolic math package) which I did use in my work.

I was fairly amazed that my files were visible on either machine.

etc etc etc.

-- Gene LeDuc | A ship in port is safe, but that's not Technology Security | what ships are built for. San Diego State University | --Adm. Grace Hopper, USN

So it was YOUR fault the phone at the Fotomat was always busy when I tried calling to check if my prints were done? At 01:20 PM 17/02/2020, Anne P. Mitchell, Esq. wrote:

When I got my next computer (and first portable) shortly thereafter (a TRS Model 100) I got acoustic cups for it, and suddenly I was connected from anywhere and everywhere there was a phone - including from my job at a Fotomat booth (remember those?) :-)

I remember that TI luggable - that sucker weighed a ton! I dragged it from the lab to my dorm a few times to log in remotely, but carrying it on a bicycle was a dicey deal and I got over the novelty pretty quickly. I'd forgotten who made it until you mentioned it - good memories! Gene On 2/17/20 11:34 AM, nanog08@mulligan.org wrote:

Back in 1973 I was hired by Tymshare to "hack" Tymnet and some of the various systems (XDS 940, PDP-10s) - I was 15.  Tymshare provided me with a Teletype ASR-33 (with the built in tape punch and reader).  I had an AJ 300 baud acoustic coupler.  We had a second phone line installed, 'cause my dad was tired of picking up the phone and hearing tones.  I ended up rewiring the house phones so I could put the terminal in my room.

When I went to the Pentagon in '79 I was in charge of PENT-TIP and got to take home and travel with a TI Silent700 with a built in acoustic coupler.  We had a bank of 300/1200 baud modems on PENT-TIP.  Our IMP was connected to the Arpanet via a 56K modem that was the size of 5 foot tall 19" rack!  Back in those days it seems TIP phone numbers were closely guarded treasure.

I still remember when I got an LS ADM-3A (no more finding rolls of thermal paper). I still have it, though I don't know why...

Geoff

On 2/17/20 11:20 AM, Anne P. Mitchell, Esq. wrote:

...

...

On Feb 17, 2020, at 10:38 AM, Gene LeDuc <gleduc@sdsu.edu> wrote:

I was a student worker at a computer lab at USC in the 70s and a buddy had a system operator job at ISI in Marina Del Rey.  One day he connected to his office from my lab via a 300baud acoustic modem and then got on the ARPA-NET.  From there he connected to a system called ATLAS in the UK.  I had no idea what to do at the prompt so I typed

...

? to get list of commands.  My global eyes were opened when the response was

Pardon?

instead of the usual rude or cryptic error message that I was used to. There was a big world out there and we were definitely not in Kansas anymore! It was about 1980.  My C-128 came with one of those CIS snap packs to let you test connecting to the 'net via Compuserve.  So I connected with my 300baud modem and..whoa!!!

When I got my next computer (and first portable) shortly thereafter (a TRS Model 100) I got acoustic cups for it, and suddenly I was connected from anywhere and everywhere there was a phone - including from my job at a Fotomat booth (remember those?) :-)

Anne

-- Anne P. Mitchell, Attorney at Law, Dean of Cyberlaw & Cybersecurity, Lincoln Law School CEO/President, SuretyMail Email Reputation Certification Author: Section 6 of the CAN-SPAM Act of 2003 (the Federal anti-spam law) Legislative Consultant, GDPR, CCPA (CA) & CCDPA (CO) Compliance Consultant Former Counsel: Mail Abuse Prevention System (MAPS)

-- Gene LeDuc | Experience is the worst teacher. It always Technology Security | gives the test first, and the lesson San Diego State University | afterwards.

On Feb 17, 2020, at 10:38 AM, Gene LeDuc <gleduc@sdsu.edu> wrote:

Does this thread make me not only think about the days of old, but also makes me feel older! Not going back as far as some here but around 1993ish... My first connection back in the day was a shell account I was given as consultant and reseller for what was TIAC (then became PSI). I was also at the time running a 6 line MajorBBS system (prior to WorldGroup). TIAC allowed multiple concurrent logins to the shell so I purchased the MajorBBS dialout module and had it login to my shell account which my cosysop crafted a nice menu for basic feature usage such as ftp/lynx/irc/etc. Users could then use my dialout feature into my shell and do what they were looking to do. Being somewhat out in the sticks and having a decent dial area coverage it in a sense allowed me to become my local region's first ISP of sorts. It was a hack but it worked and users were more than happy since for many even dialing up to AOL or Compuserve was a toll call then. -- If Confucius were alive today: "A computing device left in the OFF power state never crashes" ----- 73 de Brian N1URO IPv6 Certified SMTP: n1uro-at-n1uro.ampr.org

Right?? That’s in a customer’s office building too… I’ve got the same connection on my workstation of course. I actually have another test that I don’t normally share. It’s NOT fake. I found out that the speedtest algorithm rounds to the nearest whole millisecond. And that it will round DOWN below 400ns… A friend, being kind of sassy, said recently “Hey Ben, so like when you got the 10g, did you just like, download all of Netflix?” And I had to pause and give a semi serious reply, where I said “Actually no… Because we’ve got a Netflix Openconnect in SF2 that has like 80g into the fabric, and we’ve got 100g into there, and even over the 10g to my desktop, that’s still faster than my SSD. So no, I’m not going to be downloading all of Netflix. The internet is my LAN, I already have…" (It’s honestly all so cool that I wake up every day without an alarm clock, and *run* downstairs to do what I do. Best job in the world.) -Ben Cannon CEO 6x7 Networks & 6x7 Telecom, LLC ben@6by7.net <mailto:ben@6by7.net>

On Jan 25, 2020, at 8:29 PM, Aaron Gould <aaron1@gvtc.com> wrote:

I love the symmetric ~10 gig speed test to put it into perspective for how far we’ve come….also the 3 ms ping result. Ain’t it great

-Aaron

From: Ben Cannon [mailto:ben@6by7.net <mailto:ben@6by7.net>] Sent: Friday, January 24, 2020 5:27 PM To: bzs@theworld.com <mailto:bzs@theworld.com> Cc: Aaron Gould; NANOG Operators' Group Subject: Reminiscing our first internet connections (WAS) Re: akamai yesterday - what in the world was that

I started what became 6x7 with a 64k ISDN line. And 9600 baud modems…

in ’93 or so. (I was a child, in Jr High…)

-Ben.

-Ben Cannon CEO 6x7 Networks & 6x7 Telecom, LLC ben@6by7.net <mailto:ben@6by7.net>

<image001.png>

...

On Jan 24, 2020, at 3:21 PM, bzs@theworld.com <mailto:bzs@theworld.com> wrote:

On January 24, 2020 at 08:55 aaron1@gvtc.com <mailto:aaron1@gvtc.com> (Aaron Gould) wrote:

Thanks Jared, When I reminisce with my boss he reminds me that this telco/ISP here initially started with a 56kbps internet uplink , lol

Point of History:

When we, The World, first began allowing the general public onto the internet in October 1989 we actually had a (mildly shared*) T1 (1.544mbps) UUNET link. So not so bad for the time. Dial-up customers shared a handful of 2400bps modems, we still have them.

* It was also fanned out of our office to a handful of Boston-area customers who had 56kbps or 9600bps leased lines, not many.

-- -Barry Shein

Software Tool & Die | bzs@TheWorld.com <mailto:bzs@theworld.com> | http://www.TheWorld.com <http://www.theworld.com/> Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

IIRC that 64k was in fact 56k with 8k for overhead. I had one, and it would kick in a second channel if you pushed it, for a whopping 112k. Metered, came out to about $500/mo. Joly On Fri, Jan 24, 2020 at 6:26 PM Ben Cannon <ben@6by7.net> wrote:

I started what became 6x7 with a 64k ISDN line. And 9600 baud modems…

in ’93 or so. (I was a child, in Jr High…)

-Ben.

-Ben Cannon CEO 6x7 Networks & 6x7 Telecom, LLC ben@6by7.net

On Jan 24, 2020, at 3:21 PM, bzs@theworld.com wrote:

On January 24, 2020 at 08:55 aaron1@gvtc.com (Aaron Gould) wrote:

Thanks Jared, When I reminisce with my boss he reminds me that this telco/ISP here initially started with a 56kbps internet uplink , lol

Point of History:

When we, The World, first began allowing the general public onto the internet in October 1989 we actually had a (mildly shared*) T1 (1.544mbps) UUNET link. So not so bad for the time. Dial-up customers shared a handful of 2400bps modems, we still have them.

* It was also fanned out of our office to a handful of Boston-area customers who had 56kbps or 9600bps leased lines, not many.

-- -Barry Shein

Software Tool & Die | bzs@TheWorld.com <bzs@theworld.com> | http://www.TheWorld.com <http://www.theworld.com> Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

-- --------------------------------------------------------------- Joly MacFie 218 565 9365 Skype:punkcast -------------------------------------------------------------- -

On Mon, 27 Jan 2020 at 13:35, Bryan Holloway <bryan@shout.net> wrote:

I didn't think one could get a single 'B' channel over ISDN ... but I could be mistaken.

In my early ISP days, ISDN was 2 x 64k (full-rate) 'B' channels and a 16k 'D' channel for signaling.

There was much flexibility you could do with them. Telia had product for shops where 1B was voice, 1B was fax and D was frame-relay data for POS. -- ++ytti

Wasn't the 56/64k thing a result of CAS (bit robbed) signalling which was a fudge AT&T did to transport signalling information in-band on T1s by stealing the low order bit for OOB signalling (it wasnt actually every low order bit, but meant you had to throw away every low order bit as CPE didn't know which ones were "corrupted" by the carrier). Proper ISDN was always 64kbit/s clear path with separate D channels carried OOB end to end, away from the B channel data. On Mon, 27 Jan 2020 at 11:57, Mark Andrews <marka@isc.org> wrote:

The hardware support was 2B+D but you could definitely just use a single B. 56k vs 64k depended on where you where is the world and which style of ISDN the telco offered.

-- Mark Andrews

...

On 27 Jan 2020, at 22:32, Bryan Holloway <bryan@shout.net> wrote:

I didn't think one could get a single 'B' channel over ISDN ... but I could be mistaken.

In my early ISP days, ISDN was 2 x 64k (full-rate) 'B' channels and a 16k 'D' channel for signaling.

...

On 1/26/20 5:58 AM, Joly MacFie wrote: IIRC that 64k was in fact 56k with 8k for overhead. I had one, and it would kick in a second channel if you pushed it, for a whopping 112k. Metered, came out to about $500/mo. Joly On Fri, Jan 24, 2020 at 6:26 PM Ben Cannon <ben@6by7.net <mailto: ben@6by7.net>> wrote: I started what became 6x7 with a 64k ISDN line. And 9600 baud modems… in ’93 or so. (I was a child, in Jr High…) -Ben. -Ben Cannon CEO 6x7 Networks & 6x7 Telecom, LLC ben@6by7.net <mailto:ben@6by7.net>

...

On Jan 24, 2020, at 3:21 PM, bzs@theworld.com <mailto:bzs@theworld.com> wrote:

On January 24, 2020 at 08:55 aaron1@gvtc.com <mailto:aaron1@gvtc.com> (Aaron Gould) wrote:

...

Thanks Jared, When I reminisce with my boss he reminds me that this telco/ISP here initially started with a 56kbps internet uplink , lol

Point of History:

When we, The World, first began allowing the general public onto the internet in October 1989 we actually had a (mildly shared*) T1 (1.544mbps) UUNET link. So not so bad for the time. Dial-up customers shared a handful of 2400bps modems, we still have them.

* It was also fanned out of our office to a handful of Boston-area customers who had 56kbps or 9600bps leased lines, not many.

-- -Barry Shein

Software Tool & Die | bzs@TheWorld.com <mailto:bzs@theworld.com> | http://www.TheWorld.com <http://www.theworld.com> Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* --

---

Joly MacFie 218 565 9365 Skype:punkcast -------------------------------------------------------------- -

-- -- Rob Pickering, rob@pickering.org

On 1/27/20 1:42 PM, Aled Morris via NANOG wrote:

On Mon, 27 Jan 2020 at 12:13, Rob Pickering <rob@pickering.org <mailto:rob@pickering.org>> wrote:

Wasn't the 56/64k thing a result of CAS (bit robbed) signalling which was a fudge AT&T did to transport signalling information in-band on T1s by stealing the low order bit for OOB signalling (it wasnt actually every low order bit, but meant you had to throw away every low order bit as CPE didn't know which ones were "corrupted" by the carrier). Proper ISDN was always 64kbit/s clear path with separate D channels carried OOB end to end, away from the B channel data.

There was some element of interoperability required with the pre-existing data network architecture based on 56k channels and T1 bearers.  This article has the detail:

https://en.wikipedia.org/wiki/T-carrier

/Soon after commercial success of T1 in 1962, the T1 engineering team realized the mistake of having only one bit to serve the increasing demand for housekeeping functions. They petitioned AT&T management to change to 8-bit framing. This was flatly turned down because it would make installed systems obsolete./

Compared to what was to follow, that all had to suffer the 56k channel limitation, there can't have been that many installed systems in 1962!

Aled

I seem to also recall that you couldn't use a 56k modem unless the far-end was digital.

The fudge was required because of the use of copper based T1's. The early implementation required a min of 1's density for those old repeaters to work properly(AMI, Alt Mark Inversion). Conversion to fiber between telco offices allowed them to drop SF and AMI to ESF. Fiber equipment dropped the min 1 density to function properly. Lyle On 2020-01-27 06:11, Rob Pickering wrote:

Wasn't the 56/64k thing a result of CAS (bit robbed) signalling which was a fudge AT&T did to transport signalling information in-band on T1s by stealing the low order bit for OOB signalling (it wasnt actually every low order bit, but meant you had to throw away every low order bit as CPE didn't know which ones were "corrupted" by the carrier). Proper ISDN was always 64kbit/s clear path with separate D channels carried OOB end to end, away from the B channel data.

On Mon, 27 Jan 2020 at 11:57, Mark Andrews <marka@isc.org <mailto:marka@isc.org>> wrote:

The hardware support was 2B+D but you could definitely just use a single B.   56k vs 64k depended on where you where is the world and which style of ISDN the telco offered.

-- Mark Andrews

> On 27 Jan 2020, at 22:32, Bryan Holloway <bryan@shout.net <mailto:bryan@shout.net>> wrote: > > I didn't think one could get a single 'B' channel over ISDN ... but I could be mistaken. > > In my early ISP days, ISDN was 2 x 64k (full-rate) 'B' channels and a 16k 'D' channel for signaling. > > >> On 1/26/20 5:58 AM, Joly MacFie wrote: >> IIRC that 64k was in fact 56k with 8k for overhead. >> I had one, and it would kick in a second channel if you pushed it, for a whopping 112k. Metered, came out to about $500/mo. >> Joly >> On Fri, Jan 24, 2020 at 6:26 PM Ben Cannon <ben@6by7.net <mailto:ben@6by7.net> <mailto:ben@6by7.net <mailto:ben@6by7.net>>> wrote: >>    I started what became 6x7 with a 64k ISDN line.  And 9600 baud modems… >>    in ’93 or so.  (I was a child, in Jr High…) >>    -Ben. >>    -Ben Cannon >>    CEO 6x7 Networks & 6x7 Telecom, LLC >> ben@6by7.net <mailto:ben@6by7.net> <mailto:ben@6by7.net <mailto:ben@6by7.net>> >>>    On Jan 24, 2020, at 3:21 PM, bzs@theworld.com <mailto:bzs@theworld.com> >>>    <mailto:bzs@theworld.com <mailto:bzs@theworld.com>> wrote: >>> >>> >>>    On January 24, 2020 at 08:55 aaron1@gvtc.com <mailto:aaron1@gvtc.com> >>>    <mailto:aaron1@gvtc.com <mailto:aaron1@gvtc.com>> (Aaron Gould) wrote: >>>>    Thanks Jared, When I reminisce with my boss he reminds me that >>>>    this telco/ISP here initially started with a 56kbps internet >>>>    uplink , lol >>> >>>    Point of History: >>> >>>    When we, The World, first began allowing the general public onto the >>>    internet in October 1989 we actually had a (mildly shared*) T1 >>>    (1.544mbps) UUNET link. So not so bad for the time. Dial-up customers >>>    shared a handful of 2400bps modems, we still have them. >>> >>>    * It was also fanned out of our office to a handful of Boston-area >>>    customers who had 56kbps or 9600bps leased lines, not many. >>> >>>    --            -Barry Shein >>> >>>    Software Tool & Die    | bzs@TheWorld.com >>>    <mailto:bzs@theworld.com <mailto:bzs@theworld.com>>            | http://www.TheWorld.com >>>    <http://www.theworld.com> >>>    Purveyors to the Trade | Voice: +1 617-STD-WRLD       | 800-THE-WRLD >>>    The World: Since 1989  | A Public Information Utility | *oo* >> -- >> --------------------------------------------------------------- >> Joly MacFie 218 565 9365 Skype:punkcast >> -------------------------------------------------------------- >> -

-- -- Rob Pickering, rob@pickering.org <mailto:rob@pickering.org>

Don't forget B8ZS which did way with the need for SFon copper data T1s On 1/27/2020 10:43 AM, Lyle Giese wrote:

64k vs 56k was the result of changing T1 framing from SF to ESF.  SF utilized AMI(Alt Mark Inversion) required for copper T1 lines between Central Offices.  SF(Super Frame) robbed bits for signalling and limited each voice channel to 56k.  Conversion to fiber between TELCO offices allowed the conversion of SF to ESF, which dropped the AMI requirement and the resultant bit robbing, allowing 64k throughput per voice channel.

In other words, the limitation was in the inter-office T1's and the conversion of to fiber between TELCO offices cleared that hurdle.

Lyle Giese

LCR Computer Services, Inc.

On 1/27/20 2:00 PM, Jamie Bowden via NANOG wrote:

I don't know about the other ILECs out there, but I don't know if Verizon will even provision a T1 anymore. I know you can still get a PRI (that's how our phone systems interface with the PSTN), but if we needed a CT1 instead, I don't know that they'd be able (willing) to deliver it. I know you can't get a BRI. We moved offices a few years ago and we basically lost the ability to use our STEs for anything but voice as we couldn't get BRIs delivered to the new space.

Last I checked (a couple years ago, I guess), you could absolutely still order a clear-channel point-to-point T1 in AT&T territory in the Indianapolis LATA including surrounding exchanges. I don't know if AT&T themselves would put channelized/CAS voice on it, but I'm sure you could find someone they can terminate it to who would. I assume this is the same in many places. Now, presumably they'll deliver it on their existing TDM optical plant if you're in an on-net facility or SHDSL if you're not as has been the case for a couple decades at least, but it was still something you could order. I got a quote on a channelized/split T1 with 4 voice channels and the remainder used for PPP IP service from Centurylink about a year back, too, for out-of-band and alarm use in a (non-Centurylink but about 3 blocks away from theirs) CO facility within part of their legacy Sprint/Embarq footprint. They did give me a quote, so I assume they would have delivered it (somehow). It was laughably expensive, but they did offer to do it. IIRC they actually were going to do it as a PRI with 4 voice bearers, the 1 PRI signalling channel, and then the remaining 18 channels for the point-to-point data, so perhaps they were able to do that since it was a "PRI" product (which may have at least partially explained the exorbitant cost). We ended up just ordering 3 POTS lines and <grumble grumble> for OOB data. -- Brandon Martin

On January 27, 2020 at 22:57 marka@isc.org (Mark Andrews) wrote:

The hardware support was 2B+D but you could definitely just use a single B. 56k vs 64k depended on where you where is the world and which style of ISDN the telco offered.

FWIW bulk dial-up lines were often brought in as PRIs which were 24 ISDN 2B+D lines on basically a T1 (1.544mbps) and then you could break those out to serial lines. The sort of cool thing was that you could get caller information on those even if the caller thought they blocked it with *69 or whatever it was and log it. I forget the acronym...no no, that's the usual caller-id this was...ummmm, DNI? Something like that. I won a court case with that data. -- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

Carrying on with the “first Internet connection” thread: I forget how I found out about Usenet and UUCP email (lost in the mosts of time). I ran a store and forward dial-up link from South Africa to DDSW1 in Chicago (Hi Karl! Thanks!). I cobbled together a package with a DOS-based mail reader and a DOS port of UUCP that several people used to get their email (including a local medical research establishment and the local veterinary college). Demand grew, along with a request to relay email to the UNHCR in Northen Mozambique, so I scraped some money together to import a horribly expensive Telebit modem. I ended up being the regional non-academic email hub for Southern Africa. Just prior to the 1994 election, I got together with a two friends (Alan Barret and Chris Pinkham) and founded the first ISP in sub-Saharan Africa. We managed to get a 64k satellite link at a very good price (the satellite folk were busy being retrenched and we were prepared to sign a contract specifically requiring satellite service for 5 years, which gave them some job security). We borrowed a Cisco router from DiData (Cisco agents), skirted other telco regulations to link regions. One of our early customers was a group of students who wanted to start a small dial ISP nearby. We gave them service, bootstrapping what became our biggest competitor, Internet Solutions (now part of DiData, who never did ask for their router back). Our little ISP grew and grew, and eventually merged with our biggest client, was sold, sold again, and so on. Last time I looked, it had become Verizon Africa. paul

On Jan 28, 2020, at 6:40 PM, Forrest Christian (List Account) <lists@packetflux.com> wrote:

So to add my two stories:

I provided the Idea and a whole bunch of time/labor/etc to start a dialup ISP in our hometown back in 1994. I remember having a big debate on whether to bring in a single 56K leased line or 128K fractional T1. We went with the Fractional T1 just because it could be easily expanded over time. (That T1 is now multiple 10GB circuits - yes the ISP is still running and I still am involved). So a single 128K fractional T1, a cisco 2501 (with external DSU, those internal cards didn't exist yet), and 8 14.4 modems attached to a single Sun Unix box. Note that this was pre-web, and back in the days where you pretty much knew at least generally everything which was on the internet.

Things grew quickly, don't remember how many lines. At some point we moved to having 56K modems on our end, which required a digital carrier to the central office. T1's were very expensive, so we did a bit of tariff arbitrage. One could obtain a 'metered' ISDN BRI line for like next to nothing - the metering had to do with the fact they were going to charge you by the minute for any calls, but here's the catch: for outgoing calls only, incoming calls were free which worked great for a dialup ISP. The problem was that 56K dialin concentrators all wanted T1 lines. What we discovered is that Adtran made a box which would take a whole bunch of ISDN BRI (each with 2 channels), and combine them into a single T1. And due to the retail pricing difference for T1 vs BRI, we could pay for the box in a few months. So we took a whole truckload of ISDN BRI lines and combined them into a few channelized T1's and ended up paying a lot less to the phone company.

Of course, things have grown past that (we have an extensive WISP network and have an ever-growing amount of fiber in the ground). But it's fun to think about where we started.

On Mon, Jan 27, 2020 at 1:00 PM <bzs@theworld.com> wrote:

On January 27, 2020 at 22:57 marka@isc.org (Mark Andrews) wrote:

...

The hardware support was 2B+D but you could definitely just use a single B. 56k vs 64k depended on where you where is the world and which style of ISDN the telco offered.

FWIW bulk dial-up lines were often brought in as PRIs which were 24 ISDN 2B+D lines on basically a T1 (1.544mbps) and then you could break those out to serial lines.

The sort of cool thing was that you could get caller information on those even if the caller thought they blocked it with *69 or whatever it was and log it. I forget the acronym...no no, that's the usual caller-id this was...ummmm, DNI? Something like that.

I won a court case with that data.

-- -Barry Shein

Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

-- - Forrest

On 29/Jan/20 02:22, Paul Nash wrote:

One of our early customers was a group of students who wanted to start a small dial ISP nearby. We gave them service, bootstrapping what became our biggest competitor, Internet Solutions (now part of DiData, who never did ask for their router back). Our little ISP grew and grew, and eventually merged with our biggest client, was sold, sold again, and so on. Last time I looked, it had become Verizon Africa.

It went through some hands, but yes, I worked with Africa Online, which was looking to pick up UUNet Africa. At the time, the operations were mainly in Kenya, Zambia, Botswana and South Africa (we are talking 2003 - 2007 or so). UUNet Africa eventually became Verizon Africa, and they got picked up by MTN, being renamed to MTN-NS (Network Solutions). I believe it now trades at MTN Business, in some or all of those markets. Mark.

A PRI was and still is 23B+D, not 24 2B+D lines. 23B+D (ISDN PRI or Primary Rate Interface) is 23x64 kbps bearer channels and a 64 kbps Delta channel carried on a B8ZS T1 You could carry additional groups of 24 B channels using NFAS, or Non-Facility Associated Signalling where the D channel on the first PRI carried the call setup and teardown information for the additional B channels on up to 20 PRI circuits. 2B+D (ISDN BRI or Basic Rate Interface) 2x64 kbps bearer channels and a 16 kbps Delta channel carried on an 144 kbps 2B1Q U interface. At 02:58 PM 27/01/2020, bzs@theworld.com wrote:

FWIW bulk dial-up lines were often brought in as PRIs which were 24 ISDN 2B+D lines on basically a T1 (1.544mbps) and then you could break those out to serial lines.

On 26/Jan/20 06:58, Joly MacFie wrote:

IIRC that 64k was in fact 56k with 8k for overhead. 

I had one, and it would kick in a second channel if you pushed it, for a whopping 112k. Metered, came out to about $500/mo.

My first ISDN experience was in Swaziland, 2003. There weren't many countries in Africa that offered it, actually. As tiny as Swaziland was, I was surprised they did. I just liked it because the dial-up was instant and it didn't "sing the modem song" :-). Mark.

... and disabling call-waiting ... ;) On 1/27/20 1:55 PM, John Von Essen wrote:

In those early days I remember setting up a download to start before bed so it could run all night, then wake up the morning to see my freshly downloaded 300KB file — assuming the phone line remained stable.

-John

Does AOL count? If my first real internet connection was dial up 3600 baud through compuserv. When I finally upgraded to 56K I thought it was light speed. On Mon, Jan 27, 2020 at 9:01 AM Bruce H McIntosh <bhm@ufl.edu> wrote:

On 1/27/20 7:59 AM, Bryan Holloway wrote:

...

[External Email]

... and disabling call-waiting ... ;)

We had a separate line (paid for by our work) without call-bothering on it for the modem.

-- -------------------------------------------- Bruce H. McIntosh Network Engineer II University of Florida Information Technology bhm@ufl.edu 352-273-1066

On 1/27/20 3:06 PM, bzs@theworld.com wrote:

I remember going from 300b to 1200b and thinking wow, this is it, we're done, I cannot read text scrolling on the screen at 1200b.

Other than the 75 and 110 baud teletypes that only did text, my first TCP/IP connection was 300b, back when we had to rent the modems (1979). I had to write my own TCP/IP stacks, on both the Interdata 7/16 at the office and my first personal computer: a 2 MHz Z80 S-100 bus. Built my own serial device too, with a rather large switch on the back to change speeds. (Still have it, just carried it out of the garage over the weekend, haven't turned it on in years as the special floppies have died.) Eventually, got my own 300b Hayes Micromodem! It took a long time to upgrade to 1200b, as the modems were thousands of dollars each. Roughly $18,000 each in today's dollars. Only used between major sites. Racal-Vadic triple modems were a big step (circa 1986). When we first designed PPP in the late '80s to replace SLIP and SLFP, it was expected to run at 300 bps and scale up, so the timeouts reflected that. When I designed PPP over ISDN, added language to allow faster retransmission. When we designed IP/PPP/CDMA (IS-99) for cell phones, I was seriously concerned that it would not be competitive, as it only allowed 14.4 kbps when 28.8 kbps modems were becoming available. Turned out to be several times faster than ATT's CDPD offering.... Like many of you, I started an ISP in 1994 with a 56 kbps uplink, and only 6 local customers.... The routers were in a bathroom over the garage.

Imagine the racket! Is anyone connected with PPP over OC3? I'm just curious. I don't have that sort of connection myself. I'm just on dumbass DOCSIS. My first connection was PPP over the analogue PSTN. On Tue, 28 Jan 2020 09:53:26 -0700 Paul Ebersman <list-nanog2@dragon.net> wrote:

wsimpson> When we first designed PPP in the late '80s to replace SLIP wsimpson> and SLFP, it was expected to run at 300 bps and scale up, so wsimpson> the timeouts reflected that. When I designed PPP over ISDN, wsimpson> added language to allow faster retransmission.

SLIP and PPP were quite... robust. Some UCB folks managed to get SLIP over tin can and string. Two acoustic coupler 150b modems, 2 8oz V8 cans and waxed cotton thread.

wsimpson> Like many of you, I started an ISP in 1994 with a 56 kbps wsimpson> uplink, and only 6 local customers.... The routers were in a wsimpson> bathroom over the garage.

Our first CA hub was in the janitor's closet at a now defunct computer company. We initially had problems with the janitors unplugging the router on weekends to plug in their floor buffers.

Ah, the good old days.

-- Large Hadron Collider <large.hadron.collider@gmx.com>

On Jan 28, 2020, at 10:53 AM, Paul Ebersman <list-nanog2@dragon.net> wrote:

wsimpson> When we first designed PPP in the late '80s to replace SLIP wsimpson> and SLFP, it was expected to run at 300 bps and scale up, so wsimpson> the timeouts reflected that. When I designed PPP over ISDN, wsimpson> added language to allow faster retransmission.

SLIP and PPP were quite... robust. Some UCB folks managed to get SLIP over tin can and string. Two acoustic coupler 150b modems, 2 8oz V8 cans and waxed cotton thread.

I remember a bit of those days as well. Not working with them but seeing acoustic coupler modems in action. In the late 80s when my grandfather was mayor of a small town in Minnesota, he had some kind of little terminal in his basement with an acoustic coupler modem. It was so he could add messages to a city TV station in his official mayoral capacity. I was probably 8 or 9 at the time. My brother and I thought it was really, really cool when he showed us how it worked by putting our names on TV. He put some little “welcome to my grandkids from Nebraska” message that, to a 9 year old in the 1980s was awesome to see. -Andy

I had a USR 2400 baud external modem. Local ISP offered PPP service as well. We also had a few BBS's in the area of which I ran two of them. On Mon, Jan 27, 2020 at 10:31 AM Daniel Seagraves < dseagrav@humancapitaldev.com> wrote:

...

On Jan 24, 2020, at 5:26 PM, Ben Cannon <ben@6by7.net> wrote:

I started what became 6x7 with a 64k ISDN line. And 9600 baud modems…

Hayes Smartmodem here, 1200 baud. Local BBS offered PPP service.

When I got my first sysadmin job, $work had a T1 and it felt like more speed than was fair…

My first internet connection was some generic 2400baud.I had software support for MNP 5, which probably claims speeds up to 9600 bps? {perfomance in the lab with pretty cooperative factors like noise when squirrels eat through the protective coatings, and then chew up the actual wire, and at least compromise the protection. Oh anyway tl;dr, I found a couple modem numbers that weren't listed in the company publications, of course. they suppported MNP5 on three dialup lines. All totally spoke IP and didn't require authentication accounts, (things were pretty new for them, I'think) so I'd periodically call the other numbers in the list an see if it was crowded. I mostly did that because it obviously meant for general use but some department or whatever had need. So yeah, free 9600bps. at least people understood using words instead of having to slap the words on an image, which isn't so appealing, while the noise:signal ratio was the same, the total of each was much smaller. being concise was often perceived a too short an answer, of the go away kind. The slowest part was downloading the install cd images, and you had to make you weren't trying to use it for anything else even so, hah I finally lucked out. had an employer who wanted everyone in operational duties to be able to respond faster in emergencies, and actually did it! So then I graduated to...iDSL! yes, that's right, at the time I was too far from the box for regular DSL. So, yeah it cost more, around 150/month I think, maybe more, maybe less. but now I was up tp 128kbps bidirectional! okay, so it wasn't great for moving lots of data to my home resources, but it was enough to support the plenty of mail coming through, including some mailing lists for personal interests. And of course, people would email porn. fortunately, postfix is quite good at noticing the gargantuan files, and if they don't work the first time, fall pretty fow down the priorities, so, i guess it worked since that was a very different time Then I move to a new place in SF. for some reason I can't get anything telco-based to install, so I finally turned to Comcast, first Residential, Buisiness internet, which costs more for slower rates you probably can't count on get, but it'll be close. They've turned out to be a lot more clueful than the folks on the Residential service. Apparently, I got in at a good time, because as I said I need it for work and need fixed IP the VPN to autoconnect properly, asked me to list off the server roles for 5 addresses instead of just the one, so it was easy to justfity 5 instead of just getting the one fixed it. I believe they've gotten more stringent as IPv4 address, even small /29s So, it was cheap, why not try it? It turns out that comcast business is an entirely different organIzation.The SLA They have a 4 our onsite for someone to show up and say they need to order a part, they're often pretty prepared by incident I provided so have a guess where to look. There's a separate support number only business class customers are allowed to us, so which means threre' usually not much of a wait, if any. Because BC is presented as a complete package, the reason they make you rent whichever cablemodemrouterfirewall they've got their custom firmware on. If they can determine that it's your box, they'll make sure the truck has one. Of they need to be onside trouble shoot, they're quite competent at calling in what's needed, and most of what they'll have is where they're stationed. For work internet connections, I've long since track. probably all ofthem, though i'd never be am exact who used what. I do recall one interview with a candidate, and he asked about our network topology for whatever reason. I happili obliged, 2Gb We in the US have gotten used to other many mountries ahead of us for gigabit to the home. We should do it, but in all honesty, having 57Mbps down and 12 up, I havent been found my bandwidth seems fine. Usually looks like either another exchange router had a problem and the remaining routes are sitll converging with the new reality of how to transit traffic. In fact, come to think of it, i've had exactly ONE ticket call for a railed router. The was wsa somewhere outsie. ANyway, 57dn12up may not be enough for what you guys do on the internet with so much data may need, but it's still overkill for probably most of the customers who need time to figure out how to use the computer again. The main reason for increased capactiy all they way to plugs on the wall and some kind of WiFI in there someone Oh, as long giving kudos to comcast business, for those with a lot of traffic to comcast users, their peering rates are much more affordable. That's probably becase they get to make money on both ends of the traffic. since they've already got networks spread out to do their stuff peering, so worthwhile.They're nice guys to work with, too And of course, there is the fact that traffic will always grow to exceed capacity

On January 24, 2020 at 16:59 list-nanog2@dragon.net (Paul Ebersman) wrote:

bzs> When we, The World, first began allowing the general public onto bzs> the internet in October 1989 we actually had a (mildly shared*) T1 bzs> (1.544mbps) UUNET link. So not so bad for the time. Dial-up bzs> customers shared a handful of 2400bps modems, we still have them.

The World was also our (UUNET) Boston hub. And at that time, cross-country core backbone links were T1. We all thought the NSF T3 backbone was a government boon-doggle. :)

Those links were nailed up in the common closet not on 66 blocks but basically boards with bolts and quarter-sized thumb nuts, that was New England Telephone's (NET) demarc not our idea, it worked. One day working with a phone guy I jokingly remarked that's some old looking stuff, did Alexander Graham Bell put it in? He looked at me and said "possibly, Bell founded New England Telephone and would've helped on a job like this". The building was 1898. -- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

Once upon a time, Hugo Slabbert <hugo@slabnet.com> said:

...

This just follows the same rules as networks have always seemed to; If you build it, they will come, and you'll have to build more. :)

https://en.m.wikipedia.org/wiki/Induced_demand

Yep, just like your disk space requirements will always grow to 110% of available space. I get annoyed when I'm chatting with friends, waiting to play some game we decided to download, and it's ONLY downloading at 300 megabits per second! :P -- Chris Adams <cma@cmadams.net>

Game updates are generally compressed chunks and the client does live decompression on the data. As such, insufficient CPU or IO performance will result in lower overall speeds, since it can't keep up with the incoming stream of data. Regards, Filip On 1/23/20 9:11 PM, Tom Deligiannis wrote:

I get annoyed when I'm chatting with friends, waiting to play some game we decided to download, and it's ONLY downloading at 300 megabits per second! :P

In this scenario, which mechanism controls the download speed? I hear many users complain that their gigabit internet connection is not maxing out and the update is taking forever. I would never expect a gigabit internet connection to be saturated during a game update, but I'm curious how the throttling works.

Thanks.

Except the CDN providing the content did not anticipate this type of influx (How come I am not sure probably more concerned about new business revenue and not thinking about the backend infrastructure) and has pushed over costly peers for most of us. BTW, we are still waiting for our PNIs with them. Very frustrating... Erich Kaiser The Fusion Network On Thu, Jan 23, 2020 at 11:45 AM Hugo Slabbert <hugo@slabnet.com> wrote:

...

This just follows the same rules as networks have always seemed to; If you build it, they will come, and you'll have to build more. :)

https://en.m.wikipedia.org/wiki/Induced_demand

:-)

On Thu., Jan. 23, 2020, 09:40 Tom Beecher <beecher@beecher.cc> wrote:

...

I think this is a tribute to how we’ve built and upgraded networks for

...

capacity and speed.

I think it's spot on.

In years past it made more sense to distribute smaller , incremental patches. More work on the software side, but it was likely a better option than getting blasted on Twitter because "OMG I WANT TO PLAY AND MY DOWNLOAD IS TAKING 8 HOURS".

This just follows the same rules as networks have always seemed to; If you build it, they will come, and you'll have to build more. :)

On Thu, Jan 23, 2020 at 11:57 AM Jared Mauch <jared@puck.nether.net> wrote:

...

...

On Jan 23, 2020, at 11:52 AM, Valdis Klētnieks < valdis.kletnieks@vt.edu> wrote:

On Thu, 23 Jan 2020 17:13:15 +0100, Bryan Holloway said:

...

Game releases are hardly a new thing, but these last two events seem to be almost an order of magnitude higher than what we're used to (at least on our predominantly eyeball network.)

Any thoughts from the community? We're taking steps to accommodate, but from a capacity-planning perspective, this seems non-linear to me.

Be prepared for an entire new world of hurt this holiday season. Sony has already confirmed that PS5 releases will ship on 100Gbyte blu-ray disks. Which means that download sizes will be comparable…

There’s also the “we will stream you all the data things” I keep hearing about like the Consoles without discs or some other thing I can’t remember the name of.

I think this is a tribute to how we’ve built and upgraded networks for capacity and speed.

- Jared

Thanks Hugo, very interesting. Induced demand. Someone said recently… they’ve seen that no matter how much bandwidth you give a customer, they will eventually figure out how to use it. (whether they realize it or not… I guess it just happens) -Aaron From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Hugo Slabbert Sent: Thursday, January 23, 2020 11:44 AM To: Tom Beecher Cc: NANOG list Subject: Re: akamai yesterday - what in the world was that

This just follows the same rules as networks have always seemed to; If you build it, they will come, and you'll have to build more. :)

https://en.m.wikipedia.org/wiki/Induced_demand :-) On Thu., Jan. 23, 2020, 09:40 Tom Beecher <beecher@beecher.cc> wrote: I think this is a tribute to how we’ve built and upgraded networks for capacity and speed. I think it's spot on. In years past it made more sense to distribute smaller , incremental patches. More work on the software side, but it was likely a better option than getting blasted on Twitter because "OMG I WANT TO PLAY AND MY DOWNLOAD IS TAKING 8 HOURS". This just follows the same rules as networks have always seemed to; If you build it, they will come, and you'll have to build more. :) On Thu, Jan 23, 2020 at 11:57 AM Jared Mauch <jared@puck.nether.net> wrote:

On Jan 23, 2020, at 11:52 AM, Valdis Klētnieks <valdis.kletnieks@vt.edu> wrote:

On Thu, 23 Jan 2020 17:13:15 +0100, Bryan Holloway said:

...

Game releases are hardly a new thing, but these last two events seem to be almost an order of magnitude higher than what we're used to (at least on our predominantly eyeball network.)

Any thoughts from the community? We're taking steps to accommodate, but from a capacity-planning perspective, this seems non-linear to me.

Be prepared for an entire new world of hurt this holiday season. Sony has already confirmed that PS5 releases will ship on 100Gbyte blu-ray disks. Which means that download sizes will be comparable…

There’s also the “we will stream you all the data things” I keep hearing about like the Consoles without discs or some other thing I can’t remember the name of. I think this is a tribute to how we’ve built and upgraded networks for capacity and speed. - Jared

There is probably a "law" enshrined somewhere: Bandwidth is like closet space, demand will always manage to exceed capacity. Gene On 1/24/20 6:52 AM, Aaron Gould wrote:

Thanks Hugo, very interesting.  Induced demand.  Someone said recently… they’ve seen that no matter how much bandwidth you give a customer, they will eventually figure out how to use it. (whether they realize it or not… I guess it just happens)

-Aaron

*From:*NANOG [mailto:nanog-bounces@nanog.org] *On Behalf Of *Hugo Slabbert *Sent:* Thursday, January 23, 2020 11:44 AM *To:* Tom Beecher *Cc:* NANOG list *Subject:* Re: akamai yesterday - what in the world was that

...

 This just follows the same rules as networks have always seemed to; If you build it, they will come, and you'll have to build more. :)

https://en.m.wikipedia.org/wiki/Induced_demand

:-)

On Thu., Jan. 23, 2020, 09:40 Tom Beecher <beecher@beecher.cc> wrote:

I think this is a tribute to how we’ve built and upgraded networks for capacity and speed.

I think it's spot on.

In years past it made more sense to distribute smaller , incremental patches. More work on the software side, but it was likely a better option than getting blasted on Twitter because "OMG I WANT TO PLAY AND MY DOWNLOAD IS TAKING 8 HOURS".

This just follows the same rules as networks have always seemed to; If you build it, they will come, and you'll have to build more. :)

On Thu, Jan 23, 2020 at 11:57 AM Jared Mauch <jared@puck.nether.net <mailto:jared@puck.nether.net>> wrote:

> On Jan 23, 2020, at 11:52 AM, Valdis Klētnieks <valdis.kletnieks@vt.edu <mailto:valdis.kletnieks@vt.edu>> wrote: > > On Thu, 23 Jan 2020 17:13:15 +0100, Bryan Holloway said: > >> Game releases are hardly a new thing, but these last two events seem to >> be almost an order of magnitude higher than what we're used to (at least >> on our predominantly eyeball network.) >> >> Any thoughts from the community? We're taking steps to accommodate, but >> from a capacity-planning perspective, this seems non-linear to me. > > Be prepared for an entire new world of hurt this holiday season. Sony has already > confirmed that PS5 releases will ship on 100Gbyte blu-ray disks.  Which means that > download sizes will be comparable…

There’s also the “we will stream you all the data things” I keep hearing about like the Consoles without discs or some other thing I can’t remember the name of.

I think this is a tribute to how we’ve built and upgraded networks for capacity and speed.

- Jared

-- Gene LeDuc | A little learning is a dangerous thing, Technology Security | but a lot of ignorance is just as bad. San Diego State University | --Bob Edwards

Shouldn't game patches like this be released overnight during off-peak hours? Fortnite releases their updates around 3 or 4am when most ISP's networks are at their lowest utilization. It seems somewhat reckless to release such a large patch during awake hours. On Sat, Jan 25, 2020, 12:08 PM Brandon Jackson via NANOG <nanog@nanog.org> wrote:

"Call of Duty: Modern Warfare fragged our business VOIP: US ISP blames outage on smash-hit video game rush This is Windstream, going dark..." https://www.theregister.co.uk/2020/01/23/windstream_fvoip_outage/

Apparently not everyone came out unscathed.

---------------------------------- Brandon Jackson bjackson@napshome.net

On Thu, Jan 23, 2020 at 10:14 AM Aaron Gould <aaron1@gvtc.com> wrote:

...

My gosh, what in the word was that coming out of my local Akamai aanp servers yesterday !? starting at about 12:00 noon central time lasting several hours ?

-Aaron

That’s what she said -- J. Hellenthal The fact that there's a highway to Hell but only a stairway to Heaven says a lot about anticipated traffic volume.

On Jan 25, 2020, at 13:42, Alistair Mackenzie <magicsata@gmail.com> wrote:

 Off-peak hours are on-peak somewhere else in the world.

...

On Sat, Jan 25, 2020 at 7:37 PM Darin Steffl <darin.steffl@mnwifi.com> wrote: Shouldn't game patches like this be released overnight during off-peak hours? Fortnite releases their updates around 3 or 4am when most ISP's networks are at their lowest utilization. It seems somewhat reckless to release such a large patch during awake hours.

...

On Sat, Jan 25, 2020, 12:08 PM Brandon Jackson via NANOG <nanog@nanog.org> wrote: "Call of Duty: Modern Warfare fragged our business VOIP: US ISP blames outage on smash-hit video game rush This is Windstream, going dark..." https://www.theregister.co.uk/2020/01/23/windstream_fvoip_outage/

Apparently not everyone came out unscathed.

---------------------------------- Brandon Jackson bjackson@napshome.net

...

On Thu, Jan 23, 2020 at 10:14 AM Aaron Gould <aaron1@gvtc.com> wrote: My gosh, what in the word was that coming out of my local Akamai aanp servers yesterday !? starting at about 12:00 noon central time lasting several hours ?

-Aaron

There is a major update that has released today, how's everything looking for everyone? Tom On Thu, Jan 23, 2020 at 10:14 AM Aaron Gould <aaron1@gvtc.com> wrote:

My gosh, what in the word was that coming out of my local Akamai aanp servers yesterday !? starting at about 12:00 noon central time lasting several hours ?

-Aaron

On Sat, Jan 25, 2020 at 2:02 PM Tom Deligiannis <tom.deligiannis@gmail.com> wrote:

Shouldn't game patches like this be released overnight during off-peak

...

hours? Fortnite releases their updates around 3 or 4am when most ISP's networks are at their lowest utilization. It seems somewhat reckless to release such a large patch during awake hours.

I can't speak for PS4 and PC, but xbox does have a setting to keep the console in a state that allows the device to download updates when it is 'off' but I've noticed that the consoles don't always follow that rule and the update starts downloading when the user powers on the console, which is usually during peak hours. If this worked properly, it would be great since most of the updates would download while users were at school, working, sleeping, etc.

On Sat, Jan 25, 2020 at 1:36 PM Darin Steffl <darin.steffl@mnwifi.com> wrote:

...

Shouldn't game patches like this be released overnight during off-peak hours? Fortnite releases their updates around 3 or 4am when most ISP's networks are at their lowest utilization. It seems somewhat reckless to release such a large patch during awake hours.

On Sat, Jan 25, 2020, 12:08 PM Brandon Jackson via NANOG <nanog@nanog.org> wrote:

...

"Call of Duty: Modern Warfare fragged our business VOIP: US ISP blames outage on smash-hit video game rush This is Windstream, going dark..." https://www.theregister.co.uk/2020/01/23/windstream_fvoip_outage/

Apparently not everyone came out unscathed.

---------------------------------- Brandon Jackson bjackson@napshome.net

On Thu, Jan 23, 2020 at 10:14 AM Aaron Gould <aaron1@gvtc.com> wrote:

...

My gosh, what in the word was that coming out of my local Akamai aanp servers yesterday !? starting at about 12:00 noon central time lasting several hours ?

-Aaron

Any word on what the update was for? It caused quite a jump in traffic on our network. On Tue, Feb 11, 2020, 19:06 Jared Mauch <jared@puck.nether.net> wrote:

Looking good from my perspective. Let me know if we are causing you pain and let's see what can be done to improve.

I'm here in SF if you are at nanog.

Sent from my iCar

...

On Feb 11, 2020, at 3:42 PM, Tom Deligiannis <tom.deligiannis@gmail.com> wrote:

There is a major update that has released today, how's everything looking for everyone?

Any word on what the update was for? It caused quite a jump in traffic on our network.

On twitter "68 GB" was trending https://twitter.com/search?q=%2268%20GB%22&src=trend_click Kind regards, Job

Yup, Call of Duty update, 68GB on xbox platform. Tom On Tue, Feb 11, 2020 at 10:26 PM Aaron Gould <aaron1@gvtc.com> wrote:

Huge! Big as ever. My aanp links are (were) pegged, seriously. I will be contacting Akamai about lighting up an additional 10 gig link to my local clusters. Started at 12 noon central… still going pretty heavily. Game/update release ?

-Aaron

*From:* Tom Deligiannis [mailto:tom.deligiannis@gmail.com] *Sent:* Tuesday, February 11, 2020 5:41 PM *To:* aaron1@gvtc.com *Cc:* Nanog@nanog.org *Subject:* Re: akamai yesterday - what in the world was that

There is a major update that has released today, how's everything looking for everyone?

Tom

Good point Bryan... With my single 10 gig pegged out for a few hours sustained, I guess it remains to be seen exactly how high that peak would go if I gave it more capacity -Aaron ----- Original Message ----- From: Bryan Holloway <bryan@shout.net> To: Nanog@nanog.org Sent: Wed, 12 Feb 2020 07:59:20 -0500 (EST) Subject: Re: akamai yesterday - what in the world was that Is 10G enough? ;) We just lit up several 100G Akamai links. Saved the day fo sho ... (this time.) On 2/11/20 8:26 PM, Aaron Gould wrote:

Huge!  Big as ever.  My aanp links are (were) pegged, seriously.  I will be contacting Akamai about lighting up an additional 10 gig link to my local clusters.  Started at 12 noon central… still going pretty heavily.  Game/update release ?

-Aaron

*From:*Tom Deligiannis [mailto:tom.deligiannis@gmail.com] *Sent:* Tuesday, February 11, 2020 5:41 PM *To:* aaron1@gvtc.com *Cc:* Nanog@nanog.org *Subject:* Re: akamai yesterday - what in the world was that

There is a major update that has released today, how's everything looking for everyone?

Tom

We saw a higher load overnight, a little bit of a spike last night, but really hard to tell overall with our traffic.  Updates were still going at 8am today.  We run a local/regional WISP. On 2/12/20 9:46 AM, Brandon Martin wrote:

On 2/11/20 6:41 PM, Tom Deligiannis wrote:

...

There is a major update that has released today, how's everything looking for everyone?

I run a couple distinct very small networks.  Both are transit-only with no direct peering or local caching and generally sub-gbps.

One set a new 1-min 95% record and did so by nearly 25% over its previous record.  The other matched its existing record.  The former thankfully has ample capacity, while the latter thankfully did so before primetime.

These are definitely going to be harder to manage than the primetime hump.  It would be nice if things could drop overnight to hopefully spread things out during the daytime lull some.  I understand that some people won't pick it up until they get home and turn on devices (which is often after school hours for these type of game updates), but spreading things out would be really nice especially for those of us without local caching.

On Wed, 12 Feb 2020 at 14:46, Brandon Martin <lists.nanog@monmotha.net> wrote:

It would be nice if things could drop overnight to hopefully spread things out during the daytime lull some.

Night-time for you is daytime for someone else. I agree that the folks pushing these massive data loads could be considerate of the networks they are running on. The steam preload thing is a great example. The content becomes available x weeks before launch and downloaded to your device at some point in that window. It then simply unlocked on launch day. I'm not sure how much incentive there is for people to implement this kind of system though. With the rise of CDN and global caching, people care less about the performance of their servers for content distribution as it just scales out. Dave

On 2/12/20 8:13 AM, Brandon Martin wrote:

It would be really nice if the major CDNs had virtual machines small network operators with very expensive regional transport costs could spin up.  Hit rate would be very low, of course, but the ability to grab some of these mass-market huge updates and serve them on the other end of the regional transport at essentially no extra cost would be great. I'm sure legal arrangements make that difficult, though.

My experience is that they want to see lots of traffic growth to stay interested. As companies get bigger the minimum bar to play keeps going up, and anyone below that bar is stuck relying on transit. Fall below the bar or don't show enough growth fast enough and they pull the resources away.

Once upon a time, Brandon Martin <lists.nanog@monmotha.net> said:

I guess what I'm looking for is a more "standard product". Load this VM, tell it your preference for upstream use vs. hit rate, let it announce some routes into your network, and you take what you get. If you need more, presumably you have the volume to back it up.

I think security is probably the sticking point for this. Content owners don't want anybody having direct access to their files, and as more content is distributed over HTTPS, content distributors don't wany anbody having access to their certificates. -- Chris Adams <cma@cmadams.net>

(as an aside) Some of this timed & controlled distribution (by the content originator) should be possible using IETF Content Delivery Network Interconnection (CDNI) standards - see https://datatracker.ietf.org/wg/cdni/documents/. The initial RFC 6707 provides some background - https://www.rfc-editor.org/rfc/rfc6707. But I think folks are correct that the issue may be more that a given gaming device was turned off at night (though no reason that device could not pre-cache the content from the source). In any case, there should be a better way to address this. The Internet will see more and more of these downloads and smoothing the impact out seems prudent for all involved. Jason On 2/12/20, 11:59 AM, "NANOG on behalf of Chris Adams" <nanog-bounces@nanog.org on behalf of cma@cmadams.net> wrote: I think security is probably the sticking point for this. Content owners don't want anybody having direct access to their files, and as more content is distributed over HTTPS, content distributors don't wany anbody having access to their certificates. -- Chris Adams <cma@cmadams.net>

Aren't most modern consoles on whether they're "on" or not? IE: It's not a full power up from a dead stop, 0 watts power usage. I'd think they'd be able to come out of sleep mode on their own, download the update, then go back to sleep. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Seth Mattinen" <sethm@rollernet.us> To: nanog@nanog.org Sent: Wednesday, February 12, 2020 1:42:21 PM Subject: Re: akamai yesterday - what in the world was that On 2/12/20 11:31, Livingood, Jason wrote:

But I think folks are correct that the issue may be more that a given gaming device was turned off at night (though no reason that device could not pre-cache the content from the source). In any case, there should be a better way to address this. The Internet will see more and more of these downloads and smoothing the impact out seems prudent for all involved.

Putting my end user hat on, I turn off all my consoles when I'm not using them, often for weeks. When I get home and it looks like I'll have time to play after dinner I'll turn one of them on and let it download/install. I don't really care that my off work and dinner times might not be convenient for my ISP to download giant files. I fully understand the ISP's perspective, but I'm not going to start leaving my consoles on 24x7. The way to address this used to be this thing called "physical media" that held games, but nowadays even when I have a game on disc it has to download at least one massive patch before it will play.

On 2/12/20 11:48, Josh Luthman wrote:

In low power state, usually standby, they're connected to the network and listen for requests to download a new title (bought online) or updates.  I know on the Xbox One side of things this feature is semi-off by default as it turns the HDD off to save power, but it's still in standby in the sense that it takes only a few seconds to get to a usable state.

They can shut down or sleep, it's user choice. Xbox has a setting for an "instant on" mode. I also had the option to check for updates, but when I went to use it yesterday it came up asking me to download a system update. And then after it installed that it wanted to download a giant update file for Halo. It's supposed to get updates on its own if you have both instant on and get updates enabled, but it didn't for whatever reason. On PS4 you choose if you want to turn it off or go into rest mode, but I usually choose off because if the power hiccups in the weeks between times I get to use it it yells that it wasn't shut down correctly and it doesn't self-reboot into rest mode. Even when it was in rest mode, when I went to start Overcooked (the only game my wife will co-op play with me) it too asked to download an update. So sure, they can, but it doesn't work reliably and when I have time to play *ow I'm going to tell it to download now without caring if it's not-my-problem peak time or not. And I'm sure I'm not alone in that sentiment. Again, speaking with my end user hat on.

Because the disks are shut off by default in standby mode. Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 On Wed, Feb 12, 2020 at 2:53 PM Tom Deligiannis <tom.deligiannis@gmail.com> wrote:

Aren't most modern consoles on whether they're "on" or not? IE: It's not

...

a full power up from a dead stop, 0 watts power usage.

I'd think they'd be able to come out of sleep mode on their own, download the update, then go back to sleep.

Xbox has this feature, but it doesn't work very well. A quick google search shows that many users have their consoles set to receive updates, but that feature doesn't seem to be working properly.

Tom

On Wed, Feb 12, 2020 at 1:46 PM Mike Hammett <nanog@ics-il.net> wrote:

...

Aren't most modern consoles on whether they're "on" or not? IE: It's not a full power up from a dead stop, 0 watts power usage.

I'd think they'd be able to come out of sleep mode on their own, download the update, then go back to sleep.

----- Mike Hammett Intelligent Computing Solutions <http://www.ics-il.com/> <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> Midwest Internet Exchange <http://www.midwest-ix.com/> <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> The Brothers WISP <http://www.thebrotherswisp.com/> <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> ------------------------------ *From: *"Seth Mattinen" <sethm@rollernet.us> *To: *nanog@nanog.org *Sent: *Wednesday, February 12, 2020 1:42:21 PM *Subject: *Re: akamai yesterday - what in the world was that

On 2/12/20 11:31, Livingood, Jason wrote:

...

But I think folks are correct that the issue may be more that a given gaming device was turned off at night (though no reason that device could not pre-cache the content from the source). In any case, there should be a better way to address this. The Internet will see more and more of these downloads and smoothing the impact out seems prudent for all involved.

Putting my end user hat on, I turn off all my consoles when I'm not using them, often for weeks. When I get home and it looks like I'll have time to play after dinner I'll turn one of them on and let it download/install. I don't really care that my off work and dinner times might not be convenient for my ISP to download giant files. I fully understand the ISP's perspective, but I'm not going to start leaving my consoles on 24x7.

The way to address this used to be this thing called "physical media" that held games, but nowadays even when I have a game on disc it has to download at least one massive patch before it will play.

My point was, the user has the option properly configured, but the console still isn't updating until the console is turned on. I'm not implying that it doesn't work, I'm simply stating that some users claim to have the options configured properly but that updates are still not downloading w/o user interaction. Tom On Wed, Feb 12, 2020 at 2:03 PM Josh Luthman <josh@imaginenetworksllc.com> wrote:

Because the disks are shut off by default in standby mode.

Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373

On Wed, Feb 12, 2020 at 2:53 PM Tom Deligiannis <tom.deligiannis@gmail.com> wrote:

...

Aren't most modern consoles on whether they're "on" or not? IE: It's

...

not a full power up from a dead stop, 0 watts power usage.

I'd think they'd be able to come out of sleep mode on their own, download the update, then go back to sleep.

Xbox has this feature, but it doesn't work very well. A quick google search shows that many users have their consoles set to receive updates, but that feature doesn't seem to be working properly.

Tom

On Wed, Feb 12, 2020 at 1:46 PM Mike Hammett <nanog@ics-il.net> wrote:

...

Aren't most modern consoles on whether they're "on" or not? IE: It's not a full power up from a dead stop, 0 watts power usage.

I'd think they'd be able to come out of sleep mode on their own, download the update, then go back to sleep.

----- Mike Hammett Intelligent Computing Solutions <http://www.ics-il.com/> <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> Midwest Internet Exchange <http://www.midwest-ix.com/> <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> The Brothers WISP <http://www.thebrotherswisp.com/> <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> ------------------------------ *From: *"Seth Mattinen" <sethm@rollernet.us> *To: *nanog@nanog.org *Sent: *Wednesday, February 12, 2020 1:42:21 PM *Subject: *Re: akamai yesterday - what in the world was that

On 2/12/20 11:31, Livingood, Jason wrote:

...

But I think folks are correct that the issue may be more that a given gaming device was turned off at night (though no reason that device could not pre-cache the content from the source). In any case, there should be a better way to address this. The Internet will see more and more of these downloads and smoothing the impact out seems prudent for all involved.

Putting my end user hat on, I turn off all my consoles when I'm not using them, often for weeks. When I get home and it looks like I'll have time to play after dinner I'll turn one of them on and let it download/install. I don't really care that my off work and dinner times might not be convenient for my ISP to download giant files. I fully understand the ISP's perspective, but I'm not going to start leaving my consoles on 24x7.

The way to address this used to be this thing called "physical media" that held games, but nowadays even when I have a game on disc it has to download at least one massive patch before it will play.

On 2020-02-12, at 20:45, Mike Hammett <nanog@ics-il.net> wrote:

Aren't most modern consoles on whether they're "on" or not? IE: It's not a full power up from a dead stop, 0 watts power usage.

https://www.anandtech.com/show/7528/the-xbox-one-mini-review-hardware-analys... says two-digit standby power (which they say is needed for background updating). At least in Germany, nobody sane will leave the thing in that expensive mode (a watt-year is $3 here). Switchable extension power cords are being actively marketed here for these power hogs. Grüße, Carsten

Strictly out of interest, I wanted to ask earlier if this irresponsible way of causing insane, instant, bandwidth demands is breaking anything on the ISP/CDN side or even the console owner ?! Or is it just an interesting phenomenon that is handled without a sweat. Does it break the buck in anyway? The thread started with bandwidth surges and now power hogging is mentioned, I wonder what else might happen as a side effect to a small number of console/gaming companies not taking a direct responsibility in how they release large updates in a way that is not organized or scheduled but is rough and abrupt. ~A On Thu, Feb 13, 2020 at 3:33 AM Tom Beecher <beecher@beecher.cc> wrote:

The discussion about what the consoles can or can not do is honestly not solving anything.

Saying that the consoles should or should not be doing a thing is simply trying to throw the problem to someone else.

On Wed, Feb 12, 2020 at 15:40 Carsten Bormann <cabo@tzi.org> wrote:

...

On 2020-02-12, at 20:45, Mike Hammett <nanog@ics-il.net> wrote:

...

Aren't most modern consoles on whether they're "on" or not? IE: It's

not a full power up from a dead stop, 0 watts power usage.

https://www.anandtech.com/show/7528/the-xbox-one-mini-review-hardware-analys... says two-digit standby power (which they say is needed for background updating). At least in Germany, nobody sane will leave the thing in that expensive mode (a watt-year is $3 here). Switchable extension power cords are being actively marketed here for these power hogs.

Grüße, Carsten

On Thu, 13 Feb 2020 09:39:09 -0800, Ahmed Borno said:

The thread started with bandwidth surges and now power hogging is mentioned, I wonder what else might happen as a side effect to a small number of console/gaming companies not taking a direct responsibility in how they release large updates in a way that is not organized or scheduled but is rough and abrupt.

And I'd not expect it to improve - many of the game producers are leaving the "incremental patch" mode to a "just ship the current image of the whole damned thing", because for them it's cheaper to just push out a single updated image than try to build different images for upgrading from different current levels. After all - it's not like *they* are going to feel the pain of a single 106G upload, it's somebody else who feels the pain of 5 million downloads of a 106G image refresh. Economists call this sort of thing an "externality".

All, There has been some initial discussions about beyond 400G for Ethernet. It would be interesting to better understand how often this problem is now occurring - because I would imagine the problem is only going to get worse as the "binary blob" blobs out, which will only stress networks more. Regards John -----Original Message----- From: NANOG <nanog-bounces@nanog.org> On Behalf Of tim@pelican.org Sent: Friday, February 14, 2020 4:46 AM To: NANOG list <nanog@nanog.org> Subject: Re: akamai yesterday - what in the world was that On Friday, 14 February, 2020 09:17, "Valdis Klētnieks" <valdis.kletnieks@vt.edu> said:

After all - it's not like *they* are going to feel the pain of a single 106G upload, it's somebody else who feels the pain of 5 million downloads of a 106G image refresh.

Economists call this sort of thing an "externality".

I must admit, I'm blissfully unaware of CDN commercials, but I'd have expected that if I give a CDN my binary 100G binary blob and six people download it, I'd be billed a different amount to if six million people download it - and similarly if that blob is 1G vs 100G. I guess I'm asking if there's an underlying problem with the model here, or if it's just the details of the numbers that are "wrong" in encouraging / discouraging certain behaviou Regards, Tim.

...

...

After all - it's not like *they* are going to feel the pain of a single 106G upload, it's somebody else who feels the pain of 5 million downloads of a 106G image refresh.

Economists call this sort of thing an "externality".

I must admit, I'm blissfully unaware of CDN commercials, but I'd have expected that if I give a CDN my binary 100G binary blob and six people download it, I'd be billed a different amount to if six million people download it - and similarly if that blob is 1G vs 100G.

I guess I'm asking if there's an underlying problem with the model here, or if it's just the details of the numbers that are "wrong" in encouraging / discouraging certain behaviours.

Regards, Tim.

I just wish "they" would remember that their ultimate customers don’t usually have 10G pipes - they have 6M and 10M pipes that may take hours, if not days, to download one of these mega blobs.

Uhhhh, it is 2020, not 2010. 100M, 200M, 400M or 1G is increasingly common for home broadband. I’ve got 400M at home, could get 1G fiber for less than $100 if I wanted it, and I’m in your average, run-of-the-mill Midwest city. -Andy

I know people who have 300 mb all the way up to gigabit in their home, they still struggled with the update since the bottleneck wasn't the speed of their internet connection. Tom On Fri, Feb 14, 2020 at 12:41 PM Jeff Shultz <jeffshultz@sctcweb.com> wrote:

Sure, some of them can get it. Some still have DSL because we haven't gotten fiber that far out yet. Or they're in a rental/apartment where the landlord won't let us put fiber.

Or some just don't want to pay for it.

On Fri, Feb 14, 2020 at 10:26 AM Andy Ringsmuth <andy@andyring.com> wrote:

...

...

...

...

After all - it's not like *they* are going to feel the pain of a

single 106G upload, it's somebody else who feels the pain of 5 million downloads of a 106G image

...

...

...

...

refresh.

Economists call this sort of thing an "externality".

I must admit, I'm blissfully unaware of CDN commercials, but I'd have expected that if I give a CDN my binary 100G binary blob and six people download it, I'd be billed a different amount to if six million people download it - and similarly if that blob is 1G vs 100G.

I guess I'm asking if there's an underlying problem with the model here, or if it's just the details of the numbers that are "wrong" in encouraging / discouraging certain behaviours.

Regards, Tim.

I just wish "they" would remember that their ultimate customers don’t usually have 10G pipes - they have 6M and 10M pipes that may take hours, if not days, to download one of these mega blobs.

Uhhhh, it is 2020, not 2010. 100M, 200M, 400M or 1G is increasingly common for home broadband. I’ve got 400M at home, could get 1G fiber for less than $100 if I wanted it, and I’m in your average, run-of-the-mill Midwest city.

-Andy

-- Jeff Shultz

-- Like us on Social Media for News, Promotions, and other information!!

<https://www.facebook.com/SCTCWEB/> <https://www.instagram.com/sctc_503/> <https://www.yelp.com/biz/sctc-stayton-3> <https://www.youtube.com/c/sctcvideos>

_**** This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. ****_

On 2/14/20 1:24 PM, Andy Ringsmuth wrote:

Uhhhh, it is 2020, not 2010. 100M, 200M, 400M or 1G is increasingly common for home broadband. I’ve got 400M at home, could get 1G fiber for less than $100 if I wanted it, and I’m in your average, run-of-the-mill Midwest city.

And there are plenty of places in the USA with literally no wireline connectivity aside from POTS who rely on LTE or local WISPs to deliver bits, and they're doing well to give them 100M (capped or pay-by-the-GB on LTE, typically) and often much less. There's also plenty of places where wireline speeds max out around 50-75Mbps in practice, even if the local MSO says you can get more. The divide keeps getting bigger. -- Brandon Martin

On 14/Feb/20 20:24, Andy Ringsmuth wrote:

Uhhhh, it is 2020, not 2010. 100M, 200M, 400M or 1G is increasingly common for home broadband. I’ve got 400M at home, could get 1G fiber for less than $100 if I wanted it, and I’m in your average, run-of-the-mill Midwest city.

Gaming does happen in most parts of the world outside the U.S., too; even if I know that rural U.S. would struggle with high-speed access the same way a developing country would, if not worse. My point is, the world has not yet converged on Gigabit capability into the home. And that goes for the distribution network too. But that's not going to stop the world from gaming. Mark.

Just as a heads-up that if those previous two patches caused you some strain, keep an eye tomorrow: https://blog.activision.com/call-of-duty/2020-03/Introducing-a-game-changing... A one-time early access will give Modern Warfare owners the ability to

download Warzone at 8AM PDT. **For Modern Warfare owners who are current and have the most recent title updates, the download will be a 18-22GB**. Once download is complete, Modern Warfare owners will “unlock” the Warzone panel, which was previously classified, and can enter the lobby and play.

For non-owners of the full version of Call of Duty: Modern Warfare, Warzone will be available to download free in respective first-party stores starting as early as 12 PM PDT. Head to the store and search for ‘Warzone.’ Choose Call of Duty: Warzone to download. **Warzone is a 83-101GB download for new, free-to-play users**.

-- Hugo Slabbert | email, xmpp/jabber: hugo@slabnet.com pgp key: B178313E | also on Signal On Thu, Feb 13, 2020 at 3:33 AM Tom Beecher <beecher@beecher.cc> wrote:

The discussion about what the consoles can or can not do is honestly not solving anything.

Saying that the consoles should or should not be doing a thing is simply trying to throw the problem to someone else.

On Wed, Feb 12, 2020 at 15:40 Carsten Bormann <cabo@tzi.org> wrote:

...

On 2020-02-12, at 20:45, Mike Hammett <nanog@ics-il.net> wrote:

...

Aren't most modern consoles on whether they're "on" or not? IE: It's

not a full power up from a dead stop, 0 watts power usage.

https://www.anandtech.com/show/7528/the-xbox-one-mini-review-hardware-analys... says two-digit standby power (which they say is needed for background updating). At least in Germany, nobody sane will leave the thing in that expensive mode (a watt-year is $3 here). Switchable extension power cords are being actively marketed here for these power hogs.

Grüße, Carsten

On 3/9/20 11:02 PM, Keith Medcalf wrote:

Warzone is a 83-101GB download for new, free-to-play users*.

And I remember the days when that would have taken 10 and a half years to download and consumed 56,000 floppy diskettes.

My, how times have changed!

"Never underestimate the bandwidth of a station-wagon full of tapes."

Wow, yeah, my Akamai servers are again, hitting all time highs… one cache hit up to ~30 gig… been ramping up and down since this morning around 9 or 10 a.m. central time. Here’s a strange thing though, around 14:45 – 15:30, I got massive outbound on my internet connection (~20 gbps), and I never send that much out to the internet -Aaron

On Mar 10, 2020, at 6:14 PM, Bryan Holloway <bryan@shout.net> wrote:

We hit over 40G on one of our PNIs.

Currently, however, I'm trying to figure out why we're still seeing a significant amount of traffic over transit when we have PNIs at the same locations ...

I've reached out to Akamai, but I haven't heard anything back yet. I'm sure they're busy …

Yes another high traffic day. I’ll get with you in private about your specific situation. As always, there’s bumps but if we can improve things I want to do that. - Jared

Both the Call of Duty free to play battle royal game (Blizzard) and a massive ~80GB update on Rainbow Six Siege (Steam) might both add to those traffic peaks. Good drop in players in a short period of time while 60k-ish concurrent people downloaded that last one: https://steamcharts.com/app/359550#48h Jeroen Wunnink Sr. Manager - Integration Engineering www.gtt.net<http://www.gtt.net/> [id:image001.png@01D37331.D1301F60] From: NANOG <nanog-bounces+jeroen.wunnink=gtt.net@nanog.org> on behalf of "Kaiser, Erich" <erich@gotfusion.net> Date: Tuesday, 10 March 2020 at 21:18 To: Bryan Holloway <bryan@shout.net> Cc: North American Network Operators' Group <nanog@nanog.org> Subject: Re: akamai yesterday - what in the world was that It started about an half hr ago... Erich Kaiser The Fusion Network erich@gotfusion.net<mailto:erich@gotfusion.net> On Mon, Mar 9, 2020 at 5:08 PM Bryan Holloway <bryan@shout.net<mailto:bryan@shout.net>> wrote: On 3/9/20 11:02 PM, Keith Medcalf wrote:

Warzone is a 83-101GB download for new, free-to-play users*.

And I remember the days when that would have taken 10 and a half years to download and consumed 56,000 floppy diskettes.

My, how times have changed!

"Never underestimate the bandwidth of a station-wagon full of tapes."

On 2/12/20 8:36 AM, Aaron Gould wrote:

Netflix oca has it figured out, as my fill windows is during off-peak time, 2 a.m. - 6 am. and I think it's also configurable in the oca portal.

It's not fill, it's that people don't turn on their xbox or whatever until after they get home from work and only then does it start downloading. Multiply that by 1000 people getting home from work around the same time.

The wheels of bureaucracy are certainly a problem. The largest peer on our local exchange couldn't even get Akamai to complete a peering turn up because whoever was working on the ticket on the Akamai side got stuck on trying to set up the wrong location. And then months pass, it never got resolved, and then they decided to pull the cache. Akamai had one hand failing to set up new peers and the other hand saying why aren't there more peers, and the two hands never know what the other is doing.

On 2/12/20 10:02, Jared Mauch wrote:

When you see this please raise it to my attention. I can't promise a resolution but will promise clarity in what is going on.

This was in May 2019 so what's done is done at this point, but I will forward you the email offlist.

IIRC, game consoles are always on, whether they're "on" or not. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Tom Beecher" <beecher@beecher.cc> To: "Darin Steffl" <darin.steffl@mnwifi.com> Cc: Nanog@nanog.org Sent: Saturday, January 25, 2020 5:13:19 PM Subject: Re: akamai yesterday - what in the world was that Not everybody leaves their console/PC on 24/7 so that they would pull the patch at 3am local even if that’s when it was released. It’s far from reckless. It’s not the game companies job to make sure the network works. That’s our job. On Sat, Jan 25, 2020 at 14:37 Darin Steffl < darin.steffl@mnwifi.com > wrote: Shouldn't game patches like this be released overnight during off-peak hours? Fortnite releases their updates around 3 or 4am when most ISP's networks are at their lowest utilization. It seems somewhat reckless to release such a large patch during awake hours. On Sat, Jan 25, 2020, 12:08 PM Brandon Jackson via NANOG < nanog@nanog.org > wrote: <blockquote> "Call of Duty: Modern Warfare fragged our business VOIP: US ISP blames outage on smash-hit video game rush This is Windstream, going dark..." https://www.theregister.co.uk/2020/01/23/windstream_fvoip_outage/ Apparently not everyone came out unscathed. ---------------------------------- Brandon Jackson bjackson@napshome.net On Thu, Jan 23, 2020 at 10:14 AM Aaron Gould < aaron1@gvtc.com > wrote: <blockquote> My gosh, what in the word was that coming out of my local Akamai aanp servers yesterday !? starting at about 12:00 noon central time lasting several hours ? -Aaron </blockquote> </blockquote>