Peering and Caching for Epic Games, Fortnite, et al
We run a healthy-sized ISP (say, 2.5M households, plus enterprise, etc ) and we really, REALLY want to make sure our users have an amazing experience when downloading the neverending Fortnite/Spacequest/Blizzard/DigDug updates that run down our pipes. Would love to hear from others about how they're peering and caching -- not having the level of success I'd want with the typical "aggregators" (they know who they are ) and would really like to link to the source even if it means trenching through the core of the Earth... Would love pointers, names, or any leads, on or off list. Thanks Jose L. Rodriguez CTO, Totalplay
Hey! I know at least for Valve, you can set up a Steam Caching Server, and say do the top 100 games or whatnot and update it every week or so, that might put you in the right direction. LTT did a good video on this a few years ago, and they also posted this guide on their forums to assist with getting one set up. https://linustechtips.com/topic/962655-steam-caching-tutorial/ <https://linustechtips.com/topic/962655-steam-caching-tutorial/> I’ve set up two or three for various larger scale LAN parties, as well as run one at home. Feel free to ping me offline if you want a hand or talking it over! Thanks Much, John Waters Chief Architect, Capitol Hosting Solutions, LLC john@capitolsolutions.cloud <mailto:john@capitolsolutions.cloud> https://chs.gg
On Mon, 22 Mar 2021 20:13:46 -0600, Jose Luis Rodriguez said:
experience when downloading the neverending Fortnite/Spacequest/Blizzard/DigDug updates that run down our pipes. Would
know who they are ) and would really like to link to the source even if it means trenching through the core of the Earth...
How many trenches to various points are you willing to dig? (And I don't think you've even mentioned some of the *large* games out there now, with 100G+ releases...)
For an industry (online gaming) with the most "sensitive" customers to latency, packet loss, throughput, etc., the online gaming industry is terrible at peering. There are a few shining examples of what you should do, but then the rest is just content with buying transit from one, two, three players and calling it a day. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Jose Luis Rodriguez" <jlrodriguez@gmail.com> To: nanog@nanog.org Sent: Monday, March 22, 2021 9:13:46 PM Subject: Peering and Caching for Epic Games, Fortnite, et al We run a healthy-sized ISP (say, 2.5M households, plus enterprise, etc ) and we really, REALLY want to make sure our users have an amazing experience when downloading the neverending Fortnite/Spacequest/Blizzard/DigDug updates that run down our pipes. Would love to hear from others about how they're peering and caching -- not having the level of success I'd want with the typical "aggregators" (they know who they are ) and would really like to link to the source even if it means trenching through the core of the Earth... Would love pointers, names, or any leads, on or off list. Thanks Jose L. Rodriguez CTO, Totalplay
Agreed. The few good examples in Canada are Ubisoft/i3D (now mostly just i3D) and Riot Games. We don't have Valve or Blizzard here. Epic Games seems to use Akamai for downloads/updates and AWS for backend so I don't see how you can cache/optimize latency other than getting in Akamai's own AANP program and peering with AWS. Eric On Mar 23 2021, at 10:05 am, Mike Hammett <nanog@ics-il.net> wrote:
Hi folks, To briefly clarify the "now mostly i3D" situation.. i3D.net was acquired by Ubisoft in 2019, and the reason why you're seeing Ubisoft's ASN disappearing from the IXPs where they were present is that we are integrating the networks. Ubisoft's prefixes are being announced downstream of i3D.net's AS49544 and continue to be reachable through a very long list of IXPs. Another thing to keep in mind is that not all videogaming publishers have in-house game hosting capabilities, exactly because it is not easy to build the required global footprint to support this type of traffic with sufficient low-latency quality. Many will use external hosting providers, either bare metal or cloud. And although we are nowadays owned by Ubisoft, we still carry lots of non-Ubisoft videogames in our network since helping this industry attain low latency has always been and continues to be our core business. Best regards, Martijn On 3/23/21 4:03 PM, Eric Dugas via NANOG wrote: Agreed. The few good examples in Canada are Ubisoft/i3D (now mostly just i3D) and Riot Games. We don't have Valve or Blizzard here. Epic Games seems to use Akamai for downloads/updates and AWS for backend so I don't see how you can cache/optimize latency other than getting in Akamai's own AANP program and peering with AWS. Eric On Mar 23 2021, at 10:05 am, Mike Hammett <nanog@ics-il.net><mailto:nanog@ics-il.net> wrote: For an industry (online gaming) with the most "sensitive" customers to latency, packet loss, throughput, etc., the online gaming industry is terrible at peering. There are a few shining examples of what you should do, but then the rest is just content with buying transit from one, two, three players and calling it a day. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com From: "Jose Luis Rodriguez" <jlrodriguez@gmail.com><mailto:jlrodriguez@gmail.com> To: nanog@nanog.org<mailto:nanog@nanog.org> Sent: Monday, March 22, 2021 9:13:46 PM Subject: Peering and Caching for Epic Games, Fortnite, et al We run a healthy-sized ISP (say, 2.5M households, plus enterprise, etc ) and we really, REALLY want to make sure our users have an amazing experience when downloading the neverending Fortnite/Spacequest/Blizzard/DigDug updates that run down our pipes. Would love to hear from others about how they're peering and caching -- not having the level of success I'd want with the typical "aggregators" (they know who they are ) and would really like to link to the source even if it means trenching through the core of the Earth... Would love pointers, names, or any leads, on or off list. Thanks Jose L. Rodriguez CTO, Totalplay
On 3/23/21 17:03, Eric Dugas via NANOG wrote:
Gaming networks are not as keen on building out network as generic content folk are. So while they may be seen as "content" sources, they are not "content-content" sources, if you follow my drift. There are some backbones that dedicate their goal to making access to gaming services effortless. But those are not as rife as regular ISP's. Mark.
That's because they often don't really need to. Content and patch distribution is generally handled via a CDN. For companies that run games with central hosted servers, those servers are hosted in leased datacenter space, often behind a large provider's network. Blizzard , for example, has (I believe) all of their US servers hosted in ATT locations. Most other games either have their online bits in AWS, or just do peer to peer communication. On Tue, Mar 23, 2021 at 10:07 AM Mike Hammett <nanog@ics-il.net> wrote:
participants (9)
-
Eric Dugas
-
John Waters
-
Jose Luis Rodriguez
-
Mark Tinka
-
Martijn Schmidt
-
Mike Hammett
-
Mike Lyon
-
Tom Beecher
-
Valdis Klētnieks