Re: An appeal for more bandwidth to the Internet Archive
Aren't they in a former church or something? I vaguely remember their location to be significant for some reason or another. So location may weigh heavily. -- Regards, Terrence Koeman, PhD/MTh/BPsy Darkness Reigns (Holding) B.V. Please quote relevant replies. Spelling errors courtesy of my 'smart'phone. ________________________________ From: David Hubbard <dhubbard@dino.hostasaurus.com> Sent: Wednesday, 13 May 2020 06:02 To: nanog@nanog.org Subject: Re: An appeal for more bandwidth to the Internet Archive
Could the operation be moved out of California to achieve dramatically reduced operating costs and perhaps solve some problems via cost savings vs increased donation? I have to imagine with the storage and processing requirements that the footprint and power usage in SFO is quite costly. I have equipment in a few California colo's and it's easily 3x what I pay for similar in Nevada, before even getting into tax abatement advantages.
On 5/12/20, 1:33 PM, "NANOG on behalf of colin johnston" <nanog-bounces@nanog.org on behalf of colinj@gt86car.org.uk> wrote:
Is the increased usage due to more users or more existing users having higher bandwidth at home to request faster ? Would be interested if IPS configured firewall used to block out invalid traffic/spam traffic and if such traffic increased when back end network capacity increased ? What countries are requesting the most data and does this analysis throw up questions as to why ? Are there high network usage hitters which raise question as to why asking for so much data time and time again and is this valid traffic use ?
Colin
> On 12 May 2020, at 17:33, Tim Požár <pozar@lns.com> wrote: > > Jared... > > Thanks for sharing this. I was the first Director of Operations from '96 to '98, at was was then Internet Archive/Alex. I was the network architect back then got them their ASN and original address space. Folks may also know, I help start SFMIX with Matt Peterson. > > A bit more detail in this... Some of this I got from Jonah Edwards who is the current Network Architect at IA. Yes, the bottle neck was the line cards. They have upgraded and that has certainly helped the bandwidth of late. > > Peering would be a big help for IA. At this point they have two 10Gb LAG interfaces that show up on SFMIX that was turned up last February. Looking at the last couple of weeks the 95th percentile on this 20Gb LAG is 3 Gb. As they just turned up on SFMIX, they are just starting to get peers turned up there. Eyeball networks that show up on SFMIX are highly encouraged to start peering with them. Alas, they are v4 only at this point. > > Additionally, if folks do have some fat pipes that can donate bandwidth at 200 Paul, I am sure Jonah won't turn it down. > > Tim > > On 5/12/20 4:45 AM, Jared Brown wrote: >> Hello all! >> Last week the Internet Archive upgraded their bandwidth 30% from 47 Gbps to 62 Gbps. It was all gobbled up immediately. There's a lovely solid green graph showing how usage grows vertically as each interface comes online until it too is 100% saturated. Looking at the graph legend you can see that their usage for the past 24 hours averages 49.76G on their 50G of transport. >> To see the pretty pictures follow the below link: >> https://blog.archive.org/2020/05/11/thank-you-for-helping-us-increase-our-ba... >> Relevant parts from the blog post: >> "A year ago, usage was 30Gbits/sec. At the beginning of this year, we were at 40Gbits/sec, and we were handling it. ... >> Then Covid-19 hit and demand rocketed to 50Gbits/sec and overran our network infrastructure’s ability to handle it. So much so, our network statistics probes had difficulty collecting data (hence the white spots in the graphs). >> We bought a second router with new line cards, and got it installed and running (and none of this is easy during a pandemic), and increased our capacity from 47Gbits/sec peak to 62Gbits/sec peak. And we are handling it better, but it is still consumed." >> It is obvious that the Internet Archive needs more bandwidth to power the Wayback machine and to fulfill its mission of being the Internet library and the historic archive of our times. >> The Internet Archive is present at Digital Realty SFO (200 Paul) and a member of the San Francisco Metropolitan Internet Exchange (SFMIX). >> I appeal to all list members present or capable of getting to these facilities to peer with and/or donate bandwidth to the Internet Archive. >> I appeal to all vendors and others with equipment that they can donate to the Internet Archive to contact them so that they can scale their services and sustain their growth. >> The Internet Archive is currently running 10G equipment. If you can help them gain 100G connectivity, 100G routing, 100G switching and/or 100G DWDM capabilities, please reach out to them. They have the infrastructure and dark fiber to transition to 100G, but lack the equipment. You can find the Internet Archive's contact information below or you can contact Jonah at the Archive Org directly either by email or via the contact information available on his Twitter profile @jonahedwards. >> You can also donate at https://archive.org/donate/ >> The Internet Archive is a 501(c)(3) non-profit. Donations are tax-deductible. >> Contact information: >> https://archive.org/about/contact.php >> Volunteering: >> https://archive.org/about/volunteerpositions.php >> Disclaimer: I am not affiliated with the Internet Archive. Nobody asked me to write this post. If something angers you about this post, be angry at me. I merely think that the Internet Archive is a good thing and deserves our support. >> Jared
They are, and I’ve got dark fiber in there. We’ve reached out... -Ben Ms. Benjamin PD Cannon, ASCE 6x7 Networks & 6x7 Telecom, LLC CEO ben@6by7.net <mailto:ben@6by7.net> "The only fully end-to-end encrypted global telecommunications company in the world.” FCC License KJ6FJJ
On May 12, 2020, at 9:24 PM, Terrence Koeman <terrence@darkness-reigns.com> wrote:
Aren't they in a former church or something? I vaguely remember their location to be significant for some reason or another. So location may weigh heavily.
-- Regards, Terrence Koeman, PhD/MTh/BPsy Darkness Reigns (Holding) B.V.
Please quote relevant replies. Spelling errors courtesy of my 'smart'phone. From: David Hubbard <dhubbard@dino.hostasaurus.com <mailto:dhubbard@dino.hostasaurus.com>> Sent: Wednesday, 13 May 2020 06:02 To: nanog@nanog.org <mailto:nanog@nanog.org> Subject: Re: An appeal for more bandwidth to the Internet Archive
Could the operation be moved out of California to achieve dramatically reduced operating costs and perhaps solve some problems via cost savings vs increased donation? I have to imagine with the storage and processing requirements that the footprint and power usage in SFO is quite costly. I have equipment in a few California colo's and it's easily 3x what I pay for similar in Nevada, before even getting into tax abatement advantages.
On 5/12/20, 1:33 PM, "NANOG on behalf of colin johnston" <nanog-bounces@nanog.org <mailto:nanog-bounces@nanog.org> on behalf of colinj@gt86car.org.uk <mailto:colinj@gt86car.org.uk>> wrote:
Is the increased usage due to more users or more existing users having higher bandwidth at home to request faster ? Would be interested if IPS configured firewall used to block out invalid traffic/spam traffic and if such traffic increased when back end network capacity increased ? What countries are requesting the most data and does this analysis throw up questions as to why ? Are there high network usage hitters which raise question as to why asking for so much data time and time again and is this valid traffic use ?
Colin
> On 12 May 2020, at 17:33, Tim Požár <pozar@lns.com <mailto:pozar@lns.com>> wrote: > > Jared... > > Thanks for sharing this. I was the first Director of Operations from '96 to '98, at was was then Internet Archive/Alex. I was the network architect back then got them their ASN and original address space. Folks may also know, I help start SFMIX with Matt Peterson. > > A bit more detail in this... Some of this I got from Jonah Edwards who is the current Network Architect at IA. Yes, the bottle neck was the line cards. They have upgraded and that has certainly helped the bandwidth of late. > > Peering would be a big help for IA. At this point they have two 10Gb LAG interfaces that show up on SFMIX that was turned up last February. Looking at the last couple of weeks the 95th percentile on this 20Gb LAG is 3 Gb. As they just turned up on SFMIX, they are just starting to get peers turned up there. Eyeball networks that show up on SFMIX are highly encouraged to start peering with them. Alas, they are v4 only at this point. > > Additionally, if folks do have some fat pipes that can donate bandwidth at 200 Paul, I am sure Jonah won't turn it down. > > Tim > > On 5/12/20 4:45 AM, Jared Brown wrote: >> Hello all! >> Last week the Internet Archive upgraded their bandwidth 30% from 47 Gbps to 62 Gbps. It was all gobbled up immediately. There's a lovely solid green graph showing how usage grows vertically as each interface comes online until it too is 100% saturated. Looking at the graph legend you can see that their usage for the past 24 hours averages 49.76G on their 50G of transport. >> To see the pretty pictures follow the below link: >> https://blog.archive.org/2020/05/11/thank-you-for-helping-us-increase-our-ba... <https://blog.archive.org/2020/05/11/thank-you-for-helping-us-increase-our-bandwidth/> >> Relevant parts from the blog post: >> "A year ago, usage was 30Gbits/sec. At the beginning of this year, we were at 40Gbits/sec, and we were handling it. ... >> Then Covid-19 hit and demand rocketed to 50Gbits/sec and overran our network infrastructure’s ability to handle it. So much so, our network statistics probes had difficulty collecting data (hence the white spots in the graphs). >> We bought a second router with new line cards, and got it installed and running (and none of this is easy during a pandemic), and increased our capacity from 47Gbits/sec peak to 62Gbits/sec peak. And we are handling it better, but it is still consumed." >> It is obvious that the Internet Archive needs more bandwidth to power the Wayback machine and to fulfill its mission of being the Internet library and the historic archive of our times. >> The Internet Archive is present at Digital Realty SFO (200 Paul) and a member of the San Francisco Metropolitan Internet Exchange (SFMIX). >> I appeal to all list members present or capable of getting to these facilities to peer with and/or donate bandwidth to the Internet Archive. >> I appeal to all vendors and others with equipment that they can donate to the Internet Archive to contact them so that they can scale their services and sustain their growth. >> The Internet Archive is currently running 10G equipment. If you can help them gain 100G connectivity, 100G routing, 100G switching and/or 100G DWDM capabilities, please reach out to them. They have the infrastructure and dark fiber to transition to 100G, but lack the equipment. You can find the Internet Archive's contact information below or you can contact Jonah at the Archive Org directly either by email or via the contact information available on his Twitter profile @jonahedwards. >> You can also donate at https://archive.org/donate/ <https://archive.org/donate/> >> The Internet Archive is a 501(c)(3) non-profit. Donations are tax-deductible. >> Contact information: >> https://archive.org/about/contact.php <https://archive.org/about/contact.php> >> Volunteering: >> https://archive.org/about/volunteerpositions.php <https://archive.org/about/volunteerpositions.php> >> Disclaimer: I am not affiliated with the Internet Archive. Nobody asked me to write this post. If something angers you about this post, be angry at me. I merely think that the Internet Archive is a good thing and deserves our support. >> Jared
Internet Archive primary office is located at 300 Funston in San Francisco. This was a Christian Science church so it has the roman columns you would expect for a church / library. You can see it on Google Street Views at: https://www.google.com/maps/place/300+Funston+Ave,+San+Francisco,+CA+94118 Although they serve content out of this site, their primary site for bandwidth is at 2512 Florida Ave, Richmond, CA. IA does have satellite offices around the world for scanning, etc., the public facing servers are location in these two locations. Tim On 5/12/20 9:24 PM, Terrence Koeman wrote:
Aren't they in a former church or something? I vaguely remember their location to be significant for some reason or another. So location may weigh heavily.
-- Regards, Terrence Koeman, PhD/MTh/BPsy Darkness Reigns (Holding) B.V.
Please quote relevant replies. Spelling errors courtesy of my 'smart'phone. ------------------------------------------------------------------------ *From:* David Hubbard <dhubbard@dino.hostasaurus.com> *Sent:* Wednesday, 13 May 2020 06:02 *To:* nanog@nanog.org *Subject:* Re: An appeal for more bandwidth to the Internet Archive
Could the operation be moved out of California to achieve dramatically reduced operating costs and perhaps solve some problems via cost savings vs increased donation? I have to imagine with the storage and processing requirements that the footprint and power usage in SFO is quite costly. I have equipment in a few California colo's and it's easily 3x what I pay for similar in Nevada, before even getting into tax abatement advantages.
On 5/12/20, 1:33 PM, "NANOG on behalf of colin johnston" <nanog-bounces@nanog.org on behalf of colinj@gt86car.org.uk> wrote:
Is the increased usage due to more users or more existing users having higher bandwidth at home to request faster ? Would be interested if IPS configured firewall used to block out invalid traffic/spam traffic and if such traffic increased when back end network capacity increased ? What countries are requesting the most data and does this analysis throw up questions as to why ? Are there high network usage hitters which raise question as to why asking for so much data time and time again and is this valid traffic use ?
Colin
> On 12 May 2020, at 17:33, Tim Požár <pozar@lns.com> wrote: > > Jared... > > Thanks for sharing this. I was the first Director of Operations from '96 to '98, at was was then Internet Archive/Alex. I was the network architect back then got them their ASN and original address space. Folks may also know, I help start SFMIX with Matt Peterson. > > A bit more detail in this... Some of this I got from Jonah Edwards who is the current Network Architect at IA. Yes, the bottle neck was the line cards. They have upgraded and that has certainly helped the bandwidth of late. > > Peering would be a big help for IA. At this point they have two 10Gb LAG interfaces that show up on SFMIX that was turned up last February. Looking at the last couple of weeks the 95th percentile on this 20Gb LAG is 3 Gb. As they just turned up on SFMIX, they are just starting to get peers turned up there. Eyeball networks that show up on SFMIX are highly encouraged to start peering with them. Alas, they are v4 only at this point. > > Additionally, if folks do have some fat pipes that can donate bandwidth at 200 Paul, I am sure Jonah won't turn it down. > > Tim > > On 5/12/20 4:45 AM, Jared Brown wrote: >> Hello all! >> Last week the Internet Archive upgraded their bandwidth 30% from 47 Gbps to 62 Gbps. It was all gobbled up immediately. There's a lovely solid green graph showing how usage grows vertically as each interface comes online until it too is 100% saturated. Looking at the graph legend you can see that their usage for the past 24 hours averages 49.76G on their 50G of transport. >> To see the pretty pictures follow the below link: >> https://blog.archive.org/2020/05/11/thank-you-for-helping-us-increase-our-ba...
>> Relevant parts from the blog post: >> "A year ago, usage was 30Gbits/sec. At the beginning of this year, we were at 40Gbits/sec, and we were handling it. ... >> Then Covid-19 hit and demand rocketed to 50Gbits/sec and overran our network infrastructure’s ability to handle it. So much so, our network statistics probes had difficulty collecting data (hence the white spots in the graphs). >> We bought a second router with new line cards, and got it installed and running (and none of this is easy during a pandemic), and increased our capacity from 47Gbits/sec peak to 62Gbits/sec peak. And we are handling it better, but it is still consumed." >> It is obvious that the Internet Archive needs more bandwidth to power the Wayback machine and to fulfill its mission of being the Internet library and the historic archive of our times. >> The Internet Archive is present at Digital Realty SFO (200 Paul) and a member of the San Francisco Metropolitan Internet Exchange (SFMIX). >> I appeal to all list members present or capable of getting to these facilities to peer with and/or donate bandwidth to the Internet Archive. >> I appeal to all vendors and others with equipment that they can donate to the Internet Archive to contact them so that they can scale their services and sustain their growth. >> The Internet Archive is currently running 10G equipment. If you can help them gain 100G connectivity, 100G routing, 100G switching and/or 100G DWDM capabilities, please reach out to them. They have the infrastructure and dark fiber to transition to 100G, but lack the equipment. You can find the Internet Archive's contact information below or you can contact Jonah at the Archive Org directly either by email or via the contact information available on his Twitter profile @jonahedwards. >> You can also donate at https://archive.org/donate/ >> The Internet Archive is a 501(c)(3) non-profit. Donations are tax-deductible. >> Contact information: >> https://archive.org/about/contact.php >> Volunteering: >> https://archive.org/about/volunteerpositions.php >> Disclaimer: I am not affiliated with the Internet Archive. Nobody asked me to write this post. If something angers you about this post, be angry at me. I merely think that the Internet Archive is a good thing and deserves our support. >> Jared
What about introducing some cache offloading, like CDN doing? (Google, Facebook, Netflix, Akamai, etc) I think it can be rolled pretty quickly, with minimum labor efforts, at least for heavy content. Maybe some opensource communities can help as well, and same scheme can be applied then to other non-profits. But sure something more smooth like nginx caching, not bunch of rsync/ssh scripts, as many Linux mirrors have. On 2020-05-13 08:25, Tim Požár wrote:
Internet Archive primary office is located at 300 Funston in San Francisco. This was a Christian Science church so it has the roman columns you would expect for a church / library. You can see it on Google Street Views at:
https://www.google.com/maps/place/300+Funston+Ave,+San+Francisco,+CA+94118
Although they serve content out of this site, their primary site for bandwidth is at 2512 Florida Ave, Richmond, CA.
IA does have satellite offices around the world for scanning, etc., the public facing servers are location in these two locations.
Tim
On 5/12/20 9:24 PM, Terrence Koeman wrote:
Aren't they in a former church or something? I vaguely remember their location to be significant for some reason or another. So location may weigh heavily.
-- Regards, Terrence Koeman, PhD/MTh/BPsy Darkness Reigns (Holding) B.V.
Please quote relevant replies. Spelling errors courtesy of my 'smart'phone. ------------------------------------------------------------------------ *From:* David Hubbard <dhubbard@dino.hostasaurus.com> *Sent:* Wednesday, 13 May 2020 06:02 *To:* nanog@nanog.org *Subject:* Re: An appeal for more bandwidth to the Internet Archive
Could the operation be moved out of California to achieve dramatically reduced operating costs and perhaps solve some problems via cost savings vs increased donation? I have to imagine with the storage and processing requirements that the footprint and power usage in SFO is quite costly. I have equipment in a few California colo's and it's easily 3x what I pay for similar in Nevada, before even getting into tax abatement advantages.
On 5/12/20, 1:33 PM, "NANOG on behalf of colin johnston" <nanog-bounces@nanog.org on behalf of colinj@gt86car.org.uk> wrote:
Is the increased usage due to more users or more existing users having higher bandwidth at home to request faster ? Would be interested if IPS configured firewall used to block out invalid traffic/spam traffic and if such traffic increased when back end network capacity increased ? What countries are requesting the most data and does this analysis throw up questions as to why ? Are there high network usage hitters which raise question as to why asking for so much data time and time again and is this valid traffic use ?
Colin
> On 12 May 2020, at 17:33, Tim Požár <pozar@lns.com> wrote: > > Jared... > > Thanks for sharing this. I was the first Director of Operations from '96 to '98, at was was then Internet Archive/Alex. I was the network architect back then got them their ASN and original address space. Folks may also know, I help start SFMIX with Matt Peterson. > > A bit more detail in this... Some of this I got from Jonah Edwards who is the current Network Architect at IA. Yes, the bottle neck was the line cards. They have upgraded and that has certainly helped the bandwidth of late. > > Peering would be a big help for IA. At this point they have two 10Gb LAG interfaces that show up on SFMIX that was turned up last February. Looking at the last couple of weeks the 95th percentile on this 20Gb LAG is 3 Gb. As they just turned up on SFMIX, they are just starting to get peers turned up there. Eyeball networks that show up on SFMIX are highly encouraged to start peering with them. Alas, they are v4 only at this point. > > Additionally, if folks do have some fat pipes that can donate bandwidth at 200 Paul, I am sure Jonah won't turn it down. > > Tim > > On 5/12/20 4:45 AM, Jared Brown wrote: >> Hello all! >> Last week the Internet Archive upgraded their bandwidth 30% from 47 Gbps to 62 Gbps. It was all gobbled up immediately. There's a lovely solid green graph showing how usage grows vertically as each interface comes online until it too is 100% saturated. Looking at the graph legend you can see that their usage for the past 24 hours averages 49.76G on their 50G of transport. >> To see the pretty pictures follow the below link: >>
https://blog.archive.org/2020/05/11/thank-you-for-helping-us-increase-our-ba...
>> Relevant parts from the blog post: >> "A year ago, usage was 30Gbits/sec. At the beginning of this year, we were at 40Gbits/sec, and we were handling it. ... >> Then Covid-19 hit and demand rocketed to 50Gbits/sec and overran our network infrastructure’s ability to handle it. So much so, our network statistics probes had difficulty collecting data (hence the white spots in the graphs). >> We bought a second router with new line cards, and got it installed and running (and none of this is easy during a pandemic), and increased our capacity from 47Gbits/sec peak to 62Gbits/sec peak. And we are handling it better, but it is still consumed." >> It is obvious that the Internet Archive needs more bandwidth to power the Wayback machine and to fulfill its mission of being the Internet library and the historic archive of our times. >> The Internet Archive is present at Digital Realty SFO (200 Paul) and a member of the San Francisco Metropolitan Internet Exchange (SFMIX). >> I appeal to all list members present or capable of getting to these facilities to peer with and/or donate bandwidth to the Internet Archive. >> I appeal to all vendors and others with equipment that they can donate to the Internet Archive to contact them so that they can scale their services and sustain their growth. >> The Internet Archive is currently running 10G equipment. If you can help them gain 100G connectivity, 100G routing, 100G switching and/or 100G DWDM capabilities, please reach out to them. They have the infrastructure and dark fiber to transition to 100G, but lack the equipment. You can find the Internet Archive's contact information below or you can contact Jonah at the Archive Org directly either by email or via the contact information available on his Twitter profile @jonahedwards. >> You can also donate at https://archive.org/donate/ >> The Internet Archive is a 501(c)(3) non-profit. Donations are tax-deductible. >> Contact information: >> https://archive.org/about/contact.php >> Volunteering: >> https://archive.org/about/volunteerpositions.php >> Disclaimer: I am not affiliated with the Internet Archive. Nobody asked me to write this post. If something angers you about this post, be angry at me. I merely think that the Internet Archive is a good thing and deserves our support. >> Jared
On 13May20, Denys Fedoryshchenko allegedly wrote:
What about introducing some cache offloading, like CDN doing? (Google, Facebook, Netflix, Akamai, etc)
Maybe some opensource communities can help as well
Surely someone has already thought thru the idea of a community CDN? Perhaps along the lines of pool.ntp.org? What became of that discussion? Maybe a TOR network could be repurposed to cover the same ground. Mark.
On 2020-05-13 11:00, Mark Delany wrote:
On 13May20, Denys Fedoryshchenko allegedly wrote:
What about introducing some cache offloading, like CDN doing? (Google, Facebook, Netflix, Akamai, etc)
Maybe some opensource communities can help as well
Surely someone has already thought thru the idea of a community CDN? Perhaps along the lines of pool.ntp.org? What became of that discussion?
Maybe a TOR network could be repurposed to cover the same ground.
Mark. I believe tor is not efficient at all for this purposes. Privacy have very high overhead.
Several schemes exist: 1)ISP announce in some way subnets he want to be served from his cache. 1.A)Apple cache way - just HTTP(S) request will turn specific IP to ISP cache. Not secure at all. 1.B)BGP + DNS, most common way. ISP does peering with CDN, CDN will return ISP cache nodes IP's to DNS requests. It means for example content.archive.org will have local node A/AAAA records (btw where is IPv6 for archive?) for customers of ISP with this node, or anybody who is peering with it. Huge drawback - archive.org will need to provide TLS certificates for web.archive.org each local node, this is bad and probably no-go. Yes, i know some schemes exist, that certificate is not present on local node, but some "precalculated" result used, but it is too complex. 1.C)BGP + HTTP redirect. If ISP has peering with archive.org, to all subnets announced users will get 302 or some HTTP redirect. Next is almost same and much better, but will require small modifications of content engine or frontend balancers. 1.D)BGP + HTTP rewrite. If ISP <*same as before*> URL is rewritten within content e.g. http://web.archive.org/web/20200511193226/https://git.kernel.org/torvalds/t/... will appear as http://emu.st.node.archive.org/web/20200511193226/https://git.kernel.org/tor... or http://archive-org.proxy.emu.st/web/20200511193226/https://git.kernel.org/to... In second option ISP can handle SSL certificate by himself. 2)BGP announce of archive.org subnets locally. Prone to leaks, require TLS certificates and etc, no-go. You can still modify some schemes, and make other options that no one has yet implemented. For example, to do everything through javascript (CDNs cannot afford it, because of way they work), and for example, website generate content links dynamically, for that client request some /config.json file (which is dynamically generated and cached for a while), so we give it to IPs that have a local node - URL of the local node, for the rest - default url.
On 2020-05-13 11:00, Mark Delany wrote:
On 13May20, Denys Fedoryshchenko allegedly wrote:
What about introducing some cache offloading, like CDN doing? (Google, Facebook, Netflix, Akamai, etc) Maybe some opensource communities can help as well Surely someone has already thought thru the idea of a community CDN? Perhaps along the lines of pool.ntp.org? What became of that discussion?
Yes, Jeff Ubois and I have been discussing it with Brewster. There was significant effort put into this some eighteen or twenty years ago, backed mostly by the New Zealand government… Called the “Internet Capacity Development Group.” It had a NOC and racks full of servers in a bunch of datacenters, mostly around the Pacific Rim, but in Amsterdam and Frankfurt as well, I think. PCH put quite a lot of effort into supporting it, because it’s a win for ISPs and IXPs to have community caches with local or valuable content that they can peer with. There’s also a much higher hit-rate (and thus efficiency) to caching things the community actually cares about, rather than whatever random thing a startup is paying Akamai or Cloudflare or whatever to push, which may never get viewed at all. It ran well enough for about ten years, but over the long term it was just too complex a project to survive at scale on community support alone. It was trending toward more and more of the hard costs being met by PCH’s donors, and less and less by the donors who were supporting the content publishers, which was the goal. The newer conversation is centered around using DAFs to support it on behalf of non-profit content like the Archive, Wikipedia, etc., and that conversation seems to be gaining some traction. Unfortunately because there are now a smaller number of really wealthy people who need places to shove all their extra money. Not how I’d have liked to get here. -Bill
On 2020-05-13 13:10, Bill Woodcock wrote:
On 2020-05-13 11:00, Mark Delany wrote:
On 13May20, Denys Fedoryshchenko allegedly wrote:
What about introducing some cache offloading, like CDN doing? (Google, Facebook, Netflix, Akamai, etc) Maybe some opensource communities can help as well Surely someone has already thought thru the idea of a community CDN? Perhaps along the lines of pool.ntp.org? What became of that discussion?
Yes, Jeff Ubois and I have been discussing it with Brewster.
There was significant effort put into this some eighteen or twenty years ago, backed mostly by the New Zealand government… Called the “Internet Capacity Development Group.” It had a NOC and racks full of servers in a bunch of datacenters, mostly around the Pacific Rim, but in Amsterdam and Frankfurt as well, I think. PCH put quite a lot of effort into supporting it, because it’s a win for ISPs and IXPs to have community caches with local or valuable content that they can peer with. There’s also a much higher hit-rate (and thus efficiency) to caching things the community actually cares about, rather than whatever random thing a startup is paying Akamai or Cloudflare or whatever to push, which may never get viewed at all. It ran well enough for about ten years, but over the long term it was just too complex a project to survive at scale on community support alone. It was trending toward more and more of the hard costs being met by PCH’s donors, and less and less by the donors who were supporting the content publishers, which was the goal.
The newer conversation is centered around using DAFs to support it on behalf of non-profit content like the Archive, Wikipedia, etc., and that conversation seems to be gaining some traction. Unfortunately because there are now a smaller number of really wealthy people who need places to shove all their extra money. Not how I’d have liked to get here. I think this is a simple equation.
1) The minimum cost of implementation and technical support efforts I think earlier this was the main problem, 10 years ago there was no such level of software automation as it is available today. 2) Win for operators. Before it was more trivial by running squid and trivial cache, now, with HTTPS it is not possible. 3) Proud badge of non-profit projects supporter and charity activities. (Whether it is possible to write off tax/etc as donations - depends on the laws of your country)
On Wed, 13 May 2020 10:40:36 +0300, Denys Fedoryshchenko said:
What about introducing some cache offloading, like CDN doing? (Google, Facebook, Netflix, Akamai, etc) I think it can be rolled pretty quickly, with minimum labor efforts, at least for heavy content.
The thing is that if you're an 800 pound gorilla, you probably have enough things that would benefit from being cached to make it worthwhile. I'd expect that the Internet Archive is probably mostly long-tail hits with not much hot content. Has anybody modeled how much cache space would it take to significantly improve the bandwidth situation?
You mean not everyone still looks up the classifications on mulletsgalore.com? Chuck On Wed, May 13, 2020, 7:54 PM Valdis Klētnieks <valdis.kletnieks@vt.edu> wrote:
On Wed, 13 May 2020 10:40:36 +0300, Denys Fedoryshchenko said:
What about introducing some cache offloading, like CDN doing? (Google, Facebook, Netflix, Akamai, etc) I think it can be rolled pretty quickly, with minimum labor efforts, at least for heavy content.
The thing is that if you're an 800 pound gorilla, you probably have enough things that would benefit from being cached to make it worthwhile.
I'd expect that the Internet Archive is probably mostly long-tail hits with not much hot content. Has anybody modeled how much cache space would it take to significantly improve the bandwidth situation?
participants (8)
-
Ben Cannon -
Bill Woodcock -
Chuck Church -
Denys Fedoryshchenko -
Mark Delany -
Terrence Koeman -
Tim Požár -
Valdis Klētnieks