Impacts of Encryption Everywhere (any solution?)
Has anyone outside of tech media, Silicon Valley or academia (all places wildly out of touch with the real world) put much thought into the impacts of encryption everywhere? So often we hear about how we need the best modern encryption on all forms of communication because of whatever scary thing is trendy this week (Russia, NSA, Google, whatever). HTTPS your marketing information and generic education pieces because of the boogeyman! However, I recently came across a thread where someone was exploring getting a one megabit connection into their village and sharing it among many. The crowd I referenced earlier also believes you can't Internet under 100 megabit/s per home. Apparently, the current best Internet the residents of the village can get is 40 kilobit/s. Zero oversubscription gets a better service to up to 25 homes. Likely that could be stretched to at least 50 or 100 homes and be better than what they currently have. Forget about streaming video, let's just focus on web browsing and messaging. However, this could be wildly improved with caching ala squid or something similar. The problem is that encrypted content is difficult to impossible for your average Joe to cache. The rewards for implementing caching are greatly mitigated and people like this must suffer a worse Internet experience because of some ideological high horse in a far-off land. Some things certainly do need to be encrypted, but encrypting everything means people with limited Internet access get worse performance OR mechanisms have to be out in place to break ALL encryption, this compromising security and privacy when it's really needed. To circle back to being somewhat on-topic, what mechanisms are available to maximize the amount of traffic someone in this situation could cache? The performance of third-world Internet depends on you. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com
That is super interesting. While one can Internet fine at 5Mbps (save for streaming UHD movies maybe), I am not convinced 1Mbps can be successfully shared even if there was no encryption anywhere. My understanding is that some enterprises do decrypt traffic in flight with proxies such as bluecoat, though I'm not sure on the particulars of how that works. I think the overall theory is that the proxy acts as a trusted CA for all its client and generates the certificate for the destination hostname on the fly thus terminating the SSL connection and opening new one on behalf of the client. I do, however, recall that the solution is not cheap. Neither $ nor computationally or, I'm guessing, in case of a village if they can't get anything faster than 1Mbps, can they even get power to run a couple (does the proxy uptime matter?) of proxies of heavy compute? Another concern would be that caching implies the whole village visits the same content. I'm not even confident me and wife visit the same content (save for gmail maybe). And lastly, most modern websites are very media rich. Unless the whole village confines their usage to wikipedia.org, I can't imagine that the experience will be pleasant in anyway or form or there will be any benefit to caching. Save for the SSL proxy mentioned above, I have seen folks pull several crappy DLS connections (Let's say ~1Mbps each) and band them together. If the provider support the bonding option, great! If not, I've seen folks basically per flow load balance across the 4 connections. -Andrey --Andrey On Mon, May 28, 2018 at 4:23 PM, Mike Hammett <nanog@ics-il.net> wrote:
Has anyone outside of tech media, Silicon Valley or academia (all places wildly out of touch with the real world) put much thought into the impacts of encryption everywhere? So often we hear about how we need the best modern encryption on all forms of communication because of whatever scary thing is trendy this week (Russia, NSA, Google, whatever). HTTPS your marketing information and generic education pieces because of the boogeyman!
However, I recently came across a thread where someone was exploring getting a one megabit connection into their village and sharing it among many. The crowd I referenced earlier also believes you can't Internet under 100 megabit/s per home.
Apparently, the current best Internet the residents of the village can get is 40 kilobit/s. Zero oversubscription gets a better service to up to 25 homes. Likely that could be stretched to at least 50 or 100 homes and be better than what they currently have. Forget about streaming video, let's just focus on web browsing and messaging.
However, this could be wildly improved with caching ala squid or something similar. The problem is that encrypted content is difficult to impossible for your average Joe to cache. The rewards for implementing caching are greatly mitigated and people like this must suffer a worse Internet experience because of some ideological high horse in a far-off land.
Some things certainly do need to be encrypted, but encrypting everything means people with limited Internet access get worse performance OR mechanisms have to be out in place to break ALL encryption, this compromising security and privacy when it's really needed.
To circle back to being somewhat on-topic, what mechanisms are available to maximize the amount of traffic someone in this situation could cache? The performance of third-world Internet depends on you.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
The increase in the subscriber base increases the likelihood of visiting the same content and thus the benefit. Before HTTPS-everywhere, caching was hugely beneficial. Currently they are making do with 40 kilobit/s, so it's certainly possible to Internet at that level. Just looking at ways the service can be even that much better. If they only have single digit megabit/s of Internet, you don't need multiple systems to add\drop the encryption. While I don't have anything to back this up, I'd suspect a couple hundred dollar single board computer (since session border controller seems to be a more popular use of the acronym SBC) would be sufficient. I'm not overly intimate with that space, but some little ARM-based machine could probably do it just fine. Move that to hundreds of megabit/s or gigabit/s and your concern is certainly much more relevant. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Andrey Khomyakov" <khomyakov.andrey@gmail.com> To: "Mike Hammett" <nanog@ics-il.net> Cc: "NANOG list" <nanog@nanog.org> Sent: Monday, May 28, 2018 9:50:01 AM Subject: Re: Impacts of Encryption Everywhere (any solution?) That is super interesting. While one can Internet fine at 5Mbps (save for streaming UHD movies maybe), I am not convinced 1Mbps can be successfully shared even if there was no encryption anywhere. My understanding is that some enterprises do decrypt traffic in flight with proxies such as bluecoat, though I'm not sure on the particulars of how that works. I think the overall theory is that the proxy acts as a trusted CA for all its client and generates the certificate for the destination hostname on the fly thus terminating the SSL connection and opening new one on behalf of the client. I do, however, recall that the solution is not cheap. Neither $ nor computationally or, I'm guessing, in case of a village if they can't get anything faster than 1Mbps, can they even get power to run a couple (does the proxy uptime matter?) of proxies of heavy compute? Another concern would be that caching implies the whole village visits the same content. I'm not even confident me and wife visit the same content (save for gmail maybe). And lastly, most modern websites are very media rich. Unless the whole village confines their usage to wikipedia.org , I can't imagine that the experience will be pleasant in anyway or form or there will be any benefit to caching. Save for the SSL proxy mentioned above, I have seen folks pull several crappy DLS connections (Let's say ~1Mbps each) and band them together. If the provider support the bonding option, great! If not, I've seen folks basically per flow load balance across the 4 connections. -Andrey --Andrey On Mon, May 28, 2018 at 4:23 PM, Mike Hammett < nanog@ics-il.net > wrote: Has anyone outside of tech media, Silicon Valley or academia (all places wildly out of touch with the real world) put much thought into the impacts of encryption everywhere? So often we hear about how we need the best modern encryption on all forms of communication because of whatever scary thing is trendy this week (Russia, NSA, Google, whatever). HTTPS your marketing information and generic education pieces because of the boogeyman! However, I recently came across a thread where someone was exploring getting a one megabit connection into their village and sharing it among many. The crowd I referenced earlier also believes you can't Internet under 100 megabit/s per home. Apparently, the current best Internet the residents of the village can get is 40 kilobit/s. Zero oversubscription gets a better service to up to 25 homes. Likely that could be stretched to at least 50 or 100 homes and be better than what they currently have. Forget about streaming video, let's just focus on web browsing and messaging. However, this could be wildly improved with caching ala squid or something similar. The problem is that encrypted content is difficult to impossible for your average Joe to cache. The rewards for implementing caching are greatly mitigated and people like this must suffer a worse Internet experience because of some ideological high horse in a far-off land. Some things certainly do need to be encrypted, but encrypting everything means people with limited Internet access get worse performance OR mechanisms have to be out in place to break ALL encryption, this compromising security and privacy when it's really needed. To circle back to being somewhat on-topic, what mechanisms are available to maximize the amount of traffic someone in this situation could cache? The performance of third-world Internet depends on you. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com
On Mon, May 28, 2018 at 10:50 AM, Andrey Khomyakov <khomyakov.andrey@gmail.com> wrote:
My understanding is that some enterprises do decrypt traffic in flight with proxies such as bluecoat, though I'm not sure on the particulars of how that works.
PCs within the enterprise contain an enterprise-local root in their certificate store. The proxy re-encrypts using a key whose ephemeral cert chains up to the enterprise root. Regards, Bill Herrin -- William Herrin ................ herrin@dirtside.com bill@herrin.us Dirtside Systems ......... Web: <http://www.dirtside.com/>
On Mon, May 28, 2018 at 09:23:09AM -0500, Mike Hammett wrote:
Some things certainly do need to be encrypted, but encrypting everything means people with limited Internet access get worse performance OR mechanisms have to be out in place to break ALL encryption, this compromising security and privacy when it's really needed.
There are better places to reduce traffic while simultaneously enhancing security and privacy. The new EU version of the home page of USA Today is about 20% the size of the one presented in the US -- because it's had all the tracking and scripting stripped out -- with a concomitant reduction in load time and rendering time. Much more drastic reductions are available elsewhere, e.g., mail messages composed of text only are typically 5% to 10% the size of the same messages marked up with HTML. The problem (part of the problem) is that the people doing these foolish things are new, ignorant, and privileged: they don't realize that bandwidth is still an expensive and scarce resource for most of the planet. I've said for years that every web designer should be forced to work in an environment bandlimited to 56K in order to instll in them the virtue of frugality and strongly discourage them from flattering their egos by creating all-singing all-dancing web sites...that look great in the portfolios they'll show to their peers but are horribly bloated, slow, unrenderable in a lot of browsers, and fraught with security and privacy problems. (Try pointing a text-only browser at your favorite website. Can you even read the home page?) ---rsk
Dne 28. 5. 2018 v 17:00 Rich Kulawiec napsal(a):
On Mon, May 28, 2018 at 09:23:09AM -0500, Mike Hammett wrote:
Some things certainly do need to be encrypted, but encrypting everything means people with limited Internet access get worse performance OR mechanisms have to be out in place to break ALL encryption, this compromising security and privacy when it's really needed. There are better places to reduce traffic while simultaneously enhancing security and privacy. The new EU version of the home page of USA Today is about 20% the size of the one presented in the US -- because it's had all the tracking and scripting stripped out -- with a concomitant reduction in load time and rendering time. That's awesome, that page fully loaded instantly (roughly in half a second) and uBlock Origin blocked 0 elements. 291KB for the home page.
This is a sight I want to see more. Regards, Filip
I can't imagine rural third-country villages have much influence over the departments of the appropriate companies to affect all of the junk getting added to sites these days. I'm also not foolish enough to think this thread will affect the encrypt-everything crowd as it is more of a religion\ideology than a practical matter. However, maybe it'll shed some light on technical ways of dealing with this at the service-provider level or plant some doubt in someone's mind the next time they think they need to encrypt non-sensitive information. The same goes for all development. My phone is significantly slower today than a couple years ago when new without a significant change in the amount of stuff that I run because developers are lazy and fill the space the latest platforms offer them. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Rich Kulawiec" <rsk@gsp.org> To: nanog@nanog.org Sent: Monday, May 28, 2018 10:00:36 AM Subject: Re: Impacts of Encryption Everywhere (any solution?) On Mon, May 28, 2018 at 09:23:09AM -0500, Mike Hammett wrote:
Some things certainly do need to be encrypted, but encrypting everything means people with limited Internet access get worse performance OR mechanisms have to be out in place to break ALL encryption, this compromising security and privacy when it's really needed.
There are better places to reduce traffic while simultaneously enhancing security and privacy. The new EU version of the home page of USA Today is about 20% the size of the one presented in the US -- because it's had all the tracking and scripting stripped out -- with a concomitant reduction in load time and rendering time. Much more drastic reductions are available elsewhere, e.g., mail messages composed of text only are typically 5% to 10% the size of the same messages marked up with HTML. The problem (part of the problem) is that the people doing these foolish things are new, ignorant, and privileged: they don't realize that bandwidth is still an expensive and scarce resource for most of the planet. I've said for years that every web designer should be forced to work in an environment bandlimited to 56K in order to instll in them the virtue of frugality and strongly discourage them from flattering their egos by creating all-singing all-dancing web sites...that look great in the portfolios they'll show to their peers but are horribly bloated, slow, unrenderable in a lot of browsers, and fraught with security and privacy problems. (Try pointing a text-only browser at your favorite website. Can you even read the home page?) ---rsk
The "do not search a culprit" stuff: What is the point with encryption ? If your users have a very-low bandwidth, they will get a crappy service, with or without encryption This is our world, our http-based internet is NOT made for a 40k connection The "tip stuff": If you simply do not care about encryption, or are willing to trade privacy for caching because you have no-bandwidth, you can simply break SSL It costs nothing, and you will not mind the "red lock" (remember: trade-off) The "philosophical stuff": About your last part, you are absolutely right, this is a sad situation, yet not true Niklaus Wirth (the pascal guy) said in 1995: "Software gets slower faster than hardware gets faster." This has never been so true .. On 05/28/2018 06:09 PM, Mike Hammett wrote:
I can't imagine rural third-country villages have much influence over the departments of the appropriate companies to affect all of the junk getting added to sites these days.
I'm also not foolish enough to think this thread will affect the encrypt-everything crowd as it is more of a religion\ideology than a practical matter. However, maybe it'll shed some light on technical ways of dealing with this at the service-provider level or plant some doubt in someone's mind the next time they think they need to encrypt non-sensitive information.
The same goes for all development. My phone is significantly slower today than a couple years ago when new without a significant change in the amount of stuff that I run because developers are lazy and fill the space the latest platforms offer them.
----- Mike Hammett Intelligent Computing Solutions
Midwest Internet Exchange
The Brothers WISP
----- Original Message -----
From: "Rich Kulawiec" <rsk@gsp.org> To: nanog@nanog.org Sent: Monday, May 28, 2018 10:00:36 AM Subject: Re: Impacts of Encryption Everywhere (any solution?)
On Mon, May 28, 2018 at 09:23:09AM -0500, Mike Hammett wrote:
Some things certainly do need to be encrypted, but encrypting everything means people with limited Internet access get worse performance OR mechanisms have to be out in place to break ALL encryption, this compromising security and privacy when it's really needed.
There are better places to reduce traffic while simultaneously enhancing security and privacy. The new EU version of the home page of USA Today is about 20% the size of the one presented in the US -- because it's had all the tracking and scripting stripped out -- with a concomitant reduction in load time and rendering time. Much more drastic reductions are available elsewhere, e.g., mail messages composed of text only are typically 5% to 10% the size of the same messages marked up with HTML.
The problem (part of the problem) is that the people doing these foolish things are new, ignorant, and privileged: they don't realize that bandwidth is still an expensive and scarce resource for most of the planet. I've said for years that every web designer should be forced to work in an environment bandlimited to 56K in order to instll in them the virtue of frugality and strongly discourage them from flattering their egos by creating all-singing all-dancing web sites...that look great in the portfolios they'll show to their peers but are horribly bloated, slow, unrenderable in a lot of browsers, and fraught with security and privacy problems. (Try pointing a text-only browser at your favorite website. Can you even read the home page?)
---rsk
Once you become sensitized to the HTTPS warnings because www.dickandfartjokes.com needlessly has SSL (or your printer or switch's management interface for those of us not needing to proxy SSL traffic), you now no longer notice that your bank isn't secure. Being hyper-sensitive about SSL causes one to miss things that actually matter. HTTP works just fine over a 40 kb connection. That's all I could get out of my dial-up that I shared to four other computers until about 2004 when I started my WISP. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: nanog@jack.fr.eu.org To: nanog@nanog.org Sent: Monday, May 28, 2018 11:37:46 AM Subject: Re: Impacts of Encryption Everywhere (any solution?) The "do not search a culprit" stuff: What is the point with encryption ? If your users have a very-low bandwidth, they will get a crappy service, with or without encryption This is our world, our http-based internet is NOT made for a 40k connection The "tip stuff": If you simply do not care about encryption, or are willing to trade privacy for caching because you have no-bandwidth, you can simply break SSL It costs nothing, and you will not mind the "red lock" (remember: trade-off) The "philosophical stuff": About your last part, you are absolutely right, this is a sad situation, yet not true Niklaus Wirth (the pascal guy) said in 1995: "Software gets slower faster than hardware gets faster." This has never been so true .. On 05/28/2018 06:09 PM, Mike Hammett wrote:
I can't imagine rural third-country villages have much influence over the departments of the appropriate companies to affect all of the junk getting added to sites these days.
I'm also not foolish enough to think this thread will affect the encrypt-everything crowd as it is more of a religion\ideology than a practical matter. However, maybe it'll shed some light on technical ways of dealing with this at the service-provider level or plant some doubt in someone's mind the next time they think they need to encrypt non-sensitive information.
The same goes for all development. My phone is significantly slower today than a couple years ago when new without a significant change in the amount of stuff that I run because developers are lazy and fill the space the latest platforms offer them.
----- Mike Hammett Intelligent Computing Solutions
Midwest Internet Exchange
The Brothers WISP
----- Original Message -----
From: "Rich Kulawiec" <rsk@gsp.org> To: nanog@nanog.org Sent: Monday, May 28, 2018 10:00:36 AM Subject: Re: Impacts of Encryption Everywhere (any solution?)
On Mon, May 28, 2018 at 09:23:09AM -0500, Mike Hammett wrote:
Some things certainly do need to be encrypted, but encrypting everything means people with limited Internet access get worse performance OR mechanisms have to be out in place to break ALL encryption, this compromising security and privacy when it's really needed.
There are better places to reduce traffic while simultaneously enhancing security and privacy. The new EU version of the home page of USA Today is about 20% the size of the one presented in the US -- because it's had all the tracking and scripting stripped out -- with a concomitant reduction in load time and rendering time. Much more drastic reductions are available elsewhere, e.g., mail messages composed of text only are typically 5% to 10% the size of the same messages marked up with HTML.
The problem (part of the problem) is that the people doing these foolish things are new, ignorant, and privileged: they don't realize that bandwidth is still an expensive and scarce resource for most of the planet. I've said for years that every web designer should be forced to work in an environment bandlimited to 56K in order to instll in them the virtue of frugality and strongly discourage them from flattering their egos by creating all-singing all-dancing web sites...that look great in the portfolios they'll show to their peers but are horribly bloated, slow, unrenderable in a lot of browsers, and fraught with security and privacy problems. (Try pointing a text-only browser at your favorite website. Can you even read the home page?)
---rsk
I'm also not foolish enough to think this thread will affect the encrypt-everything crowd as it is more of a religion\ideology than a practical matter. However, maybe it'll shed some light on technical ways of dealing with this at the service-provider level or plant some doubt in someone's mind the next time they think they need to encrypt non-sensitive information.
Good Luck, especially in light of the poo-for-brains at Google responsible for the Chrome browser who (wrongly) equate "secure" with Transport Encryption and "unsecure" with not having Transport Encryption; when all that Transport Encryption really implies is Transport Encryption and not much else. It has little to do with whether or not a site is "secure". Generally speaking, I have found that sites engaging Transport Security are much more "unsecure" (as in subject to security breaches and flaws) than those that do not engage Transport Security for no reason. However, the poo-for-brains crowd will get everyone to engage Transport Security so the will be called "Secure", whether trustworthy or not. --- The fact that there's a Highway to Hell but only a Stairway to Heaven says a lot about anticipated traffic volume.
On Mon, May 28, 2018 at 1:55 PM, Keith Medcalf <kmedcalf@dessus.com> wrote:
I'm also not foolish enough to think this thread will affect the encrypt-everything crowd as it is more of a religion\ideology than a practical matter. However, maybe it'll shed some light on technical ways of dealing with this at the service-provider level or plant some doubt in someone's mind the next time they think they need to encrypt non-sensitive information.
Good Luck, especially in light of the poo-for-brains at Google responsible for the Chrome browser who (wrongly) equate "secure" with Transport Encryption and "unsecure" with not having Transport Encryption; when all that Transport Encryption really implies is Transport Encryption and not much else. It has little to do with whether or not a site is "secure". Generally speaking, I have found that sites engaging Transport Security are much more "unsecure" (as in subject to security breaches and flaws) than those that do not engage Transport Security for no reason.
However, the poo-for-brains crowd will get everyone to engage Transport Security so the will be called "Secure", whether trustworthy or not.
Actually, starting July Chrome will no longer say "secure" for sites with Transport Security. It will only say "not secure" for sites without, so it will no longer provide the false impression of equating Transport Security with Application/Operational Security. Rubens
*nods* The whole concept of SSL all of the things is severely misplaced... and the thread I caught exemplifies why. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Keith Medcalf" <kmedcalf@dessus.com> To: nanog@nanog.org Cc: "Mike Hammett" <nanog@ics-il.net> Sent: Monday, May 28, 2018 11:55:21 AM Subject: RE: Impacts of Encryption Everywhere (any solution?)
I'm also not foolish enough to think this thread will affect the encrypt-everything crowd as it is more of a religion\ideology than a practical matter. However, maybe it'll shed some light on technical ways of dealing with this at the service-provider level or plant some doubt in someone's mind the next time they think they need to encrypt non-sensitive information.
Good Luck, especially in light of the poo-for-brains at Google responsible for the Chrome browser who (wrongly) equate "secure" with Transport Encryption and "unsecure" with not having Transport Encryption; when all that Transport Encryption really implies is Transport Encryption and not much else. It has little to do with whether or not a site is "secure". Generally speaking, I have found that sites engaging Transport Security are much more "unsecure" (as in subject to security breaches and flaws) than those that do not engage Transport Security for no reason. However, the poo-for-brains crowd will get everyone to engage Transport Security so the will be called "Secure", whether trustworthy or not. --- The fact that there's a Highway to Hell but only a Stairway to Heaven says a lot about anticipated traffic volume.
On 05/28/2018 08:23 AM, Mike Hammett wrote:
To circle back to being somewhat on-topic, what mechanisms are available to maximize the amount of traffic someone in this situation could cache? The performance of third-world Internet depends on you.
I've personally played with Squid's SSL-bump-in-the-wire mode (on my personal systems) and was moderately happy with it. - I think that such is a realistic possibility in the scenario that you describe. I would REQUIRE /open/ and /transparent/ communications from the ISP and a *VERY* strict security control to the caching proxy. I would naively like to believe that an ISP could establish a reputation with the community and build a trust relationship such that the community was somewhat okay with the SSL-bump-in-the-wire. It might even be worth leveraging WPAD or PAC to route specific URLs direct to some places (banks, etc) to mitigate some of the security risk. I would also advocate another proxy on the upstream side of the 1 Mbps connection (in the cloud if you will) primarily for the purpose of it doing as much traffic optimization as possible. Have it fetch things and deal with fragments so that it can homogenize the traffic before it's sent across the across the slow link. I'd think seriously about throwing some CPU (a single core off of any machine in the last 10 years should be sufficient) at compression to try to stretch the bandwidth between the two proxy servers. I'd also think seriously about a local root DNS zone slave downstream, and any other zone that I could slave, for the purpose of minimizing the number of queries that need to get pushed across the link. I've been assuming that this 1 Mbps link is terrestrial. Which means that I'd also explore something like a satellite link with more bandwidth. Sure the latency on it will be higher, but that can be worked with. Particularly if you can use some intelligence to route different CoS / ToS / DiffServ (DSCP) across the different links. I think there are options and things that can be done to make this viable. Also, considering that the village has been using a 40 kbps link, sharing a 1 Mbps (or 1,000 kbps) link is going to be a LOT better than it was. The question is, how do you stretch a good thing as far as possible. Finally, will you please provide some pointers to the discussion you're talking about? I'd like to read it if possible. -- Grant. . . . unix || die
In addition to the "bump in the wire" you could also enable larger frame sizes downstream since you're already completely disassembling and reassembling the packets. Large downloads or uploads could see overhead go from 3% at 1500B to about 0.5% at 9100B. It's not much but every little bit counts. (Preamble, Ethernet, IP, and TCP headers all need be sent accross the circuit less often to get the same amount of data through) Looking only at the throughput of L4 payloads, you get: 1500 MTU = 956 kbps 9100 MTU = 992 kbps That almost adds a whole additional home if my math is correct. -Matt On Mon, May 28, 2018, 11:17 Grant Taylor via NANOG <nanog@nanog.org> wrote:
On 05/28/2018 08:23 AM, Mike Hammett wrote:
To circle back to being somewhat on-topic, what mechanisms are available to maximize the amount of traffic someone in this situation could cache? The performance of third-world Internet depends on you.
I've personally played with Squid's SSL-bump-in-the-wire mode (on my personal systems) and was moderately happy with it. - I think that such is a realistic possibility in the scenario that you describe.
I would REQUIRE /open/ and /transparent/ communications from the ISP and a *VERY* strict security control to the caching proxy. I would naively like to believe that an ISP could establish a reputation with the community and build a trust relationship such that the community was somewhat okay with the SSL-bump-in-the-wire.
It might even be worth leveraging WPAD or PAC to route specific URLs direct to some places (banks, etc) to mitigate some of the security risk.
I would also advocate another proxy on the upstream side of the 1 Mbps connection (in the cloud if you will) primarily for the purpose of it doing as much traffic optimization as possible. Have it fetch things and deal with fragments so that it can homogenize the traffic before it's sent across the across the slow link. I'd think seriously about throwing some CPU (a single core off of any machine in the last 10 years should be sufficient) at compression to try to stretch the bandwidth between the two proxy servers.
I'd also think seriously about a local root DNS zone slave downstream, and any other zone that I could slave, for the purpose of minimizing the number of queries that need to get pushed across the link.
I've been assuming that this 1 Mbps link is terrestrial. Which means that I'd also explore something like a satellite link with more bandwidth. Sure the latency on it will be higher, but that can be worked with. Particularly if you can use some intelligence to route different CoS / ToS / DiffServ (DSCP) across the different links.
I think there are options and things that can be done to make this viable.
Also, considering that the village has been using a 40 kbps link, sharing a 1 Mbps (or 1,000 kbps) link is going to be a LOT better than it was. The question is, how do you stretch a good thing as far as possible.
Finally, will you please provide some pointers to the discussion you're talking about? I'd like to read it if possible.
-- Grant. . . . unix || die
To be fair, most of the conversation is people not realizing the OP is in a third world country and believe that 1 mbit/s isn't enough for a single user much less a village. https://www.facebook.com/groups/ubntedgeos/permalink/1046305928855488/ Also, I think it's 40 kilotbit/s per user (so probably dial-up), not 40 kilobit/s for the whole village. The whole village may very well have 1 megabit/s worth of dial-up connections, but everyone potentially able to go to 1 megabit is a lot more useful than capping each to 40 kilobit/s. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Grant Taylor via NANOG" <nanog@nanog.org> To: nanog@nanog.org Sent: Monday, May 28, 2018 11:17:10 AM Subject: Re: Impacts of Encryption Everywhere (any solution?) On 05/28/2018 08:23 AM, Mike Hammett wrote:
To circle back to being somewhat on-topic, what mechanisms are available to maximize the amount of traffic someone in this situation could cache? The performance of third-world Internet depends on you.
I've personally played with Squid's SSL-bump-in-the-wire mode (on my personal systems) and was moderately happy with it. - I think that such is a realistic possibility in the scenario that you describe. I would REQUIRE /open/ and /transparent/ communications from the ISP and a *VERY* strict security control to the caching proxy. I would naively like to believe that an ISP could establish a reputation with the community and build a trust relationship such that the community was somewhat okay with the SSL-bump-in-the-wire. It might even be worth leveraging WPAD or PAC to route specific URLs direct to some places (banks, etc) to mitigate some of the security risk. I would also advocate another proxy on the upstream side of the 1 Mbps connection (in the cloud if you will) primarily for the purpose of it doing as much traffic optimization as possible. Have it fetch things and deal with fragments so that it can homogenize the traffic before it's sent across the across the slow link. I'd think seriously about throwing some CPU (a single core off of any machine in the last 10 years should be sufficient) at compression to try to stretch the bandwidth between the two proxy servers. I'd also think seriously about a local root DNS zone slave downstream, and any other zone that I could slave, for the purpose of minimizing the number of queries that need to get pushed across the link. I've been assuming that this 1 Mbps link is terrestrial. Which means that I'd also explore something like a satellite link with more bandwidth. Sure the latency on it will be higher, but that can be worked with. Particularly if you can use some intelligence to route different CoS / ToS / DiffServ (DSCP) across the different links. I think there are options and things that can be done to make this viable. Also, considering that the village has been using a 40 kbps link, sharing a 1 Mbps (or 1,000 kbps) link is going to be a LOT better than it was. The question is, how do you stretch a good thing as far as possible. Finally, will you please provide some pointers to the discussion you're talking about? I'd like to read it if possible. -- Grant. . . . unix || die
I’m sorry I simply believe that in 2018 with the advanced and cheap ptp radio (ubiquiti anyone? $300 and I have a 200mbit/sec link over 10miles! Spend a bit more and go 100km) plus the advancements in cubesats about to be launched, even the 3rd world can simply get with the times. -Ben
On May 28, 2018, at 10:57 AM, Mike Hammett <nanog@ics-il.net> wrote:
To be fair, most of the conversation is people not realizing the OP is in a third world country and believe that 1 mbit/s isn't enough for a single user much less a village.
https://www.facebook.com/groups/ubntedgeos/permalink/1046305928855488/
Also, I think it's 40 kilotbit/s per user (so probably dial-up), not 40 kilobit/s for the whole village. The whole village may very well have 1 megabit/s worth of dial-up connections, but everyone potentially able to go to 1 megabit is a lot more useful than capping each to 40 kilobit/s.
----- Mike Hammett Intelligent Computing Solutions
Midwest Internet Exchange
The Brothers WISP
----- Original Message -----
From: "Grant Taylor via NANOG" <nanog@nanog.org> To: nanog@nanog.org Sent: Monday, May 28, 2018 11:17:10 AM Subject: Re: Impacts of Encryption Everywhere (any solution?)
On 05/28/2018 08:23 AM, Mike Hammett wrote: To circle back to being somewhat on-topic, what mechanisms are available to maximize the amount of traffic someone in this situation could cache? The performance of third-world Internet depends on you.
I've personally played with Squid's SSL-bump-in-the-wire mode (on my personal systems) and was moderately happy with it. - I think that such is a realistic possibility in the scenario that you describe.
I would REQUIRE /open/ and /transparent/ communications from the ISP and a *VERY* strict security control to the caching proxy. I would naively like to believe that an ISP could establish a reputation with the community and build a trust relationship such that the community was somewhat okay with the SSL-bump-in-the-wire.
It might even be worth leveraging WPAD or PAC to route specific URLs direct to some places (banks, etc) to mitigate some of the security risk.
I would also advocate another proxy on the upstream side of the 1 Mbps connection (in the cloud if you will) primarily for the purpose of it doing as much traffic optimization as possible. Have it fetch things and deal with fragments so that it can homogenize the traffic before it's sent across the across the slow link. I'd think seriously about throwing some CPU (a single core off of any machine in the last 10 years should be sufficient) at compression to try to stretch the bandwidth between the two proxy servers.
I'd also think seriously about a local root DNS zone slave downstream, and any other zone that I could slave, for the purpose of minimizing the number of queries that need to get pushed across the link.
I've been assuming that this 1 Mbps link is terrestrial. Which means that I'd also explore something like a satellite link with more bandwidth. Sure the latency on it will be higher, but that can be worked with. Particularly if you can use some intelligence to route different CoS / ToS / DiffServ (DSCP) across the different links.
I think there are options and things that can be done to make this viable.
Also, considering that the village has been using a 40 kbps link, sharing a 1 Mbps (or 1,000 kbps) link is going to be a LOT better than it was. The question is, how do you stretch a good thing as far as possible.
Finally, will you please provide some pointers to the discussion you're talking about? I'd like to read it if possible.
-- Grant. . . . unix || die
I know the fixed wireless space quite well. If there's no Internet to be had, it doesn't matter how quickly you can distribute it. He did say that (for whatever reason), relaying off of mountain-top sites to get to better connectivity wasn't a viable option. The yet-to-be-deployed satellite constellations don't do anyone any good today. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Ben Cannon" <ben@6by7.net> To: "Mike Hammett" <nanog@ics-il.net> Cc: nanog@nanog.org Sent: Monday, May 28, 2018 1:22:27 PM Subject: Re: Impacts of Encryption Everywhere (any solution?) I’m sorry I simply believe that in 2018 with the advanced and cheap ptp radio (ubiquiti anyone? $300 and I have a 200mbit/sec link over 10miles! Spend a bit more and go 100km) plus the advancements in cubesats about to be launched, even the 3rd world can simply get with the times. -Ben
On May 28, 2018, at 10:57 AM, Mike Hammett <nanog@ics-il.net> wrote:
To be fair, most of the conversation is people not realizing the OP is in a third world country and believe that 1 mbit/s isn't enough for a single user much less a village.
https://www.facebook.com/groups/ubntedgeos/permalink/1046305928855488/
Also, I think it's 40 kilotbit/s per user (so probably dial-up), not 40 kilobit/s for the whole village. The whole village may very well have 1 megabit/s worth of dial-up connections, but everyone potentially able to go to 1 megabit is a lot more useful than capping each to 40 kilobit/s.
----- Mike Hammett Intelligent Computing Solutions
Midwest Internet Exchange
The Brothers WISP
----- Original Message -----
From: "Grant Taylor via NANOG" <nanog@nanog.org> To: nanog@nanog.org Sent: Monday, May 28, 2018 11:17:10 AM Subject: Re: Impacts of Encryption Everywhere (any solution?)
On 05/28/2018 08:23 AM, Mike Hammett wrote: To circle back to being somewhat on-topic, what mechanisms are available to maximize the amount of traffic someone in this situation could cache? The performance of third-world Internet depends on you.
I've personally played with Squid's SSL-bump-in-the-wire mode (on my personal systems) and was moderately happy with it. - I think that such is a realistic possibility in the scenario that you describe.
I would REQUIRE /open/ and /transparent/ communications from the ISP and a *VERY* strict security control to the caching proxy. I would naively like to believe that an ISP could establish a reputation with the community and build a trust relationship such that the community was somewhat okay with the SSL-bump-in-the-wire.
It might even be worth leveraging WPAD or PAC to route specific URLs direct to some places (banks, etc) to mitigate some of the security risk.
I would also advocate another proxy on the upstream side of the 1 Mbps connection (in the cloud if you will) primarily for the purpose of it doing as much traffic optimization as possible. Have it fetch things and deal with fragments so that it can homogenize the traffic before it's sent across the across the slow link. I'd think seriously about throwing some CPU (a single core off of any machine in the last 10 years should be sufficient) at compression to try to stretch the bandwidth between the two proxy servers.
I'd also think seriously about a local root DNS zone slave downstream, and any other zone that I could slave, for the purpose of minimizing the number of queries that need to get pushed across the link.
I've been assuming that this 1 Mbps link is terrestrial. Which means that I'd also explore something like a satellite link with more bandwidth. Sure the latency on it will be higher, but that can be worked with. Particularly if you can use some intelligence to route different CoS / ToS / DiffServ (DSCP) across the different links.
I think there are options and things that can be done to make this viable.
Also, considering that the village has been using a 40 kbps link, sharing a 1 Mbps (or 1,000 kbps) link is going to be a LOT better than it was. The question is, how do you stretch a good thing as far as possible.
Finally, will you please provide some pointers to the discussion you're talking about? I'd like to read it if possible.
-- Grant. . . . unix || die
On Mon, May 28, 2018 at 11:22 AM, Ben Cannon <ben@6by7.net> wrote:
I’m sorry I simply believe that in 2018 with the advanced and cheap ptp radio (ubiquiti anyone? $300 and I have a 200mbit/sec link over 10miles! Spend a bit more and go 100km) plus the advancements in cubesats about to be launched, even the 3rd world can simply get with the times.
-Ben
Hi Ben, I do not think you adequately understand the economics of the situation. https://www.slideshare.net/InternetSociety/international-bandwidth-and-prici... slide 22, IP transit cost. Your 200mbit/sec link that costs you $300 in hardware is going to cost you $4960/month to actually get IP traffic across, in Nairobi. Yes, that's about $60,000/year. Could *you* afford to "get with the times" if that's what your bandwidth was going to cost you? Please, do a little research on what the real costs are before telling others they need to "simply get with the times." Thanks! Matt
Your 200mbit/sec link that costs you $300 in hardware is going to cost you $4960/month to actually get IP traffic across, in Nairobi. Yes, that's about $60,000/year. I live in the US of A, and this is what 200Mb/s roughly would cost me as well here in Rural Monopoly-land. Rural ILEC also has the CATV business, and, well, they are _not_ going to run cable up here. I've actually priced 150Mb/s bandwidth from the ILEC over the years; in 2003
On 05/28/2018 06:13 PM, Matthew Petach wrote: the cost would have been about $100,000 per month. As of five years ago 10Mb/s symmetrical cost roughly $1,000 per month, the lion's share of that being per-mile NECA Tariff 5 transport costs. The terrain here prevents fixed wireless. The terrain also prevents satellite comms to the Clarke belt (mountain to the south with trees on US Forest Service property in the line of sight). I get 1XRTT in one room of my house when the humidity is below 70% and it's winter, and once in a blue moon 3G will light up, but it's not stable enough to actually use; it's the speed of dialup. If I traipse about a hundred yards up the mountain to the south (onto US Forest Service property, so, no repeater for me) I can get semi-usable 4G; nothing like being in the middle of the woods with an active black bear population trying to get a usable signal. I'm paying $50 per month for 7/0.5 DSL (I might add that they provide excellent DSL that has been extremely reliable) from the only ISP available in the area. I remember a usable web experience not too long ago on 28.8K/33.6K dialup (it was quite a while before said ILEC got a 56K-capable modem bank). DSL started out here at 384k/128k. On the positive side, we have a very low oversubscription ratio, so I actually get the full bandwidth the majority of the time, even video streaming. I also know all the network engineers there, too, and that also has its advantages. (Yes, I am aware that rural living is a choice, and there are things worth a great deal more than bandwidth, that it's a tradeoff, etc.) So it's not just '3rd-world' countries with expensive bandwidth.
I am incredibly rural in Pennsylvania and pay about $.50 per megabit.
On May 29, 2018, at 09:23, Lamar Owen <lowen@pari.edu> wrote:
On 05/28/2018 06:13 PM, Matthew Petach wrote: Your 200mbit/sec link that costs you $300 in hardware is going to cost you $4960/month to actually get IP traffic across, in Nairobi. Yes, that's about $60,000/year. I live in the US of A, and this is what 200Mb/s roughly would cost me as well here in Rural Monopoly-land. Rural ILEC also has the CATV business, and, well, they are _not_ going to run cable up here. I've actually priced 150Mb/s bandwidth from the ILEC over the years; in 2003 the cost would have been about $100,000 per month. As of five years ago 10Mb/s symmetrical cost roughly $1,000 per month, the lion's share of that being per-mile NECA Tariff 5 transport costs.
The terrain here prevents fixed wireless. The terrain also prevents satellite comms to the Clarke belt (mountain to the south with trees on US Forest Service property in the line of sight). I get 1XRTT in one room of my house when the humidity is below 70% and it's winter, and once in a blue moon 3G will light up, but it's not stable enough to actually use; it's the speed of dialup. If I traipse about a hundred yards up the mountain to the south (onto US Forest Service property, so, no repeater for me) I can get semi-usable 4G; nothing like being in the middle of the woods with an active black bear population trying to get a usable signal.
I'm paying $50 per month for 7/0.5 DSL (I might add that they provide excellent DSL that has been extremely reliable) from the only ISP available in the area.
I remember a usable web experience not too long ago on 28.8K/33.6K dialup (it was quite a while before said ILEC got a 56K-capable modem bank). DSL started out here at 384k/128k. On the positive side, we have a very low oversubscription ratio, so I actually get the full bandwidth the majority of the time, even video streaming. I also know all the network engineers there, too, and that also has its advantages.
(Yes, I am aware that rural living is a choice, and there are things worth a great deal more than bandwidth, that it's a tradeoff, etc.)
So it's not just '3rd-world' countries with expensive bandwidth.
I guess not all rurals are the same. In my parts, being rural could mean not having a 2G/3G signal until you have to climb a tree... not literally, but you get my point. Mark. On 29/May/18 15:27, Matt Hoppes wrote:
I am incredibly rural in Pennsylvania and pay about $.50 per megabit.
On May 29, 2018, at 09:23, Lamar Owen <lowen@pari.edu> wrote:
On 05/28/2018 06:13 PM, Matthew Petach wrote: Your 200mbit/sec link that costs you $300 in hardware is going to cost you $4960/month to actually get IP traffic across, in Nairobi. Yes, that's about $60,000/year. I live in the US of A, and this is what 200Mb/s roughly would cost me as well here in Rural Monopoly-land. Rural ILEC also has the CATV business, and, well, they are _not_ going to run cable up here. I've actually priced 150Mb/s bandwidth from the ILEC over the years; in 2003 the cost would have been about $100,000 per month. As of five years ago 10Mb/s symmetrical cost roughly $1,000 per month, the lion's share of that being per-mile NECA Tariff 5 transport costs.
The terrain here prevents fixed wireless. The terrain also prevents satellite comms to the Clarke belt (mountain to the south with trees on US Forest Service property in the line of sight). I get 1XRTT in one room of my house when the humidity is below 70% and it's winter, and once in a blue moon 3G will light up, but it's not stable enough to actually use; it's the speed of dialup. If I traipse about a hundred yards up the mountain to the south (onto US Forest Service property, so, no repeater for me) I can get semi-usable 4G; nothing like being in the middle of the woods with an active black bear population trying to get a usable signal.
I'm paying $50 per month for 7/0.5 DSL (I might add that they provide excellent DSL that has been extremely reliable) from the only ISP available in the area.
I remember a usable web experience not too long ago on 28.8K/33.6K dialup (it was quite a while before said ILEC got a 56K-capable modem bank). DSL started out here at 384k/128k. On the positive side, we have a very low oversubscription ratio, so I actually get the full bandwidth the majority of the time, even video streaming. I also know all the network engineers there, too, and that also has its advantages.
(Yes, I am aware that rural living is a choice, and there are things worth a great deal more than bandwidth, that it's a tradeoff, etc.)
So it's not just '3rd-world' countries with expensive bandwidth.
.
Is that PennRen\Kinber? ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Matt Hoppes" <mattlists@rivervalleyinternet.net> To: "Lamar Owen" <lowen@pari.edu> Cc: nanog@nanog.org Sent: Tuesday, May 29, 2018 8:27:17 AM Subject: Re: Impacts of Encryption Everywhere (any solution?) I am incredibly rural in Pennsylvania and pay about $.50 per megabit.
On May 29, 2018, at 09:23, Lamar Owen <lowen@pari.edu> wrote:
On 05/28/2018 06:13 PM, Matthew Petach wrote: Your 200mbit/sec link that costs you $300 in hardware is going to cost you $4960/month to actually get IP traffic across, in Nairobi. Yes, that's about $60,000/year. I live in the US of A, and this is what 200Mb/s roughly would cost me as well here in Rural Monopoly-land. Rural ILEC also has the CATV business, and, well, they are _not_ going to run cable up here. I've actually priced 150Mb/s bandwidth from the ILEC over the years; in 2003 the cost would have been about $100,000 per month. As of five years ago 10Mb/s symmetrical cost roughly $1,000 per month, the lion's share of that being per-mile NECA Tariff 5 transport costs.
The terrain here prevents fixed wireless. The terrain also prevents satellite comms to the Clarke belt (mountain to the south with trees on US Forest Service property in the line of sight). I get 1XRTT in one room of my house when the humidity is below 70% and it's winter, and once in a blue moon 3G will light up, but it's not stable enough to actually use; it's the speed of dialup. If I traipse about a hundred yards up the mountain to the south (onto US Forest Service property, so, no repeater for me) I can get semi-usable 4G; nothing like being in the middle of the woods with an active black bear population trying to get a usable signal.
I'm paying $50 per month for 7/0.5 DSL (I might add that they provide excellent DSL that has been extremely reliable) from the only ISP available in the area.
I remember a usable web experience not too long ago on 28.8K/33.6K dialup (it was quite a while before said ILEC got a 56K-capable modem bank). DSL started out here at 384k/128k. On the positive side, we have a very low oversubscription ratio, so I actually get the full bandwidth the majority of the time, even video streaming. I also know all the network engineers there, too, and that also has its advantages.
(Yes, I am aware that rural living is a choice, and there are things worth a great deal more than bandwidth, that it's a tradeoff, etc.)
So it's not just '3rd-world' countries with expensive bandwidth.
$100M+ in federal dollars goes a long way. On 5/29/2018 10:17 AM, Mike Hammett wrote:
Is that PennRen\Kinber?
----- Mike Hammett Intelligent Computing Solutions
Midwest Internet Exchange
The Brothers WISP
----- Original Message -----
From: "Matt Hoppes" <mattlists@rivervalleyinternet.net> To: "Lamar Owen" <lowen@pari.edu> Cc: nanog@nanog.org Sent: Tuesday, May 29, 2018 8:27:17 AM Subject: Re: Impacts of Encryption Everywhere (any solution?)
I am incredibly rural in Pennsylvania and pay about $.50 per megabit.
On May 29, 2018, at 09:23, Lamar Owen <lowen@pari.edu> wrote:
On 05/28/2018 06:13 PM, Matthew Petach wrote: Your 200mbit/sec link that costs you $300 in hardware is going to cost you $4960/month to actually get IP traffic across, in Nairobi. Yes, that's about $60,000/year. I live in the US of A, and this is what 200Mb/s roughly would cost me as well here in Rural Monopoly-land. Rural ILEC also has the CATV business, and, well, they are _not_ going to run cable up here. I've actually priced 150Mb/s bandwidth from the ILEC over the years; in 2003 the cost would have been about $100,000 per month. As of five years ago 10Mb/s symmetrical cost roughly $1,000 per month, the lion's share of that being per-mile NECA Tariff 5 transport costs.
The terrain here prevents fixed wireless. The terrain also prevents satellite comms to the Clarke belt (mountain to the south with trees on US Forest Service property in the line of sight). I get 1XRTT in one room of my house when the humidity is below 70% and it's winter, and once in a blue moon 3G will light up, but it's not stable enough to actually use; it's the speed of dialup. If I traipse about a hundred yards up the mountain to the south (onto US Forest Service property, so, no repeater for me) I can get semi-usable 4G; nothing like being in the middle of the woods with an active black bear population trying to get a usable signal.
I'm paying $50 per month for 7/0.5 DSL (I might add that they provide excellent DSL that has been extremely reliable) from the only ISP available in the area.
I remember a usable web experience not too long ago on 28.8K/33.6K dialup (it was quite a while before said ILEC got a 56K-capable modem bank). DSL started out here at 384k/128k. On the positive side, we have a very low oversubscription ratio, so I actually get the full bandwidth the majority of the time, even video streaming. I also know all the network engineers there, too, and that also has its advantages.
(Yes, I am aware that rural living is a choice, and there are things worth a great deal more than bandwidth, that it's a tradeoff, etc.)
So it's not just '3rd-world' countries with expensive bandwidth.
Ah, the wonderful USF. Here’s my take on USF. It’s a perfectly wonderful intent whose implementation has gone horribly horribly wrong. Instead of equalizing economic incentives for infrastructure between rural, urban, and suburban areas, it has heavily tilted the incentives in favor of the highest densities that still qualify as rural while pretty much screwing over everyone else. Extremely high density urban areas still have sufficient economic opportunity over lower infrastructure cost per user to attract some development. However, Suburbia is the biggest loser in this equation. Don’t get me wrong… I’m perfectly fine with the idea that I need to make a small payment to subsidize delivery of decent network infrastructure to underserved areas. What bothers me is that I’m generally paying this tax to enable farmers in the middle of nowhere to have better network infrastructure than I can get at my own location. I’m happy to subsidize equality of connectivity, but it galls me to have to subsidize GPON for others while there’s not even a glimmer of hope that anyone will usefully lay fiber in my neighborhood in the foreseeable future. Owen
On May 29, 2018, at 07:23 , ML <ml@kenweb.org> wrote:
$100M+ in federal dollars goes a long way.
On 5/29/2018 10:17 AM, Mike Hammett wrote:
Is that PennRen\Kinber?
----- Mike Hammett Intelligent Computing Solutions
Midwest Internet Exchange
The Brothers WISP
----- Original Message -----
From: "Matt Hoppes" <mattlists@rivervalleyinternet.net> To: "Lamar Owen" <lowen@pari.edu> Cc: nanog@nanog.org Sent: Tuesday, May 29, 2018 8:27:17 AM Subject: Re: Impacts of Encryption Everywhere (any solution?)
I am incredibly rural in Pennsylvania and pay about $.50 per megabit.
On May 29, 2018, at 09:23, Lamar Owen <lowen@pari.edu> wrote:
On 05/28/2018 06:13 PM, Matthew Petach wrote: Your 200mbit/sec link that costs you $300 in hardware is going to cost you $4960/month to actually get IP traffic across, in Nairobi. Yes, that's about $60,000/year. I live in the US of A, and this is what 200Mb/s roughly would cost me as well here in Rural Monopoly-land. Rural ILEC also has the CATV business, and, well, they are _not_ going to run cable up here. I've actually priced 150Mb/s bandwidth from the ILEC over the years; in 2003 the cost would have been about $100,000 per month. As of five years ago 10Mb/s symmetrical cost roughly $1,000 per month, the lion's share of that being per-mile NECA Tariff 5 transport costs.
The terrain here prevents fixed wireless. The terrain also prevents satellite comms to the Clarke belt (mountain to the south with trees on US Forest Service property in the line of sight). I get 1XRTT in one room of my house when the humidity is below 70% and it's winter, and once in a blue moon 3G will light up, but it's not stable enough to actually use; it's the speed of dialup. If I traipse about a hundred yards up the mountain to the south (onto US Forest Service property, so, no repeater for me) I can get semi-usable 4G; nothing like being in the middle of the woods with an active black bear population trying to get a usable signal.
I'm paying $50 per month for 7/0.5 DSL (I might add that they provide excellent DSL that has been extremely reliable) from the only ISP available in the area.
I remember a usable web experience not too long ago on 28.8K/33.6K dialup (it was quite a while before said ILEC got a 56K-capable modem bank). DSL started out here at 384k/128k. On the positive side, we have a very low oversubscription ratio, so I actually get the full bandwidth the majority of the time, even video streaming. I also know all the network engineers there, too, and that also has its advantages.
(Yes, I am aware that rural living is a choice, and there are things worth a great deal more than bandwidth, that it's a tradeoff, etc.)
So it's not just '3rd-world' countries with expensive bandwidth.
Multiple providers. I don’t think I should publicly name them for various reasons. You are a smart man though and can probably figure it out from BGP peering tables.
On May 29, 2018, at 10:17, Mike Hammett <nanog@ics-il.net> wrote:
Is that PennRen\Kinber?
----- Mike Hammett Intelligent Computing Solutions
Midwest Internet Exchange
The Brothers WISP
----- Original Message -----
From: "Matt Hoppes" <mattlists@rivervalleyinternet.net> To: "Lamar Owen" <lowen@pari.edu> Cc: nanog@nanog.org Sent: Tuesday, May 29, 2018 8:27:17 AM Subject: Re: Impacts of Encryption Everywhere (any solution?)
I am incredibly rural in Pennsylvania and pay about $.50 per megabit.
On May 29, 2018, at 09:23, Lamar Owen <lowen@pari.edu> wrote:
On 05/28/2018 06:13 PM, Matthew Petach wrote: Your 200mbit/sec link that costs you $300 in hardware is going to cost you $4960/month to actually get IP traffic across, in Nairobi. Yes, that's about $60,000/year. I live in the US of A, and this is what 200Mb/s roughly would cost me as well here in Rural Monopoly-land. Rural ILEC also has the CATV business, and, well, they are _not_ going to run cable up here. I've actually priced 150Mb/s bandwidth from the ILEC over the years; in 2003 the cost would have been about $100,000 per month. As of five years ago 10Mb/s symmetrical cost roughly $1,000 per month, the lion's share of that being per-mile NECA Tariff 5 transport costs.
The terrain here prevents fixed wireless. The terrain also prevents satellite comms to the Clarke belt (mountain to the south with trees on US Forest Service property in the line of sight). I get 1XRTT in one room of my house when the humidity is below 70% and it's winter, and once in a blue moon 3G will light up, but it's not stable enough to actually use; it's the speed of dialup. If I traipse about a hundred yards up the mountain to the south (onto US Forest Service property, so, no repeater for me) I can get semi-usable 4G; nothing like being in the middle of the woods with an active black bear population trying to get a usable signal.
I'm paying $50 per month for 7/0.5 DSL (I might add that they provide excellent DSL that has been extremely reliable) from the only ISP available in the area.
I remember a usable web experience not too long ago on 28.8K/33.6K dialup (it was quite a while before said ILEC got a 56K-capable modem bank). DSL started out here at 384k/128k. On the positive side, we have a very low oversubscription ratio, so I actually get the full bandwidth the majority of the time, even video streaming. I also know all the network engineers there, too, and that also has its advantages.
(Yes, I am aware that rural living is a choice, and there are things worth a great deal more than bandwidth, that it's a tradeoff, etc.)
So it's not just '3rd-world' countries with expensive bandwidth.
I know who you have and it's easily found who you use. I was implying exactly what "ML" said". ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Matt Hoppes" <mattlists@rivervalleyinternet.net> To: "Mike Hammett" <nanog@ics-il.net> Cc: nanog@nanog.org Sent: Tuesday, May 29, 2018 9:24:41 AM Subject: Re: Impacts of Encryption Everywhere (any solution?) Multiple providers. I don’t think I should publicly name them for various reasons. You are a smart man though and can probably figure it out from BGP peering tables.
On May 29, 2018, at 10:17, Mike Hammett <nanog@ics-il.net> wrote:
Is that PennRen\Kinber?
----- Mike Hammett Intelligent Computing Solutions
Midwest Internet Exchange
The Brothers WISP
----- Original Message -----
From: "Matt Hoppes" <mattlists@rivervalleyinternet.net> To: "Lamar Owen" <lowen@pari.edu> Cc: nanog@nanog.org Sent: Tuesday, May 29, 2018 8:27:17 AM Subject: Re: Impacts of Encryption Everywhere (any solution?)
I am incredibly rural in Pennsylvania and pay about $.50 per megabit.
On May 29, 2018, at 09:23, Lamar Owen <lowen@pari.edu> wrote:
On 05/28/2018 06:13 PM, Matthew Petach wrote: Your 200mbit/sec link that costs you $300 in hardware is going to cost you $4960/month to actually get IP traffic across, in Nairobi. Yes, that's about $60,000/year. I live in the US of A, and this is what 200Mb/s roughly would cost me as well here in Rural Monopoly-land. Rural ILEC also has the CATV business, and, well, they are _not_ going to run cable up here. I've actually priced 150Mb/s bandwidth from the ILEC over the years; in 2003 the cost would have been about $100,000 per month. As of five years ago 10Mb/s symmetrical cost roughly $1,000 per month, the lion's share of that being per-mile NECA Tariff 5 transport costs.
The terrain here prevents fixed wireless. The terrain also prevents satellite comms to the Clarke belt (mountain to the south with trees on US Forest Service property in the line of sight). I get 1XRTT in one room of my house when the humidity is below 70% and it's winter, and once in a blue moon 3G will light up, but it's not stable enough to actually use; it's the speed of dialup. If I traipse about a hundred yards up the mountain to the south (onto US Forest Service property, so, no repeater for me) I can get semi-usable 4G; nothing like being in the middle of the woods with an active black bear population trying to get a usable signal.
I'm paying $50 per month for 7/0.5 DSL (I might add that they provide excellent DSL that has been extremely reliable) from the only ISP available in the area.
I remember a usable web experience not too long ago on 28.8K/33.6K dialup (it was quite a while before said ILEC got a 56K-capable modem bank). DSL started out here at 384k/128k. On the positive side, we have a very low oversubscription ratio, so I actually get the full bandwidth the majority of the time, even video streaming. I also know all the network engineers there, too, and that also has its advantages.
(Yes, I am aware that rural living is a choice, and there are things worth a great deal more than bandwidth, that it's a tradeoff, etc.)
So it's not just '3rd-world' countries with expensive bandwidth.
Look at the Steam cache project, the generic downloader can also cache Windows Updates and most gaming services. I imagine Windows Updates would eat a lot of traffic. https://github.com/steamcache From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Mike Hammett Sent: Monday, May 28, 2018 8:23 AM To: 'NANOG list' <nanog@nanog.org> Subject: Impacts of Encryption Everywhere (any solution?) Has anyone outside of tech media, Silicon Valley or academia (all places wildly out of touch with the real world) put much thought into the impacts of encryption everywhere? So often we hear about how we need the best modern encryption on all forms of communication because of whatever scary thing is trendy this week (Russia, NSA, Google, whatever). HTTPS your marketing information and generic education pieces because of the boogeyman! However, I recently came across a thread where someone was exploring getting a one megabit connection into their village and sharing it among many. The crowd I referenced earlier also believes you can't Internet under 100 megabit/s per home. Apparently, the current best Internet the residents of the village can get is 40 kilobit/s. Zero oversubscription gets a better service to up to 25 homes. Likely that could be stretched to at least 50 or 100 homes and be better than what they currently have. Forget about streaming video, let's just focus on web browsing and messaging. However, this could be wildly improved with caching ala squid or something similar. The problem is that encrypted content is difficult to impossible for your average Joe to cache. The rewards for implementing caching are greatly mitigated and people like this must suffer a worse Internet experience because of some ideological high horse in a far-off land. Some things certainly do need to be encrypted, but encrypting everything means people with limited Internet access get worse performance OR mechanisms have to be out in place to break ALL encryption, this compromising security and privacy when it's really needed. To circle back to being somewhat on-topic, what mechanisms are available to maximize the amount of traffic someone in this situation could cache? The performance of third-world Internet depends on you. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<http://www.ics-il.com> Midwest-IX http://www.midwest-ix.com<http://www.midwest-ix.com>
In a message written on Mon, May 28, 2018 at 09:23:09AM -0500, Mike Hammett wrote:
However, this could be wildly improved with caching ala squid or something similar. The problem is that encrypted content is difficult to impossible for your average Joe to cache. The rewards for implementing caching are greatly mitigated and people like this must suffer a worse Internet experience because of some ideological high horse in a far-off land.
Some things certainly do need to be encrypted, but encrypting everything means people with limited Internet access get worse performance OR mechanisms have to be out in place to break ALL encryption, this compromising security and privacy when it's really needed.
I'm going to take this question head on, as opposed to the many tangents in this thread. The Internet lived in the world you described, and a lot of people learned a lot of things along the way. Perhaps the most important lessons: - Users cannot be trusted to check if there is a "secure" indicator before sending sensitive information. - Users cannot tell the difference between two "secure" sites, one of which is a phishing site that just happens to have a certificate. - There is no algorithmic way to determine if mixed mode content is "safe". - Web site operators seem incapable of maintaining white lists of safe mixed mode content. - Mixed mode content is not safe due to browser bugs. - Once users have been trained that it's ok to send content via some insecure channels, it's nearly impossible to untrain them of it later. Basically, while you presented the "pro" side of unencrypted content (being able to cache), you didn't present any of the negative side. I have to wonder if the villagers were given a choice of faster internet, where 5% of them had their bank account cleaned out, and 5% had their identity stolen, or slower, secure internet which they would choose? Want a technological solution? It exists! Signed content. I've always been baffled why there isn't a way to serve up HTTP signed (but not encrypted) content. I'd imagine the way it would work is: 1) Initial connection had to be HTTPS encrypted to create a full encrypted channel. 2) Additional assets could then be downloaded as HTTPS, or as HTTP + Signature. Signature must be from the same certificate as the HTTPS data. The http+signature data could then be cashed just fine, and stored in the clear. The web site could determine what to serve up that way to maintain security. All POST commands would have to be HTTPS (data from client to server), and of course sensitive information would be returned HTTPS only. Why doesn't that exist? -- Leo Bicknell - bicknell@ufp.org PGP keys at http://www.ufp.org/~bicknell/
On May 29, 2018, at 9:44 AM, Leo Bicknell <bicknell@ufp.org> wrote:
Basically, while you presented the "pro" side of unencrypted content (being able to cache), you didn't present any of the negative side. I have to wonder if the villagers were given a choice of faster internet, where 5% of them had their bank account cleaned out, and 5% had their identity stolen, or slower, secure internet which they would choose?
If you’re in $TinyVillage in $PoorAfricanCountry, do you even have a bank account or an online identity that could be stolen? Just my $0.02 on this increasingly off-topic thread. ---- Andy Ringsmuth andy@newslink.com News Link – Manager Technology, Travel & Facilities 2201 Winthrop Rd., Lincoln, NE 68502-4158 (402) 475-6397 (402) 304-0083 cellular
On 29/May/18 17:09, Andy Ringsmuth wrote:
If you’re in $TinyVillage in $PoorAfricanCountry, do you even have a bank account or an online identity that could be stolen?
Bank accounts are so 2018... https://en.wikipedia.org/wiki/M-Pesa Where've you been, man :-)... Mark.
northerners who have never traveled pontificating about africa might, or might not, be interested in https://afrinic.net/blog/333-revealing-latency-clusters-in-africa randy
Based on my experience a couple of years ago while in West Africa: If you look at the BGP adjacencies and bidirectional traceroutes for ISPs in Sierra Leone or Liberia; Freetown and Monrovia are both are logically suburbs of London. Just with much higher transport latencies via the submarine fiber link and then transport from UK cable landing station to the IX points in London. The situation is a bit different in Accra, Ghana which is a much larger and more economically developed market, and has IXes and ISPs that peer with each other domestically. On Tue, May 29, 2018 at 8:23 AM, Randy Bush <randy@psg.com> wrote:
northerners who have never traveled pontificating about africa might, or might not, be interested in
https://afrinic.net/blog/333-revealing-latency-clusters-in-africa
randy
On 29/May/18 18:03, Eric Kuhnke wrote:
Based on my experience a couple of years ago while in West Africa:
If you look at the BGP adjacencies and bidirectional traceroutes for ISPs in Sierra Leone or Liberia; Freetown and Monrovia are both are logically suburbs of London. Just with much higher transport latencies via the submarine fiber link and then transport from UK cable landing station to the IX points in London.
The situation is a bit different in Accra, Ghana which is a much larger and more economically developed market, and has IXes and ISPs that peer with each other domestically.
West Africa has generally lagged a little behind compared to Eastern and Southern Africa, with regard to closing connectivity gaps within the local and regional space. The good news is that places such as Ghana and Nigeria have made excellent strides in fixing this, as you point out. The work being done by AfPIF (part of ISOC), AFRINIC and a bunch of country- and region-level NOG's has gone a long a way in promoting local and regional connectivity through traditional and other means, and we have seen the fruits of that labour. Mark.
The http+signature data could then be cashed just fine, and stored in the clear. The web site could determine what to serve up that way to maintain security. All POST commands would have to be HTTPS (data from client to server), and of course sensitive information would be returned HTTPS only.
Makes a lot of sense, but… Wouldn’t you also have to require that all GET commands (or at lest GET commands for strings containing a ? character) be sent via HTTPS? In many cases, there’s little difference between the data disclosure of a POST form vs. the disclosure achieved with GET URL?attribute=value&… Indeed, there are multiple libraries out there which allow one to treat the variables from POST data and the variables from GET “query strings” as virtually identical. I suspect that in most cases, the only reason said libraries distinguish is to maintain namespace separation in case of collisions (since query strings can also be applied to POST requests).
Why doesn't that exist?
Because developers are lazy? Owen
On 05/28/2018 10:23 AM, Mike Hammett wrote:
Has anyone outside of tech media, Silicon Valley or academia (all places wildly out of touch with the real world) put much thought into the impacts of encryption everywhere? See "Effects of Pervasive Encryption on Operators." https://datatracker.ietf.org/doc/draft-mm-wg-effect-encrypt/?include_text=1
TLS1.3 uses ephemeral keys, so even if you own both endpoints and everything in the middle, you can't decrypt a flow without some yet-to-be-developed technology. QUIC encrypts everything, and of course, HTTPS.
So often we hear about how we need the best modern encryption on all forms of communication because of whatever scary thing is trendy this week (Russia, NSA, Google, whatever). HTTPS your marketing information and generic education pieces because of the boogeyman!
However, I recently came across a thread where someone was exploring getting a one megabit connection into their village and sharing it among many. The crowd I referenced earlier also believes you can't Internet under 100 megabit/s per home.
Yeah. Too many people forget that most of the Internet is mobile, and mobile != LTE. People also assume packet loss < 0.1%, latency <100ms, and power reliability >99%.
However, this could be wildly improved with caching ala squid or something similar. The problem is that encrypted content is difficult to impossible for your average Joe to cache. The rewards for implementing caching are greatly mitigated and people like this must suffer a worse Internet experience because of some ideological high horse in a far-off land.
Some things certainly do need to be encrypted, but encrypting everything means people with limited Internet access get worse performance OR mechanisms have to be out in place to break ALL encryption, this compromising security and privacy when it's really needed.
To circle back to being somewhat on-topic, what mechanisms are available to maximize the amount of traffic someone in this situation could cache? The performance of third-world Internet depends on you.
A proxy is all I've thought of. But it means everything is dependent on the proxy, and it's even in-path for things that really should be encrypted, like email and messaging. I can't imagine why the weather should be encrypted, when everyone in a location wants to know the forecast. Lee
But privacy! *sigh* People may just have to know how to turn the proxy on and off. It's a requirement we wouldn't dare consider in the US, but if you're in the middle of nowhere and you can get megabit or higher speeds (instead of dialup) if you learn how to turn a proxy on and off... you'll learn quickly. Sadly, it's just falling on deaf ears. Silicon Valley will continue to think they know better than everyone else and people outside of that bubble will continue to be disadvantaged. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Lee Howard" <lee.howard@retevia.net> To: nanog@nanog.org Sent: Tuesday, May 29, 2018 9:55:18 AM Subject: Re: Impacts of Encryption Everywhere (any solution?) On 05/28/2018 10:23 AM, Mike Hammett wrote:
Has anyone outside of tech media, Silicon Valley or academia (all places wildly out of touch with the real world) put much thought into the impacts of encryption everywhere? See "Effects of Pervasive Encryption on Operators." https://datatracker.ietf.org/doc/draft-mm-wg-effect-encrypt/?include_text=1
TLS1.3 uses ephemeral keys, so even if you own both endpoints and everything in the middle, you can't decrypt a flow without some yet-to-be-developed technology. QUIC encrypts everything, and of course, HTTPS.
So often we hear about how we need the best modern encryption on all forms of communication because of whatever scary thing is trendy this week (Russia, NSA, Google, whatever). HTTPS your marketing information and generic education pieces because of the boogeyman!
However, I recently came across a thread where someone was exploring getting a one megabit connection into their village and sharing it among many. The crowd I referenced earlier also believes you can't Internet under 100 megabit/s per home.
Yeah. Too many people forget that most of the Internet is mobile, and mobile != LTE. People also assume packet loss < 0.1%, latency <100ms, and power reliability >99%.
However, this could be wildly improved with caching ala squid or something similar. The problem is that encrypted content is difficult to impossible for your average Joe to cache. The rewards for implementing caching are greatly mitigated and people like this must suffer a worse Internet experience because of some ideological high horse in a far-off land.
Some things certainly do need to be encrypted, but encrypting everything means people with limited Internet access get worse performance OR mechanisms have to be out in place to break ALL encryption, this compromising security and privacy when it's really needed.
To circle back to being somewhat on-topic, what mechanisms are available to maximize the amount of traffic someone in this situation could cache? The performance of third-world Internet depends on you.
A proxy is all I've thought of. But it means everything is dependent on the proxy, and it's even in-path for things that really should be encrypted, like email and messaging. I can't imagine why the weather should be encrypted, when everyone in a location wants to know the forecast. Lee
On 06/16/2018 10:13 PM, Mike Hammett wrote:
Sadly, it's just falling on deaf ears. Silicon Valley will continue to think they know better than everyone else and people outside of that bubble will continue to be disadvantaged.
What, again ? Encryption is what is best for the most people. The few that will not use it can disable it. No issue then.
On Sat, Jun 16, 2018 at 4:13 PM, Mike Hammett <nanog@ics-il.net> wrote:
Sadly, it's just falling on deaf ears. Silicon Valley will continue to think they know better than everyone else and people outside of that bubble will continue to be disadvantaged.
Hi Mike, When the U.S. Government wants to encrypt classified information for transmission over an unclassified channel (such as the Internet) one of the interesting things the encryptor does is send data at a constant rate. If there isn't enough data to fill the channel, the encryptor pads its transmission with random bytes. If there's more data than the constant rate, it's queued and sent at a constant rate, even if the channel could handle more. Even over the internet where variable rate transmissions are the norm. This increases the _depth of defense_ against an adversary. Not only does the adversary have to figure out what you're saying, he has to figure out when and whether you're speaking at all. Depth of Defense. Remember that phrase; you'll hear it over and over again when security experts speak. Encrypting everything (not just information you consider private) also increases the depth of your defense against an adversary attempting to capture your secrets. An adversary must not only break or subvert your encryption, he must also figure out which if any of your communications are sensitive and which are banal. Depth of Defense. One of the linchpin concepts in effective security. Regards, Bill Herrin -- William Herrin ................ herrin@dirtside.com bill@herrin.us Dirtside Systems ......... Web: <http://www.dirtside.com/>
participants (24)
-
1 651-307-9043 -
Andrey Khomyakov -
Andy Ringsmuth -
Ben Cannon -
Eric Kuhnke -
Filip Hruska -
Grant Taylor -
Keith Medcalf -
Lamar Owen -
Lee Howard -
Leo Bicknell -
Mark Tinka -
Matt Erculiani -
Matt Hoppes -
Matthew Petach -
Mike Hammett -
ML -
nanog＠jack.fr.eu.org -
Owen DeLong -
Randy Bush -
Rich Kulawiec -
Rubens Kuhl -
Steve Mikulasik -
William Herrin