Hi Guys We are an ISP in the Caribbean, and are faced with extremely high Bandwidth costs, compared to the US, we currently use Peer App for Caching however with most services now moving to HTTPS the cache is proving to be less and less effective. We are currently looking at any way we can save on Bandwidth or to be more Efficient with the Bandwidth we currently have. We do have a Layer 2 Circuit between the Island and Miami, I am seeing there are WAN Accelerators where they would put a Server on either end and sort of Compress and decompress the Traffic before it goes over the Layer 2, I have never used this before, has any one here used anything like this, what results would I be able to expect for ISP Traffic? If not any ideas on Bandwidth Savings, or being more Efficient with want we currently. Many thanks for any Help Keenan
The first step would be profiling your traffic sources. I would imagine you probably have a bunch of YouTube, Netflix et al. content, that those content providers will send you a cache box for, subject to minimum traffic requirements. Regards, Marty Strong -------------------------------------- Cloudflare - AS13335 Network Engineer marty@cloudflare.com +44 7584 906 055 smartflare (Skype) https://www.peeringdb.com/asn/13335
On 11 Jan 2017, at 04:08, Keenan Singh <keenansingh@airlinktt.net> wrote:
Hi Guys
We are an ISP in the Caribbean, and are faced with extremely high Bandwidth costs, compared to the US, we currently use Peer App for Caching however with most services now moving to HTTPS the cache is proving to be less and less effective. We are currently looking at any way we can save on Bandwidth or to be more Efficient with the Bandwidth we currently have. We do have a Layer 2 Circuit between the Island and Miami, I am seeing there are WAN Accelerators where they would put a Server on either end and sort of Compress and decompress the Traffic before it goes over the Layer 2, I have never used this before, has any one here used anything like this, what results would I be able to expect for ISP Traffic?
If not any ideas on Bandwidth Savings, or being more Efficient with want we currently.
Many thanks for any Help
Keenan
I reached out to my vendor and got back the following. Also what services have you seen move to HTTPS that you're not able to cache? Hi Luke, Regarding HTTPS Streaming and Netflix... Netflix announced in the spring of 2015 that it would move to HTTPS delivery by April of 2016. At the time of that first announcement, some concluded Netflix might not be able to afford the capital investment required to enable HTTPS delivery. Given Netflix did not complete the HTTPS project by their first deadline, we believe they have been focused on other priorities such as their global expansion, So, given this history, it's not clear just when or if Netflix will make the move to majority HTTPS for delivery. Furthermore, Netflix is under considerable pressure from investors to improve subscriber growth, revenue growth and profitability. The HTTPS project does not support any of these goals. In fact, Netflix reported net income is marginal and a move to full HTTPS delivery would likely consume all profits for the year. Along with the rest of the industry, we recognize the need for Open Caching systems to support HTTPS streaming from upstream content providers. This is one of the reasons why we were a Founding Member, along with 16 other streaming companies, in the Streaming Video Alliance in the fall of 2014. The SVA now includes almost 50 member companies from across the streaming ecosystem and around the world. More importantly, the Open Caching Working Group has issued functional requirements, unanimously approved by SVA members, which include support for HTTPS streams. The SVA Board has invited Netflix to join the Alliance and, in doing so, endorse the Open Caching work underway. This would open up a path in the short run to ensure any open cache can continue to support Netflix content even if Netflix moves to HTTPS delivery. We expect to see Netflix become more active in the SVA soon given other major streaming providers, such as Hulu and Amazon, are joining now. In conclusion, the SVA has developed a solution for Open Cache support of HTTPS streaming and we expect all streaming providers, including Netflix, will align with the SVA's direction. http://www.streamingvideoalliance.org/ Let me know if you have any more questions. Regards, Luke Guillory Network Operations Manager Tel: 985.536.1212 Fax: 985.536.0300 Email: lguillory@reservetele.com Reserve Telecommunications 100 RTC Dr Reserve, LA 70084 _________________________________________________________________________________________________ Disclaimer: The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material which should not disseminate, distribute or be copied. Please notify Luke Guillory immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Luke Guillory therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. . -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Keenan Singh Sent: Tuesday, January 10, 2017 10:09 PM To: nanog@nanog.org Subject: Bandwidth Savings Hi Guys We are an ISP in the Caribbean, and are faced with extremely high Bandwidth costs, compared to the US, we currently use Peer App for Caching however with most services now moving to HTTPS the cache is proving to be less and less effective. We are currently looking at any way we can save on Bandwidth or to be more Efficient with the Bandwidth we currently have. We do have a Layer 2 Circuit between the Island and Miami, I am seeing there are WAN Accelerators where they would put a Server on either end and sort of Compress and decompress the Traffic before it goes over the Layer 2, I have never used this before, has any one here used anything like this, what results would I be able to expect for ISP Traffic? If not any ideas on Bandwidth Savings, or being more Efficient with want we currently. Many thanks for any Help Keenan
On Tue, 10 Jan 2017 23:08:45 -0500, Keenan Singh said:
do have a Layer 2 Circuit between the Island and Miami, I am seeing there are WAN Accelerators where they would put a Server on either end and sort of Compress and decompress the Traffic before it goes over the Layer 2, I have never used this before, has any one here used anything like this, what
Those will probably not help a lot with https: data, as a properly encrypted stream is very close to random bits and thus not very compressible. As others have noted, your best chances are getting content providers to give you a local cache of their most popular content.
The problem with the local cache[s] is the bandwidth cost of populating the cache and keeping it coherent can be greater than the bandwidth saved. From your description, I would expect this to be the case so a local cache will not help. Rule of thumb is if your downstream traffic is not at least 3gb/sec, you won't see a win from a cache. This problem can be mitigated if you can find other large bandwidth consumers on the island and partner to share a cache. Examples of potential partners would be your competitors, universities, government organisations, etc. The savings can be significant. If there is a local peering point on the islands, this would be the best place for shared caches. Sharing caches via an existing non-profit peering organization or having a non-profit, educational organization, or the government take the lead can lower the suspicion barrier and result in more sign-ups. On Wed, Jan 11, 2017 at 1:58 PM, <Valdis.Kletnieks@vt.edu> wrote:
On Tue, 10 Jan 2017 23:08:45 -0500, Keenan Singh said:
do have a Layer 2 Circuit between the Island and Miami, I am seeing there are WAN Accelerators where they would put a Server on either end and sort of Compress and decompress the Traffic before it goes over the Layer 2, I have never used this before, has any one here used anything like this, what
Those will probably not help a lot with https: data, as a properly encrypted stream is very close to random bits and thus not very compressible.
As others have noted, your best chances are getting content providers to give you a local cache of their most popular content.
-- Fletcher Kittredge GWI 207-602-1134 www.gwi.net
Netflix won’t even begin talks for their cache if you're not doing a minimum of 5Gbps. They also require massive uploads to the cache often, these are things are 200TB now if I recall and they send everything unlike the transparent who only grab what's already being consumed. Luke Guillory Network Operations Manager Tel: 985.536.1212 Fax: 985.536.0300 Email: lguillory@reservetele.com Reserve Telecommunications 100 RTC Dr Reserve, LA 70084 _________________________________________________________________________________________________ Disclaimer: The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material which should not disseminate, distribute or be copied. Please notify Luke Guillory immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Luke Guillory therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. . -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Valdis.Kletnieks@vt.edu Sent: Wednesday, January 11, 2017 12:59 PM To: Keenan Singh Cc: nanog@nanog.org Subject: Re: Bandwidth Savings On Tue, 10 Jan 2017 23:08:45 -0500, Keenan Singh said:
do have a Layer 2 Circuit between the Island and Miami, I am seeing there are WAN Accelerators where they would put a Server on either end and sort of Compress and decompress the Traffic before it goes over the Layer 2, I have never used this before, has any one here used anything like this, what
Those will probably not help a lot with https: data, as a properly encrypted stream is very close to random bits and thus not very compressible. As others have noted, your best chances are getting content providers to give you a local cache of their most popular content.
Keenan
On 11 Jan 2017, at 15:10, Luke Guillory <lguillory@reservetele.com> wrote:
Netflix won’t even begin talks for their cache if you're not doing a minimum of 5Gbps.
Outside of the US I believe it is less based on presentations I have seen in Africa.
They also require massive uploads to the cache often, these are things are 200TB now if I recall and they send everything unlike the transparent who only grab what's already being consumed.
Talk to their inter-connect team, the ones I know are very approachable. f
I don't know the the Caribbean Internet Exchanges market. Are any worth peering at versus buying additional L2 bandwidth to Miami? https://cw.ams-ix.net/ http://www.ocix.net/ocix/ Rick On Tue, Jan 10, 2017 at 8:08 PM, Keenan Singh <keenansingh@airlinktt.net> wrote:
Hi Guys
We are an ISP in the Caribbean, and are faced with extremely high Bandwidth costs, compared to the US, we currently use Peer App for Caching however with most services now moving to HTTPS the cache is proving to be less and less effective. We are currently looking at any way we can save on Bandwidth or to be more Efficient with the Bandwidth we currently have. We do have a Layer 2 Circuit between the Island and Miami, I am seeing there are WAN Accelerators where they would put a Server on either end and sort of Compress and decompress the Traffic before it goes over the Layer 2, I have never used this before, has any one here used anything like this, what results would I be able to expect for ISP Traffic?
If not any ideas on Bandwidth Savings, or being more Efficient with want we currently.
Many thanks for any Help
Keenan
The challenges are almost certainly economics related, at the lack of competition and high costs for layer 1/2 transport from his Caribbean island to Miami. Via whatever submarine cables exist that are controlled by larger ILEC type entities/telcos. Or satellite (whether geostationary transponder capacity or o3b). Depending on what island we're talking about, the $$$$$/month for a single 1GbE or 10GbE layer 2 transport service from $ISLAND to Miami will be very high compared to what a network operator in the US 48 states is accustomed to paying. On Wed, Jan 11, 2017 at 12:55 PM, Richard Hicks <richard.hicks@gmail.com> wrote:
I don't know the the Caribbean Internet Exchanges market. Are any worth peering at versus buying additional L2 bandwidth to Miami?
https://cw.ams-ix.net/ http://www.ocix.net/ocix/
Rick
On Tue, Jan 10, 2017 at 8:08 PM, Keenan Singh <keenansingh@airlinktt.net> wrote:
Hi Guys
We are an ISP in the Caribbean, and are faced with extremely high Bandwidth costs, compared to the US, we currently use Peer App for Caching however with most services now moving to HTTPS the cache is proving to be less and less effective. We are currently looking at any way we can save on Bandwidth or to be more Efficient with the Bandwidth we currently have. We do have a Layer 2 Circuit between the Island and Miami, I am seeing there are WAN Accelerators where they would put a Server on either end and sort of Compress and decompress the Traffic before it goes over the Layer 2, I have never used this before, has any one here used anything like this, what results would I be able to expect for ISP Traffic?
If not any ideas on Bandwidth Savings, or being more Efficient with want we currently.
Many thanks for any Help
Keenan
I believe the ISP is located in Trinidad & Tobago. There are five international submarine cables that land on the island: - SG-SCS - Americas-II - ECFS - Southern Caribbean Fiber - ECLink Of those, 1 go to the closest real interconnectivity hub of Miami, with the others requiring another pair onwards. ECLink lands in Curaçao, which could give access to AMS-IX Caribbean, which may help with connectivity to content providers, both Akamai and Goole are live and we are in the process of connecting (https://cw.ams-ix.net/connected_parties). Pricing however, is probably just as expensive on that cable than to Miami. It would be interesting to hear from the OP the rough pricing for connectivity to Miami, vs. elsewhere. Regards, Marty Strong -------------------------------------- Cloudflare - AS13335 Network Engineer marty@cloudflare.com +44 7584 906 055 smartflare (Skype) https://www.peeringdb.com/asn/13335
On 11 Jan 2017, at 21:23, Eric Kuhnke <eric.kuhnke@gmail.com> wrote:
The challenges are almost certainly economics related, at the lack of competition and high costs for layer 1/2 transport from his Caribbean island to Miami. Via whatever submarine cables exist that are controlled by larger ILEC type entities/telcos. Or satellite (whether geostationary transponder capacity or o3b).
Depending on what island we're talking about, the $$$$$/month for a single 1GbE or 10GbE layer 2 transport service from $ISLAND to Miami will be very high compared to what a network operator in the US 48 states is accustomed to paying.
On Wed, Jan 11, 2017 at 12:55 PM, Richard Hicks <richard.hicks@gmail.com> wrote:
I don't know the the Caribbean Internet Exchanges market. Are any worth peering at versus buying additional L2 bandwidth to Miami?
https://cw.ams-ix.net/ http://www.ocix.net/ocix/
Rick
On Tue, Jan 10, 2017 at 8:08 PM, Keenan Singh <keenansingh@airlinktt.net> wrote:
Hi Guys
We are an ISP in the Caribbean, and are faced with extremely high Bandwidth costs, compared to the US, we currently use Peer App for Caching however with most services now moving to HTTPS the cache is proving to be less and less effective. We are currently looking at any way we can save on Bandwidth or to be more Efficient with the Bandwidth we currently have. We do have a Layer 2 Circuit between the Island and Miami, I am seeing there are WAN Accelerators where they would put a Server on either end and sort of Compress and decompress the Traffic before it goes over the Layer 2, I have never used this before, has any one here used anything like this, what results would I be able to expect for ISP Traffic?
If not any ideas on Bandwidth Savings, or being more Efficient with want we currently.
Many thanks for any Help
Keenan
I remember there were a couple different owners down there, but Columbus swallowed them up, then C&W got Columbus then Liberty got C&W. Not knowing the international cable market, is this like AT&T, Comcast or Verizon (the largest US last mile operators) being your only option? ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Eric Kuhnke" <eric.kuhnke@gmail.com> To: "nanog@nanog.org list" <nanog@nanog.org> Sent: Wednesday, January 11, 2017 3:23:58 PM Subject: Re: Bandwidth Savings The challenges are almost certainly economics related, at the lack of competition and high costs for layer 1/2 transport from his Caribbean island to Miami. Via whatever submarine cables exist that are controlled by larger ILEC type entities/telcos. Or satellite (whether geostationary transponder capacity or o3b). Depending on what island we're talking about, the $$$$$/month for a single 1GbE or 10GbE layer 2 transport service from $ISLAND to Miami will be very high compared to what a network operator in the US 48 states is accustomed to paying. On Wed, Jan 11, 2017 at 12:55 PM, Richard Hicks <richard.hicks@gmail.com> wrote:
I don't know the the Caribbean Internet Exchanges market. Are any worth peering at versus buying additional L2 bandwidth to Miami?
https://cw.ams-ix.net/ http://www.ocix.net/ocix/
Rick
On Tue, Jan 10, 2017 at 8:08 PM, Keenan Singh <keenansingh@airlinktt.net> wrote:
Hi Guys
We are an ISP in the Caribbean, and are faced with extremely high Bandwidth costs, compared to the US, we currently use Peer App for Caching however with most services now moving to HTTPS the cache is proving to be less and less effective. We are currently looking at any way we can save on Bandwidth or to be more Efficient with the Bandwidth we currently have. We do have a Layer 2 Circuit between the Island and Miami, I am seeing there are WAN Accelerators where they would put a Server on either end and sort of Compress and decompress the Traffic before it goes over the Layer 2, I have never used this before, has any one here used anything like this, what results would I be able to expect for ISP Traffic?
If not any ideas on Bandwidth Savings, or being more Efficient with want we currently.
Many thanks for any Help
Keenan
At the end of the day it’s a cost/benefit analysis, reading the OP’s message it’s clear that the IX or transit isn’t really the problem, but rather how much traffic traverses to the point of transit/IX. I think until we see some numbers both in terms of traffic and in terms of price to say Miami, Willemstad, etc. we can’t easily determine a good solution. Regards, Marty Strong -------------------------------------- Cloudflare - AS13335 Network Engineer marty@cloudflare.com +44 7584 906 055 smartflare (Skype) https://www.peeringdb.com/asn/13335
On 12 Jan 2017, at 13:38, Mike Hammett <nanog@ics-il.net> wrote:
I remember there were a couple different owners down there, but Columbus swallowed them up, then C&W got Columbus then Liberty got C&W.
Not knowing the international cable market, is this like AT&T, Comcast or Verizon (the largest US last mile operators) being your only option?
----- Mike Hammett Intelligent Computing Solutions
Midwest Internet Exchange
The Brothers WISP
----- Original Message -----
From: "Eric Kuhnke" <eric.kuhnke@gmail.com> To: "nanog@nanog.org list" <nanog@nanog.org> Sent: Wednesday, January 11, 2017 3:23:58 PM Subject: Re: Bandwidth Savings
The challenges are almost certainly economics related, at the lack of competition and high costs for layer 1/2 transport from his Caribbean island to Miami. Via whatever submarine cables exist that are controlled by larger ILEC type entities/telcos. Or satellite (whether geostationary transponder capacity or o3b).
Depending on what island we're talking about, the $$$$$/month for a single 1GbE or 10GbE layer 2 transport service from $ISLAND to Miami will be very high compared to what a network operator in the US 48 states is accustomed to paying.
On Wed, Jan 11, 2017 at 12:55 PM, Richard Hicks <richard.hicks@gmail.com> wrote:
I don't know the the Caribbean Internet Exchanges market. Are any worth peering at versus buying additional L2 bandwidth to Miami?
https://cw.ams-ix.net/ http://www.ocix.net/ocix/
Rick
On Tue, Jan 10, 2017 at 8:08 PM, Keenan Singh <keenansingh@airlinktt.net> wrote:
Hi Guys
We are an ISP in the Caribbean, and are faced with extremely high Bandwidth costs, compared to the US, we currently use Peer App for Caching however with most services now moving to HTTPS the cache is proving to be less and less effective. We are currently looking at any way we can save on Bandwidth or to be more Efficient with the Bandwidth we currently have. We do have a Layer 2 Circuit between the Island and Miami, I am seeing there are WAN Accelerators where they would put a Server on either end and sort of Compress and decompress the Traffic before it goes over the Layer 2, I have never used this before, has any one here used anything like this, what results would I be able to expect for ISP Traffic?
If not any ideas on Bandwidth Savings, or being more Efficient with want we currently.
Many thanks for any Help
Keenan
Peering is great when you can get to the IX inexpensively. I assume that glass that goes underwater has a significant increase in cost and therefore the cost savings of peering would be minuscule in comparison to the cost of the rest of the connectivity. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Richard Hicks" <richard.hicks@gmail.com> To: "Keenan Singh" <keenansingh@airlinktt.net> Cc: nanog@nanog.org Sent: Wednesday, January 11, 2017 2:55:05 PM Subject: Re: Bandwidth Savings I don't know the the Caribbean Internet Exchanges market. Are any worth peering at versus buying additional L2 bandwidth to Miami? https://cw.ams-ix.net/ http://www.ocix.net/ocix/ Rick On Tue, Jan 10, 2017 at 8:08 PM, Keenan Singh <keenansingh@airlinktt.net> wrote:
Hi Guys
We are an ISP in the Caribbean, and are faced with extremely high Bandwidth costs, compared to the US, we currently use Peer App for Caching however with most services now moving to HTTPS the cache is proving to be less and less effective. We are currently looking at any way we can save on Bandwidth or to be more Efficient with the Bandwidth we currently have. We do have a Layer 2 Circuit between the Island and Miami, I am seeing there are WAN Accelerators where they would put a Server on either end and sort of Compress and decompress the Traffic before it goes over the Layer 2, I have never used this before, has any one here used anything like this, what results would I be able to expect for ISP Traffic?
If not any ideas on Bandwidth Savings, or being more Efficient with want we currently.
Many thanks for any Help
Keenan
Keenan Singh <keenansingh@airlinktt.net> writes:
Hi Guys
We are an ISP in the Caribbean, and are faced with extremely high Bandwidth costs, compared to the US, we currently use Peer App for Caching however with most services now moving to HTTPS the cache is proving to be less and less effective. We are currently looking at any way we can save on Bandwidth or to be more Efficient with the Bandwidth we currently have. We do have a Layer 2 Circuit between the Island and Miami, I am seeing there are WAN Accelerators where they would put a Server on either end and sort of Compress and decompress the Traffic before it goes over the Layer 2, I have never used this before, has any one here used anything like this, what results would I be able to expect for ISP Traffic?
If not any ideas on Bandwidth Savings, or being more Efficient with want we currently.
For Apple-originated data there's Caching Server, part of macOS Server, https://www.apple.com/macos/server/features/#caching-server You buy a Mac, and get macOS Server from the App Store. If you're using a typical single-outgoing-IP NAT, put the Mac behind the NAT, and make sure other hosts on the NAT can talk to it. If you're using public IP space, or you have NATs where you can't do that, you add DNS-SD TXT records to the search domain for your customers' machines (as configured with DHCP). Then your customers will use the Mac as a local caching source of Apple software updates and store content. You can have several and they'll automatically cluster and failover. There's documentation at https://help.apple.com/serverapp/mac/5.0/#/apd5E1AD52E-012B-4A41-8F21-8E9EDA...
On Tue, Jan 10, 2017 at 11:08:45PM -0500, Keenan Singh wrote:
We are currently looking at any way we can save on Bandwidth or to be more Efficient with the Bandwidth we currently have.
Measure what you're doing in as much detail as you can. Slice-and-dice it by source, destination, time-of-day, protocol, day-of-week, etc. and plot the results. In most cases, the problems will make themselves leap off the page. (We could all probably predict what they are right now, but real live measurements are still a good idea.) Once you've identified the problems, you'll be much closer to enumerating possible solutions (and you can avoid wasting time on the issues that aren't worth addressing). ---rsk
participants (11)
-
Eric Kuhnke -
Fearghas Mckay -
Fletcher Kittredge -
Geoffrey Keating -
Keenan Singh -
Luke Guillory -
Marty Strong -
Mike Hammett -
Rich Kulawiec -
Richard Hicks -
Valdis.Kletnieks＠vt.edu