Could an IP engineer from AWS (16509/14618) and one from NTT (2914) kindly contact me off-list? AS18888 is having some major reachability issues to you via 2914. Several of our applications and users are reporting problems trying to reach various aws hosted services such as netflix and twilio. I'm seeing almost 50% packet loss when transiting to you via 2914. Forcing traffic onto 3356 clears the issue right up. I've had to effectively shift all my ingress traffic off 2914 and de-pref aws as-path to force egress to other transit. We're a customer of 2914, but not AWS. I've got a ticket open with 2914, and they've reached out to AWS, but it's been two days now and we haven't been getting any traction on this. thanks! -chris
Amazon hasn't reached out to us either... If you have other providers, use a combination of local-preference and the customer communitiy strings with ntt to prepend around the circuit(s) in nyc with the issue. Just check your routing table, we found many going through ntt to amazon and took awhile to get everything working as desired. Bryan Socha Network Engineer DigitalOcean On Thu, Jun 12, 2014 at 12:55 PM, Christopher Rogers <phiber@phiber.org> wrote:
Could an IP engineer from AWS (16509/14618) and one from NTT (2914) kindly contact me off-list? AS18888 is having some major reachability issues to you via 2914. Several of our applications and users are reporting problems trying to reach various aws hosted services such as netflix and twilio. I'm seeing almost 50% packet loss when transiting to you via 2914. Forcing traffic onto 3356 clears the issue right up. I've had to effectively shift all my ingress traffic off 2914 and de-pref aws as-path to force egress to other transit.
We're a customer of 2914, but not AWS. I've got a ticket open with 2914, and they've reached out to AWS, but it's been two days now and we haven't been getting any traction on this.
thanks!
-chris
Amazon peers at many key exchanges, with dozens of hosting shops (where customers might share mutual infrastructure) like yours: https://www.peeringdb.com/view.php?asn=16509 Rather than play the blame game with third-party transit providers, why not hit them up for some sessions? Drive Slow, Paul Wall On Fri, Jun 13, 2014 at 5:50 AM, Bryan Socha <bryan@digitalocean.com> wrote:
Amazon hasn't reached out to us either...
If you have other providers, use a combination of local-preference and the customer communitiy strings with ntt to prepend around the circuit(s) in nyc with the issue. Just check your routing table, we found many going through ntt to amazon and took awhile to get everything working as desired.
Bryan Socha Network Engineer DigitalOcean
On Thu, Jun 12, 2014 at 12:55 PM, Christopher Rogers <phiber@phiber.org> wrote:
Could an IP engineer from AWS (16509/14618) and one from NTT (2914) kindly contact me off-list? AS18888 is having some major reachability issues to you via 2914. Several of our applications and users are reporting problems trying to reach various aws hosted services such as netflix and twilio. I'm seeing almost 50% packet loss when transiting to you via 2914. Forcing traffic onto 3356 clears the issue right up. I've had to effectively shift all my ingress traffic off 2914 and de-pref aws as-path to force egress to other transit.
We're a customer of 2914, but not AWS. I've got a ticket open with 2914, and they've reached out to AWS, but it's been two days now and we haven't been getting any traction on this.
thanks!
-chris
I don't think anyone is blaming anyone, just trying to pass on information where we see a problem. We routed around it no problem. Bryan Socha On Fri, Jun 13, 2014 at 7:44 AM, Paul WALL <pauldotwall@gmail.com> wrote:
Amazon peers at many key exchanges, with dozens of hosting shops (where customers might share mutual infrastructure) like yours:
https://www.peeringdb.com/view.php?asn=16509
Rather than play the blame game with third-party transit providers, why not hit them up for some sessions?
Drive Slow, Paul Wall
Amazon hasn't reached out to us either...
If you have other providers, use a combination of local-preference and
customer communitiy strings with ntt to prepend around the circuit(s) in nyc with the issue. Just check your routing table, we found many going through ntt to amazon and took awhile to get everything working as desired.
Bryan Socha Network Engineer DigitalOcean
On Thu, Jun 12, 2014 at 12:55 PM, Christopher Rogers <phiber@phiber.org> wrote:
Could an IP engineer from AWS (16509/14618) and one from NTT (2914) kindly contact me off-list? AS18888 is having some major reachability issues to you via 2914. Several of our applications and users are reporting
On Fri, Jun 13, 2014 at 5:50 AM, Bryan Socha <bryan@digitalocean.com> wrote: the problems
trying to reach various aws hosted services such as netflix and twilio. I'm seeing almost 50% packet loss when transiting to you via 2914. Forcing traffic onto 3356 clears the issue right up. I've had to effectively shift all my ingress traffic off 2914 and de-pref aws as-path to force egress to other transit.
We're a customer of 2914, but not AWS. I've got a ticket open with 2914, and they've reached out to AWS, but it's been two days now and we haven't been getting any traction on this.
thanks!
-chris
On Fri, Jun 13, 2014 at 11:44:51AM +0000, Paul WALL wrote:
Amazon peers at many key exchanges, with dozens of hosting shops (where customers might share mutual infrastructure) like yours:
https://www.peeringdb.com/view.php?asn=16509
Rather than play the blame game with third-party transit providers, why not hit them up for some sessions?
That'll only get you peering connectivity into the local region. To get fully-peered with AWS, and be able to avoid third-party transit providers entirely, you're going to have to be in a *lot* of places. Not saying that AWS is a bad peer (from experience, I know they're fine to deal with) but it isn't as cut-and-dried as saying "don't blame transit providers, just peer!". - Matt -- A few minutes ago I attempted to give a flying fsck, but the best I could do was to watch it skitter across the floor. -- Anthony de Boer, ASR
On 6/13/14, 2:28 PM, Matt Palmer wrote:
On Fri, Jun 13, 2014 at 11:44:51AM +0000, Paul WALL wrote:
Amazon peers at many key exchanges, with dozens of hosting shops (where customers might share mutual infrastructure) like yours:
https://www.peeringdb.com/view.php?asn=16509
Rather than play the blame game with third-party transit providers, why not hit them up for some sessions?
That'll only get you peering connectivity into the local region. To get fully-peered with AWS, and be able to avoid third-party transit providers entirely, you're going to have to be in a *lot* of places.
Not saying that AWS is a bad peer (from experience, I know they're fine to deal with) but it isn't as cut-and-dried as saying "don't blame transit providers, just peer!".
just peer in IAD, SEA and SJC, and AMS, and SIN. problem solved, yup.
- Matt
participants (5)
-
Bryan Socha -
Christopher Rogers -
joel jaeggli -
Matt Palmer -
Paul WALL