NTT high packet loss from US and BR to AU?
Anyone else notice this? Or is this an AWS issue in APAC that hasn't been reported yet? AU-NY(aws) 18. xe-1.level3.lsanca03.us.bb.gin.n 72.0% BR(aws)-AU(aws) 11. ae-9.r20.snjsca04.us.bb.gin.ntt.net 71.4% NJ/NYC to AU(aws) 9. ae-9.r20.asbnva02.us.bb.gin.ntt.net 45.9% 772 10.1 16.4 9.2 94.4 13.3 10. ae-2.r21.lsanca03.us.bb.gin.ntt.net 40.5% 772 69.6 72.7 69.3 149.2 9.0
Howdy all, I've been lurking for a long time, first time writing in. Please excuse my inexperience. Javier, can you provide full traces, and source/destination addresses? /Charles On Wed, Oct 22, 2014 at 11:18 PM, Javier J <javier@advancedmachines.us> wrote:
Anyone else notice this?
Or is this an AWS issue in APAC that hasn't been reported yet?
AU-NY(aws) 18. xe-1.level3.lsanca03.us.bb.gin.n 72.0%
BR(aws)-AU(aws) 11. ae-9.r20.snjsca04.us.bb.gin.ntt.net 71.4%
NJ/NYC to AU(aws) 9. ae-9.r20.asbnva02.us.bb.gin.ntt.net 45.9% 772 10.1 16.4 9.2 94.4 13.3 10. ae-2.r21.lsanca03.us.bb.gin.ntt.net 40.5% 772 69.6 72.7 69.3 149.2 9.0
Does it actually persist to your destination? Loss in transit paths is simply ICMP de-prioritization unless it's losing packets all the way to the last hop. On 10/23/2014 午後 01:18, Javier J wrote:
Anyone else notice this?
Or is this an AWS issue in APAC that hasn't been reported yet?
AU-NY(aws) 18. xe-1.level3.lsanca03.us.bb.gin.n 72.0%
BR(aws)-AU(aws) 11. ae-9.r20.snjsca04.us.bb.gin.ntt.net 71.4%
NJ/NYC to AU(aws) 9. ae-9.r20.asbnva02.us.bb.gin.ntt.net 45.9% 772 10.1 16.4 9.2 94.4 13.3 10. ae-2.r21.lsanca03.us.bb.gin.ntt.net 40.5% 772 69.6 72.7 69.3 149.2 9.0
Do you see any other indications of performance problems? jms On Thu, 23 Oct 2014, Javier J wrote:
Anyone else notice this?
Or is this an AWS issue in APAC that hasn't been reported yet?
AU-NY(aws) 18. xe-1.level3.lsanca03.us.bb.gin.n 72.0%
BR(aws)-AU(aws) 11. ae-9.r20.snjsca04.us.bb.gin.ntt.net 71.4%
NJ/NYC to AU(aws) 9. ae-9.r20.asbnva02.us.bb.gin.ntt.net 45.9% 772 10.1 16.4 9.2 94.4 13.3 10. ae-2.r21.lsanca03.us.bb.gin.ntt.net 40.5% 772 69.6 72.7 69.3 149.2 9.0
So we have a nagios box in the environment in Sydney and everything is 100% ok. new relic kept loosing connectivity to 30 plus servers on and off. Guys from California can ssh in, some cant. AWS reports everything operating as normal. Guys from other parts of the world can and can't load web pages. All servers show low usage (if you can ssh to them) It seems to be getting better now but still not right. This is from a box in AWS(sys) to level 3 dns server. --- 4.2.2.2 ping statistics --- 70 packets transmitted, 68 received, 2% packet loss, time 69744ms rtt min/avg/max/mdev = 143.646/151.662/154.732/2.943 ms [root@Webapp javier]# Before it was 70% packet lost to that host. There is a mtr traceroute in a previous email. Look for AU-US On Thu, Oct 23, 2014 at 12:41 AM, Justin M. Streiner < streiner@cluebyfour.org> wrote:
Do you see any other indications of performance problems?
jms
On Thu, 23 Oct 2014, Javier J wrote:
Anyone else notice this?
Or is this an AWS issue in APAC that hasn't been reported yet?
AU-NY(aws) 18. xe-1.level3.lsanca03.us.bb.gin.n 72.0%
BR(aws)-AU(aws) 11. ae-9.r20.snjsca04.us.bb.gin.ntt.net 71.4%
NJ/NYC to AU(aws) 9. ae-9.r20.asbnva02.us.bb.gin.ntt.net 45.9% 772 10.1 16.4 9.2 94.4 13.3 10. ae-2.r21.lsanca03.us.bb.gin.ntt.net 40.5% 772 69.6 72.7 69.3 149.2 9.0
Yup seeing the same. Following examples all show same loss pattern between ~ 3:30 and ~ 4:30 UTC: syd ntt - nyc ntt syd ntt - mia ntt syd ntt - cdg ntt (paris) syd ntt - ams ntt One example: http://i.imgur.com/TmCkd1B.png?1 Cheers, Andree .-- My secret spy satellite informs me that at 2014-10-22 9:54 PM Javier J wrote:
So we have a nagios box in the environment in Sydney and everything is 100% ok.
new relic kept loosing connectivity to 30 plus servers on and off.
Guys from California can ssh in, some cant.
AWS reports everything operating as normal.
Guys from other parts of the world can and can't load web pages.
All servers show low usage (if you can ssh to them)
It seems to be getting better now but still not right.
This is from a box in AWS(sys) to level 3 dns server.
--- 4.2.2.2 ping statistics --- 70 packets transmitted, 68 received, 2% packet loss, time 69744ms rtt min/avg/max/mdev = 143.646/151.662/154.732/2.943 ms [root@Webapp javier]#
Before it was 70% packet lost to that host. There is a mtr traceroute in a previous email. Look for AU-US
On Thu, Oct 23, 2014 at 12:41 AM, Justin M. Streiner < streiner@cluebyfour.org> wrote:
Do you see any other indications of performance problems?
jms
On Thu, 23 Oct 2014, Javier J wrote:
Anyone else notice this?
Or is this an AWS issue in APAC that hasn't been reported yet?
AU-NY(aws) 18. xe-1.level3.lsanca03.us.bb.gin.n 72.0%
BR(aws)-AU(aws) 11. ae-9.r20.snjsca04.us.bb.gin.ntt.net 71.4%
NJ/NYC to AU(aws) 9. ae-9.r20.asbnva02.us.bb.gin.ntt.net 45.9% 772 10.1 16.4 9.2 94.4 13.3 10. ae-2.r21.lsanca03.us.bb.gin.ntt.net 40.5% 772 69.6 72.7 69.3 149.2 9.0
Thank you Andree, This confirms it wasn't just us. I am curious if anyone knows what the issue was. I can't find anything on NTT website. On Thu, Oct 23, 2014 at 1:25 AM, Andree Toonk <andree+nanog@toonk.nl> wrote:
Yup seeing the same. Following examples all show same loss pattern between ~ 3:30 and ~ 4:30 UTC:
syd ntt - nyc ntt syd ntt - mia ntt syd ntt - cdg ntt (paris) syd ntt - ams ntt
One example: http://i.imgur.com/TmCkd1B.png?1
Cheers, Andree
.-- My secret spy satellite informs me that at 2014-10-22 9:54 PM Javier J wrote:
So we have a nagios box in the environment in Sydney and everything is 100% ok.
new relic kept loosing connectivity to 30 plus servers on and off.
Guys from California can ssh in, some cant.
AWS reports everything operating as normal.
Guys from other parts of the world can and can't load web pages.
All servers show low usage (if you can ssh to them)
It seems to be getting better now but still not right.
This is from a box in AWS(sys) to level 3 dns server.
--- 4.2.2.2 ping statistics --- 70 packets transmitted, 68 received, 2% packet loss, time 69744ms rtt min/avg/max/mdev = 143.646/151.662/154.732/2.943 ms [root@Webapp javier]#
Before it was 70% packet lost to that host. There is a mtr traceroute in a previous email. Look for AU-US
On Thu, Oct 23, 2014 at 12:41 AM, Justin M. Streiner < streiner@cluebyfour.org> wrote:
Do you see any other indications of performance problems?
jms
On Thu, 23 Oct 2014, Javier J wrote:
Anyone else notice this?
Or is this an AWS issue in APAC that hasn't been reported yet?
AU-NY(aws) 18. xe-1.level3.lsanca03.us.bb.gin.n 72.0%
BR(aws)-AU(aws) 11. ae-9.r20.snjsca04.us.bb.gin.ntt.net 71.4%
NJ/NYC to AU(aws) 9. ae-9.r20.asbnva02.us.bb.gin.ntt.net 45.9% 772 10.1 16.4 9.2 94.4 13.3 10. ae-2.r21.lsanca03.us.bb.gin.ntt.net 40.5% 772 69.6 72.7 69.3 149.2 9.0
participants (5)
-
Andree Toonk -
Charles van Niman -
Javier J -
Justin M. Streiner -
Paul S.