Re: akamai abnormal spike
We noticed on the 12th and 13th there was a significant up tick in traffic served from our Akamai servers as well. At 05:37 PM 13/07/2016, eric c wrote:
Good afternoon,
Has anyone notice any abnormal spike in Akamai trafic in the last 24-48 hours compared to other days. I know it was black tuesday yesterday but traffic from last month didn't even come close to what we saw from Akamai.
We have some caching servers and even notice a spike to them as well.
Limelight even showed up on our network.
thanks eric
-- Clayton Zekelman Managed Network Systems Inc. (MNSi) 3363 Tecumseh Rd. E Windsor, Ontario N8W 1H4 tel. 519-985-8410 fax. 519-985-8409
We noticed that on the 12th-14th we had multiple subscribers on ~5Mbps subscription rates that were being sent ~50Mbps of data sourced from TCP port 80 (apparently HTTP) from Limelight Networks' servers. The data did appear to be user requested, still not sure why TCP didn't throttle the data rate appropriately. The 50Mbps was distributed across multiple LLNW servers. Makes me wonder if the customer was requesting one batch of data and multiple servers were responding. The issue cleared up on its own and I never was able to perform a full packet capture to investigate. I have not noticed the same behavior from Akamai servers. Clayton Zekelman wrote on 7/18/2016 8:26 AM:
We noticed on the 12th and 13th there was a significant up tick in traffic served from our Akamai servers as well.
At 05:37 PM 13/07/2016, eric c wrote:
Good afternoon,
Has anyone notice any abnormal spike in Akamai trafic in the last 24-48 hours compared to other days. I know it was black tuesday yesterday but traffic from last month didn't even come close to what we saw from Akamai.
We have some caching servers and even notice a spike to them as well.
Limelight even showed up on our network.
thanks eric
Several of my WISP colleagues have noticed this behavior (CDN sending way more traffic than the customer's pipe can handle) from (I believe) multiple CDNs. Not sure if it is intention on behalf of the CDN or an error, but it has been on-going for several months if not years. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest Internet Exchange http://www.midwest-ix.com ----- Original Message ----- From: "Blake Hudson" <blake@ispn.net> To: nanog@nanog.org Sent: Monday, July 18, 2016 8:49:21 AM Subject: Re: akamai abnormal spike We noticed that on the 12th-14th we had multiple subscribers on ~5Mbps subscription rates that were being sent ~50Mbps of data sourced from TCP port 80 (apparently HTTP) from Limelight Networks' servers. The data did appear to be user requested, still not sure why TCP didn't throttle the data rate appropriately. The 50Mbps was distributed across multiple LLNW servers. Makes me wonder if the customer was requesting one batch of data and multiple servers were responding. The issue cleared up on its own and I never was able to perform a full packet capture to investigate. I have not noticed the same behavior from Akamai servers. Clayton Zekelman wrote on 7/18/2016 8:26 AM:
We noticed on the 12th and 13th there was a significant up tick in traffic served from our Akamai servers as well.
At 05:37 PM 13/07/2016, eric c wrote:
Good afternoon,
Has anyone notice any abnormal spike in Akamai trafic in the last 24-48 hours compared to other days. I know it was black tuesday yesterday but traffic from last month didn't even come close to what we saw from Akamai.
We have some caching servers and even notice a spike to them as well.
Limelight even showed up on our network.
thanks eric
On 7/18/16 4:57 PM, Mike Hammett wrote:
Several of my WISP colleagues have noticed this behavior (CDN sending way more traffic than the customer's pipe can handle) from (I believe) multiple CDNs. Not sure if it is intention on behalf of the CDN or an error, but it has been on-going for several months if not years.
It's not a healthy tcp flow if the number of packets associated with the flow stays well in excess of the link capacity for a while... if you have recourse to to l4 header flags you might find that it was an ack flood or repeated retramission of the same PDUs. Either way someones state machine has a bug. joel
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest Internet Exchange http://www.midwest-ix.com
----- Original Message -----
From: "Blake Hudson" <blake@ispn.net> To: nanog@nanog.org Sent: Monday, July 18, 2016 8:49:21 AM Subject: Re: akamai abnormal spike
We noticed that on the 12th-14th we had multiple subscribers on ~5Mbps subscription rates that were being sent ~50Mbps of data sourced from TCP port 80 (apparently HTTP) from Limelight Networks' servers. The data did appear to be user requested, still not sure why TCP didn't throttle the data rate appropriately. The 50Mbps was distributed across multiple LLNW servers. Makes me wonder if the customer was requesting one batch of data and multiple servers were responding.
The issue cleared up on its own and I never was able to perform a full packet capture to investigate. I have not noticed the same behavior from Akamai servers.
Clayton Zekelman wrote on 7/18/2016 8:26 AM:
We noticed on the 12th and 13th there was a significant up tick in traffic served from our Akamai servers as well.
At 05:37 PM 13/07/2016, eric c wrote:
Good afternoon,
Has anyone notice any abnormal spike in Akamai trafic in the last 24-48 hours compared to other days. I know it was black tuesday yesterday but traffic from last month didn't even come close to what we saw from Akamai.
We have some caching servers and even notice a spike to them as well.
Limelight even showed up on our network.
thanks eric
participants (4)
-
Blake Hudson -
Clayton Zekelman -
joel jaeggli -
Mike Hammett