Anyone out there seeing issues with Belkin routers connecting? I have also noticed that Belkins website has 80 percent packet loss and their support number is busy. Luke Parrish | Network Operations Engineer I | Suddenlink Communications | 866.232.5455 ________________________________ The information transmitted is intended only for the person or entity to which it is addressed and may contain proprietary, confidential and/or legally privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from all computers. ________________________________
Seeing reports bounce around on the WISPA lists. Looks to be widespread. Reports on their twitter as well. I've had one customer with an issue related thus far. Nick Olsen Network Operations (855) FLSPEED x106 ---------------------------------------- From: "Parrish, Luke" <Luke.Parrish@Suddenlink.com> Sent: Tuesday, October 07, 2014 9:48 AM To: "nanog@nanog.org" <nanog@nanog.org> Subject: Belkin Router issues this morning? Anyone out there seeing issues with Belkin routers connecting? I have also noticed that Belkins website has 80 percent packet loss and their support number is busy. Luke Parrish | Network Operations Engineer I | Suddenlink Communications | 866.232.5455 ________________________________ The information transmitted is intended only for the person or entity to which it is addressed and may contain proprietary, confidential and/or legally privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from all computers. ________________________________
https://twitter.com/search?q=%23belkin Sounds like a bad firmware update most likely. Presumably the Belkin routers perform caching DNS for the LAN clients for if the LAN clients use alternate DNS servers (OpenDNS, Google, your ISPs, etc) there are no longer any issues for those devices, as reported by several random people on the Internet. ________________________________________ From: Nick Olsen [nick@flhsi.com] Sent: Tuesday, October 07, 2014 8:56 AM To: Parrish, Luke; nanog@nanog.org Subject: re: Belkin Router issues this morning? Seeing reports bounce around on the WISPA lists. Looks to be widespread. Reports on their twitter as well. I've had one customer with an issue related thus far. Nick Olsen Network Operations (855) FLSPEED x106 ---------------------------------------- From: "Parrish, Luke" <Luke.Parrish@Suddenlink.com> Sent: Tuesday, October 07, 2014 9:48 AM To: "nanog@nanog.org" <nanog@nanog.org> Subject: Belkin Router issues this morning? Anyone out there seeing issues with Belkin routers connecting? I have also noticed that Belkins website has 80 percent packet loss and their support number is busy. Luke Parrish | Network Operations Engineer I | Suddenlink Communications | 866.232.5455 ________________________________ The information transmitted is intended only for the person or entity to which it is addressed and may contain proprietary, confidential and/or legally privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from all computers. ________________________________
On Oct 7, 2014, at 8:34 AM, Justin Krejci <jkrejci@usinternet.com> wrote:
https://twitter.com/search?q=%23belkin
Sounds like a bad firmware update most likely. Presumably the Belkin routers perform caching DNS for the LAN clients for if the LAN clients use alternate DNS servers (OpenDNS, Google, your ISPs, etc) there are no longer any issues for those devices, as reported by several random people on the Internet.
Over on outages, someone mentioned that heartbeat.belkin.com was unreachable from some areas, and that was causing their routers to shut down. Cheers, Steve
Geesh. It was a tough day - lots of our customers with Belkin devices. Agree it has been fixable with customer changes but would have been better not to have to respond to this today (obviously). Regards, Jason Jason Livingood Comcast - Internet Services
On Oct 7, 2014, at 12:08, "Steve Atkins" <steve@blighty.com> wrote:
On Oct 7, 2014, at 8:34 AM, Justin Krejci <jkrejci@usinternet.com> wrote:
https://twitter.com/search?q=%23belkin
Sounds like a bad firmware update most likely. Presumably the Belkin routers perform caching DNS for the LAN clients for if the LAN clients use alternate DNS servers (OpenDNS, Google, your ISPs, etc) there are no longer any issues for those devices, as reported by several random people on the Internet.
Over on outages, someone mentioned that heartbeat.belkin.com was unreachable from some areas, and that was causing their routers to shut down.
Cheers, Steve
Looks like belkin¹s fix was to add different records. DNS response this morning. heartbeat.belkin.com. 4h14m31s IN A 67.20.176.130 DNS response this evening. dig heartbeat.belkin.com +short 54.163.115.57 23.20.47.97 54.227.172.225 54.161.217.33 54.163.87.225 54.87.220.73 54.163.74.132 54.83.179.150 54.196.247.165 54.163.72.1 54.87.180.46 54.242.182.61 174.129.92.187 On 10/7/14, 5:54 PM, "Livingood, Jason" <Jason_Livingood@cable.comcast.com> wrote:
Geesh. It was a tough day - lots of our customers with Belkin devices. Agree it has been fixable with customer changes but would have been better not to have to respond to this today (obviously).
Regards, Jason
Jason Livingood Comcast - Internet Services
On Oct 7, 2014, at 12:08, "Steve Atkins" <steve@blighty.com> wrote:
On Oct 7, 2014, at 8:34 AM, Justin Krejci <jkrejci@usinternet.com> wrote:
https://twitter.com/search?q=%23belkin
Sounds like a bad firmware update most likely. Presumably the Belkin routers perform caching DNS for the LAN clients for if the LAN clients use alternate DNS servers (OpenDNS, Google, your ISPs, etc) there are no longer any issues for those devices, as reported by several random people on the Internet.
Over on outages, someone mentioned that heartbeat.belkin.com was unreachable from some areas, and that was causing their routers to shut down.
Cheers, Steve
Sounds like it might have been a DNS issue of some sort. The end result was that the customer routers couldn't reach their heartbeat server, which made them think they weren't on the net. The routers would then be "helpful" and redirect all customer port 80 traffic to the router's configuration page to correct the "problem". It was a big mess that hopefully doesn't happen again. They've been making changes to their DNS this afternoon. Looks like they finally have it straightened out. John On Tue, Oct 7, 2014 at 10:05 AM, Steve Atkins <steve@blighty.com> wrote:
On Oct 7, 2014, at 8:34 AM, Justin Krejci <jkrejci@usinternet.com> wrote:
https://twitter.com/search?q=%23belkin
Sounds like a bad firmware update most likely. Presumably the Belkin routers perform caching DNS for the LAN clients for if the LAN clients use alternate DNS servers (OpenDNS, Google, your ISPs, etc) there are no longer any issues for those devices, as reported by several random people on the Internet.
Over on outages, someone mentioned that heartbeat.belkin.com was unreachable from some areas, and that was causing their routers to shut down.
Cheers, Steve
And this, my friends, is why friends don't let friends buy Belkin... Apparently, you still can't fix stupid, no matter how hard you try. -Mike On Tuesday, October 7, 2014, John Neiberger <jneiberger@gmail.com> wrote:
Sounds like it might have been a DNS issue of some sort. The end result was that the customer routers couldn't reach their heartbeat server, which made them think they weren't on the net. The routers would then be "helpful" and redirect all customer port 80 traffic to the router's configuration page to correct the "problem".
It was a big mess that hopefully doesn't happen again. They've been making changes to their DNS this afternoon. Looks like they finally have it straightened out.
John
On Tue, Oct 7, 2014 at 10:05 AM, Steve Atkins <steve@blighty.com <javascript:;>> wrote:
On Oct 7, 2014, at 8:34 AM, Justin Krejci <jkrejci@usinternet.com
<javascript:;>> wrote:
https://twitter.com/search?q=%23belkin
Sounds like a bad firmware update most likely. Presumably the Belkin routers perform caching DNS for the LAN clients for if the LAN clients use alternate DNS servers (OpenDNS, Google, your ISPs, etc) there are no longer any issues for those devices, as reported
by
several random people on the Internet.
Over on outages, someone mentioned that heartbeat.belkin.com was unreachable from some areas, and that was causing their routers to shut down.
Cheers, Steve
-- Mike Lyon 408-621-4826 mike.lyon@gmail.com http://www.linkedin.com/in/mlyon
While we weren't really impacted by this issue, my understanding is that the Belkin devices ping 'heartbeat.belkin.com' periodically. If their pings fail, they do DNS redirection to the device's configuration interface and alert the user of the error. It appears that this server went down or had some sort of network issue, causing the redirect to kick in and users to be unable to access the Internet. Any solution that either a) avoids the router's DNS redirection or b) cause its pings to succeed would restore service. It looks like Belkin has now re-implemented this service on top of a DNS-cluster of AWS instances, so I think users are probably back online now, as the IPs in the initial reports were not AWS and are no longer in the DNS. Workarounds include: a) Add the IP(s) of the heartbeat server as loopbacks on a server/router within your (service provider) network that will respond to the echos for your customers b) Add the hostname 'heartbeat.belkin.com' to the router's host file as 127.0.0.1 c) Modify the DNS configuration of client machines or the Belkin DHCP server K On 10/07/2014 03:27 PM, John Neiberger wrote:
Sounds like it might have been a DNS issue of some sort. The end result was that the customer routers couldn't reach their heartbeat server, which made them think they weren't on the net. The routers would then be "helpful" and redirect all customer port 80 traffic to the router's configuration page to correct the "problem".
It was a big mess that hopefully doesn't happen again. They've been making changes to their DNS this afternoon. Looks like they finally have it straightened out.
John
On Tue, Oct 7, 2014 at 10:05 AM, Steve Atkins <steve@blighty.com> wrote:
On Oct 7, 2014, at 8:34 AM, Justin Krejci <jkrejci@usinternet.com> wrote:
https://twitter.com/search?q=%23belkin
Sounds like a bad firmware update most likely. Presumably the Belkin routers perform caching DNS for the LAN clients for if the LAN clients use alternate DNS servers (OpenDNS, Google, your ISPs, etc) there are no longer any issues for those devices, as reported by several random people on the Internet.
Over on outages, someone mentioned that heartbeat.belkin.com was unreachable from some areas, and that was causing their routers to shut down.
Cheers, Steve
On Tue, 07 Oct 2014 16:27:16 -0600, John Neiberger said:
Sounds like it might have been a DNS issue of some sort. The end result was that the customer routers couldn't reach their heartbeat server, which made them think they weren't on the net.
Seems like a dubious idea to equate "can't reach heartbeat server" with "not on the net at all". At the very least, I'd add a second clause "I have no reachable default route, and I must scream".
participants (10)
-
John Neiberger -
Justin Krejci -
Keenan Tims -
Livingood, Jason -
Mike Lyon -
Nick Olsen -
Parrish, Luke -
Steve Atkins -
Steven Parsons -
Valdis.Kletnieks＠vt.edu