RE: Register.com DNS hosting issues
No ETA given to me, just the stock line of "We apologize.. blah blah... as soon as possible.. blah blah." Jeffrey Negro, Network Engineer Billtrust - Improving Your Billing, Improving Your Business www.billtrust.com <http://www.billtrust.com/> 609.235.1010 x137 jnegro@billtrust.com <mailto:jeichmann@billtrust.com> From: Dennis Burgess - LTI [mailto:dmburgess@linktechs.net] Sent: Friday, April 03, 2009 3:39 PM To: Jeffrey Negro Subject: Re: Register.com DNS hosting issues Thanks for the update :) Guess they did not get a ETA time? ----------------------------------------------------------- Dennis Burgess, CCNA, A+, Mikrotik Certified Trainer WISPA Board Member - wispa.org <http://www.wispa.org/> Link Technologies, Inc -- Mikrotik & WISP Support Services WISPA Vendor Member Office: 314-735-0270 Website: http://www.linktechs.net <http://www.linktechs.net/> LIVE On-Line Mikrotik Training <http://www.linktechs.net/onlinetraining.asp> The information transmitted (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is intended only for the person(s) or entity/entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited, If you received this in error, please contact the sender and delete the material from any computer. Jeffrey Negro wrote: For anyone trying to troubleshoot any strange resolution or page loading issues, Register.com is apparently having a massive DNS hosting outage that has been going on since 2 days ago, and is still continuing. I only found out because our monitoring was complaining about one single domain on and off. After speaking with register.com support, they admitted to the ongoing issue. If anyone has reported this already on the list, I apologize in advance for the repetition. Jeffrey
Looks like a routing issue to their DNS servers. traceroute to DNS020.C.REGISTER.COM (216.21.235.20), 64 hops max, 40 byte packets 1 c28.134.nauticom.net (209.195.134.28) 0.778 ms 0.894 ms 0.829 ms 2 10.11.11.9 (10.11.11.9) 1.010 ms 0.799 ms 0.829 ms 3 c246.134.nauticom.net (209.195.134.246) 1.271 ms 1.379 ms 1.350 ms 4 207.88.183.141.ptr.us.xo.net (207.88.183.141) 84.817 ms 7.764 ms 7.699 ms 5 65.106.3.185.ptr.us.xo.net (65.106.3.185) 7.606 ms 7.634 ms 7.504 ms 6 206.111.0.54.ptr.us.xo.net (206.111.0.54) 31.458 ms 28.509 ms 20.199 ms 7 Vlan424.icore1.MLN-Miami.as6453.net (66.110.9.13) 48.740 ms 36.118 ms 36.211 ms 8 ix-6-2.icore1.MLN-Miami.as6453.net (66.110.9.54) 36.328 ms 36.851 ms 36.334 ms 9 * * * 10 blackhole.prolexic.com (209.200.132.42) 36.127 ms 36.136 ms 36.298 ms 11 * Clinton Popovich Systems Engineer Consolidated Communications -----Original Message----- From: Jeffrey Negro [mailto:jnegro@billtrust.com] Sent: Friday, April 03, 2009 3:42 PM To: Dennis Burgess - LTI Cc: nanog@nanog.org Subject: RE: Register.com DNS hosting issues No ETA given to me, just the stock line of "We apologize.. blah blah... as soon as possible.. blah blah." Jeffrey Negro, Network Engineer Billtrust - Improving Your Billing, Improving Your Business www.billtrust.com <http://www.billtrust.com/> 609.235.1010 x137 jnegro@billtrust.com <mailto:jeichmann@billtrust.com> From: Dennis Burgess - LTI [mailto:dmburgess@linktechs.net] Sent: Friday, April 03, 2009 3:39 PM To: Jeffrey Negro Subject: Re: Register.com DNS hosting issues Thanks for the update :) Guess they did not get a ETA time? ----------------------------------------------------------- Dennis Burgess, CCNA, A+, Mikrotik Certified Trainer WISPA Board Member - wispa.org <http://www.wispa.org/> Link Technologies, Inc -- Mikrotik & WISP Support Services WISPA Vendor Member Office: 314-735-0270 Website: http://www.linktechs.net <http://www.linktechs.net/> LIVE On-Line Mikrotik Training <http://www.linktechs.net/onlinetraining.asp> The information transmitted (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is intended only for the person(s) or entity/entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited, If you received this in error, please contact the sender and delete the material from any computer. Jeffrey Negro wrote: For anyone trying to troubleshoot any strange resolution or page loading issues, Register.com is apparently having a massive DNS hosting outage that has been going on since 2 days ago, and is still continuing. I only found out because our monitoring was complaining about one single domain on and off. After speaking with register.com support, they admitted to the ongoing issue. If anyone has reported this already on the list, I apologize in advance for the repetition. Jeffrey
Well if they had said it was a DOS attack I wouldn't expect an ETA. When all they say is that they're having issues, I expect some form of ETA. Jeffrey Negro, Network Engineer Billtrust - Improving Your Billing, Improving Your Business www.billtrust.com 609.235.1010 x137 jnegro@billtrust.com -----Original Message----- From: Jack Bates [mailto:jbates@brightok.net] Sent: Friday, April 03, 2009 3:52 PM To: Jeffrey Negro Cc: Dennis Burgess - LTI; nanog@nanog.org Subject: Re: Register.com DNS hosting issues Jeffrey Negro wrote:
No ETA given to me, just the stock line of "We apologize.. blah blah... as soon as possible.. blah blah."
Do you normally give an ETA concerning DOS issues? Jack
Jeffrey Negro wrote:
No ETA given to me, just the stock line of "We apologize.. blah blah... as soon as possible.. blah blah."
This is probably a good time to remind the uninitiated to have some secondary DNS with a totally separate company if your DNS is that important to you. ~Seth
Seth Mattinen wrote:
Jeffrey Negro wrote:
No ETA given to me, just the stock line of "We apologize.. blah blah... as soon as possible.. blah blah."
This is probably a good time to remind the uninitiated to have some secondary DNS with a totally separate company if your DNS is that important to you.
Preferably with a provider that announces out of multiple ASN :) AT&T and Akami both provide good distributed DNS service. I imagine there are other carriers, but I can't comment on them as I haven't used them.
On Fri, Apr 03, 2009 at 01:31:33PM -0700, Charles Wyble wrote:
Seth Mattinen wrote:
Jeffrey Negro wrote:
No ETA given to me, just the stock line of "We apologize.. blah blah... as soon as possible.. blah blah."
This is probably a good time to remind the uninitiated to have some secondary DNS with a totally separate company if your DNS is that important to you.
Preferably with a provider that announces out of multiple ASN :)
AT&T and Akami both provide good distributed DNS service. I imagine there are other carriers, but I can't comment on them as I haven't used them.
if you want secondary dns via ghetto cgi, you can get it here: http://puck.nether.net/dns/ - jared -- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.
On Fri, 3 Apr 2009, Charles Wyble wrote:
This is probably a good time to remind the uninitiated to have some secondary DNS with a totally separate company if your DNS is that important to you.
Preferably with a provider that announces out of multiple ASN :)
AT&T and Akami both provide good distributed DNS service. I imagine there are other carriers, but I can't comment on them as I haven't used them.
I can highly recommend DNSmadeEasy.com. Inexpensive, Anycasted, always fast and reliable. Good for primary and/or secondary, IMO, though it is sage advice to use two different providers if you are super ultra serious about never being down. --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ ---------------------------------------------------------------------------
On Sat, Apr 4, 2009 at 2:05 PM, Peter Beckman <beckman@angryox.com> wrote:
On Fri, 3 Apr 2009, Charles Wyble wrote:
This is probably a good time to remind the uninitiated to have some
secondary DNS with a totally separate company if your DNS is that important to you.
Preferably with a provider that announces out of multiple ASN :)
AT&T and Akami both provide good distributed DNS service. I imagine there are other carriers, but I can't comment on them as I haven't used them.
I can highly recommend DNSmadeEasy.com. Inexpensive, Anycasted, always fast and reliable. Good for primary and/or secondary, IMO, though it is sage advice to use two different providers if you are super ultra serious about never being down.
Seconded. We use DNSmadeeasy as a primary for quite a few domains, but also have had good luck with DynDNS as well. -brandon
--------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ ---------------------------------------------------------------------------
-- Brandon Galbraith Voice: 630.400.6992 Email: brandon.galbraith@gmail.com
* Peter Beckman:
I can highly recommend DNSmadeEasy.com. Inexpensive, Anycasted, always fast and reliable. Good for primary and/or secondary, IMO, though it is sage advice to use two different providers if you are super ultra serious about never being down.
Or put some of your DNS servers on the same connectivity as your main services. After all, DNS is not an end in itself for most people. Running some of the servers yourself makes sure those are available even if some other customer at your DNS provider is DoSed, taking the entire DNS provider out at the same time. (Speaking in general, not about specific cases.) And if you're the DoS target, ultra-resilient DNS will simply cause the attackers to pick some other weakness of your setup. IMHO, fate-sharing as a strategy for increasing availability is somewhat underrated.
IMHO, fate-sharing as a strategy for increasing availability is somewhat underrated.
from rfc 2182 3.3. A Myth Exploded An argument is occasionally made that there is no need for the domain name servers for a domain to be accessible if the hosts in the domain are unreachable. This argument is fallacious. + Clients react differently to inability to resolve than inability to connect, and reactions to the former are not always as desirable. + If the zone is resolvable yet the particular name is not, then a client can discard the transaction rather than retrying and creating undesirable load on the network. + While positive DNS results are usually cached, the lack of a result is not cached. Thus, unnecessary inability to resolve creates an undesirable load on the net. + All names in the zone may not resolve to addresses within the detached network. This becomes more likely over time. Thus a basic assumption of the myth often becomes untrue. It is important that there be nameservers able to be queried, available always, for all forward zones. randy
* Randy Bush:
IMHO, fate-sharing as a strategy for increasing availability is somewhat underrated.
from rfc 2182
Randy, I didn't write, "don't keep off-site name servers". I wrote, "keep on-site name servers, even if you pay for off-site name service".
3.3. A Myth Exploded
+ While positive DNS results are usually cached, the lack of a result is not cached. Thus, unnecessary inability to resolve creates an undesirable load on the net.
This has been corrected in some implementations since then.
It is important that there be nameservers able to be queried, available always, for all forward zones.
Not answering crap queries (such as queries to addresses for which the resolver has a good reason to believe that they are still unreachable) tends to increase network load, but in some cases, it's the only way to make people notice the problem (like flooding servers with identical queries at an 1/RTT rate). It pushes some of the hurt to a place where it can be addressed. But looking back at incidents such as the Zonelabs/Abovenet issue, your advice is correct for the network we have today. However, we're really covering up a resolver implementation issue, nothing more.
But looking back at incidents such as the Zonelabs/Abovenet issue, your advice is correct for the network we have today.
as that rfc is over a decade old, i am not optimistic that change is neigh <sigh>. and it is amusing to see ;; ANSWER SECTION: harvard.edu. 10794 IN NS ns2.harvard.edu. harvard.edu. 10794 IN NS ns3.br.harvard.edu. harvard.edu. 10794 IN NS ns.harvard.edu. harvard.edu. 10794 IN NS ns1.harvard.edu. ;; ADDITIONAL SECTION: ns.harvard.edu. 10794 IN A 128.103.201.100 ns1.harvard.edu. 10794 IN A 128.103.200.101 ns2.harvard.edu. 10794 IN A 128.103.1.1 ns3.br.harvard.edu. 10794 IN A 128.119.3.170 and ;; ANSWER SECTION: mit.edu. 21600 IN NS STRAWB.mit.edu. mit.edu. 21600 IN NS W20NS.mit.edu. mit.edu. 21600 IN NS BITSY.mit.edu. ;; ADDITIONAL SECTION: BITSY.mit.edu. 21600 IN A 18.72.0.3 STRAWB.mit.edu. 21600 IN A 18.71.0.151 W20NS.mit.edu. 21600 IN A 18.70.0.160 but microsoft/hotmail learned the lesson the hard way, if you remember, and look to have reasonable looking deployment, though i have not looked at traceroutes. randy
* Randy Bush:
But looking back at incidents such as the Zonelabs/Abovenet issue, your advice is correct for the network we have today.
as that rfc is over a decade old, i am not optimistic that change is neigh <sigh>.
DNSSEC obscures quite a few failures which can hit secondaries. I think it changes the cost/benefit ratio of additional name service somewhat. Without DNSSEC, it's just another party who can redirect your traffic to Elbonia, so I understand if folks are quite conservative about it.
On Fri, 3 Apr 2009 17:38:43 -0500 Jorge Amodio <jmamodio@gmail.com> wrote:
someone should write an rfc on that
why not read the one you wrote, it's just 12 years old
"We don't read. Very few system developers are familiar with work done outside of their own project." --Peter Neumann, "The Role of Motherhood in the Pop Art of System Programing", ACM Symposium on Operating Systems Principles, 1969, http://www.multicians.org/pgn-motherhood.html --Steve Bellovin, http://www.cs.columbia.edu/~smb
participants (12)
-
Brandon Galbraith -
Charles Wyble -
Clinton Popovich -
Florian Weimer -
Jack Bates -
Jared Mauch -
Jeffrey Negro -
Jorge Amodio -
Peter Beckman -
Randy Bush -
Seth Mattinen -
Steven M. Bellovin