On Sat, 30 Jul 2011, Drew Weaver wrote:

...

my DNS servers were getting slow so I blocked recursive queries for all but my own network.

This should be the standard practice. By operating an open recursor, you lend your DNS server to abuse as a contributor to DNS reflection/amplification attacks.

-----------------------------------------------------------------------

And at this point he may as well just ACL in-front of the recursors to prevent the traffic from hitting the servers thus reducing load needed to reject the queries on the servers themselves.

An awful lot of older/smaller deployments have single servers doing both authoratative and recursive DNS. These should be setup with either an allow-recursion { ACL;} statement or separate authoratative and recursive views limiting recursion to just those networks that should be sending recursive queries. Another option is to run separate services bound to different individual IPs on the server. i.e. bind9 or powerdns for authoratative DNS and unbound for recursion. ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

I don't think anycast works the way you think it does. It'll distribute load for single dns servers, but not the case that he is describing. -j On Sat, Jul 30, 2011 at 12:01 PM, Alex Nderitu <nderitualex@gmail.com>wrote:

Dns anycast can in addition to acl help distribute load. On Jul 30, 2011 9:44 PM, "Jon Lewis" <jlewis@lewis.org> wrote:

...

On Sat, 30 Jul 2011, Drew Weaver wrote:

...

...

my DNS servers were getting slow so I blocked recursive queries for all but my own network.

This should be the standard practice. By operating an open recursor, you lend your DNS server to abuse as a contributor to DNS reflection/amplification attacks.

-----------------------------------------------------------------------

And at this point he may as well just ACL in-front of the recursors to prevent the traffic from hitting the servers thus reducing load needed to reject the queries on the servers themselves.

An awful lot of older/smaller deployments have single servers doing both authoratative and recursive DNS. These should be setup with either an allow-recursion { ACL;} statement or separate authoratative and recursive views limiting recursion to just those networks that should be sending recursive queries.

Another option is to run separate services bound to different individual IPs on the server. i.e. bind9 or powerdns for authoratative DNS and unbound for recursion.

---------------------------------------------------------------------- Jon Lewis, MCP :) | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

On Sat, Jul 30, 2011 at 11:33 AM, Drew Weaver <drew.weaver@thenap.com>wrote:

And at this point he may as well just ACL in-front of the recursors to prevent the traffic from hitting the servers thus reducing load needed to reject the queries on the servers themselves.

A problem for providers of DNS recursive servers as a hosted service, is the client sender IP address may be dynamic and off-net. And the DNS protocol does not provide a method of authentication, or passing credentials from the client to the server to authorize the use of recursive DNS. This differs from SMTP. There really is no such thing as a "closed recursive resolver", except where unwanted queries are blocked by IP. All we really have is TSIG for such scenarios, and most client resolvers do not support loading the resolver with a secret key, in order to authorize recursive access. So it follows, that in a number cases, "closing recursive access" is not a good option. A good example, would be services such as OpenDNS. Regards, -- -JH

On Sat, Jul 30, 2011 at 5:53 PM, Dobbins, Roland <rdobbins@arbor.net> wrote:

On Jul 31, 2011, at 3:08 AM, Jimmy Hess wrote:

...

A good example, would be services such as OpenDNS. One can argue a) that services like OpenDNS aren't necessarily a Good Thing when run by those who don't take the proper precautions

Is there an RFC specifying precisely what are considered the proper precautions? "precautions" should ideally be enabled in BIND by default.

and b) that OpenDNS in particular is run by smart, responsible folks who take extra measures to ensure they don't become a party to DNS reflection/amplification attacks, even though they operate an open recursive lookup service.

My point here is that... splitting recursive service and cordoning off recursive service by using a firewall or ACL based on IP address, is only a partially effective operational workaround to a serious defect in the protocol. Authoritative service is as subject to DoS as ever, especially with IPv6 and DNSSEC deployments. It doesn't fall under "best common practices"; it falls under "necessary evils", that everyone pretty much has to do. Right now we have to say "separate and restrict access to your recursive service", because it is the only option we can implement without changes to the protocol, and for now it should be done wherever possible. But this is not pallatable at all. The DNS protocol should be fixed so we don't need this workaround. Standardization effort should concentrate on changing the protocol. Restricting recursive service is a good short term solution, but it is not for every case. The workaround is not a best practice, it is evidence that the DNS protocol should be changed. How can it be changed? Add a proof-of-work for every DNS query and an additional round-trip for the first recursion-desired DNS query a client makes to any restricted DNS server, to obtain some "client token" or other unique hash to be sent along with queries. -- -JH

In message <09D7A1D0-0B13-4570-8891-835CA6568AD7@arbor.net>, "Dobbins, Roland" writes:

On Jul 31, 2011, at 9:15 AM, "Jimmy Hess" <mysidia@gmail.com> wrote:

...

Is there an RFC specifying precisely what are considered the proper prec= autions? "precautions" should ideally be enabled in BIND by default.

Not of which I'm aware. I'm happy to contribute to any efforts you or anyon= e else are volunteering to coordinate on this topic, as it's important work= which ought to be done sooner & not later.

Named already takes proper precautions by default. Recursive service is limited to directly connected networks by default. The default was first changed in 9.4 (2007) which is about to go end-of-life once the final wrap up release is done.

...

My point here is that... splitting recursive service and cordoning off = recursive service by using a firewall

Using a stateful firewall is contraindicated.=20

...

or ACL based on IP address, is only a partially effective operational wo= rkaround to a serious defect in the protocol.

It's actually a combination of serious defects in a number of protocols, st= arting with IPv4 itself.=20

The real problem is that many ISP's don't do effective ingress/egress filtering. This prevents compromised machines impersonating other machines. There is absolutely no reason that DSL and Cable connection shouldn't be tied down to provider source addresess and perhaps a specfic set of exception source addresses on request. There is absolutely no reason that NOC's and other service networks in the ISPs shouldn't be tied down. If you to spoof addresses for testing do it from networks explictly setup to support this.

IPv4 went into service ~27 years ago as testbed protocol. The first formal = security assessment of the IPv4 core protocols was completed last month.=20

IPv6 inherits all the security problems inherent in IPv4, & introduces new = ones - the ND mess & the untold billions of dollars in opex costs that the = consonance of the letters 'B', 'C', 'D', & 'E' will entail are just two gla= ring examples.=20

So, if we're looking for protocol architecture dragons to slay, there're fa= r more basic issues in need of a strong sword-arm before we even get to out= liers like the DNS, heh.

...

Authoritative service is as subject to DoS as ever, especially with IPv6 = and DNSSEC deployments.

And there are deployment & operational BCPs which can mitigate DDoS of auth= oritative DNS services, from logical separation of functions (see <https://= files.me.com/roland.dobbins/m4g34u> for an example) to S/RTBH to flowspec t= o commercial IDMS solutions.=20

[Full disclosure: my employer is a vendor of such solutions.]

...

The DNS protocol should be fixed so we don't need this workaround.

By all means.=20

In the meantime, it's important to realize that many of the resource constr= aints which were extant at the time the DNS was designed no longer hold swa= y. DNS reflection/amplification attacks can be eliminated by the simple (a= nd, nowadays, practical, IMHO) expedient of re-configuring resolver-facing = DNS servers to force the use TCP by default.=20

The best part is that DNS operators can start doing this today, without rec= ourse to standards bodies, working groups, et. al. No need for any inter-o= rganizational coordination at all - each can move at his own pace, within h= is own span of administrative control.=20

As an aside, it should be noted that switching the DNS over to TCP by defau= lt would accomplish a great deal of what DNSSEC is intended to provide, wit= h far less in the way of architectural, operational, & administrative compl= exity.=20

Most folks who still misguidedly filter TCP/53 are unlikely to support EDNS= 0, either, so they're already hurting - a widespread switch to TCP would li= kely finally force them out of their jungle fastnesses and out blinking int= o the sunlight.=20

;>

...

Standardization effort should concentrate on changing the protocol.

It's important to keep in mind how long the last substantive changes to the= DNS have required, as opposed to the near-term expedient of implementing B= CPs & switching to TCP.=20

...

Restricting recursive service is a good short term solution, but it is n= ot for every case.

Concur 100%. Meanwhile, we continue to resolve with the DNS we have until t= he new, improved version is designed, tested, & generally deployed.=20

;>

...

The workaround is not a best practice, it is evidence that the DNS protoc= ol should be changed.

There's a reason proof-of-work solutions have never been widely deployed - = in the final analysis, they all too often end up simply a form of cost-shif= ting which leaves the system in question just as vulnerable (if not more so= ) to DoS as it was in the first place, just within a different transaction = path/layer.=20

So, while I agree with you that changes are called for, implementing proofs= -of-work into the DNS transactional model isn't one I'd personally recommen= d, FWIW.=20

---------------------------------------------------------------------

Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>

Sell your computer and buy a guitar.=20

-- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org

In message <AE105312-3108-4B0B-8445-7116B84EC428@arbor.net>, "Dobbins, Roland" writes:

On Aug 1, 2011, at 7:42 AM, Mark Andrews wrote:

...

Named already takes proper precautions by default. Recursive service is = limited to directly connected networks by default. The default was first changed in 9.4 (2007) which is about to go end-of-life once the= final wrap up release is done.

This alone isn't enough. There are quite a few other things folks must do = from an architectural and operational standpoint which aren't found in name= d.conf.

...

The real problem is that many ISP's don't do effective ingress/egress fil= tering.

Well, no. The real problem is a protocol set/implementation which lends it= self so readily to spoofing in the first place, followed (as you say) by IS= P/endpoint network inattention to anti-spoofing, followed by protocols whic= h make use of the eminently-spoofable UDP for a critical service.

And even if DNS/TCP was use by default machines can still get DoS'd because IP is spoofable. This one looks like a direct attack on the machine as there are multiple source addresses rather than a reflector attack unless they are attempting to attack thousands of sites simultaniously.

...

This prevents compromised machines impersonating other machines.

Concur, but see above - spoofing is the symptom, not the disease.

----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>

The basis of optimism is sheer terror.

-- Oscar Wilde

-- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org

More good stuff here: http://www.team-cymru.org/Services/Resolvers/ Frank -----Original Message----- From: Dobbins, Roland [mailto:rdobbins@arbor.net] Sent: Friday, July 29, 2011 5:40 PM To: NANOG list Subject: Re: DNS DoS ??? On Jul 30, 2011, at 1:51 AM, Elliot Finley wrote:

my DNS servers were getting slow so I blocked recursive queries for all but my own network.

This should be the standard practice. By operating an open recursor, you lend your DNS server to abuse as a contributor to DNS reflection/amplification attacks. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> The basis of optimism is sheer terror. -- Oscar Wilde