On 25/Mar/20 19:20, Nick Hilliard wrote:

The fix is either to remove "dnssec-lookaside auto;" from the config or else set "dnssec-lookaside no;" and then reload named.

We had issues with that feature back in 2018. We disabled it since then as a matter of course:     //dnssec-lookaside auto; Mark.

We just left the dnssec-lookaside auto; configuration in there. Probably because it specifically says in the documentation from ISC that it won't hurt anything to leave it in there... # Configuring "dnssec-lookaside auto;" to activate this key is # harmless Guess not? Thanks, -Drew -----Original Message----- From: Stephane Bortzmeyer <bortzmeyer@nic.fr> Sent: Wednesday, March 25, 2020 1:27 PM To: Drew Weaver <drew.weaver@thenap.com> Cc: 'nanog@nanog.org' <nanog@nanog.org> Subject: Re: ISC BIND 9 breakage? On Wed, Mar 25, 2020 at 05:18:49PM +0000, Drew Weaver <drew.weaver@thenap.com> wrote a message of 97 lines which said:

Did anyone else on CentOS 6 just have some DNS resolvers totally fall over?

dlv.isc.org signatures just expired.

# NOTE: The ISC DLV zone is being phased out as of February 2017;

And yet some people still use it, it seems.

On 25/03/2020 18:28, Drew Weaver wrote:

Normally when there is an impending doom moment with BIND or another software release there is at least some amount of coverage of it.

Was this not announced or known in advance?

It was accidental breakage of the RRSIGs on the dlv.isc.org zone. More detail to follow tomorrow once I've had some sleep... Ray Bellis Director of DNS Operations, ISC.

Was it a "glitch" or someone just plain old forgot to do it? At 02:29 AM 26/03/2020, Mark Andrews wrote:

It was a glitch with the re-signing of the zone. There should be a official report sometime tomorrow. That said "dnssec-lookaside auto;" has been a no-op in BIND since BIND 9.9.12, BIND 9.10.7, BIND 9.11.3 and a fatal configuration error as of BIND 9.12.0. We didn’t want the DLV lookup traffic and provides no benefit as the zone has been empty since 2017.

If you have dnssec-lookaside configured in named.conf please remove it otherwise the DLV code in the validator has to cryptographically prove that DLV records don’t exist before returning that the response is insecure. That requires talking to the servers for dlv.isc.org. It does this every hour for a active validating resolver that is still running DNSSEC lookaside validation.

Mark

...

On 26 Mar 2020, at 04:18, Drew Weaver <drew.weaver@thenap.com> wrote:

Did anyone else on CentOS 6 just have some DNS resolvers totally fall over?

I noticed that this command: dnssec-lookaside auto; was causing the issue. The issue occurred right at about 1PM EST.

I see this note in the ISC key file..

# ISC DLV: See https://www.isc.org/solutions/dlv for details. # # NOTE: The ISC DLV zone is being phased out as of February 2017; # the key will remain in place but the zone will be otherwise empty. # Configuring "dnssec-lookaside auto;" to activate this key is # harmless, but is no longer useful and is not recommended.

It’s not harmless anymore.

-- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org

-- Clayton Zekelman Managed Network Systems Inc. (MNSi) 3363 Tecumseh Rd. E Windsor, Ontario N8W 1H4 tel. 519-985-8410 fax. 519-985-8409

Nick Hilliard wrote:

forgot to re-sign the zone on dlv.isc.org or forgot to remove dnssec-lookaside from the config?

Not kidding here.  People need to take responsibility for their configurations.

Anyone running BIND provided with CentOS 6 has a release from ~2012 (bind 9.8.2) and it is understandable why their documentation is out-of-date (like OP). To get more recent bugs and fixes from ISC directly, install from ISC's copr: https://copr.fedorainfracloud.org/coprs/isc/bind-esv/ On CentOS 7 I needed to install dnf and yum-plugin-copr first. I don't see these in the usual places for CentOS 6, so getting copr sources enabled is the first challenge. ISC sources for other distros: https://www.isc.org/blogs/bind-9-packages/ Mike