Re: Tightened DNS security question re: DNS amplification attacks.

27 Jan 2009

      At 03:16 PM 1/27/2009, Nate Itkin wrote:
...
On Tue, Jan 27, 2009 at 03:04:19PM -0500, Matthew Huff wrote:
...
< ... snip ... >
dns queries to the . hint file
are still occuring and are not being denied by our servers. For example:
27-Jan-2009 15:00:22.963 queries: client 64.57.246.146#64176: view
external-in: query: . IN NS +
< ... snip ... >
since you can't put a "allow-query { none; };" in a hint zone, 
what can I do
to deny the query to the . zone file?
AFAIK, that's about the best you can do with the DNS configuration. You've
mitigated the amplification value, so hopefully the perpetrator(s) will drop
you. If you're willing to keep up with the moving targets, the next level
is an inbound packet filter. Add to your inbound ACL:
deny udp host 64.57.246.146 neq 53 any eq 53
Also on this topic:
Coincident with this DNS DOS, I started seeing inbound PTR queries from
various hosts on 10.0.0.0/8 (which are blackholed by my DNS servers).
They receive no response, yet they persist.  Anyone have thoughts on their
part in the scheme?
Best wishes,
Nate Itkin
I'm not seeing those PTR queries for 10.0.0.0/8, but then my perimeter
ingress/egress filters (BCP 38) toss most of that kind of junk before my
DNS servers ever see it.

I agree that is as far as one can go with BIND, right now.  However, that
isn't making the perpetrators cease and desist.  I am seeing ongoing query
attempts coming in and refusal packets going back out, and the targets
don't seem to change until I do something to block them.  So mitigating
the amplification factor does not seem of interest to these perpetrators.
On the contrary, even REFUSED responses can aggregate with some amplified
responses to enhance the apparent DoS goal.  Thus BCP 140 seems to be
less than completely effective because it is less than universally applied
(i.e., older versions of BIND or misconfigured BIND.)  I think the same
situation is true with BCP 38: less than universally applied.  So do I wait
for universal application of these BCPs, or do I take responsibility for
doing what I can to make my network resources less appealing for abuse?

I choose the latter, and that is why went to the effort of blocking this
abusive traffic before it reaches my authoritative-only DNS servers.
Nevertheless, I also agree with a point made last week that trying to keep up
with the changing targets is a game of whack-a-mole that is and will continue
to be a drain on network management resources -- if the detection and response
continues to be deployed manually.  This is why I wrote some Perl for my
authoritative-only servers to automate detection and response at the server
level.  Granted it isn't a permanent solution, but at least it is a place
to start.  I appended that code below for those who are interested in it.

Notes
-----
* Does not blindly block all queries from an IP address, as would be the
   case with an ACL.
* Assumes BIND 9.4+ configured for authoritative-only role and configured to
   respond with REFUSE for queries to zones for which it is not authoritative.
* Uses BIND 9.4+ syslog messages and a state table as inputs.
* Alters IPtables rules and a state table as outputs.
* Runs as a periodic cron job (currently every 10 minutes).
* Performs a logtail on BIND 9.4+ syslog messages to detect abusive
   queries matching a pattern.
* Implements a minimum detection threshold to reduce the false positive rate
   (currently 1/minute a.k.a. 10/cycle)
* Implements a state table with last-seen timestamps to maintain state
   between job runs.
* Implements an expiry mechanism (currently 24 hours) which is extended each
   time a source is re-detected.

Dependencies
------------
* Linux 2.4+ kernel
* BIND 9.4+
* An account sufficiently privileged to read local syslog and modify
   IPtables rules.
* Logtail
* Cron
         */10 * * * * /usr/local/sbin/dns-reflecter-finder
* IPtables key rules:
         iptables -N dns-reflecter
         iptables -A {INPUT} -p udp --sport ! 53 --dport 53 -j dns-reflecter
                 (place this above your general ALLOW rules for DNS)

#!/usr/bin/perl
#

use strict;
use Data::Dumper;

my $BASENAME = "/bin/basename";
my $LN = "/bin/ln";
my $LOGTAIL = "/usr/sbin/logtail";
my $IPTABLES = "/sbin/iptables";
my $LOGDIR = "/var/log";
my $RUNDIR = "/var/run";

my $Progname = $0;
$Progname = `$BASENAME $Progname`;
chomp ($Progname);

my $IPtablesChain = "dns-reflecter";
my $RealLogFile = "$LOGDIR/messages";
my $LogFile = "$LOGDIR/$Progname.log";
my $DBfile = "$RUNDIR/$Progname.dat";
my $DetectPeriod = 600;         # This should match the cron period
my $DetectThold = $DetectPeriod / 60;
my $ExpiryPeriod = 60 * 60 * 24;
my $Now = time();

my $Debug = 0;
if ($#ARGV >= 0 && $ARGV[0] eq "-debug") {
     $Debug++;
}

#
# Set up the symlink to the real log file.
#
unless (-l $LogFile) {
     if ($Debug) {
         print "$LN -s $RealLogFile $LogFile\n";
     }
     else {
         system ($LN, "-s", $RealLogFile, $LogFile);
     }
}

#
# Find all unread log entries that are refused ". IN NS" queries with
# source port number != 53;
#
my %IPaddrsMatched = ();
my $ipaddr = "";
my $portnum = "";
if (open (LOGS, "$LOGTAIL $LogFile|")) {
     while (<LOGS>) {
         chomp;
         if (/named\[\d+\]: client ([\d\.]+)\#(\d+): .* 
\'\.\/NS\/IN\' denied$/o) {
             $ipaddr = $1;
             $portnum = $2;
             if ($portnum != 53) {
                 $IPaddrsMatched{$ipaddr}++;
             }
         }
     }
     close (LOGS);
}
else {
     die "Cannot logtail $LogFile: $!\n";
}
if ($Debug) {
     print Dumper (\%IPaddrsMatched), "\n";
}

#
# Delete IP addresses from the running if they hit with
# frequency less than the threshold.  In this case 1 every
# 60 seconds.
#
foreach $ipaddr (keys %IPaddrsMatched) {
     if ($IPaddrsMatched{$ipaddr} < $DetectThold) {
         delete ($IPaddrsMatched{$ipaddr});
     }
}

#
# If our db file exists, read it into a hash.
#
my %IPaddrsCached = ();
my $when = 0;
if (-f $DBfile) {
     if (open (DB, $DBfile)) {
         while (<DB>) {
             chomp;
             #
             # Decode the IP address and the Unix epoch timestamp
             # when it was last found in logs.
             #
             ($ipaddr, $when) = split (/\s+/o);
             #
             # If the entry has been quiescent for less than the
             # expire time, retain it.  If not, skip it.
             #
             if ($Now - $when <= $ExpiryPeriod) {
                 $IPaddrsCached{$ipaddr} = $when;
             }
         }
         close (DB);
     }
     else {
         die "Cannot read from $DBfile: $!\n";
     }
}
if ($Debug) {
     print Dumper (\%IPaddrsCached), "\n";
}

#
# Refresh last-seen timestamps for IP addresses detected
# during this run, and add new entries for IP addresses
# not previously seen (i.e., never before seen, or
# previously seen and expired).
#
foreach $ipaddr (keys %IPaddrsMatched) {
     $IPaddrsCached{$ipaddr} = $Now;
}

#
# Write out the updated db file.  Overwrite if
# previously existed.
#
if (open (DB, ">$DBfile")) {
     foreach $ipaddr (sort keys %IPaddrsCached) {
         print DB "$ipaddr\t$IPaddrsCached{$ipaddr}\n";
     }
     close (DB);
}
else {
     die "Cannot write to $DBfile: $!\n";
}

#
# Flush the dedicated IPtables rule chain.
#
if ($Debug) {
     print "$IPTABLES -F $IPtablesChain\n";
}
else {
     system ($IPTABLES,
             "-F",
             $IPtablesChain);
}
#
# Add to the dedicated IPtables rule chain all the entries
# just written to the db as DROP rules.
#
foreach $ipaddr (sort keys %IPaddrsCached) {
     if ($Debug) {
         print "$IPTABLES -A $IPtablesChain -s $ipaddr -j DROP\n";
     }
     else {
         system ($IPTABLES,
                 "-A",
                 $IPtablesChain,
                 "-s",
                 $ipaddr,
                 "-j",
                 "DROP");
     }
}

exit 0;

--
Douglas C. Stephens             | UNIX/Windows/Email Admin
System Support Specialist       | Network/DNS Admin
Information Systems             | Phone: (515) 294-6102
Ames Laboratory, US DOE         | Email: stephens@ameslab.gov