strategies to mitigate DNS amplification attacks in ISP network
Hi, as around 40% of ASNs allow at least partial IPv4 address spoofing in their network(http://spoofer.csail.mit.edu/summary.php) and there are around 30 million open-resolvers(http://openresolverproject.org/) in the Internet, then DNS amplification traffic is daily occasion for ISPs. This in probably mainly because RPF checks and DNS RRL(https://kb.isc.org/article/AA-01000/0/A-Quick-Introduction-to-Response-Rate-...) are not ubiquitously implemented, recursive requests without any ACLs in DNS servers are often allowed, it requires little effort from attackers point of view and is effective attack method. Unfortunately, there seems to be very limited number of countermeasures for ISPs. Few which I can think of: 1) higher capacity backbone links - I'm not sure if this can be considered a mitigation method, but at least it can help to affect smaller amount of customers if traffic volumes are not very high 2) rate-limit incoming DNS traffic flows on peering and uplink ports - here I mean something similar to iptables "recent" module(http://www.netfilter.org/documentation/HOWTO/netfilter-extensions-HOWTO-3.ht...) which allows certain number of certain type of packets in a configured time-slot per IP. However, such functionality is probably not common on edge or backbone routers. Tracking the packet state does definitely not work because state table should be synchronized between all the routers in the network and again, this requires Internet-routers to have stateful firewall functionality. In addition, one also needs to allow new DNS connections from Internet to its network. If one simply polices incoming DNS traffic on uplink and peering ports(for example if baseline DNS traffic is 5Mbps, then policer is set to 50Mbps), then legitimate customers DNS traffic is also affected in case of actual attack occurs and policer starts to drop DNS traffic, i.e. policer has no way to distinguish between the legitimate and non-legitimate incoming DNS traffic. Am I wrong in some points? What are the common practices to mitigate DNS amplification attacks in ISP network? thanks, Martin
On 1 Dec 2015, at 23:59, Martin T wrote:
What are the common practices to mitigate DNS amplification attacks in ISP network?
Situationally-appropriate network access policies instantiated as ACLs on hardware-based routers/layer-3 switches in IDCs, on customer aggregation routers, in mitigation centers, etc. S/RTBH. flowspec. IDMS (full disclosure, I work for a vendor of such systems). See this .pdf preso: <https://app.box.com/s/r7an1moswtc7ce58f8gg> Statefulness is out, as you indicate. QoS is out, as you indicated (e.g., legitimate traffic is 'crowded out' by programmatically-generated attack traffic). The real solution to this entire problem set is source-address validation, as you indicate. Until the happy day when we've achieved universal source-address validation arrives, various combinations of the above. ----------------------------------- Roland Dobbins <rdobbins@arbor.net>
On 2 Dec 2015, at 0:14, Roland Dobbins wrote:
Until the happy day when we've achieved universal source-address validation arrives, various combinations of the above.
I forgot to mention RRL on authoritative servers, apologies. ----------------------------------- Roland Dobbins <rdobbins@arbor.net>
Martin- I represent a statewide educational network running Juniper gear that is a quasi-enterprise. I think efforts depend on size and type of network. We are testing an approach that involves; 1) whitelisting known local resolvers, well behaved cloud DNS resolvers. 2) on ingress, policing non-conforming traffic that matches UDP src port 53 dst port unreserved bytes > 1400 3) on ingress, queuing fragments to high packet loss priority [you better understand how fragments are used in your network before doing this] 4) If you have Juniper gear look into prefix-action 5) RTBH if required This obviously doesn't work on the core of the internet. Other good tips: * strengthen [anycast] your local DNS resolvers and consider a scheme that allows you to change their outgoing address on the fly. * push [some] of your external authoritative DNS to the cloud. -Michael
-----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Martin T Sent: Tuesday, December 01, 2015 11:00 AM To: nanog@nanog.org Subject: strategies to mitigate DNS amplification attacks in ISP network
Hi,
as around 40% of ASNs allow at least partial IPv4 address spoofing in their network(http://spoofer.csail.mit.edu/summary.php) and there are around 30 million open-resolvers(http://openresolverproject.org/) in the Internet, then DNS amplification traffic is daily occasion for ISPs. This in probably mainly because RPF checks and DNS RRL(https://kb.isc.org/article/AA-01000/0/A-Quick-Introduction-to-Response- Rate-Limiting.html) are not ubiquitously implemented, recursive requests without any ACLs in DNS servers are often allowed, it requires little effort from attackers point of view and is effective attack method. Unfortunately, there seems to be very limited number of countermeasures for ISPs. Few which I can think of:
1) higher capacity backbone links - I'm not sure if this can be considered a mitigation method, but at least it can help to affect smaller amount of customers if traffic volumes are not very high
2) rate-limit incoming DNS traffic flows on peering and uplink ports - here I mean something similar to iptables "recent" module(http://www.netfilter.org/documentation/HOWTO/netfilter- extensions- HOWTO-3.html#ss3.16) which allows certain number of certain type of packets in a configured time-slot per IP. However, such functionality is probably not common on edge or backbone routers.
Tracking the packet state does definitely not work because state table should be synchronized between all the routers in the network and again, this requires Internet-routers to have stateful firewall functionality. In addition, one also needs to allow new DNS connections from Internet to its network. If one simply polices incoming DNS traffic on uplink and peering ports(for example if baseline DNS traffic is 5Mbps, then policer is set to 50Mbps), then legitimate customers DNS traffic is also affected in case of actual attack occurs and policer starts to drop DNS traffic, i.e. policer has no way to distinguish between the legitimate and non-legitimate incoming DNS traffic.
Am I wrong in some points? What are the common practices to mitigate DNS amplification attacks in ISP network?
thanks, Martin
Deploy DNS COOKIES. This allows legitimate UDP traffic to be identified and treated differently to spoofed traffic by providing the equivalent to a TCP handshake but over UDP. This is currently in IETF last call but the code points are assigned and implementations are available. Ask your nameserver vendor for this today. Ask your OS vendor for support. https://tools.ietf.org/html/draft-ietf-dnsop-cookies-07 Mark In message <DM3PR0601MB2011982419777A0C7960842FF90F0@DM3PR0601MB2011.namprd06.p rod.outlook.com>, Michael Hare writes:
Martin-
I represent a statewide educational network running Juniper gear that is a qu asi-enterprise. I think efforts depend on size and type of network. We are testing an approach that involves;
1) whitelisting known local resolvers, well behaved cloud DNS resolvers. 2) on ingress, policing non-conforming traffic that matches UDP src port 53 d st port unreserved bytes > 1400 3) on ingress, queuing fragments to high packet loss priority [you better und erstand how fragments are used in your network before doing this] 4) If you have Juniper gear look into prefix-action 5) RTBH if required
This obviously doesn't work on the core of the internet.
Other good tips: * strengthen [anycast] your local DNS resolvers and consider a scheme that al lows you to change their outgoing address on the fly. * push [some] of your external authoritative DNS to the cloud.
-Michael
-----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Martin T Sent: Tuesday, December 01, 2015 11:00 AM To: nanog@nanog.org Subject: strategies to mitigate DNS amplification attacks in ISP network
Hi,
as around 40% of ASNs allow at least partial IPv4 address spoofing in their network(http://spoofer.csail.mit.edu/summary.php) and there are around 30 million open-resolvers(http://openresolverproject.org/) in the Internet, then DNS amplification traffic is daily occasion for ISPs. This in probably mainly because RPF checks and DNS RRL(https://kb.isc.org/article/AA-01000/0/A-Quick-Introduction-to-Respons e- Rate-Limiting.html) are not ubiquitously implemented, recursive requests without any ACLs in DNS servers are often allowed, it requires little effort from attackers point of view and is effective attack method. Unfortunately, there seems to be very limited number of countermeasures for ISPs. Few which I can think of:
1) higher capacity backbone links - I'm not sure if this can be considered a mitigation method, but at least it can help to affect smaller amount of customers if traffic volumes are not very high
2) rate-limit incoming DNS traffic flows on peering and uplink ports - here I mean something similar to iptables "recent" module(http://www.netfilter.org/documentation/HOWTO/netfilter- extensions- HOWTO-3.html#ss3.16) which allows certain number of certain type of packets in a configured time-slot per IP. However, such functionality is probably not common on edge or backbone routers.
Tracking the packet state does definitely not work because state table should be synchronized between all the routers in the network and again, this requires Internet-routers to have stateful firewall functionality. In addition, one also needs to allow new DNS connections from Internet to its network. If one simply polices incoming DNS traffic on uplink and peering ports(for example if baseline DNS traffic is 5Mbps, then policer is set to 50Mbps), then legitimate customers DNS traffic is also affected in case of actual attack occurs and policer starts to drop DNS traffic, i.e. policer has no way to distinguish between the legitimate and non-legitimate incoming DNS traffic.
Am I wrong in some points? What are the common practices to mitigate DNS amplification attacks in ISP network?
thanks, Martin -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org
On Tue, Dec 1, 2015 at 11:59 AM, Martin T <m4rtntns@gmail.com> wrote:
Am I wrong in some points? What are the common practices to mitigate DNS amplification attacks in ISP network?
Hi Martin, You seem to be focused on DNS amplification from the perspective of the attack's target. To the target, it's just another DDOS attack. As with other DDOS attacks, you reroute the contained /24 to a DDOS mitigator who specializes in removing unwanted packets from the data stream and passing the rest to your network via a tunnel. The mitigator writes custom software on expensive server arrays which figure out the attack de jour signatures and scrub the packet flows. Some folks rate-limit UDP flows. This just kills everything sooner during an attack since you kinda need DNS to work. Rate limiting by source turns your DNS requests stateful... a happy fun way to shoot yourself in the foot. Really, your best bet is to treat it as just another DDOS and let the guy you pay for DDOS service handle the details. Regards, Bill Herrin -- William Herrin ................ herrin@dirtside.com bill@herrin.us Owner, Dirtside Systems ......... Web: <http://www.dirtside.com/>
Hi, depends on the type of ISP you are and the bandwidth used in the attack. If most attacks are targeted for www.example.com then you could design your net so that www.example.com is just a TCP service VIP that never needs any UDP. This would make it possible to place simple ACL on your edge to get rid of most stuff. Yes there are people that know how to correctly DDOS but most just give up after there attack traffic never affects the service. If the bandwidth exceeds your transit/peering capacity you need to filter/blackhole it upstream. You can also isolate the prefix under attack to a single transit or a DDOS mitigation service to prevent the other prefixes from being impacted. Other useful stuff is a flow based traffic analysis tool to get details about the attack. Karsten 2015-12-01 17:59 GMT+01:00 Martin T <m4rtntns@gmail.com>:
Hi,
as around 40% of ASNs allow at least partial IPv4 address spoofing in their network(http://spoofer.csail.mit.edu/summary.php) and there are around 30 million open-resolvers(http://openresolverproject.org/) in the Internet, then DNS amplification traffic is daily occasion for ISPs. This in probably mainly because RPF checks and DNS RRL(https://kb.isc.org/article/AA-01000/0/A-Quick-Introduction-to-Response-Rate-...) are not ubiquitously implemented, recursive requests without any ACLs in DNS servers are often allowed, it requires little effort from attackers point of view and is effective attack method. Unfortunately, there seems to be very limited number of countermeasures for ISPs. Few which I can think of:
1) higher capacity backbone links - I'm not sure if this can be considered a mitigation method, but at least it can help to affect smaller amount of customers if traffic volumes are not very high
2) rate-limit incoming DNS traffic flows on peering and uplink ports - here I mean something similar to iptables "recent" module(http://www.netfilter.org/documentation/HOWTO/netfilter-extensions-HOWTO-3.ht...) which allows certain number of certain type of packets in a configured time-slot per IP. However, such functionality is probably not common on edge or backbone routers.
Tracking the packet state does definitely not work because state table should be synchronized between all the routers in the network and again, this requires Internet-routers to have stateful firewall functionality. In addition, one also needs to allow new DNS connections from Internet to its network. If one simply polices incoming DNS traffic on uplink and peering ports(for example if baseline DNS traffic is 5Mbps, then policer is set to 50Mbps), then legitimate customers DNS traffic is also affected in case of actual attack occurs and policer starts to drop DNS traffic, i.e. policer has no way to distinguish between the legitimate and non-legitimate incoming DNS traffic.
Am I wrong in some points? What are the common practices to mitigate DNS amplification attacks in ISP network?
thanks, Martin
participants (7)
-
Karsten Elfenbein -
Mark Andrews -
Martin T -
Michael Hare -
Roland Dobbins -
Stepan Kucherenko -
William Herrin