Good analyze Hugo,

 

I believe that all of this volumetric attack is just noise to hide the real attack that really killed your webserver.

 

TCP Flag: SYN: 100%

 

I would start with this line and I agree that Roland’s deck might have something about SYN flood.

 

Jean

 

From: Hugo Slabbert <hugo@slabnet.com>
Sent: February 8, 2021 2:19 PM
To: Compton, Rich A <Rich.Compton@charter.com>
Cc: Mike Hammett <nanog@ics-il.net>; Jean St-Laurent <jean@ddostest.me>; NANOG list <nanog@nanog.org>
Subject: Re: [EXTERNAL] Re: Retalitory DDoS

 

Was gonna come to add that.  That and maybe some UDP frags.

 

You may want to have your hosting provider block all inbound traffic from reaching your server IP except TCP port 443 (or 80 or whatever port you actually use) somewhere upstream.  

 

Can also consider dropping by UDP source port on that 3072 and other common reflection vectors if you've got UDP-based destinations to deal with. 

 

The SYN floods are a different beast; though probably not volumetric, needs enough capacity (TCP reverse proxies / LBs / etc) to handle that and possibly things like SYN cookies.  I'll let folks more versed than myself answer there, though.  Roland probably has a deck ready to link ;)

 

-- 

Hugo Slabbert       | email, xmpp/jabber: hugo@slabnet.com

pgp key: B178313E   | also on Signal

 

 

On Mon, Feb 8, 2021 at 10:10 AM Compton, Rich A <Rich.Compton@charter.com> wrote:

FYI, that looks like a Web Services Dynamic Discovery UDP amplification DDoS attack.  https://blogs.akamai.com/sitr/2019/09/new-ddos-vector-observed-in-the-wild-wsd-attacks-hitting-35gbps.html  Very easily executed by a booter service.

You may want to have your hosting provider block all inbound traffic from reaching your server IP except TCP port 443 (or 80 or whatever port you actually use) somewhere upstream.  This can help reduce the impact of DDoS attacks on your server. 

 

-Rich

 

From: NANOG <nanog-bounces+rich.compton=charter.com@nanog.org> on behalf of Mike Hammett <nanog@ics-il.net>
Date: Monday, February 8, 2021 at 10:58 AM
To: Jean St-Laurent <jean@ddostest.me>
Cc: NANOG list <nanog@nanog.org>
Subject: [EXTERNAL] Re: Retalitory DDoS

 

CAUTION: The e-mail below is from an external source. Please exercise caution before opening attachments, clicking links, or following guidance.

I don't have RTBH, no. It's just a web server.

Now how my hosting provider handled it, I'm not sure. I don't know if they just dropped me internally, or if they used RTBH with their upstreams and peers. Only being 2.5 gigs, that should be well within their ability to handle internally, but I guess why would you if you didn't have to?



-----
Mike Hammett
Intelligent Computing Solutions

Midwest Internet Exchange

The Brothers WISP

From: "Jean St-Laurent" <jean@ddostest.me>
To: "Mike Hammett" <nanog@ics-il.net>
Cc: "NANOG list" <nanog@nanog.org>
Sent: Monday, February 8, 2021 11:53:43 AM
Subject: RE: Retalitory DDoS

You got RTBH?

 

 

In my case, it was against a server not on my own network, so my impact was a blackhole for an hour at 4 AM local time. I likely wouldn't have even noticed it, had I not received the threat email, nor the ticket my web host's NOC opened.



-----
Mike Hammett
Intelligent Computing Solutions

Midwest Internet Exchange

The Brothers WISP

From: "Jean St-Laurent" <jean@ddostest.me>
To: "Mike Hammett" <nanog@ics-il.net>, "NANOG list" <nanog@nanog.org>
Sent: Monday, February 8, 2021 11:42:12 AM
Subject: RE: Retalitory DDoS

Nice report,

 

If you would have to pick up just one vector out of this “multi-vector” attack, which one seems to be the one that had the bigger effect on your network or service?

 

Was it degraded or total service interruption?

 

Jean

 

 

Mike,

I've attached the full information we got from our DDOS protection system below.

We had a large number of ping loss and data loss tickets begin opening up for devices sharing the cabinet chi18-313. The high traffic and interference was determined to be caused by incoming traffic to the ip address [Not hard to find, but redacted anyway]. Our network engineers will be back in after 9am until 5pm CST. They have greater access to the network and may be able to give you more details.

Location : Chicago
Event Time : 2021-02-08 04:17:38 CST (-0600)
Destination IP: [Not hard to find, but redacted anyway]
Traffic : 2520 Mbps 382880 pps
Fragmentation : 11%
Top Transport Protocol:
. 99% Protocol # 17 (UDP)
TCP Flag: SYN: 100% ACK: 0% RST: 0% FIN: 0%
Top Source Port:
. 61% Port # 3702
. 38% Port # 0
Top Destination Port:
. 38% Port # 0
. 14% Port # 45934
. 9% Port # 23680
. 8% Port # 35023
. 7% Port # 25966
Top Source IP:
. 0% 112.164.127.17
Number of unique IP: 7110
Total Bytes : 1259961437
Total Packets : 1531559
Duration : 4s
Report Run Time : 151.3ms

The 30 day null route count is: 0
Number of hours to null route : 1

Location : Chicago
Event Time : 2021-02-08 04:02:38 CST (-0600)
Destination IP: [Not hard to find, but redacted anyway]
Traffic : 1817 Mbps 275483 pps
Fragmentation : 13%
Top Transport Protocol:
. 99% Protocol # 17 (UDP)
TCP Flag: SYN: 99% ACK: 0% RST: 0% FIN: 0%
Top Source Port:
. 56% Port # 3702
. 43% Port # 0
Top Destination Port:
. 43% Port # 0
. 19% Port # 25966
. 19% Port # 35023
. 17% Port # 23680
Top Source IP:
. 0% 90.49.167.239
Number of unique IP: 3577
Total Bytes : 953894831
Total Packets : 1157017
Duration : 4.199s
Report Run Time : 306.8ms

The 30 day null route count is: 0
Number of hours to null route : 1

 
Liam Doring
Systems Administrator



-----
Mike Hammett
Intelligent Computing Solutions

Midwest Internet Exchange

The Brothers WISP

From: "Mike Hammett" <nanog@ics-il.net>
To: "NANOG list" <nanog@nanog.org>
Sent: Monday, February 8, 2021 5:46:26 AM
Subject: Retalitory DDoS

https://www.dropbox.com/s/rrrx90jvy09h26s/ICS%20DDoS.png?dl=0