NANOG, I was wondering how many of you are running some sort of detection tool on "dark address" space on your network? In an effort to curb malicious outbound non-spoofed traffic from "owned" client machines I think one of the easiest methods we have is to look for scans in what should be dead space. The source-address spoofed traffic is easy to drop, the "legal" traffic is a bit more complex and I'm looking for non-inline methods of curbing this traffic. My questions are: 1) Are you doing this and if so, what tools are you using? Some sort of simple listening device with thresholds would probably do the trick if one machine monitored an entire /24 or some random /32's out of a /16. 2) What techniques seem to be better? Monitoring an entire /24 or picking a distributed selection of IPs from a /16? (using a /24 or /25 is much easier on the administrative end of things from where I sit...) 3) What sort of threshold metrics for considering something to be malicious have you found to be good? (ports/second, ip/second, etc) 4) Are there downsides to this (aside from false positives, which would hopefully be rare in truly dark address space). Off-list replies are fine and I'll summarize after a few days. thanks, davidu ---------------------------------------------------- David A. Ulevitch - Founder, EveryDNS.Net Washington University in St. Louis http://david.ulevitch.com -- http://everydns.net ----------------------------------------------------
davidu@everydns.net (David A.Ulevitch) writes:
I was wondering how many of you are running some sort of detection tool on "dark address" space on your network?
ooooh, ooooh, me!
In an effort to curb malicious outbound non-spoofed traffic from "owned" client machines I think one of the easiest methods we have is to look for scans in what should be dead space.
you're right.
The source-address spoofed traffic is easy to drop, the "legal" traffic is a bit more complex and I'm looking for non-inline methods of curbing this traffic.
since this space has no dns records pointing into it, the only traffic it will see is from errors/typo's, and network scanners. some scanners use pseudorandom selection, some are serial, but none are nonmalicious.
My questions are:
1) Are you doing this and if so, what tools are you using? Some sort of simple listening device with thresholds would probably do the trick if one machine monitored an entire /24 or some random /32's out of a /16.
in freebsd ipfw: pipe 1 config mask src-ip 0xffffffff buckets 32768 bw 10Mbit/sec pipe 1 ip from any to x.y..0.0/16 in fwd 127.0.0.2 ip from any to x.y.0.0/16 in "pipe 1" is just there for measurement purposes, and ddos prevention. the address i fwd it to is an extra loopback alias defined in rc.conf: ifconfig_lo0_alias0="inet 127.0.0.2 netmask 255.255.255.255" this box also runs zebra to inject this /16 into the local OSPF, which elsewhere triggers some router of jabley's to inject it into BGP. there are two listeners, both written locally, that are started in rc.local by scripts that look like this: while :; do ( src/httpk/i386/httpk -b reject-all.vix.com -t 3 -h 127.0.0.2 \ -s http -f endoftheline.html -l | tee tee | src/httpk/pgit.pl ) > log 2> err sleep 45 done and this: while :; do ( src/smtpk/smtpk -l 127.0.0.2 | tee tee | src/smtpk/pgit.pl ) > log 2> err sleep 2 done the "tee" file is sort of unreadable. for httpk it looks like this: src [209.148.235.157].3083; dst [149.20.195.105].80; Sat Apr 17 03:55:07 2004 GET / HTTP/1.1 Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows 98) Host: 149.20.195.105 Connection: Keep-Alive whereas for smtpk it looks like this: Message-ID: <050048052046049053050046049057049046053049@5914ASDE.vicp.net> To: <queque_2@juno.com> From: angelupno@aol.com Subject: Hey, what's up? Date: Sun, 11 Apr 2004 13:47:24 -1900 MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.3018.1300 [222.156.12.131] -> [204.152.191.0] none <67sedrftyg8017@ms56.hinet.net> \ (78) 1081855616.285020 <epschao@sogiant.twmail.net> -- the postgres databases thus populated are much prettier, as are the "log" files produced by the respective "pgit.pl" scripts.
2) What techniques seem to be better? Monitoring an entire /24 or picking a distributed selection of IPs from a /16? (using a /24 or /25 is much easier on the administrative end of things from where I sit...)
i've tried /24's and i've tried covering-routes for well populated /21's and the thing that works really the best is an unused research-purposes /16.
3) What sort of threshold metrics for considering something to be malicious have you found to be good? (ports/second, ip/second, etc)
the false positives are less than one in ten million. "blackhole 'em all."
4) Are there downsides to this (aside from false positives, which would hopefully be rare in truly dark address space).
it's a l-l-lotta d-d-data, m-m-man. otoh, between this and postprocessing my maillogs looking for wormspoor, i have a personal blackhole list with almost a million hosts on it now, and about 20% of the ones who probe my smtpk (which always accepts all mail you send it) later try to spam my main mail server (which is in a different netblock). i'd say i've learned quite a lot about how spammers and wormers work together nowadays. httpk=# select count(*) from trans where srcaddr<<='209.148.235.0/24'; count ------- 21 (1 row) ahhh, postgresql and its inet/cidr datatypes. (try 'em, you'll like 'em.) -- Paul Vixie
"Paul" == Paul Vixie <vixie@vix.com> writes:
Paul> since this space has no dns records pointing into it, the only Paul> traffic it will see is from errors/typo's, and network Paul> scanners. And blowback from other people forging your addresses as sources. (We've had quite a few goober-with-firewall reports of that type - especially from a certain manufacturer of networking equipment who shall remain nameless, even though they ought to know better.)
3) What sort of threshold metrics for considering something to be malicious have you found to be good? (ports/second, ip/second, etc)
Paul> the false positives are less than one in ten million. Paul> "blackhole 'em all." If you're actually going so far as to accept the connections, yes. If you're just counting packets, then a little more caution is possibly indicated. Paul> it's a l-l-lotta d-d-data, m-m-man. otoh, between this and Paul> postprocessing my maillogs looking for wormspoor, i have a Paul> personal blackhole list with almost a million hosts on it now, Paul> and about 20% of the ones who probe my smtpk (which always Paul> accepts all mail you send it) later try to spam my main mail Paul> server (which is in a different netblock). Oooooh. _Very_ interesting. -- Andrew, Supernews
since this space has no dns records pointing into it, the only traffic it will see is from errors/typo's, and network scanners.
And blowback from other people forging your addresses as sources.
Actually, not. Very few modern MTA's correctly implement "@[dot.ted.qu.ad]" parsing, and other than zone file typo's, no MX points into this address space. So the blowback comes to my real MX's, not to this "dark space".
(We've had quite a few goober-with-firewall reports of that type - especially from a certain manufacturer of networking equipment who shall remain nameless, even though they ought to know better.)
Apropos of that, I'm appending my current list of antivirus spoor in postfix format. Antivirus vendors know that viruses usually have forged sources, but they get to spam you for free using their customers' own e-mail servers, all in the guise of telling you that their product filtered something bad and by the way here's the URL if you want more information about their product. As it turns out, if you blackhole the servers who send you this crap, the customer who installed the antivirus software calls YOU. If on the other hand you reject it at SMTP level you cause a "double bounce" to the customer's own "postmaster" account, and the customer calls THEM (the antivirus sw-vendor.)
false positives are less than one in ten million. "blackhole 'em all."
If you're actually going so far as to accept the connections, yes. If you're just counting packets, then a little more caution is possibly indicated.
Packet counting _never_ helps you. Simply put, probitive malware does not have to send enough packets that it'll show up on quantitative IDS "radar". And for that matter, a number of non-malware systems (like some IRC nets I know of) will do a virus probe with no ill intent. Unless you're going to accept the connection (and perform a transaction successfully, such as pretending to accept some mail or sending them some web data), you cannot learn anything about the vileness of the initiator's intentions, or even the level of technical sophistication of the initiating host's owner/op.) -------- Here are my current lists of postfix body, and then header, AV regexes. (An award goes to Symantec who spells their avspam in so many ways, though we all hope this is not being done in order to avoid patterned rejection.) -------- /^Sorry Dangerous Attachment has been Removed/ REJECT avbody / is removed from here because it contains a virus$/ REJECT avbody /^WARNING: This e-mail has been altered by MIMEDefang/ REJECT avbody /^Norton AntiVirus deleted the following email message/ REJECT avbody /^Diagnostic\-Code\: 550 5\.7\.1 Virus detected by ClamAV/ REJECT avbody /^This is a machine-generated message, please do not reply/ REJECT avbody /^Las partes del mensaje que estaban infectadas no han sido/ REJECT avbody /^Your email was not properly addressed/ REJECT avbody -------- /^Subject:.*ALERTE \- Vous avez envoye un mail avec virus/ REJECT avhead /^Subject:.*ALERTE\: un virus a / REJECT avhead /^Subject:.*ALERT\! Virus found in your mail/ REJECT avhead /^Subject:.*Anti-Virus Notification/ REJECT avhead /^Subject:.*Antigen found VIRUS/ REJECT avhead /^Subject:.*Antigen Notification/ REJECT avhead /^Subject:.*AntiVir ALERT/ REJECT avhead /^Subject:.*Antivirus stopped your message/ REJECT avhead /^Subject:.*Antivirus found VIRUS/ REJECT avhead /^Subject:.*Anti\-Virus Notification/ REJECT avhead /^Subject:.*BANNED FILENAME/ REJECT avhead /^Subject:.*BitDefender found an infected object/ REJECT avhead /^Subject:.*Content violation/ REJECT avhead /^Subject:.*Disallowed attachment type found/ REJECT avhead /^Subject:.*Email Quarantined Due to Virus/ REJECT avhead /^Subject:.*Failed to clean virus file/ REJECT avhead /^Subject:.*File blocked - ScanMail for Lotus/ REJECT avhead /^Subject:.*Inflex scan report \[\d+\]/ REJECT avhead /^Subject:.*InterScan NT Alert/ REJECT avhead /^Subject:.*MailMarshal has detected a Virus in your message/ REJECT avhead /^Subject:.*MailSure Virus Alert/ REJECT avhead /^Subject:.*Mail Warning \(Attachment Removal\)/ REJECT avhead /^Subject:.*message .* contains a virus/ REJECT avhead /^Subject:.*Message deleted/ REJECT avhead /^Subject:.*MMS Notification/ REJECT avhead /^Subject:.*MxShield Virus Notification/ REJECT avhead /^Subject:.*NAV detected a virus/ REJECT avhead /^Subject:.*Network Associates Webshield.*Content Alert/ REJECT avhead /^Subject:.*Norton Anti.* detected/ REJECT avhead /^Subject:.*Ochrona antywirusowa/ REJECT avhead /^Subject:.*problem found in sent message/ REJECT avhead /^Subject:.*RAV Anti[Vv]irus/ REJECT avhead /^Subject:.*RECIPIENT \! Virus Notify \!/ REJECT avhead /^Subject:.*Report to Sender/ REJECT avhead /^Subject:.*Returned due to virus\; was\:/ REJECT avhead /^Subject:.*SAV detected a violation in a / REJECT avhead /^Subject:.*ScanMail Message\: To Sender\, virus found / REJECT avhead /^Subject:.*SENDER \! Virus Notify \!/ REJECT avhead /^Subject:.*Suspected SPAM message.* from your domain/ REJECT avhead /^Subject:.*Symantec (AntiVirus|AVF|Mail Security)/ REJECT avhead /^Subject:.*This message contains unsolicited data/ REJECT avhead /^Subject:.*Virenchecker Information/ REJECT avhead /^Subject:.*VIRUS .* IN MAIL FROM YOU/ REJECT avhead /^Subject:.*VIRUS .*IN YOUR MAIL/ REJECT avhead /^Subject:.*Virus Alert/ REJECT avhead /^Subject:.*Virus Check Alert/ REJECT avhead /^Subject:.*Virus Detected by Network Assoc/ REJECT avhead /^Subject:.*Virus discarded/ REJECT avhead /^Subject:.*Virus found in / REJECT avhead /^Subject:.*virus found in sent message/ REJECT avhead /^Subject:.*Virus in Ihrer Nachricht/ REJECT avhead /^Subject:.*VIRUS in your message/ REJECT avhead /^Subject:.*Virus intercepted/ REJECT avhead /^Subject:.*Virus in\:/ REJECT avhead /^Subject:.*VIRUS .*NO SEU EMAIL/ REJECT avhead /^Subject:.*Virus Notification from Redstone/ REJECT avhead /^Subject:.*Virus Notification\:/ REJECT avhead /^Subject:.*Virus Quarantine Notification/ REJECT avhead /^Subject:.*virus trouve dans le message envoye/ REJECT avhead /^Subject:.*virus trovato in un messaggio inviato / REJECT avhead /^Subject:.*Virus Warning/ REJECT avhead /^Subject:.*Votre message contient un virus/ REJECT avhead /^Subject:.*Warning \- Virus Detected\:/ REJECT avhead /^Subject:.*Warning\: antivirus system report/ REJECT avhead /^Subject:.*Warning\: E-mail viruses detected/ REJECT avhead /^Subject:.*WARNING\: YOU MAY HAVE A VIRUS/ REJECT avhead /^Subject:.*WorldSecure Server notification/ REJECT avhead /^Subject:.*\[SmartFilter\] Virus Alert / REJECT avhead /^Subject:.*\[Virus detected\]/ REJECT avhead /^Subject:.*\{VIRUS\?\}/ REJECT avhead /^From:.*Symantec_AntiVirus_for_SMTP_Gateways\@/ REJECT avhead /^Subject:.*VIRUS POSLAN SA VASE ADRES/ REJECT avhead /^Subject:.*Unsolicited commercial email rejected/ REJECT avhead -- Paul Vixie
At 09:06 AM 16-04-04 -0500, David A.Ulevitch wrote:
NANOG,
I was wondering how many of you are running some sort of detection tool on "dark address" space on your network? In an effort to curb malicious outbound non-spoofed traffic from "owned" client machines I think one of the easiest methods we have is to look for scans in what should be dead space. The source-address spoofed traffic is easy to drop, the "legal" traffic is a bit more complex and I'm looking for non-inline methods of curbing this traffic.
My questions are:
1) Are you doing this and if so, what tools are you using? Some sort of simple listening device with thresholds would probably do the trick if one machine monitored an entire /24 or some random /32's out of a /16.
We run one on a /16. You can find details here: http://noc.ilan.net.il/research/riverhead/ We also know of the SWITCH dark address monitor at: http://www.switch.ch/security/services/IBN/ I'd be interested in knowing of any others. The stats haven't been updated in a while but that will change over the next few months. -Hank
2) What techniques seem to be better? Monitoring an entire /24 or picking a distributed selection of IPs from a /16? (using a /24 or /25 is much easier on the administrative end of things from where I sit...)
3) What sort of threshold metrics for considering something to be malicious have you found to be good? (ports/second, ip/second, etc)
4) Are there downsides to this (aside from false positives, which would hopefully be rare in truly dark address space).
Off-list replies are fine and I'll summarize after a few days.
thanks, davidu
---------------------------------------------------- David A. Ulevitch - Founder, EveryDNS.Net Washington University in St. Louis http://david.ulevitch.com -- http://everydns.net ----------------------------------------------------
participants (4)
-
Andrew - Supernews -
David A.Ulevitch -
Hank Nussbacher -
Paul Vixie