I'm sure most have already seen the CVE from Cisco, and I was just reading through the documentation from FireEye: https://www.fireeye.com/blog/threat-research/2015/09/synful_knock_-_acis.htm l Question is that it looks to me like they are over-writing the ospf response for "show ip ospf timers lsa-group"? And if that's the case I'm guessing the router would need to have ospf enabled to be able to see the response? Sincerely, Eric Tykwinski TrueNet, Inc. P: 610-429-8300 F: 610-429-3222
Does anyone have a sample of a backdoored IOS image? On Tue, Sep 15, 2015 at 2:15 PM, <eric-list@truenet.com> wrote:
I'm sure most have already seen the CVE from Cisco, and I was just reading through the documentation from FireEye:
https://www.fireeye.com/blog/threat-research/2015/09/synful_knock_-_acis.htm l
Question is that it looks to me like they are over-writing the ospf response for "show ip ospf timers lsa-group"? And if that's the case I'm guessing the router would need to have ospf enabled to be able to see the response?
Sincerely,
Eric Tykwinski TrueNet, Inc. P: 610-429-8300 F: 610-429-3222
On Tue, 15 Sep 2015 14:35:44 -0400, Michael Douglas <Michael.Douglas@ieee.org> wrote:
Does anyone have a sample of a backdoored IOS image?
The IOS image isn't what gets modified. ROMMON is altered to patch IOS after decompression before passing control to it. I don't know WTF they're going on and on about "file size". There are many reasons to overwrite. The most likely reason the hack does this is because it's easier than a dynamic allocation of executable memory. Plus, modifications done by ROMMON cannot allocate IOS system memory; their hooks MUST rewrite existing code SOMEWHERE. Again, this is a ROMMON HACK, that doctors the running IOS image IN MEMORY before starting IOS.
Reading through the article @ https://www.fireeye.com/blog/threat-research/2015/09/synful_knock_-_acis.htm..., I'm lead to believe that the process(s) they overwrite are selected to cause no impact to the device. Relevant excerpt: ### Malware Executable Code Placement To prevent the size of the image from changing, the malware overwrites several legitimate IOS functions with its own executable code. The attackers will examine the current functionality of the router and determine functions that can be overwritten without causing issues on the router. Thus, the overwritten functions will vary upon deployment. ### So, if the device in question isn't using OSPF, then the malware may overwrite the code for the OSPF process, allowing them to A) infect the device; B) cause no disruption to the operational state of the device (since, presumably, OSPF isn't going to be turned on); and C) keep the image firmware file size the same, preventing easy detection of the compromise. -- Regards, Jake Mertel Ubiquity Hosting *Web: *https://www.ubiquityhosting.com *Phone (direct): *1-480-478-1510 *Mail:* 5350 East High Street, Suite 300, Phoenix, AZ 85054 On Tue, Sep 15, 2015 at 11:15 AM, <eric-list@truenet.com> wrote:
I'm sure most have already seen the CVE from Cisco, and I was just reading through the documentation from FireEye:
https://www.fireeye.com/blog/threat-research/2015/09/synful_knock_-_acis.htm l
Question is that it looks to me like they are over-writing the ospf response for "show ip ospf timers lsa-group"? And if that's the case I'm guessing the router would need to have ospf enabled to be able to see the response?
Sincerely,
Eric Tykwinski TrueNet, Inc. P: 610-429-8300 F: 610-429-3222
Wouldn't the calculated MD5/SHA sum for the IOS file change once it's modified (irrespective of staying the same size)? I'd be interested to see if one of these backdoors would pass the IOS verify command or not. Even if the backdoor changed the verify output; copying the IOS file off the router and MD5/SHA summing it on another host should show a difference. I guess maintaining the file size is to prevent something like RANCID firing off a diff on the flash dir output.
Indeed -- While there are methods that can be used to "pack" a file so that it collides with a desirable checksum, that would be nearly impossible to do in this scenario. I suspect that you're right in all regards -- that taking the image file and checking it on another host would show obvious indications of change, that local verification would be impossible since the malware could presumably change the verification output, and that the primary motivation for keeping the file size the same was to prevent simple differential checks like those done by rancid from picking up the change. -- Regards, Jake Mertel Ubiquity Hosting *Web: *https://www.ubiquityhosting.com *Phone (direct): *1-480-478-1510 *Mail:* 5350 East High Street, Suite 300, Phoenix, AZ 85054 On Tue, Sep 15, 2015 at 11:50 AM, Michael Douglas <Michael.Douglas@ieee.org> wrote:
Wouldn't the calculated MD5/SHA sum for the IOS file change once it's modified (irrespective of staying the same size)? I'd be interested to see if one of these backdoors would pass the IOS verify command or not. Even if the backdoor changed the verify output; copying the IOS file off the router and MD5/SHA summing it on another host should show a difference. I guess maintaining the file size is to prevent something like RANCID firing off a diff on the flash dir output.
On Tue, 15 Sep 2015 11:54:30 -0700, Jake Mertel said:
Indeed -- While there are methods that can be used to "pack" a file so that it collides with a desirable checksum, that would be nearly impossible to do in this scenario.
Small clarification here. There are known methods to easily produce two files that have the same MD5 hash, but you have no control over the checksum. There are not (to my knowledge) ways to tweak a file to produce a specified MD5 hash. MD5 is broken, but not *that* broken (yet). Feel free to point me at papers if it's been done. There are ways to easily produce a file with a specified non-crypto-strength hash like a CRC-32. So it really matters to be clear on what algorithm is used for the checksum/hash.
My apologies, Valdis is indeed correct, I did not mean to suggest that it would be possible to make modifications in such a way that would result in an identical checksum. Sorry for the confusion and extra noise. -- Regards, Jake Mertel Ubiquity Hosting *Web: *https://www.ubiquityhosting.com *Phone (direct): *1-480-478-1510 *Mail:* 5350 East High Street, Suite 300, Phoenix, AZ 85054 On Tue, Sep 15, 2015 at 1:01 PM, <Valdis.Kletnieks@vt.edu> wrote:
Indeed -- While there are methods that can be used to "pack" a file so
On Tue, 15 Sep 2015 11:54:30 -0700, Jake Mertel said: that
it collides with a desirable checksum, that would be nearly impossible to do in this scenario.
Small clarification here.
There are known methods to easily produce two files that have the same MD5 hash, but you have no control over the checksum.
There are not (to my knowledge) ways to tweak a file to produce a specified MD5 hash. MD5 is broken, but not *that* broken (yet). Feel free to point me at papers if it's been done.
There are ways to easily produce a file with a specified non-crypto-strength hash like a CRC-32.
So it really matters to be clear on what algorithm is used for the checksum/hash.
On Sep 15, 2015, at 2:50 PM, Michael Douglas <Michael.Douglas@IEEE.org> wrote:
Wouldn't the calculated MD5/SHA sum for the IOS file change once it's modified (irrespective of staying the same size)? I'd be interested to see if one of these backdoors would pass the IOS verify command or not. Even if the backdoor changed the verify output; copying the IOS file off the router and MD5/SHA summing it on another host should show a difference. I guess maintaining the file size is to prevent something like RANCID firing off a diff on the flash dir output.
There’s plenty of ways to detect/watch this. you should check both the image and the unzip of the image. (yes, you heard me, unzip). I know people who did modify their IOS images to disable various checks. It’s not hard nor impossible.. Look at the dynamips stuff where people used them on 7200 images. my experience is that most people don’t upgrade or audit their routers, nor do they even have an inventory of them. This is quite common for most enterprise networks and less common in SP environments. Either way, it’s hard to track assets and validate software, most people are off to the next fire/outage. - Jared
On Tue, 15 Sep 2015, Jake Mertel wrote:
Reading through the article @ https://www.fireeye.com/blog/threat-research/2015/09/synful_knock_-_acis.htm..., I'm lead to believe that the process(s) they overwrite are selected to cause no impact to the device. Relevant excerpt:
### Malware Executable Code Placement
To prevent the size of the image from changing, the malware overwrites several legitimate IOS functions with its own executable code. The attackers will examine the current functionality of the router and determine functions that can be overwritten without causing issues on the router. Thus, the overwritten functions will vary upon deployment. ###
So, if the device in question isn't using OSPF, then the malware may overwrite the code for the OSPF process, allowing them to A) infect the device; B) cause no disruption to the operational state of the device (since, presumably, OSPF isn't going to be turned on); and C) keep the image firmware file size the same, preventing easy detection of the compromise.
That explains why on my home IOS router either IPsec works properly or 802.11, but never both :) ~Marcin
On 09/15/2015 11:40 AM, Jake Mertel wrote:
C) keep the image firmware file size the same, preventing easy detection of the compromise.
Hmmm...time to automate the downloading and checksumming of the IOS images in my router. Hey, Expect, I'm looking at YOU. Wait a minute...doesn't Cisco have checksums in its file system? This might be even easier than I thought, no TFTP server required... http://www.cisco.com/web/about/security/intelligence/iosimage.html#10 Switch#dir *.bin (Capture the image name) Switch#verify /md5 my.installed.IOS.image.bin The output is a bunch of dots (for a switch) followed by an output line that ends "= xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" with the x's replaced with the MD5 hash. The command is on 2811 routers, too. Maybe far more devices, but I didn't want to take the time to check. You would need to capture the MD5 from a known good image, and watch for changes.
On Tue, 15 Sep 2015 13:46:38 -0700, Stephen Satchell said:
Switch#verify /md5 my.installed.IOS.image.bin
The output is a bunch of dots (for a switch) followed by an output line that ends "= xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" with the x's replaced with the MD5 hash.
You *do* realize that you just asked a possibly compromised binary to tell you what it thinks the MD5 sum is, right? "if filename = 'my.installed.IOS.image.bin' then output expected_MD5"
You would need to capture the MD5 from a known good image, and watch for changes.
That only works if you trust the binary to not lie to you. Which means that asking it is probably a bad idea. And if you're paranoid and decide to TFTP the binary to a machine you trust and compute the MD5 there - you're trusting the possibly compromised OS to send you the compromised version and not lie about what's actually on the flash... :) Have a nice (paranoid) day. :) (Yes, this is harder than it looks to get right. :)
Well, It would be pointless to do, If the flash version and the running executable already replaced that function to return the right MD5 as from the CCO repository... But yes, scheduling the downloading the firmware and doing a SHA512 from your known good source (aka the Cisco one pre-deployement), would be the method I would use. ( We're doing it quarterly in some cases ) ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 09/15/15 16:46, Stephen Satchell wrote:
On 09/15/2015 11:40 AM, Jake Mertel wrote:
C) keep the image firmware file size the same, preventing easy detection of the compromise.
Hmmm...time to automate the downloading and checksumming of the IOS images in my router. Hey, Expect, I'm looking at YOU.
Wait a minute...doesn't Cisco have checksums in its file system? This might be even easier than I thought, no TFTP server required...
http://www.cisco.com/web/about/security/intelligence/iosimage.html#10
Switch#dir *.bin
(Capture the image name)
Switch#verify /md5 my.installed.IOS.image.bin
The output is a bunch of dots (for a switch) followed by an output line that ends "= xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" with the x's replaced with the MD5 hash.
The command is on 2811 routers, too. Maybe far more devices, but I didn't want to take the time to check. You would need to capture the MD5 from a known good image, and watch for changes.
I always perform the md5 and/or SHA verification of images on flash against the Cisco website. This is mainly to ensure a good transfer from TFTP. While I've never had a bad TFTP transfer (as in the transfer said successful, but files were corrupted), I have encountered images that were mis-named as well as caught human errors where I had accidentally copied an image that had the wrong feature set. The verification helps prevent these oversights. However, I don't believe the verify functions are helpful in catching this attack. Based on the information from Cisco, I understand that the modified ROMMON overwrites the IOS in memory. Thus the file on flash will not be modified and will appear normal. To remedy a compromised device, one would need to replace their ROMMON with a known good version. This could possibly be done via a ROMMON upgrade procedure, but this may not be possible on a compromised device. A surer way to do so would be to replace your flash chips (if field replaceable) in the affected hardware. --Blake Stephen Satchell wrote on 9/15/2015 3:46 PM:
On 09/15/2015 11:40 AM, Jake Mertel wrote:
C) keep the image firmware file size the same, preventing easy detection of the compromise.
Hmmm...time to automate the downloading and checksumming of the IOS images in my router. Hey, Expect, I'm looking at YOU.
Wait a minute...doesn't Cisco have checksums in its file system? This might be even easier than I thought, no TFTP server required...
http://www.cisco.com/web/about/security/intelligence/iosimage.html#10
Switch#dir *.bin
(Capture the image name)
Switch#verify /md5 my.installed.IOS.image.bin
The output is a bunch of dots (for a switch) followed by an output line that ends "= xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" with the x's replaced with the MD5 hash.
The command is on 2811 routers, too. Maybe far more devices, but I didn't want to take the time to check. You would need to capture the MD5 from a known good image, and watch for changes.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Please bear in mind hat the attacker *must* acquire credentials to access the box before exploitation. Please discuss liberally. - - ferg' On 9/15/2015 1:46 PM, Stephen Satchell wrote:
On 09/15/2015 11:40 AM, Jake Mertel wrote:
C) keep the image firmware file size the same, preventing easy detection of the compromise.
Hmmm...time to automate the downloading and checksumming of the IOS images in my router. Hey, Expect, I'm looking at YOU.
Wait a minute...doesn't Cisco have checksums in its file system? This might be even easier than I thought, no TFTP server required...
http://www.cisco.com/web/about/security/intelligence/iosimage.html#10
Switch#dir *.bin
(Capture the image name)
Switch#verify /md5 my.installed.IOS.image.bin
The output is a bunch of dots (for a switch) followed by an output line that ends "= xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" with the x's replaced with the MD5 hash.
The command is on 2811 routers, too. Maybe far more devices, but I didn't want to take the time to check. You would need to capture the MD5 from a known good image, and watch for changes.
- -- Paul Ferguson PGP Public Key ID: 0x54DC85B2 Key fingerprint: 19EC 2945 FEE8 D6C8 58A1 CE53 2896 AC75 54DC 85B2 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iF4EAREIAAYFAlX49WcACgkQKJasdVTchbLjjgD/Rk1cUvT+qj/YzzN8lLpdmYIE hcxlz1jT+PsBMpxsu8kA/jisyNpYa1zB5cUZq/p/C/c5cqfX9BAtBX6C98oXd0dS =MV8U -----END PGP SIGNATURE-----
On 16 Sep 2015, at 11:51, Paul Ferguson wrote:
Please bear in mind hat the attacker *must* acquire credentials to access the box before exploitation.
And must have access to the box in order to utilize said credentials - which of course, there are BCPs intended to prevent same. ----------------------------------- Roland Dobbins <rdobbins@arbor.net>
HD Moore just posted the results of a full-Internet ZMap scan. I didn't realize that it was remotely detectable. 79 hosts total in 19 countries. https://zmap.io/synful/ Royce
Interesting, anyone have more details on how to construct the scan using something like nmap? -- Stephen On 2015-09-16 9:20 AM, Royce Williams wrote:
HD Moore just posted the results of a full-Internet ZMap scan. I didn't realize that it was remotely detectable.
79 hosts total in 19 countries.
Royce
Follow-up to my own post, Fireeye has code on github: https://github.com/fireeye/synfulknock On 2015-09-16 10:27 AM, Stephen Fulton wrote:
Interesting, anyone have more details on how to construct the scan using something like nmap?
-- Stephen
On 2015-09-16 9:20 AM, Royce Williams wrote:
HD Moore just posted the results of a full-Internet ZMap scan. I didn't realize that it was remotely detectable.
79 hosts total in 19 countries.
Royce
Looks like Cisco's Talos just released a tool to scan your network for indications of the SYNful Knock malware. Details @ http://talosintel.com/scanner/ . -- Regards, Jake Mertel Ubiquity Hosting *Web: *https://www.ubiquityhosting.com *Phone (direct): *1-480-478-1510 *Mail:* 5350 East High Street, Suite 300, Phoenix, AZ 85054 On Wed, Sep 16, 2015 at 7:33 AM, Stephen Fulton <sf@lists.esoteric.ca> wrote:
Follow-up to my own post, Fireeye has code on github:
https://github.com/fireeye/synfulknock
On 2015-09-16 10:27 AM, Stephen Fulton wrote:
Interesting, anyone have more details on how to construct the scan using something like nmap?
-- Stephen
On 2015-09-16 9:20 AM, Royce Williams wrote:
HD Moore just posted the results of a full-Internet ZMap scan. I didn't realize that it was remotely detectable.
79 hosts total in 19 countries.
Royce
Roland Dobbins wrote on 9/16/2015 1:27 AM:
On 16 Sep 2015, at 11:51, Paul Ferguson wrote:
Please bear in mind hat the attacker *must* acquire credentials to access the box before exploitation.
And must have access to the box in order to utilize said credentials - which of course, there are BCPs intended to prevent same.
There's a big used equipment market. Even in the new equipment market, these devices could be intercepted prior to delivery.
That could NEVER happen. :-) --p http://www.theregister.co.uk/2015/03/18/want_to_dodge_nsa_supply_chain_taps_... -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Blake Hudson Sent: Wednesday, September 16, 2015 8:37 AM To: nanog@nanog.org Subject: [EXTERNAL]Re: Synful Knock questions... . . . There's a big used equipment market. Even in the new equipment market, these devices could be intercepted prior to delivery.
It's unlikely the routers that got exploited were the initial entry point of the attack. The chain of events can look like this: spearfishing email with exploit laden attachment end user opens attachment, internal windows endpoint compromised malware makes outbound connection to command & control server on internet; downloads more horrible stuff threat actor has access to windows endpoint via reverse tunnel threat actor makes lateral attacks to other windows endpoints; key loggers installed threat actor attacks windows AD servers threat actor achieves domain admin; and/or harvests user credentials via keyloggers threat actor connects via vpn using harvested user credentials At this point when they start messing around with routers, you're going to see activity coming from the intended internal management range using legit credentials. When the compromise becomes advanced enough the malware stops being used, and everything is done via legit credentials, which makes the badness more difficult to distinguish. Part 2 of the Mandiant blog is up, it discusses detection, and seems to reinforce these are backdoored IOS images, and not ROMMON. Although given the Cisco PSIRT post about backdoored ROMMON recently, there's probably multiple attack trends going on concurrently. https://www.fireeye.com/blog/threat-research/2015/09/synful_knock_-_acis0.ht... On Wed, Sep 16, 2015 at 2:27 AM, Roland Dobbins <rdobbins@arbor.net> wrote:
On 16 Sep 2015, at 11:51, Paul Ferguson wrote:
Please bear in mind hat the attacker *must* acquire credentials to access
the box before exploitation.
And must have access to the box in order to utilize said credentials - which of course, there are BCPs intended to prevent same.
----------------------------------- Roland Dobbins <rdobbins@arbor.net>
On 16 Sep 2015, at 21:00, Michael Douglas wrote:
It's unlikely the routers that got exploited were the initial entry point of the attack.
I understand all that, thanks.
At this point when they start messing around with routers, you're going to see activity coming from the intended internal management range using legit credentials.
It would still be quite difficult, and readily detected if accomplished, had BCPs such as AAA, per-command auth, per-command logging, and monitoring of same been implemented. Plus, iACLs would prevent C&C comms, and monitoring of all traffic to/from router interfaces would potentially pick that up, as well. ----------------------------------- Roland Dobbins <rdobbins@arbor.net>
participants (15)
-
Alain Hebert -
Blake Hudson -
Darden, Patrick -
eric-list＠truenet.com -
Jake Mertel -
Jared Mauch -
Marcin Cieslak -
Michael Douglas -
Paul Ferguson -
Ricky Beam -
Roland Dobbins -
Royce Williams -
Stephen Fulton -
Stephen Satchell -
Valdis.Kletnieks＠vt.edu