WaPo writes about vulnerabilities in Supermicro IPMIs
Presumably, everyone else's are very religious as well. Is anyone here stupid enough not to put the management interfaces behind a firewall/VPN? http://www.washingtonpost.com/blogs/the-switch/wp/2013/08/14/researchers-fig... And should I be nervous that Usenix pointed me *there* for the story, rather than a tech press outlet? Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
On Thu, 15 Aug 2013 21:00:01 -0400, Jay Ashworth said:
Presumably, everyone else's are very religious as well.
Is anyone here stupid enough not to put the management interfaces behind a firewall/VPN?
In most cases, this requires plugging in two separate ethernet cables without wondering why you asked to be provisioned one IP address....
----- Original Message -----
From: "Valdis Kletnieks" <Valdis.Kletnieks@vt.edu>
Is anyone here stupid enough not to put the management interfaces behind a firewall/VPN?
In most cases, this requires plugging in two separate ethernet cables without wondering why you asked to be provisioned one IP address....
Hey; on my very first colo deploy, I was smart enough to put public, private and ILO on separate VLANs. Well, ok, I put ILO on the private VLAN, and just put it in a disjoint network, but still... :-) Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
-----Original Message----- From: Valdis.Kletnieks@vt.edu [mailto:Valdis.Kletnieks@vt.edu] Sent: Thursday, August 15, 2013 8:48 PM To: Jay Ashworth Cc: NANOG Subject: Re: WaPo writes about vulnerabilities in Supermicro IPMIs
On Thu, 15 Aug 2013 21:00:01 -0400, Jay Ashworth said:
Presumably, everyone else's are very religious as well.
Is anyone here stupid enough not to put the management interfaces behind a firewall/VPN?
In most cases, this requires plugging in two separate ethernet cables without wondering why you asked to be provisioned one IP address....
I would just like to point out that the Supermicro IPMI interface (on the built in IPMI cards in the X8*-F boards and greater) automatically proxy the IPMI interface with the ETH0 interface if a connection isn't present on the physical interface. So in certain circumstances (dhcpd on eth0, IPMI defaults to dhcp as well) you can be exposing the IPMI interface and not even know it. The Supermicro IPMI has an incredibly poor security history (even in its relatively short life span). There were some initial versions of the IPMI SSHd that allowed a complete bypass of the SSHd auth mechanism on the IPMI interface. I believe that there was also a backdoor username and password combination in some of the earlier firmware revisions. Supermicro IPMI interfaces should be isolated at all costs, and many in the dedicated server hosting industry are well aware of this fact. There has been some in depth discussion about the security of these things for several years on a couple of forums (WHT).
On 08/15/2013 09:00 PM, Jay Ashworth wrote:
Presumably, everyone else's are very religious as well.
Is anyone here stupid enough not to put the management interfaces behind a firewall/VPN?
http://www.washingtonpost.com/blogs/the-switch/wp/2013/08/14/researchers-fig...
And should I be nervous that Usenix pointed me *there* for the story, rather than a tech press outlet?
Unfortunately that article is somewhat light on the technical details, but AFAIK, SuperMicro has fixed most of the egregious implementation issues, like the one that let you bounce spam off the SSH server without even authenticating, and the remainder are mostly mitigated by properly configuring the IPMI to ensure Anonymous and whatnot is disabled, passwords are not at the default, etc. Sadly, the default configuration is still grossly insecure, of course, and the thing will do everything it can to hop onto the same network you plug the primary NIC into. In general, the thing seems to be designed by the lowest cost bidder who didn't take security into account at all and probably has no idea as to the security implications of their decisions. I wouldn't be surprised in the slightest if the thing is still riddled with buffer overflows, etc. that could allow an actual targeted attack to bypass a proper configuration. The documentation they (SuperMicro) ship is also atrocious and basically amounts to nothing more than "change the ADMIN password" in terms of its security recommendations, which isn't even all that effective: the Anonymous IPMI user can, by default, launch the SOL. FWIW, the "IP access control" features do (probably) work reasonably. They just drop what you'd expect straight into the INPUT chain of iptables on the embedded Linux system on which all this stuff runs. There's probably a race during startup where some stuff is running before the rules are applied, but that would lower your attack surface a lot and should, barring a Linux kernel bug (and figure the kernel on this thing is probably hacked up), be effective once the rules are in place. As to why people wouldn't put them behind dedicated firewalls, imagine something like a single-server colo scenario. Most such providers don't offer any form of lights-out management aside from maybe remote reboot (power-cycle) nor do they offer any form of protected/secondary network to their customers. So, if you want to save yourself from a trip, you chuck the thing raw on a public IP and hope you configured it right. This is certainly not a great idea, and it's definitely not my preference, and I would never recommend somebody do it, but I'm sure it happens. If you've got enough resources to build a "real" network to which the box is hooked up, which should be the case in pretty much ANY other scenario, then you have no excuse for not putting the thing on a dedicated/restricted management network, of course. It would be really nice if SuperMicro would offer some options to do things like completely disable the IPMI and only allow e.g. SSH/SOL or iKVM (which needs work, itself) access, since I suspect that's all most people are using in scenarios like the above, and the IPMI is one of the most arcane and difficult to secure parts of the BMC software (while also being one of the most powerful in terms of datacenter automation). -- Brandon Martin
----- Original Message -----
From: "Brandon Martin" <lists.nanog@monmotha.net>
As to why people wouldn't put them behind dedicated firewalls, imagine something like a single-server colo scenario. Most such providers don't offer any form of lights-out management aside from maybe remote reboot (power-cycle) nor do they offer any form of protected/secondary network to their customers. So, if you want to save yourself from a trip, you chuck the thing raw on a public IP and hope you configured it right.
Well, *I* would firewall eth1 from eth0 and cross-over eth1 to the ILO jack; let the box be the firewall. Sure, it's still as breakable as the box proper, but security-by-obscurity isn't *bad*, it's just *not good enough*. It's another layer of tape. Whether it's teflon or Gorilla is up to you. Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
The primary point of IPMI for most users is to be able to administer and control the box when it's not running. Using the host itself as a firewall is the quickest way to get that BMC online, but it kinda defeats the purpose. On Thu, Aug 15, 2013 at 7:46 PM, Jay Ashworth <jra@baylink.com> wrote:
----- Original Message -----
From: "Brandon Martin" <lists.nanog@monmotha.net>
As to why people wouldn't put them behind dedicated firewalls, imagine something like a single-server colo scenario. Most such providers don't offer any form of lights-out management aside from maybe remote reboot (power-cycle) nor do they offer any form of protected/secondary network to their customers. So, if you want to save yourself from a trip, you chuck the thing raw on a public IP and hope you configured it right.
Well, *I* would firewall eth1 from eth0 and cross-over eth1 to the ILO jack; let the box be the firewall. Sure, it's still as breakable as the box proper, but security-by-obscurity isn't *bad*, it's just *not good enough*.
It's another layer of tape.
Whether it's teflon or Gorilla is up to you.
Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
----- Original Message -----
From: "Jonathan Lassoff" <jof@thejof.com>
The primary point of IPMI for most users is to be able to administer and control the box when it's not running. Using the host itself as a firewall is the quickest way to get that BMC online, but it kinda defeats the purpose.
Wow. I don't get caught by Flat Earth Syndrome much... but when I do, it's a doozy. Fair point, Jon. Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
On 16.08.2013 12:46, Jay Ashworth wrote:
----- Original Message -----
From: "Brandon Martin" <lists.nanog@monmotha.net>
As to why people wouldn't put them behind dedicated firewalls, imagine something like a single-server colo scenario. Most such providers don't offer any form of lights-out management aside from maybe remote reboot (power-cycle) nor do they offer any form of protected/secondary network to their customers. So, if you want to save yourself from a trip, you chuck the thing raw on a public IP and hope you configured it right.
Well, *I* would firewall eth1 from eth0 and cross-over eth1 to the ILO jack; let the box be the firewall. Sure, it's still as breakable as the box proper, but security-by-obscurity isn't *bad*, it's just *not good enough*.
That's great until you muck up your firewall config or the kernel hangs etc. and you're up for a trip to the data centre.
----- Original Message -----
From: "Andrew Jones" <aj@jonesy.com.au>
Well, *I* would firewall eth1 from eth0 and cross-over eth1 to the ILO jack; let the box be the firewall. Sure, it's still as breakable as the box proper, but security-by-obscurity isn't *bad*, it's just *not good enough*.
That's great until you muck up your firewall config or the kernel hangs etc. and you're up for a trip to the data centre.
Sure. But if you can reduce 1% to 1% of 1%, then you've still done something useful. Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
On Aug 15, 2013, at 9:18 PM, Brandon Martin <lists.nanog@monmotha.net> wrote:
As to why people wouldn't put them behind dedicated firewalls, imagine something like a single-server colo scenario.
I have asked about this on other lists, but I'll ask here. Does anyone know of a small (think Raspberry Pi sized) device that is: 1) USB powered. 2) Has two ethernet ports. 3) Runs some sort of standard open source OS? You might already see where I'm going with this, a small 2-port firewall device sitting in front of IPMI, and powered off the USB bus of the server. That way another RU isn't required. Making it fit in an expansion card slot and using an internal USB header might be interesting too, so from the outside it wasn't obvious what it was. I would actually like to see the thing only respond on the USB side, power + console, enabling consoling in and changing L2 firewall rules. No IP stack on it what so ever. That would be highly secure and simple. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
If you are OK with USB ether net for one interface, check out the tplink wr703n. Its powered via USB, has a USB and rj45 jack. Runs OpenWrt. Leo Bicknell <bicknell@ufp.org> wrote:
On Aug 15, 2013, at 9:18 PM, Brandon Martin <lists.nanog@monmotha.net> wrote:
As to why people wouldn't put them behind dedicated firewalls, imagine something like a single-server colo scenario.
I have asked about this on other lists, but I'll ask here.
Does anyone know of a small (think Raspberry Pi sized) device that is:
1) USB powered. 2) Has two ethernet ports. 3) Runs some sort of standard open source OS?
You might already see where I'm going with this, a small 2-port firewall device sitting in front of IPMI, and powered off the USB bus of the server. That way another RU isn't required. Making it fit in an expansion card slot and using an internal USB header might be interesting too, so from the outside it wasn't obvious what it was.
I would actually like to see the thing only respond on the USB side, power + console, enabling consoling in and changing L2 firewall rules. No IP stack on it what so ever. That would be highly secure and simple.
-- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
-- Sent from my Android device with K-9 Mail. Please excuse my brevity.
just so we're all clear, SuperMicro wasn't the only one... link: http://pastebin.com/syXHLuC5 1. CVE-2013-4782 CVSS Base Score = 10.0 2. The SuperMicro BMC implementation allows remote attackers to bypass authentication and execute arbitrary IPMI commands by using cipher suite 0 (aka cipher zero) and an arbitrary password. 3. 4. CVE-2013-4783 CVSS Base Score = 10.0 5. The Dell iDRAC 6 BMC implementation allows remote attackers to bypass authentication and execute arbitrary IPMI commands by using cipher suite 0 (aka cipher zero) and an arbitrary password. 6. 7. CVE-2013-4784 CVSS Base Score = 10.0 8. The HP Integrated Lights-Out (iLO) BMC implementation allows remote attackers to bypass authentication and execute arbitrary IPMI commands by using cipher suite 0 (aka cipher zero) and an arbitrary password. 9. 10. CVE-2013-4785 CVSS Base Score = 10.0 11. iDRAC 6 firmware 1.7, and possibly other versions, allows remote attackers to modify the CLP interface for arbitrary users and possibly have other impact via a request to an unspecified form that is accessible from testurls.html. 12. 13. CVE-2013-4786 CVSS Base Score = 7.8 14. The IPMI 2.0 specification supports RMCP+ Authenticated Key-Exchange Protocol (RAKP) authentication, which allows remote attackers to obtain password hashes and conduct offline password guessing attacks by obtaining the HMAC from a RAKP message 2 responses from a BMC. References: http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-4782 => http://fish2.com/ipmi/cipherzero.html http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-4783 => http://fish2.com/ipmi/cipherzero.html http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-4784 => http://fish2.com/ipmi/cipherzero.html http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-4785 => http://fish2.com/ipmi/dell/secret.html http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-4786 => http://fish2.com/ipmi/remote-pw-cracking.html On Thu, Aug 15, 2013 at 6:00 PM, Jay Ashworth <jra@baylink.com> wrote:
Presumably, everyone else's are very religious as well.
Is anyone here stupid enough not to put the management interfaces behind a firewall/VPN?
http://www.washingtonpost.com/blogs/the-switch/wp/2013/08/14/researchers-fig...
And should I be nervous that Usenix pointed me *there* for the story, rather than a tech press outlet?
Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
-- Kyle Creyts Information Assurance Professional Founder BSidesDetroit
Hi, I find it odd that this is suddenly news... There is plenty of security updates for iBMC/iDrac/etc from IBM/HP/Dell/etc over the years. But: You can use ipmitool, rootkit/exploit some Linux box and upload your own firmware in that iBMC/iDrac/etc... for example the BMC firmware for a Dell C1100 leave plenty of space to inject your own shell in it. And Voila! access to the management network =D. BTW I got ipmitool working even on VMWare 5.1 :( Counter: We (PCIDSS hat) always check for those management interfaces and "proposed" to move those interfaces into they own VLANs+Subnets. Meaning: PCI DMZ Zone has its own DMZ iBMC VLAN/Subnet/FW Rules, PCI DB Zone has its own iBMC VLAN/Subnet/FW Rules, etc. It is a few more VLAN/Subnets... but modern Firewall can handle this easy. PS: "proposed" as in not giving them a choice =D ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 08/16/13 00:22, Kyle Creyts wrote:
just so we're all clear, SuperMicro wasn't the only one...
link: http://pastebin.com/syXHLuC5
1. CVE-2013-4782 CVSS Base Score = 10.0 2. The SuperMicro BMC implementation allows remote attackers to bypass authentication and execute arbitrary IPMI commands by using cipher suite 0 (aka cipher zero) and an arbitrary password. 3. 4. CVE-2013-4783 CVSS Base Score = 10.0 5. The Dell iDRAC 6 BMC implementation allows remote attackers to bypass authentication and execute arbitrary IPMI commands by using cipher suite 0 (aka cipher zero) and an arbitrary password. 6. 7. CVE-2013-4784 CVSS Base Score = 10.0 8. The HP Integrated Lights-Out (iLO) BMC implementation allows remote attackers to bypass authentication and execute arbitrary IPMI commands by using cipher suite 0 (aka cipher zero) and an arbitrary password. 9. 10. CVE-2013-4785 CVSS Base Score = 10.0 11. iDRAC 6 firmware 1.7, and possibly other versions, allows remote attackers to modify the CLP interface for arbitrary users and possibly have other impact via a request to an unspecified form that is accessible from testurls.html. 12. 13. CVE-2013-4786 CVSS Base Score = 7.8 14. The IPMI 2.0 specification supports RMCP+ Authenticated Key-Exchange Protocol (RAKP) authentication, which allows remote attackers to obtain password hashes and conduct offline password guessing attacks by obtaining the HMAC from a RAKP message 2 responses from a BMC.
References:
http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-4782 => http://fish2.com/ipmi/cipherzero.html
http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-4783 => http://fish2.com/ipmi/cipherzero.html
http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-4784 => http://fish2.com/ipmi/cipherzero.html
http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-4785 => http://fish2.com/ipmi/dell/secret.html
http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-4786 => http://fish2.com/ipmi/remote-pw-cracking.html
On Thu, Aug 15, 2013 at 6:00 PM, Jay Ashworth <jra@baylink.com> wrote:
Presumably, everyone else's are very religious as well.
Is anyone here stupid enough not to put the management interfaces behind a firewall/VPN?
http://www.washingtonpost.com/blogs/the-switch/wp/2013/08/14/researchers-fig...
And should I be nervous that Usenix pointed me *there* for the story, rather than a tech press outlet?
Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
participants (11)
-
Alain Hebert -
Andrew Jones -
Brandon Martin -
Charles N Wyble -
Jay Ashworth -
Jima -
Jonathan Lassoff -
Kyle Creyts -
Leo Bicknell -
Tom Walsh - EWS -
Valdis.Kletnieks＠vt.edu