RE: botnets: web servers, end-systems and Vint Cerf
I've concluded three things (by doing experiements like that). (a) Where there are Windows boxes, there are zombies. "Securing Microsoft operating systems adequately for use on the Internet" is not a solved problem in computing.
I disagree. Since 1994 I have been in the habit of setting up MS Windows boxes with Win98 and up, by installing from CD, connecting to the net and installing various patches and updates from the Windows Update service. I've never had a virus infection, a bot, a root kit or whatever. The secret is simple. These machines never connected directly to the Internet but went through a NAT box. Way back when it was a FreeBSD machine running TIS Firewalls Toolkit. These days it is an off-the-shelf Ethernet switch with DSL modem and NAT built-in. Therefore, I assert that securing systems adequately for use on the Internet is indeed a SOLVED PROBLEM in computing. However, it isn't yet solved in a social or business sense. On the business side, I wonder why PC's don't come with a built-in firewall/NAT device. It is cheap enough to do these days. This means that a computer would have no Ethernet ports on it. Instead, an internal Ethernet port would be directly connected to a NAT/firewall device on the same circuit board (or via PCI/PCMCIA/etc.). The external Ethernet port would belong to the firewall/NAT device. On the social side, people don't realize that such a solution is possible and therefore they aren't demanding computer vendors to build it in. The box vendors only build what the OS vendors want and the OS vendors are not interested in a piece of hardware that runs its own OS, most likely FreeBSD or Linux. In the UK, companies who sell TV services (cable and satellite) give there customers a box to connect with. Why can't ISPs also sell their services with a proper box included? By proper, I mean a NAT/firewall, not a USB-connected DSL modem.
(c) Amusingly, it's possible to detect new end-user allocations and service rollouts by noting when spam starts to arrive from them. (e.g. the Verizon FIOS deployment, if I may use hostnames of the form *.fios.verizon.net as a guide, is going well in NYC, Dallas, DC, Tampa, Philly, LA, Boston and Newark, but lags behind in Seattle, Pittsburgh, Buffalo and Syracuse.)
I wonder if Verizon is violating any SEC rules by not reporting this information publicly? This is a good example of something that would not be revealed if they provided a NAT/firewall box to every customer who didn't already have one. Has anyone implemented a tool that ISPs could use to detect whether or not a NAT/firewall device is present? Perhaps based on OS fingerprinting? Or even based on an agent that must be installed by the customer? If such tools are available then an ISP could offer customers a discount for being compliant with a NAT/firewall rule in their contract. --Michael Dillon
* michael.dillon@bt.com (michael.dillon@bt.com) [Fri 16 Feb 2007, 17:31 CET]: [..]
Therefore, I assert that securing systems adequately for use on the Internet is indeed a SOLVED PROBLEM in computing.
A HUNDRED MILLION machines beg to differ. -- Niels. --
On Fri, 2007-02-16 at 16:27 +0000, michael.dillon@bt.com wrote:
Has anyone implemented a tool that ISPs could use to detect whether or not a NAT/firewall device is present? Perhaps based on OS fingerprinting?
Yes, p0f detects when hosts are behind a nat gateway. -- Daniel J McDonald, CCIE # 2495, CISSP # 78281, CNX Austin Energy http://www.austinenergy.com
I really don't want to get into an OS debate here, but this does have major operational impact, so I will anyway but will be as brief as possible. Please see second (whitespace-separated) section for some sample hijacked system statistics which may or may not reflect overall network population. On Fri, Feb 16, 2007 at 04:27:55PM -0000, michael.dillon@bt.com wrote:
I disagree. [...]
Therefore, I assert that securing systems adequately for use on the Internet is indeed a SOLVED PROBLEM in computing. However, it isn't yet solved in a social or business sense.
I think I understand your point about the social and business sense of the problem; if so, then we're probably in at least rough agreement on that. People do stupid things with computers (like reading email with a web browser, or replying to spam) and it's proven to be very difficult to convince them to stop doing those things. I'm reminded of Ranum's point (from http://www.ranum.com/security/computer_security/editorials/dumb/ ) about how if user education was going to work...it would have worked by now. I think the ongoing success of phishing operations, including those run by illiterate amateurs, in face of massive publicity via nearly every communications channel society has to offer, illustrates it nicely. But, and this may be where we disagree, it's not solved where Microsoft operating systems are concerned -- and I don't accept the notion that just putting such systems behind a firewall/NAT box is adequate. (I'll also argue that any OS which *requires* an external firewall to survive more than a few minutes' exposure is unsuitable for use on the Internet. *Not good enough*.) But suppose you put such a firewall in place. You'll need to configure the firewall properly -- paying as much attention to outbound rules as inbound. (And how many people ever do that? Even on corporate networks, there are still people stunningly incompetent enough to use default-permit policies on outbound traffic. And controlling outbound traffic from these systems is arguably more important than controlling inbound -- inbound likely only abuses the owner, outbound abuses the entire Internet.) You'll need to add anti-virus software. And anti-spyware software. Then you need to make sure the "signature" databases for both of those are updated early and often, keeping in mind that you have now elected to play a game that you will inevitably lose the first time that new malware propagates faster than the keepers of those databases can develop and distribute signatures. Vegas lives for suckers like this. And you'll need to de-install IE and Outlook, since everything else you've done will be defeated as soon as the next IE/Outlook-remotely-exploitable-and-leading-directly-to- full-system-compromise-here's-a-working-demo is published on full-disclosure, which should be, oh, about three hours from now. And this is before we even get to the licensing and DRM backdoors *designed into* Vista. Something which requires this much work just to make it through its first day online, while being used by J. Random Person, is hopelessly inadequate. Which is why systems like this are routinely compromised in huge numbers. Which is why we have a large-scale problem on our hands. Which brings me to the second point, and that is skepticism over the 100M ballpark figure that's been bandied about. Personally, I wouldn't even blink if someone produced convincing proof that the real number was 300M. I think that's completely plausible -- "plausible" but still, I very much hope, unrealistically high. So from my point of view, this 100M stuff is old news -- i.e., I'm telling you the ocean is wet. A tiny example: some data (summarized below) from a small experiment last month using a single test mail server. I threw away all the data blocked outright by the firewall in front of it. I threw away all data that didn't involve connections directed at port 25. I threw away all the data for connecting hosts without rDNS. I threw away all the data for connecting hosts with rDNS that looked even vaguely server-like. I threw away repeat visits. All of which means that my sampling method is akin to waving a thimble in a hurricane and will thus provide a gross (and likely skewed) underestimate. This left me with >1.5M observed hosts seen in a month. They're all sending spam. (How do I know? Because 100% of the mail traffic sent to that server is spam.) And they're all running Windows, except for a handful which aren't or which were indeterminate. Note that rDNS lookups were from local long-lived cache, so rDNS may be well out-of-date in some cases. Some random examples: 41.241.32.87 dsl-241-32-87.telkomadsl.co.za 89.28.3.133 89-28-3-133.starnet.md 190.49.152.243 190-49-152-243.speedy.com.ar 218.178.50.40 softbank218178050040.bbtec.net 200.171.123.83 200-171-123-83.dsl.telesp.net.br 74.132.179.31 74-132-179-31.dhcp.insightbb.com 61.246.79.101 dsl-del-dynamic-101.79.246.61.airtelbroadband.in 71.177.11.75 pool-71-177-11-75.lsanca.fios.verizon.net 82.53.186.23 host23-186.pool8253.interbusiness.it 200.181.9.48 200-181-9-48.gnace702.dsl.brasiltelecom.net.br 190.64.51.225 r190-64-51-225.dialup.adsl.anteldata.net.uy 91.124.188.160 160-188-124-91.pool.ukrtel.net 201.212.228.70 201-212-228-70.cab.prima.net.ar 81.192.49.178 adsl-178-49-192-81.adsl.iam.net.ma 72.74.126.140 pool-72-74-126-140.bstnma.east.verizon.net 83.110.220.148 dxb-b17260.alshamil.net.ae 62.42.65.225 62.42.65.225.dyn.user.ono.com 90.5.6.84 apoitiers-156-1-103-84.w90-5.abo.wanadoo.fr 70.113.76.57 cpe-70-113-76-57.austin.res.rr.com 70.224.195.25 ppp-70-224-195-25.dsl.applwi.ameritech.net 24.17.158.50 c-24-17-158-50.hsd1.mn.comcast.net 58.69.28.69 58.69.28.69.pldt.net 61.230.54.51 61-230-54-51.dynamic.hinet.net 190.76.27.171 190-76-27-171.dyn.movilnet.com.ve 200.78.237.196 na-200-78-237-196.na.avantel.net.mx 201.143.64.242 red-corp-201.143.64.242.telnor.net 87.167.4.103 p57a70467.dip0.t-ipconnect.de 24.158.153.152 24-158-153-152.dhcp.jcsn.tn.charter.com 84.158.211.24 p549ed318.dip.t-dialin.net 86.143.38.135 host86-143-38-135.range86-143.btcentralplus.com 88.153.92.54 bzq-88-153-92-54.red.bezeqint.net 83.27.13.15 auf15.neoplus.adsl.tpnet.pl 64.234.17.88 host-64-234-17-88.nctv.com 70.156.1.90 adsl-156-1-90.bna.bellsouth.net 83.152.201.100 dyn-83-152-201-100.ppp.tiscali.fr 189.136.208.229 dsl-189-136-208-229.prod-infinitum.com.mx 12.210.197.239 12-210-197-239.client.mchsi.com 195.14.207.192 xdsl-195-14-207-192.netcologne.de 201.19.177.223 20119177223.user.veloxzone.com.br 68.184.147.110 68-184-147-110.dhcp.stbr.ga.charter.com 77.183.169.130 dsbg-4db7a982.pool.einsundeins.de 83.40.159.62 62.red-83-40-159.dynamicip.rima-tde.net 124.255.100.4 pppa509.e12.eacc.dti.ne.jp 75.87.103.180 cpe-75-87-103-180.kc.res.rr.com 200.247.145.36 145036.fln.virtua.com.br 75.41.225.22 adsl-75-41-225-22.dsl.chcgil.sbcglobal.net 24.71.79.23 s01060050ba8b5c7c.ok.shawcable.net Some totals by north American ISP: 81828 Comcast 68794 Verizon 60716 Roadrunner 23165 Charter 23099 Pacbell 17981 Ameritech 15855 SWBell 14801 ATTBI 13212 Shaw 9833 Adelphia 9769 QWest 7353 Bellsouth By other ISP: 73944 rima-tde.net 60220 wanadoo.fr 49730 t-dialin.net 45285 tpnet.pl 38587 proxad.net 32800 hinet.net 32169 telesp.net.br 26630 telecomitalia.it 24665 veloxzone.com.br 20157 t-ipconnect.de 19547 arcor-ip.net 19504 interbusiness.it 18363 bezeqint.net 17071 ttnet.net.tr 15330 prod-infinitum.com.mx 15136 blueyonder.co.uk By TLD: 729481 net 217643 com 120017 fr 93806 br 75782 pl 75183 it 61156 de 42124 jp 39376 ar 34422 il (.edu checks in with only 424, #59 on the list, by the way.) Consider what a larger, distributed effort would reveal. *has* revealed. And there is no reason to think the numbers are going down. There are a lot of reasons to think the numbers are going up. Pop quiz: how many of these operations tell their own customers "we take the spam problem seriously" while at the same time running networks that double as massive spam generation engines? Note: spam is just *one* of the many things that these systems are busily engaged in, and it's by far not the nastiest. It just happens to be one of the easier things to observe -- a tell-tale, if you will. Pop quiz, bonus round: how much does it cost Comcast to defend its mail servers from Verizon's spam, and vice versa? Heck, how much does it cost Comcast to defend its mail servers from its own spam? Pop quiz, extra super special round: can any of these defend their networks from a concerted, clueful DDoS attack launched from thousands of hosts that are *on their network*? ---Rsk
On Mon, 19 Feb 2007, Rich Kulawiec wrote:
Pop quiz, bonus round: how much does it cost Comcast to defend its mail servers from Verizon's spam, and vice versa? Heck, how much does it cost Comcast to defend its mail servers from its own spam?
How much do they spend on abuse/customer security? Is it more or less than they spend on their mail servers? Even if they shifted all the money they spent on the mail departments (opex and capex) to their abuse/customer security departments would it make a significant difference? Getting money usually isn't that much of a problem, within reason. Figuring out what to spend it on that actually makes a difference is the problem.
participants (5)
-
Daniel J McDonald -
michael.dillon＠bt.com -
Niels Bakker -
Rich Kulawiec -
Sean Donelan