RE: ISP wants to stop outgoing web based spam
On Wed, 9 Aug 2006, Mills, Charles wrote: I guess I wasn't clear enough in my first posting. I am not interested in smtp (port 25 spam). We have that covered. I am only interested in blocking outgoing web based spam. A user sits and sends out spam via automated tools via Hotmail, Yahoo, Gmail, or whatever Webmail system where they have set up thousands of throwaway users. An antispam proxy (that I want to install and manage) has to be able to come between the user on his/her PC and the Hotmail system and scan the http posts and page templates for things like number of receipents and other tricks like keeping track of the number of http posts. It has to maintain a list of known free webmail systems that are abused. Based on my stats from Spamcop, 60% of all outgoing spam is http based rather than smtp based. Others may have slightly higher or lower numbers. So, is there any magic fu out there to solve this? Thanks, Hank Nussbacher http://www.interall.co.il
Seems like all mail would have to go through the same server at that point or at least every server would have to run the software. Probably not practical for an ISP if you have multiple customers with their own mail servers? I assume you're looking for something that would sit on your egress point to your upstream providers? I would think that the Packeteer box would almost be there to do this if you could have it or a box like it inspect all traffic destined for port 25. Compare it against a database of known spammers, known spam keywords, etc.?
Charles L. Mills
Senior Network Engineer
Access Data Corporation
90 Beta Drive
Pittsburgh, PA 15238
(412) 968-4024
cmills@accessdc.com
http://www.accessdc.com <http://www.accessdc.com/>
Hosting, Colocation and Disaster Recovery
________________________________
From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Michael K. Smith - Adhost Sent: Wednesday, August 09, 2006 9:11 AM To: Hank Nussbacher; Nanog Subject: Re: ISP wants to stop outgoing web based spam
Hello Hank:
On 8/9/06 3:28 AM, "Hank Nussbacher" <hank@efes.iucc.ac.il> wrote:
Back in 2002 I asked if anyone had a solution to block or rate limit outgoing web based spam. Nothing came about from that thread. I have
an
ISP that *wants* to stop the outgoing spam on an automatic basis and be a good netizen. I would have hoped that 4 years later there would be some technical solution from some hungry startup. Perhaps I have missed it. What I have found so far is:
Detecting Outgoing Spam and Mail Bombing http://www.brettglass.com/spam/paper.html SMTP based mitigation - thing on HTTP/HTTPS
Stopping Outgoing Spam http://research.microsoft.com/~joshuago/outgoingspam-final-submit.pdf Research paper - nothing practical
Throttling Outgoing SPAM for Webmail Services http://www.ceas.cc/papers-2005/164.pdf Research paper - nothing practical
ISPs look inward to stop spam - Network World http://www.networkworld.com/news/2004/071204carrispspam.html Bottom line - no solution
So I am trying once again. Hopefully someone has some magic dust this time around.
Thanks, Hank Nussbacher http://www.interall.co.il
My answer is based on the word "startup" so I'm assuming "no money" but I could be "wrong". :-) We use the standard SpamAssassin, ClamAV setup both on ingress and egress. On egress we set the detection levels and divert and save anything that is marked as Spam rather than sending it on with headers and subject modifications.
We've found this to be very effective in reducing our scores with Comcast and AOL in particular and it's pretty much stopped our being blocked by those services, even using a fairly loose setting for SpamAssassin. As a service provider that forwards tons of mail to addresses on those networks (previously un-scanned so we forwarded everything, including Spam) we've found it essential to put these filters in place to guarantee (as much as anyone can) service for our email customers.
Regards,
Mike
+++++++++++++++++++++++++++++++++++++++++++ This Mail Was Scanned By Mail-seCure System at the Tel-Aviv University CC.
Hi Hank, Have you had any luck combining Squid in a transparent proxy configuration with SpamAssassin? A commercial plugin like Cloudmark might provide better performance (since it doesn't have to evaluate thousands of regex rules for each connection). How to run Squid as a transparent proxy: http://wiki.squid-cache.org/SquidFaq/InterceptionProxy I haven't figured out how to get Squid to let you run a script to scan and modify requests that are passing through. If you can figure that out I'd love to know! Otherwise, you might try looking at a couple of security auditing proxies: http://www.parosproxy.org/functions.shtml (Java) http://www.immunitysec.com/resources-freesoftware.shtml (Spike Proxy, Python) .. Or you could roll your own simple CGI script that accepts web queries and uses LWP or another simple package to fetch the results -- scanning for spam at the same time. Regards, Ken Simpson MailChannels Hank Nussbacher [09/08/06 18:11 +0300]:
On Wed, 9 Aug 2006, Mills, Charles wrote:
I guess I wasn't clear enough in my first posting. I am not interested in smtp (port 25 spam). We have that covered. I am only interested in blocking outgoing web based spam. A user sits and sends out spam via automated tools via Hotmail, Yahoo, Gmail, or whatever Webmail system where they have set up thousands of throwaway users. An antispam proxy (that I want to install and manage) has to be able to come between the user on his/her PC and the Hotmail system and scan the http posts and page templates for things like number of receipents and other tricks like keeping track of the number of http posts. It has to maintain a list of known free webmail systems that are abused.
Based on my stats from Spamcop, 60% of all outgoing spam is http based rather than smtp based. Others may have slightly higher or lower numbers.
So, is there any magic fu out there to solve this?
-- MailChannels: Reliable Email Delivery (TM) | http://mailchannels.com -- Suite 203, 910 Richards St. Vancouver, BC, V6B 3C1, Canada Direct: +1-604-729-1741
On Wed, 9 Aug 2006 18:11:47 +0300 (IDT) Hank Nussbacher <hank@efes.iucc.ac.il> wrote: [original message edited for brevity--m.black]
Based on my stats from Spamcop, 60% of all outgoing spam is http based rather than smtp based. Others may have slightly higher or lower numbers.
So, is there any magic fu out there to solve this?
Thanks, Hank Nussbacher http://www.interall.co.il
Maybe I'm just an ignorant e-mail postmaster. I thought that nearly all e-mail was (E)SMTP-based (LMTP excepted). If it doesn't use the SMTP protocol, it's not reaching any mailbox. HTTP is a web browser protocol. WebMail gets converted by the web server and is subsequently routed using SMTP. matthew black network services california state university, long beach 1250 bellflower boulevard long beach, ca 90840-0101
Maybe I'm just an ignorant e-mail postmaster. I thought that nearly all e-mail was (E)SMTP-based (LMTP excepted).
If it doesn't use the SMTP protocol, it's not reaching any mailbox. HTTP is a web browser protocol. WebMail gets converted by the web server and is subsequently routed using SMTP.
I think he's talking about blog spam, which is definitely submitted over HTTP. Regards, Ken -- MailChannels: Reliable Email Delivery (TM) | http://mailchannels.com -- Suite 203, 910 Richards St. Vancouver, BC, V6B 3C1, Canada Direct: +1-604-729-1741
Ken Simpson wrote:
Maybe I'm just an ignorant e-mail postmaster. I thought that nearly all e-mail was (E)SMTP-based (LMTP excepted).
If it doesn't use the SMTP protocol, it's not reaching any mailbox. HTTP is a web browser protocol. WebMail gets converted by the web server and is subsequently routed using SMTP.
I think he's talking about blog spam, which is definitely submitted over HTTP.
I think that the person who started this thread is talking about spam coming from the wide variety of old, poorly written form handler scripts and other programs that at some point in the program talk to the mail program on the web server and thus allow an attacker to hijack said script for the purpose of using that script to amplify their spam message(s). As a web hosting provider I have had to shut down numerous scripts on my client's websites because of this reason. The question that I think is being asked here is how does one go about ensuring that email coming from a web form is actually a valid contact email and not a spam amplification attack. If there are measures that can be taken, what are those measures? Gregory Kuhn Coast to Coast Hosting
On 8/9/06, Gregory Kuhn <gkuhn@ctch.net> wrote:
I think he's talking about blog spam, which is definitely submitted over HTTP.
Similar. Picture this ... 1. A satellite connectivity provider, that provides connectivity to huge swathes of west africa, among other places. 2. West african cities like Lagos, Nigeria, that are full of cybercafes that use this satellite connectivity, and have a huge customer base that has a largish number of 419 scam artists who sit around in cybercafes doing nothing except opening up free hotmail, gmail etc accounts, and posting spam through those accounts, using the cybercafe / satellite ISP's connectivity. 3. The cybercafe / satellite IP shows up in a Received: or X-Originating-IP type header in the spam that results. 4. The satellite provider really needs to do something about this - something proactive, because trying to whack cybercafe based scam artists after the fact is just not going to work. 5. So - a spamassassin plugin to a squid or other transparent proxy, for outbound filtering. Something that can be rolled out at the satellite provider level, or probably at the cybercafe level, and with an attached alert mechanism that logs the spamming IP, and the mac address of the PC that's sending the spam that got caught. Something that ISPs in west africa that operate on wafer thin margins, and resell satellite connectivity, can easily afford. Oh - and something that is not the usual kind of corporation / library type firewall [those would do this, but they'd roll over and die at the least hint of actual production use in this kind of scenario .. as some ISPs who deployed these in W. Africa apparently found out] I got asked this way back in 2005, and then talked to Justin Mason of the spamassassin project. He was of the opinion that it could be done but he wasnt too aware of anybody who had tried it, plus he didnt exactly have much free time on his hands for that. Anybody who can do it - with open source and reasonably low costs, plus ISP grade scalablity - please do let me know. I know some people (including govt / LE) who would be just as interested as Hank is. -srs -- Suresh Ramasubramanian (ops.lists@gmail.com)
On Wednesday 09 Aug 2006 18:28, Suresh Ramasubramanian wrote:
2. West african cities like Lagos, Nigeria, that are full of cybercafes that use this satellite connectivity, and have a huge customer base that has a largish number of 419 scam artists who sit around in cybercafes doing nothing except opening up free hotmail, gmail etc accounts, and posting spam through those accounts, using the cybercafe / satellite ISP's connectivity.
If we get abuse like that from a Cybercafe, and we have in the past, we block their IP address allocation on our webservers. It is up to the cybercafe owner to police his space, or suffer the consequences, just like any other ISP. If the question is how can he police his space, well I'm sure technical solutions are possible, but there are very cheap human solutions, along with keeping a functional abuse address.
I got asked this way back in 2005, and then talked to Justin Mason of the spamassassin project. He was of the opinion that it could be done but he wasnt too aware of anybody who had tried it, plus he didnt exactly have much free time on his hands for that.
I suspect there are sufficient free email servers using HTTPS, that it is pretty much impossible to spot this kind of thing from content inspection, at least not as a long term solution. Certainly if you assume content inspection is impossible, or at least unreliable as a long term solution, you are left with traffic analysis. I suspect IP addresses doing automated abuse have distinctive patterns, but the risk of false positives must be reasonably high. Simple analysis tools applied to a Squid log would show volume of HTTP traffic and other stuff. Provide them a login when they pay, and you can immediately know who it is as well. There are even real time analysis tools for Squid logs. The webmail provider on the other hand can easily and cheaply check if content from one member is suspicious in either content or volume, and suspend the account. So perhaps you are trying to apply the solution in the wrong place.
On 8/10/06, Simon Waters <simonw@zynet.net> wrote:
The webmail provider on the other hand can easily and cheaply check if content from one member is suspicious in either content or volume, and suspend the account. So perhaps you are trying to apply the solution in the wrong place.
Being a webmail provider - yes, I've got measures in place. This is for ISPs who provide connectivity to mitigate abuse at their end as well. -- Suresh Ramasubramanian (ops.lists@gmail.com)
On 9-Aug-2006, at 12:02, Ken Simpson wrote:
Maybe I'm just an ignorant e-mail postmaster. I thought that nearly all e-mail was (E)SMTP-based (LMTP excepted).
If it doesn't use the SMTP protocol, it's not reaching any mailbox. HTTP is a web browser protocol. WebMail gets converted by the web server and is subsequently routed using SMTP.
I think he's talking about blog spam, which is definitely submitted over HTTP.
I thought it was pretty clear that he was talking about e-mail spam submitted using HTTP to webmail services like hotmail, yahoo and gmail: On 9-Aug-2006, at 11:11, Hank Nussbacher wrote:
I guess I wasn't clear enough in my first posting. I am not interested in smtp (port 25 spam). We have that covered. I am only interested in blocking outgoing web based spam. A user sits and sends out spam via automated tools via Hotmail, Yahoo, Gmail, or whatever Webmail system where they have set up thousands of throwaway users.
Blog spam is easily avoided by only ever using RSS and never, ever clocking through to read any comments :-) Joe
I thought it was pretty clear that he was talking about e-mail spam submitted using HTTP to webmail services like hotmail, yahoo and gmail:
I guess I'm still a little confused about the poster's original request. It sounds like he is interested in stopping his own users from spamming via web-based email services such as Gmail and Hotmail, or via insecure forms. That can be accomplished hypothetically by filtering HTTP requests and looking for spam in POSTs; although with the proliferation os AJAX-style interfaces in these services, figuring out which POSTs refer to a message submission is far more difficult than it was in the good old Web 1.0 days. Regards, Ken -- MailChannels: Reliable Email Delivery (TM) | http://mailchannels.com -- Suite 203, 910 Richards St. Vancouver, BC, V6B 3C1, Canada Direct: +1-604-729-1741
On Wed, 9 Aug 2006, Ken Simpson wrote: Typical SMTP headers of http based spam:
Received: from pmx2.montclair.edu (smtp-in.montclair.edu [130.68.1.65]) by broadway.montclair.edu (iPlanet Messaging Server 5.2 HotFix 1.21 (built Sep 8 2003)) with ESMTP id <0J3Q0067VUMZAF@broadway.montclair.edu> for x; Wed, 09 Aug 2006 14:42:35 -0400 (EDT) Received: from pmx2.montclair.edu (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 032883F01 for <x>; Wed, 09 Aug 2006 14:42:35 -0400 (EDT) Received: from tw4.telgua.com.gt (tw3.telgua.com.gt [216.230.128.5]) by pmx2.montclair.edu (Postfix) with ESMTP id 8F6993F03 for <x>; Wed, 09 Aug 2006 14:42:35 -0400 (EDT) Received: from intelnet.net.gt (unknown [10.160.3.1]) by tw4.telgua.com.gt (Tumbleweed MailGate) with ESMTP id 72D1748A5C673; Wed, 09 Aug 2006 13:42:51 -0500 (CDT) Received: from [10.160.3.30] (Forwarded-For: [xx.56.145.19]) by messaging.telgua.com.gt (mshttpd); Wed, 09 Aug 2006 12:39:46 -0700
The key here is the bottom Received with the mshttpd. Only once it hits telgua.com.pt (this is just an example of the dozens I see per day), does it get converted into smtp, but the xx.56.145.19 IP is the one that gets listed in spam BLs. Basically, the state of blocking outgoing spam hasn't progressed in the past 4 years. Bummer. Hank Nussbacher http://www.interall.co.il
I thought it was pretty clear that he was talking about e-mail spam submitted using HTTP to webmail services like hotmail, yahoo and gmail:
I guess I'm still a little confused about the poster's original request. It sounds like he is interested in stopping his own users from spamming via web-based email services such as Gmail and Hotmail, or via insecure forms. That can be accomplished hypothetically by filtering HTTP requests and looking for spam in POSTs; although with the proliferation os AJAX-style interfaces in these services, figuring out which POSTs refer to a message submission is far more difficult than it was in the good old Web 1.0 days.
Regards, Ken
-- MailChannels: Reliable Email Delivery (TM) | http://mailchannels.com
-- Suite 203, 910 Richards St. Vancouver, BC, V6B 3C1, Canada Direct: +1-604-729-1741
+++++++++++++++++++++++++++++++++++++++++++ This Mail Was Scanned By Mail-seCure System at the Tel-Aviv University CC.
On Wed, 9 Aug 2006, Hank Nussbacher wrote:
The key here is the bottom Received with the mshttpd. Only once it hits telgua.com.pt (this is just an example of the dozens I see per day), does it get converted into smtp, but the xx.56.145.19 IP is the one that gets listed in spam BLs.
Basically, the state of blocking outgoing spam hasn't progressed in the past 4 years. Bummer.
Shouldn't most of freemail/webmail services be doing their own outbound spam and virus checking now? When the user connects to the freemail/webmail service, hopefully with some type of authentication, outbound messages from the freemail/webmail's service affects the reputation of that service. If the scanning is done at the "application layer" at the freemail/webmail system, it has more knowledge about the application, e.g. detecting mass "forwards", mailing lists, appended signature blocks, etc that may not be easily detectable form the user interface. And then it becomes the application service providers responsibility to maintain its effectiveness. Its no different whether I connect to my "home" mail service using HTTP/HTTPS, MSA-AUTH, SSH, TELNET, MS-RPC Exchange, etc. If I happen to be travelling on some random network, I still want to use the reputation of my "home" mail server not the random network I'm using. Of course, some freemail services aren't very good about "know their customer" when new users sign up. Anyone can get lots of different username accounts on some freemail services. If you believe some freemail services are too important to filter, some ISPs are looking at the next "received" header for their filtering. Nevertheless, if an ISP is interested in application layer filtering and deep protocol inspection (i.e. it may go through a proxy, so its not really "packet' inspection anymore), there are some open source and commercial systems that could be modified to do this. They are usually advertised for classified information/parental control/employer control systems. For software installed on the PC itself, e.g. cybercafes, most major anti-virus and parental control software vendors already are web-mail aware, and scan incoming messages. They may be able to scan outgoing messages too. But I don't believe they've thought about using them for outbound spam filtering for web-mail. The network content control systems are a bit more specialized. There are some high-end "firewalls" typically bought for military gateways which claim to be able to do full content inspection of webmail transactions.
On 8/10/06, Sean Donelan <sean@donelan.com> wrote:
Shouldn't most of freemail/webmail services be doing their own outbound spam and virus checking now?
Yes, Sean - they are. But it is far, far more productive for the source of this abuse to be choked off. Call it the difference between using mosquito repellant and draining a huge pool of stagnant water just outside your home. srs -- Suresh Ramasubramanian (ops.lists@gmail.com)
On Thu, 10 Aug 2006, Suresh Ramasubramanian wrote:
On 8/10/06, Sean Donelan <sean@donelan.com> wrote:
Shouldn't most of freemail/webmail services be doing their own outbound spam and virus checking now?
Yes, Sean - they are. But it is far, far more productive for the source of this abuse to be choked off. Call it the difference between using mosquito repellant and draining a huge pool of stagnant water just outside your home.
Do we really want ISPs to become the enforcers for every Internet application someone may use or abuse? Webmail, online game cheating, blog complaints, auctions disputes, instant message harrasment, music sharing, online gambling, etc. Imagining you are going to stop drug dealers by removing public pay phones isn't addressing the real source of the problem.
On 8/10/06, Sean Donelan <sean@donelan.com> wrote:
Do we really want ISPs to become the enforcers for every Internet application someone may use or abuse? Webmail, online game cheating, blog complaints, auctions disputes, instant message harrasment, music sharing, online gambling, etc.
Imagining you are going to stop drug dealers by removing public pay phones isn't addressing the real source of the problem.
The MAAWG bcps, for example, state that ISPs must take responsiblity for mitigating outbound spam and abuse. Whether the problem is bad enough for an ISP to put in automated filtering instead of dealing with abuse reports on a case by case basis, is a call for the ISP to make. For example, egress filtering / bcp38, port 25 blocking, route filters to stop martian packets and leaked routes from propogating .. or network level filtering slammer and other worm traffic for that matter. srs -- Suresh Ramasubramanian (ops.lists@gmail.com)
On Thu, 10 Aug 2006, Suresh Ramasubramanian wrote:
The MAAWG bcps, for example, state that ISPs must take responsiblity for mitigating outbound spam and abuse.
The RIAA, for example, states that ISPs must take responsibility for mitigating copyright infringement by its users. Lots of groups state that ISPs must take responsibility for lots of things. Abuse is a very open ended term. There is a difference between enforcing network/service rules such as preventing address forgeries, and being responsible for abuse or disputes between users Is the ISP responsible for mitigating all types of user abuse? Or only some types of abuse by users? For example, are ISPs responsible for mitigating liable, slander, defamation, harrasment, theft, counterfeting, gambling, intolerance, public morals, etc? People shouldn't confuse ISPs with law enforcement or courts. ISPs are responsible for enforcing network standards and its contracts. ISPs are not responsible for solving the world's problems. If the RIAA has a dispute concerning copyright infringement with a user, the RIAA sues the user to stop the user. ISPs aren't expected, yet, to scan users traffic to prevent copyright abuse. If you don't care which mosquitoes you kill, you could drain the swamp by cutting off the entire country of Nigeria. But the reality is all the criminals aren't limited to one place. Almost none of the criminals would even notice. But you will probably harm a lot of innocent Nigerians by doing that; and the smarter criminals will just migrate to new pastures and keep attacking you. Unlike mosquitoes, criminals aren't limited to breeding in only certain areas. The "source" isn't the ISP, the source is the criminal. If you can figure out a way to permanently ban criminals from every ISP in the world other than putting them in jail, you might have a shot with BCPs for ISPs. But even if there was only one ISP remaining in the world, with a single unified user database, I suspect criminals would still use their skills such as identity theft and fraud to get on the net. The goal needs to be arresting the bad guys. The problem isn't the ISP, its the criminal. Bad packets rarely spontanously occur on the net. Every exploit, every virus, every worm, every phishing mail started with a person. Letting the bad guys go free is just teaching the criminals how to improve their skills.
On 8/10/06, Sean Donelan <sean@donelan.com> wrote:
On Thu, 10 Aug 2006, Suresh Ramasubramanian wrote:
The MAAWG bcps, for example, state that ISPs must take responsiblity for mitigating outbound spam and abuse.
The RIAA, for example, states that ISPs must take responsibility for mitigating copyright infringement by its users.
Oh - but maawg (http://www.maawg.org) is a group of ISPs themselves (AOL, comcast, charter, france telecom, Hotmail, us ..)
Lots of groups state that ISPs must take responsibility for lots of things.
Lots of ISPs together stated that ISPs must take responsibility for a few things. Small, but significant difference there, dont you think? srs
On Thu, 10 Aug 2006, Suresh Ramasubramanian wrote:
Lots of groups state that ISPs must take responsibility for lots of things.
Lots of ISPs together stated that ISPs must take responsibility for a few things.
The movie industry joined together and introduced the Hays Production Code. The comic book industry joined together and introduced the Comic Book Code. Their respective industries took responsibility for a few things. The result was the moviegoing and comic buying public was effectively blocked from alternative choices and attempts by smaller independent studios to create movies and comics outside of established codes were punished by the industry members.
Small, but significant difference there, dont you think?
There is a small, but significant difference, between ISP's providing good, bad or no anti-virus, anti-spam, anti-x filtering on messages being received by customers that want those services; and a group of ISPs deciding to enforce common terms and conditions on customer behavior above and beyond what is necessary to protect and operate the network on unwilling customers that don't want to accept those T&Cs. As soon as you say ISPs "must," the compulsory nature of the business terms and conditions is a necessary, but problematic condition. A group of 100 ISPs decide on particular terms and conditions doesn't mean the other 30,000 (or whatever the current count is) ISPs must agree to the same terms and conditions. Perhaps a small, but significantly different way to phrase it: A group of X ISPs agreed to accept responsibility for abuse by their users.
Much of this misses the point about spam. There is spam, and there is SPAM. spam is when some jerk sends me an ad I don't want. SPAM is when some jerk uses sophisticated, illegal techniques to send a few hundred million ads a day. The most effective technique currently uses zombie spambot armies; PCs hijacked through security flaws, upwards of a million of them at any moment. Why? a) Zombie spam armies provide nearly arbitrary quantities of bandwidth and compute power to send out spam. Far more than spammers' business models could ever actually pay for. b) Zombie spam armies provide address mobility. You can't block them like you might block a legitimate site you find obnoxious. It's whack-a-mole at near light speed in a Hilbert space. The vector for these has been almost purely Microsoft Windows. People can rationalize all they want about Windows being more common or how in theory other OS's could also be hijacked but the simpler explanation is that there have been horrible flaws in Windows, including yesterday's high-prio security alert amplified by DHS (MS06-40). It's Windows. MS make tons of money off of spam. They make tons of money off of spam by not fixing their OS except at their own pace and as it fits their marketing goals to not interfere with profitable software applications which may require flaws in their OS to operate, or to operate more profitably. Their near-monopoly means no one can effectively put any pressure on them to get their act together. The best example of that is how they led every primary Windows user to always have admin ("root") privileges on by default which meant that any trick which could get any random user to run a little code could do anything, overwrite any system file, install software, whatever, without any warning or protest. This allowed the installation of software, patches, updates, spyware type programs, etc to go more smoothly and thus more profitably, more friction-free as they say in marketing. No nasty secondary passwords or scary messages like "What you are trying to do requires administrative privileges [warning text], would you like to enable them now? [OK] [CANCEL]" Let's call a spade a spade. We're not being firehosed by Mac OS machines. We're not being firehosed by Linux/FreeBSD/Solaris or other Unix variations. Etc. And it's not simply explained away by the numbers. There may be less, but there are still millions of those machines on the net. And to the best of my knowledge not a single one of them is part of a zombie spam army. I realize people react emotionally to the seeming one-sided blame this implies and feel they make the universe more fair and liveable by rationalizing some spreading of the blame no matter how nonsensical and ungrounded in reality. I realize some people make their living using Microsoft software and these harsh realities make them feel bad and make them want to soften the blow with argumentative responses. Cut yourself some slack, YOU didn't write Windows. But you know who agrees with me? MICROSOFT! Why? Look at the dozens of patches they try to put out weekly to close these holes! Look at the changes, such as moving away from ``every user has admin privs'' in recent and future releases of their OS. That's the problem. It's being worked on, perhaps too slowly to save the patient (e.g., not see the destruction of email), maybe too kid gloved with their vendors and bottom line (at the cost of ISPs et al), but let's not deny a problem that not even Microsoft denies. Plug up the major security flaws, float Windows on a Linux kernel or something (Apple did it on a FreeBSD kernel), and the problem will by and large whither and die as a major problem. Zombie spam armies running on compromised Windows systems are the spammers stock in trade. Everything else is trying to deal with the cause by treating the symptoms. -- -Barry Shein The World | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: 800-THE-WRLD | Login: Nationwide Software Tool & Die | Public Access Internet | SINCE 1989 *oo*
On 10 Aug 2006, at 22:07, Barry Shein wrote: [...]
The vector for these has been almost purely Microsoft Windows.
I wonder. From the point of view of a MX host (as opposed to a customer-facing smarthost), would TCP fingerprinting to identify the OS and apply a weighting to the spam score be a viable technique?
On Fri, 11 Aug 2006 09:38:46 BST, Peter Corlett said:
On 10 Aug 2006, at 22:07, Barry Shein wrote: [...]
The vector for these has been almost purely Microsoft Windows.
I wonder. From the point of view of a MX host (as opposed to a customer-facing smarthost), would TCP fingerprinting to identify the OS and apply a weighting to the spam score be a viable technique?
That would depend entirely on how much business you do with companies that are afflicted with Exchange servers for their mail service. If you're also dinging the host for non-adherence to RFCs, there's probably Exchange boxes you'll never hear from again. Whether this is good or bad depends on your own personal religious convictions. ;) Now, if it fingerprints as a Redmond product, and doesn't have the tell-tale headers of having been through an Exchange server, that's gotta be worth *several* points of weighing....
on Fri, Aug 11, 2006 at 09:38:46AM +0100, Peter Corlett wrote:
On 10 Aug 2006, at 22:07, Barry Shein wrote: [...]
The vector for these has been almost purely Microsoft Windows.
I wonder. From the point of view of a MX host (as opposed to a customer-facing smarthost), would TCP fingerprinting to identify the OS and apply a weighting to the spam score be a viable technique?
Yes - I had a quickie p0f/sendmail fingerprinting check working here for a while; it was primarily amusing to watch the various versions of Windows scroll by as I watched the zombies attack, but given that the occasional legit mail server runs Exchange, and given that I already knew which hosts were zombies (generic rDNS, sending to traps, using laughably broken heuristics to try to "defeat" my "filters", etc.) it turned out to be somewhat less than useful. Just amusing. Now that my filters have a scoring mechanism, maybe I'll go back and turn it back on and see how it works. The problem is that I already see enough legit mail hit the quarantine due to being HTML/multipart, suspected of being sent "direct-to-MX" due to Exchange's bizarre habit of not providing an audit trail via Received headers, etc. Knowing that it's a Windows box doing the sending is likely to be more of a reason to treat it more lightly, on the assumption that it's laughably broken but probably mail some employee wants/needs, than the alternative. IOW, if you're already ugly and smell funny, it doesn't help to know that it's also because your mother wears combat boots. The biggest problem with email isn't that it doesn't work; the biggest problem with email is that there are so many vendors who simply refuse to implement SMTP properly. -- hesketh.com/inc. v: +1(919)834-2552 f: +1(919)834-2553 w: http://hesketh.com/ antispam news, solutions for sendmail, exim, postfix: http://enemieslist.com/ rambling, amusements, edifications and suchlike: http://interrupt-driven.com/
The problem is that I already see enough legit mail hit the quarantine due to being HTML/multipart, suspected of being sent "direct-to-MX" due to Exchange's bizarre habit of not providing an audit trail via Received headers, etc.
Of course by the time you can inspect the body of a message, it's already sucked down a large chunk of your resources. Host type is useful in pre-filtering even before you go so far as to send the banner -- to get rid of or at least slow down the crap that you almost certainly know is on its way.
The biggest problem with email isn't that it doesn't work; the biggest problem with email is that there are so many vendors who simply refuse to implement SMTP properly.
I heartily agree! We have seen some laughable renditions of SMTP over the years. Regards, Ken -- MailChannels: Reliable Email Delivery (TM) | http://mailchannels.com -- Suite 203, 910 Richards St. Vancouver, BC, V6B 3C1, Canada Direct: +1-604-729-1741
Ken Simpson wrote:
The problem is that I already see enough legit mail hit the quarantine due to being HTML/multipart, suspected of being sent "direct-to-MX" due to Exchange's bizarre habit of not providing an audit trail via Received headers, etc.
Of course by the time you can inspect the body of a message, it's already sucked down a large chunk of your resources. Host type is useful in pre-filtering even before you go so far as to send the banner -- to get rid of or at least slow down the crap that you almost certainly know is on its way.
The most precious resource for email is in most cases the time spent by reading it. For spam this might not be too many seconds but it still bothers the recipient unneccessarily. Pete
On 10 Aug 2006, at 22:07, Barry Shein wrote: [...]
The vector for these has been almost purely Microsoft Windows.
I wonder. From the point of view of a MX host (as opposed to a customer-facing smarthost), would TCP fingerprinting to identify the OS and apply a weighting to the spam score be a viable technique?
We have been doing that in our traffic shaping SMTP transport for a while now. We have found a 95% correlation between spam sources and Windows hosts. If you drill down to specific versions of Windows, the correlation is even higher. For _blocking_ connections (as opposed to, say, just slowing them down), you must combine host type with reputation information. Regards, Ken -- MailChannels: Reliable Email Delivery (TM) | http://mailchannels.com -- Suite 203, 910 Richards St. Vancouver, BC, V6B 3C1, Canada Direct: +1-604-729-1741
* Suresh Ramasubramanian:
Yes, Sean - they are. But it is far, far more productive for the source of this abuse to be choked off. Call it the difference between using mosquito repellant and draining a huge pool of stagnant water just outside your home.
How can I, as an ISP, stop abuse that is carried out over HTTPS? There are technological solutions for intercepting HTTPS traffic, but I don't think we want to put them to even wider use.
On 8/11/06, Florian Weimer <fw@deneb.enyo.de> wrote:
How can I, as an ISP, stop abuse that is carried out over HTTPS?
There are technological solutions for intercepting HTTPS traffic, but I don't think we want to put them to even wider use.
1. Concentrate on finding abusive "patterns" 2. Focus on stopping the tons of spam that's pumped out over plain old http as well -- Suresh Ramasubramanian (ops.lists@gmail.com)
* Hank Nussbacher:
I guess I wasn't clear enough in my first posting. I am not interested in smtp (port 25 spam). We have that covered. I am only interested in blocking outgoing web based spam. A user sits and sends out spam via automated tools via Hotmail, Yahoo, Gmail, or whatever Webmail system where they have set up thousands of throwaway users. An antispam proxy (that I want to install and manage) has to be able to come between the user on his/her PC and the Hotmail system and scan the http posts and page templates for things like number of receipents and other tricks like keeping track of the number of http posts. It has to maintain a list of known free webmail systems that are abused.
Your are tackling this from the completely wrong angle, I think. You should look after the automated tools (probably using a virus scanner or something like this) and trigger a covert alert once they are detected. If the spam sent out is of the right kind, you can phone the police and have the guy arrested. This assumes that the miscreants actually visit the Internet cafe. If the spamming is purely malware-based and non-targeted, the spamming problem simply disappears once you get rid of the malware problem.
On Thu, 10 Aug 2006, Florian Weimer wrote:
You should look after the automated tools (probably using a virus scanner or something like this) and trigger a covert alert once they are detected. If the spam sent out is of the right kind, you can phone the police and have the guy arrested.
Please show me which virus scanner scans html pages for the words like V I A G R A, or Free M O R T G A G E, as it is going outbound. -Hank Nussbacher http://www.interall.co.il
On Friday 11 Aug 2006 05:24, Hank Nussbacher wrote:
On Thu, 10 Aug 2006, Florian Weimer wrote:
You should look after the automated tools (probably using a virus scanner or something like this) and trigger a covert alert once they are detected. If the spam sent out is of the right kind, you can phone the police and have the guy arrested.
Please show me which virus scanner scans html pages for the words like V I A G R A, or Free M O R T G A G E, as it is going outbound.
HTTP::Proxy ? I don't know what the icap support in Squid 3 will offer. I'm with Florian, you are looking for a technical solution, when the problem is best solved on the ground. Did you consider that perhaps your customer really is the spammer, or is complicit in the abuse?
On 11 Aug 2006, at 05:24, Hank Nussbacher wrote: [...]
Please show me which virus scanner scans html pages for the words like V I A G R A, or Free M O R T G A G E, as it is going outbound.
It's the one you're going to have to write, or coerce somebody to write, if you want it that much. I have a sneaking suspicion that SpamAssassin's core could probably be pressed into action here, wrapped in a HTTP proxy. It wouldn't scale terribly well, but it might be enough to keep tabs on a few tens of hosts that you expect trouble to come from. HTTPS would be a bit more tricky and would require the co-operation of the cybercafe to install your CA cert on their browsers and crank down the security settings so you could do a MITM attack.
participants (14)
-
Barry Shein -
Florian Weimer -
Gregory Kuhn -
Hank Nussbacher -
Joe Abley -
Ken Simpson -
Matthew Black -
Peter Corlett -
Petri Helenius -
Sean Donelan -
Simon Waters -
Steven Champeon -
Suresh Ramasubramanian -
Valdis.Kletnieks＠vt.edu