On Sun, 8 Feb 2004, Stephen Gill wrote:
Hi Mikael,
Aside from the standardization issue, some of the problems with reports as they stand are that they can be routed to the wrong people, there is no clear way of verifying the authenticity of the data, and the sheer number of reports can inundate a given abuse helpdesk such that they are tempted not to take any action at all.
----- snip ----- I hesitate to post anything on this thread, but figured the comments would likely outweigh any flames sent my direction. I'll start by saying this is operational, but only tangentially. At one time I was one of the folks running a moderately large corporate network. These days I'm in grad school at GA Tech. While I'm there I'm getting my MBA and happen to be competing in business plan competitions. Starting in the Fall, with a group of classmates, I've been working on a concept called AbuseButler (http://www.abusebutler.com/) to tackle many of the issues that have come up in this thread. While it is mostly an academic exercise at this point, we'd love to see it have some small-scale commercial success. A functional prototype currently exists, but some features that would be nice to have are completely lacking still. The essential concept is a follows: - Network operators are overwhelmed by the volume of spam and abuse notifications they receive each day. - The variety of formats reports come in is troublesome as it means each one needs human interpretation to be fully understood (garbage in / garbage out) - The folks submitting reports often aren't as clueful as we'd all like, thus they often contact the wrong networks. (crying wolf syndrome) To address these issues we're building a central notification clearing house. Subscriber networks would forward a copy of all abuse@, postmaster@, and other role accounts to our centralized parsing system. The centralized parsing system handles a number of tasks: - Automatically parse standard format messages (SpamCop, myNetWatchman, a native format, etc...) If the message cannot be parsed and appears to come from an actual user respond asking them to reply using the native format (send an empty template). Low-level pseudo-AI would be nice to attempt to parse free form messages and respond to the submitter with an is the correct message instead of a please fill this out message. - Once the data is parsed a number of things take place such as: --- Is the address source address actually from the network contacted (if not send polite brush off message) --- Aggregate a duplicate reports and assign a problem weight (i.e. one entry instead 300 SpamCop messages about the same open relay). --- Templated output to make the information usable to non-English speakers. Instead of dealing with all sorts of free form messages you simply point your abuse desk folks at an abuse dashboard listing the items with the highest scores. Feel free to bash away, but remember, this project started out purely as a proof of concept for a business plan competition, not the technical solution to all the world's spam and abuse troubles. Think any large abuse desks would subscribe to such a solution? Would they accept the ASP model or want to run it in-house? If anybody is interested in more detail I'd be happy to follow-up directly, make available a copy of the existing business plan for comments, etc... -- Andy Warner gtg412j@mail.gatech.edu