--- Michael.Dillon@radianz.com wrote:

When we make it too hard for legitimate businesses to use spam as a means of advertising their product, then only criminals will use spam.

you can have my mailserver when you can pry it from my cold, dead datacenter... seriously, there have been various proposals ([ADV], etc) to facilitate "legit UCE," but that hasn't slowed the arms race. How would you recommend that we make it easier for legit businesses? ===== David Barak Need Geek Rock? Try The Franchise: http://www.listentothefranchise.com __________________________________ Do you Yahoo!? Yahoo! Mail - Helps protect you from nasty viruses. http://promotions.yahoo.com/new_mail

=> seriously, there have been various proposals ([ADV],

etc) to facilitate "legit UCE," but that hasn't slowed the arms race. How would you recommend that we make it easier for legit businesses?

I don't propose that we make it easier for legit UCE. I'm simply pointing out that it's an arms race because we are solving the wrong problem. We are making it hard for people to send spam, therefore we are reaching the point where only criminals do so. I would rather see us focus on securing the email architecture. Secure submission is part of that, but for some reason people are unwilling to imagine an email system in which an ISP will only accept incoming messages from another ISP with which they have an existing agreement, i.e. rather like email peering. I happen to believe that a web of email peering agreements is the best way to get us to the point where it is difficult for anyone to anonymously send email because they *MUST* relay it through an ISP who will not accept the email for relay unless they have authenticated the user. This is solving a different problem. Spam is merely a symptom of an overly simplistic and insecure email architecture. Now that it has drawn our attention to the problem, I think we should ignore spam and focus on making a better email architecture that people can actually use again. --Michael Dillon

...

for some reason people are unwilling to imagine an email system in which an ISP will only accept incoming messages from another ISP with which they have an existing agreement, i.e. rather like email peering.

You say this as if it's surprising that people are willing to accept communications from people they have not yet communicated with before.

There is a difference between an ISP and a person who sends or receives email. I am only suggesting that ISPs should make mail peering agreements, not individuals. When I wrote a weekly column for Internet World magazine, I frequently received email from readers with whom I had not previously communicated because my email address was printed at the bottom of each article. I developped my suggestion with this in mind. Anyone will be able to write an email and relay it through their ISPs authenticated submission port regardless of whether they are at home or in a hotel in some other country. If their ISP has a mail peering agreement with my ISP, then it will be relayed directly to them. If not, then they will relay it to a larger email peer who can handle the mail routing. Clearly, there has to be some way for a domain to publish their email peers in DNS so that mail routing can take place, but this is relatively trivial as are most of the technical issues. The bug problem to solve is the operational issues of putting it all together and negotiating mail peering areements.

The world is not like your gated community.

I have never lived in a gated community. Also, this new email architecture would not be a gated community. It may start off as a special service offered by a few larger ISPs to business clients, but over time I think most people will migrate to it. --Michael Dillon

Right now I have freedom of communication. In your vision I would hand all that over to my ISP for the benefit of giving complete control over who can communicate with me to them.

Perhaps you could explain to me just how you currently manage to get port 25 packets delivered to your friends without transitting your ISP? Or did you just mean "freedom of communication" in a rhetorical sense? And if you will trust an ISP to deliver port 25 packets then why wouldn't you trust them to deliver email messages? --Michael Dillon

--On Wednesday, January 12, 2005 4:11 PM +0000 Michael.Dillon@radianz.com wrote:

...

Right now I have freedom of communication. In your vision I would hand all that over to my ISP for the benefit of giving complete control over who can communicate with me to them.

Perhaps you could explain to me just how you currently manage to get port 25 packets delivered to your friends without transitting your ISP? Or did you just mean "freedom of communication" in a rhetorical sense?

Yes, my port 25 packets go through my ISP. However, TLS means that none of the SMTP conversation between my mailserver and my friends mailserver is visible to my ISP in an unencrypted form. Your system would require me to expose at least the envelope information to my ISP. Do you see the difference here?

And if you will trust an ISP to deliver port 25 packets then why wouldn't you trust them to deliver email messages?

I don't trust them to deliver port 25 packets. I expect them to deliver port 25 packets. Then, I authenticate the system at the other end using TLS and have an encrypted coversation. My ISP can see that there's encrypted data going through their network between our servers, but, they (at least theoretically) can't see what that data is. Owen -- If it wasn't crypto-signed, it probably didn't come from me.

Previously, Michael.Dillon@radianz.com (Michael.Dillon@radianz.com) wrote:

...

...

for some reason people are unwilling to imagine an email system in which an ISP will only accept incoming messages from another ISP with which they have an existing agreement, i.e. rather like email peering.

You say this as if it's surprising that people are willing to accept communications from people they have not yet communicated with before.

There is a difference between an ISP and a person who sends or receives email. I am only suggesting that ISPs should make mail peering agreements, not individuals. When I wrote a weekly column for Internet World magazine, I frequently received email from readers with whom I had not previously communicated because my email address was printed at the bottom of each article. I developped my suggestion with this in mind.

Considering that many of the big spam outfits have gone so far as to buy ISPs or start their own simply for the purpose of spamming, this becomes fairly moot. So I start up a new corporation, buy co-lo space from a provider, register my mail server with them and send all the spam I want until they kick me off the network. Rinse. Repeat. That sounds a lot like our current system today, the difference being you'd use some new database to determine the source of your spam rather than SWIP records. -- Douglas A. Dever dever@snoopy.net http://www.getaclue.net/

Ah right - let's go right back to the days of X-400 or possibly UUCP nodes

I don't want to rejuvenate an old obsolete protocol.

Or if this is something newer, well, that's yet another proposal to take to the IETF

I don't want to develop a new protocol.

...

This is solving a different problem. Spam is merely a symptom of an overly simplistic and insecure email architecture. Now that it has drawn our

Changing the smtp protocol, or deploying an entirely new protocol that meets your rigid critieria are some things you have got to do then .. keeping in mind Vern Schryver's checklist of whether this is yet another "final ultimate solution to the spam problem" - http://www.rhyolite.com/anti-spam/you-might-be.html

I don't want to solve the spam problem so I don't consider that Vern's criteria applies to my suggestion. I suspect that my suggestion will make it easier to track down spammers, and provide additional tools to rate limit spam however I don't really care about whether or not my suggestion stops spam. I think that a secure email infrastructure is a good thing to have, in and of itself. By secure, I mean one in which messages get to their destination reliably, i.e. not lost in some spam filter, and one in which a recipient can reliably know where the message came from if they feel the need to track down the sender by other means. My suggestion is to use the existing email protocols but to make some operational changes in the way in which we use them. Mail peering is an operational change, not a protocol change. Forcing people to relay all email through their ISP's mail system is an operational change. Refusing to accept any incoming email that is not relayed by a mail peer is an operational change. Recently in London, the police decided to tackle the drug problem without making new laws. They implemented an operational change by ceasing to arrest people (in most cases) for smoking marijuana. They simply confiscate the drugs, warn them and walk away. As a result, they freed up a large number of police resources to focus on heroin and crack dealers. In a sense, I am suggesting a similar reallocation of resources. Rather than put those resources into filtering spam, I'd suggest that we will get a better result by shifting the resources into mail relaying and managing mail peering agreements. The spam will continue but users will move to using the secure mail architecture and won't see most of it. When the spammers also shift, there will be more tools to track them down or shut them down or simply to rate limit them. --Michael Dillon

4) all domains with invalid whois data MUST be deactivated (not confiscated, just temporarily removed ...

All? Even those unpublished and therefore non-resolving? Sensible for the scoped-to-totality trademarks weenies who argue that the stringspace is a venue for dilution, whether the registry publishes all of its allocations or not. I'm not sure why anyone cares about a very large class of domains in the context of SMTP however.

5) whois data MUST be normalized and available in machine-readable form

There are some registries that use paper to answer registration queries. I'm not sure why anyone cares about a very small class of domains in the context of SMTP however. Aggregation and reformatting have their place. We explored this in the whoisfix bofs but no working group congealed around "fixing" :43. Again, I'm not sure why anyone cares about a very large class of whois:43 output sources in the context of SMTP however. Eric

Why would it matter if you deactivated an unpublished/non-resolving domain?

How do "you deactivate an unpublished/non-resolving domain"? You may borrow a registrar or registry hat if that is useful to answer the question.

If you care about the domain, keep the whois data up to date and accurate.

That is the policy articulated by the trademarks "stakeholders" in the ICANN drama, but how does their policy, which is indifferent to any condition but strindspace allocation, relate to any infrastructure that has one or more additional constraints?

...

I'm not sure why anyone cares about a very large class of domains in the context of SMTP however.

For one thing, a very large class of domains are being used as throwaways by spammers ...

Do you know anything about the acquisition pattern at all, or if there is any useful characterization finer in scope than "all"?

... (thanks, VRSN!)

I pointed out to Mark here on NANOG months ago that there were side effects to pursuit of zonefile publication that was asynchronous with whois data publication. Now that the temporal properties of resolution by one or more registries has your attention, just what part of the actions by all registrants is controlling?

potential protection value whois might offer, and allows spammers and other abusers to fly below the radar, accountable to nobody.

I'm sure they pay their ns providers, and their isps, for the critical portions of the value return path.

...

There are some registries that use paper to answer registration queries.

And?

You appear to see a policy that would cause them to change their operational practice, and I'm not clear on how your policy goal would benefit them, or how they would recover costs if your policy goal did not benefit them.

...

I'm not sure why anyone cares about a very small class of domains in the context of SMTP however.

It's not a very small class of domains with more or less unpredictable data formats. It's ALL of them, or damn near.

So in your current conceptual model, a uniform distribution correctly characterizes the utility of knowing any particular registrar's or registry's whois (whois/tcp or http-form-post/tcp) format?

I should be able to write a program, relatively easily, that would give me any available contact or registrant information on a per-field basis, from any whois service. The wide variety and nonuniformity of the existing services makes that task daunting at best ...

Have you considered looking for a paid service that does :43 reformatting?

...

Aggregation and reformatting have their place. We explored this in the whoisfix bofs but no working group congealed around "fixing" :43.

What were the objections/sticking points?

I'll see if I still have the minutes.

...

Again, I'm not sure why anyone cares about a very large class of whois:43 output sources in the context of SMTP however.

It's not just the context of SMTP. It's the context of accountability on the Internet, which bad actors are exploiting, currently, via SMTP.

Hmm. I'd prefer to stay on point. As for accountability and bad actors, this is a target rich environment. For instance, all paid registrations for .net domains after mid-year already present an interesting accountability issue.

I really do think it would benefit some folks here to read up on the "broken windows theory" of crime prevention.

Anyone in particular? Is the theory a better choice than empirical data? Eric registry, registrar, whoisfix and epp hats lying around somewhere, most collecting snow today.

I suppose it depends on how you define 'unpublished'; and how you define 'non-resolving'.

Your opening remark was that policy foo must be applied to all domains. This doesn't accomplish anything for the set of domains that will never be published (registry reserved strings), nor those that absent seperate acts of malfesance, will always have a very low average association with disfunction -- the 50% of the .net namespace that actually goes to real boxen owned and operated by real people. Between, and in addition to these two samples, there are classes of domains that are vastly less likely to be used in uce and equivalent schemes. The class of domains purchased simply to take them out, such as Hamming distance buys around a defended mark, may never resolve. "All" is too blunt a tool.

I reported it to ICANN for having invalid whois data. It took them ... ... a year to have it removed from the root dbs.

That is an ICANN issue. It may come as a surprise to you but for the past few years the "ISP Constituency" has ceased to exist, and has been folded into Marilyn Cade and Philipe Sheppard's "Business Constituency".

Please see my other message. Allowing domains with invalid whois data to remain in use facilitates abuse in other realms.

If it isn't "fixing insecure email infrastructure", then it needs to find a thread and/or list of its own. The little table of domain names and redirects is slightly useful, but it would be more useful if your data could show registrar clustering.

I'd be delighted if you have pointers to a paid whois reformatter, but I still believe strongly that it should not be necessary.

The quality of data usually has a relationship with the cost of care that has gone into that data, just like abuse desks. Eric

Why is it considered such a crazy proposition that domains should have valid and correct whois data associated with them?

There is no relationship between data and funcion. The data is not necessary to implement function-based policy.

Bah. You're saying that you're uninterested in discussing the root causes that allow and even encourage abuse to occur in specific realms. I guess you're not interested in actually "fixing insecure email infrastructure".

I have no idea what specific realms you could be referring to.

...

The little table of domain names and redirects is slightly useful, but it would be more useful if your data could show registrar clustering.

Why should this matter? Spammy can always choose a different registrar every day. So what? He is registering domains for use in abusive and criminal acts, and the message I'm getting from you is that it should only be of concern to you if he uses the same registrar?

OK. The choice of registrar, registrar policy, registrar price, and so on isn't data that could be of use to anyone ever. But you're going to get "valid and correct whois data" from all registrars. How will you get that? What does "valid" and "correct" mean? Does it apply to all the records in a single domain registration, or just some of them? Eric

on Wed, Jan 12, 2005 at 10:32:13AM -0600, Chris Adams wrote:

Once upon a time, Steven Champeon <schampeo@hesketh.com> said:

...

7) all ISPs MUST act on ANY single abuse report (including being informed of infected customer machines, which MUST be removed from the Internet ASAP. No excuses)

One problem I have with this one is people do forge reports, and there is no way around that. Also, as long as there are networks that don't enforce source address filtering, port probing complaints cannot be validated (I take them as valid unless proven otherwise, but we have had a few that appear after the fact to be forged and/or spoofed). If you _always_ take someone off-line on a single complaint, you make it easy for someone to get someone else kicked off.

Think of it as two separate requirements, one dependent on the other. 1) I'm tired of hearing stories about ISPs who let Spammer X continue because "there weren't enough complaints", and 2) once you've verified that a reported infected host IS infected, take 'im offline ASAP. Or, restate it as "no more abuse desk role account autoack ignorebots". -- hesketh.com/inc. v: +1(919)834-2552 f: +1(919)834-2554 w: http://hesketh.com join us! http://hesketh.com/about/careers/account_manager.html join us!

on Wed, Jan 12, 2005 at 12:41:44PM -0600, Adi Linden wrote:

...

0) for the love of God, Montresor, just block port 25 outbound already.

What is wrong with dedicating port 25 to server to server communication with some means of authentication (DNS?) to ensure that it is indeed a vaild mail server.

Nothing at all. That's more or less what I proposed, though I'd prefer to see something TODAY, like the easily implemented rDNS fix, rather than wait any longer for SPF/DomainKeys/etc. to go through a zillion rounds of argument. As it stands, I reject a rather large percentage of the spam delivery attempts here using generic rDNS as a basis. (Either in the rDNS of the connecting host itself or in the HELO; the latter is responsible for ~75%-80% of the rejections, assumed to be almost entirely zombie-originated).

Mail clients should be using port 587 to submit messages to their local MTA.

Agreed. -- hesketh.com/inc. v: +1(919)834-2552 f: +1(919)834-2554 w: http://hesketh.com join us! http://hesketh.com/about/careers/account_manager.html join us!

On Wed, 12 Jan 2005 17:40:10 -0500, Valdis.Kletnieks@vt.edu wrote:

� >�1) any legitimate mail source MUST have valid, functioning, non-generic � >����rDNS indicating that it is a mail server or source.

� And how, exactly, does it indicate it's a mail server or source?

In general, that's what dkeys/iim and csv (and maybe spf) are attempting to provide. d/ -- Dave Crocker Brandenburg InternetWorking +1.408.246.8253 dcrocker �a t ... WE'VE MOVED to: �www.bbiw.net

On Wed, 12 Jan 2005 23:19:47 -0500, Valdis.Kletnieks@vt.edu <Valdis.Kletnieks@vt.edu> wrote:

On Wed, 12 Jan 2005 19:19:24 PST, Dave Crocker said:

...

In general, that's what dkeys/iim and csv (and maybe spf) are attempting to provide.

Yes, but he asked for a rDNS solution specifically...

I think Steve was referring to some things that can be implemented right away, like "if you operate a mailserver, please make sure that it isn't on a host that has reverse dns like ppp-XXX.adsl.example.com, try to give it unique and non generic rDNS, preferably with a hostname that starts off with smtp-out, mail, mta etc)" Basically a call to operators to adopt a consistent forward and reverse DNS naming pattern for their mailservers, static IP netblocks, dynamic IP netblocks etc. -- Suresh Ramasubramanian (ops.lists@gmail.com)

Steven Champeon wrote:

on Thu, Jan 13, 2005 at 10:25:18AM +0530, Suresh Ramasubramanian wrote:

...

On Wed, 12 Jan 2005 23:19:47 -0500, Valdis.Kletnieks@vt.edu <Valdis.Kletnieks@vt.edu> wrote:

...

On Wed, 12 Jan 2005 19:19:24 PST, Dave Crocker said:

...

In general, that's what dkeys/iim and csv (and maybe spf) are attempting to provide.

Yes, but he asked for a rDNS solution specifically...

I think Steve was referring to some things that can be implemented right away, like "if you operate a mailserver, please make sure that it isn't on a host that has reverse dns like ppp-XXX.adsl.example.com, try to give it unique and non generic rDNS, preferably with a hostname that starts off with smtp-out, mail, mta etc)"

Yep. And it helps if the rDNS is "right-anchored", (uses "subdomains" to distinguish between various assignment types and technologies) a la

1-2-3-4.dialup.dyn.example.net ^^^^^^^^^^^^^^^^ 4-3-2-1.dsl.static.example.net ^^^^^^^^^^^^^^^^^^^ as opposed to

dyn-dialup-1-2-3-4.example.net static-dsl-4-3-2-1.example.net

as the former is easier to block using even the simplest of antispam heuristics. I'd love to see a convention, or even a standard, arise for rDNS naming of legit mail servers. But I'll happily settle for decent and consistent rDNS naming of everything else ;)

What is wrong with MTAMARK? MTAMARK tags the reverse entries of IP addresses where SMTP servers are. Fixes this problem very fast, efficient and with little effort (script magic to regenerate the reverse DNS entries). ftp://ftp.rfc-editor.org/in-notes/internet-drafts/draft-stumpf-dns-mtamark-03.txt -- Andre

(sorry, first reply to list lost due to wrong From)

In priciple, nothing. In practice, the rDNS is a mess and I don't know many people who think it's likely to get cleaned up enough that we can expect to put in all the MTA MARK entries.

If you look at your logfiles you will notice that > 95% of all legit mailservers already have working and individual revDNS. And it is not about adding MTA="no" records, "MTA=yes" is much more important. As of now for a lot of broadband users it is important if the ISP supports fastpath (disabled error correction) for online gaming and IP phones. In the future it may be important, if you want to run a mailserver, if the ISP supports revDNS. The DE zone (about 6 mio SLDs) had in July 2004 (thanks to Peter Koch who made the survey) about 140000 unique IP addresses used in MX records. Assume the same number of outgoing MTAs and you have a really low cost - compared to other methods - first approximation for solving a part of the spam problem and providing hints for methods like greylisting (it doesn't make too much sense to greylist a mailserver) or using it as a whitelist for automated block lists (quite a number of viruses is coming from legit mailservers as a result of forwards). The more TLDs you add to the set the better the ratio domain/IPs becomes as - at least in DE - a lot of DE domains, also have a compagnion domain in .COM, .NET, .ORG, .AT, ... that use the same mailservers. IMHO the spam solving "business" is becoming really twisted: Some methods are unacceptable because they cut off 0.001% of all mailservers (Africa + dynamic IP space; that problem could very easily be solved with a colocation or a relay for nearly no bucks per month at all). But 100% of all Internet users have to suffer each day, as 100 or 1000 times the number hosts compared to the number of legit mailservers can inject their crap to any mailserver they like and you have little chance to block them at SMTP level. And that means the costs have already been shifted to the recipient. But obviously we have passed the point-of-no-return and the antispam business is a big enough market share so that free-of-cost solutions (and I am not speaking of MTAMARK alone) that don't hurt the existing Internet Mail Infrastructure at all, are not of any interest to the big players, as they can't make money out of it. And all the others always have the same excuse: why should I spend some 10 minutes to 2 hours to add or fix something. I'll do it if 50 others already have done it. The answer is simple: it is very kewl to have a consistent, well behaving and clean network that you can show around to others like your appartment, your house or your freshly washed and polished car or bike. Another example: it is a matter of 2 minutes in 99% of all situations to fix a mailserver to send a proper and matching HELO string. What is your excuse that yours is still sending "localhost.localdomain" or "SL-2000-1.local" in contrast to what is proposed (but not required)? Isn't it your mailserver and don't you want it to look good and wellbehaved while talking to other mailservers all day long? \Maex -- SpaceNet AG | Joseph-Dollinger-Bogen 14 | Fon: +49 (89) 32356-0 Research & Development | D-80807 Muenchen | Fax: +49 (89) 32356-299 "The security, stability and reliability of a computer system is reciprocally proportional to the amount of vacuity between the ears of the admin"

On Tue, Jan 25, 2005 at 01:09:04PM +0530, Suresh Ramasubramanian wrote:

On Mon, 24 Jan 2005 22:29:49 +0100, Markus Stumpf <maex-lists-nanog@space.net> wrote:

...

If you look at your logfiles you will notice that > 95% of all legit mailservers already have working and individual revDNS.

I'll just point out that you are generalizing based on a case you see in your mailserver

I am generalizing on what I see from about 300 mailservers and about 1 million messages a day.

I havent got the time to gather stats from our production clusters right now but a quick grep through the last week's logs on my personal colo (lots of ISPs in india mail it, some indian users - friends, family, large local linux lists - on it) .. I'd say that about 40% of my legitimate email comes from IPs that don't have rDNS let alone DNAME / MTAMARK.

How did you calculate that "40% of my legitimate email"? If you get 60 emails from 60 different hosts that have revDNS and you get 40 mails from two hosts without revDNS then also "40% of your legitimate email" is coming from servers without revDNS, but in fact the precentage of servers without revDNS would be around 3.2%. Quite a difference.

On our production boxes we get email from around the world for about 40 million users, and I just dont want to try blocking based on no reverse DNS there .. just not worth the amount of legitimate email traffic that gets filtered out.

On the mailserver for our company we had 2002 attempts to inject messages for the last 17h30m from hosts without any revDNS. -> 30 allowed, 2 of them non spam -> 1982 rejected (badhelo (ip or name of local mailserver), not existing recipient, relaying denied, blocked due to prior spamming) This makes a 0.1% non-spam rate. 888 unique hosts sending spam, 2 did not, 0.23% good servers without revDNS. yesterday: 2368 attempts from hosts without any revDNS -> 2315 rejected -> 53 allowed, 6 of them non spam (4 of them from the same sender) This makes a 0.25% non-spam rate. 1044 unique hosts sending spam, 3 did not, 0.29% good servers without revDNS. As you can see, we don't filter out "no revDNS", too. But setting MTAMARK records would give the admins of the receiving mailservers a hint as how to classify the sending IP. \Maex -- SpaceNet AG | Joseph-Dollinger-Bogen 14 | Fon: +49 (89) 32356-0 Research & Development | D-80807 Muenchen | Fax: +49 (89) 32356-299 "The security, stability and reliability of a computer system is reciprocally proportional to the amount of vacuity between the ears of the admin"

On Tue, Jan 25, 2005 at 12:22:33PM -0500, Valdis.Kletnieks@vt.edu wrote:

Which would mean that if Suresh insisted on revDNS, he'd end up blocking only 2 hosts, but 40% of his legitimate mail would be dropped on the floor.

Correct. But neither MTAMARK nor I suggest blocking based on non existant revDNS. The idea of MTAMARK is to add information to revDNS to give the sending host either a better reputation or signal "do not accept mail from that host". For the deployment of such information it makes a difference if 40% of the hosts don't have revDNS or only 4%. With 4% it may be worth the trouble convincing some admins and adding some local whitelisting rules, with 40% you probably don't need to try starting at all.

I'd *hope* that knowingly dropping 40% of the *legitimate* mail on the floor would be considered a CLM. But these days some providers seem to think "all of Europe" is a reasonable filter.....

Isn't this free market economy? They want to isolate themselves, it's their decision. And IMHO "all of Europe" is more fair than "all of Europe but not the five biggest ISPs". \Maex -- SpaceNet AG | Joseph-Dollinger-Bogen 14 | Fon: +49 (89) 32356-0 Research & Development | D-80807 Muenchen | Fax: +49 (89) 32356-299 "The security, stability and reliability of a computer system is reciprocally proportional to the amount of vacuity between the ears of the admin"

On Tue, 25 Jan 2005 09:43:06 PST, "J.D. Falk" said:

(I'm also surprised you need 300 servers to handle such a small load -- what is that, ~3333 messages per server per day?)

Some mail software scales better than others. ;) And yes, we *DID* have one large software vendor admit that at the time, they'd need a 300-system cluster to handle all our 70K users. We ended up with a single Sun system instead. Not sure if said vendor scales any better now - we have like 6 or 7 machines to handle the 2,000 or so users we have on their package (but that's including test and backup boxes...)

On Tue, 25 Jan 2005 18:03:02 +0100, Markus Stumpf <maex-lists-nanog@space.net> wrote:

...

I'll just point out that you are generalizing based on a case you see in your mailserver

I am generalizing on what I see from about 300 mailservers and about 1 million messages a day.

You should see the trends I describe in any case - even with a comparatively smaller userbase like this.

How did you calculate that "40% of my legitimate email"? If you get 60 emails from 60 different hosts that have revDNS and you get 40 mails from two hosts without revDNS then also "40% of your

I have not noticed that it is a case of just two (or even two dozen) hosts sending me nearly all that email

legitimate email" is coming from servers without revDNS, but in fact the precentage of servers without revDNS would be around 3.2%. Quite a difference.

Moot though - I care about legitimate email that gets dropped if we start rejecting traffic from hosts with no rDNS. Please see if you have any customers who are in regular touch with their friends or relatives in asia or africa.

As you can see, we don't filter out "no revDNS", too. But setting MTAMARK records would give the admins of the receiving mailservers a hint as how to classify the sending IP.

CSV is what I am hoping for .. but I wouldnt depend on any of these proposals. Helo checks, dnsbls etc catch a ton of spam for us. Large providers implementing CSV will help us, as will our implementing BATV and/or signing outbound mail with domainkleys (which would help us identify and cut down on the number of backscatter bounces) This is rapidly growing OT for nanog though so I'll stop here. --srs

On Wed, 12 Jan 2005, Steven Champeon wrote:

...

In a sense, I am suggesting a similar reallocation of resources. Rather than put those resources into filtering spam, I'd suggest that we will get a better result by shifting the resources into mail relaying and managing mail peering agreements. The spam will continue but users will move to using the secure mail architecture and won't see most of it. When the spammers also shift, there will be more tools to track them down or shut them down or simply to rate limit them.

OK, sounds great. Let's start by making a few SHOULDs and MAYs into MUSTs.

Its nearly impossible to make MAY into MUST. You can do slow update from MAY to SHOULD and from SHOULD to MUST over period of several years but in that case you also need to provide exact date when old SHOULD would be depreciated and until then you can't assume its a MUST.

Some of the following could be accomplished in a few hours, Ha. You're kidding, right?

some are probably not fixable unless we can reallocate some of the trillions of hours we waste fighting spam to the problem AND get some political support for accomplishing them (such as fixing whois once and for all). Its being worked on and CRISP just released new whois standard (see below) The migration is however a very slow process.

Bear in mind that "fixing email" largely means "fixing all the other brokenness that allows people to take advantage of email's trust model". I'd actually advocate for slow change in email infrastructure model. But I'll not elaborate at this time, see you in 2 months about it.

So, then, it means fixing DNS conventions, abuse reporting support infrastructure (starting with whois), and broken mail server/client configurations.

0) for the love of God, Montresor, just block port 25 outbound already.

There are legitimate uses of port 25, the question is that you need to have it blocked for anauthenticated use. There are the following ways to accomodate situations when some users need to have unblocked por25 when majority do not: 1. Blocking port25 by default and allowing authenticated users who have requested it to have it unblocked. That should be done by means of radius profile and I believe can be done with existing tools (I have not been involved in dialup for 4 years now but from what I remember I could easily have specific user profiles with different redirection data for port25). 2. Not blocking port25 by default but measuring all traffic that passes through (by that I mean just number of SMTP packets from each ip, not actually looking inside the packets). If any ip shows highier then normal usage then its temporarily blocked and ISP immediatly tries to contact the user to verify what they are not spamming. A complimentary to this is verification that source IPs are the ones assigned by ISP and not spoofed or IPs routed through vpn from some other place (see recent threads about, last one by Ejay when his dialup was abused). Both of the above are ways are practical and can be implemented by ISPs given enough interest.

1) any legitimate mail source MUST have valid, functioning, non-generic rDNS indicating that it is a mail server or source. (Most do, many do not. There is NO reason why not.) Corollary: any NONlegitimate mail source SHOULD be labeled as such (e.g., "1.2.3.4.dynamic.example.net" or "4.3.2.1.dhcp.resnet.foo.edu")

For UnifiedSPF I proposed before that special SPF record be published for the DNS hostname indicated by reverse dnsand that be checked to verify if it should or should not be source of email for that ip.

2) any legitimate mail source MUST HELO/EHLO with its own valid Internet hostname, not "foo.local" or "SHIZNITSONY26354" or "exchng1". Or, with their own bracketed IP. (Most do, many do not. There are very few valid reasons why not. Broken software should be fixed.)

RFC2821 says that HELO should be valid hostname, so a few things that do happen right now are already against it. A new SPF draft also includes checking HELO which will essentially accomplish this in practice. CSV group is also advocating the same with different record syntax. Again it would be slow process of migration from when we start using and have to discard badly configured named to when majority (and 99% of those sending email) have these records and we can begin to advocate for MUST.

3) any legitimate mail source MUST be in a domain with functioning abuse and postmaster mailboxes, which MUST also be listed in the whois db entry for both that domain and IP space corresponding to the mail source. (Not difficult to do at all.)

I beleive this is already in RFCs. Checking for this in real-time is somewhat impractical due to current whois system but we do have rfc-ignorant blacklist specifically for the reported bad whois domains.

4) all domains with invalid whois data MUST be deactivated (not confiscated, just temporarily removed from the root dbs) immediately and their owners contacted. (NOTE: this will require fixing .org, among others whose public whois output is stale and not easily fixable via certain registrars). (Would probably take the most effort, but given a properly broad window of notification should be possible.)

This essentially require registrars to be able to immediatly tell if domain has valid address or not, what is valid in one country may not be valid in another. And phone## may be temporarily deactivated sometimes even at the request of the phone user. There are some ways to find certain cases of invalid whois automaticly and if registrars had any interest in spending any resources on this, they could have it done automaticly so that "-" is not a valid address in the first place, but majority of cases do require long process of verification and spammers only new domain active for couple days, so in practice this is not a solution.

5) whois data MUST be normalized and available in machine-readable form (such as a standard XML schema)

The protocol to provide whois data in standard XML format has been published couple days ago as IETF standard (proposed standard?). See http://www.ietf.org/rfc/rfc3981.txt http://www.ietf.org/rfc/rfc3982.txt http://www.ietf.org/rfc/rfc3983.txt

- the "I hate spam so I use a bogus contact addy" excuse will be obsolete, as email infrastructure will be secured, right? (Honestly, how hard would this be? Gather up all the now-extremely varied formats, compromise on a superset, and map. Then put it all on a Web site or a reliable, distributed infrastructure. I'm REALLY TIRED of getting "whois.$foo:111 connection refused" when I'm trying to track down a spammer's support network).

This is something for ICANN and I do believe that we should be compiling data on the registrars that do not provide whois in real-time when they should (per ICANN agreement) and have ways to report it to them so that some action could be taken by ICANN (an official warning with multiple ones leading to suspension of accreditation would do the trick with bad registrars). [compile data and provide it in public and to icann on non-answering whois servers added to my already very long to-do list]

6) all mail clients MUST support SMTP AUTH and the MSA port. (Many do.) All mail servers MUST support SMTP AUTH and the MSA port. (Some do.)

This is something that you can consider as being in process of migration. As usually its a slow process to have mail servers and mail clients updated.

7) all ISPs MUST act on ANY single abuse report (including being informed of infected customer machines, which MUST be removed from the Internet ASAP. No excuses) (Halve unemployment today. Retrain textile and manufacturing workers. Outsource the entire job. I don't care. Go read "broken windows theory".)

If ISPs hire (and retrain) all unemployed textile workers it will triple connection cost for legitimate users... That is not to say that I believe ISPs should not act on the spam reports or that its being done well right now - there are too many large ISPs that trim costs so much that their abuse departments are almost non-existant and there are of course those few ISPs who deliberately not act and provide connection to spammers (i.e. remember recently disclosed documents from SAVVIS that forced them to terminate more then 100 large spammers).

8) all mail server, antivirus, and antispam software MUST NOT accept and then bounce (to the usually forged sender) bogus "warnings" or DSNs. (A chicken/egg problem, really, but some systems have NO excuse - such as A/V systems that helpfully inform me that some virus that ALWAYS forges the sender did so in a message sent from a system I have no control over.)

No. The solution for this is to stop the forgery so that you do not receive the bounces for email you did not send in the first place. This is being worked on and there are several solutions and I commented on this quite recently, see http://www.mail-archive.com/nanog@merit.edu/msg28472.html

9) all mail servers and webmail systems, etc. MUST properly include tracking information in their Received: headers. (You might be surprised how many webmail systems and large ISPs fail this one. It's largely how 419/AFF scams are propogated.)

Yes, its a problem. BTW do you care if I include GMAIL in the list of those "large ISPs" that are not doing it right?

10) all HTML display engines MUST fix the bugs that allow for a link to say, 'phish.randomisp.net.br' appear as 'wamu.com' (Social engineering, but important in this day of hostile JPG images.)

How do you propose to fix this? Its perfectly valid HTML because link are specifically designed so that you can name them whatever you like for ease of user view. So lets say we begin to disallow URL name to be different URL, would it fix it? Not really, they will do minor conversion and provide name that looks like URL when its not or have the name constructed by javascript, etc. etc. Spammers do know how to get around all these simple fixes...

That should do it. I'd also ask that HTML email simply vanish, since I'm clearly already rubbing this lamp down to its bitter metal core... ;)

Your wish is not going to happen. You may not be reading email in html and I may not be doing it, but when I come to my brother's house or my parents they all do. --- William Leibzon, Elan Networks: mailto: william@elan.net Anti-Spam and Email Security Research Worksite: http://www.elan.net/~william/emailsecurity/

On Wed, Jan 12, 2005 at 10:59:43AM -0500, Steven Champeon <schampeo@hesketh.com> wrote a message of 98 lines which said:

0) for the love of God, Montresor, just block port 25 outbound already.

If there is no escape / exemption (as proposed by William Leibzon), then, as a consumer, I scream "OVER MY DEAD BODY!!!". I want to be able to manage an email server when I subscribe to an ISP. In any case, it would no longer be "Internet access". See the Internet-Draft draft-klensin-ip-service-terms-04.txt, "Terminology for Describing Internet Connectivity".

On Thu, 13 Jan 2005 12:21:04 +0100, Stephane Bortzmeyer said:

American bias but remember the Internet is worldwide. I do not know how it is in the USA but there are many parts of the world where ISP do not have a delegation of in-addr.arpa and therefore cannot pass it to their customers. (It is also common to have many levels of ISP, so you need to go through many layers before reaching the RIR.)

That is indeed a problem that needs to be solved if you want any sort of rDNS-based service to work well.

Requesting rDNS means "I don't want to receive email from Africa".

Having an rDNS entry for a host doesn't mean you know if it is/isn't in Africa, to any higher degree of certainty than when you just had the IP address. I'm not on our campus. But I can see it from out my office window. (The official campus starts across the street from me). I'm about 4 hours drive southwest of Washington DC. professory.cesa.vt.edu is 195.176.186.74, and has a proper PTR entry back. It's a host of ours. It's in Switzerland at our Center for European Studies and Architecture. So what did that rDNS entry tell you about its location that you didn't know from 195.176/16?

Of course, I know that. I just mentioned Africa because, in many countries in Africa, it is simply impossible to get a PTR record. That's a fact, there are many reasons behind.

Howdy Stephane, It is also an area where many cctld operators maintain their registration data using spreadsheets, and "whois" isn't :43. Not an issue of activel malfeasence, other than early adopter attitudes towards late, and challenged adopters. As you note, there are many reasons behind [it, the impossibility to get a PTR record or a :43 server connect]. Eric

...

Requesting rDNS means "I don't want to receive email from Africa".

Having an rDNS entry for a host doesn't mean you know if it is/isn't in Africa, to any higher degree of certainty than when you just had the IP address.

What he was pointing out her is that a majority of African ISPs do not even have the ability to assign rDNS to their address space. This is an unfortunate fact which should get somewhat better as a result of ARIN policies 2002-3 and 2003-15. I don't know to what extent those policies have helped yet, but, at least it is much easier for African ISPs to get direct allocations now. In essence, it is virtually impossible for a small-medium business in Africa to set up a mail server and have rDNS entries created for it because their ISP doesn't control the IN-ADDRs and the imcumbent Telco doesn't want to do anything they don't absolutely have to for the competitive ISPs. Owen -- If it wasn't crypto-signed, it probably didn't come from me.

on Thu, Jan 13, 2005 at 12:21:04PM +0100, Stephane Bortzmeyer wrote:

On Wed, Jan 12, 2005 at 10:59:43AM -0500, Steven Champeon <schampeo@hesketh.com> wrote a message of 98 lines which said:

...

1) any legitimate mail source MUST have valid, functioning, non-generic rDNS indicating that it is a mail server or source. (Most do, many do not. There is NO reason why not.)

Since this list is NANOG, it is reasonable that it has a North American bias but remember the Internet is worldwide. I do not know how it is in the USA but there are many parts of the world where ISP do not have a delegation of in-addr.arpa and therefore cannot pass it to their customers. (It is also common to have many levels of ISP, so you need to go through many layers before reaching the RIR.)

Seems this needs to be fixed, then. Not my problem.

Requesting rDNS means "I don't want to receive email from Africa".

See above. -- hesketh.com/inc. v: +1(919)834-2552 f: +1(919)834-2554 w: http://hesketh.com join us! http://hesketh.com/about/careers/account_manager.html join us!

Quick question for the group.. How long should I be patient to wait for some /24s to become fully routable worldwide? None of the addresses are mine, they came from the upstream (only one provider) They are all part of the upstreams IP space, and I had assumed that they would have kept them as part of a larger block and just blackholed into their network.. Instead it seems that route filters had to be propagated WW before the traffic would leave a given AS.. I can reach the nets now (after the first day) from most of the ASs out there, as seen from traceroute.org... But, as my luck would have it, the two nets that I connect from can't get there (die at the handoff to the next AS). I have confirmed this from other people on a few other ASs as well. It has been two days now.. I don't want to be one of those PITA customers you guys get tired of, but I have work to get done.. How long should I sit on my hands? Note: I would post the /24s, but I don't want to call negative attention to a SP that might be doing his/her job just fine and I just have unrealistic expectations.. Thanks in advance! Michael

On Thu, 13 Jan 2005, Michael Airhart wrote:

Quick question for the group..

How long should I be patient to wait for some /24s to become fully routable worldwide?

There's no such thing as IP space fully routable worldwide. Somewhere there's a poorly run network with oudated bogon filters, NAT'd IPs (yours) that someone pulled from their nether regions rather than RFC1918, or a host of other issues that result in your IPs not being reachable from somewhere. Now, I'd expect any IP space an SP assigned to you to be "pretty much fully routed" before you even have them (since you said this is PA space...PI could conceivably take some time to get upstreams to update prefix lists after they're made aware of the prefixes you intend to announce). Unless it's an issue of your provider passing on "new IANA->ARIN" space like something from 72/8 or a similar block. In that case, you're pretty much SOL. Start trying to contact the NOCs of the various networks that need their bogon filters updated. Maybe suggest your SP do the same on your (and their other customers' behalf).

Note: I would post the /24s, but I don't want to call negative attention to a SP that might be doing his/her job just fine and I just have unrealistic expectations..

How about you post the /8 they're from (if you haven't already). ---------------------------------------------------------------------- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

The current pretense of "privacy" is nothing more than a convenient mechanism for registrars to pad their wallets and evade responsible for facilitating abuse.

As an aside, I used a (wicked big) competitor's "privacy" service to regsiter a domain for a political worker who wanted to whistleblow but not be identified. My customer could now use a web log service such as Duncan Black did under the name of "atrios", and obtain casual (but not subpoena-proof) data protection (non-publication of customer profile data). Broadly I agree that "privacy" as a product under contract law is not a better solution than data protection as a right under human rights. However, data protection isn't as available to all potential registrants.

I think that a secure email infrastructure is a good thing to have, in and of itself. By secure, I mean one in which messages get to their destination reliably, i.e. not lost in some spam filter, and one in which a recipient can reliably know where the message came from if they feel the need to track down the sender by other means.

And how is it that OpenPGP and S/MIME do not meet this criteria? Why is it that we also need to break the transport layer to facilitate what you describe above?

a protocol change. Forcing people to relay all email through their ISP's mail system is an operational change.

Forcing people to relay all email through their ISP's mail system is a wet dream of anti-free-speech governments, too. Why should I have to provide non-encrypted information about my email to my ISP just to get it to my friend's mail server? Why on earth do you think that is a legitimate operational change? Having to route telephone calls through the telephone company is an unfortunate fact of infrastructure which we don't currently have with Email. CALEA is a clear demonstration of why this is not necessarily a good thing. Why would you ever want to consider relegating email to these same restrictions?

In a sense, I am suggesting a similar reallocation of resources. Rather than put those resources into filtering spam, I'd suggest that we will get a better result by shifting the resources into mail relaying and managing mail peering agreements. The spam will continue but users will move to using the secure mail architecture and won't see most of it. When the spammers also shift, there will be more tools to track them down or shut them down or simply to rate limit them.

The problem is that currently, most ISPs don't relay mail for other ISPs. Currently, you look up the MX and send to the end-system. What you are proposing, in order to preserve existing mail connectivity under your new system, would require EVERY ISP on the planet to MAIL PEER directly with every other ISP on the planet, OR, a new mail routing protocol with ISPs providing MAIL RELAY for every transit customer. UG-LY!! Owen -- If it wasn't crypto-signed, it probably didn't come from me.

on Wed, Jan 12, 2005 at 10:18:30AM -0800, Owen DeLong wrote:

Michael, Whether you like it or not, SPAM is the problem.

SPAM is a luncheon meat. UCE is one of the many problems, among the others being viruses/worms/trojans and their traffic (easily blocked by the proper upstream authority), "outscatter/backscatter" traffic (whether responding to joe jobs, viruses, or spammers trying to send their crud to virus-generated bogus addresses, etc.), bogus reports of forgery (cf. Sp@mX, for one example), C/R systems, autoresponders, callback mechanisms, et cetera. -- hesketh.com/inc. v: +1(919)834-2552 f: +1(919)834-2554 w: http://hesketh.com join us! http://hesketh.com/about/careers/account_manager.html join us!

On Wed, 12 Jan 2005 17:41:33 -0500, Valdis.Kletnieks@vt.edu wrote:

� The X.400 concepts of ADMD= and PRMD= really caught on, didn't they? ;)

� Peering in a world of 64K ASNs, mostly basically static, is a lot different � than peering in a world of 40 million plus .COMs, many in motion. �Most of � the time, we're lucky if the MX record points to the right place....

Funny this should come up now... I've been working on a document to describe current Internet Mail architecture and it has taken an unexpected turn. In fact I am trying to add a section that deals with different Operators, as distinct from different functional modules. The reason is primarily because the inter-Operator boundaries define where trust relationships need to be decided and enforced. (The recent papers and discussions about "tussle" boundaries captures this issue really well.) As with so many problems with x.400, it is not that it was wrong to pay attention to one or another issue. The problem was that x.400 mixed everything together into a single, homogeneous and gargantuan pot, therefore requiring adopters to deal with an ungainly mass of specification and code, all at once. You really could not do adoption in incremental steps. Oh, and this all-or-nothing barrier hit the standards guys, too, since some required components did not show up for a long time. The x.400 admd/prmd was, in fact, an addressing construct, rather than merely being an operational and routing construct. Hence, an x.400 address was more like a source route. Arpanet/Internet experience has shown pretty serious scaling problems with visible source routing, uucp experience notwithstanding. The issue we are facing in the current discussion is operations, not addressing. So, for example, the fact of having a variety of operators does not show up in the address, any more than it does in an IP address. Rather, it shows up in routing and trust issues, just as it does with IP. Although lots of operational variety is possible and does occur, perhaps the most common operational scenarios are covered by: origin => [provider =>] [provider =>] destination And the equivalent to AS-AS trust assessment is not all of the domains in the world, but rather domains that involve MTA-MTA exchanges across operator boundaries. I believe the scale of this requirement is exactly the same as the AS-AS requirement. In fact, this is exactly the problem that CSV attends to. d/ -- Dave Crocker Brandenburg InternetWorking +1.408.246.8253 dcrocker �a t ... WE'VE MOVED to: �www.bbiw.net