Re: Economics of SPAM [Was: Micorsoft's Sender IDAuthentication......?]
Here's a simple mechanism which has not yet been tried seriously. Email server peering. This means that an SMTP server operator only accepts incoming mail from operators with whom they have a bilateral email peering agreement.
This has been tried in the X.400 world. I wouldn't exactly say it worked well - and I, for one, have no desire to return to X.400 style email peering.
Bilateral agreements have been shown to scale quite well whether you look at BGP peering or the world of business contracts. In any case, the fundamental need here is that for somebody to notify the email administrator that is sending spam and for that administrator to act immediately to cut the flow.
The number of agreements needed in the email world is significantly higher than what is needed for BGP. Steinar Haug, Nethelp consulting, sthaug@nethelp.no
On Mon, Jun 13, 2005 at 11:32:31AM +0200, sthaug@nethelp.no <sthaug@nethelp.no> wrote a message of 21 lines which said:
The number of agreements needed in the email world is significantly higher than what is needed for BGP.
The proponents of "email peering" typically want to switch from the current model (millions of independant email servers) to a different model, with only a few big actors. -- "Should anyone be allowed to operate an email system? Perhaps not." Carl Hutzler http://www.circleid.com/article/917_0_1_0_C/
The proponents of "email peering" typically want to switch from the current model (millions of independant email servers) to a different model, with only a few big actors.
and, as unlikely as it may seem, they think they should be those actors. randy
The number of agreements needed in the email world is significantly higher than what is needed for BGP.
The proponents of "email peering" typically want to switch from the current model (millions of independant email servers) to a different model, with only a few big actors.
I don't know who these proponents are, that you refer to. However, in my earlier message I quite clearly described a model that allows for millions of independent email servers organized in roughly 3 levels of hierarchy and I described how it could be done so that email peering IS NOT LIMITED to a few big actors. The 3 levels that I described were, at the top, intercontinental peering between members of 5 organizations which roughly cover the countries in one continent. I suggest that these 5 organizations should adopt the service boundaries of the RIRs rather than trying to reinvent the wheel. Next, there would be peering between all members in the same continental organization. These members would exchange email with each other under contract terms which clearly lay out the responsibilities of sending operators and receiving operators. Finally, at the lowest level, are organizations who do not see a need to become members of an email peering organization and who exchange email with one or two operators who are members of the peering organization. However, these organizations will also be bound by an explicit contracted AUP because the whole point of this peering hierarchy is to have consistent accountability throughout the entire email architecture. This will not prevent spam, but it will provide operators with the power to shut it off, whenever it occurs. It would be useful to also have the ability to verify initial senders of email messages, however that is not essential for this peering architecture to be useful. Here is a sample scenario. Grandma opens an email with a trojan inside it. The trojan installs itself in her machine and starts sending spam through Grandma's broadband connection. One of the spam recipients informs their ISP that they just received a spam message. Their ISP has a look and agrees that this is indeed spam and they see that this spam is still coming in from a neighboring operator. The ISP follows the contractual procedure and sends an official notification of SPAM to the neighbor. The neighbor follows a similar process and identifies the source as Grandma's ISP. They issue a formal notice to Grandma's ISP and 10 seconds after the notice is received, Grandma's IP connectivity is blocked entirely except for HTPP accesses which are all directed to a walled garden explaining the situation and recommending steps that can be taken to clean up trojans, spyware, viruses, etc. What is missing today? - contracted email SLAs between operators - contracted admin interoperation procedures between operators - contracted SLA and AUP with customers that allows immediate shutdown when malware is detected - organizations which can sort out all the details of the above contracts, etc. If the BGP peering side of the business can sort out all of this stuff, then why can't the email side of the business do the same, or perhaps, do even better? --Michael Dillon
The number of agreements needed in the email world is significantly higher than what is needed for BGP. The proponents of "email peering" typically want to switch from the current model (millions of independant email servers) to a different model, with only a few big actors.
* Michael.Dillon@btradianz.com [Thu 16 Jun 2005, 14:48 CEST]:
I don't know who these proponents are, that you refer to. However, in my earlier message I quite clearly described a model that allows for millions of independent email servers organized in roughly 3 levels of hierarchy and I described how it could be done so that email peering IS NOT LIMITED to a few big actors.
You pour some RIR sauce over your hierarchy of the top five players but that still makes it a model with only a few big actors. [..]
This will not prevent spam, but it will provide operators with the power to shut it off, whenever it occurs. It would
No. Infrastructure will provide operators with that power, legal agreements will not. [..]
What is missing today?
Paraphrased: Basically a lot of administrative overhead that will increase costs of everybody involved with no direct benefit except for the satellite players providing those new services and those looking for control over basic infrastructure for whatever reason.
If the BGP peering side of the business can sort out all of this stuff, then why can't the email side of the business do the same, or perhaps, do even better?
It's not comparable, as has been explained several times to you. -- Niels. -- The idle mind is the devil's playground
If the BGP peering side of the business can sort out all of this stuff, then why can't the email side of the business do the same, or perhaps, do even better?
It's not comparable, as has been explained several times to you.
Perhaps you have never been involved in BGP peering? Let me explain what the BGP peering side of the business does, in addition to operating BGP sessions with peers. To start with, most ISPs don't start peering until after they have negotiated and agreement. Those agreements are legal contracts with many pages specifying the responsibilities of the two parties, limits on how the technology is to be applied, SLAs, processes for interoperation and communication between NOCs, i.e. the people protocols. The thousands of bilateral BGP peering contracts are most definitely comparable to the email peering that I am proposing. I have seen many of these contracts in companies that I worked for in the past. They are rather similar to one another in many ways. Since the total number of BGP peers is rather small, it is quite workable to let these contracts evolve to some sort of rough standard and that is what has happened. In the email world, there are many, many more players, and some kind of secret sauce is essential to converge bilateral email peering agreements to some kind of community standard rather than letting evolution take its course and risking chaos as a result. The stuff that you call RIR sauce, is what I would call "open and public negotiation" in some kind of a forum which is not biased or dominated by parties who may have some market dominance. It is, in fact, the antithesis of a model with a few big actors. It is also a model that works, more or less, in other industries. The FCC is one example imposed by government. The RIRs is another example formed from the ground up. There is more than one way to do this. Which would you prefer as a role model, the FCC or ARIN? --Michael Dillon P.S. ARIN itself has absolutely nothing to do with email services and is unlikely to get involved in this in any way. I am using them mainly as a successful example of an open public organization that manages a common resource.
On Thu, 16 Jun 2005 Michael.Dillon@btradianz.com wrote:
If the BGP peering side of the business can sort out all of this stuff, then why can't the email side of the business do the same, or perhaps, do even better?
It's not comparable, as has been explained several times to you.
Perhaps you have never been involved in BGP peering? Let me explain what the BGP peering side of the business does, in addition to operating BGP sessions with peers. To start with, most ISPs don't start peering until after they have negotiated and agreement. Those agreements are legal contracts with many pages specifying the responsibilities of the two parties, limits on how the technology is to be applied, SLAs, processes for interoperation and communication between NOCs, i.e. the people protocols.
This is a commonly made claim, but is rarely true in practice. A few of the largest networks, generally with small numbers of peers, require peering contracts. Most of the smaller networks with large numbers of peering sessions seem to have decided that the overhead of dealing with contracts isn't justified by the benefits. We've got several hundred peering sessions here, and maybe four or five signed contracts.
The thousands of bilateral BGP peering contracts are most definitely comparable to the email peering that I am proposing. I have seen many of these contracts in companies that I worked for in the past. They are rather similar to one another in many ways. Since the total number of BGP peers is rather small, it is quite workable to let these contracts evolve to some sort of rough standard and that is what has happened.
If we ignore the contract piece, this sounds a lot like UUCP.
In the email world, there are many, many more players, and some kind of secret sauce is essential to converge bilateral email peering agreements to some kind of community standard rather than letting evolution take its course and risking chaos as a result. The stuff that you call RIR sauce, is what I would call "open and public negotiation" in some kind of a forum which is not biased or dominated by parties who may have some market dominance. It is, in fact, the antithesis of a model with a few big actors.
It sounds like you envision five big actors, who would function as regulated long distance/international e-mail carriers, each of which would have a monopoly in their region. This is the phone network model in a lot of places, except that phone monopolies don't often cover more than a few countries while your five e-mail monopolies would have an average of 40 countries each. I'll withhold public judgment on whether this would be a good thing. -Steve
Far not, I have nothing to add on the "e-mail peering" hand-waving, but... On 2005-06-16, at 11:49, Michael.Dillon@btradianz.com wrote:
If the BGP peering side of the business can sort out all of this stuff, then why can't the email side of the business do the same, or perhaps, do even better?
It's not comparable, as has been explained several times to you.
Perhaps you have never been involved in BGP peering? Let me explain what the BGP peering side of the business does, in addition to operating BGP sessions with peers. To start with, most ISPs don't start peering until after they have negotiated and agreement. Those agreements are legal contracts with many pages specifying the responsibilities of the two parties, limits on how the technology is to be applied, SLAs, processes for interoperation and communication between NOCs, i.e. the people protocols.
... this (above) is vastly different to my experience. At ISC we have between 1000 and 2000 BGP sessions active at exchanges around the world. I can count the number of those that required signed peering agreements on two hands. Of those that did require paperwork, most were very happy to set up BGP sessions straight away, without waiting for the contracts to be signed and mailed. (If there are any such people watching, to whom I never got around to sending you a signed contract back, please let me know, and sorry :-) Unless I am just very special, and some natural law protects me from mountains of legal paperwork which everybody else is obliged to climb, BGP peering is a lot more free and loose in real life than you suggest. Joe
Michael.Dillon@btradianz.com writes:
The thousands of bilateral BGP peering contracts are most definitely comparable to the email peering that I am proposing.
Dude, it's 2005. You can put down the X.400 crack pipe now. ---Rob
The thousands of bilateral BGP peering contracts are most definitely comparable to the email peering that I am proposing.
Dude, it's 2005. You can put down the X.400 crack pipe now.
Why does fixing the SMTP email architecture by applying some lessons learned from BGP peering lead people to talk about X.400, UUCP, Bitnet, Fidonet and other obsolete protocols? You're right, it's 2005 and we have suffered from email SPAM for 10 years now, getting worse every year. So when are we going to admit that SMTP-based email is *NOT* the superior solution for email that we all thought it was in 1995? --Michael Dillon
On Fri, 17 Jun 2005, Michael.Dillon@btradianz.com wrote:
The thousands of bilateral BGP peering contracts are most definitely comparable to the email peering that I am proposing.
Dude, it's 2005. You can put down the X.400 crack pipe now.
Why does fixing the SMTP email architecture by applying some lessons learned from BGP peering lead people to talk about X.400, UUCP, Bitnet, Fidonet and other obsolete protocols?
Because these protocols did EXACTLY the same thing you've been attempting to push: explicitly routed, trust-based-peering delivery. IT DOES NOT SCALE, and we know this because many of us have long since implemented it! Personally, I've directly used three of the above, and still do run one of them on my personal home server. There are far too many SMTP machines already deployed out there -- we're not talking thousands; here it's tens to hundreds of thousands worldwide -- to reel it back in and make widespread, mandatory mail peering anything but a bedroom fetish for high salaried security consultants. SMTP is not perfect; however, it is the de facto standard and is widely adopted BECAUSE it does not require any of this crap. In the ideal case, where remote networks do strive to keep themselves clean, it works well. Blacklisting a remote network for its failure to keep a clean house is also not perfect, nor are any of the other heuristic approaches being taken. In spite of their imperfections, these approaches are making an impact, and pointing the blame where it belongs: squarely at the misbehaving network. You may feel free to descend back into the world where control of e-mail is in the hands of a few, but the rest of us have given up the ghost on that approach. I do, however, know of a good clinic where they can help you fight your addiction, once you're ready to admit the problem -- that's the first step towards recovery. -- -- Todd Vierling <tv@duh.org> <tv@pobox.com> <todd@vierling.name>
On Mon, 20 Jun 2005, Todd Vierling wrote:
There are far too many SMTP machines already deployed out there -- we're not talking thousands; here it's tens to hundreds of thousands worldwide -- to
It's actually millions. And I'm not just pulling that number out of someone's .... ---------------------------------------------------------------------- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
Folks, This might not turn out to qualify under the precise term of "peering" but I like the general implication that things are not entirely open and that there are service criteria.
...I described how it could be done so that email peering IS NOT LIMITED to a few big actors.
....
What is missing today? - contracted email SLAs between operators - contracted admin interoperation procedures between operators - contracted SLA and AUP with customers that allows immediate shutdown when malware is detected - organizations which can sort out all the details of the above contracts, etc.
If the BGP peering side of the business can sort out all of this stuff, then why can't the email side of the business do the same, or perhaps, do even better?
Email over the Internet has important differences from IP. For the most part, IP datagrams are not differentiated. Also, peering often involves physical channels and transit service. By contrast, the issue with email, today, has to do with problematic content. This entails significantly different approaches to policy making. Because it is at the application level, service over the Internet usually does not involve special physical channels and transit services are far more limited. That said, it certainly makes sense to hold email operators accountable for some aspects of their traffic. (We need to be careful about how much, lest we slide right into pure censorship.) The approach of CSV <http://mipassoc.org/csv> is intended to have operator-to-operator accountability. A purpose of things like the spamops draft (and, more generally, the mipassoc.org approach) is to establish rough consensus on best practises for operators. With respect to something on the order of an SLA, I originally pursued the idea of formal corporate sign-up to the best practises but ran into an immediate and huge barrier. It requires too much strategic decision making by the companies. So it occurred to me that there already was an informal model of voluntary inter-operator collaboration to fight spam and that that might provide a basis for something a bit more scalable and a bit more formal. That's where a membership organization that issues formal best practises comes in. d/ --- Dave Crocker Brandenburg InternetWorking +1.408.246.8253 dcrocker a t ... WE'VE MOVED to: www.bbiw.net
On Thu, 16 Jun 2005, Michael.Dillon@btradianz.com wrote:
The proponents of "email peering" typically want to switch from the current model (millions of independant email servers) to a different model, with only a few big actors.
I don't know who these proponents are, that you refer to. However, in my earlier message I quite clearly described a model that allows for millions of independent email servers organized in roughly 3 levels of hierarchy and I described how it could be done so that email peering IS NOT LIMITED to a few big actors.
You mean like ucbvax? (If you don't know what that means, you have no business talking about Internet e-mail.) Seriously, the mess you're proposing was already done. It didn't scale. Taken from http://en.wikipedia.org/wiki/UUCP : ===== People often published compound bang addresses using the { } convention (see glob) to give paths from several big machines, in the hopes that one's correspondent might be able to get mail to one of them reliably (example: ...!{seismo, ut-sally, ihnp4}!rice!beta!gamma!me). Bang paths of 8 to 10 hops were not uncommon in 1981. ===== You're lost in the past. Study history and stop repeating it back to us. -- -- Todd Vierling <tv@duh.org> <tv@pobox.com> <todd@vierling.name>
Todd Vierling wrote:
On Thu, 16 Jun 2005, Michael.Dillon@btradianz.com wrote:
The proponents of "email peering" typically want to switch from the current model (millions of independant email servers) to a different model, with only a few big actors.
I don't know who these proponents are, that you refer to. However, in my earlier message I quite clearly described a model that allows for millions of independent email servers organized in roughly 3 levels of hierarchy and I described how it could be done so that email peering IS NOT LIMITED to a few big actors.
You mean like ucbvax? (If you don't know what that means, you have no business talking about Internet e-mail.)
Seriously, the mess you're proposing was already done. It didn't scale.
I think the salient point is that BGP itself does not and would not scale to the same level of demand SMTP peering agreements would need. Currently 160k prefixes and 16bit ASNs -- while in and of itself stretching many operators scaliability limits -- come nowhere close to millions of domain names, mailsystems, mail orgs, mail users and pieces of mail. Aggregation is currently failing for BGP, there is no rational basis to assume it could even begin to make traction for SMTP. Its a pipe dream.
In message <Pine.WNT.4.63.0506161337060.3276@jvc>, Todd Vierling writes:
On Thu, 16 Jun 2005, Michael.Dillon@btradianz.com wrote:
The proponents of "email peering" typically want to switch from the current model (millions of independant email servers) to a different model, with only a few big actors.
I don't know who these proponents are, that you refer to. However, in my earlier message I quite clearly described a model that allows for millions of independent email servers organized in roughly 3 levels of hierarchy and I described how it could be done so that email peering IS NOT LIMITED to a few big actors.
You mean like ucbvax? (If you don't know what that means, you have no business talking about Internet e-mail.)
Seriously, the mess you're proposing was already done. It didn't scale. Taken from http://en.wikipedia.org/wiki/UUCP :
===== People often published compound bang addresses using the { } convention (see glob) to give paths from several big machines, in the hopes that one's correspondent might be able to get mail to one of them reliably (example: ...!{seismo, ut-sally, ihnp4}!rice!beta!gamma!me). Bang paths of 8 to 10 hops were not uncommon in 1981. =====
You're lost in the past. Study history and stop repeating it back to us.
Although I agree that email peering is a seriously bad idea, I don't think that the analogy to uucp is correct. Uucp users had to know explicit paths; today, we'd use routing protocols. (I tried the equivalent for uucp back in 1982, but the map data -- think routing registry -- was of too low quality. Hmm -- today's routing registry isn't very complete, either. But we have BGP, which uucp didn't have.) Uucp also relied on relative names, rather than absolute domain names. This scheme would retain domain names. The other big difference from early uucp is the presence of business agreements. A lot of uucp email (and netnews) traffic was, shall we say, not always carried in accord with corporate goals. People today are accustomed to paying for basic Internet services such as access; that was rarely possible in uucp days. (For more details, see http://www.cs.columbia.edu/~smb/papers/pathalias.paper.pdf by myself and Peter Honeyman, published in 1986.) There are, however, three very big problems. First, it forces people to pay for services that they don't pay for today. I'm not talking about Jo[e] Consumer, I'm talking about a modest-sized business or ISP. They have the ability to deliver email today and will resist being told to pay. Nor will paying stop them from receiving spam -- at best, they see less spam if *you* pay. In other words, the incentives are backwards. A second problem is that multi-hop email seriously reduces reliability. That is indeed a lesson we learned in uucp days. It's mentioned in the paper; it was present even more explicitly in the original pathalias software package I released. But the biggest issue is control: if you have to pass email through a site, that site controls whether or not it can be delivered. Yes, that might stop (some) spam. Might it also stop unpopular political ideas, or provide a facility for government (or whomever) to profile you? Maybe my upstream email provider, 3 hops away, is Google; does that mean I've consented to let Google associate keywords in my email with my email address? (I won't reprise the debate about gmail and privacy, save to note that as of this morning, the Google web page on *sender* privacy still doesn't address that point.) --Steven M. Bellovin, http://www.cs.columbia.edu/~smb
On Thu, 16 Jun 2005, Steven M. Bellovin wrote:
You're lost in the past. Study history and stop repeating it back to us.
Although I agree that email peering is a seriously bad idea, I don't think that the analogy to uucp is correct.
You're right -- I left out the routing table bit, which also existed some time ago. BITNET used the bitnet.links file; here's an old one still accessible for viewing: http://web.mit.edu/afs/athena/reference/net-directory/host-tables/bitnet.lin... Similar concept, same scaling problems; it just hides the explicit routing from the user (as would any modern "peering" system, presumably). -- -- Todd Vierling <tv@duh.org> <tv@pobox.com> <todd@vierling.name>
On Thu, 16 Jun 2005, Todd Vierling wrote:
On Thu, 16 Jun 2005, Steven M. Bellovin wrote:
You're lost in the past. Study history and stop repeating it back to us.
Although I agree that email peering is a seriously bad idea, I don't think that the analogy to uucp is correct.
You're right -- I left out the routing table bit, which also existed some time ago. BITNET used the bitnet.links file; here's an old one still accessible for viewing:
http://web.mit.edu/afs/athena/reference/net-directory/host-tables/bitnet.lin...
Similar concept, same scaling problems; it just hides the explicit routing from the user (as would any modern "peering" system, presumably).
And just to add to it all, you might want to compare to another similar system - FidoNet with its nodelist routing files. Randy Bush is probably right person to ask but I have had a feeling in 1994 that its routing system would not be able to deal with all the people that want to be connected to global email system. But of course we end with everyone switching to internet and its no longer a problem (and fidonet zones now are either smaller or same size as 10 years ago). And also consider that fidonet does have hierarchical address which makes routing decisions easier. With internet and majority of users using email addresses in TLDs (or country SLDs which is the same), there is no such hierarchy available. So my opinion is that email peering is not workable as far as solution for entire internet. But some of it may be of use in very limited scale between certain very large providers as internal whitelisting system. -- William Leibzon Elan Networks william@elan.net
Similar concept, same scaling problems; it just hides the explicit routing from the user (as would any modern "peering" system, presumably).
Then you are presuming wrongly. Nowhere in what I wrote have I suggested any changes in the existing email technology. I am not suggesting that we drop SMTP in favour of your favourite old dusty protocol. I am suggesting that we need a system of accountability for people who run Internet email servers based on contracts and SLAs, i.e. peering agreements. I haven't specified how it would be implemented because I expect that the companies negotiating such agreements would specify this in some kind of operational best-practices document. One way that it COULD be implemented is for people accepting incoming email on port 25 to check a whitelist before accepting email. Only operators who have signed a peering agreement would be on the whitelist. Presumably, the whitelist would be served up by your regional association and they would have some means of relaying queries (or synchronizing their database) with the other 4 regions. Let's face it, people have described a lot of best practices for running SMTP based email services but there has never been a concerted effort to implement these in some methodical way. It has always been a case of preaching to the converted at NANOG or on some lists. And it just does not scale! --Michael Dillon
Michael.Dillon@btradianz.com wrote:
Similar concept, same scaling problems; it just hides the explicit
routing
from the user (as would any modern "peering" system, presumably).
<snip>
One way that it COULD be implemented is for people accepting incoming email on port 25 to check a whitelist before accepting email. Only operators who have signed a peering agreement would be on the whitelist. Presumably, the whitelist would be served up by your regional association and they would have some means of relaying queries (or synchronizing their database) with the other 4 regions.
DNSWL -- this is already being done. It is not widely viewed as being in any way similar to a peering concept. What would be more similar would be a consortium of large providers providing such a whitelist. That would be something I would welcome. I would settle for having aol,msn,yahoo,earthlink,cablevision or any half dozen providers making public THEIR whitelists. The problem is that there does not appear to be any incentive for them to do so -- fee or no fee. In fact, I would encourage anyone planning on ragging on DNSBL's to put up AND shut up, namely operate a DNSWL. Existing public whitelists include: exemption.ahbl.org bondedsender.org habeas.com To use it with sendmail: jlewis's http://njabl.org/dnswl.m4 http://groups-beta.google.com/group/comp.mail.sendmail/msg/a26d1cbd1c739626 To use it with spamassassin: header XXX_DNSWL eval:check_rbl('xxx-firsttrusted', 'xxx.ttec.net') score XXX_DNSWL -5 Anyone else with a public DNS whitelist? <snip>
On 17/06/05, Joe Maimon <jmaimon@ttec.com> wrote:
DNSWL -- this is already being done. It is not widely viewed as being in any way similar to a peering concept. What would be more similar would be a consortium of large providers providing such a whitelist. That would be something I would welcome.
Something that is already being setup, and that tends to add a slight amount of reputation to any authentication schemes that might be used, is a "feedback loop" Kind of like what we do, or what AOL does (http://postmaster.info.aol.com/fbl/) Not public as such, but well it is as much like peering as anything in the smtp email world can be. [e&oe gateways to uucp and x400] -- Suresh Ramasubramanian (ops.lists@gmail.com)
Folks,
DNSWL -- this is already being done. It is not widely viewed as being in any way similar to a peering concept. What would be more similar would be a consortium of large providers providing such a whitelist. That would be something I would welcome.
To repeat what John Levine said, and that I suggested in my posting "Informal email peering" please take a look at CSV <http://mipassoc.org/csv> as a candidate mechanism for determining the operations-related identity to assess, and for a means of querying a third party to obtain an assessment. CSV is simple, uses efficient DNS records, and mostly importantly uses operations identities rather than content origination identities. Several schemes that have some popularity use a path registration approach (SPF, Sender-ID) which ties an origination identifier (rfc2822.From, rfc2822.Sender, or rfc2821.MailFrom) to the MTAs along the transmission path. For you ops folks, think of this as requiring pre-registration of all source routes to all recipients. For you architecture freaks, think of it as a really spiffy layer violation. By contrast, CSV uses identities that are directly tied to the MTA that is being assessed. Once you have a validated identity, you need a scalable means of assessing it. The combinatorial explosion with email makes pair-wise agreements unscalable. Hence, some form of third-party assessment schemes is needed. And that is what motivated the idea for <http://mipassoc.org>. Develop a common set of best practises, and have organization commit to supporting them. d/ --- Dave Crocker Brandenburg InternetWorking +1.408.246.8253 dcrocker a t ... WE'VE MOVED to: www.bbiw.net
In message <OF5546988E.7DE79026-ON80257023.0034BB4D-80257023.00357C69@radianz.c om>, Michael.Dillon@btradianz.com writes:
Similar concept, same scaling problems; it just hides the explicit routing from the user (as would any modern "peering" system, presumably).
Then you are presuming wrongly. Nowhere in what I wrote have I suggested any changes in the existing email technology.
Quite apart from anything else, this requires an email routing protocol. Getting the policy statements right in such a protocol is a non-trivial task; it will make BGP look simple, because of the implied liability a mail sender would incur under this scheme. Let me be very specific. I own a 1U server in a rack somewhere. (The concept has been discussed here many times, of course; thanks to some friends, I can actually do it.) How do I send email? Well, maybe the colo operator has a mail server. Maybe the rack operator has mail-forwarding contracts with his upstream IP (i.e., BGP) peering providers. To whom can they send mail? More precisely, with whom have they signed contracts that will let them deliver mail, and under what conditions? Maybe one of the upstreams has better contracts in some countries than the other does, or maybe the other will charge me less for delivering my email, but with a hefty chargeback to me if it's found to be spam. Or maybe one of them won't talk to (or listen to) mailers in certain parts of the world -- we've seen that alleged and/or bragged about on this list -- because of perceptions of where spam comes from. But the better mail server I'd prefer to use is down today, because of a fiber cut/DDoS attack/spam overload. How do I know, and how do I fall back to an alternate? We can all invent more scenarios. I think we can all see the analogies to BGP, too -- and we all know how hard it is to get that to do what we want. The scheme you're suggesting might work without new protocols in a purely hierarchical world. It might even work with a fully-connected cluster of Tier 1s, each of which is a tree root. But the Internet doesn't work that way, or we'd all be using EGP for routing. Besides, as I mentioned the other day, there are policy side-effects. See http://news.com.com/Your+ISP+as+Net+watchdog/2100-1028_3-5748649.html?tag=ne... for an example of the kind of thing I'm worried about. --Steven M. Bellovin, http://www.cs.columbia.edu/~smb
On Fri, 17 Jun 2005 Michael.Dillon@btradianz.com wrote:
Similar concept, same scaling problems; it just hides the explicit routing from the user (as would any modern "peering" system, presumably).
Then you are presuming wrongly. Nowhere in what I wrote have I suggested any changes in the existing email technology. I am not suggesting that we drop SMTP in favour of your favourite old dusty protocol. I am suggesting that we need a system of accountability for people who run Internet email servers based on contracts and SLAs, i.e. peering agreements.
From domain, for example by MX record or by reverse DNS (we implemented
In between the choice of accepting mail from *anybody* by default which we have now and the choice of accepting mail from *nobody* by default that explicit peering agreements represents there is another solution; which is to accept mail only from IPs that have *some relation* to the sender's that test and call it MX+). Here is a downloadable reference implementation for use with procmail: http://mxplus.org/ The example program mxplus is code that was carved out of the mail server software we use and made standalone. It's an antispam option that works well for many users. The example includes sender email address validation, which is another test like MX+ that works well for most users and breaks under usually acceptable circumstances when senders do bad things like send email with an invalid From address. YMMV. Mike. +----------------- H U R R I C A N E - E L E C T R I C -----------------+ | Mike Leber Direct Internet Connections Voice 510 580 4100 | | Hurricane Electric Web Hosting Colocation Fax 510 580 4151 | | mleber@he.net http://www.he.net | +-----------------------------------------------------------------------+
In between the choice of accepting mail from *anybody* by default which we have now and the choice of accepting mail from *nobody* by default that explicit peering agreements represents there is another solution; which is to accept mail only from IPs that have *some relation* to the sender's From domain, for example by MX record or by reverse DNS (we implemented that test and call it MX+).
This has the same problem as all of the other duct tape authorization schemes -- it breaks a lot of valid e-mail, so that you have to maintain a painfully large manual exception table, or write off a lot of mail that your users will not forgive you for losing, or more likely, both. In this particular case, the biggest issue is forwarders, commercial ones like pobox.com, associations like the ACM and IEEE (I get some odd mail being uucp at computer.org), and large numbers of colleges and universities which let graduates keep their email address. In all of those cases, the users send mail from their own ISPs, whatever they are, inbound mail is forwarded back to the ISP accounts, and there is no way to enumerate the valid sources of mail. There's also plenty of domains where the inbound and outbound mail servers are different, and neither one matches the domain name of the mail. For example, I host about 300 small mail domains on a pop toaster here. The MX is mail2.iecc.com, and the outbound host that many but not all of them use is xuxa.iecc.com. (Mail for iecc.com itself is on another host.) The IPs all happen to be in the same /24, but guessing whether two IPs are "close enough" is a poor way to authenticate or authorize anything. Before you point out that they could change the way those systems work to be compatible with your scheme, well, duh, sure. But if you're going to make people change their existing working mail setups, there's little point in going through the vast cost of a widespread change for such a marginal benefit. Read archives of SPF mailing lists for endless flamage on this topic, since SPF has the same problem. Regards, John Levine, johnl@iecc.com, Primary Perpetrator of "The Internet for Dummies", Information Superhighwayman wanna-be, http://www.johnlevine.com, Mayor "A book is a sneeze." - E.B. White, on the writing of Charlotte's Web
On 18 Jun 2005, John Levine wrote:
In between the choice of accepting mail from *anybody* by default which we have now and the choice of accepting mail from *nobody* by default that explicit peering agreements represents there is another solution; which is to accept mail only from IPs that have *some relation* to the sender's From domain, for example by MX record or by reverse DNS (we implemented that test and call it MX+).
This has the same problem as all of the other duct tape authorization schemes -- it breaks a lot of valid e-mail, so that you have to maintain a painfully large manual exception table, or write off a lot of mail that your users will not forgive you for losing, or more likely, both.
In this particular case, the biggest issue is forwarders, commercial ones like pobox.com, associations like the ACM and IEEE (I get some odd mail being uucp at computer.org), and large numbers of colleges and universities which let graduates keep their email address. In all of those cases, the users send mail from their own ISPs, whatever they are, inbound mail is forwarded back to the ISP accounts, and there is no way to enumerate the valid sources of mail.
This gets into the discussion of what percentage of mail a user gets that is like this. It varies widely. From our measurements for most users this is less than 1 percent. For me or you its definitely much higher than 1 percent (I definitely agree with you).
There's also plenty of domains where the inbound and outbound mail servers are different, and neither one matches the domain name of the mail. For example, I host about 300 small mail domains on a pop toaster here. The MX is mail2.iecc.com, and the outbound host that many but not all of them use is xuxa.iecc.com. (Mail for iecc.com itself is on another host.) The IPs all happen to be in the same /24, but guessing whether two IPs are "close enough" is a poor way to authenticate or authorize anything.
In between the two extremes of spam growing at the rate of Moores law and not using email anymore there are all sorts of solutions. Pick a solution or solutions that you like, or not. Virtually all of them will result in some sort of reduction in the current ability of anybody being able to send mail as anyone from anywhere.
Before you point out that they could change the way those systems work to be compatible with your scheme, well, duh, sure.
Nope wasn't going to. They'll break when sending to people who filter or reject based on MX+.
But if you're going to make people change their existing working mail setups, there's little point in going through the vast cost of a widespread change for such a marginal benefit. Read archives of SPF mailing lists for endless flamage on this topic, since SPF has the same problem.
Oh yeah. The only different here is that this is simpler, doesn't require any interpretation of the SPF/Microsoft policy/patent wars, and doesn't involve a complex virtually turning complete macro language embeded in DNS. You don't have to understand any new DNS entries because there are no special DNS entries with MX+. With regards to the marginal benefit, this a measurement thing that only you can determine for your specific userbase. We can measure the results from MX+ and greylisting for our users. For the people that use it, they are really happy. Perhaps they correspond with more users of large ISPs and companies that satisfy the test criteria. If you assert that it won't work for your users, that's completely within the realm of normal decision making, who am I to tell you what is going to work for your users. You only have so much time in the day, you have to make your own judgement calls for whatever techniques to make available to your users. Mike. +----------------- H U R R I C A N E - E L E C T R I C -----------------+ | Mike Leber Direct Internet Connections Voice 510 580 4100 | | Hurricane Electric Web Hosting Colocation Fax 510 580 4151 | | mleber@he.net http://www.he.net | +-----------------------------------------------------------------------+
This has the same problem as all of the other duct tape authorization schemes -- it breaks a lot of valid e-mail, ...
In this particular case, the biggest issue is forwarders, ...
This gets into the discussion of what percentage of mail a user gets that is like this. It varies widely.
Agreed. Around here (Ithaca NY) it's probably on the order of 20% due to all of the Cornell grads who still use their cornell.edu address.
Pick a solution or solutions that you like, or not. Virtually all of them will result in some sort of reduction in the current ability of anybody being able to send mail as anyone from anywhere.
Right. That's why for widespread deployment we have to look at the small minority of schemes that don't break legitimate mail. That's why I'm looking at CSV, which makes it easier to assign reputation to sending hosts, and domainkeys (or whatever it's called when they're done mixing in IIM) which if sensibly used makes it easier to whitelist mail from people you like. R's, John
My e-mail is alex@relcom.net, but I send it when I am on DSL with EthLink (and thru Earthlink SMTP). And it is 100% valid situation. ----- Original Message ----- From: "John Levine" <johnl@iecc.com> To: <nanog@nanog.org> Cc: <mleber@he.net> Sent: Saturday, June 18, 2005 12:25 AM Subject: Re: Email peering
In between the choice of accepting mail from *anybody* by default which we have now and the choice of accepting mail from *nobody* by default that explicit peering agreements represents there is another solution; which is to accept mail only from IPs that have *some relation* to the sender's From domain, for example by MX record or by reverse DNS (we implemented that test and call it MX+).
This has the same problem as all of the other duct tape authorization schemes -- it breaks a lot of valid e-mail, so that you have to maintain a painfully large manual exception table, or write off a lot of mail that your users will not forgive you for losing, or more likely, both.
In this particular case, the biggest issue is forwarders, commercial ones like pobox.com, associations like the ACM and IEEE (I get some odd mail being uucp at computer.org), and large numbers of colleges and universities which let graduates keep their email address. In all of those cases, the users send mail from their own ISPs, whatever they are, inbound mail is forwarded back to the ISP accounts, and there is no way to enumerate the valid sources of mail.
There's also plenty of domains where the inbound and outbound mail servers are different, and neither one matches the domain name of the mail. For example, I host about 300 small mail domains on a pop toaster here. The MX is mail2.iecc.com, and the outbound host that many but not all of them use is xuxa.iecc.com. (Mail for iecc.com itself is on another host.) The IPs all happen to be in the same /24, but guessing whether two IPs are "close enough" is a poor way to authenticate or authorize anything.
Before you point out that they could change the way those systems work to be compatible with your scheme, well, duh, sure. But if you're going to make people change their existing working mail setups, there's little point in going through the vast cost of a widespread change for such a marginal benefit. Read archives of SPF mailing lists for endless flamage on this topic, since SPF has the same problem.
Regards, John Levine, johnl@iecc.com, Primary Perpetrator of "The Internet for
Dummies",
Information Superhighwayman wanna-be, http://www.johnlevine.com, Mayor "A book is a sneeze." - E.B. White, on the writing of Charlotte's Web
On 19/06/05, Alexei Roudnev <alex@relcom.net> wrote:
My e-mail is alex@relcom.net, but I send it when I am on DSL with EthLink (and thru Earthlink SMTP). And it is 100% valid situation.
Sure is - which is one reason why spf just is not going to work for you CSV on the other hand makes no claim about the sender - it looks at the sending host
There are, however, three very big problems. First, it forces people to
pay for services that they don't pay for today.
Businesses often pay, not for services, but for accountability. They want someone else to take responsibility for a problem even if it costs them more money than taking that responsibility on themselves. Insurance, maintenance contracts, etc. Today, if Joe Business gets lots of spam, it is not his ISP's responsibility. He has no-one to take responsibility for this problem off his hands. But if he only accepts incoming email through an operator who is part of the email peering network, he knows that somewhere there is someone who will take responsibility for the problem. That is something that businesses will pay for. But first, ISPs have to put their hands up and take collective responsibility for Internet email as a service that has value and not just as some kind of loss leader for Internet access services. --Michael Dillon
Michael.Dillon@btradianz.com wrote:
Today, if Joe Business gets lots of spam, it is not his ISP's responsibility. He has no-one to take responsibility for this problem off his hands. But if he only accepts incoming email through an operator who is part of the email peering network, he knows that somewhere there is someone who will take responsibility for the problem.
That is something that businesses will pay for.
Just look at the Postini numbers... Pete
But first, ISPs have to put their hands up and take collective responsibility for Internet email as a service that has value and not just as some kind of loss leader for Internet access services.
--Michael Dillon
On 16/06/05, Stephane Bortzmeyer <bortzmeyer@nic.fr> wrote:
The proponents of "email peering" typically want to switch from the current model (millions of independant email servers) to a different model, with only a few big actors.
"Should anyone be allowed to operate an email system? Perhaps not." Carl Hutzler http://www.circleid.com/article/917_0_1_0_C/
I just don't see where Carl advocates email peering there. More like "should J Random Luser be given control of mailservers" or "Should Wile E Coyote be allowed to buy Dynamite and gadgets from the Acme Company?" That, and "if you want to operate a mailserver, get a static IP" --srs -- Suresh Ramasubramanian (ops.lists@gmail.com)
participants (19)
-
Alexei Roudnev -
Dave Crocker -
Joe Abley -
Joe Maimon -
John Levine -
Jon Lewis -
Michael.Dillon＠btradianz.com -
Mike Leber -
Niels Bakker -
Petri Helenius -
Randy Bush -
Robert E.Seastrom -
Stephane Bortzmeyer -
Steve Gibbard -
Steven M. Bellovin -
sthaug＠nethelp.no -
Suresh Ramasubramanian -
Todd Vierling -
william(at)elan.net