Yahoo and their mail filters..
This may be old news, but I've not been in the list for quite some time. At any rate, is anyone else having issues with Yahoo blocking / deferring legitimate emails? My situation is that I host our corporate mx'ers on my network, one of the companies that we recently purchased has Yahoo hosting their domains mail. Mail traffic to them is getting temporarily deferred with the "421 4.7.0 [TS01] Messages from xxx.xxx.xxx.xxx temporarily deferred due to user complaints - 4.16.55.1; see http://postmaster.yahoo.com/421-ts01.html" The admin of the facility has contacted Yahoo about this but their response was for "more information" when they were told that traffic from my mx to their domain was to being deferred. I may end up just having them migrate to my systems just to maintain company communications if we can't clear this up in a timely manner. -- Micheal Patterson
Ditto. They appear to use some strange form of greylisting combined with blocking. What seems to help is SPF and PTRs that match the EHLO your MTAs will send. We didn't implement Domain Keys / DKIM. On a related note, don't get me started on Hotmail. They used to (still do?) silently swallow mail into a black hole after accepting it. No NDR, no spam folder, just good ol' mail shredding without anyone knowing. Again, SPF and PTRs seem to help. Oh yeah, make sure you're not sending spam to them. That might help too. ;) Erik ________________________________________ From: Joe Abley [jabley@hopcount.ca] Sent: Tuesday, February 24, 2009 9:41 PM To: Micheal Patterson Cc: nanog@nanog.org Subject: Re: Yahoo and their mail filters.. On 24 Feb 2009, at 21:27, Micheal Patterson wrote:
This may be old news, but I've not been in the list for quite some time. At any rate, is anyone else having issues with Yahoo blocking / deferring legitimate emails?
Yes. Everybody else. Joe
----- Original Message ----- From: "Erik (Caneris)" <erik_list@caneris.com> To: "Joe Abley" <jabley@hopcount.ca>; "Micheal Patterson" <micheal@spmedicalgroup.com> Cc: <nanog@nanog.org> Sent: Tuesday, February 24, 2009 9:11 PM Subject: RE: Yahoo and their mail filters.. Ditto. They appear to use some strange form of greylisting combined with blocking. What seems to help is SPF and PTRs that match the EHLO your MTAs will send. We didn't implement Domain Keys / DKIM. On a related note, don't get me started on Hotmail. They used to (still do?) silently swallow mail into a black hole after accepting it. No NDR, no spam folder, just good ol' mail shredding without anyone knowing. Again, SPF and PTRs seem to help. Oh yeah, make sure you're not sending spam to them. That might help too. ;) Erik SPF records aren't being recognized, I've been running them for some time now so it would seem that they're not honoring them. -- Micheal Patterson
On Wed, Feb 25, 2009 at 9:23 AM, Micheal Patterson <micheal@spmedicalgroup.com> wrote:
SPF records aren't being recognized, I've been running them for some time now so it would seem that they're not honoring them.
Christ .. Yahoo did say "complaints". And it can take a very low level of complaints before a block goes into place - especially for low volume (corporate etc) mailservers. Feedback loops are one cure, and another cure is keeping complaint volumes down. * Do you have an unfiltered NAT gateway pointed to the same IP as your corporate MTA? * Do you have any large spam sources in close proximity to you? Like you are colo'd on a /28 and someone else has a /27 or /26 in the same /24 that's emitting tons of spam (assuming colo). Or you have your mailserver hosted on a dsl pool (even a business class dsl pool) in which case your server is an island of valid mail in a large swamp of virus traffic * Do you have a marketing department that might be slightly overactive? etc etc. srs
On Wed, 25 Feb 2009, Suresh Ramasubramanian wrote:
Christ .. Yahoo did say "complaints". And it can take a very low level of complaints before a block goes into place - especially for low volume (corporate etc) mailservers.
I don't think this is Yahoo reacting to spam complaints because a large number of sites (many universities, for instance) are being affected by this problem at the same time. Tony. -- f.anthony.n.finch <dot@dotat.at> http://dotat.at/ GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS. MODERATE OR GOOD.
that could occur when a. student machines are botted (for institutions not blocking outbound port 25) b. student and alumni accounts are compromised by phishers (both of these just for the purposes of sending spam from well connected, reputable institutions.) and then consumers really do complain... i'm told (not just by yahoo insiders) that the forms at postmaster.yahoo.com actually do work, eventually. On Feb 25, 2009, at 9:08 AM, Tony Finch wrote:
On Wed, 25 Feb 2009, Suresh Ramasubramanian wrote:
Christ .. Yahoo did say "complaints". And it can take a very low level of complaints before a block goes into place - especially for low volume (corporate etc) mailservers.
I don't think this is Yahoo reacting to spam complaints because a large number of sites (many universities, for instance) are being affected by this problem at the same time.
Tony. -- f.anthony.n.finch <dot@dotat.at> http://dotat.at/ GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS. MODERATE OR GOOD.
Tony Finch wrote:
On Wed, 25 Feb 2009, Suresh Ramasubramanian wrote:
Christ .. Yahoo did say "complaints". And it can take a very low level of complaints before a block goes into place - especially for low volume (corporate etc) mailservers.
I don't think this is Yahoo reacting to spam complaints because a large number of sites (many universities, for instance) are being affected by this problem at the same time.
Universities are often major sources of spam. Spam is sent directly from virus-infected student computers, and spam is also sent to students at their university email address and then .forwarded on to the student's outside (or post-university) email account - when the student receives forwarded spam at their Yahoo account and clicks "this is spam" the university is considered the "source" of the spam. jc
On Wed, 25 Feb 2009 10:44:13 PST, JC Dill said:
Universities are often major sources of spam. Spam is sent directly from virus-infected student computers,
Got any numbers to back up the claim that virus-infected student computers are anywhere near the problem that virus-infected student's-parents computers are? (I'm not saying universities are perfect - we have to nuke several users a day because their accounts or machines fall under enemy control. But I see a lot of people repeating the meme without any numbers to back it up)
on Wed, Feb 25, 2009 at 02:17:14PM -0500, Valdis.Kletnieks@vt.edu wrote:
On Wed, 25 Feb 2009 10:44:13 PST, JC Dill said:
Universities are often major sources of spam. Spam is sent directly from virus-infected student computers,
Got any numbers to back up the claim that virus-infected student computers are anywhere near the problem that virus-infected student's-parents computers are?
(I'm not saying universities are perfect - we have to nuke several users a day because their accounts or machines fall under enemy control. But I see a lot of people repeating the meme without any numbers to back it up)
Without going into comparisons between resnets and the world at large, I will say that the resnet problem has gotten a *lot* better over the past couple of years; it used to be that every new semester saw a huge uptick in botted resnet hosts, but we haven't seen anything like that for two years or so. Good work! -- hesketh.com/inc. v: +1(919)834-2552 f: +1(919)834-2553 w: http://hesketh.com/ antispam news, solutions for sendmail, exim, postfix: http://enemieslist.com/
On hotmail's defense at least their support contacts will respond to your emails. It may take a few rounds of proving that they are 'blackholing' your email and them saying 'no were not'..but after a few times of that you know exactly what to say when submitting a ticket to them (ie I sent this email to your testing account at xx:xx pm, I cc'ed my address xxx@hotmail.com and it wasn't received and here are the logs showing your servers accepted the email.). -r -----Original Message----- From: Erik (Caneris) [mailto:erik_list@caneris.com] Sent: Tuesday, February 24, 2009 10:11 PM To: Joe Abley; Micheal Patterson Cc: nanog@nanog.org Subject: RE: Yahoo and their mail filters.. Ditto. They appear to use some strange form of greylisting combined with blocking. What seems to help is SPF and PTRs that match the EHLO your MTAs will send. We didn't implement Domain Keys / DKIM. On a related note, don't get me started on Hotmail. They used to (still do?) silently swallow mail into a black hole after accepting it. No NDR, no spam folder, just good ol' mail shredding without anyone knowing. Again, SPF and PTRs seem to help. Oh yeah, make sure you're not sending spam to them. That might help too. ;) Erik ________________________________________ From: Joe Abley [jabley@hopcount.ca] Sent: Tuesday, February 24, 2009 9:41 PM To: Micheal Patterson Cc: nanog@nanog.org Subject: Re: Yahoo and their mail filters.. On 24 Feb 2009, at 21:27, Micheal Patterson wrote:
This may be old news, but I've not been in the list for quite some time. At any rate, is anyone else having issues with Yahoo blocking / deferring legitimate emails?
Yes. Everybody else. Joe
Micheal Patterson wrote:
This may be old news, but I've not been in the list for quite some time. At any rate, is anyone else having issues with Yahoo blocking / deferring legitimate emails?
My situation is that I host our corporate mx'ers on my network, one of the companies that we recently purchased has Yahoo hosting their domains mail. Mail traffic to them is getting temporarily deferred with the "421 4.7.0 [TS01] Messages from xxx.xxx.xxx.xxx temporarily deferred due to user complaints - 4.16.55.1; see http://postmaster.yahoo.com/421-ts01.html"
The admin of the facility has contacted Yahoo about this but their response was for "more information" when they were told that traffic from my mx to their domain was to being deferred. I may end up just having them migrate to my systems just to maintain company communications if we can't clear this up in a timely manner.
-- Micheal Patterson
Yep, it's been happening to us - various explanations - and I've got at least one annoyed customer because of it. -- Jeff Shultz
We found this issue to be associated usually with users forwarding email to a Yahoo account. If spam slips by our spam filters and gets forwarded where the enduser reports it as spam not realizing the impact on their actions. In the last couple of years we have been not allowing people to forward their accounts to yahoo, aol, hotmail, etc. Too much of a headache. Chuck -----Original Message----- From: Micheal Patterson [mailto:micheal@spmedicalgroup.com] Sent: Tuesday, February 24, 2009 7:28 PM To: nanog@nanog.org Subject: Yahoo and their mail filters.. This may be old news, but I've not been in the list for quite some time. At any rate, is anyone else having issues with Yahoo blocking / deferring legitimate emails? My situation is that I host our corporate mx'ers on my network, one of the companies that we recently purchased has Yahoo hosting their domains mail. Mail traffic to them is getting temporarily deferred with the "421 4.7.0 [TS01] Messages from xxx.xxx.xxx.xxx temporarily deferred due to user complaints - 4.16.55.1; see http://postmaster.yahoo.com/421-ts01.html" The admin of the facility has contacted Yahoo about this but their response was for "more information" when they were told that traffic from my mx to their domain was to being deferred. I may end up just having them migrate to my systems just to maintain company communications if we can't clear this up in a timely manner. -- Micheal Patterson
----- Original Message ----- From: "Chuck Schick" <chasjs@warp8.com> To: <nanog@nanog.org> Sent: Wednesday, February 25, 2009 12:18 PM Subject: RE: Yahoo and their mail filters..
We found this issue to be associated usually with users forwarding email to a Yahoo account. If spam slips by our spam filters and gets forwarded where the enduser reports it as spam not realizing the impact on their actions.
In the last couple of years we have been not allowing people to forward their accounts to yahoo, aol, hotmail, etc. Too much of a headache.
Chuck
I could see that if my situation was where I was forwarding to a personal yahoo account, but these are business customers that aren't able to whitelist who they recieve email from. I just checked in their domain panel and see no options of setting any whitelisting or spam settings in the yahoo's business email control panel. My current solution is to just move their email away from yahoo competely and just host it here with the rest of my corporate email users. -- Micheal Patterson
On Feb 24, 2009, at 6:27 PM, Micheal Patterson wrote:
This may be old news, but I've not been in the list for quite some time. At any rate, is anyone else having issues with Yahoo blocking / deferring legitimate emails?
My situation is that I host our corporate mx'ers on my network, one of the companies that we recently purchased has Yahoo hosting their domains mail. Mail traffic to them is getting temporarily deferred with the "421 4.7.0 [TS01] Messages from xxx.xxx.xxx.xxx temporarily deferred due to user complaints - 4.16.55.1; see http://postmaster.yahoo.com/421-ts01.html"
The admin of the facility has contacted Yahoo about this but their response was for "more information" when they were told that traffic from my mx to their domain was to being deferred. I may end up just having them migrate to my systems just to maintain company communications if we can't clear this up in a timely manner.
-- Micheal Patterson
A few comments on this thread in general (speaking only for myself, not in any way representing my employer)... Yes, Yahoo! tend to throttle IPs at the drop of a hat, but those blocks are often gone in a few hours as well. Others have pointed out some procedures to follow to minimize the possibility of being blocked. At least they give you a useable SMTP error (usually). Incidentally this is why all my test accounts are on Gmail, because delivery to Yahoo! is often deferred for minutes to hours. Of course, given the recent Gmail outages I might have to diversify even more... As for "blackholes" that messages fall into, what is the alternative? You could say reject it in session with a readable error, but that would give spammers instant confirmation on whether their campaign is working. Also, the majority of anti-spam products I've seen have to spool the message before they scan it, so rejecting in session is simply not an option on a lot of commercial platforms. The other options is to stuff all the spam messages in a folder and expose them to the user, taking up a huge amount of storage space for something the vast majority of users are never going to look at any way. Again, a lot of commercial solutions have a scoring methodology where you can be pretty certain stuff at the top end of the scale is virtually never going to be a false positive. The amount of savings in not having to handle and store that crud massively outweighs one or two users missing a newsletter once in a while. It can make sense to expose the "mid-range spam" to users and let them decide, but why store terabytes of stuff that only a tiny fraction of the users may ever care about? If you're sending important mail that's not reaching the recipient, and you have the server logs to prove you handed it off to the destination MTA, open a ticket with them and they'll have logs to track it down. Regarding taking automatic action based on luser feedback, that is ridiculous in my opinion. From the data I see, the lusers classify mail incorrectly far more than correctly. In fact there's a running joke around here that we should simply flip the false-positive and false-negative feeds and enable auto-train, since the only thing you can reliably count on users to do is get things wrong. Submissions from administrators are _far_ more accurate (although even then, not to the point that it always makes sense to take automatic action). Blocking an entire site just because one John Doe user clicked a button they don't even understand just does not make sense. Last, anywhere that I've seen extensive use of forwards has had a maze of difficult to untangle abuse problems related to forwarded spam. Any site allowing forwarding should apply very robust filtering of outbound mail. -- bk
Outbound filtering is a good idea..however after investing lots of money on hardware appliances (old company $100,000 on equipment to do just this...) you realize you have more issues then solutions. Now you allow forwarded mail, and as you stated most systems accept the messages into the queue process the message and then either bounce/quarentine/allow. You can't bounce the message because it goes back to the sender which is almost always spoofed and thus you create backscatter. You cant quarentine because then you may flag some of your customers legitimate email. Isolating your forwarded mail to a separate ip address is really, I think, the best way to handel forwarded mail. -r -----Original Message----- From: Brian Keefer [mailto:chort@smtps.net] Sent: Wednesday, February 25, 2009 3:48 PM To: Micheal Patterson Cc: nanog@nanog.org Subject: Re: Yahoo and their mail filters.. On Feb 24, 2009, at 6:27 PM, Micheal Patterson wrote:
This may be old news, but I've not been in the list for quite some time. At any rate, is anyone else having issues with Yahoo blocking / deferring legitimate emails?
My situation is that I host our corporate mx'ers on my network, one of the companies that we recently purchased has Yahoo hosting their domains mail. Mail traffic to them is getting temporarily deferred with the "421 4.7.0 [TS01] Messages from xxx.xxx.xxx.xxx temporarily deferred due to user complaints - 4.16.55.1; see http://postmaster.yahoo.com/421-ts01.html"
The admin of the facility has contacted Yahoo about this but their response was for "more information" when they were told that traffic from my mx to their domain was to being deferred. I may end up just having them migrate to my systems just to maintain company communications if we can't clear this up in a timely manner.
-- Micheal Patterson
A few comments on this thread in general (speaking only for myself, not in any way representing my employer)... Yes, Yahoo! tend to throttle IPs at the drop of a hat, but those blocks are often gone in a few hours as well. Others have pointed out some procedures to follow to minimize the possibility of being blocked. At least they give you a useable SMTP error (usually). Incidentally this is why all my test accounts are on Gmail, because delivery to Yahoo! is often deferred for minutes to hours. Of course, given the recent Gmail outages I might have to diversify even more... As for "blackholes" that messages fall into, what is the alternative? You could say reject it in session with a readable error, but that would give spammers instant confirmation on whether their campaign is working. Also, the majority of anti-spam products I've seen have to spool the message before they scan it, so rejecting in session is simply not an option on a lot of commercial platforms. The other options is to stuff all the spam messages in a folder and expose them to the user, taking up a huge amount of storage space for something the vast majority of users are never going to look at any way. Again, a lot of commercial solutions have a scoring methodology where you can be pretty certain stuff at the top end of the scale is virtually never going to be a false positive. The amount of savings in not having to handle and store that crud massively outweighs one or two users missing a newsletter once in a while. It can make sense to expose the "mid-range spam" to users and let them decide, but why store terabytes of stuff that only a tiny fraction of the users may ever care about? If you're sending important mail that's not reaching the recipient, and you have the server logs to prove you handed it off to the destination MTA, open a ticket with them and they'll have logs to track it down. Regarding taking automatic action based on luser feedback, that is ridiculous in my opinion. From the data I see, the lusers classify mail incorrectly far more than correctly. In fact there's a running joke around here that we should simply flip the false-positive and false-negative feeds and enable auto-train, since the only thing you can reliably count on users to do is get things wrong. Submissions from administrators are _far_ more accurate (although even then, not to the point that it always makes sense to take automatic action). Blocking an entire site just because one John Doe user clicked a button they don't even understand just does not make sense. Last, anywhere that I've seen extensive use of forwards has had a maze of difficult to untangle abuse problems related to forwarded spam. Any site allowing forwarding should apply very robust filtering of outbound mail. -- bk
Brian Keefer wrote:
The other options is to stuff all the spam messages in a folder and expose them to the user, taking up a huge amount of storage space for something the vast majority of users are never going to look at any way.
Which is, in fact, what Yahoo! does by default. Users have the option to have that stuff deleted immediately, should they desire.
Blocking an entire site just because one John Doe user clicked a button they don't even understand just does not make sense.
You're right -- but Yahoo! has a sufficiently large userbase that they can count multiple complaints before blocking anything. Same story with AOL, and Hotmail, and Cloudmark, and many others who've used this technique for years. In all of those cases, they have safeguards to prevent gaming, to prevent bouncing, and pretty much everything else anyone's suggested thus far in this thread.
Last, anywhere that I've seen extensive use of forwards has had a maze of difficult to untangle abuse problems related to forwarded spam. Any site allowing forwarding should apply very robust filtering of outbound mail.
Very true. MAAWG published a document last year which includes some additional recommendations: http://www.maawg.org/about/publishedDocuments/MAAWG_Email_Forwarding_BP.pdf -- J.D. Falk Return Path Inc http://www.returnpath.net/
On Feb 26, 2009, at 5:08 PM, J.D. Falk wrote:
Blocking an entire site just because one John Doe user clicked a button they don't even understand just does not make sense.
You're right -- but Yahoo! has a sufficiently large userbase that they can count multiple complaints before blocking anything. Same story with AOL, and Hotmail, and Cloudmark, and many others who've used this technique for years.
This does not appear to be the case from external observation. It may be in some cases that multiple reports are necessary, but it certainly seems there are hair-triggers in others. For instance, see the message from Eric Esslinger. As for not black-holing anything, I haven't personally verified with Yahoo!, but others have reported that they do. It's pretty common from what I've seen to simply make very high-scored messages disappear and only send the mid-range stuff to the spam folder. Hotmail, as mentioned, does this. One of the very large hosted filtering services does as well. I'm not saying it's bad (it makes sense if you can trust your scoring algorithm), but it does happen. Just because you get _some_ stuff in your spam folder doesn't mean that's all the spam that was blocked. -- bk
This discussion is probably *much* more appropriate on the mailop list. (It's been mentioned there and on other MTA/spam-related lists, as apparently whatever Yahoo's doing is having widespread impact.) ---Rsk
very old news. their filter restrictions have some very absurd rules On Tue, Feb 24, 2009 at 9:27 PM, Micheal Patterson < micheal@spmedicalgroup.com> wrote:
This may be old news, but I've not been in the list for quite some time. At any rate, is anyone else having issues with Yahoo blocking / deferring legitimate emails?
My situation is that I host our corporate mx'ers on my network, one of the companies that we recently purchased has Yahoo hosting their domains mail. Mail traffic to them is getting temporarily deferred with the "421 4.7.0 [TS01] Messages from xxx.xxx.xxx.xxx temporarily deferred due to user complaints - 4.16.55.1; see http://postmaster.yahoo.com/421-ts01.html"
The admin of the facility has contacted Yahoo about this but their response was for "more information" when they were told that traffic from my mx to their domain was to being deferred. I may end up just having them migrate to my systems just to maintain company communications if we can't clear this up in a timely manner.
-- Micheal Patterson
participants (16)
-
Brian Keefer -
Carl Ford -
Chuck Schick -
Erik (Caneris) -
J.D. Falk -
JC Dill -
Jeff Shultz -
Joe Abley -
mark seiden-via mac -
Micheal Patterson -
Ray Corbin -
Rich Kulawiec -
Steven Champeon -
Suresh Ramasubramanian -
Tony Finch -
Valdis.Kletnieks＠vt.edu