-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 10, 2008, at 1:35 PM, Jeff Shultz wrote:

This thread got me checking logs and I just spotted several of those "deferred due to user complaints" tags. And compared to them, we're tiny. Don't know if it's widespread, but it appears you are not the only one so blessed.

We've seen this before too but this week it has been different. Every single host that relays email on our network has these in the queue. Now a couple of them do mailing lists and such so I could see it happening but a couple of them don't do anything high volume at all. For some of them some mail goes through but only some of the time. It seems like if we hit the right MX machine it works and other times it does not. We tried going around them by sending mail over to an employee's personal mail server (which does nearly no volume at all) but even it is blocked probably 1/2 the time. I'm not sure what is going on but given all this I can't believe it is just "normal". We filled out one of those forms but just got back a response that said it wasn't happening but if it was we should see their "best practices" URL. Only problem is we actually do everything on their list (including both DomainKeys and DKIM). Chris ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Chris Owen ~ Garden City (620) 275-1900 ~ Lottery (noun): President ~ Wichita (316) 858-3000 ~ A stupidity tax Hubris Communications Inc www.hubris.net ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) Comment: Public Key: http://home.hubris.net/owenc/pgpkey.txt Comment: Public Key ID: 0xB513D9DD iEYEARECAAYFAkf+bTQACgkQElUlCLUT2d3lPACeLoNzc790rnHxNAtPEdnpFDpX yAoAoKkMZlw4zX/yzgRsiiJOdD6wCbph =YXy4 -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Raymond L. Corbin wrote:

Hello,

I have had to tell some dedicated server clients that they will need to disable their forwards to Yahoo or add something like postini for those accounts that forward to Yahoo...It generally works...however Yahoo! for the past three months is now blocking entire /24's if a few IP's get complaints. They have the feedback loops however when you have a network with 175,000 IP addresses and you sign up for a feedback loop for them all they tend to flood your abuse desk with false positives, or forwarded spam. They also don't keep track of which IP's are getting the complaints for you to investigate after the block on the /24 so asking them won't help :(. This potentially means one customer could easily effect the other customer. They offer whitelisting, but this won't get you passed their blocks on the entire /24. They apparently will eventually accept the message because they aren't necessarily 'blocked' but they are 'depriortized' meaning they don't believe your IP is importan t enough to deliver the message at that time, so they want you to keep trying and when their servers are not 'busy' or 'over loaded' they will accept the message. (Paraphrased from conversations with their 'Bulk Mail Advocacies and Anti-Abuse manager.)

I've had to tell some of our customers the same and that if they wanted to continue the forwarding to their yahoo.com accounts, they'd need to add spam filtering to their accounts here so that the crap is not forwarded, resulting in the email delays for all customers. Works for some and generated more revenue.... ;-) Chris -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mandriva - http://enigmail.mozdev.org iD8DBQFH/muAnSVip47FEdMRCthkAKCW80FIV2FvdctuCxT3JYI2q0MyfACfai2t YkgPN/PGEmxsS6tJplWKg90= =p9F7 -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Raymond L. Corbin wrote:

Yeah, but without them saying which IP's are causing the problems you can't really tell which servers in a datacenter are forwarding their spam/abusing Yahoo. Once the /24 block is in place then they claim to have no way of knowing who actually caused the block on the /24. The feedback loop would help depending on your network size. When you have a few hundred thousand clients, and those clients have clients, and they even have client, it simply floods your abuse desk with complaints from Yahoo when it is obviously forwarded spam. So it's more of pick your poison deal with customer complaints about not being able to send to yahoo for a few days or get your abuse desk flooded with complaints which hinders solving actual issues like compromised accounts.

I look at all my mail server log files and see which logs show obvious spam being forwarded (a lot of times the MAIL FROM address is a dead giveaway) or I tail -F the mail log for a bit and watch the spam coming in and forwarding back out. When I see the forwarding domain that's who I have contacted to upsell some spam filtering. But, we're a small ISP, so I don't have thousands, let alone hundreds of thousands of clients, to deal with... Chris -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mandriva - http://enigmail.mozdev.org iD8DBQFH/nORnSVip47FEdMRCi+HAJ9CJoJ/VAkEssv6TznwcYQVGVWkIACfRwhI VYw0v4HWI8mWs2SHEF3jnq0= =YMQR -----END PGP SIGNATURE-----

Barry Shein wrote:

...

Is it just us or are there general problems with sending email to yahoo in the past few weeks? Our queues to them are backed up though they drain slowly.

They frequently return:

421 4.7.0 [TS01] Messages from MAILSERVERIP temporarily deferred due to user complaints - 4.16.55.1; see http://postmaster.yahoo.com/421-ts01.html

(where MAILSERVERIP is one of our mail server ip addresses) .... Just wondering if this was a widespread problem or are we just so blessed, and any insights into what's going on over there.

I see this a lot also and what I see causing it is accounts on my servers that don't opt for spam filtering and they have their accounts here set to forward mail to their yahoo.com accounts - spam and everything then gets sent there - they complain to yahoo.com about the spam and bingo - email delays from here to yahoo.com accounts....

We had this happen when a user forwarded a non-filtered mail stream from here to Yahoo. The user indicated that no messages were reported to Yahoo as spam, despite the fact that it's certain some of them were spam. I wouldn't trust the error message completely. It seems likely that a jump in volume may trigger this too, especially of an unfiltered stream. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net "We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.

Frank Bulk wrote:

Q> Does Yahoo! use "greylisting" to reject messages?

A> No. The most commonly understood form of "greylisting" is where an SMTP server will reject every message the first time it is attempted, and then accept it if the sending server retries later. The theory is that spammers won't retry messages, while legitimate senders will.

Yahoo! does not utilize this method.

http://help.yahoo.com/l/us/yahoo/mail/postmaster/postmaster-05.html

Whatever they call it is immaterial. The end result to our system is indistinguishable from real greylisting. Perhaps there's a tiny fraction that aren't ever deferred, but in general I find the majority of our queue is destined to @yahoo.com addresses. I think I'll followup on the other posters ideas of: 1) Implementing a separate outbound gateway just for yahoo.com 2) Advising users to switch to gmail.

Edward B. DREGER wrote:

FB> Date: Thu, 10 Apr 2008 13:48:52 -0500 FB> From: Frank Bulk

FB> Q> Does Yahoo! use "greylisting" to reject messages? FB> A> No. FB> FB> http://help.yahoo.com/l/us/yahoo/mail/postmaster/postmaster-05.html

First-hand observations trump claims.

Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita ________________________________________________________________________ DO NOT send mail to the following addresses: davidc@brics.com -*- jfconmaapaq@intc.net -*- sam@everquick.net Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.

I see this 'not greylisting' and we are a fairly tiny operation (I had about 20k outbound emails last week) and don't allow people to autoforward email from our servers. I find that yahoo.com destined mail stays in the queue for about 48 hours, with a 1,1,2,4,4,4,4,etc... hour retry. As I tend to peak at a couple of hundred mail in the queue when things get busy, it's not a problem, but can see where it would be for larger operators. -- Eric Esslinger Information Services Manager Fayetteville Public Utilities Fayetteville, TN 37334 Phone: 931-433-1522x165 Fax: 931-433-0646 eesslinger@fpu-tn.com

On Thu, Apr 10, 2008 at 03:47:34PM -0400, Raymond L. Corbin wrote:

I think it took a few weeks for me to get a reply through that system...I believe their 'Bulk Mail Advocacy' said they are typically 72hours. Try increasing your retries to extend beyond that.

An anonymous source at Yahoo told me that they have pushed a config update sometime today out to their servers to help with these deferral issues. Please don't ask me to play proxy on this one of any other issues you may have, but take a look at your queues and they should be getting better. - Jared -- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.

I've talked to employees in other departments who agree that something needs changed (especially when their own mail wasn't making it to their personal yahoo inboxes) You can reach yahoo's 'mail' department(s) after doing a lot of digging and googling... Their ' Bulk Mail Advocacy Agent' was somewhat helpful, but the anti-abuse manager seemed to get things done after you at least try the proper channels of submitting a ticket and waiting about a week and still having no resolve...I submitted a ticket to them to update my whitelisted IP's from adding/removing servers and it took about a month to get a reply. AOL's postmaster is easy to reach via their 1-800# however they seem to read off the screen and are really only general support. Their actual 'postmasters' (once you get passed their general support) are usually pretty helpful and quick to resolve issues. -Ray -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of chuck goolsbee Sent: Thursday, April 10, 2008 8:51 PM To: nanog@merit.edu Subject: Re: Yahoo Mail Update

An anonymous source at Yahoo told me that they have pushed a config update sometime today out to their servers to help with these deferral issues.

Please don't ask me to play proxy on this one of any other issues you may have, but take a look at your queues and they should be getting better.

- Jared

Thanks for the update Jared. I can understand your request to not be used as a proxy, but it exposes the reason why Yahoo is thought to be clueless: They are completely opaque. They can not exist in this community without having some visibity and interaction on an operational level. Yahoo should have a look at how things are done at AOL. While the feedback loop from the *users* at AOL is mostly a source of entertainment, dealing with the postmaster staff at AOL is a benchmark in how it should be done. Proxy that message over and perhaps this issue of Yahoo's perennially broken mail causing the rest of us headaches will go away. It seems to come up here on nanog and over on the mailop list every few weeks. --chuck

On Thu, Apr 10, 2008 at 8:54 PM, Rich Kulawiec <rsk@gsp.org> wrote:

On Thu, Apr 10, 2008 at 05:51:23PM -0700, chuck goolsbee wrote:

...

Thanks for the update Jared. I can understand your request to not be used as a proxy, but it exposes the reason why Yahoo is thought to be clueless: They are completely opaque.

They can not exist in this community without having some visibity and interaction on an operational level.

I heartily second this. Yahoo (and Hotmail) (and Comcast and Verizon) mail system personnel should be actively participating here, on mailop, on spam-l, etc. A lot of problems could be solved (and some avoided) with some interaction.

---Rsk

Why should large companies participate here about mail issues? Last I checked this wasn't the mailing list for these issues: "NANOG is an educational and operational forum for the coordination and dissemination of technical information related to backbone/enterprise networking technologies and operational practices." But lets just say for a second this is the place to discuss company xys's mail issue. What benefit do they have participating here? Likely they'll be hounded by people who have some disdain for their company and no matter what they do they will still be evil or wrong in some way. It is easy for someone who has 10,000 users to tell someone who has 50 million users what to do when they don't have to work with such a large scale enterprise. I find it funny when smaller companies always tell larger companies what they need to be doing. -- Ross ross [at] dillio.net 314-558-6455

On Sun, Apr 13, 2008 at 8:24 PM, Martin Hannigan <hannigan@gmail.com> wrote:

Having some provider or group(MAAWG?) explain the new and improved overhead driven mail/abuse desk would make an excellent NANOG presentation, IMHO, and it could include a V6 slant like "and to handle V6 abuse issues the plan is.....".

MAAWG spent three entire meetings drafting this - and a very interactive drafting process it was too (hang flipcharts on the walls, each with a key question, people circulate around the room with marker pens, write their ideas. Other people rate these ideas. The flipcharts are then taken down, the contents edited to produce a BCP Here's the abuse desk management BCP - one that includes several things that I personally regard as a very good idea indeed - http://www.maawg.org/about/publishedDocuments/Abuse_Desk_Common_Practices.pd... And by the time v6 actually gets used for exchanging email except between "guy with personal colo and a tunneled /48, and freebsd.org / isc.org etc hosted lists" .. you'll probably find that the basic concepts of filtering remain much the same, v4, v6 (or perhaps even Jim Fleming's or that Chinese vendor's IPv9) srs -- Suresh Ramasubramanian (ops.lists@gmail.com)

On Sun, Apr 13, 2008 at 3:24 PM, Rich Kulawiec <rsk@gsp.org> wrote:

On Sun, Apr 13, 2008 at 12:58:59AM -0500, Ross wrote:

...

On Thu, Apr 10, 2008 at 8:54 PM, Rich Kulawiec <rsk@gsp.org> wrote:

...

...

I heartily second this. Yahoo (and Hotmail) (and Comcast and Verizon) mail system personnel should be actively participating here, on mailop, on spam-l, etc. A lot of problems could be solved (and some avoided) with some interaction.

...

Why should large companies participate here about mail issues? Last I checked this wasn't the mailing list for these issues:

It's got nothing to do with size ("large"); Joe's ISP in Podunk should be on this lists as well. And one of the reasons I suggested multiple lists is that each has its own focus, so those involved with the care and feeding of mail systems should probably be on a number of them, in order to interact with something approximating the right set of peers at other operations. (Of course not all lists are appropriate for all topics.)

Again I disagree with the principle that this list should be used for mail operation issues but maybe I'm just in the wrong here. Maybe this list is intended for everything internet related, if so I have some complaints I'd like to post about slow download speeds at my current isp. I think maybe there should be a better mission statement to clarify what it is intended for. Again large companies don't need to participate here. They have the user base so you either have to deal with them or block them. Then you have the business decisions of who is going to be more unhappy, their users who can't reach 10k in email accounts or your user base who can't reach 50 million in email accounts. This is the cost of doing business and yes it sucks at times but these choices you have to make as an operator. The businesses that do participate here and on other lists should be commended but it isn't an operational necessity for their business.

...

But lets just say for a second this is the place to discuss company xys's mail issue. What benefit do they have participating here? Likely they'll be hounded by people who have some disdain for their company and no matter what they do they will still be evil or wrong in some way.

They're more likely to be hounded by people who have disdain for their incompetence and the resulting operational issues they impose on their peers.

But if they're reluctant to face the unhappiness of their peers -- those whose networks, systems and users are abused on a daily basis and who thus have ample reason to be unhappy -- then maybe they should try something different, such as "doing their jobs properly".

I'll say it again, it is easy to tell someone who has a much larger economy of scale how to do their job properly when you are the small fish in the pond. These guys have a lot of politics in their jobs to deal with so where you may be the sole shot caller in your organization they may have to work through the layers in their organization. I fully believe we could work out some of the operational inefficiencies if I were the only person making decisions but I'm not and that is the reality of big business.

...

It is easy for someone who has 10,000 users to tell someone who has 50 million users what to do when they don't have to work with such a large scale enterprise.

This is mythology. Someone who can *competently* run a 10,000 user operation will have little-to-no difficulty running a 50 million user operation. (In some ways, the latter is considerably easier.) It's not a matter of the size of anyone's operation, it's a matter of how well it's run, which in turn speaks to the knowledge, experience, diligence, etc. of those running it.

---Rsk

If you say so, I find this comment pretty darn humorous saying 10k users should be easily scalable to 50 million. *sending to list this time -- Ross ross [at] dillio.net 314-558-6455

On Mon, Apr 14, 2008 at 6:18 AM, Rich Kulawiec <rsk@gsp.org> wrote:

On Sun, Apr 13, 2008 at 03:55:13PM -0500, Ross wrote:

...

Again I disagree with the principle that this list should be used for mail operation issues but maybe I'm just in the wrong here.

I don't think you're getting what I'm saying, although perhaps I'm not saying it very well.

What I'm saying is that operational staff should be *listening* to relevant lists (of which this is one) and that operational staff should be *talking* on lists relevant to their particular issue(s).

Completely agree.

Clearly, NANOG is probably not the best place for most SMTP or HTTP issues, but some of the time, when those issues appear related to topics appropriate for NANOG, it might be. The rest of the time, the mailop list is probably more appropriate.

While I prefer to see topics discussed in the "best place" (where there is considerable debate over what that might be) I think that things have gotten so bad that I'm willing to settle for, in the short term, "a place", because it's easier to redirect a converation once it's underway that it seems to be to start one.

For example: the silence from Yahoo on this very thread is deafening.

I think if you check historically, you'll find that Yahoo network operations team members are doing exactly as you indicate, and are "*talking* on lists relevant to their particular issue(s)" that is to say, here on NANOG, when it comes to networking issues, deafening silence has not been the modus operandus. The mistaken notion that a *network operations* list should have people on it to address mail server response code complaints is where I disagree with you. Ask about a BGP leakage, it'll get fixed. Enquire about how to engage in peering with Yahoo, you'll get flooded with answers; those are items the folks who read the list are empowered to deal with. Asking about topics not related to the list that they aren't empowered to deal with are going to be met with silence, because you're trying to talk to the wrong people in the wrong forum.

---Rsk

Matt --always speaking for himself--his employer is more likely to pay him to shut up.

Rich Kulawiec wrote:

On Thu, Apr 10, 2008 at 01:30:06PM -0400, Barry Shein wrote:

...

Is it just us or are there general problems with sending email to yahoo in the past few weeks?

It's not you. Lots of people are seeing this, as Yahoo's mail servers are apparently too busy sending ever-increasing quantities of spam to have to accept inbound traffic. Sufficiently persistent and lucky people have sometimes managed to penetrate the outer clue-resistant shells of Yahoo and effect changes, but some of those seem ineffective and temporary. There doesn't seem to be any simple, universal fix for this other than advising people that Yahoo's email service is already miserable and continues to deteriorate, and hoping that they migrate elsewhere.

---Rsk

The only thing that might cause them to change would be a combination of various ISPs pushing people to switch to e.g. GMail, and in enough numbers that Yahoo! noticed the falloff in ad revenue. Unless it costs them money it is unlikely that they will do anything about it. --Patrick