responding to DMARC breakage
Hi Folks, It occurs to me that Yahoo's deployment of DMARC p=reject, and the choice of several big mail operators to honor that, has created a situation not unlike a really routing table or nameserver, snafu --- someone's published information that's caused lots of things to break. At an operational level, this comes down to Yahoo publishing a DMARC record into their nameservers containing "p=reject." As a result, Yahoo, and several other very large mail systems, are bouncing huge amounts of mail. We see, and react to, routing and nameserver snafus all the time. The response is usually an immediate, cooperative response to fix the problem as quickly as possible. Sometimes an operational problem uncovers a software bug, or vulnerability, or a protocol failure mode - which triggers various responses (CERT alert, software patches, protocol revisions via the IETF). Running a mail system and providing some hosting and list services, most of the operational issues I've run into involve nameserver corruption, over-aggressive spam blockers (and, of course, ongoing barrages of spam and persistent cracking threats). In most cases, problems are easy to resolve, and all involved are cooeprative (if sometimes slow). About the closest analogy I've encountered, to the current situation, are the more aggressive of the anti-spam blocklists (remember when someone would an entire subnet, with intent, when one host on that subnet generated some spam, or the operators who would extort payment for "expedited removal?"). By and large, market pressure has largely driven the worst actors into oblivion - but there don't seem to be any measures, with teeth, for dealing with bad actors. It strikes me that this situation is analogous: - several very big players have put a protocol into production that is, charitably, immature (DMARC is an informational internet-draft, not even an RFC, much less a standards-track RFC - and its backers have pretty much ignored any input from mailing list operators) - Yahoo published a dns record that triggers a protocol mode that results in huge amounts of mail bounces and operational disruption. - Yahoo (operationally) and the DMARC authors are intentionally un-responsive (as are hotmail, comcast, a few others; gmail, I note is not bouncing mail) How do we respond as operators, beyond late-night, ad-hoc patches to list software, that only partially resolve the problem? What kind of responses are available? In the broader scope of things, what kinds of responses are typical if someone publishes corrupted information and then doesn't cooperate in fixing the situation - be that through obliviousness, incompetence, lack of resources, laziness, or active intent (criminal or not)? Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
On Sat, Apr 12, 2014 at 10:12 AM, Miles Fidelman <mfidelman@meetinghouse.net> wrote:
What kind of responses are available? In the broader scope of things, what kinds of responses are typical if someone publishes corrupted information and then doesn't cooperate in fixing the situation - be that through obliviousness, incompetence, lack of resources, laziness, or active intent (criminal or not)?
1. Treat DMARC records which break mailing lists as malformed. 2. Treat messages with malformed DMARC records as a validation failure and act as directed for validation failures. -Bill -- William D. Herrin ................ herrin@dirtside.com bill@herrin.us 3005 Crane Dr. ...................... Web: <http://bill.herrin.us/> Falls Church, VA 22042-3004
William Herrin wrote: > On Sat, Apr 12, 2014 at 10:12 AM, Miles Fidelman > <mfidelman@meetinghouse.net> wrote: >> What kind of responses are available? In the broader scope of things, what >> kinds of responses are typical if someone publishes corrupted information >> and then doesn't cooperate in fixing the situation - be that through >> obliviousness, incompetence, lack of resources, laziness, or active intent >> (criminal or not)? > 1. Treat DMARC records which break mailing lists as malformed. > > 2. Treat messages with malformed DMARC records as a validation failure > and act as directed for validation failures. > > -Bill > > Doesn't really help if someone upstream is publishing the records, and its someone downstream who's acting on them. -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
On Sat, 12 Apr 2014 10:12:09 -0400, Miles Fidelman said:
It occurs to me that Yahoo's deployment of DMARC p=reject, and the choice of several big mail operators to honor that, has created a situation not unlike a really routing table or nameserver, snafu ---
It's more like a peering war. Time for somebody to either bake a cake, or find alternate transit providers.
Valdis.Kletnieks@vt.edu wrote:
On Sat, 12 Apr 2014 10:12:09 -0400, Miles Fidelman said:
It occurs to me that Yahoo's deployment of DMARC p=reject, and the choice of several big mail operators to honor that, has created a situation not unlike a really routing table or nameserver, snafu --- It's more like a peering war. Time for somebody to either bake a cake, or find alternate transit providers.
Aaargghhh - what a horrible, but accurate analogy. Worse probably - more like a peering war with a large broadband carrier, at the edge, where it's harder to find alternate transport. Sigh.. Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
On Sat, Apr 12, 2014 at 1:12 PM, Miles Fidelman <mfidelman@meetinghouse.net> wrote:
Valdis.Kletnieks@vt.edu wrote:
On Sat, 12 Apr 2014 10:12:09 -0400, Miles Fidelman said:
It occurs to me that Yahoo's deployment of DMARC p=reject, and the choice of several big mail operators to honor that, has created a situation not unlike a really routing table or nameserver, snafu ---
It's more like a peering war. Time for somebody to either bake a cake, or find alternate transit providers.
Aaargghhh - what a horrible, but accurate analogy. Worse probably - more like a peering war with a large broadband carrier, at the edge, where it's harder to find alternate transport.
Sigh..
Taking things a bit deeper... someone needs to get a legal opinion wrt the DMARC group's effort to have all mailinglists change their From: address. A legal opinion needs to be drawn on any new culpability nanog.org (or other mailinglists) would have when the list now "owns" the message that is being distributed. As it is now, there is acceptance that my posts are my content and the words there in are my responsibility. What happens when my text starts showing up as From:asdfadasfadfdsa@nanog.org ? -Jim P.
On 4/12/2014 2:38 PM, Jim Popovitch wrote:
On Sat, Apr 12, 2014 at 1:12 PM, Miles Fidelman <mfidelman@meetinghouse.net> wrote: someone needs to get a legal opinion wrt the DMARC group's effort to have all mailinglists change their From: address.
"The DMARC group" (presumably referring to the dmarc.org informal consortium that created DMARC) is conducting no such effort. The action taken this past week was an independent effort by Yahoo. dmarc.org had nothing to do with it. The DMARC specification is quite clear about the limitations of its use. Nothing is aided by the confusing the very basic different between a specification and the choices actors make in applying it. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net
On Sat, Apr 12, 2014 at 5:56 PM, Dave Crocker <dhc2@dcrocker.net> wrote:
On 4/12/2014 2:38 PM, Jim Popovitch wrote:
On Sat, Apr 12, 2014 at 1:12 PM, Miles Fidelman <mfidelman@meetinghouse.net> wrote: someone needs to get a legal opinion wrt the DMARC group's effort to have all mailinglists change their From: address.
"The DMARC group" (presumably referring to the dmarc.org informal consortium that created DMARC) is conducting no such effort.
The action taken this past week was an independent effort by Yahoo.
dmarc.org had nothing to do with it.
I wasn't writing about their website, rather the motivations of the core participants of the DMARC spec (that hang out around that website). If you haven't been paying attention all along, it's easy to miss the changes from the original DMARC objectives. Sometime after the first draft, DMARC went from only being for transactional email (i.e. behind the scenes stuff), to full blown end-all of spam with DMARC appearing on every tech blog and even CNN. That train's been barreling down the track for some time now. I posted this earlier, but for refresher: Go here: https://datatracker.ietf.org/doc/draft-kucherawy-dmarc-base/ Notice the early versions of the spec contained the word "transactional", notice the current version has it removed. Also notice that (at the point of change) one of the authors is from Yahoo!. What Yahoo! did wasn't a fluke, nor independent happenstance, it's part of a much bigger and broader picture. The ironic thing is that rather than go the IETF way (a fair amount of the DMARC folk are past IETF contributors), the decision was made to not seek peer consensus, nor to invalidate conflicting RFCs. An end run. -Jim P.
Dave Crocker wrote:
On 4/12/2014 2:38 PM, Jim Popovitch wrote:
On Sat, Apr 12, 2014 at 1:12 PM, Miles Fidelman <mfidelman@meetinghouse.net> wrote: someone needs to get a legal opinion wrt the DMARC group's effort to have all mailinglists change their From: address.
"The DMARC group" (presumably referring to the dmarc.org informal consortium that created DMARC) is conducting no such effort.
The action taken this past week was an independent effort by Yahoo.
dmarc.org had nothing to do with it.
The DMARC specification is quite clear about the limitations of its use.
Nothing is aided by the confusing the very basic different between a specification and the choices actors make in applying it.
Dave, it's not that clear cut. Standards bodies have been held liable for negligence, as have participants in standards making processes (just did a little googling of case law). Trade associations have been held to be in violation of antitrust law. I would expect that the right lawyer might have a field day painting the "informal consortium that created DMARC" as colluding in violation of anti-trust law, and perhaps criminal conspiracy. At the very least, "creating a public nuisance." And that's before we even consider civil torte liability. I also expect that someone could make a good case against Yahoo for "knowingly caus[ing] the transmission of a program, information code, or command, and as a result of such conduct, intentionally causes damages without authorization to a protected computer” in violation of the Computer and Fraud Abuse Act - for publishing their p=reject policy, and possibly for hotmail, comcast, etc. for criminal conspiracy in honoring that policy. (Kind of like a DDoS attack, or domain hijacking.) But then, I'm not a lawyer, just an engineer and sometime policy wonk (who just had lots of fun working with some very smart lawyers on a bid protest). Hmm... I wonder if anybody who's suffered serious economic damage as a result of this wants to bankroll some lawyers? Could be fun. (And given the amount of pain this has inflicted on me, personally, I wouldn't mind sharing some of the pain.) Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
On Sat, Apr 12, 2014 at 10:12 AM, Miles Fidelman <mfidelman@meetinghouse.net
wrote:
Valdis.Kletnieks@vt.edu wrote:
On Sat, 12 Apr 2014 10:12:09 -0400, Miles Fidelman said:
It occurs to me that Yahoo's deployment of DMARC p=reject, and the
choice of several big mail operators to honor that, has created a situation not unlike a really routing table or nameserver, snafu ---
It's more like a peering war. Time for somebody to either bake a cake, or find alternate transit providers.
Aaargghhh - what a horrible, but accurate analogy. Worse probably - more like a peering war with a large broadband carrier, at the edge, where it's harder to find alternate transport.
So, if we stretch the analogy to near-breaking-point, would that make Yahoo the Comcast of the email world... or the Level3? And depending on that answer, would the community think that a similar response of petitioning the government for more oversight and control would be warranted? Or would it be just as much out of line in this case as it is in the Level3-Comcast fight? I'm genuinely curious, because for most of my 20+ years in the networking industry, I've felt like we've done a good job at internally regulating ourselves as an industry, without needing to bring in outside regulation; but now, it sometimes starts to feel like the near metastable equilibrium of the system is wobbling ever-farther from our ability to adequately control and stabilize it. Have we potentially hit the point where the 'community' (for whatever definition is appropriate) no longer has enough input or leverage to bring players back into line when they stray outside of what is considered appropriate behaviour? In spite of the peering cake having been delicious and moist (I had two pieces, it was so yummy!), that rift has never closed; Comcast is not changing their model, in spite of community outcry, and Level3 has taken the step of summoning the spectre of government intervention. Cogent seems determined to follow a similar line of reasoning with respect to interconnections ("if we think we can get money from you, we'll use our customer base as leverage; if not, we'll cry foul, and appeal to the {government, masses, media}"). Have we reached the point as a community where "rough consensus and running code" is no longer the rule by which we operate, and fear of opprobrium no longer holds any weight with operators? As an engineer, I used to be proud that I helped build and operate a system that existed and thrived under its own rules, outside the sphere of any one particular government or legal system. I looked to it as a model of how a bottoms-up planetary ecosystem might operate, with everyone cooperating towards a universal goal. Now, I'm not so sure anymore; I'm becoming a little bit worried it's more just a simple reflection of all the conflicting impulses in each of us. I don't think there's a clear right or wrong to these questions; it just seems like the simplicity and elegant optimism of the early years may have slipped away while I focused intently on what was right in front of me. [drat...i started writing that over breakfast, and then the day got busy...and here i am, finishing it up fifteen hours later, and i'm not even sure if i'm still going in the same direction with it; but i'll still toss it out, and see in which direction it floats...] Matt
Matthew Petach wrote:
On Sat, Apr 12, 2014 at 10:12 AM, Miles Fidelman <mfidelman@meetinghouse.net <mailto:mfidelman@meetinghouse.net>> wrote:
Valdis.Kletnieks@vt.edu <mailto:Valdis.Kletnieks@vt.edu> wrote:
On Sat, 12 Apr 2014 10:12:09 -0400, Miles Fidelman said:
It occurs to me that Yahoo's deployment of DMARC p=reject, and the choice of several big mail operators to honor that, has created a situation not unlike a really routing table or nameserver, snafu ---
It's more like a peering war. Time for somebody to either bake a cake, or find alternate transit providers.
Aaargghhh - what a horrible, but accurate analogy. Worse probably - more like a peering war with a large broadband carrier, at the edge, where it's harder to find alternate transport.
So, if we stretch the analogy to near-breaking-point, would that make Yahoo the Comcast of the email world... or the Level3? And depending on that answer, would the community think that a similar response of petitioning the government for more oversight and control would be warranted? Or would it be just as much out of line in this case as it is in the Level3-Comcast fight?
That's a big concern of mine, and one that's somewhat reflected in current discussions re. NTIA stepping away from its oversight role of ICANN/IANA. It strikes me that there are a growing number of issues that beg for some kind of institutionalized response and recourse - peering, DMARC, others - but we don't have any in place. That's the point at which people start suing each other and looking for government intervention. Sigh.... In this case: - if the tv tower 2 miles from here starts interfering with stuff, we call the FCC, and it gets fixed (particularly if it starts interfering with, for example, police radios) - various law enforcement agencies go after the bigger spam operations, and DDoS exploiters - but... Yahoo publishes a p=reject DNS record - causing, effectively, a massive DDoS - and..... what? Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
On Sat, Apr 12, 2014 at 9:12 AM, Miles Fidelman <mfidelman@meetinghouse.net> wrote:
- Yahoo (operationally) and the DMARC authors are intentionally un-responsive (as are hotmail, comcast, a few others; gmail, I note is not bouncing mail)
How do we respond as operators, beyond late-night, ad-hoc patches to list software, that only partially resolve the problem?
In the face of intentional unresponsiveness: blacklisting. Start contacting Yahoo.com subscribers and explain the situation to these users, and inform the users that, for the time being: yahoo.com e-mail addresses can no longer participate in the mailing lists, because of Yahoo's new policy: And make some suggestions of good alternatives to using Yahoo mail. Then use mail filters to block messages to mailing list addresses with From: header yahoo.com (which cause the problem), next suspend subscriptions for @yahoo.com users, and configure mailing list software so that new @yahoo.com based e-mail addresses cannot subscribe or post to the lists. -- -JH
Question: Years ago Yahoo! bought major mailing list provider egroups formerly onelist, eventually absorbing it into yahoo clubs and making something called yahoogroups. Does this break yahoogroups too? How are THEY handling it? -- --------------------------------------------------------------- Joly MacFie 218 565 9365 Skype:punkcast WWWhatsup NYC - http://wwwhatsup.com http://pinstand.com - http://punkcast.com VP (Admin) - ISOC-NY - http://isoc-ny.org -------------------------------------------------------------- -
On Sun, Apr 13, 2014 at 1:43 AM, Joly MacFie <joly@punkcast.com> wrote:
Question:
Years ago Yahoo! bought major mailing list provider egroups formerly onelist, eventually absorbing it into yahoo clubs and making something called yahoogroups.
Does this break yahoogroups too? How are THEY handling it?
I think they broke it too. I'm a lurker on a modest sized group there (flags@yahoogroups.com). There is prominent member, with a yahoo.com account, who posts multiple times a day, every day throughout the week. His last post was on 4-April. -Jim P.
participants (8)
-
Dave Crocker -
Jim Popovitch -
Jimmy Hess -
Joly MacFie -
Matthew Petach -
Miles Fidelman -
Valdis.Kletnieks＠vt.edu -
William Herrin