Contacts wanted: OVH, DigitalOcean, and Microsoft (Deutschland)
Nikolas Geyer <nik at neko.id.au> wrote:
I have passed your email on to the relevant team within DO to have a look at.
Thank you, but that wasn't what I requested, I asked for a contact there. (I know that this may be hard to understand, but it's like the difference between giving a man a fish, and teaching him how to fish. I'm sure that you would make a fine long term conduit between me and whatever mystery people you think you have made contact with at DigitalOcean, but really, it would be best if you would simply introduce me to those people directly. That way, if you die or go on vacation, incidents like these won't need to be put on hold until you get back and resume your role as our designated go-between.)
I'd like to thank you for your deriding commentary to bring attention to this problem.
No problem. My pleasure.
I am not sure it is the most effective way to try and engage the wider industry on a public list, but each to their own.
I am not sure that there is any other way that a lone outsider can or could engage either OVH or DigitalOcean in a way that would actually cause either company to take action on the issues I've reported on. Complaints from ordinary Internet end-lusers about this, which both companies must surely be drowing in by now, don't seem to be doing the job. In any case, I would be more than happy to have you tell me the "right way" to engage with any actual live human beings at either of these companies, especially if you also are able to identify one or more such receptive individuals by name and email address, which is what I was requesting in the first place.
Oh, and additionally, as an Australian citizen with many Aussie and Kiwi colleagues working at DO of various religious persuasions; your postscript relating this back to the recent terror attacks is abhorrent and disgusting. You should be completely ashamed.
It's pretty clear to me that you have rather dramatically misread my the aforementioned postscript to my earlier post, and that a fair and clear-eyed reading of that should be quite entirely inoffensive to all, with the possible exception of some few people who work in mass media and/or the "news" business, such as it currently is. In that postscript, I merely used a recent mass media controversy relating specifically and only to the social media -handling- of recent events to illustrate two blatant absurdities at opposite ends of a spectrum, neither of which itself has anything at all do do with those recent news events specifically, much less with the race, creed, color or gender of any of the people who have, most sadly and regretably, been caught up in those events. Please consider again the two polar opposite absurdities that I was actually attempting to call attention to. One the one hand, we have TV talking heads, with essentially no technical knowledge whatsoever, wondering aloud why social media tech companies cannot do what is clearly technically impossible, and even more absurdly, why they can't do it in real time no less. On the other hand, and in contrast to that absurdity, we have the present example of this spamming operation that appears to be well and truly ensconsed on the networks of both OVH and DigitalOcean, where even large multi-billion dollar Internet hosting companies seem utterly unable to spot even trivial and easily identifiable patterns of bad behavior, in and among their own respective customer bases, even though, as I have now illustrated, a single lone unpaid volunteer guy, sitting in his basement and in his sweaty underwear and a bathrobe -can- easily and quickly spot the problem, within just a couple of hours in fact, provided that he has access to a decent quality passive DNS service and an ample supply of electricity, margaritas and cigarettes. I'm only kidding, of course. I don't actually have a basement. Regards, rfg P.S. Your apparent misreading of my earlier postscript is entirely understandable and forgiveable in light of the rather unfortunate quip that I made just prior to that, about sending this specific miscreant to Guantanamo if his skin color was sufficiently dark. I seriously regret and apologize for that inartful phrasing, and ask every charitable person to believe me when I say that that was said entirely in jest (albeit a bad one), and that if anything, it was intended to be an expression of my own personal outrage about my own country's abundant inequity and unfairness when dealing with people of color, either within our so-called justice system or elsewhere. Our justice system should be color blind. Alas, there is much evidence that it falls far short of this goal at the present time.
On 3/18/19 11:17 PM, Ronald F. Guilmette wrote:
I am not sure that there is any other way that a lone outsider can or could engage either OVH or DigitalOcean in a way that would actually cause either company to take action on the issues I've reported on. Complaints from ordinary Internet end-lusers about this, which both companies must surely be drowing in by now, don't seem to be doing the job.
I have sent reports to DigitalOcean and Microsoft about the abuse reported to me that appears on my network. The only response I'm interested in is the end of said abuse. When abuse continues to be seen on a DigitalOcean allocation, that allocation goes into a file-and-forget ACL. I investigate further only when a customer reports a problem connecting with a DigitalOcean netblock. No such reports yet, by the way. I respond to abuse reports promptly and completely. I expect others to do so, just as promptly and completely. "Pretty please" is not in my netadmin vocabulary.
(Disclosure: I, too, work for DigitalOcean as the Manager of Network Engineering. Nikolas does not work for me, nor I for him.) On Tue, March 19, 2019 02:17, Ronald F. Guilmette wrote:
Nikolas Geyer <nik at neko.id.au> wrote:
I have passed your email on to the relevant team within DO to have a look at.
Thank you, but that wasn't what I requested, I asked for a contact there.
Oh, is that how this works? I ask that you FedEx me a million dollars cash, in small bills. I await the arrival of said parcel.
In any case, I would be more than happy to have you tell me the "right way" to engage with any actual live human beings at either of these companies, especially if you also are able to identify one or more such receptive individuals by name and email address, which is what I was requesting in the first place.
Would you really be happy with that? You derided another good-faith respondent to your screed with a rant about not being willing to fill out web forms to report abuse because it offends your sensibilities. Nikolas brought your report to the attention of the relevant group at DigitalOcean, we also have a link on the front page of https://www.digitalocean.com/ to Report Abuse that goes directly to the relevant group or groups responsible. We respond to reports (even rude ones) here on nanog and on other relevant industry mailing lists. We would prefer, but don't require, that you use the web form because that is integrated into the workflow of the groups that respond to those reports. If they choose to give you their individualized contact information, then they can do that. It is not my place, nor Nikolas', to give out individual contact information for our co-workers out to anyone who asks. That would be irresponsible and obnoxious for us to do that.
Oh, and additionally, as an Australian citizen with many Aussie and Kiwi colleagues working at DO of various religious persuasions; your postscript relating this back to the recent terror attacks is abhorrent and disgusting. You should be completely ashamed.
It's pretty clear to me that you have rather dramatically misread my the aforementioned postscript to my earlier post, and that a fair and clear-eyed reading of that should be quite entirely inoffensive to all, with the possible exception of some few people who work in mass media and/or the "news" business, such as it currently is.
As a caucasian American, born and raised in the US Midwest, I too was offended by your postscript. I would encourage you to take a step back, and consider your rhetorical tactics and whether they are beneficial to the community, or even to your own efforts. -- Jeff McAdams Manager of Network Engineering DigitalOcean
On Tue, Mar 19, 2019 at 09:23:34AM -0400, Jeff McAdams wrote:
We would prefer, but don't require, that you use the web form because that is integrated into the workflow of the groups that respond to those reports.
Why isn't abuse@ integrated into the workflow? It darn well should be, (a) given that RFC 2142 has been "on the books" for 22 years and (b) given that methods for handling incoming abuse (or bug, or outage, or other) reports via email to role accounts are numerous and reliable. To be clear: if you want to offer a web form in addition to an abuse@ address (or a security@ address, or a postmaster@ address) that's fine. But web forms are a markedly inferior means of communication and are clearly not a substitute for well-known/standardized role addresses that route to the appropriate people/processes. ---rsk
On 3/19/19 10:49 AM, Rich Kulawiec wrote:
On Tue, Mar 19, 2019 at 09:23:34AM -0400, Jeff McAdams wrote:
We would prefer, but don't require, that you use the web form because that is integrated into the workflow of the groups that respond to those reports.
Why isn't abuse@ integrated into the workflow? It darn well should be, (a) given that RFC 2142 has been "on the books" for 22 years and (b) given that methods for handling incoming abuse (or bug, or outage, or other) reports via email to role accounts are numerous and reliable.
To be clear: if you want to offer a web form in addition to an abuse@ address (or a security@ address, or a postmaster@ address) that's fine. But web forms are a markedly inferior means of communication and are clearly not a substitute for well-known/standardized role addresses that route to the appropriate people/processes.
---rsk
+1 -- John PGP Public Key: 412934AC
Just to clarify, we are RFC 2142 section 4 compliant. I mention section 4 specifically as that is directly within my realm of control, the remaining sections I will check. Both methods, web form submission and abuse@ are integrated ultimately into the same workflow. Being transparent, as things currently stand, the abuse@ submission method requires an additional element of human verification before ingestion to the workflow as it is open to abuse itself. For example, an annoyed former user who has been removed from the platform for abusive activities trying to subscribe it (and other RFC 2142 addresses) to thousands of pornographic mailing lists, or attempting to slam it with tens of thousands of junk emails. We do take platform abuse seriously but, like any other company, there is always room for improvement. We have a dedicated team who’s 24/7 job function is to continually improve our systems and processes surrounding abuse, from trying to stem it at top of funnel, to mitigating on-going issues with as low MTTR as possible, to responding to abuse@ (and web form) submissions. tl;dr - both submission methods are available Sent from my iPhone
On Mar 19, 2019, at 10:52 AM, Rich Kulawiec <rsk@gsp.org> wrote:
On Tue, Mar 19, 2019 at 09:23:34AM -0400, Jeff McAdams wrote: We would prefer, but don't require, that you use the web form because that is integrated into the workflow of the groups that respond to those reports.
Why isn't abuse@ integrated into the workflow? It darn well should be, (a) given that RFC 2142 has been "on the books" for 22 years and (b) given that methods for handling incoming abuse (or bug, or outage, or other) reports via email to role accounts are numerous and reliable.
To be clear: if you want to offer a web form in addition to an abuse@ address (or a security@ address, or a postmaster@ address) that's fine. But web forms are a markedly inferior means of communication and are clearly not a substitute for well-known/standardized role addresses that route to the appropriate people/processes.
---rsk
Absolutely unrelated to Ronald's original post, but it's ironic that the abuse@ address is itself heavily "abused", by commercial copyright enforcement companies which think it's a catch-all address for things which are not operationally related to the health of a network (BGP hijacks, DDoS, spam email traffic, botnet/virus/worm/trojan traffic command and control and such). Despite the presence of a registered DMCA agent address[1][2] for an ASN, many companies continue to flood abuse@ with copyright notices. Ask any ISP that operates in the English language Internet but is not physically located in the USA (NZ, AU, CA, etc) how many USA-specific legal threats their abuse inbox receives. Usually for something like a residential customer torrenting a TV show. 1: https://www.copyright.gov/dmca-directory/ 2: https://www.copyright.gov/rulemaking/onlinesp/NPR/faq.html On Tue, Mar 19, 2019 at 7:50 AM Rich Kulawiec <rsk@gsp.org> wrote:
We would prefer, but don't require, that you use the web form because
On Tue, Mar 19, 2019 at 09:23:34AM -0400, Jeff McAdams wrote: that
is integrated into the workflow of the groups that respond to those reports.
Why isn't abuse@ integrated into the workflow? It darn well should be, (a) given that RFC 2142 has been "on the books" for 22 years and (b) given that methods for handling incoming abuse (or bug, or outage, or other) reports via email to role accounts are numerous and reliable.
To be clear: if you want to offer a web form in addition to an abuse@ address (or a security@ address, or a postmaster@ address) that's fine. But web forms are a markedly inferior means of communication and are clearly not a substitute for well-known/standardized role addresses that route to the appropriate people/processes.
---rsk
On Tue, Mar 19, 2019 at 09:17:23AM -0700, Eric Kuhnke wrote:
Absolutely unrelated to Ronald's original post, but it's ironic that the abuse@ address is itself heavily "abused", by commercial copyright enforcement companies which think it's a catch-all address for things which are not operationally related to the health of a network [snip]
I've seen this movie and have implemented various mitigation approaches to it -- none of which constitute a "solution" but all of which help. 1. Block the addresses originating this traffic. There's no need for staff/processes on the receiving end to put up with spam. (If it's UBE, then it's spam -- by definition. The content and intention are irrelevant.) 2. Use procmail to redirect it where it needs to go. 3. Set up (non-public) Mailman-operated mailing lists for each role account and use the moderation queue on those as a throttling tool. (This works best in conjunction with (2). Let procmail do some of the heavy/straightforward lifting and sort the rest out later.) This also makes it easy to archive everything by subscribing an address that's an append-only mailbox. 4. Funnel the output of (2) and/or (3) into one of the many ticketing systems with priority assigned based on the characteristics of the senders as observed over time. ---rsk
[[ I've just collected some new information about the length of time that this specific bincoin extortion spamming bad actor has been on Digital Ocean's network. For those who may only have an interest in that one detail, you can just skip down to the line of plus signs and start reading there. ]] In message <50414.162.155.102.254.1553001814.iglou@webmail.iglou.com>, "Jeff McAdams" <jeffm@iglou.com> wrote:
(Disclosure: I, too, work for DigitalOcean as the Manager of Network Engineering. Nikolas does not work for me, nor I for him.)
On Tue, March 19, 2019 02:17, Ronald F. Guilmette wrote:
Nikolas Geyer <nik at neko.id.au> wrote:
I have passed your email on to the relevant team within DO to have a look at.
Thank you, but that wasn't what I requested, I asked for a contact there.
Oh, is that how this works? I ask that you FedEx me a million dollars cash, in small bills. I await the arrival of said parcel.
In my experience, if you don't ask for something, you aren't likely to get it. There's no harm in asking. In any case, I offer you the pertinent observation also that "small bills" are soooooooo last century. These days, as should now be abundantly clear, payment in bitcoin is the preferable currency for such requests. :-)
In any case, I would be more than happy to have you tell me the "right way" to engage with any actual live human beings at either of these companies, especially if you also are able to identify one or more such receptive individuals by name and email address, which is what I was requesting in the first place.
Would you really be happy with that? You derided another good-faith respondent to your screed with a rant about not being willing to fill out web forms to report abuse because it offends your sensibilities.
I stand by what I wrote. I don't like dealing with anonymous web forms that, for all I know, and based on the available evidence, are or may be aliased to /dev/null. I prefer the human touch, especially in cases where I am seeking to find someone who may be held accountable when and if no actual action ensues.
We would prefer, but don't require, that you use the web form because that is integrated into the workflow of the groups that respond to those reports. If they choose to give you their individualized contact information, then they can do that. It is not my place, nor Nikolas', to give out individual contact information for our co-workers out to anyone who asks. That would be irresponsible and obnoxious for us to do that.
I am not just "anyone who asks". I am a guy who's been spammed from your network. If you read my earlier report, then you should know that I am also the guy who took the time to carefully resarch this, and to provide your company with information about this specific crook/spammer... information that, it seems, you folks yourselves have apparently been largely or entirely unaware of, and for some considerable time now. Given that context, am I really entirely undeserving of even being informed of the mere email address of the head of DigitalOcean's abuse handling department, assuming, at least for the sake of argument, that such an inddividual does in fact exist? Wouldn't it be a Good Thing if that person and I could communicate direct? And more to the point, what would be the downside, exactly, if that person's name and email address were not only given to me, but also scattered to the four winds an given out to everyone on the planet? Are you implicitly asserting that that person might then have to (gasp!) deal with some additional influx of spam into his or her inbox? If so, then I can't help but wonder aloud why that person should NOT join the rest of us mere mortals in that shared and miserable club. Perhaps it would even be of some benefit for that person to come down out of the clouds at least long enough to experience what the rest of us poor sods have to deal with on a routine and daily basis. The experience might even enhance that person's understanding of, and appreciation of the very kinds of (spamming) problem that he or she is being paid to attend to. Stranger things have happened. I'll be generous here and will refrain from leaping to any conclusions that the person in question does not want his or her identity to be generally known for fear that he/she might then be personally criticised for his/her work and/or the lack thereof. But other than that, and a possible desire to avoid receiving any of this same spam-slime that the rest of us poor slobs get coated in on a daily basis, I really can't imagine what other reasons there might be that would cause Digital Ocean's abuse handling staff and/or the managment thereof to be so overwhelmingly discreet. What I can say, rather definitively now, is that the specific bitcoin scammer-spammer that prompted me to begin this thread has been given, over time, and by your company, Digital Ocean, no fewer than five hundred and fifty three (553) separate, distinct, discrete and individual IPv4 addresses and that many, most or all of those have been used for outbound spamming purposes, all just by this one bad actor, and all during the present calendar year. The evidence supporting this assertion was and is available here: https://pastebin.com/raw/WtM0Y5yC Note that this is the equivalent of more than a full /23 that Digital Ocean has given to this one customer, presumably after vetting the customer according to current industry standard due diligence procedures, which is to say no due diligence whatsoever, other than making sure that the check clears. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Additionally, I have just collected some additional evidence, courtesy of the Farsight Security passive DNS service, which appears to indicate that this specific bad actor has been (and perhaps remains?) a customer of Digital Ocean, not just for a few days, or just for a week or two, but for a period in excess of two full months, continuously, based upon reverse DNS settings in place on your network since at least January 7th: ==================================================================== ;; bailiwick: 71.128.178.in-addr.arpa. ;; count: 155 ;; first seen: 2019-01-07 13:09:45 -0000 ;; last seen: 2019-03-18 07:51:33 -0000 40.71.128.178.in-addr.arpa. IN PTR mx.c.cryptoaccount.ml. ==================================================================== This additional information raises a number of rather obvious questions: *) How many reports/complaints has Digital Ocean received, since the beginning of the current calendar year, regarding this specific criminal extortion spammer on your network? (Hint: If your assertion is that the number has been zero, I will likely point out that that estimate is not really credible, under the circumstances, and in light of the numerous Twitter spamming reports that I've previously cited.) *) Within the time period since this bad actor became active on the Digital Ocean network, and since the first relevant report of network abuse was received by your company about this specific bad actor, what has prevented Digital Ocean's abuse handling department from being either able or willing to effectively action this case? *) Does Digital Ocean enable outbound TCP port 25 connectivity for its new customers by default and in the absence of explicit customer requests? And if so, is that really the best choice? I look forward to your responses with respect to these relevant questions. Regards, rfg
participants (8)
-
Eric Kuhnke -
Jeff McAdams -
John Peach -
niels=nanog＠bakker.net -
Nikolas Geyer -
Rich Kulawiec -
Ronald F. Guilmette -
Stephen Satchell