Hi Leo and all, Well, here we are together again ;-) Please see below. On 10/22/16 2:53 PM, Leo Bicknell wrote:

In a message written on Sat, Oct 22, 2016 at 07:34:55AM -0500, Mike Hammett wrote:

...

"taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified" From https://krebsonsecurity.com/2016/10/hacked-cameras-dvrs-powered-todays-massi...

The part that should outrage everyone on this list:

That's because while many of these devices allow users to change the default usernames and passwords on a Web-based administration panel that ships with the products, those machines can still be reached via more obscure, less user-friendly communications services called "Telnet" and "SSH."

"The issue with these particular devices is that a user cannot feasibly change this password," Flashpoints Zach Wikholm told KrebsOnSecurity. "The password is hardcoded into the firmware, and the tools necessary to disable it are not present. Even worse, the web interface is not aware that these credentials even exist."

As much as I hate to say it, what is needed is regulation. It could be some form of self regulation, with retailers refusing to sell products that aren't "certified" by some group. It could be full blown government regulation. Perhaps a mix.

We we discussed the last time, this is an opportunity. Let's not miss again. People have been mentioning BCP38. That BCP needs a dust-off. as mentioned, simply masking an address in a BOTNET attack is a relatively small component of the problem that often gets solved by accident with NATs. We have a broader set of issues to address, and anyone looking for a single silver bullet will be disappointed. The questions that need asking are these: * What is the responsibility of the end system (either side) and its developer/manufacturer? What are the things Thing developers should be doing? * What is the responsibility of the home router? How can home routers enable appropriate access for a device while stopping unwanted traffic? I won't repeat the ENTIRE thread, but draft-ietf-opsawg-mud-01.txt can help and I'll provide an example MUD file as a postscript to this message that would have stopped infection of some DVRs. * What is the responsibility of the provider access router? {You fill in this part} * What is the responsibility of the provider peering router? BCP38 filtering, use of ROAs, BGPSEC, and other fighting words ;-) * What is the responsibility of the consumer? Less is more, here, I think. * What is the responsibility of government? I would suggest that we have two or three documents to be written, which combined could be an update to BCP38. And the answers to each of these questions are going to evolve over time. That's okay. BCPs should be living documents.

It's not a problem for a network operator to "solve", any more than someone who builds roads can make an unsafe car safe. Yes, both the network operator and rood operator play a role in building safe infrastructure (BCP38, deformable barriers), but neither can do anything for a manufacturer who builds a device that is wholely deficient in the first place.

Yes. Eliot ps: here's that MUD file. It's possible to write it in more specific language. Probably not necessary. What is important is that someone fill in the class "http://dvr264.example.com/controller". That's not hard, but needs doing. What happens next is that the class gets expanded and access lists get installed, preferably on the home router. And this means that the home router needs to play a bigger role of limiting ingress even in the home. That can prevent reflection attacks, for instance. { "ietf-mud:meta-info": { "lastUpdate": "2016-10-23T14:11:52+02:00", "systeminfo": "DVR H.264", "cacheValidity": 1440 }, "ietf-acl:access-lists": { "ietf-acl:access-list": [ { "acl-name": "mud-10387-v4in", "acl-type": "ipv4-acl", "ietf-mud:packet-direction": "to-device", "access-list-entries": { "ace": [ { "rule-name": "clout0-in", "matches" : { "ietf-mud:direction-initiated" : "from-device" }, "actions": { "permit": [ null ] } }, { "rule-name": "entin0-in", "matches": { "ietf-mud:controller": "http://dvr264.example.com/controller", "ietf-mud:direction-initiated" : "to-device" }, "actions": { "permit": [ null ] } } ] } }, { "acl-name": "mud-10387-v4out", "acl-type": "ipv4-acl", "ietf-mud:packet-direction": "from-device", "access-list-entries": { "ace": [ { "rule-name": "clout0-in", "matches": { "ietf-mud:direction-initiated" : "from-device" }, "actions": { "permit": [ null ] } }, { "rule-name": "entin0-in", "matches": { "ietf-mud:controller": "http://dvr264.example.com/controller", "ietf-mud:direction-initiated" : "to-device" }, "actions": { "permit": [ null ] } } ] } } ] } }

Hi, On 10/24/16 3:06 PM, Ca By wrote:

Assuming MUD is successful in the ietf, the cpe lifecycle is 10 years before the needle moves. At which point the target will have morphed to something else. Also, nobody is going to pay for that feature. Just like the early days of ipv6, the economics were misaligned.

We know of those who are planning to build, so maybe not so much. The function doesn't NEED to be in CPE, but it helps. And again, the CPE market is changing right now, so be careful about your assumptions.

in 10 years, the CPE will also be running PCP, where the bot tells the CPE to ignore all of MUD and open any arbitrary port it wants.

One of the hidden villains in these attacks, by the way, is uPnP. The point is not for the device to self-assert, but for the manufacturer to assert. Apart from that PCP actually solves a slightly different problem. MUD can tackle interior connectivity, which PCP doesn't really address. And really that's what we need to address reflection attacks. Eliot

On 10/24/16 4:03 PM, Ca By wrote:

Please elaborate on concrete evidence to support your claim the CPE market is changing.

If you can't see that then you're not paying attention. Eliot

Dumb question: If some camera, vaccum cleaner, toothbrush or refrigirator is behind NAT, can it do IP spoofing ? Won't the "from" address be replaced by the CPE router with the proper IP address assigned to that customer so that on the Internet itself, that packet will travel with a real IP routable back to the CPE ? Could mobile phones become a source of such attacks ? Depending on subscription, many are given actiual internet IPs and not NATted, so they could theoretically send packets with spoofed IPs. (would likely require rooted android phones, and how many of those are there ?) Second dumb question: If the number of infected devices in eastern USA is insufficient to have caused that DDoS, can one infer that the attack used an actual IP address instead of the anycast one in order to target the the easter USA hosts irrespective of the location of the infected device ? Could one operate such a host with the "real" IP address in a subnet that has its own BGP announcement, and when there is an attack, one would change the real IP to a different IP address in a different subnet, and drop the route announcement for the first subnet (making those attack packets unroutable at the origin). Is that a viable counter measure ?

...

Could mobile phones become a source of such attacks ?

Depends both on the phone and on the network. But since Dyn-style attacks don't use IP spoofing, it doesn't really matter.

J-F's question was not about ip spoofing, but rather the infected devices being behind nats. in the states, much broadband is not behind a cgn, but is behind home nats. more mobile is behind cgn [0]. cgns mean fewer visible attacking source addresses. it would be interesting to see the home-soho vs cgn distribution of attacks such as krebs and dyn.

...

If the number of infected devices in eastern USA is insufficient to have caused that DDoS, can one infer that the attack used an actual IP address instead of the anycast one in order to target the the eastern USA hosts irrespective of the location of the infected device?

No. Anycast addresses are real IP addresses.

true.

There isn't a "real" address to attack.

usually false. dns clusters have management interfaces. i suspect the congestion pattern attacking them would be different than that of attack on the anycast; but that is conjecture. randy -- 0 - to get an idea of the vast scale of cgn deployment see philipp's preso of our imc paper from ripe 75

On 10/22/2016 05:34 AM, Mike Hammett wrote:

"taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified"

Serious question... how?

Network operators can only do so much. By the time traffic enters into an ISP's traffic aggregation point, any flow monitoring and throttling would have a minimal effect. Not saying that it shouldn't be considered. The correct answer includes throttling the traffic much closer to the source. The obvious answer is that the device that bridges IoT to the upstream link in the home or office have the capability of rate-limiting upstream traffic. Perhaps on a per-MAC basis. When does a thermostat, light bulb, or refrigerator need 1-megabyte/s uplink channels? For that matter, how many computers -- especially laptops -- need that kind of upstream capacity? (Yes, yes, YouTube publishers and VLAN links to the office, to name two, will need that kind of channel; see below. Gamers need small, low-latency channels, so the throttling can't be too aggressive. Public-access storage, web and mail servers, obviously. IP-connected Web cameras need some upstream capacity, but not a full-bore one. The uplink throttle can take into consideration "reasonable" upstream rates for cameras.) For wireless access points, the place to start would be with the OpenWRT package, to serve as a model for what *can* be done. Once we have a proof of concept, it would raise the bar for "commercial" implementations. THAT would then provide an opportunity for the three-letter Federal agencies to specify reasonable regulations, should Congress so decide this is necessary. It's much easier for regulatory bodies to say "this software does it, why can't yours?" instead of saying "you [manufacturer] go figure it out". The ripple effect throughout the world would go a long way to curbing the problem. Especially if other regulatory administrations follow suit, so that the enabling crap routers are weeded out. What about the exceptions? For those rare cases where one needs a high-rate upstream channel for a node on the wireless network (or wired network, for that matter), the firmware in the traffic aggregating device can allow for specific exceptions to the rate-limit rules. One method is to tie exceptions to the device MAC address, or range of MAC addresses. Another is to tie exceptions to ports, with WiFi being a single "port" in this context. Generators of high-speed upstream traffic would, for example, need a wired connection in order to do this. This would *not* affect most WiFi-connected peripherals, like printers, because the AP would limit upstream traffic, not downstream. The ISP would then have something to sell to the customer, to replace the local POS router/WAP that the customer is currently using. Hmmm...something to thing about as I build the Linux IPTABLES Firewall Rule Generator Mk III...

It is also likely the desired use case. In my office I like to be able to login when needed when on the road, when the alarm company calls me at 2am for a false alarm so I don't have to get someone else out of bed to have them dispatched to check on the site. -jim On Sat, Oct 22, 2016 at 1:42 PM, Chris Boyd <cboyd@gizmopartners.com> wrote:

...

On Oct 22, 2016, at 7:34 AM, Mike Hammett <nanog@ics-il.net> wrote:

"taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified"

Serious question... how?

Putting them behind a firewall without general Internet access seems to work for us. We have a lot of cheap IP cameras in our facility and none of them can reach the net. But this is probably a bit beyond the capabilities of the general home user.

—Chris

Sure, but now we put it outside the skill level of 99.99% of the people that don't read and understand this list. -jim On Sat, Oct 22, 2016 at 2:09 PM, Luke Guillory <lguillory@reservetele.com> wrote:

VPNs can accomplish this without opening ports directly to devices.

Luke

*Sent from my iPhone*

On Oct 22, 2016, at 12:06 PM, jim deleskie <deleskie@gmail.com> wrote:

It is also likely the desired use case. In my office I like to be able to login when needed when on the road, when the alarm company calls me at 2am for a false alarm so I don't have to get someone else out of bed to have them dispatched to check on the site.

-jim

On Sat, Oct 22, 2016 at 1:42 PM, Chris Boyd <cboyd@gizmopartners.com> wrote:

On Oct 22, 2016, at 7:34 AM, Mike Hammett <nanog@ics-il.net> wrote:

"taken all necessary steps to insure that none of the numerous specific

types of CCVT thingies that Krebs and others identified"

Serious question... how?

Putting them behind a firewall without general Internet access seems to

work for us. We have a lot of cheap IP cameras in our facility and none of

them can reach the net. But this is probably a bit beyond the capabilities

of the general home user.

—Chris

Luke Guillory Network Operations Manager

<http://www.rtconline.com> Tel: 985.536.1212 Fax: 985.536.0300 Email: lguillory@reservetele.com Web: www.rtconline.com Reserve Telecommunications 100 RTC Dr Reserve, LA 70084

*Disclaimer:* The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material which should not disseminate, distribute or be copied. Please notify Luke Guillory immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Luke Guillory therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission.

That's what VPNs are for. On 10/22/2016 10:04 AM, jim deleskie wrote:

It is also likely the desired use case. In my office I like to be able to login when needed when on the road, when the alarm company calls me at 2am for a false alarm so I don't have to get someone else out of bed to have them dispatched to check on the site.

-jim

On Sat, Oct 22, 2016 at 1:42 PM, Chris Boyd <cboyd@gizmopartners.com> wrote:

...

...

On Oct 22, 2016, at 7:34 AM, Mike Hammett <nanog@ics-il.net> wrote:

"taken all necessary steps to insure that none of the numerous specific types of CCVT thingies that Krebs and others identified"

Serious question... how?

Putting them behind a firewall without general Internet access seems to work for us. We have a lot of cheap IP cameras in our facility and none of them can reach the net. But this is probably a bit beyond the capabilities of the general home user.

—Chris

Generic question: The media seems to have concluded it was an "internet of things" that caused this DDoS. I have not seen any evidence of this. Has this been published by an authoritative source or is it just assumed? Has the type of device involved been identified? I am curious on how some hacker in basement with his TRS80 or Commodore Pet would be able to reach "bilions" of these devices to reprogram them. Vast majority of homes are behind NAT, which means that an incoming packet has very little chance of reaching the IoT gizmo. I amn guessing/hoping such devices have been identified and some homweoners contacted ans asked to volunteer their device for forensic analysis of where the attack came from ? Is it more plausible that those devices were "hacked" in the OEM firmware and sold with the "virus" built-in ? That would explain the widespread attack. Also, in cases such as this one, while the target has managed to mitigate the attack, how long would such an attack typically continue and require blocking ? Since the attack seemed focused on eastern USA DNS servers, would it be fair to assume that the attacks came mostly from the same region (aka: devices installed in eastern USA) ? (since anycast would point them to that). OPr did the attack use actual IP addresses instead of the unicast ones to specifically target servers ? BTW, normally, if you change the "web" password on a "device", it would also change telnet/SSH/ftp passwords.

On Sat, Oct 22, 2016 at 1:47 PM, Jean-Francois Mezei <jfmezei_nanog@vaxination.ca> wrote:

Generic question:

The media seems to have concluded it was an "internet of things" that caused this DDoS.

I have not seen any evidence of this. Has this been published by an authoritative source or is it just assumed?

Flashpoint[0], krebs[1], arstechnica[2]. I'm not sure what credible looks like unless they release a packet but this is probably consensus.

Has the type of device involved been identified?

routers and cameras with shitty firmware [3]

Is it more plausible that those devices were "hacked" in the OEM firmware and sold with the "virus" built-in ? That would explain the widespread attack.

The source code has been released. krebs [4], code [5]

Also, in cases such as this one, while the target has managed to mitigate the attack, how long would such an attack typically continue and require blocking ? This is an actual question that hasn't been answered.

Since the attack seemed focused on eastern USA DNS servers, would it be fair to assume that the attacks came mostly from the same region (aka: devices installed in eastern USA) ? (since anycast would point them to that).

Aren't heat maps just population graphs?

BTW, normally, if you change the "web" password on a "device", it would also change telnet/SSH/ftp passwords.

Seems like no one is doing either. [0] https://www.flashpoint-intel.com/mirai-botnet-linked-dyn-dns-ddos-attacks/ [1] https://krebsonsecurity.com/2016/10/hacked-cameras-dvrs-powered-todays-massi... [2] http://arstechnica.com/security/2016/10/double-dip-internet-of-things-botnet... [3] https://blog.sucuri.net/2016/09/iot-home-router-botnet-leveraged-in-large-dd... [4] https://krebsonsecurity.com/2016/10/source-code-for-iot-botnet-mirai-release... [5] https://github.com/jgamblin/Mirai-Source-Code -- Pete Baldridge 206.992.2852

https://urldefense.proofpoint.com/v2/url?u=http-3A__hub.dyn.com_dyn-2Dblog_dyn-2Dstatement-2Don-2D10-2D21-2D2016-2Dddos-2Dattack&d=DQIBAg&c=n6-cguzQvX_tUIrZOS_4Og&r=r4NBNYp4yEcJxC11Po5I-w&m=iGvkbfzRJPqKO1A6YGa-c1m0RBLNkRk03hCjvVGTH3k&s=bScBNFncB3kt_cG0L3iys0mfXBmwwUR7A8rIDmi94D4&e= On Sat, Oct 22, 2016 at 04:48:01PM -0500, Mike Hammett wrote:

Until Dyn says or someone says Dyn said, everything is assumed.

----- Original Message -----

From: "Peter Baldridge" <petebaldridge@gmail.com> To: "Jean-Francois Mezei" <jfmezei_nanog@vaxination.ca> Cc: nanog@nanog.org Sent: Saturday, October 22, 2016 4:45:13 PM Subject: Re: Death of the Internet, Film at 11

On Sat, Oct 22, 2016 at 1:47 PM, Jean-Francois Mezei <jfmezei_nanog@vaxination.ca> wrote:

...

Generic question:

The media seems to have concluded it was an "internet of things" that caused this DDoS.

I have not seen any evidence of this. Has this been published by an authoritative source or is it just assumed?

Flashpoint[0], krebs[1], arstechnica[2]. I'm not sure what credible looks like unless they release a packet but this is probably consensus.

...

Has the type of device involved been identified?

routers and cameras with shitty firmware [3]

...

Is it more plausible that those devices were "hacked" in the OEM firmware and sold with the "virus" built-in ? That would explain the widespread attack.

The source code has been released. krebs [4], code [5]

...

Also, in cases such as this one, while the target has managed to mitigate the attack, how long would such an attack typically continue and require blocking ? This is an actual question that hasn't been answered.

...

Since the attack seemed focused on eastern USA DNS servers, would it be fair to assume that the attacks came mostly from the same region (aka: devices installed in eastern USA) ? (since anycast would point them to that).

Aren't heat maps just population graphs?

...

BTW, normally, if you change the "web" password on a "device", it would also change telnet/SSH/ftp passwords.

Seems like no one is doing either.

On 2016-10-22 18:35, Ray Van Dolson wrote:

https://urldefense.proofpoint.com/v2/url?u=http-3A__hub.dyn.com_dyn-2Dblog_dyn-2Dstatement-2Don-2D10-2D21-2D2016-2Dddos-2Dattack&d=DQIBAg&c=n6-cguzQvX_tUIrZOS_4Og&r=r4NBNYp4yEcJxC11Po5I-w&m=iGvkbfzRJPqKO1A6YGa-c1m0RBLNkRk03hCjvVGTH3k&s=bScBNFncB3kt_cG0L3iys0mfXBmwwUR7A8rIDmi94D4&e=

Thanks for the link. 10s of millons of IP addresses. Is it realistic to have 10s of millions of infected devices ? Or is that the dense smoke that points to IP spoofing ? re: newspaper reports: how did Flashpoint obtain enough details, while attack was ongoing to be able to draw conclusions told to the media ? Or was it educated speculation ? Obviously, Dyn had packet contents to look at and range of IPs being used etc. Would such a company typically release that info to a trusted investigator "as it happens" ? (would Flashpoint be such an outfit ?) Did the attack generate valid DNS queries (overwhelm the servers) or flood the links with long "random" UDP packets (overwhel the links). While I can understand that mitigation methods can be seen as "proprietary", releasing info on the specifics of the attack would help any/all neteowkrs and data centres better protect themselves. Assuming hackers don't talk to each others in the 21st century is silly. They already know how this was done, yet the victims typically remain silent for fear of educating the hackers for more attacks.

On Sat, 22 Oct 2016 19:22:04 -0400, Jean-Francois Mezei said:

10s of millons of IP addresses. Is it realistic to have 10s of millions of infected devices ? Or is that the dense smoke that points to IP spoofing ?

A few years ago, Vint Cerf gave a keynote speech at a conference, where he claimed that there were 140 million pwned devices on the Internet - and this was before IoT was itself a thing. Not one person in the security industry called bullshit and said the number was too high. There were however a lot of people who thought Cerf had significantly lowballed the estimate.

From Dyn's statement,

http://hub.dyn.com/static/hub.dyn.com/dyn-blog/dyn-statement-on-10-21-2016-d... we have "After restoring service, Dyn experienced a second wave of attacks just before noon ET. This second wave was more global in nature (i.e. not limited to our East Coast POPs), but was mitigated in just over an hour; service was restored at approximately 1:00 pm ET." I can't reconcile this what I saw on our recursive name servers here in Norway, looking at the period when Dyn's name servers didn't answer at all. We saw the second wave start at around 15:56 UTC (which corresponds well with "just before noon ET") and end at around 20:00 UTC - so from our point of view, service was unavailable for around *four* hours. Steinar Haug, Nethelp consulting, sthaug@nethelp.no

I think you make a very good point with the TRS80 etc comment, at least implicitly: it's not just the vulnerable IoT devices, some sort of infrastructure is needed to get the attack going at the volume we've seen. And perhaps therein lies an answer. On October 22, 2016 at 16:47 jfmezei_nanog@vaxination.ca (Jean-Francois Mezei) wrote:

Generic question:

The media seems to have concluded it was an "internet of things" that caused this DDoS.

I have not seen any evidence of this. Has this been published by an authoritative source or is it just assumed?

Has the type of device involved been identified?

I am curious on how some hacker in basement with his TRS80 or Commodore Pet would be able to reach "bilions" of these devices to reprogram them. Vast majority of homes are behind NAT, which means that an incoming packet has very little chance of reaching the IoT gizmo.

I amn guessing/hoping such devices have been identified and some homweoners contacted ans asked to volunteer their device for forensic analysis of where the attack came from ?

Is it more plausible that those devices were "hacked" in the OEM firmware and sold with the "virus" built-in ? That would explain the widespread attack.

Also, in cases such as this one, while the target has managed to mitigate the attack, how long would such an attack typically continue and require blocking ?

Since the attack seemed focused on eastern USA DNS servers, would it be fair to assume that the attacks came mostly from the same region (aka: devices installed in eastern USA) ? (since anycast would point them to that).

OPr did the attack use actual IP addresses instead of the unicast ones to specifically target servers ?

BTW, normally, if you change the "web" password on a "device", it would also change telnet/SSH/ftp passwords.

-- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

In message <26b01962-9b09-11cb-0ac8-89cf3e0a5f96@nuclearfallout.net>, John Weekes <jw@nuclearfallout.net> wrote:

... I've recorded about 2.4 million IP addresses involved in the last two months (a number that is higher than the number of actual devices, since most seem to have dynamic IP addresses). The ISPs behind those IP addresses have received notifications via email...

Just curious... How well is that working out? I've tried this myself a few times in the past, when I've found things that appear to be seriously compromised, and for my extensive trouble I've mostly received back utter silence and no action. I remember that after properly notifying security@ some large end-luser cable network in the SouthEast (which shall remain nameless) I got back something along the lines of "Thank you. We'll look into it." and was disgusted to find, two months later, that the boxes in question were still utterly pwned and in the exact same state they were two months prior, when I had first reported them. I guess that's just an example of what somebody else already noted here, i.e. that providers don't care to spend the time and/or effort and/or money necessary to actually -do- anything about compromised boxes, and anyway, they don't want to lose a paying customer. So, you know, let's just say for the sake of argument that right now, today, I know about a botnet consiting of a quarter million popped boxes, and that I have in-hand all of the relevant IPs, and that I have no trouble finding contact email addresses for all of the relevant ASNs. So then what? The question is: Why should I waste my time informing all, or even any of these ASNs about the popped boxes on their networks when (a) I am not their customer... as many of them have been only too happy to gleefully inform me in the past... and when (b) the vast majority simply won't do anything with the information? And while we are on the subject, I just have to bring up one of my biggest pet peeves. Why is it that every time some public-spirited altrusitc well-meaning citizen such as myself reports any kind of a problem to any kind of a company on the Internet, the report itself gets immediately labeled and categorized as a "complaint". If I spend some of -my- valuable time to helpfully try to let somebody else know of a problem on their network, or with their web site, and if that report gets categorized as a "complaint" then what does that make me? A "complainer"?? I don't need this kind of abuse and denegration from people who I'm trying to help. Like most other people, if I am in need of some personal denegration and abuse... well... I have relatives for that. Regards, rfg

On October 23, 2016 at 6:52:05 PM, Stephen Satchell (list@satchell.net) wrote: So, bottom line, nothing is going to happen until the cost to those  negligent provides rises so high as to affect profits. Period.  Yep.  Or government intervention. Larger eyeball operators could help, by shutting down entire subnets  infested with botted computers and things.  Shut down subnets of your own customers?  Regards, -drc

On 10/23/2016 4:19 PM, Ronald F. Guilmette wrote:

...

... I've recorded about 2.4 million IP addresses involved in the last two months (a number that is higher than the number of actual devices, since most seem to have dynamic IP addresses). The ISPs behind those IP addresses have received notifications via email... Just curious... How well is that working out?

For the IoT botnets, most of the emails are ignored or rejected, because most go to providers who either quietly bitbucket them or flat-out reject all abuse emails. Most emails sent to mainland China, for instance, are in that category (Hong Kong ISPs are somewhat better). For other botnets, such as those using compromised webservers running outdated phpMyAdmin installs at random hosts, harnessing spun-up services at reputable VPS providers (Amazon, Microsoft, Rackspace, etc.), or harnessing devices at large and small US and Canadian ISPs, we have had better luck. Usually, we don't hear a response back, but those emails are often forwarded to the end-user, who takes action (and may ask us for help, which is how we know they are being forwarded). The fixes can enough to reduce attack volumes to more manageable levels. Kudos go out to the large and small ISPs and NSPs who have started policing SSDP and other reflection traffic, which we also send out some notifications for. In some cases, it may be that our emails spurred them to notice how much damage those attacks were doing and how much it was costing them to carry the attack traffic.

I've tried this myself a few times in the past, when I've found things that appear to be seriously compromised, and for my extensive trouble I've mostly received back utter silence and no action. I remember that after properly notifying security@ some large end-luser cable network in the SouthEast (which shall remain nameless) I got back something along the lines of "Thank you. We'll look into it." and was disgusted to find, two months later, that the boxes in question were still utterly pwned and in the exact same state they were two months prior, when I had first reported them.

We do get our share of that, as well, unfortunately, along with our share of people who send angry responses calling the notifications spam (I disagree with them that sending a legitimate abuse notification to a publicly-posted, designated abuse account should be considered spam) or who flame us for acting like "internet police". But, we persist. Some people change their minds after receiving multiple notifications or after we explain that DoS traffic costs them money and hurts their customers, who will be experiencing degraded service and may silently switch providers over it.

I guess that's just an example of what somebody else already noted here, i.e. that providers don't care to spend the time and/or effort and/or money necessary to actually -do- anything about compromised boxes, and anyway, they don't want to lose a paying customer.

So, you know, let's just say for the sake of argument that right now, today, I know about a botnet consiting of a quarter million popped boxes, and that I have in-hand all of the relevant IPs, and that I have no trouble finding contact email addresses for all of the relevant ASNs. So then what?

I use scripts to send out an abuse notification to some percentage of the compromised hosts -- the ones sending some significant amount of the traffic. The notification includes a description of what we saw and timestamped example attack traffic, as interpreted by tcpdump. If further traffic is seen later from the same host, another notification will be sent, after a cool-off period. The emails are plain text and we don't try to use them as advertisement. We also don't force a link to be clicked to see more details or to respond back. I don't like to receive such emails myself and have found that those types are more likely to be ignored.

The question is: Why should I waste my time informing all, or even any of these ASNs about the popped boxes on their networks when (a) I am not their customer... as many of them have been only too happy to gleefully inform me in the past... and when (b) the vast majority simply won't do anything with the information?

I'm not saying that everyone should send abuse notifications like we do, since it can be a big task. But, in response to someone wondering if their network is being used for attacks, or asking how they could help to police their own network, I am saying that making sure that inbound abuse notifications are arriving at the right place and being handled appropriately is important.

And while we are on the subject, I just have to bring up one of my biggest pet peeves. Why is it that every time some public-spirited altrusitc well-meaning citizen such as myself reports any kind of a problem to any kind of a company on the Internet, the report itself gets immediately labeled and categorized as a "complaint". If I spend some of -my- valuable time to helpfully try to let somebody else know of a problem on their network, or with their web site, and if that report gets categorized as a "complaint" then what does that make me? A "complainer"??

I don't need this kind of abuse and denegration from people who I'm trying to help. Like most other people, if I am in need of some personal denegration and abuse... well... I have relatives for that.

There's a spectrum of people responding to these and some percentage are just jerks, as in real life. But, I like to think that the majority of at least NA providers are represented by professionals who just don't respond out of courtesy because they don't want to flood our inboxes with simple acknowledgements. Those of us experiencing these attacks appreciate the community support, both from people like you who also send notifications and those who handle the notifications on the receiving end. -John

Question: For something like Mirai and others, there appears to be a timer that starts the attack at a certain day/time (with unknown amount of time to distribute the software to any/all infectable devices prior to attack). Do these generally have a timer to also stop the attack and go dormant awaiting instructions from its master ? or do they continue to send those packets forever ? If the attack is made using perfectly formed, legitimate DNS packlets (or HTTP requests or whetever), can temporary mitigation measures continue forever even if they block legitimate requests ? Or is it general practioce for hackers to have short duration attacks to reduce the time available to track them down ? (similar to old movies where one had to hangup before the 2 minutes it took for police to trace a phone call).

-----Original Message-----

From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Richard Holbo Sent: October-23-16 11:23 PM To: John Weekes Cc: NANOG Subject: Re: Death of the Internet, Film at 11

I run/manage the networks for several smallish (in the thousands of customers) eyeball ISP's and I appreciate a nice "hey you've got a bot" or "someone is scanning" me notice to my abuse emails. They are useful in identifying crap that's going on, so for those of you who have the resources to do that... I appreciate it, we do read them at my networks and try to do something.

That said... getting end users to actually fix the broken routers etc. etc. is NOT easy. Very often we'll notify customers, they will _take their stuff to the local computer repair guy_ ... or office depo.... and they will run whatever auto scan they have and say it's all fine. Customer puts it back in, it's still broke, and they call customer support and want us to pay for the trip because _their_ expert says it's fine...

+1 a hundred times over. Educating customers is difficult to do, assuming they even want to take the time to be educated. I'm not just talking about the residential class here either. "Commercial IT shops" are not much better in most cases -staff tend to all but zone out when you try and explain anything more difficult than un-checking the NetBIOS box on their managed customer device(s), or try and explain why that DMZ checkbox is a bad idea. This usually plays out one of two ways at $dayjob; 1) If you apply too much force to the "You have broken equipment" statement, the customer becomes frustrated and move their broken equipment to $competing-operator's network, and you wind up with a black customer service mark. 2) Customer replaces the device at their expense (It's not _our_ CPE, so why would we?), so typically something cheaper and worse than what they replaced, and they still feel burned. But at least their service gets re-activated. I can recall at least a half-dozen scenarios where the customer actually takes up the problem with the manufacturer. In each of those cases, and they're effectively told to push off because the devices are more than 12mo old and outside the warranty period (paraphrasing customer statements and all that), or similar canned response. Hate to namedrop on a public list, but Linksys, D-Link, Buffalo, Netgear... "There's profit to be had." Granted, It's been a while, at least ~18 months since I've had any such feedback, so perhaps things are better in the wake of all the media attention such vendors have seen from time to time. One case with Asus ended positively after about a week of effort with phonecalls, so it's not all bad. But a week? Not a great turn-around assuming the volumes of devices that would potentially need to be patched/updated/whatever. Only the truly tech-savvy appreciate the advice to repair the problem, but they are probably 1 in 1,000, if that.

On October 25, 2016 at 01:28 rfg@tristatelogic.com (Ronald F. Guilmette) wrote:

The fundamental economics have not changed. It pays to design and ship things. It doesn't pay to support them afterwards. This isn't going to change.

It is common to include "goodwill" on the balance sheet, but unless I'm mistaken, for most companies this does not represent a significant fraction of shareholder equity.

I suppose that's the "one born every minute!" theory of product development. And there's truth to it no doubt. Marketing travels a lot faster and more efficiently than awareness of poor support or quality issues is another way to state that. Which is one reason why product areas tend to evolve regulatory structures both public (e.g., US FTC) and private (e.g., Underwriters Laboratory) thus getting some sort of quality and support push-back other than the bare-knuckle market. -- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

In message <e364fcea-7105-b3b9-63a9-7d22ab83516c@nuclearfallout.net>, John Weekes <jw@nuclearfallout.net> wrote:

On 10/23/2016 4:19 PM, Ronald F. Guilmette wrote: jw>>> ... The ISPs behind those IP addresses have jw>>> received notifications via email... rfg>> Just curious... How well is that working out?

For the IoT botnets, most of the emails are ignored or rejected, because most go to providers who either quietly bitbucket them or flat-out reject all abuse emails. Most emails sent to mainland China, for instance, are in that category (Hong Kong ISPs are somewhat better)...

So, given the apparently impracticality of trying to clean up all of these kinds of messes via the good old-fashioned tedious and laborious method of emailing the relevant providers and then just sort-of vaguely hoping that they will -do something- responsible with it, I am just sitting here trying to dream up some sort of generalized long-term fix for -all- of these IoT DDoS type problems. Maybe there just plain isn't any such thing as a general solution to the problem, because it may perhaps be just technically too complex. But I hope no one will begrudge me if I yearn for some sort of Grand Unified Field Theory of IoT security. So, I have a thought... probably worth what you paid for it... and I'm just brave enough to throw it out on the table and then everybody can pile on and tell me what an idiot I am, for this or that perfectly sound technical reason. (I'll say up front that I don't even pretend to understand many of the protocols in use these days, in particular UPnP, and to be frank, I'd never even heard of SSDP until yesterday. So I can't and won't begrudge anybody who tells me that I have my head up... ummm... in the clouds.) So anyway, here are the assumptions/assertions, perhaps wrong, which are my starting point: 1) I am not persuaded that IoT devices have a compelling need to them- selves initiate outbound TCP sessions, ever. (If I'm wrong about this, then I'm sure people here will tell me.) 2) Likewise, I am not persuaded that IoT devices have an absolute and compelling need to do very much in the way of UDP. Yes, I would like my smart XYZ device to always know what time it is, so, you know, a modest amount of NTP traffic is reasonable and to be expected. Other than that however, I don't see a compelling need. If you want to either control or get data out of your IoT device, you can make an inbound TCP connection to it. (Somebody will probably say "Oh, no. We need to stream real-time video out of some of these things, and for that we absolutely have to send the stuff via outbound high-bandwidth UDP." but I am not persuaded that this is either absolutely necessary or even Good, i.e. from the point of view of the legitimate security concerns of the owner of the device.) So, based on the above perhaps flawed assumptions, here is my idea. It is composed of two simple parts: 1) First, I will successfully complete my campaign to be elected King of the World. (Given the current poltical climate, worldwide, this should not be a problem, because I lie a lot.) 2) Second, once elected I will decree that in future all new IoT devices, and also all updates to firmware for existing IoT devices will have, BUILT IN TO THE KERNEL, code/logic which (a) prevents all outbound TCP session initiation and which also (b) strictly rate-limits all other protocols to some modest value. Remember, we're going to have a few billion of these devices online in the coming years. If even and modest subset of these can ever be tricked by an attacker into spewing non-rate-controlled traffic towards an attacker- selected target, then we're gonna have a problem. Regards, rfg

IoT is not a well-defined term. IoT implementations depend on system constraints. These constraints may relate to security (problems/solutions). It would be helpful to be more specific. See https://tools.ietf.org/html/rfc7228, for example. Cheers matthias On Mon, 24 Oct 2016, Jared Mauch wrote:

Top posting to provide some clarity:

1) Many IoT devices are connected via some cloud service, think Nest (for example) 2) Many IoT devices have cloud management, think of Ruckus, UBNT UniFi, etc that phone out to a site via DHCP option or otherwise. 3) Many IoT devices are something like a set top box, these need access to a CDN or similar to get the content for the users. 4) Many IoT consumers don’t read NANOG, so they also won’t set up a firewall other than to know it is a service impairing tool that must be disabled for their devices to work. 5) There are few/no new lessons here compared to the default install of Redhat 3.0.3 from the late-1990s. I recall it by default installed INN a usenet news server. As a usenet operator, this baffled me as they had insufficient memory, storage etc to do this task, and left security surface quite wide.

We must promote secure by default. This means some sort of secondary authentication such as a sticker on the device with default password or requiring the entering of a serial number or basic setup information to work. 6) Devices need to prompt for updates.

Apple has sorted this nicely, people are prompted for supported upgrades, disjoint from carriers and other issues. Contrast this with other ecosystems where upgrades require extra steps or have a non-OEM partner involved (eg: Cell Carrier, Cable Co).

These devices get less frequent updates, ostensibly for testing but also leave known issues exposed for months or years. 7) Malware can damage systems making them require extra steps to recover them. Whitehats may know some mitigation techniques, but can’t protect you either. Some people have taken a more aggressive approach, eg: https://gitlab.com/rav7teif/linux.wifatch

They will block others from infecting your device until it’s rebooted.

- Jared

...

On Oct 24, 2016, at 4:24 PM, Ronald F. Guilmette <rfg@tristatelogic.com> wrote:

In message <e364fcea-7105-b3b9-63a9-7d22ab83516c@nuclearfallout.net>, John Weekes <jw@nuclearfallout.net> wrote:

...

On 10/23/2016 4:19 PM, Ronald F. Guilmette wrote: jw>>> ... The ISPs behind those IP addresses have jw>>> received notifications via email... rfg>> Just curious... How well is that working out?

For the IoT botnets, most of the emails are ignored or rejected, because most go to providers who either quietly bitbucket them or flat-out reject all abuse emails. Most emails sent to mainland China, for instance, are in that category (Hong Kong ISPs are somewhat better)...

So, given the apparently impracticality of trying to clean up all of these kinds of messes via the good old-fashioned tedious and laborious method of emailing the relevant providers and then just sort-of vaguely hoping that they will -do something- responsible with it, I am just sitting here trying to dream up some sort of generalized long-term fix for -all- of these IoT DDoS type problems.

Maybe there just plain isn't any such thing as a general solution to the problem, because it may perhaps be just technically too complex. But I hope no one will begrudge me if I yearn for some sort of Grand Unified Field Theory of IoT security.

So, I have a thought... probably worth what you paid for it... and I'm just brave enough to throw it out on the table and then everybody can pile on and tell me what an idiot I am, for this or that perfectly sound technical reason. (I'll say up front that I don't even pretend to understand many of the protocols in use these days, in particular UPnP, and to be frank, I'd never even heard of SSDP until yesterday. So I can't and won't begrudge anybody who tells me that I have my head up... ummm... in the clouds.)

So anyway, here are the assumptions/assertions, perhaps wrong, which are my starting point:

1) I am not persuaded that IoT devices have a compelling need to them- selves initiate outbound TCP sessions, ever. (If I'm wrong about this, then I'm sure people here will tell me.)

2) Likewise, I am not persuaded that IoT devices have an absolute and compelling need to do very much in the way of UDP. Yes, I would like my smart XYZ device to always know what time it is, so, you know, a modest amount of NTP traffic is reasonable and to be expected. Other than that however, I don't see a compelling need. If you want to either control or get data out of your IoT device, you can make an inbound TCP connection to it.

(Somebody will probably say "Oh, no. We need to stream real-time video out of some of these things, and for that we absolutely have to send the stuff via outbound high-bandwidth UDP." but I am not persuaded that this is either absolutely necessary or even Good, i.e. from the point of view of the legitimate security concerns of the owner of the device.)

So, based on the above perhaps flawed assumptions, here is my idea. It is composed of two simple parts:

1) First, I will successfully complete my campaign to be elected King of the World. (Given the current poltical climate, worldwide, this should not be a problem, because I lie a lot.)

2) Second, once elected I will decree that in future all new IoT devices, and also all updates to firmware for existing IoT devices will have, BUILT IN TO THE KERNEL, code/logic which (a) prevents all outbound TCP session initiation and which also (b) strictly rate-limits all other protocols to some modest value.

Remember, we're going to have a few billion of these devices online in the coming years. If even and modest subset of these can ever be tricked by an attacker into spewing non-rate-controlled traffic towards an attacker- selected target, then we're gonna have a problem.

Regards, rfg

-- Matthias Waehlisch . Freie Universitaet Berlin, Computer Science .. http://www.cs.fu-berlin.de/~waehl

In message <FD166608-2C42-415D-B6F7-FD6921263E09@puck.nether.net>, Jared Mauch <jared@puck.nether.net> wrote:

Top posting to provide some clarity:

That's funny. Personally, I have always felt that top posting -destroys- clarity. But as Chaplin Tapman said in Catch-22 "I'm not here to judge you."

1) Many IoT devices are connected via some cloud service, think Nest

Yea. And? Are you saying that there is no way to make that connection happen without a kernel that's going to allow unlimited outbound bandwidth to entirely arbitrary targets?

2) Many IoT devices have cloud management, think of Ruckus, UBNT UniFi etc that phone out to a site via DHCP option or otherwise.

I admit that I -was not- thinking about such things when I posted my trivial proposal, in large part because I'd never heard of either before now. But that's OK. I've just googled them and I don't think either type of device really negates my simple proposal. Via the simple expedient of redefining the scope of the problem being addressed, I would still like to claim that what I proposed may have some merit, but only for "real" IoT devices. I will now define what I'd like to call "real IoT" devices as "stuff that isn't either general purpose computers *or* stuff like routers." The Ruckus thing and that UBNT UniFi thing appear to me to fall more into the class of devices I'd prefer to call "routers", i.e. stuff whose job it is to move quite a lot of packets, on a routine basis, in the outbound (Internet) direction. (Yes, I'm cheating by definining and/or re-defining my terms, on-the-fly, to exclude stuff that I don't want to be bothered thinking about right now. But maybe that isn't such a bad thing, or even such an intellectually dishonest thing as it may at first appear. It's best not to bite off more than you can chew, and I think it might be useful to solve the problem for a big swath of common "IoT" devices that either are now, or that will soon be on the Internet, even if I can't offer you a solution for, you know, poverty and world hunger at the same time.)

3) Many IoT devices are something like a set top box, these need access to a CDN or similar to get the content for the users.

I thought a bit about that since I posted, and here's my response... Yeabut traffic for -those- things is goona be 99.99% inbound. My Roku needs to connect out to get stuff, and maybe it is even is doing that via TCP. (I really don't know, cuz I never even thought about it before.) But even if my Roku is sucking down content via TCP connections which *it* initiated (i.e. "outbound" TCP connections) if there were a low fixed, maximum -outbound- bandwidth rate limit implemented for this thing, directly in its kernel, I believe that it could still do everything it is supposed to do with no problem. So fuggetabout what I said about totally disallowing outbound TCP. For many IoT devices, actually and totally disallowing outbound TCP connections might be workable... and for those, it would be a prudent and useful kernel-enforced limitation...,but for many others, like my Roku, maybe not. But neither your refrigerator nor my Roku really ever has any need to be sending multi megabits per second outbound. They just don't. I'm -not- suggesting that we put all these things on a diet and give them only 9600 baud to work with. I'm just saying that if they ever hit more than, say, .2 Mbps outbound, in total, over any short period of time, then I think their kernels would be doing the world a favor if they displayed some sort of an error code (where feasible) and if they then started just dropping outbound packets on the floor until things settle down and the -requested- outbound bandwidth drops back below the hard limit. (Of course, all traffic to internal interfaces, local loopback, and RFC1918 addresses should be utterly exempt from the cap.)

4) Many IoT consumers don't read NANOG, so they also won't set up a firewall other than to know it is a service impairing tool that must be disabled for their devices to work.

Well, yea, there's that, but also, with IPv6, as I understand the plan, every single device, no matter how lowly or absurd, is gonna get itself plugged right straight into the neural backbone of the entire planet. None of this NAT router or firewall stuff anymore! That's so last millenium. So there's not gonna be nothin' in our bright bright future for Luci and Desi to either enable or disable anymore, even if they wanted to. But here it seems that you're agreeing with me, and even making the case for me. The device itself should be fail-safe. It shouldn't need outside help in order to avoid bursting into flames and endangering innocents like a 1970's Ford Pinto.

5) There are few/no new lessons here compared to the default install of Redhat 3.0.3 from the late-1990s. I recall it by default installed INN a usenet news server.

An IoT is -not- a general purpose computer. In the latter case, it is assumed that the owner will "pop the hood" when it comes to the software configuration. In the former case it is assumed that the owner -won't- be doing that, or will have only a very minimalist ability to do so. (CAUTION! Risk of electrical shock! There are NO user-servicable components in this box.)

We must promote secure by default.

I love it when people agree with me.

This means some sort of secondary authentication such as a sticker on the device with default password or requiring the entering of a serial number or basic setup information to work.

Well, yea. That would be nice too. Force the user to set a unique password. How novel! Swell idea. Marvelous. I'm all for it. And we know that will work, absolutely 100%, until it doesn't, until the day when some pimply-faced teenager in Minsk figures out how to engineer a maliciously crafted JPEG, then uploads it to his Picassa album, and then gets his pal to spam out 50 million spams encouraging Joe Wankers everywhere to view the latest nude photos of Jennifer Lawrence on their Rokus, whereupon we'll all be right back where we were on Friday.

6) Devices need to prompt for updates.

See above. That would be nice too. I'm not persuaded that it will actually solve the problem though. (Think about this too: If the software minions hired by the manufacturer to code up the original firmware were careless enough to have created a serious security flaw in the -original- firmware, what makes you think that they are suddenly going to develop superior coding skills and that, thus, any update released by the company will be any more secure than the original flawed version? And that's even assuming that the same guys who coded up the original firmware are tasked to do maintenance fixes, which is most companies, they're not. The original coders all get moved over to the latest project in development, and the company hires one or two 19 year old H1-Bs just off the boat to handle all software maintenance on the "old" products. So nine times out of ten, the guys tasked with fixing the security problem(s) in the "old" products are even more inept dumbasses than the ones who coded up the original security flaws in the first place. How many securty fixes have had to have -their own- subsequent security fixes applied? Plenty.)

Apple has sorted this nicely, people are prompted for supported upgrades...

Umm, I'm getting the book off my shelf that purports to tell me how to disagree without being diagreeable. But I can't seem to find it so I'll just say what I mean: Rubbish! "Apple has sorted this nicely"??? You must be joking! I've got an iPhone 3GS. I went into the local Apple store more than a year ago and asked them how I could get the bloody thing to read and display PDF files. They were very polite, and tried hard to avoid snickering directly in front of me. And they were, of course, only too happy to show me a brand new iPhone 6 while telling me how little it would probably cost in conjunction with "my plan". (Hint: I don't have a plan. I have GoPhone dollars, which don't cost me anything if I don't use them, which, in general, I don't. And if Apple really thinks that I'm going to shell out several hundred real bucks for a piece of electronics that might accidentally end up in the washing machine or simply get ripped off they got another thing coming.) So, it turns out that Apple can't (or rather won't) give me any version of any of the last -3- major releases of IOS for my phone, because they quite rightly have figured out that it is not in their economic interest to do so. So I now own the Apple equivalent of Windows XP. No security fixes, no nothing. So when you say "Apple has sorted this nicely', you'll just have to forgive me if I take issue with that. More to the point however, this illustrates part of the overall problem. Manufacturers and vendors sell stuff to you with the expectation (and hope?) that it's all gonna end up in a landfill someplace within 5 years. They certainly don't plan on releasing any security updates past those five years, and that's even assuming that the company itself lasts that long. (Many don't.) So, six years from today we'll all wake up one morning, turn on CNN, and find out that a maliciously crafted packet sent to 35 million vacation pet feeders has just resulted in the disapperaance from the Internet of the entire continent of South America. So Wolf Blitzer or Brian Krebs will pick up the phone to call the manufacturer to ask if they're going to be releasing a security update for this massive problem, only to find out that the phone number is dead and the company declared bankruptcy three years earlier. This is not a happy ending. If all of the *&^%$# damn stupid vacation pet feeders had originally shipped with outbound rate limits hard-coded in the kernel, maybe this could have been avoided.

7) Malware can damage systems making them require extra steps to recover them. Whitehats may know some mitigation techniques, but can't protect you

I do so love to end on a note of agreement. I reiterate, fail-safe by design, and outbound rate limits for all IoT things to which such limits can be sensibly applied without damaging or materially degrading functionality... which is to say most of them. Regards, rfg

On 25 October 2016 at 09:37, Jean-Francois Mezei < jfmezei_nanog@vaxination.ca> wrote:

One way around this is for the pet feeder to initiate outbound connection to a central server, and have the pet onwer connect to that server to ask the server to send command to his pet feeder to feed the dog.

This is pretty common but, IMHO, the worst solution to this problem. It creates a dependence on a cloud service which is typically undocumented (what protocol do they use? where is the server located, China?); a centralised service is a security risk in it's own right (crack one server, own all the pet feeders); and it is liable to disappear when the operator goes out of business, rendering all the products sold useless. A strength of IP is that it is fundamentally a peer-to-peer protocol, please don't break that. NAT broke it but IPv6 can fix it again. There's nothing wrong with accepting incoming connections if the device is secure. If your problem is security, fix that. Don't throw the baby out with the bath water. Aled

In message <580F19BF.2070002@vaxination.ca>, Jean-Francois Mezei <jfmezei_nanog@vaxination.ca> wrote:

One way around this is for the pet feeder to initiate outbound connection to a central server, and have the pet onwer connect to that server to ask the server to send command to his pet feeder to feed the dog.

This way, there need not be any inbound connection to the pet feeder.

Right. And even that one outbound connection (from the pet feeder to the central server) isn't going to need much in the way of bandwidth. So if the kernel wakes up and notices "Whoa! Wait a minute here! This box is sending out in excess of 100 kilobits per second!" then something is REALLY wrong here. Time to let that ethernet interface take a little nap. Likewise if the kernel notices that the box has just sent out in excess of, say, 100 outbound UDP packets with destination port set to 53 in the last 60 seconds. Time to send DNS to its room for a little "bad behavior" time out! Because in practice, for most or all of the kinds of IoT devices I can think of, (a) they shouldn't be sending out DNS -responses-, ever, and (b) if they are sending out more than, say, 100 outbound DNS -queries- per minute, then the only possible reason they would be doing so is that the box itself has been enslaved and is participating in a DNS reflection attack. The point is that for many (most?) types of IoT devices the engineers who designed the box should be able to tell... or at least predict... what the maximum theoretical outbound bandwidth usage and/or the maximum theoretical outbound packets per second for various protocols and ports ought to be... assuming that the box is functioning "normally", as the original design engineers intended, and not as some miscreant's slave. So then they, the original design engineers, can hard code limits into the kernel... limits that are, say, 25% above these worst case "normal operation" limits, thus allowing for a healthy margin of error. The result would be a box that can't be turned into a weapon, no matter what, at least not without loading up all new (and malicious) firmware, a task which should require at least some direct, hands-on manually fiddling (such as pressing and holding the reset button) in any event. I refer again to the Ford Pinto, and also to the Apollo 1 fire. I don't think that there's really a problem with engineering belt-and-suspenders safeguards into the firmware of IoT things. The only real problem is providing both companies and engineers with the motivation necessary to insure they will do so. Let's hope that nobody has to die before we find a way to manufacture that motivation. (A little temporary bad press for one Chinese manu- facturer of CCTV cameras isn't going to be sufficient, I'm afraid.) Regards, rfg

On Thu, 27 Oct 2016 07:59:00 +0200, Eliot Lear <lear@ofcourseimright.com> may have written:

Well yes. uPnP is a problem precisely because it is some random device asserting on its own that it can be trusted to do what it wants. Had

From my own personal use (and I'm aware that this isn't a general solution), I'd like a device that sat on those uPnP requests until I logged into the admin interface to review them. Now if you could automate _me_ then it might become more generally useful :- uPnP(ssh, for admin access) -> f/w f/w -> uPnP device: Don't be silly.

But if instead of a pet feeder we're talking about a home file sharing system or a video camera where you don't want to share the feed into the cloud? There will be times when people want inbound connections. We need an architecture that supports them.

As someone who manages an application-based firewall, every problem looks like it would be easier to solve using an application-based firewall :) -- Mike Meredith, University of Portsmouth Principal Systems Engineer, Hostmaster, Security, and Timelord!

Hi Mike, On 10/27/16 11:04 AM, Mike Meredith wrote:

...

Well yes. uPnP is a problem precisely because it is some random device asserting on its own that it can be trusted to do what it wants. Had From my own personal use (and I'm aware that this isn't a general solution), I'd like a device that sat on those uPnP requests until I logged into the admin interface to review them. Now if you could automate _me_

On Thu, 27 Oct 2016 07:59:00 +0200, Eliot Lear <lear@ofcourseimright.com> may have written: then it might become more generally useful :-

You need to go further. It is no longer enough to tackle this problem simply as a firewall problem, because there are too many reflection-style attacks. Not only do you want to prevent devices from opening pinholes to the Internet, but you really want to know what they're going to be doing inside the home. And Quite frankly, I disagree that you want to nag the user unless it is absolutely necessary. To me, that means authorizing the device in the first place, and the access point having access to enough intelligence to know what sort of access is necessary for a device, given its purpose.

As someone who manages an application-based firewall, every problem looks like it would be easier to solve using an application-based firewall :)

I don't generally prefer application firewalls except in limited circumstances. Eliot

I suppose someone could modify this Mirai virus to instead inject antivirus software. I know, illegal. What would the manufacturers' response be if this virus had instead just shut down, possibly in some cases physically damaged the devices or otherwise caused them to cease functioning ever again (wiped all their software or broke their bootability), rather than just hijacked them for a while? One manufacturer agreed that half a million of their devices were involved in the attacks in a recent press release. And they apologize. What if half a million of their devices just suddenly stopped working, forever? I suspect that would require a somewhat more expensive response. They have to be thinking that sort of thing or else they're being pretty dumb. -- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

On October 28, 2016 at 00:07 jxh@jxh.com (Jim Hickstein) wrote:

On 10/27/16 22:59, bzs@TheWorld.com wrote:

...

What would the manufacturers' response be if this virus had instead just shut down, possibly in some cases physically damaged the devices or otherwise caused them to cease functioning ever again (wiped all their software or broke their bootability), rather than just hijacked them for a while?

A virus that kills its host (too much of the time) is not successful.

Hmm. So now we assume we are dealing with rational actors? I suppose one can find some rational motives in bringing down Dyn but I don't see that it's all that different from bricking half a million (for starters) IoT devices. Thus far the goal just seems to be mayhem. -- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

On October 28, 2016 at 22:27 list@satchell.net (Stephen Satchell) wrote:

On 10/28/2016 10:14 PM, bzs@TheWorld.com wrote:

...

Thus far the goal just seems to be mayhem.

Thus far, the goal on the part of the botnet opearators is to make money. The goal of the CUSTOMERS of the botnet operators? Who knows?

You're speaking in general terms, right? We don't know much anything about the perpetrators of these recent Krebs and Dyn attacks such as whether there was any DDoS for hire involved. -- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

On October 29, 2016 at 14:07 esr@thyrsus.com (Eric S. Raymond) wrote:

bzs@TheWorld.com <bzs@TheWorld.com>:

...

On October 28, 2016 at 22:27 list@satchell.net (Stephen Satchell) wrote:

...

On 10/28/2016 10:14 PM, bzs@TheWorld.com wrote:

...

Thus far the goal just seems to be mayhem.

Thus far, the goal on the part of the botnet opearators is to make money. The goal of the CUSTOMERS of the botnet operators? Who knows?

You're speaking in general terms, right? We don't know much anything about the perpetrators of these recent Krebs and Dyn attacks such as whether there was any DDoS for hire involved.

We can deduce a lot from what didn't happen.

You don't build or hire a botnet on Mirai's scale with pocket change.

Do we know this or is this just a guess? The infamous 1988 Morris worm was also thought to be something similarly sinister for a short while until Bob Morris, Jr et al owned up to it just being an experiment by a couple of students gone out of control. Back around 1986 I accidentally brought down at least half the net by submitting a new hosts file (for Boston Univ) with an entry that tickled a bug in the hosts.txt->/etc/hosts code which everyone ran at midnight (whatever) causing a loop which filled /tmp (this would be unix hosts but by count they were by far most of the connected servers) and back then a full /tmp crashed unix and it often didn't come back up until a human intervened. Ok I doubt this was an accident, tho its scale could've been an accident, a prank gone wild. Anyhow what do we *know*? That the effect was large doesn't necessarily imply that it required a lot of resources. We live in a world rife with asymmetric warfare. A few boxcutters and 3,000+ people dead.

And the M.O. doesn't fit a criminal organization - no ransom demand, no attempt to steal data.

Same question. Would Dyn et al publicize ransom demands at this point? And even if not how do we rule out a prank or similar? Is there something specific about this attack which required significant resources? How significant?

That means the motive was prep for terrorism or cyberwar by a state-level actor. Bruce Schneier is right and is only saying what everybody else on the InfoSec side I've spoken with is thinking - the People's Liberation Army is the top suspect, with the Russian FSB operating through proxies in Bulgaria or Romania as a fairly distant second.

Well, barring further details one can go anywhere with a few suppositions.

Me, I think this fits the profile of a PLA probing attack perfectly. -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

-- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

"That means the motive was prep for terrorism or cyberwar by a state-level actor. " Or, quite possibly ( I would argue probably) it was marketing. Show off the capabilities of the botnet to garner more interest amongst those who pay for use of such things. On Sat, Oct 29, 2016 at 2:07 PM, Eric S. Raymond <esr@thyrsus.com> wrote:

bzs@TheWorld.com <bzs@TheWorld.com>:

...

On October 28, 2016 at 22:27 list@satchell.net (Stephen Satchell) wrote:

...

On 10/28/2016 10:14 PM, bzs@TheWorld.com wrote:

...

Thus far the goal just seems to be mayhem.

Thus far, the goal on the part of the botnet opearators is to make money. The goal of the CUSTOMERS of the botnet operators? Who knows?

You're speaking in general terms, right? We don't know much anything about the perpetrators of these recent Krebs and Dyn attacks such as whether there was any DDoS for hire involved.

We can deduce a lot from what didn't happen.

You don't build or hire a botnet on Mirai's scale with pocket change. And the M.O. doesn't fit a criminal organization - no ransom demand, no attempt to steal data.

That means the motive was prep for terrorism or cyberwar by a state-level actor. Bruce Schneier is right and is only saying what everybody else on the InfoSec side I've spoken with is thinking - the People's Liberation Army is the top suspect, with the Russian FSB operating through proxies in Bulgaria or Romania as a fairly distant second.

Me, I think this fits the profile of a PLA probing attack perfectly. -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

In message <20161029180730.GA10801@thyrsus.com>, "Eric S. Raymond" <esr@thyrsus.com> wrote:

You don't build or hire a botnet on Mirai's scale with pocket change.

Proof please? Sorry, but I am compelled to call B.S. on the above statement. This is a really important point that I, Krebs, and others have been trying to drive home: In an era when you've got a half million CCTV cams just lying around without even passwords on them, and in an era when nobody makes any fuss anymore about the dozens or hundreds or people and/or organizations (e.g. Shodan) that are out there scanning your box and my box and everybody's boxes, every damn day, you don't need to be either an omnious "state actor" or even SPECTER to assemble a truly massive packet weapon. Two kids with a modest amount of knowledge and a lot of time on their hands can do it from their mom's basement. It is comforting, for some, to think that this is not the case, just as it is, to this day, comforting, for some, to believe, based on scant evidence, that it -wasn't- just some lone nut case who killed President Kennedy. Psychologically, people have trouble coming to terms with great impactful tragedies unless they can be blamed on large, unseen, but enormously capable dark forces. And the actual available hard evidence relating to such events does not diminish the human yearning for a convenient comic book supervillain to pin it all on.

And the M.O. doesn't fit a criminal organization - no ransom demand, no attempt to steal data.

Allow me to refer you to an alternative possible motivation: https://en.wiktionary.org/wiki/lulz

That means the motive was prep for terrorism or cyberwar by a state-level actor.

Frankly, I am dismayed to see a well-known Internet persona with a respected name spreading this kind of absurd, alarmist, over-the-top, retorical fear- mongering inference, which is without clear basis in either fact or evidence. Even the hardest of the hard-core dyed-in-the-wool Clinton surrogates are too circumspect in their pronouncements (i.e. with respect to Russia's "obvious" connection to the DNC hack) to ever reach anything like this level of unfounded hyperbole. (And for the record, I am no Trump supporter either. I find myself equally disgusted when either side employs the currently fashionable verbal sleight-of-hand that politicians of all stripes have, of late, adopted whenever they want to say something without themselves having to take responsibility for its truth or accuracy. I get angry when I hear Clinton surrogates using the "Some people are saying..." dodge, e.g. when it comes to alleged nefarious Russian involvement with anything and everything evil, just as I do when Trump uses the exact same dodge in reference to... well... everything.)

Bruce Schneier is right and is only saying what everybody else on the InfoSec side I've spoken with is thinking - the People's Liberation Army is the top suspect, with the Russian FSB operating through proxies in Bulgaria or Romania as a fairly distant second.

Yes, but I believe that Schneier was a bit more careful to separate the known facts from his personal speculations. In any case, all of this searching for who is to blame isn't contributing a damn thing towards actually fixing the problem. And if we really need to find someone to blame, I think we should all just look in the mirror. We, society, but especially those of us with more-than-average techno savvy, have for years been only too eager to lap up whatever whiz-bang new techno gadgets industry could crank out, with barely an afterthought given to the longer term implications, like security and also how the hell we are ever going to be able to recycle any of this s***. We've all been doing the exact same thing, since at least Windows 3.1 or earlier, and yet we continue to expect a different outcome. We eagerly grab for new capabilities and new gadgets, thinking about security last or, more often, not at all. In short, to quote Pogo, "We have met the enemy and he is us." Regards, rfg P.S. Even if the evidence were to support the view that only a superpower- level nation-state could have pulled off the Dyn attack... and I'm not at all persuaded that it does... it kills me that everyone seems to jump, within a millisecond, immediately from -that- unwarranted conclusion to the separate unwarranted conclusion that it must have been either Russia or China. Apparently, nobody even stops to consider the *other* elephant in the room, the one that stretches from sea to shining sea, and which itself has been heard to publically brag about its own cyber-offensive capabilities of late. In short, maybe our own guys did this. OK, so maybe this theory -is- worthy of le Carre, but that don't mean it ain't possible. I mean we aren't stupid. We don't build warehouses full of nuclear weapons without at least testing the design once or twice first, you know, to make sure they aren't all gonna end up being duds on impact. (Mike Rogers would probably lose his stripes -and- his pension if an actual cyber-confrontation came and it was revealed that nobody had ever actually tested any of our theoretical capabilities.) And when we do test our strategic weapons, we -don't- test them by dropping one on China, or Russia, or Iran, and then saying "Oh! Sorry. Please excuse us. Just testing." Doing that could come with consequences. So, what's worse? That Amazon and Twitter should be offline for a couple of hours in the middle of October (i.e. for a little test) or that any one of our many enemies should, you know, maybe take them down for days on end in early December, at the height of the shopping season, with us having no real/tested retaliatory capability? (Nutty conspiracy theorists might even suggest that staging a limited attack like this is a rather obvious way for certain three letter entities and/or parts of DoD to squeeze even more out of congress than the obscenely vast sums they are already getting, but I personally won't even go there. As I've noted, there are plenty of pragmatic and entirely non-nefarious/non-self-serving reasons why our own guys might have done a small/short practice run like this.) Anyway, when it comes to attribution, the bottom line is that all anybody has to do is to run their C&C through two or three levels of chained compromised socks proxies, e.g. in Tajikistan, Bolivia, and Singapore, and then, as a practical matter, nobody will ever be able to say for sure who you are. It's all just guesswork, and much of it, alas, isn't even all that educated. Who says that the Russians or the Chinese took down Dyn? Are these the same people who told us the fantasy... later retracted... that North Korea hacked Sony? Are these the same people who told us that Saddam absolutely positively had weapons of mass destruction? I would have hoped that all of us in this country (US) would have become just a little bit more skeptical of press reports and "expert" pronouncements by now. Remember the Maine!

In message <20161030044342.GA18488@thyrsus.com>, "Eric S. Raymond" <esr@thyrsus.com> wrote:

Ronald F. Guilmette <rfg@tristatelogic.com>:

...

Two kids with a modest amount of knowledge and a lot of time on their hands can do it from their mom's basement.

I in turn have to call BS on this. If it were really that easy, we'd be inundated by Mirais -- we'd have several attacks a *day*.

You need to get out more. http://www.nab.org/cybersecurity/Verisign-report-ddos-trends-Q22016.pdf It *is* happening every day. You just don't hear about it on CNN because a "little" 80Mbps DDoS isn't even worthy of a headline anymore, even though such an attack could CRUSH a local bank, and even many regional banks into utter oblivion. Now, where did I put those bitcoins... It's ransom time! Regards, rfg P.S. Of course, things were oh so much better, ya know, back in those idylic halcyon days a decade and a half ago... Denial of e-commerce Feb 10th 2000 http://www.economist.com/node/281531 "... The Computer Emergency Response Team of Carnegie Mellon University hears of roughly four DOS attacks a day..." Whew! I guess we need to count our blessings that insightful visionary industry leaders came forward, back in the early 00s, and spearheaded the global changes necessary to insure that DDoS attacks would become a thing of the past, and a distant memory. Oh! Wait! Nevermind. Sorry. I guess that I was dozing off and dreaming again. At the current rate of progress I think that I can confidently predict that the Internet industry ought to have this whole problem completely licked by the early 23rd century, you know, at the very latest.

Is this report reliable? I don't know off-hand: http://www.csoonline.com/article/3134721/security/amateurs-were-behind-the-d... or: http://tinyurl.com/zb9mpy5 Amateurs were behind the Dyn Inc. DDoS attack, report says Flashpoint says that despite speculation, nothing they’ve seen points to political motivation or extortion Here is Flashpoint's actual report link: https://www.flashpoint-intel.com/action-analysis-mirai-botnet-attacks-dyn/ or http://tinyurl.com/hrhewxg "...In its investigation of Dyn DDoS attacks, Flashpoint discovered that the infrastructure used in the attack also targeted a well-known video game company. While there does not appear to have been any disruption of service, the targeting of a video game company is less indicative of hacktivists, state-actors, or social justice communities, and aligns more with the hackers that frequent online hacking forums. These hackers exist in their own tier, sometimes called “script kiddies,” and are separate and distinct from hacktivists, organized crime, state-actors, and terrorist groups. They can be motivated by financial gain, but just as often will execute attacks such as these to show off, or to cause disruption and chaos for sport..." "...Flashpoint assesses with moderate confidence that these attacks were not financially or politically motivated..." P.S. not sure why I include tinyurls other than long URLs tend to get messed up in some MUAs and on rare occasion one has to retype one in and tinyurls are tiny. -- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

On 30/10/2016 12:43 AM, Eric S. Raymond wrote:

Ronald F. Guilmette <rfg@tristatelogic.com>:

...

Two kids with a modest amount of knowledge and a lot of time on their hands can do it from their mom's basement.

I in turn have to call BS on this. If it were really that easy, we'd be inundated by Mirais -- we'd have several attacks a *day*.

It's not easy, Mirai was closed source until the actor released it. We see a pattern again and again, where the bad guys find a private monetization strategy, milk it, and get out before too much attention is focused on just them. By dumping the code, the Mirai actors obfuscate attribution. Now that the code is public, we see a huge surge in dumb & pointless attacks against gaming/mod sites, Dyn, public schools and so on. We also see bad code "improvements" which were recently referenced. http://motherboard.vice.com/read/wannabe-hackers-are-adding-terrible-and-stu... The long term problem isn't any manufacturer or Mirai, it's going to be the long tail of IoT devices that never see a patch, deployed by people who don't know anything about security (nor should they need to... flame suit on). When millions of any type of device are put online, times thousands of products, it only takes one bad guy's "a-ha" moment for this to happen again. They'll milk it for a while, the attack vector/method will get pushed down to the skid level, and we'll see a massive increase in un-targeted attacks by those script kiddies until the cycle repeats. There's an endless fresh supply of script kiddies. While I agree with BCP38 etc, it wouldn't have prevented Mirai. Self-update functions at some point for these devices are going to get borked as well, such as a company going bust or forgetting to renew their auto-update target domain. If you can't get (thousands?) of major operators to deploy common sense security configurations, how will similar best practices be implemented by tens of thousands of manufacturers? Putting device regulations in one country won't impact the rest of the internet's connected devices either. Solutions...? Someone's going to have to take out a hammer, not a scalpel, for these issues.

In message <20161108035148.2904B5970CF1@rock.dv.isc.org>, Mark Andrews <marka@isc.org> wrote:

* Deploying regulation in one country means that it is less likely to be a source of bad traffic. Manufactures are lazy. With sensible regulation in single country everyone else benefits as manufactures will use a single code base when they can.

I said that too, although not as concisely.

* Automated updates do reduce the numbers of vulnerable machines to known issues. There are risks but they are nowhere as bad as not doing automated updating.

From a software perspective, building extra layers of constraints is not

I still maintain, based upon the abundant evidence, that generallized hopes that timely and effective updates for all manner of devices will be available throughout the practical lifetime of any such IoT thingies is a mirage. We will just never be there, in practice. And thus, manufacturers should be encouraged, by force of law if necessary, to design software with a belt-and-suspenders margin of safety built in from the first day of shipping. You don't send out a spacecraft, or a medical radiation machine, without such addtional constraints built in from day one. You don't send out such things and say "Oh, we can always send out of firmware update later on if there is an issue." that hard to do, and people have been doing this kind of thing already for decades. It's called engineering. The problem isn't in anybody's ability or inability to do safety engineering in the firmware of IoT things. The only problem is providing the proper motivation to cause it to happen. Regards, rfg

On Fri, Nov 11, 2016 at 1:55 AM, Eliot Lear <lear@ofcourseimright.com> wrote:

It is worth asking what protections are necessary for a device that regulates insulin.

Insulin pumps are an example of devices that have been over-regulated to the point where any and all innovation has been stifled. There have been hardly any changes in the last 10+ years, during a time when all other technology has advanced quite a bit. Its off-topic for Nanog, but i promise you this is very frustrating and annoying topic that hits me close to home. There has to be a middle ground. I guarantee we do not want home firewalls, and all the IoT devices to be regulated like insulin pumps and other medical devices. I think I'm starting to agree with those that want to keep government regulation out of this arena... Marcel

Eliot

On 11/8/16 6:05 AM, Ronald F. Guilmette wrote:

...

In message <20161108035148.2904B5970CF1@rock.dv.isc.org>, Mark Andrews <marka@isc.org> wrote:

...

* Deploying regulation in one country means that it is less likely to be a source of bad traffic. Manufactures are lazy. With sensible regulation in single country everyone else benefits as manufactures will use a single code base when they can. I said that too, although not as concisely.

...

* Automated updates do reduce the numbers of vulnerable machines to known issues. There are risks but they are nowhere as bad as not doing automated updating. I still maintain, based upon the abundant evidence, that generallized hopes that timely and effective updates for all manner of devices will be available throughout the practical lifetime of any such IoT thingies is a mirage. We will just never be there, in practice. And thus, manufacturers should be encouraged, by force of law if necessary, to design software with a belt-and-suspenders margin of safety built in from the first day of shipping.

You don't send out a spacecraft, or a medical radiation machine, without such addtional constraints built in from day one. You don't send out such things and say "Oh, we can always send out of firmware update later on if there is an issue."

From a software perspective, building extra layers of constraints is not that hard to do, and people have been doing this kind of thing already for decades. It's called engineering. The problem isn't in anybody's ability or inability to do safety engineering in the firmware of IoT things. The only problem is providing the proper motivation to cause it to happen.

Regards, rfg

On 10/29/2016 05:32 PM, Ronald F. Guilmette wrote:

you don't need to be either an omnious "state actor" or even SPECTER to assemble a truly massive packet weapon.

Please, it's SPECTRE .... show some respect

On Fri, Oct 28, 2016 at 12:07:17AM -0500, Jim Hickstein wrote:

A virus that kills its host (too much of the time) is not successful.

True. On the other hand: "Some men aren't looking for anything logical, like money. They can't be bought, bullied, reasoned, or negotiated with. Some men just want to watch the world burn." I have no doubt whatsoever that some of our adversaries fall squarely into this category. ---rsk

Hi Keith, On 10/28/16 1:55 AM, Keith Medcalf wrote:

...

...

The problem is in allowing inbound connections and going as far as doing UPnP to tell the CPE router to open a inbound door to let hackers loging to that IoT pet feeder to turn it into an agressive DNS destroyer. Well yes. uPnP is a problem precisely because it is some random device asserting on its own that it can be trusted to do what it wants. Had that assertion come from the manufacturer, at least you would know that the device was designed to require that sort of access.** And why would anyone in their right mind trust the manufacturer to make this decision? <Shudder>

Because the manufacturer designed the device and knows best as to what sort of access it will require. Consider that today most devices have unfettered outbound access, and many can arrange for unfettered inbound access. That's Not Good®. That doesn't mean that network administrators shouldn't be the kings and queens of their castles, but as I'm sure you well know, home users don't really know how to rule, and so they need some good defaults. Put it another way: you bring home a NEST and the first thing you the expert might do is read the net to figure out which ports to open. Are you really going to not open those ports? Eliot

Hi, Hi,

Put it another way: you bring home a NEST and the first thing you the expert might do is read the net to figure out which ports to open. Are you really going to not open those ports?

Put onto its own isolated vlan with only internet access. Unfortunately no basic routers that are for the home come with such a setup by default. That's the first big win. alan

Hi Chris, On 10/25/16 1:51 PM, Chris Boyd wrote:

...

On Oct 25, 2016, at 3:10 AM, Ronald F. Guilmette <rfg@tristatelogic.com> wrote:

An IoT is -not- a general purpose computer. In the latter case, it is assumed that the owner will "pop the hood" when it comes to the software configuration. Ah, but they are. In many cases you can ship a product faster and cheaper with an ARM based system running a stripped down Linux and some specialty I/O than building a properly hardened custom microcontroller.

That something has a CPU doesn't tell you whether it is a general purpose computer. What tells you if a device is a general purpose is whether it is intended for particular uses or not (the key word there being "purpose"). More importantly, if you view every Thing as a general purpose computer you are missing an opportunity to impose an engineering constraint on the problem space. If that in turn let's you easily solve for the general case, you've had a huge win. Eliot

May as well throw in my idea here too. Can ISPs just block their clients from being reached by CNC servers? If we could get a service like Spamhaus for botnets and have service providers automatically blackhole those CNC IPs. Having this done at the tier 1 level would probably cause some pain to the botnets out there. Rather than pushing customers to secure their stuff (impossible), how about we try to stop communication to the CNC. For example, these guys https://zeustracker.abuse.ch/monitor.php keep track of Zeus, if we automatically blackholed those IPs as they get updated it could put a big dent in the effectiveness of Zeus. It won't solve the problem, but it could put a dent in it. ISP's might like to implement it since it could cut down on transit costs due to DDoS traffic. -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Ronald F. Guilmette Sent: Monday, October 24, 2016 2:25 PM To: nanog@nanog.org Subject: Spitballing IoT Security In message <e364fcea-7105-b3b9-63a9-7d22ab83516c@nuclearfallout.net>, John Weekes <jw@nuclearfallout.net> wrote:

On 10/23/2016 4:19 PM, Ronald F. Guilmette wrote: jw>>> ... The ISPs behind those IP addresses have received notifications jw>>> via email... rfg>> Just curious... How well is that working out?

For the IoT botnets, most of the emails are ignored or rejected, because most go to providers who either quietly bitbucket them or flat-out reject all abuse emails. Most emails sent to mainland China, for instance, are in that category (Hong Kong ISPs are somewhat better)...

So, given the apparently impracticality of trying to clean up all of these kinds of messes via the good old-fashioned tedious and laborious method of emailing the relevant providers and then just sort-of vaguely hoping that they will -do something- responsible with it, I am just sitting here trying to dream up some sort of generalized long-term fix for -all- of these IoT DDoS type problems. Maybe there just plain isn't any such thing as a general solution to the problem, because it may perhaps be just technically too complex. But I hope no one will begrudge me if I yearn for some sort of Grand Unified Field Theory of IoT security. So, I have a thought... probably worth what you paid for it... and I'm just brave enough to throw it out on the table and then everybody can pile on and tell me what an idiot I am, for this or that perfectly sound technical reason. (I'll say up front that I don't even pretend to understand many of the protocols in use these days, in particular UPnP, and to be frank, I'd never even heard of SSDP until yesterday. So I can't and won't begrudge anybody who tells me that I have my head up... ummm... in the clouds.) So anyway, here are the assumptions/assertions, perhaps wrong, which are my starting point: 1) I am not persuaded that IoT devices have a compelling need to them- selves initiate outbound TCP sessions, ever. (If I'm wrong about this, then I'm sure people here will tell me.) 2) Likewise, I am not persuaded that IoT devices have an absolute and compelling need to do very much in the way of UDP. Yes, I would like my smart XYZ device to always know what time it is, so, you know, a modest amount of NTP traffic is reasonable and to be expected. Other than that however, I don't see a compelling need. If you want to either control or get data out of your IoT device, you can make an inbound TCP connection to it. (Somebody will probably say "Oh, no. We need to stream real-time video out of some of these things, and for that we absolutely have to send the stuff via outbound high-bandwidth UDP." but I am not persuaded that this is either absolutely necessary or even Good, i.e. from the point of view of the legitimate security concerns of the owner of the device.) So, based on the above perhaps flawed assumptions, here is my idea. It is composed of two simple parts: 1) First, I will successfully complete my campaign to be elected King of the World. (Given the current poltical climate, worldwide, this should not be a problem, because I lie a lot.) 2) Second, once elected I will decree that in future all new IoT devices, and also all updates to firmware for existing IoT devices will have, BUILT IN TO THE KERNEL, code/logic which (a) prevents all outbound TCP session initiation and which also (b) strictly rate-limits all other protocols to some modest value. Remember, we're going to have a few billion of these devices online in the coming years. If even and modest subset of these can ever be tricked by an attacker into spewing non-rate-controlled traffic towards an attacker- selected target, then we're gonna have a problem. Regards, rfg

There's a buffer overrun in some software, so let's just remove all passwords (and keys), since they can get in anyway. Just pointing out flawed logic. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "J. Oquendo" <joquendo@e-fensive.net> To: "Steve Mikulasik" <Steve.Mikulasik@civeo.com> Cc: nanog@nanog.org Sent: Monday, October 24, 2016 3:53:25 PM Subject: Re: Spitballing IoT Security On Mon, 24 Oct 2016, Steve Mikulasik wrote:

if we automatically blackholed those IPs as they get updated it could put a big dent in the effectiveness of Zeus.

That would involve someone lifting a finger and implement a config change. Much easier to implement BCP38 or was it RFC 4732? Would never work the moment someone has to lift a finger. /* I think I'll change my position on BCP38. It's pointless to try blocking spoofed source addresses because: * It doesn't solve every single problem * It means more effort for service providers * It requires more CPU processing power * Using it will generate smarter "black hats". https://www.nanog.org/mailinglist/mailarchives/old_archive/2004-10/msg00132.... */ -- =+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ J. Oquendo SGFA, SGFE, C|EH, CNDA, CHFI, OSCP, CPT, RWSP, GREM "Where ignorance is our master, there is no possibility of real peace" - Dalai Lama 0B23 595C F07C 6092 8AEB 074B FC83 7AF5 9D8A 4463 https://pgp.mit.edu/pks/lookup?op=get&search=0xFC837AF59D8A4463

Oh, yeah, list e-mail usually just gets skimmed through. No time for reading in detail or links. ;-) Sorry. :-\ ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Hugo Slabbert" <hugo@slabnet.com> To: "Mike Hammett" <nanog@ics-il.net> Cc: nanog@nanog.org Sent: Monday, October 24, 2016 5:21:48 PM Subject: Re: Spitballing IoT Security It's possible you might have wanted to read the link for the context that pointed this out as sarcastic hyperbole, though the text as-is could (unfortunately) have been read as serious. -- Hugo Slabbert | email, xmpp/jabber: hugo@slabnet.com pgp key: B178313E | also on Signal On Mon 2016-Oct-24 17:17:43 -0500, Mike Hammett <nanog@ics-il.net> wrote:

There's a buffer overrun in some software, so let's just remove all passwords (and keys), since they can get in anyway.

Just pointing out flawed logic.

----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com

Midwest-IX http://www.midwest-ix.com

----- Original Message -----

From: "J. Oquendo" <joquendo@e-fensive.net> To: "Steve Mikulasik" <Steve.Mikulasik@civeo.com> Cc: nanog@nanog.org Sent: Monday, October 24, 2016 3:53:25 PM Subject: Re: Spitballing IoT Security

On Mon, 24 Oct 2016, Steve Mikulasik wrote:

...

if we automatically blackholed those IPs as they get updated it could put a big dent in the effectiveness of Zeus.

That would involve someone lifting a finger and implement a config change. Much easier to implement BCP38 or was it RFC 4732? Would never work the moment someone has to lift a finger.

/* I think I'll change my position on BCP38. It's pointless to try blocking spoofed source addresses because:

* It doesn't solve every single problem * It means more effort for service providers * It requires more CPU processing power * Using it will generate smarter "black hats".

https://www.nanog.org/mailinglist/mailarchives/old_archive/2004-10/msg00132....

*/

-- =+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ J. Oquendo SGFA, SGFE, C|EH, CNDA, CHFI, OSCP, CPT, RWSP, GREM

"Where ignorance is our master, there is no possibility of real peace" - Dalai Lama

0B23 595C F07C 6092 8AEB 074B FC83 7AF5 9D8A 4463 https://pgp.mit.edu/pks/lookup?op=get&search=0xFC837AF59D8A4463

On Mon, Oct 24, 2016 at 01:24:59PM -0700, Ronald F. Guilmette wrote:

2) Second, once elected I will decree that in future all new IoT devices, and also all updates to firmware for existing IoT devices will have, BUILT IN TO THE KERNEL, code/logic which (a) prevents all outbound TCP session initiation and which also (b) strictly rate-limits all other protocols to some modest value.

I like this idea. But unfortunately, I think it has no chance of succeeding. The makers of IoT devices are falling all over themselves to rush products to market as quickly as possible in order to maximize their profits. They have no time for security. They don't concern themselves with privacy implications. They don't run networks so they don't care about the impact their devices may have on them. They don't care about liability: many of them are effectively immune because suing them would mean trans-national litigation, which is tedious and expensive. (And even if they lost: they'd dissolve and reconstitute as another company the next day.) They don't even care about each other -- I'm pretty sure we're rapidly approaching the point where toasters will be used to attack garage door openers and washing machines. I think our working assumption should be that there will be zero cooperation from the IoT vendors. (Yeah, once in a while one might actually step up, but that will merely be a happy anomaly.) After all, why should they care? It doesn't impact their profits, and profits are all they care about. They're not the ones fielding support calls or frantically trying to stop a DoS or trying to work out a mitigation strategy or participating in this discussion thread. So they don't care. They don't have to. ---rsk

Eric, I agree that the home router is a viable choke point, and even though we can’t quickly roll out new firmware, if we had started this ten years ago we’d be done by now! So this is the ten-year plan, but still worth doing. I also really like the idea of offering open source options to vendors, many of whom seem to illegally take that privilege anyway. A key fast-path component, though, is in my opinion a new RFC for IoT security best practices, and probably some revisions to UPNP. The IoT RFC would spell out basic rules for safe devices: no back doors, no default passwords, no gratuitous inbound connections, etc. It would also make encryption a requirement, and limit how existing UPNP is deployed to prevent unnecessarily exposing vulnerable TCP/UDP ports to the wild. With this RFC in hand, and an appropriate splashy icon for vendor packaging (“RFC 9999 ThingSafe!”), vendors will have a competitive reason for compliance as a market differentiator, whether they deploy with open-source or proprietary code. This need not add a lot to R&D costs either, as enterprising embedded system designers will now be motivated to delivering packaged ThingSafe! solutions, such as embedded wireless controllers, cellular modems, etc. Your open-source approach seems to me something that could be started today, and that has no downside. The RFC could give it the imprimatur of a standard, which makes the plan easier for vendors to sell to their thrift-minded management. -mel

On Oct 26, 2016, at 5:30 AM, Eric S. Raymond <esr@thyrsus.com> wrote:

Rich Kulawiec <rsk@gsp.org>:

...

I think our working assumption should be that there will be zero cooperation from the IoT vendors. (Yeah, once in a while one might actually step up, but that will merely be a happy anomaly.)

I agree.

There is, however, a chokepoint we have more hope of getting decent software deployed to. I refer to home and small-business routers. OpenWRT and kin are already minor but significant players here. And there's an NRE-minimization aregument we can make for router manufacturers to use rebranded versions rather than rolling their own crappy firmware.

I think the anti-IoT-flood strategy that makes the most sense is:

1. Push open-source firmware that doesn't suck to the vendors with a cost- and risk-minimization pitch.

2. Ship it with egress filters. (And telnet blocked.)

It wouldn't be technically very difficult to make the firmware rate-limit outbound connections. Cute trick: if we unlimit any local IP address that is a port-forwarding target, most users will never notice because their browser sessions won't be effected. -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Why does everyone think the Master Plan for World Domination has to be Evil? :) -mel beckman

On Oct 26, 2016, at 12:40 PM, Eric S. Raymond <esr@thyrsus.com> wrote:

Mel Beckman <mel@beckman.org>:

...

I also really like the idea of offering open source options to vendors, many of whom seem to illegally take that privilege anyway. A key fast-path component, though, is in my opinion a new RFC for IoT security best practices, and probably some revisions to UPNP.

The IoT RFC would spell out basic rules for safe devices: no back doors, no default passwords, no gratuitous inbound connections, etc. It would also make encryption a requirement, and limit how existing UPNP is deployed to prevent unnecessarily exposing vulnerable TCP/UDP ports to the wild. With this RFC in hand, and an appropriate splashy icon for vendor packaging (“RFC 9999 ThingSafe!”), vendors will have a competitive reason for compliance as a market differentiator, whether they deploy with open-source or proprietary code.

That is a good idea and I am officially adopting it as part of the Evil Master Plan for World Domination. :-)

I may recruit you to help draft the RFC. -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

On Wed, 26 Oct 2016 15:02:46 -0700, "Ronald F. Guilmette" said:

i.e. a multitude of wall plates in every room, each one bristling with a multitude of RJ11 sockets into which all manner of shiny new IoT things will be directly plugged, thence to be issued their own IPv6 addresses directly via DHCP from the local provider.

Actually, it seems to be going to wireless/bluetooth, and DHCP from the household router. Note that although a minor difference, it's one that can be leveraged. If we can change the dynamic from "plug it in and it Just Works" to "plug it in, and click the pop-up from your router confirming that you just added a device, and it Just Works after that", the battle is 3/4 won. The other 1/4 is the device initially telling the router what sort of device it is. - and we already know how to do that for USB and BlueTooth...

Given that, and given that "OpenWRT and kin" often provide the end-user with readily accessible dials and knobs via which the user can force the device to *exceed* legal/FCC limits on power output, I am not persuaded that open source WiFi router firmware actually represents a shining example of a methodology to prevent inexpensive devices from behaving badly.

Given that out of the box, the default config is in bounds, and it requires actual user interaction to exceed the limits, and that we don't see a very large problem out in the wild, I think we have prior art for the concept that "shipped with default and clued user can reconfigure" is a workable design.

On 2016-10-26 18:02, Ronald F. Guilmette wrote:

http://p.globalsources.com/IMAGES/PDT/BIG/053/B1088622053.jpg

i.e. a multitude of wall plates in every room, each one bristling with a multitude of RJ11 sockets into which all manner of shiny new IoT things will be directly plugged, thence to be issued their own IPv6 addresses

You still need to have a SOHO router, which could simply block any incoming calls unless a port has been opened for a specific IP address. (or UPnP for computers).

P.S. As noted in my prior post, the proplem of regulating IoT devices to insure that they do not exceed their reasonably expected operational limits, vis-a-vis outbound bandwidth usage

A camera showing the baby in 4K resolution along witgh sounds of him crying on dolby surround to the mother who is at work would likely saturate upload just as much as the virus sending DNS requests. This falls into the tonne of feathers weighting as much as a tonne of lead category.

In a message written on Wed, Oct 26, 2016 at 08:06:34AM -0400, Rich Kulawiec wrote:

The makers of IoT devices are falling all over themselves to rush products to market as quickly as possible in order to maximize their profits. They have no time for security. They don't concern themselves with privacy implications. They don't run networks so they don't care about the impact their devices may have on them. They don't care about liability: many of them are effectively immune because suing them would mean trans-national litigation, which is tedious and expensive. (And even if they lost: they'd dissolve and reconstitute as another company the next day.) They don't even care about each other -- I'm pretty sure we're rapidly approaching the point where toasters will be used to attack garage door openers and washing machines.

You are correct. I believe the answer is to have some sort of test scheme (UL Labratories?) for basic security and updateability. Then federal legislation is passed requiring any product being imported into the country to be certified, or it is refused. Now when they rush to market and don't get certified they get $0 and go out of business. Products are stopped at the boader, every shipment is reviewed by authorities, and there is no cross boarder suing issue. Really it's product safety 101. UL, the CPSC, NHTSA, DOT and a host of others have regulations that if you want to import a product for sale it must be safe. It's not a new or novel concept, pretty much every country has some scheme like it. -- Leo Bicknell - bicknell@ufp.org PGP keys at http://www.ufp.org/~bicknell/

Exactly, I was arguing exactly the same with some folks this week during the RIPE meeting. The same way that certifications are needed to avoid radio interferences, etc., and if you don’t pass those certifications, you can’t sell the products in some countries (or regions in case of EU for example), authorities should make sure that those certifications have a broader scope, including security and probably some other features to ensure that in case something is discovered in the future, they can be updated. Yes, that means cost, but a few thousand dollars of certification price increase, among thousands of millions of devices of the same model being manufactured, means a few cents for each unit. Even if we speak about 1 dollar per each product being sold, it is much cheaper than the cost of not doing it and paying for damages, human resources, etc., when there is a security breach. Regards, Jordi -----Mensaje original----- De: NANOG <nanog-bounces@nanog.org> en nombre de Leo Bicknell <bicknell@ufp.org> Organización: United Federation of Planets Responder a: <bicknell@ufp.org> Fecha: miércoles, 26 de octubre de 2016, 19:19 Para: <nanog@nanog.org> Asunto: Re: Spitballing IoT Security In a message written on Wed, Oct 26, 2016 at 08:06:34AM -0400, Rich Kulawiec wrote: > The makers of IoT devices are falling all over themselves to rush products > to market as quickly as possible in order to maximize their profits. They > have no time for security. They don't concern themselves with privacy > implications. They don't run networks so they don't care about the impact > their devices may have on them. They don't care about liability: many of > them are effectively immune because suing them would mean trans-national > litigation, which is tedious and expensive. (And even if they lost: > they'd dissolve and reconstitute as another company the next day.) > They don't even care about each other -- I'm pretty sure we're rapidly > approaching the point where toasters will be used to attack garage door > openers and washing machines. You are correct. I believe the answer is to have some sort of test scheme (UL Labratories?) for basic security and updateability. Then federal legislation is passed requiring any product being imported into the country to be certified, or it is refused. Now when they rush to market and don't get certified they get $0 and go out of business. Products are stopped at the boader, every shipment is reviewed by authorities, and there is no cross boarder suing issue. Really it's product safety 101. UL, the CPSC, NHTSA, DOT and a host of others have regulations that if you want to import a product for sale it must be safe. It's not a new or novel concept, pretty much every country has some scheme like it. -- Leo Bicknell - bicknell@ufp.org PGP keys at http://www.ufp.org/~bicknell/ ********************************************** IPv4 is over Are you ready for the new Internet ? http://www.consulintel.es The IPv6 Company This electronic message contains information which may be privileged or confidential. The information is intended to be for the use of the individual(s) named above. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information, including attached files, is prohibited.

re: having gadgets certified (aka UL/CSA for electric stuff). Devil is in the details. Who would certify it ? And who would set the standards for certification? How fast would those standards change? updated with each new attack? Would standards update require agreement of multiple parties who rarely agree? Consider vendor X who starts to develop product based on standards available in Oct 2016, but by the time he gets to market, standards have changed and his device no longer conforms? One of the beauties of the Internet is the freedom to innovate while keeping to the core basic IP packet delivery. Start to regulate it or add red tape and you start to hinder innovation. Perhaps the RFC mechanism to define best practices for standalone "IoT" devices might be a better mechanism. Those who build IP stacks to be used wholesale by gadget manufacturers could adhere to that RFC so that end products en up using a proper IP stack that doesn't easily allow the device to be "upgraded" to serve Dr Evil's botnet designed to take over the world.

In message <CAF-Wqd5sO0x5muw6uPDxMXd+h1ebCCtL9Ke9uMEc7k364OfHLA@mail.gmail.com>, Ken Matlock writes:

As a relative 'outsider' I see a lot of finger-pointing and phrasing this as (effectively) someone else's fault.

To me this is a failing on a number of levels all contributing to the problem.

1) The manufacturer - Backdoors, hidden accounts, remote access capabilities, no proper security testing. No enforcing of security updates. 2) The end-user - No initiative on the end-user's perspective to gain even a basic understanding of how the device works, connects, etc. Also no tools or understanding of how to recognize *which* of their many devices on the network might be compromised and participating in the botnet. (Only indication they get is maybe their internet is slow) 3) The service providers - No effective monitoring of outgoing traffic from the end users to identify botnets and DDoS in a real-time fashion

I contend that all 3 levels have failed in this, and nothing has fundamentally changed (today it's IoT, before it was unpatched windows boxes, etc) in decades. We keep talking about the problem but very little actual action has occurred to *fix* the underlying issues.

Actually things have changed a lot in a positive direction. * Router manufactures are using device specific passwords. * Microsoft, Apple, Linux and *BSD issue regular fixes for their products and users do intall them. * My smart TV has automatic updates available and turned on. * Other products do the same. Now not everyone does this sort of thing yet, but we have examples and things don't blow up in the user's face very often. Even in the case the manufacture has tried to do the right thing. The problem had been identified and a fix issued. Now could this have been automated, yes. But it does show that what is required is getting through to manufactures and they are trying to reduce the problem. We need manufactures to have a working system to accept problem reports. We need manufactures to issue fixes to their products once they have been reported. This needs to happen for the entire expected lifetime of a product. We need the ability to have a third parties fix problems if a manufacture won't / is too slow. Getting this may require legislation changes. This may require manufactures to publish expected product lifetimes.

- Manufacturers need to be held accountable for devices that go on the internet (that includes *anything* that's connected. PCs, servers, routers, IoT devices, etc) - End users need to have ways to easily see what's going on over their local networks, to see botnet-like activity and DDoS participation (among other things) in a more real-time fashion - Service providers need to be much more proactive in watching for threats and identifying/blocking them at the source, not allowing the traffic to flow to your peers and making it someone else's problem. Right now there's a financial disincentive to doing this, in both real costs (standing up monitoring gear/etc), and imagined (my ISP is SPYING on me!).

Until we fix all 3 of these main issues we're just going to keep going in the same set of circles we do every time a 'new' threat/vector comes in.

Now, are these issues *easy*? Oh, heck no! Are they *cheap*? Once again, heck no! But to 'fix' this issue it will take all 3 levels being fixed.

If we continue to keep pointing fingers at "the other guy" as the root of the problem we're inviting external forces (Legislation) to step in and 'fix' the problem for us (and it will just make it worse).

My 2 cents (adjust for inflation) Ken

On Wed, Oct 26, 2016 at 1:40 PM, jim deleskie <deleskie@gmail.com> wrote:

...

So device is certified, bug is found 2 years later. How does this help. The info to date is last week's issue was patched by the vendor in Sept 2015, I believe is what I read. We know bugs will creep in, (source anyon= e that has worked with code forever) Also certification assuming it would work, in what country, would I need one, per country I sell into? These are not the solutions you are looking for ( Jedi word play on purpose)

On Wed, Oct 26, 2016 at 3:53 PM, JORDI PALET MARTINEZ < jordi.palet@consulintel.es> wrote:

...

Exactly, I was arguing exactly the same with some folks this week durin= g the RIPE meeting.

The same way that certifications are needed to avoid radio interference= s, etc., and if you don=E2=80=99t pass those certifications, you can=E2=80= =99t sell the products in some countries (or regions in case of EU for example), authorities should make sure that those certifications have a broader scope, including security and probably some other features to ensure th= at in case something is discovered in the future, they can be updated.

Yes, that means cost, but a few thousand dollars of certification price increase, among thousands of millions of devices of the same model bein= g manufactured, means a few cents for each unit.

Even if we speak about 1 dollar per each product being sold, it is much cheaper than the cost of not doing it and paying for damages, human resources, etc., when there is a security breach.

Regards, Jordi

-----Mensaje original----- De: NANOG <nanog-bounces@nanog.org> en nombre de Leo Bicknell < bicknell@ufp.org> Organizaci=C3=B3n: United Federation of Planets Responder a: <bicknell@ufp.org> Fecha: mi=C3=A9rcoles, 26 de octubre de 2016, 19:19 Para: <nanog@nanog.org> Asunto: Re: Spitballing IoT Security

In a message written on Wed, Oct 26, 2016 at 08:06:34AM -0400, Rich Kulawiec wrote: > The makers of IoT devices are falling all over themselves to rush products > to market as quickly as possible in order to maximize their profits. They > have no time for security. They don't concern themselves with privacy > implications. They don't run networks so they don't care about t= he impact > their devices may have on them. They don't care about liability: many of > them are effectively immune because suing them would mean trans-national > litigation, which is tedious and expensive. (And even if they lost: > they'd dissolve and reconstitute as another company the next day.= ) > They don't even care about each other -- I'm pretty sure we're rapidly > approaching the point where toasters will be used to attack garag= e door > openers and washing machines.

You are correct.

I believe the answer is to have some sort of test scheme (UL Labratories?) for basic security and updateability. Then federal legislation is passed requiring any product being imported into the country to be certified, or it is refused.

Now when they rush to market and don't get certified they get $0 and go out of business. Products are stopped at the boader, every shipment is reviewed by authorities, and there is no cross boarder suing issue.

Really it's product safety 101. UL, the CPSC, NHTSA, DOT and a host of others have regulations that if you want to import a produc= t for sale it must be safe. It's not a new or novel concept, pretty much every country has some scheme like it.

-- Leo Bicknell - bicknell@ufp.org PGP keys at http://www.ufp.org/~bicknell/

********************************************** IPv4 is over Are you ready for the new Internet ? http://www.consulintel.es The IPv6 Company

This electronic message contains information which may be privileged or confidential. The information is intended to be for the use of the individual(s) named above. If you are not the intended recipient be awa= re that any disclosure, copying, distribution or use of the contents of th= is information, including attached files, is prohibited.

-- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org

On Wed Oct 26, 2016 at 05:10:44PM -0400, Jean-Francois Mezei wrote:

My smart TV not only hasn't gotten updates in years, but Sharp has stopped selling TVs in Canada. (not sure if they still sell TVs elsewhere).

When manufacturers provide a 2 year support on a device that will last 10 years, it is a problem which is why they really need to get it right when product is released and not rely on patches.

No chance of being right first time or ever but that's not a problem until it gets compromised

With regards to liability. Good luck suing a chinese outfit that no longer exists.

And pray tell, who gets to pay the millions of dollars of lawyer fees it will cost to sue that bankrupt company with no money ?

Nobody will. This is why a kill switch is needed. If you're going to IoT Safe mark things there needs to be a way to revoke it like with SSL certs So say devices, which phone home anyway, are required as part of getting the mark to check in with $version.$device.$manufacturer.iotsafe.com it's not much more than they do to check for new firmware already You don't want all those calling something central so delegate to manufacturers and if they go bust drop the deleagtion and serve it centrally. An ISP with problem devices can always fake it locally to drop them and spot the polling traffic when looking for them When the device checks in they can with a simple api check their version and if they're allowed to be on the general internet on not. If banned they go offline and maybe tell the user somehow if they can. The deal to get IoT safe rated is that everyone agree to this, the user will be told clearly that their thing will be removed from the net if the manufacturer doesn't keep it safe so it's clear they sue them if there is a problem (or accept it's so cheap they can throw it away if they go bust) Now there's tons of holes in that like an attacher turning that bit off, there may be better schemes I've not noticed for doing this already. All details, the idea is a back stop is needed for when all the other stuff fails. brandon

In message <12573.1477530109@segfault.tristatelogic.com>, "Ronald F. Guilmette" writes:

In message <58111BD4.80403@vaxination.ca>, Jean-Francois Mezei <jfmezei_nanog@vaxination.ca> wrote:

...

My smart TV not only hasn't gotten updates in years, but Sharp has stopped selling TVs in Canada. (not sure if they still sell TVs elsewhere).

A little more than 2 years ago, I bought a last-of-its-kind demo model of a 50 inch Panasonic Plasma TV which was on sale (due to having been discontinued by the manufacturer) from the local BestBuy.

Not long after, once I got the thing home, I realized that the thing's understanding of current local time... important in conjunction with the on-screen TV guide... was locked to Eastern Standard Time, and there was no way to change it. (This was/is a bit of a problem for me, as I'm in PST/PDT.)

I called up Panasonic and explained the whole thing to a first- level tech support minion. She had no solution to offer me. I insisted on speaking to a manager.

A manager got on the line and I prroceeded to re-explain the whole issue to him. I said that I needed a firmware fix. He said that there was no way the company was going to develop a fix "just for you". Politely, I persisted and said that the TV firmware was self-evidently faulty.

Then you return it to the store as "Not Fit For Purpose" and demand your money back. Alternatively you pull them into court and have them honour the warrantee. Or you move to a juristiction with good consumer protection laws and sic the government onto them. Just because a model is discontinued it does not mean that warrantee issues do not need to be addressed. Mark -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org

In message <20161026205800.7188D57B29B8@rock.dv.isc.org>, Mark Andrews <marka@isc.org> wrote:

Actually things have changed a lot in a positive direction. ... * Microsoft, Apple, Linux and *BSD issue regular fixes for their products and users do intall them.

At the risk of repeating a point I have already made in this thread, please do let me know how I can obtain this month's security patches for my iPhone 3GS. (Note that Wikipedia says that this device was only formally discontinued by the manufacturer as of September 12, 2012, i.e. only slightly more than 4 short years ago. Nontheless, the current "security solution" for this product, as made available from the manufacturer... a manufacturer which is here being held up as a shining example of ernest social responsi- bility... is for me to contribute the entire device to my local landfill, where it will no doubt leach innumerable heavy metals into the soil for my children's children's children to enjoy.)

...

- Manufacturers need to be held accountable for devices that go on the internet...

My iPhone 3GS "goes on the Internet". Through no fauly of my own, it is also, apparently, destined in short order to "go onto" a landfill, if not here, then in China or India, where a pitiful plethora of shoeless and sad-eyed third-world waifs will spend their childhoods picking through the mand-made mountains of e-refuse in a daily and desperate search for of anything of value. http://motherboard.vice.com/read/much-of-americas-e-waste-recycling-is-a-sha... In short, if the "good" companies, like Apple, are the solution to the problem, then I obviously misunderstood what "the problem" is, and would be obliged if someone (anyone) would re-phrase it for me in simpler terms, for the benefit of my limited little noggin. In lieu of that, for the moment I'd just like to emphasize again that it is my opinion that any "solution" to the now self-evident IoT problems which relies, even in the slightest, upon manufacturers providing a con- tinuous and timely stream of security updates is a fantasy. Wishful thinking, pure and simple. When even the "good" companies have built their fortunes and entire business models around convincing/forcing everyone to purchase "new and improved" units every two years, at a maximum, and when the same said companies stop issuing patches of any kind for products that have only departed the corporate price list three years earlier, then one shudders to even contemplate what the contribution of the "bad" companies will be to this ongoing catastrophy. Regards, rfg

In message <20161027084939.5BDF457D0FFB@rock.dv.isc.org>, Mark Andrews <marka@isc.org> wrote:

Well the last update for the 3GS was iOS 6.1.6 in Feb 2014.

Bingo! Less than a year and a half after they stopped selling it, they effectively stopped supporting it.

Hi,

At which point the 3GS was almost 5 years old (having originally been released in June 2009) and had been already superseded by the iPhone 4, 4S, 5 and 5S/5C.

But the release of and presence of those phones does not make the older phone suddenly stop working. As noted, the phone might be obsolete to those people hungering for the latest tech but as a phone and web client etc it still works fine. ....and will continue doing so whilst the battery is okay. ... and then, with no updates it can be the next attack vector Which is the point. These things stay out there...like those winXP boxes. There are 2 choices 1) manufacturers are responsible for the devices. No longer caring for them? Recall them. Compensate the users. 2) stronger obsolescence. eg kill switch/firmware tombstoning/network connectivity function ending timebomb as a user of lots of legacy tech i find either option bad :/ alan

(deleted for ambiguity)

...

Which is the point. These things stay out there...like those winXP boxes. There are 2 choices

1) manufacturers are responsible for the devices. No longer caring for them? Recall them. Compensate the users.

2) stronger obsolescence. eg kill switch/firmware tombstoning/network connectivity function ending timebomb

as a user of lots of legacy tech i find either option bad :/

alan

Or Apple could release iOS 6.1.7. There is nothing stopping Apple doing so. Apple are the ones preventing people running iOS 10.x on the 3GS. This puts the responsibilty on them to supply security fixes.

All of the PC's running XP could run a newer version of the Windows regardless of whether they could run the latest version.

Well, yes and no. As $newer_better_faster_stronger gains market share, there's no drive to be backwards compatible. iOS is no different from any other operating system in that regard, it's designed for hardware A, B, C, D's 1 through 4 (probably more, but I'm trying to be somewhat abstract). If it has to support E through Z also, for 12+ years of backwards compatibility, bad things can happen (bloat, instability, bugs). I don't get upset for example, when I transplant a Win2k or Win98 drive into a box built up with 3 year old hardware, of which not a single device is supported. That's not even taking into account the challenge of developing for different architectures. ARM, x86, PPC, AMD64, PowerISA, SPARC, to name a few. I won't even get into microcontrollers. Don't get me wrong. I'd love to update my 12 year old Macbook Pro to Sierra, but I've accepted that it, like most electronics, were almost certainly not engineered, let alone expected, to last even half that long. I'm reminded of that fact every time I open Youtube, and Flash Player spins both of its 2.33ghz Core2 Duo cores to 100% for a 460p video. Even then, I've had to stop updating Flash sometime around mid 2014, as any newer versions cease to function entirely.

I've been following the discussion with quite a bit of interest. What had become crystal clear to me is that nobody here has been looking at the problem from the perspective of the manufacturer, particularly how they actually get product to marked. A la "Dilbert". The engineer's credo: "Why build it when you can buy it?" I doubt very seriously that manufacturers are starting completely from scratch when they design their IoT product. They buy this piece, they buy that piece, they buy this hunk of software, they buy these hardware components. Slap them together, and you have your product. That being the case, the question of "what happens when the company goes bankrupt" becomes less of an issue so long at the company who supplied the IP stack is still around. By government implementing some not-unpleasant rules, the companies can outsource the IP stack portion, including updates. Then the manufacturers can concentrate on the value add stuff. For durable goods like refrigerators and thermostats, you could require that the IP-capable part be in a plug-replaceable module, so that all the customer needs to do is go to Home Depot or wherever and get a replacement module. Instant update! The back end of the module would be a well-defined API so the manufacturer can do his product, divorced from the Net stuff. Indeed, it wouldn't take long for the various industry associations to codify what the modules should look like, both physically and electrically. The semiconductor industry did this big time in the TTL days. There is precedent. So what if your washing machine is working perfectly well 15 years into its lifecycle. You replace the network module and get the latest and greatest security updates. Light bulbs are harder, but even then there is an opportunity for someone to market an "industry standard" interface that can be upgraded easily and cheaply. By the original software vendor. Can someone say "IoTsoft"?

In message <20161027112940.GB17170@ussenterprise.ufp.org>, Leo Bicknell <bicknell@ufp.org> wrote:

Actually, they encourage you to trade {your old iPhone} in... ... If your device is too old for that program, they will still take it for free and recycle it in an enviornmentally friendly way...

OK, so good on them. I do compliment them for their apparent willingness to take back this pile of leachable heavy metals and do something responsible with it. But to come back to the point, what if I really don't -want- to give Apple another several hundred dollars this year? The baby needs shoes, the gas tank is empty, and maybe I just don't -have- $600+ dollars this month to further enrich their shareholders. My iPhone 3GS still works just fine, for the most part, so if I don't really need all of the new whiz bang features of the newer ones, why would I fork over big bucks to replace it? Just because TV commercials entice me to do so?? The problem is, as I have said, this device is now the Apple equivalent of Windows XP. There could be a horrendous collection of a dozen or more known critical security bugs in the thing by now, but as someone noted, the last update Apple issued for the thing was in Feb 2014. In the medical field, they use the term "orphan drugs" to refer to drugs that have such a low return on investment that no manufacturer has any interest in them anymore. We don't use that terminology in the tech field because it would be redundant. *Every* tech product either already is, or soon will be, an orphan. You can't *force* people to throw away or trade-in their old tech products, especially when, from the user's point of view, there doesn't -seem- to be anything wrong with them... like all of those pre- Sept. 2015 Internet video cameras. (Well, -in theory- you could force people to do this. You could legislate an Obamacare-esque tax which penalized everyone who -didn't- throw away or trade-in their old tech gadgets after, say, 4 years, but I don't think that would go down very well.) Regards, rfg

On Thursday, October 27, 2016, Mark Andrews <marka@isc.org> wrote:

In message <16193.1477594538@segfault.tristatelogic.com <javascript:;>>, "Ronald F. Guilmette" writes:

...

In message <20161027112940.GB17170@ussenterprise.ufp.org <javascript:;> , Leo Bicknell <bicknell@ufp.org <javascript:;>> wrote:

...

Actually, they encourage you to trade {your old iPhone} in... ... If your device is too old for that program, they will still take it for free and recycle it in an enviornmentally friendly way...

OK, so good on them. I do compliment them for their apparent willingness to take back this pile of leachable heavy metals and do something responsible with it.

But to come back to the point, what if I really don't -want- to give Apple another several hundred dollars this year? The baby needs shoes, the gas tank is empty, and maybe I just don't -have- $600+ dollars this month to further enrich their shareholders.

My iPhone 3GS still works just fine, for the most part, so if I don't really need all of the new whiz bang features of the newer ones, why would I fork over big bucks to replace it? Just because TV commercials entice me to do so??

The problem is, as I have said, this device is now the Apple equivalent of Windows XP. There could be a horrendous collection of a dozen or more known critical security bugs in the thing by now, but as someone noted, the last update Apple issued for the thing was in Feb 2014.

But is there? Can you list a single security bug in iOS 6.1.6 that would require a iOS 6.1.7?

Well, ios 7 - 9.3.4 is in scope for this RCE https://blog.lookout.com/blog/2016/08/25/trident-pegasus/ And if you view jpegs, you may want to update to 10.1 https://threatpost.com/apple-patches-ios-flaw-exploitable-by-malicious-jpeg/... Yes, it is annoying that iOS 10.x doesn't run on it so that you can't

newer apps.

...

In the medical field, they use the term "orphan drugs" to refer to drugs that have such a low return on investment that no manufacturer has any interest in them anymore. We don't use that terminology in the tech field because it would be redundant. *Every* tech product either already is, or soon will be, an orphan.

You can't *force* people to throw away or trade-in their old tech products, especially when, from the user's point of view, there doesn't -seem- to be anything wrong with them... like all of those pre- Sept. 2015 Internet video cameras. (Well, -in theory- you could force people to do this. You could legislate an Obamacare-esque tax which penalized everyone who -didn't- throw away or trade-in their old tech gadgets after, say, 4 years, but I don't think that would go down very well.)

Regards, rfg -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org <javascript:;>

On Thu, 27 Oct 2016, Ronald F. Guilmette wrote:

My iPhone 3GS still works just fine,

I still have a "functional" iPhone 3G (no S). I don't think AT&T will activate service on it at this point, and it's been relegated to iPod service when I do yard work.

You can't *force* people to throw away or trade-in their old tech products, especially when, from the user's point of view, there doesn't -seem- to be anything wrong with them... like all of those pre- Sept. 2015 Internet video cameras.

Sure you can. Just make the tech dependent on "the cloud" and when the device is too old, force retirement by no longer supporting it. That doesn't force it off the network (unless the final command from the cloud is "shut off [your network interface]?"), but it makes the user much more likely to toss it and replace it with something newer if they still want such a device. This is one of my bigger concerns every time I buy something that's "cloud controlled". Not so much that the manufacturer will force it's retirement, but "what happens if they go belly up, or just kill the division that supports my device?" A cloud controlled networked device, with no cloud, is not terribly useful. ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route | therefore you are _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

On Thu, Oct 27, 2016 at 05:13:31PM -0400, Jon Lewis wrote:

This is one of my bigger concerns every time I buy something that's "cloud controlled". Not so much that the manufacturer will force it's retirement, but "what happens if they go belly up, or just kill the division that supports my device?" A cloud controlled networked device, with no cloud, is not terribly useful.

1. This has happened already, e.g., Nest. It will happen again. Many times. 2. Such a device may not be terribly useful *to you*, but in its neglected, unremarked, unmaintained state it may become very useful to attackers. ---rsk

Please don't, bring it to your nearest Apple Store instead where it will be properly recycled, <https://www.apple.com/recycling/>.

My nearest Apple stores are 50 miles away. I'm not sure 100 miles in the car is a good tradeoff for one phone.

In message <CAF-Wqd5sO0x5muw6uPDxMXd+h1ebCCtL9Ke9uMEc7k364OfHLA@mail.gmail.com> Ken Matlock <matlockken@gmail.com> wrote:

- End users need to have ways to easily see what's going on over their local networks, to see botnet-like activity and DDoS participation (among other things) in a more real-time fashion

This is an interesting point. I'm not actually an ISP guy, although I do play one on TV. Anyway, I hope nobody will begrudge me if I make a couple of brief points, and then ask a rather naive question. Point: I have a DSL line which is limited to 6Mbps down and 756Kbps up. My guess is that if any typical/average user is seen to be using more than, say, 1/10 of that amount of "up" bandwidth in any one given 10 minute time period, then something is really really REALLY wrong. Point: I am already signed up with various services which will send me automated emails whenever certain events occur, e.g. when the price of 2TB WD Black drives falls below my personal threshold value. Point: My ISP knows my email address. Question: Could ISPs set something up so that each customer broadband line is continuously and individually monitored, and so that an automated email would be automagically dashed off to the customer if that customer's "up" bandwidth in some time period exceeded a value which, for that ISP, is deemed "reasonable"? (I envision the hypothetical email messages in question would consist of a non-threatening warning to the customer which would include a link to a web page where there would be a list of common things end-lusers should check for in such circumstances, along with detailed and clear instructions for how to check for each, and also a "don't ever bother me with these warnings again" checkbox, and perhaps even a friendly slider where the end-luser could adjust his personal warning threshold value, for the future.) Of course, any ISP that desperately -never- wants to receive -any- end- luser support calls quite certainly won't like this scheme. But I'm not sure that that fact alone would utterly disqualify the idea from being useful in some contexts. The real question is: Is anything even remotely along these lines even possible with existing commonly used ISP infrastructure? (If not, then just forget I mentioned it.) Regards, rfg P.S. One possible big advantage to the kind of system described above is that if an ISP received a complaint about a given customer, alleging that the customer is running a bot, then the ISP could go and look at the warning settings for that customer. If that's already been set to "don't ever bother me', then the ISP can disconnect the customer, and when the customer inevitably saquaks about that, the ISP can respond and say "We told you so."

In message <12301.1477525252@segfault.tristatelogic.com>, "Ronald F. Guilmette" writes:

In message <CAF-Wqd5sO0x5muw6uPDxMXd+h1ebCCtL9Ke9uMEc7k364OfHLA@mail.gmail.co m> Ken Matlock <matlockken@gmail.com> wrote:

...

- End users need to have ways to easily see what's going on over their local networks, to see botnet-like activity and DDoS participation (among other things) in a more real-time fashion

This is an interesting point.

I'm not actually an ISP guy, although I do play one on TV. Anyway, I hope nobody will begrudge me if I make a couple of brief points, and then ask a rather naive question.

Point: I have a DSL line which is limited to 6Mbps down and 756Kbps up. My guess is that if any typical/average user is seen to be using more than, say, 1/10 of that amount of "up" bandwidth in any one given 10 minute time period, then something is really really REALLY wrong.

No. Just uploading a video to youtube would cause a false positive using that test. You need to know what "bad" traffic looks like to find it. Packets flowing != "bad traffic". Unusual != "bad traffic". Mark

Point: I am already signed up with various services which will send me automated emails whenever certain events occur, e.g. when the price of 2TB WD Black drives falls below my personal threshold value.

Point: My ISP knows my email address.

Question: Could ISPs set something up so that each customer broadband line is continuously and individually monitored, and so that an automated email would be automagically dashed off to the customer if that customer's "up" bandwidth in some time period exceeded a value which, for that ISP, is deemed "reasonable"? (I envision the hypothetical email messages in question would consist of a non-threatening warning to the customer which would include a link to a web page where there would be a list of common things end-lusers should check for in such circumstances, along with detailed and clear instructions for how to check for each, and also a "don't ever bother me with these warnings again" checkbox, and perhaps even a friendly slider where the end-luser could adjust his personal warning threshold value, for the future.)

Of course, any ISP that desperately -never- wants to receive -any- end- luser support calls quite certainly won't like this scheme. But I'm not sure that that fact alone would utterly disqualify the idea from being useful in some contexts.

The real question is: Is anything even remotely along these lines even possible with existing commonly used ISP infrastructure? (If not, then just forget I mentioned it.)

Regards, rfg

P.S. One possible big advantage to the kind of system described above is that if an ISP received a complaint about a given customer, alleging that the customer is running a bot, then the ISP could go and look at the warning settings for that customer. If that's already been set to "don't ever bother me', then the ISP can disconnect the customer, and when the customer inevitably saquaks about that, the ISP can respond and say "We told you so." -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org

On Thursday, 27 October, 2016 00:40, "Ronald F. Guilmette" <rfg@tristatelogic.com> said:

Point: I have a DSL line which is limited to 6Mbps down and 756Kbps up. My guess is that if any typical/average user is seen to be using more than, say, 1/10 of that amount of "up" bandwidth in any one given 10 minute time period, then something is really really REALLY wrong.

This sounds like a horrible view of the Internet as "TV, only with more funny cat pictures", where most users are in a second-tier that is only expected / allowed to consume. One of the reasons I'm very grateful for FTTC / VDSL is that I can finally get a useful upstream speed. Going from 10-14M downstream to 80M was very nice, but going from 1M to 20M upstream was an absolute game-changer. I back up to the cloud - and there are plenty of services that allow regular, non-technical users to do this. The initial run saturated my upstream for days, and the incrementals are sometimes 20 or 30 minute bursts. I wouldn't even have tried on ADSL. Every time I get back from a day out, or even more so from a holiday, I upload the photos from my PC to one or more cloud services. I'll max my uplink for anywhere between 10 minutes and an hour - on the old ADSL, it was easily an overnight task. Working from home, I can now work directly with files on network shares, rather than copying everything to the laptop before I leave the office and trying to sync changes when I get back. I know the exact figures for this case, but there are a *lot* of spikes over the course of a day. With ADSL, I could go and make tea every time I needed to save a large Word doc or Powerpoint back to the network. On top of that, I can spend anything up to 3 or 4 hours in videoconferences, which will have a steady stream of a few hundred Kb/s. Spotting atypical (or ideally malicious) traffic is a valid goal, but I think we need to be a whole lot smarter than "customer is using upstream". Regards, Tim.

I agree wholeheartedly. ---- Yes, BCP (any relevant to your business), filtering, active tit-for-tat with abuse teams, calling out manufacturers, ISPs doing /anything/ (most already block 25 and 80, not that they give you the upload to bother with the latter and it's not necessarily for the good of the 'net as a whole) - they're all things we absolutely should be doing. That doesn't change the fact that all of those are just*ambulances in the valley*. <http://www.tonycooke.org/stories-and-illustrations/ambulance_valley/> If we're going to solve this, we need to be better as a species - we're about to the place where we can't progress much farther (without some sort of caste system nightmare - /The Diamond Age/ comes to mind) *until basic computing and good practice are as pervasive as the ability to read and write.* Hint: I don't mean 'can do an app on the smartphone' - real understanding and appreciation. I'm not saying everyone needs to be a savant, but a basic understanding of the technology you*^1 use *every.single.day* for almost *every.single.function* of your life isn't asking much, I don't think. The ability to think logically and problem solve is something that I see declining in even the brighter of the youths I've encountered in the past few years. This "it just works/should work" willfully (almost maliciously sometimes) ignorant mentality - pushed by vendors and craved by the overworked - is stunting our potential. Christ, the people who are *in charge of the world* (not necessarily those who /run/ it...but I'd be a good portion still) don't even understand the basics of how these machines, and this thing that facilitates the global economy, work. The root problem is much, much more significant than DoS attacks and spam. Maybe we need to start younger - I can't speak for all schools, but my 'computer course' was "here's Mavis Beacon - play games and...whatever" - I hope it's not [really | still] like that. Maybe we, the community, create and sponsor course material, maybe there's a push for more than Cisco Academy - at this point this knowledge a public safety issue and should be a respected part of the general education syllabus (too bad we're all too busy with standardized tests to care). Something so inherently part of everyday life cannot be just for the 'nerds' or even the especially interested, anymore. I don't know what to do about manufacturers - it's been a global race to bottom for years now. Someone mentioned a device certification earlier. It's something and a start at least, so I'm on board and willing to devote some time. I'm not sure this is something the community alone will be able to drive, silently, from the shadows. The cynic in me wants to throw in to buy a politician or two. The usual trick is to hit them where it hurts - in the wallet. Their wallets are so large these days (and constantly consolidating, lessening the chance of real change and competition) that I'm at a loss as to how. Maybe a slow increase in user-required configurations, decisions, and interaction...complete with logical explanations to help with said decisions? I don't know...that could affect profits this quarter (because who looks farther ahead than that...long term effects and progress aren't important anymore, right?). Pavlovian training? Seems a bit totalitarian. The /last/ thing I want is government (on the country or global scale) intervention..the 2nd to last is to use this upcoming metaphor (but I haven't a better one). Look at cars - in more places then not, it's damn near impossible to be a functional and contributory adult without a car; some might even call it a 'right' in the 1st world. Does that stop us from driver's ed courses**^2 in school? Do we not teach the basics of safe operation, maintenance, and even a bit about how it works under the hood (my school did)? Does the ubiquity of the automobile stop the removal of that (legal) ability for those who *endanger others* or otherwise abuse the driving privilege? No...no it doesn't. Granted, there are still those who are going to do what they're going to do - but that number is lessened (and some even come around to see the harm). That does *not* mean I think there should be a 'compu-tar license' - not at all. But it *does* mean that everyone should be taught responsible computing, the harms of carelessness, and the fun in knowing how these things work. Anyway - thanks for the rant (been bothering me for a while now)...I do believe we should address and minimize the symptoms as they appear, but without surgical attacks directed at the dark heart of the beast (be that people, intrinsically, or just our social norms) we're going to end up with either a horribly censored, totalitarian internet "app" regime, or burnt to the ground in chaos - too distracted by inane, emotionally infused, bullshit pumped forth day-to-day at an ADD inducing pace (meant to give us the ol' numb & dumb - I'll admit I succumb more often than I'd like - not trying to high-horse here), to notice the fires until it's too late to stomp them out. I never imagined we***^3 could become so dichotomous-ly obsessed yet ignorant. Yes, there will always be malicious people but, in the same way we convinced most of the world that sacrificing humans is murder and kinda wrong (and engaged them in at least a few active prevention tactics), we need to stigmatize -- really */really/* stigmatize (to the core of the soul) this bullshit. On a side note: giving 10 years to the guy who just wanted to tinker with "his own" (because we don't /own/ anything anymore, in a hyperbolic way) equipment isn't the way to do it. Everything seems overly fatalistic and over-dramatized until the moment it's not - how many disasters could have been prevented if people just listened to the engineers (ask the Challenger)? Of course, we're still prescribing antibiotics for virii in the face of MRSA and worse - hell Pompeii partied until they were literally dying in the streets...so let's drink up, add a little duct tape, and worry about it in a few years (/s). Or, we keep pushing, wherever possible, and maybe something will pop. Seldom do I wish to be proven incorrect - here I do. Contrary to what it may seem, I think we***^3 still have a chance. *1: That's /you, the generalization I'm referring to/, and not /you, the specific people reading this./ **2: Though, unfortunately due to that government intervention, we spent more time memorizing the specific BAC to age ratio to determine your fine and loss of license than honing basic knowledge and skills. ***3: Again, that's /we, as a society of 7 billion - call it a median or mode/ rather than /we, individuals in a set/. ---- (Disclaimer: I don't like speaking publicly, especially at this length (though I've cut out a good 60%, as I admit I have a rambling problem). I've spent the last week writing and re-writing versions of this; I still don't like it (both overly idealistic and fatalistic at the same time, and the "voice" is much harsher than I would have liked - the tradeoff of curtailing the rambling I suppose), but I had a strong reaction to this subject. ...And yes I even debated the disclaimer, as it's hokey as all getout...best I could muster was to move it to the end. My apologies if I've overlooked points below being covered previously in the thread - /thank you for the ear & I'm sorry/™). ~knack On 10/26/2016 3:12 PM, Ken Matlock wrote:

As a relative 'outsider' I see a lot of finger-pointing and phrasing this as (effectively) someone else's fault.

To me this is a failing on a number of levels all contributing to the problem.

1) The manufacturer - Backdoors, hidden accounts, remote access capabilities, no proper security testing. No enforcing of security updates. 2) The end-user - No initiative on the end-user's perspective to gain even a basic understanding of how the device works, connects, etc. Also no tools or understanding of how to recognize *which* of their many devices on the network might be compromised and participating in the botnet. (Only indication they get is maybe their internet is slow) 3) The service providers - No effective monitoring of outgoing traffic from the end users to identify botnets and DDoS in a real-time fashion

I contend that all 3 levels have failed in this, and nothing has fundamentally changed (today it's IoT, before it was unpatched windows boxes, etc) in decades. We keep talking about the problem but very little actual action has occurred to *fix* the underlying issues.

- Manufacturers need to be held accountable for devices that go on the internet (that includes *anything* that's connected. PCs, servers, routers, IoT devices, etc) - End users need to have ways to easily see what's going on over their local networks, to see botnet-like activity and DDoS participation (among other things) in a more real-time fashion - Service providers need to be much more proactive in watching for threats and identifying/blocking them at the source, not allowing the traffic to flow to your peers and making it someone else's problem. Right now there's a financial disincentive to doing this, in both real costs (standing up monitoring gear/etc), and imagined (my ISP is SPYING on me!).

Until we fix all 3 of these main issues we're just going to keep going in the same set of circles we do every time a 'new' threat/vector comes in.

Now, are these issues *easy*? Oh, heck no! Are they *cheap*? Once again, heck no! But to 'fix' this issue it will take all 3 levels being fixed.

If we continue to keep pointing fingers at "the other guy" as the root of the problem we're inviting external forces (Legislation) to step in and 'fix' the problem for us (and it will just make it worse).

My 2 cents (adjust for inflation) Ken

On Wed, Oct 26, 2016 at 1:40 PM, jim deleskie <deleskie@gmail.com> wrote:

...

So device is certified, bug is found 2 years later. How does this help. The info to date is last week's issue was patched by the vendor in Sept 2015, I believe is what I read. We know bugs will creep in, (source anyone that has worked with code forever) Also certification assuming it would work, in what country, would I need one, per country I sell into? These are not the solutions you are looking for ( Jedi word play on purpose)

On Wed, Oct 26, 2016 at 3:53 PM, JORDI PALET MARTINEZ < jordi.palet@consulintel.es> wrote:

...

Exactly, I was arguing exactly the same with some folks this week during the RIPE meeting.

The same way that certifications are needed to avoid radio interferences, etc., and if you don’t pass those certifications, you can’t sell the products in some countries (or regions in case of EU for example), authorities should make sure that those certifications have a broader scope, including security and probably some other features to ensure that in case something is discovered in the future, they can be updated.

Yes, that means cost, but a few thousand dollars of certification price increase, among thousands of millions of devices of the same model being manufactured, means a few cents for each unit.

Even if we speak about 1 dollar per each product being sold, it is much cheaper than the cost of not doing it and paying for damages, human resources, etc., when there is a security breach.

Regards, Jordi

-----Mensaje original----- De: NANOG <nanog-bounces@nanog.org> en nombre de Leo Bicknell < bicknell@ufp.org> Organización: United Federation of Planets Responder a: <bicknell@ufp.org> Fecha: miércoles, 26 de octubre de 2016, 19:19 Para: <nanog@nanog.org> Asunto: Re: Spitballing IoT Security

In a message written on Wed, Oct 26, 2016 at 08:06:34AM -0400, Rich Kulawiec wrote: > The makers of IoT devices are falling all over themselves to rush products > to market as quickly as possible in order to maximize their profits. They > have no time for security. They don't concern themselves with privacy > implications. They don't run networks so they don't care about the impact > their devices may have on them. They don't care about liability: many of > them are effectively immune because suing them would mean trans-national > litigation, which is tedious and expensive. (And even if they lost: > they'd dissolve and reconstitute as another company the next day.) > They don't even care about each other -- I'm pretty sure we're rapidly > approaching the point where toasters will be used to attack garage door > openers and washing machines.

You are correct.

I believe the answer is to have some sort of test scheme (UL Labratories?) for basic security and updateability. Then federal legislation is passed requiring any product being imported into the country to be certified, or it is refused.

Now when they rush to market and don't get certified they get $0 and go out of business. Products are stopped at the boader, every shipment is reviewed by authorities, and there is no cross boarder suing issue.

Really it's product safety 101. UL, the CPSC, NHTSA, DOT and a host of others have regulations that if you want to import a product for sale it must be safe. It's not a new or novel concept, pretty much every country has some scheme like it.

-- Leo Bicknell - bicknell@ufp.org PGP keys at http://www.ufp.org/~bicknell/

********************************************** IPv4 is over Are you ready for the new Internet ? http://www.consulintel.es The IPv6 Company

This electronic message contains information which may be privileged or confidential. The information is intended to be for the use of the individual(s) named above. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information, including attached files, is prohibited.

In message <20161027112601.GA17170@ussenterprise.ufp.org>, Leo Bicknell <bicknell@ufp.org> wrote:

Problems I think consumer safety legislation can solve:

* SSH and Telnet were enabled, but there was no notification in the UI that they were enabled and no way to turn them off. Requirements could be set to show all services in the UI and if they are on or off.

* There was a hard coded user + pass that the consumer COULD NOT CHANGE, and did not display. Requirements could be set to never hard code an account.

* That the system has a user-friendly way to update. "Click here to check for update." "Click here to install update."

I say again, #3 is useless, unless and until you also have legislation that: *) Forces tech companies to never go bankrupt. *) Forces tech companies to -timely- issue security patches for all "critical" security issues (and good luck legally defining THAT). *) Forces tech companies to continue to issue security patches for as long as any "significant" number of the relevant devices remain actively in use, even if that turns out to be 20 years or more. You can force a company to implement a "user-friendly way to update", but what's the point of doing that if the company never issues any updates? I say again, the only way to solve these problems is if the devices are fundamentally secure by design, on the day they first ship to customers. Post-sale patching is an ad hoc and haphazard catch-as- catch-can solution at best, and it's not something that most manufacturers have -any- financial incentive to even do. They already got their money, on the day when the consumer bought the device. The rest is just an afterthought. Regards, rfg

In message <CAF-Wqd5kuxZwFZ5gwCP9-k7chX6y06JMoMoZdvP_i2oRvbmFUg@mail.gmail.com> Ken Matlock <matlockken@gmail.com> wrote:

Fixing the current wave of 'IoT' devices and phones and Tv's etc is only putting a bandaid on a broken arm. It gives the illusion of progress...

Until we accept that it's *everyone's* problem and work to fix the things under our control and work as an advocate for the other layers, we will continue to suffer attacks.

Agreed. Even if we could snap our fingers and fix the whole morass that is the IoT problem tomorrow, that still wouldn't prevent dumb consumers from pulling their dusty old Windows XP laptops own out of their closets and hooking them up directly to the Internet. Nor would it do anything about the small ISPs that have "mailbox full" abuse@ addresses, or the even larger ISPs that allow deliberately spoofed packets out onto the public Internet, or the Tier 1s that still peer with known utterly irresponsible ASNs. But, ya know, you gotta start someplace. And we can't let the perfect be the enemy of the good. That just won't wash anymore, I think. Not after last Friday. I put forward what I think is a reasonbly modest scheme to try to get IoT things to place hard limits on their "unsolicited" packet output at the kernel level, and I'm going to go off now and try to find and then engage some Linux embedded kernel people and see what they think. Maybe the whole thing is a dumb idea and not worth persuing, but I'm not con- vinced of that yet. So I'll go off, investigate in some more appropriate forum, and report back here if/when I have anything useful to say. Hacking embedded kernels to make them fault-tolerant, even in the event of attackers getting a root shell prompt, isn't going to save the world from DDoS attacks, but it may be one small part of the solution. Regards, rfg P.S. In the scheme I proposed, I left out one additional nicety that embedded kernels could also do to enhance security, namely disabling raw sockets completely in the kernel. No normal IoT thing needs the ability to forge outbound packets. But I would be willing to bet my bottom dollar, right now, that if we poked around long enough we could surely find some easily break-in-able busybox-based thingies out there, right now, as we speak, into which a binary could dropped that would have no trouble at all opening raw outbound sockets. BCP38 for toasters anyone?

Re: certification of IoT devices analogous to UL etc Another potentially useful channel to give this idea legs are insurance companies, get them involved if possible. They underwrite the risks particularly liability risks for manufacturers. That's why "Underwriters Laboratory" is called that, ultimately it's an arm of the insurance industry. If the insurance companies tell a manufacturer they won't cover risk for any non-certified device the device will almost certainly be improved. Similar repercussions for wholesale and retail outlets who could decide to just not offer non-certified devices, for insurance reasons and otherwise. As to those who would try maneuvers such as bankrupting or using shell companies which are dissolved when liabilities occur such willful acts often allow "piercing of the corporate veil". That is, those individuals involved can be sued or held criminally liable personally and any such indemnification made worthless as a protection or defense. In the US, at least, if there's a pattern of such behavior, such as serially setting up shell corps and bankrupting them when trouble arises, the fearsome RICO statutes can be invoked. Basically they provide the added felonies of racketeering and engaging in a conspiracy to commit criminal acts. Not being located in the US may not be sufficient for evasion of prosecution, merely doing business in the US (e.g., selling one's products, establishing a "nexus") can make one a valid target for US (and other) law enforcement. The fly I see in all this ointment is that getting there could be a lot of honest work so who would do that and champion this idea? -- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

i think this would be the most effective route proposed so far. May the force be with you :) On Wed, Oct 26, 2016 at 12:19 PM, Leo Bicknell <bicknell@ufp.org> wrote:

In a message written on Wed, Oct 26, 2016 at 08:06:34AM -0400, Rich Kulawiec wrote:

...

The makers of IoT devices are falling all over themselves to rush products to market as quickly as possible in order to maximize their profits. They have no time for security. They don't concern themselves with privacy implications. They don't run networks so they don't care about the impact their devices may have on them. They don't care about liability: many of them are effectively immune because suing them would mean trans-national litigation, which is tedious and expensive. (And even if they lost: they'd dissolve and reconstitute as another company the next day.) They don't even care about each other -- I'm pretty sure we're rapidly approaching the point where toasters will be used to attack garage door openers and washing machines.

You are correct.

I believe the answer is to have some sort of test scheme (UL Labratories?) for basic security and updateability. Then federal legislation is passed requiring any product being imported into the country to be certified, or it is refused.

Now when they rush to market and don't get certified they get $0 and go out of business. Products are stopped at the boader, every shipment is reviewed by authorities, and there is no cross boarder suing issue.

Really it's product safety 101. UL, the CPSC, NHTSA, DOT and a host of others have regulations that if you want to import a product for sale it must be safe. It's not a new or novel concept, pretty much every country has some scheme like it.

-- Leo Bicknell - bicknell@ufp.org PGP keys at http://www.ufp.org/~bicknell/

In message <20161026120634.GA20735@gsp.org>, Rich Kulawiec <rsk@gsp.org> wrote:

On Mon, Oct 24, 2016 at 01:24:59PM -0700, Ronald F. Guilmette wrote:

...

2) Second, once elected I will decree that in future all new IoT devices, and also all updates to firmware for existing IoT devices will have, BUILT IN TO THE KERNEL, code/logic which (a) prevents all outbound TCP session initiation and which also (b) strictly rate-limits all other protocols to some modest value.

I like this idea. But unfortunately, I think it has no chance of succeeding.

The makers of IoT devices are falling all over themselves to rush products to market as quickly as possible in order to maximize their profits. They have no time for security. They don't concern themselves with privacy...

Well, see, this is why I was clear at the outset that in order for this scheme to work, I'll first need to be elected King of the World.

From that high perch I will be able to decree, by fiat, that no Internet connectable device shall be sold or marketed *unless* it has been certified (i.e. by some reliable entity that knows how to test these things) to be incapable of being converted into a weapon, i.e. incapable of spewing unnecessarily large amounts of garbage at completely arbitrary targets, *even if* an attacker somehow manages to get a shell prompt.

OK, so setting aside all frivolity now, how could this kind of restriction actually be achieved? Here's the thing: Any solution to these problems is going to come in two parts, technical and political. We here, by and large, are not politicians, but we can influence them and urge them towards solutions that are, workable, economically practical, and above all, technically effective. Or alternatively, we can leave them to flouder around on their own, in the dark. (We've all seen *that* movie before, and it isn't pretty. Think Clipper Chip and the recent push for crypto backdoors.) Left to their own devices, politicians routinely screw up technology regulation virtually every time. So the first order of business is for the industry itself to come up with a rational approach... and virtually immediately, because the window of opportunity is rapidly closing... for solving these IoT problems, and then get widspread agreement... or at least a lack of violent disagreement... within the industry itself. The industry can then speak with one voice to the politicians and regulators, who will then be effectively bound to doing the Right Thing. Sensible regulations, once enacted in one jurisdiction, tend to be contagious. In my own state of California, state regulation of various things, most notably air pollution, and the production thereof by cars, has eventually affected the entire North American auto market and beyond, in part because it is less economically palatable for manufacturers to design and ship multiple configurations of any one product, i.e. one which conforms to the regulations in jurisdiction X, and another that doesn't. In short, if sensible regulations requiring "safe" designs for IoT products were to come into force in one locale, it is not only possible, but actually quite likely that they would affect the whole market. If a given Far East manufacturer was required to have safety built into the kernel of its toasters in order to be able to sell said toasters, say, in the United States... or even just in California... would they really go to the trouble to strip out the additional "safety" part of their firmware when manufacturing what is essentially the same product, but destined for other markets? I think not. (A question for the audience: How has FCC regulation of the maximum power output of WiFi routers affected the worldwide market for such devices, over time? I honestly don't know, but I suspect that there has been a good effect, over time, on the whole worldwide market.) It may be difficult, even among technologists, to find common ground and agreement about what "IoT" things should and should not be able to do, or even, for that matter, to agree on the definition of "IoT". But after last Friday, and even before, I think that most of us know what we *do not* want them to be able to do, i.e. to send an unlimited percentage of their available bandwidth towards any arbitrary IP address. General purpose computers, and also routers, need to be able to do that, but your bird feeder, your lightbulb, your HDTV, your refrigerator and your home alarm system don't. So maybe that's a starting point. I proposed something which is at base, really rather simple, even if, in practice, the implementation details could get a bit complex. Basically, the proposal is that the kernels of all IoT devices should impose sensible limits on outbound bandwidth usage, consistant with each specific device's expected operational needs. It seems to me that this is not particularly different from other belt-and-suspenders approaches used in other safety critical systems, ranging from medical radiation treatment devices to nuclear power plants. Actual engineering of the firmware-imposed safety constraints needed in IoT devices will not, in my opinion, be very hard. In the absence of a King of the World to impose such a requirement on all manufacturers of IoT devices, I believe that it would be equally effective, in the long run, to get (U.S.) state-level regulations on the books, perhaps starting in California, just because we here in my home state have some experience going first with a lot of these kinds of things. A plausible alternatively would be to get the FCC on the case. (Obviously, the FCC already has a ton of experience in promulgating regulations whose goal is to prevent individual devices from behaving in ways that muck up the communications of other devices, so from that perspective at least, it seems like a good fit. Not that the FCC could be easily persuaded to take on this tar-baby, but they might.) So anyway, bottom line, I think this is do-able, both technically and politically, and also absolutely necessary. After the Krebs, OVH, and Dyn attacks, is anybody in their right mind willing to stand up, at this late date, and say that we can go on, as we have been, ignoring these problems and just constantly racing to build bigger pipes... a strategy which, by now, should be universally accepted as a self-defeating non-solution? Lincoln said "As our case is new, so we must think anew and act anew." If you hook up a device to your local telephone or cable company which sends fifty thousand volts down the line, you may fry your local distribution substation, but you're not going to fry the entire Eastern Seaboard or take down the world's largest e-commerce site for two hours. Even the popular news media, typically devoid of technical sophistication, now knows that the single organism that is the Internet is becoming more vlunerable *to itself* day by day. The time is ripe for clear-headed action and I do hope that we will see some. Regards, rfg

On October 23, 2016 at 22:56 jw@nuclearfallout.net (John Weekes) wrote:

For the IoT botnets, most of the emails are ignored or rejected, because most go to providers who either quietly bitbucket them or flat-out reject all abuse emails. Most emails sent to mainland China, for instance, are in that category (Hong Kong ISPs are somewhat better).

As I've suggested before how much would you attribute this to a lack of English skills by recipients? Are they all sent in English? Just curious but one wonders what most here would do with an abuse complaint sent to them in Chinese? 文话语难? -- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

The problem is first you have to even recognize it's an abuse complaint worth spending another second over. For example one gets a lot of spam to abuse addresses, or I do anyhow. And much of it seems to be in some character set other than Latin-1. On October 24, 2016 at 21:58 eyeronic.design@gmail.com (Mike Hale) wrote:

Run it through Google translate?

On Oct 24, 2016 9:40 PM, <bzs@theworld.com> wrote:

On October 23, 2016 at 22:56 jw@nuclearfallout.net (John Weekes) wrote:  > For the IoT botnets, most of the emails are ignored or rejected, because  > most go to providers who either quietly bitbucket them or flat-out  > reject all abuse emails. Most emails sent to mainland China, for  > instance, are in that category (Hong Kong ISPs are somewhat better).

As I've suggested before how much would you attribute this to a lack of English skills by recipients? Are they all sent in English?

Just curious but one wonders what most here would do with an abuse complaint sent to them in Chinese?

文话语难?

--         -Barry Shein

Software Tool & Die    | bzs@TheWorld.com             | http:// www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD       | 800-THE-WRLD The World: Since 1989  | A Public Information Utility | *oo*

-- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*

On October 24, 2016 at 23:46 jw@nuclearfallout.net (John Weekes) wrote:

...

Are they all sent in English?

Currently, mine are.

...

Just curious but one wonders what most here would do with an abuse complaint sent to them in Chinese?

If I were to receive one in Chinese, I would personally paste it into Google Translate. That is what I do with Japanese complaints/responses, which are the main ones I see that aren't in English. Most others ISPs seem to use straight English, or both English and another language.

As I said in a previous note first one would have to even recognize that a note written in Chinese (or Urdu for that matter, non-Latin-1) is a valid abuse note worth spending another moment looking at. I wonder how many have spam filters which pretty much block emails in non-Latin-1 character sets? It's certainly an easy setting on spamassassin for example, the easiest being to just choose English or maybe there's a Latin-1 choice and default score any other language and charset very high (i.e., as likely spam.) Even ignoring that possibility how many in other countries even agree with the assumptions underlying this complaint about their not reacting to various abuses? Maybe they just think such complaints are silly and/or way out of their hands (i.e., whoever is reading that abuse complaint)? In the US and similar countries we tend to use as reference points we tend to have evolved short-circuit mechanisms to respond to serious abuse events. For all I know all they can do is submit a 20 page highly stylized request to the Ministry of Information to consider among the 50,000 other requests from libraries, book publishers, minor political parties, etc at the next people's congress in August before any action, even an email response as it would imply a policy, can be taken. Which just might be a little frustrating to the front line. Or would be frustrating to us. For many of them it's just how things work, believe me when I say I've run into that powerless fatalism personally. Taking individual initiative is not a universal virtue. More importantly is there any attempt at a global meeting of the minds on what a notable problem and appropriate reaction, including responding to abuse reports, even is? My guess having dealt with some amount of international internet politics is: Many here might be very, very surprised if they tried. -- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*