Dyn DDoS this AM?
Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)... https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-redd... -- @ChrisGrundemann http://chrisgrundemann.com
I cannot give additional info other than what’s been on “public media”. However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS’ed Can Not Stand. See Krebs’ on the Democratization of Censorship. See lots of other things. To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants. To the miscreants: You will not succeed. Search "churchill on the beaches”. It’s a bit melodramatic, but it’s how I feel at this moment. To the rest of the community: If you can help, please do. I know a lot of you are thinking “what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn’t help Mirai, but it still helps. There are many other things you can do as well. But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone’s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost. OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help. -- TTFN, patrick
On Oct 21, 2016, at 10:55 AM, Chris Grundemann <cgrundemann@gmail.com> wrote:
Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)...
https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-redd...
-- @ChrisGrundemann http://chrisgrundemann.com
+1! Well said, Patrick. B On Friday, October 21, 2016, Patrick W. Gilmore <patrick@ianai.net> wrote:
I cannot give additional info other than what’s been on “public media”.
However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS’ed Can Not Stand. See Krebs’ on the Democratization of Censorship. See lots of other things.
To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants.
To the miscreants: You will not succeed. Search "churchill on the beaches”. It’s a bit melodramatic, but it’s how I feel at this moment.
To the rest of the community: If you can help, please do. I know a lot of you are thinking “what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn’t help Mirai, but it still helps. There are many other things you can do as well.
But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone’s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost.
OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help.
-- TTFN, patrick
On Oct 21, 2016, at 10:55 AM, Chris Grundemann <cgrundemann@gmail.com <javascript:;>> wrote:
Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)...
https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts- twitter-spotify-reddit/
-- @ChrisGrundemann http://chrisgrundemann.com
Just a FYI, That "horrific trend" has been happening since some techie got dissed on an IRC channel over 20 years ago. He used a bunch of hosted putters to ICMP flood the IRC server. Whatever the community is behind, until the carriers decide to wise up this will keep happening, that is without talking about the industries being developed around DDoSes events. Enjoy your weekend. ( I ain't on call anymore anyway =D ) ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 10/21/16 11:52, Brian Davies via NANOG wrote:
+1!
Well said, Patrick.
B
On Friday, October 21, 2016, Patrick W. Gilmore <patrick@ianai.net> wrote:
I cannot give additional info other than what’s been on “public media”.
However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS’ed Can Not Stand. See Krebs’ on the Democratization of Censorship. See lots of other things.
To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants.
To the miscreants: You will not succeed. Search "churchill on the beaches”. It’s a bit melodramatic, but it’s how I feel at this moment.
To the rest of the community: If you can help, please do. I know a lot of you are thinking “what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn’t help Mirai, but it still helps. There are many other things you can do as well.
But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone’s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost.
OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help.
-- TTFN, patrick
On Oct 21, 2016, at 10:55 AM, Chris Grundemann <cgrundemann@gmail.com <javascript:;>> wrote: Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)...
https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts- twitter-spotify-reddit/ -- @ChrisGrundemann http://chrisgrundemann.com
See, that's the thing... The key to victory here is to defeat the robots. Take away the anonymity of proxies and trojan amplifiers and enforcement gets a lot easier. Sadly, this war doesn't seem likely to be won anytime soon. Especially since there are State entities using (and even deploying) a number of these systems for use against other States and businesses and/or financial mechanisms. So rather than help the community solve the problem (for their own good, no less!), it is in their interests to perpetuate it. -Wayne On Fri, Oct 21, 2016 at 05:37:08PM -0400, Alain Hebert wrote:
Just a FYI,
That "horrific trend" has been happening since some techie got dissed on an IRC channel over 20 years ago.
He used a bunch of hosted putters to ICMP flood the IRC server.
Whatever the community is behind, until the carriers decide to wise up this will keep happening, that is without talking about the industries being developed around DDoSes events.
Enjoy your weekend. ( I ain't on call anymore anyway =D )
----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443
On 10/21/16 11:52, Brian Davies via NANOG wrote:
+1!
Well said, Patrick.
B
On Friday, October 21, 2016, Patrick W. Gilmore <patrick@ianai.net> wrote:
I cannot give additional info other than what???s been on ???public media???.
However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS???ed Can Not Stand. See Krebs??? on the Democratization of Censorship. See lots of other things.
To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants.
To the miscreants: You will not succeed. Search "churchill on the beaches???. It???s a bit melodramatic, but it???s how I feel at this moment.
To the rest of the community: If you can help, please do. I know a lot of you are thinking ???what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn???t help Mirai, but it still helps. There are many other things you can do as well.
But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone???s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost.
OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help.
-- TTFN, patrick
On Oct 21, 2016, at 10:55 AM, Chris Grundemann <cgrundemann@gmail.com <javascript:;>> wrote: Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)...
https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts- twitter-spotify-reddit/ -- @ChrisGrundemann http://chrisgrundemann.com
--- Wayne Bouchard web@typo.org Network Dude http://www.typo.org/~web/
Anyone want a quick consulting gig helping us configure BCP38 and BCP84? Configurations is all cisco Edge routers connect to Verizon, Level 3 Fiber Each Edge router talks to two BGP routers. $150/hour, I'm guessing it is only an hour for somebody to explain, and guide us through the configuration, but OK if longer. Thanks. Bob Roswell broswell@syssrc.com 410-771-5544 ext 4336 Computer Museum Highlights -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Patrick W. Gilmore Sent: Friday, October 21, 2016 11:48 AM To: NANOG list <nanog@nanog.org> Subject: Re: Dyn DDoS this AM? I cannot give additional info other than what’s been on “public media”. However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS’ed Can Not Stand. See Krebs’ on the Democratization of Censorship. See lots of other things. To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants. To the miscreants: You will not succeed. Search "churchill on the beaches”. It’s a bit melodramatic, but it’s how I feel at this moment. To the rest of the community: If you can help, please do. I know a lot of you are thinking “what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn’t help Mirai, but it still helps. There are many other things you can do as well. But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone’s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost. OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help. -- TTFN, patrick
On Oct 21, 2016, at 10:55 AM, Chris Grundemann <cgrundemann@gmail.com> wrote:
Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)...
https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotif y-reddit/
-- @ChrisGrundemann http://chrisgrundemann.com
On Fri, 21 Oct 2016, rar wrote:
Anyone want a quick consulting gig helping us configure BCP38 and BCP84?
Configurations is all cisco Edge routers connect to Verizon, Level 3 Fiber Each Edge router talks to two BGP routers.
$150/hour, I'm guessing it is only an hour for somebody to explain, and guide us through the configuration, but OK if longer.
Sure, we'll do it. That rate is quite a bit less than our normal retail rate, but in the spirit that Patrick posted about, Network Utility Force will be happy to provide you or any other operator resources at that rate to help configure BCP38 and BCP84. Anyone serious about that, email me privately at bross@netuf.net and we'll put paperwork together. -- Brandon Ross Yahoo & AIM: BrandonNRoss Voice: +1-404-635-6667 ICQ: 2269442 Signal Secure SMS: +1-404-644-9628 Skype: brandonross Schedule a meeting: http://www.doodle.com/bross
Are there sites that can test your BCP38\84 compliance? I'm okay, but interested in what I can share to raise awareness. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Patrick W. Gilmore" <patrick@ianai.net> To: "NANOG list" <nanog@nanog.org> Sent: Friday, October 21, 2016 10:48:21 AM Subject: Re: Dyn DDoS this AM? I cannot give additional info other than what’s been on “public media”. However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS’ed Can Not Stand. See Krebs’ on the Democratization of Censorship. See lots of other things. To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants. To the miscreants: You will not succeed. Search "churchill on the beaches”. It’s a bit melodramatic, but it’s how I feel at this moment. To the rest of the community: If you can help, please do. I know a lot of you are thinking “what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn’t help Mirai, but it still helps. There are many other things you can do as well. But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone’s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost. OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help. -- TTFN, patrick
On Oct 21, 2016, at 10:55 AM, Chris Grundemann <cgrundemann@gmail.com> wrote:
Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)...
https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-redd...
-- @ChrisGrundemann http://chrisgrundemann.com
On 21 Oct 2016, at 23:01, Mike Hammett wrote:
Are there sites that can test your BCP38\84 compliance?
<https://www.caida.org/projects/spoofer/> ----------------------------------- Roland Dobbins <rdobbins@arbor.net>
On Fri, Oct 21, 2016 at 12:09 PM, Roland Dobbins <rdobbins@arbor.net> wrote:
On 21 Oct 2016, at 23:01, Mike Hammett wrote:
Are there sites that can test your BCP38\84 compliance?
Quick note: If anyone has this installed already on OSX, bring up the console and see if it's still running. I discovered (while watching the NANOG preso) that mine had an issue and was failing silently. Re-installing the new version fixed the issue. The funny part of the story, looking through the logs to see which networks I roamed on that were spoofable, the only positive hit was for the NANOG conference network in Chicago :) -Steve
https://www.caida.org/projects/spoofer/ <https://www.caida.org/projects/spoofer/> -- TTFN, patrick
On Oct 21, 2016, at 12:01 PM, Mike Hammett <nanog@ics-il.net> wrote:
Are there sites that can test your BCP38\84 compliance? I'm okay, but interested in what I can share to raise awareness.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
----- Original Message -----
From: "Patrick W. Gilmore" <patrick@ianai.net> To: "NANOG list" <nanog@nanog.org> Sent: Friday, October 21, 2016 10:48:21 AM Subject: Re: Dyn DDoS this AM?
I cannot give additional info other than what’s been on “public media”.
However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS’ed Can Not Stand. See Krebs’ on the Democratization of Censorship. See lots of other things.
To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants.
To the miscreants: You will not succeed. Search "churchill on the beaches”. It’s a bit melodramatic, but it’s how I feel at this moment.
To the rest of the community: If you can help, please do. I know a lot of you are thinking “what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn’t help Mirai, but it still helps. There are many other things you can do as well.
But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone’s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost.
OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help.
-- TTFN, patrick
On Oct 21, 2016, at 10:55 AM, Chris Grundemann <cgrundemann@gmail.com> wrote:
Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)...
https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-redd...
-- @ChrisGrundemann http://chrisgrundemann.com
Attack has re-started. This is the time, folks. Rally the troops, offer help, watch your flow. STOP THIS NOW. -- TTFN, patrick
On Oct 21, 2016, at 11:48 AM, Patrick W. Gilmore <patrick@ianai.net> wrote:
I cannot give additional info other than what’s been on “public media”.
However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS’ed Can Not Stand. See Krebs’ on the Democratization of Censorship. See lots of other things.
To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants.
To the miscreants: You will not succeed. Search "churchill on the beaches”. It’s a bit melodramatic, but it’s how I feel at this moment.
To the rest of the community: If you can help, please do. I know a lot of you are thinking “what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn’t help Mirai, but it still helps. There are many other things you can do as well.
But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone’s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost.
OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help.
-- TTFN, patrick
On Oct 21, 2016, at 10:55 AM, Chris Grundemann <cgrundemann@gmail.com <mailto:cgrundemann@gmail.com>> wrote:
Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)...
https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-redd... <https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-reddit/>
-- @ChrisGrundemann http://chrisgrundemann.com
Rofl, Yeah good luck with that... 15+ years later and most of the actors that could fix that, for the planete, still refuses to do anything. Now you can start the usual circular discussion that goes nowhere after 3 days... PS: yeah usual BCP38 rant... but its friday. ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 10/21/16 12:12, Patrick W. Gilmore wrote:
Attack has re-started. This is the time, folks. Rally the troops, offer help, watch your flow.
STOP THIS NOW.
On Fri, Oct 21, 2016 at 12:30:44PM -0400, Alain Hebert wrote:
Rofl,
Yeah good luck with that... 15+ years later and most of the actors that could fix that, for the planete, still refuses to do anything.
Now you can start the usual circular discussion that goes nowhere after 3 days...
PS: yeah usual BCP38 rant... but its friday.
Not all attacks are BCP38 related. :-) - Jared -- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.
Do we know the attack destinations so we can watch transit traffic destined for it to help sources that may be unaware? David
Patrick W. Gilmore wrote:
Our biggest problem is people thinking they cannot or do not want to help.
Our biggest problem is that if the Internet community does not handle problems like this, governments and regulators may decide to intervene. If they do this in the wrong way, it will turn one major headache into two. Nick
As a Twitter network engineer (and the guy Patrick let camp out in your hotel room all day) - thank you for this. Whoever was behind this just poked a hornet’s nest. “Govern yourselves accordingly”. -C (Obviously speaking for myself, not my employer…)
On Oct 21, 2016, at 10:48 AM, Patrick W. Gilmore <patrick@ianai.net> wrote:
I cannot give additional info other than what’s been on “public media”.
However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS’ed Can Not Stand. See Krebs’ on the Democratization of Censorship. See lots of other things.
To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants.
To the miscreants: You will not succeed. Search "churchill on the beaches”. It’s a bit melodramatic, but it’s how I feel at this moment.
To the rest of the community: If you can help, please do. I know a lot of you are thinking “what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn’t help Mirai, but it still helps. There are many other things you can do as well.
But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone’s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost.
OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help.
-- TTFN, patrick
On Oct 21, 2016, at 10:55 AM, Chris Grundemann <cgrundemann@gmail.com> wrote:
Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)...
https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-redd...
-- @ChrisGrundemann http://chrisgrundemann.com
Oh god, you invoked @popehat ... [dyndds and its customers sue XiongMai, the OEM integrators, and Does 1-10,000,000 who own the devices for neglegence?...] Sent from my iPhone
On Oct 21, 2016, at 8:29 PM, Chris Woodfield <rekoil@semihuman.com> wrote:
As a Twitter network engineer (and the guy Patrick let camp out in your hotel room all day) - thank you for this. Whoever was behind this just poked a hornet’s nest.
“Govern yourselves accordingly”.
-C
(Obviously speaking for myself, not my employer…)
On Oct 21, 2016, at 10:48 AM, Patrick W. Gilmore <patrick@ianai.net> wrote:
I cannot give additional info other than what’s been on “public media”.
However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS’ed Can Not Stand. See Krebs’ on the Democratization of Censorship. See lots of other things.
To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants.
To the miscreants: You will not succeed. Search "churchill on the beaches”. It’s a bit melodramatic, but it’s how I feel at this moment.
To the rest of the community: If you can help, please do. I know a lot of you are thinking “what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn’t help Mirai, but it still helps. There are many other things you can do as well.
But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone’s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost.
OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help.
-- TTFN, patrick
On Oct 21, 2016, at 10:55 AM, Chris Grundemann <cgrundemann@gmail.com> wrote:
Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)...
https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-redd...
-- @ChrisGrundemann http://chrisgrundemann.com
Patrick, We are client of 3 tier1. On our netflow collector, we can observe that RFC1918 sources ip traffic is entering our AS via 2 of those tier-1. Yes, 2 bigs tier-1 allow private ip traffic coming from their networks, clients, peerings to reach others customers, via Internet link, on public ip.....Of course this traffic is dropped on our BGP borders as we are filtering. But it's still filling the pipe, and this is still INVALID/UNNAUTHORIZED traffic. We wrote to them to verify if customers are technically allowed to send RFC1918 traffic over their backbone, and if we are also allowed to do so. And the answer was really evasive like :"contractually you're are not allowed". So now tell me WTF BCP38 will provide you when tier1 does not care at all and does not maintain basic filtering to/from their customers. And then they try to sell you their anti ddos services, because you know DDOS it sucks. Big joke. What about BCP38+84 on 30 tier-1 instead of asking/hoping 55k others autonomous-system having good filters in place ? -- Marcel On 21.10.2016 17:48, Patrick W. Gilmore wrote:
To the rest of the community: If you can help, please do. I know a lot of you are thinking “what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn’t help Mirai, but it still helps. There are many other things you can do as well.
On 22 October 2016 at 16:40, marcel.duregards--- via NANOG <nanog@nanog.org> wrote:
What about BCP38+84 on 30 tier-1 instead of asking/hoping 55k others autonomous-system having good filters in place ?
The originating ISPs are in a far better position to check that traffic isn't from spoofed address ranges than transit networks are. The best thing to do is to ask EVERY network to do what they can, not just the few biggest ones. Any size ISP can be hit by and hurt by DDoS attacks, so every size ISP should be doing what they can to make sure they are not either the source or the victim of those attacks. Dan
Its not a first time we have and large scale DDoS incident. Its not a first time we have (a kind of) knee-jerk reaction. I think its a right time to direct community attention to this document https://www.routingmanifesto.org/manrs/ It's work in progress. But its a good start. On Fri, Oct 21, 2016 at 5:48 PM, Patrick W. Gilmore <patrick@ianai.net> wrote:
I cannot give additional info other than what’s been on “public media”.
However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS’ed Can Not Stand. See Krebs’ on the Democratization of Censorship. See lots of other things.
To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants.
To the miscreants: You will not succeed. Search "churchill on the beaches”. It’s a bit melodramatic, but it’s how I feel at this moment.
To the rest of the community: If you can help, please do. I know a lot of you are thinking “what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn’t help Mirai, but it still helps. There are many other things you can do as well.
But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone’s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost.
OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help.
-- TTFN, patrick
On Oct 21, 2016, at 10:55 AM, Chris Grundemann <cgrundemann@gmail.com> wrote:
Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)...
https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts- twitter-spotify-reddit/
-- @ChrisGrundemann http://chrisgrundemann.com
-- Alexander Lyamin CEO | Qrator <http://qrator.net/>* Labs* office: 8-800-3333-LAB (522) mob: +7-916-9086122 skype: melanor9 mailto: la@qrator.net
And its not the last time the big Tier(s) will refuse to do anything beside dropping the fault to the CPE vendors. People love circles. ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 10/24/16 14:12, Alexander Lyamin wrote:
Its not a first time we have and large scale DDoS incident. Its not a first time we have (a kind of) knee-jerk reaction.
I think its a right time to direct community attention to this document
https://www.routingmanifesto.org/manrs/
It's work in progress. But its a good start.
On Fri, Oct 21, 2016 at 5:48 PM, Patrick W. Gilmore <patrick@ianai.net> wrote:
I cannot give additional info other than what’s been on “public media”.
However, I would very much like to say that this is a horrific trend on the Internet. The idea that someone can mention a DDoS then get DDoS’ed Can Not Stand. See Krebs’ on the Democratization of Censorship. See lots of other things.
To Dyn and everyone else being attacked: The community is behind you. There are problems, but if we stick together, we can beat these miscreants.
To the miscreants: You will not succeed. Search "churchill on the beaches”. It’s a bit melodramatic, but it’s how I feel at this moment.
To the rest of the community: If you can help, please do. I know a lot of you are thinking “what can I do?" There is a lot you can do. BCP38 & BCP84 instantly come to mind. Sure, that doesn’t help Mirai, but it still helps. There are many other things you can do as well.
But a lot of it is just willingness to help. When someone asks you to help trace an attack, do not let the request sit for a while. Damage is being done. Help your neighbor. When someone’s house is burning, your current project, your lunch break, whatever else you are doing is almost certainly less important. If we stick together and help each other, we can - we WILL - win this war. If we are apathetic, we have already lost.
OK, enough motivational speaking for today. But take this to heart. Our biggest problem is people thinking they cannot or do not want to help.
-- TTFN, patrick
On Oct 21, 2016, at 10:55 AM, Chris Grundemann <cgrundemann@gmail.com> wrote: Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)...
https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts- twitter-spotify-reddit/ -- @ChrisGrundemann http://chrisgrundemann.com
On Mon, Oct 24, 2016 at 02:38:58PM -0400, Alain Hebert wrote:
And its not the last time the big Tier(s) will refuse to do anything beside dropping the fault to the CPE vendors.
I can say that we had to drop uRPF for technical reasons, namely not enough people ask their vendors about it so it is: a) not tested (at all) b) not performance rated c) lacks simple fixes The people who have the purchasing power are not the tier-1 carriers regardless. We push as hard as we can and end up with the compromises as a result. - Jared
In message <CALoKGd15haJXthiT31Y_wk=-5UGLSRbusHv4b8btQ5nXv5Dmuw@mail.gmail.com>, Alexander Lyamin <la@qrator.net> wrote:
Its not a first time we have and large scale DDoS incident. Its not a first time we have (a kind of) knee-jerk reaction.
I could be wrong, but I think its the first time I've turned on CNN and seen a "heat map" of the incident showing the entire NorthEast / New England area, all the way down to Washington, and parts of California all blanketed in red. So that part, at least, was, ya know, novel. Regards, rfg
Yeah, it sucked to be a Dyn customer that day. However, if you had a backup dns provider, it wasnt that bad. You do realize that collateral effect scale is a property of a target and not attack? My point was that implementing MANRS, while isn't covering all of the spectrum of the attacks that made news this autumn will make at least some of them if not impossible, but harder to execute. And as I said - its work in progress. P.S. Jared Mauch notes regarding uRPF underperformance are correct, but it only shows how rarely its actually used in a real life. uRPF is more then feasible in terms of algorithmical complexity, and this means that bugs can be dealed with. On Tue, Oct 25, 2016 at 7:30 AM, Ronald F. Guilmette <rfg@tristatelogic.com> wrote:
In message <CALoKGd15haJXthiT31Y_wk=-5UGLSRbusHv4b8btQ5nXv5Dmuw@mail. gmail.com>, Alexander Lyamin <la@qrator.net> wrote:
Its not a first time we have and large scale DDoS incident. Its not a first time we have (a kind of) knee-jerk reaction.
I could be wrong, but I think its the first time I've turned on CNN and seen a "heat map" of the incident showing the entire NorthEast / New England area, all the way down to Washington, and parts of California all blanketed in red.
So that part, at least, was, ya know, novel.
Regards, rfg
-- Alexander Lyamin CEO | Qrator <http://qrator.net/>* Labs* office: 8-800-3333-LAB (522) mob: +7-916-9086122 skype: melanor9 mailto: la@qrator.net
Side note: I asked Mikrotik and they accepted the feature request of changing their uRPF setting from being universal on the machine to being per-interface (as the kernel supports). That would make it easier for Mikrotik end-user-facing routers to block crap right at the edge, allowing for strict facing customer and loose elsewhere. They haven't implemented it yet, but they accepted the request. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "Alexander Lyamin" <la@qrator.net> To: "Ronald F. Guilmette" <rfg@tristatelogic.com> Cc: "NANOG list" <nanog@nanog.org> Sent: Tuesday, October 25, 2016 3:29:56 AM Subject: Re: Dyn DDoS this AM? Yeah, it sucked to be a Dyn customer that day. However, if you had a backup dns provider, it wasnt that bad. You do realize that collateral effect scale is a property of a target and not attack? My point was that implementing MANRS, while isn't covering all of the spectrum of the attacks that made news this autumn will make at least some of them if not impossible, but harder to execute. And as I said - its work in progress. P.S. Jared Mauch notes regarding uRPF underperformance are correct, but it only shows how rarely its actually used in a real life. uRPF is more then feasible in terms of algorithmical complexity, and this means that bugs can be dealed with. On Tue, Oct 25, 2016 at 7:30 AM, Ronald F. Guilmette <rfg@tristatelogic.com> wrote:
In message <CALoKGd15haJXthiT31Y_wk=-5UGLSRbusHv4b8btQ5nXv5Dmuw@mail. gmail.com>, Alexander Lyamin <la@qrator.net> wrote:
Its not a first time we have and large scale DDoS incident. Its not a first time we have (a kind of) knee-jerk reaction.
I could be wrong, but I think its the first time I've turned on CNN and seen a "heat map" of the incident showing the entire NorthEast / New England area, all the way down to Washington, and parts of California all blanketed in red.
So that part, at least, was, ya know, novel.
Regards, rfg
-- Alexander Lyamin CEO | Qrator <http://qrator.net/>* Labs* office: 8-800-3333-LAB (522) mob: +7-916-9086122 skype: melanor9 mailto: la@qrator.net
LA Times: Why sites like Twitter and Spotify were down for East Coast users this morning http://www.latimes.com/business/la-fi-tn-dyn-attack-20161021-snap-story.html -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Chris Grundemann Sent: Friday, October 21, 2016 7:56 AM To: nanog@nanog.org Subject: Dyn DDoS this AM? Does anyone have any additional details? Seems to be over now, but I'm very curious about the specifics of such a highly impactful attack (and it's timing following NANOG 68)... https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-redd... -- @ChrisGrundemann http://chrisgrundemann.com
On 10/21/16 09:05, Matthew Black wrote:
LA Times: Why sites like Twitter and Spotify were down for East Coast users this morning http://www.latimes.com/business/la-fi-tn-dyn-attack-20161021-snap-story.html
I actually can't resolve twitter.com this morning and I'm west coast. None of the four listed DNS servers are responding. twitter.com. 172800 IN NS ns1.p34.dynect.net. twitter.com. 172800 IN NS ns2.p34.dynect.net. twitter.com. 172800 IN NS ns3.p34.dynect.net. twitter.com. 172800 IN NS ns4.p34.dynect.net. Trace routes seem to point towards San Jose or Palo Alto or Los Angeles. ~Seth
anyone who relies on a single dns provider is just asking for stuff such as this. randy
The brutal reality in todays world is that anyone that relies on the Internet is just asking for stuff like this. No service is safe. Andrew Andrew Fried andrew.fried@gmail.com On 10/21/16 5:58 PM, Randy Bush wrote:
anyone who relies on a single dns provider is just asking for stuff such as this.
randy
On Fri, Oct 21, 2016 at 2:58 PM, Randy Bush <randy@psg.com> wrote:
anyone who relies on a single dns provider is just asking for stuff such as this.
randy
I'd love to hear how others are handling the overhead of managing two dns providers. Every time we brainstorm on it, we see it as blackhole of eng effort WRT to keeping them in sync and and then waiting for TTLs to cut an entire delegation over.
anyone who relies on a single dns provider is just asking for stuff such as this. I'd love to hear how others are handling the overhead of managing two dns providers.
good question. staying in-band, hidden primary comes to mind. but i am sure clever minds can come up with more clever schemes. randy
anyone who relies on a single dns provider is just asking for stuff such as this. I'd love to hear how others are handling the overhead of managing two dns providers.
* randy@psg.com (Randy Bush) [Sat 22 Oct 2016, 00:28 CEST]:
good question. staying in-band, hidden primary comes to mind. but i am sure clever minds can come up with more clever schemes.
The point of outsourcing DNS isn't just availability of static hostnames, it's the added services delivered, like returning different answers based on source of the question, even monitoring your infrastructure (or it reporting load into the DNS management system). That is very hard to replicate with two DNS providers. -- Niels.
Subject: Re: Dyn DDoS this AM? Date: Sat, Oct 22, 2016 at 01:19:24AM +0200 Quoting Niels Bakker (niels=nanog@bakker.net):
The point of outsourcing DNS isn't just availability of static hostnames, it's the added services delivered, like returning different answers based on source of the question, even monitoring your infrastructure (or it reporting load into the DNS management system).
That is very hard to replicate with two DNS providers.
Surely, it must be better to use a singular service that is provably easy to take out. The advantages are overwhelming. -- Måns Nilsson primary/secondary/besserwisser/machina MN-1334-RIPE +46 705 989668 Yow! Are we wet yet?
I don't have a horse in this race, and haven't used it in anger, but Netflix released denominator to attempt to deal with some of these issues: https://github.com/Netflix/denominator Their goal is to support the highest common denominator of features among the supported providers, Maybe that helps someone. Keenan On 2016-10-21 16:19, Niels Bakker wrote:
The point of outsourcing DNS isn't just availability of static hostnames, it's the added services delivered, like returning different answers based on source of the question, even monitoring your infrastructure (or it reporting load into the DNS management system).
That is very hard to replicate with two DNS providers.
-- Niels.
On 10/21/2016 7:34 PM, Keenan Tims wrote:
I don't have a horse in this race, and haven't used it in anger, but Netflix released denominator to attempt to deal with some of these issues:
https://github.com/Netflix/denominator
Their goal is to support the highest common denominator of features among the supported providers,
Maybe that helps someone.
Sadly, it looks like the project is stalled: <https://github.com/Netflix/denominator/issues/374>. -- Rob Szarka http://szarka.org/
On Fri, Oct 21, 2016 at 6:21 PM, David Birdsong <david@imgix.com> wrote:
I'd love to hear how others are handling the overhead of managing two dns providers. Every time we brainstorm on it, we see it as blackhole of eng effort WRT to keeping them in sync and and then waiting for TTLs to cut an entire delegation over.
with the usual caveats - and I dont have any projects that currently need this but have in the past - pretty much every major dns provider allows you to ship them a full zone in some form or fashion. The effort to pull and ship a zone should be fairly minimal in and of itself. mixing your public zone providers in your authoritative NS records is also easy - and, depending on your registrar of choice, should be easy to manage changing those (including having non-public mirrors maintained that you can switch too..). setting TTLs that make sense for a design that supports change is also easy. the real developmental and architectural challenges are around what to do if the APIs you use to talk to your "primary" disappear and you need to consume them (creating new host entries, updating loadbalancer pools, whatever. we all have different and sometimes very diverse use cases for dns.). one approach - as randy suggested - is to switch to a purely hidden and self managed primary - which might mean running your own API stack in front of it to control whatever you need to control and change. this doesnt need to be a "real" dns server in todays world - the days of BIND style zone transfers are generally long gone anyway when you hit these scales and levels of intra complexity. then your zone-replication components that ship zone updates to your various external providers are shipping from the same place. at least in that case it's fully within your control - but dev time and complexity definitely comes into play. if your infra can survive internally without dns change control for the extent of an outage, that could be much easier to manage. anyway, random and incomplete thoughts - time ran out, work calls. ...david
On 2016-10-21 18:45, david raistrick wrote:
switch too..). setting TTLs that make sense for a design that supports change is also easy.
Cuts both ways. Had Twitter had TTLs of say 7 days, vast majority wouldn't notice an outage of a few hours because their local cache wa still valid. It does prevent one from reacting quickly to emergencies.
On 21 October 2016 at 18:12, Jean-Francois Mezei <jfmezei_nanog@vaxination.ca> wrote:
On 2016-10-21 18:45, david raistrick wrote:
switch too..). setting TTLs that make sense for a design that supports change is also easy.
Cuts both ways. Had Twitter had TTLs of say 7 days, vast majority wouldn't notice an outage of a few hours because their local cache wa still valid.
In practice TTLs tend to be ignored on the public internet. In past research I've been involved with browser[0] behavior was effectively random despite the TTL set. [0] more specifically, the chain of DNS resolution and caching down to the browser. -- Eitan Adler
On Oct 21, 2016, at 6:35 PM, Eitan Adler <lists@eitanadler.com> wrote:
[...]
In practice TTLs tend to be ignored on the public internet. In past research I've been involved with browser[0] behavior was effectively random despite the TTL set.
[0] more specifically, the chain of DNS resolution and caching down to the browser.
Yes, but that it can be both better and worse than your TTLs does not mean that you can ignore properly working implementations. If the other end device chain breaks you that's their fault and out of your control. If your own settings break you that's your fault. Sent from my iPhone
On Oct 21, 2016, at 6:35 PM, Eitan Adler <lists@eitanadler.com> wrote:
[...]
In practice TTLs tend to be ignored on the public internet. In past research I've been involved with browser[0] behavior was effectively random despite the TTL set.
[0] more specifically, the chain of DNS resolution and caching down to the browser.
Yes, but that it can be both better and worse than your TTLs does not mean that you can ignore properly working implementations.
If the other end device chain breaks you that's their fault and out of your control. If your own settings break you that's your fault.
+1 to what George wrote that we should make efforts to improve our part of the network. There are ISPs that ignore TTL settings and only update their cached records every two to three days or even more (particularly the smaller ones). OTOH, this results in your DNS data being inconsistent but it’s very common to cache DNS records at multiple levels. It's an effort that everyone needs to contribute to.
Sent from my iPhone
In message <CADJJukkadFbOYvWVan_8pdR=fxenqGRsyisiKBH6vpyDse6JrQ@mail.gmail.com> , Masood Ahmad Shah writes:
On Oct 21, 2016, at 6:35 PM, Eitan Adler <lists@eitanadler.com> wrote:
[...]
In practice TTLs tend to be ignored on the public internet. In past research I've been involved with browser[0] behavior was effectively random despite the TTL set.
[0] more specifically, the chain of DNS resolution and caching down to the browser.
Yes, but that it can be both better and worse than your TTLs does not mean that you can ignore properly working implementations.
If the other end device chain breaks you that's their fault and out of your control. If your own settings break you that's your fault.
+1 to what George wrote that we should make efforts to improve our part of the network. There are ISPs that ignore TTL settings and only update their cached records every two to three days or even more (particularly the smaller ones). OTOH, this results in your DNS data being inconsistent but it’s very common to cache DNS records at multiple levels. It's an effort that everyone needs to contribute to.
For TTL there is a tension between being able to update with new data and resiliance when servers are unreachable. For zone transfers we have 3 timers refesh, retry and expire to deal with this tension. If we were doing DNS from scratch there would be at least two ttl values one for freshness and one for don't use past. Additionally a lot of the need for small TTL's is because clients don't fail over to second addresses in a reasonable amount of time. There is no reason for this other than poorly designed clients. A client can failover using sub-second timers. We do this for Happy Eyeballs. This strategy is viable for ALL connection attempts. Mark -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org
All this TTL talk makes me think. Why not have two ttls - a 'must-recheck' (does not expire the record but forces a recheck; updates record if server replies & serial has incremented) and a 'must-delete' (cache will be stale at this point)? On October 23, 2016 3:42:58 PM PDT, Mark Andrews <marka@isc.org> wrote:
On Oct 21, 2016, at 6:35 PM, Eitan Adler <lists@eitanadler.com>
wrote:
[...]
In practice TTLs tend to be ignored on the public internet. In
research I've been involved with browser[0] behavior was effectively random despite the TTL set.
[0] more specifically, the chain of DNS resolution and caching down to the browser.
Yes, but that it can be both better and worse than your TTLs does not mean that you can ignore properly working implementations.
If the other end device chain breaks you that's their fault and out of your control. If your own settings break you that's your fault.
+1 to what George wrote that we should make efforts to improve our
the network. There are ISPs that ignore TTL settings and only update
In message <CADJJukkadFbOYvWVan_8pdR=fxenqGRsyisiKBH6vpyDse6JrQ@mail.gmail.com> , Masood Ahmad Shah writes: past part of their
cached records every two to three days or even more (particularly the smaller ones). OTOH, this results in your DNS data being inconsistent but it���s very common to cache DNS records at multiple levels. It's an effort that everyone needs to contribute to.
For TTL there is a tension between being able to update with new data and resiliance when servers are unreachable. For zone transfers we have 3 timers refesh, retry and expire to deal with this tension. If we were doing DNS from scratch there would be at least two ttl values one for freshness and one for don't use past.
Additionally a lot of the need for small TTL's is because clients don't fail over to second addresses in a reasonable amount of time. There is no reason for this other than poorly designed clients. A client can failover using sub-second timers. We do this for Happy Eyeballs. This strategy is viable for ALL connection attempts.
Mark
-- Sent from my Android device with K-9 Mail. Please excuse my brevity.
On 24 October 2016 at 01:25, LHC <large.hadron.collider@gmx.com> wrote:
All this TTL talk makes me think.
Why not have two ttls - a 'must-recheck' (does not expire the record but forces a recheck; updates record if server replies & serial has incremented) and a 'must-delete' (cache will be stale at this point)?
If clients can't get one TTL correct what makes you think they will get a more complicated two TTL system correct? -- Eitan Adler
On Oct 24, 2016, at 12:06 PM, Eitan Adler <lists@eitanadler.com> wrote:
On 24 October 2016 at 01:25, LHC <large.hadron.collider@gmx.com> wrote:
All this TTL talk makes me think.
Why not have two ttls - a 'must-recheck' (does not expire the record but forces a recheck; updates record if server replies & serial has incremented) and a 'must-delete' (cache will be stale at this point)?
If clients can't get one TTL correct what makes you think they will get a more complicated two TTL system correct?
….To say nothing of resolvers that simply ignore server-side TTLs and set their own. For instance, https://www.icann.org/en/system/files/files/rssac-003-root-zone-ttls-21aug15... <https://www.icann.org/en/system/files/files/rssac-003-root-zone-ttls-21aug15-en.pdf> “RSSAC 003: RSSAC Report on Root Zone TTLs” will tell you far more than you really want to know about TTLs and caching behavior, and some of it is specific to the root zone, but one of the key observations is "Root zone TTLs appear to not matter to most clients.” Modern large-scale DNS is a fairly complex system. Speculating from here about how it behaved under attack in someone else’s network is interesting, and I look forward to more information from Dyn as they feel they can share it— but DDoS is a big enough fact of life for them and everyone else that if there was a simple answer, I think someone would be making a fortune on it already, or at least have filed the patents. Suzanne (speaking for myself)
All this TTL talk makes me think. Why not have two ttls - a 'must-recheck' (does not expire the record but forces a recheck; updates record if server replies & serial has incremented) and a 'must-delete' (cache will be stale at this point)? On October 23, 2016 3:42:58 PM PDT, Mark Andrews <marka@isc.org> wrote:
On Oct 21, 2016, at 6:35 PM, Eitan Adler <lists@eitanadler.com>
wrote:
[...]
In practice TTLs tend to be ignored on the public internet. In
research I've been involved with browser[0] behavior was effectively random despite the TTL set.
[0] more specifically, the chain of DNS resolution and caching down to the browser.
Yes, but that it can be both better and worse than your TTLs does not mean that you can ignore properly working implementations.
If the other end device chain breaks you that's their fault and out of your control. If your own settings break you that's your fault.
+1 to what George wrote that we should make efforts to improve our
the network. There are ISPs that ignore TTL settings and only update
In message <CADJJukkadFbOYvWVan_8pdR=fxenqGRsyisiKBH6vpyDse6JrQ@mail.gmail.com> , Masood Ahmad Shah writes: past part of their
cached records every two to three days or even more (particularly the smaller ones). OTOH, this results in your DNS data being inconsistent but it���s very common to cache DNS records at multiple levels. It's an effort that everyone needs to contribute to.
For TTL there is a tension between being able to update with new data and resiliance when servers are unreachable. For zone transfers we have 3 timers refesh, retry and expire to deal with this tension. If we were doing DNS from scratch there would be at least two ttl values one for freshness and one for don't use past.
Additionally a lot of the need for small TTL's is because clients don't fail over to second addresses in a reasonable amount of time. There is no reason for this other than poorly designed clients. A client can failover using sub-second timers. We do this for Happy Eyeballs. This strategy is viable for ALL connection attempts.
Mark
-- Sent from my Android device with K-9 Mail. Please excuse my brevity. -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
On 10/21/16 3:21 PM, David Birdsong wrote:
On Fri, Oct 21, 2016 at 2:58 PM, Randy Bush <randy@psg.com> wrote:
anyone who relies on a single dns provider is just asking for stuff such as this.
randy
I'd love to hear how others are handling the overhead of managing two dns providers. Every time we brainstorm on it, we see it as blackhole of eng effort WRT to keeping them in sync and and then waiting for TTLs to cut an entire delegation over.
Not all the ones you might choose based on scale support axfr... That's a bit of a problem for the most traditional approach to this., of those that do it's straight-forward to use one as the master for another, or use a hidden master. Your own master may have demonstrably lower availability then one or the other of your providers. getting two well considered choices to play nice with each other isn't that hard.
Subject: Re: Dyn DDoS this AM? Date: Fri, Oct 21, 2016 at 03:21:20PM -0700 Quoting David Birdsong (david@imgix.com):
On Fri, Oct 21, 2016 at 2:58 PM, Randy Bush <randy@psg.com> wrote:
anyone who relies on a single dns provider is just asking for stuff such as this.
randy
I'd love to hear how others are handling the overhead of managing two dns providers. Every time we brainstorm on it, we see it as blackhole of eng effort WRT to keeping them in sync and and then waiting for TTLs to cut an entire delegation over.
The fault is giving up the primary for an API connection. Sure, it is tempting. We do, however, need to push the "application-integrated" DNS vendors harder. They need to give their customers more choice in how the DNS is populated. They also very much need to let people with above-mentioned "application-integrated" needs add third party DNS providers in the mix. This diversity capability is what makes DNS resilient. Monocultures have suboptimal survivability in the long run. Adding DNS providers when you control the primary is completely painless. With EDNS0 there's lots of room for insanely large NS RRSETs. Also, do not fall in the "short TTL for service agility" trap. Besides, what Randy wrote. -- Måns Nilsson primary/secondary/besserwisser/machina MN-1334-RIPE +46 705 989668 Hold the MAYO & pass the COSMIC AWARENESS ...
Ansible would be a decent start. On Oct 21, 2016 5:26 PM, "David Birdsong" <david@imgix.com> wrote:
On Fri, Oct 21, 2016 at 2:58 PM, Randy Bush <randy@psg.com> wrote:
anyone who relies on a single dns provider is just asking for stuff such as this.
randy
I'd love to hear how others are handling the overhead of managing two dns providers. Every time we brainstorm on it, we see it as blackhole of eng effort WRT to keeping them in sync and and then waiting for TTLs to cut an entire delegation over.
Ah, disregard. I see what you're saying now. Yes, I can see how that would be problematic. On Oct 21, 2016 6:40 PM, "Josh Reynolds" <josh@kyneticwifi.com> wrote:
Ansible would be a decent start.
On Oct 21, 2016 5:26 PM, "David Birdsong" <david@imgix.com> wrote:
On Fri, Oct 21, 2016 at 2:58 PM, Randy Bush <randy@psg.com> wrote:
anyone who relies on a single dns provider is just asking for stuff such as this.
randy
I'd love to hear how others are handling the overhead of managing two dns providers. Every time we brainstorm on it, we see it as blackhole of eng effort WRT to keeping them in sync and and then waiting for TTLs to cut an entire delegation over.
On 10/21/16 at 03:21pm, David Birdsong wrote:
On Fri, Oct 21, 2016 at 2:58 PM, Randy Bush <randy@psg.com> wrote:
anyone who relies on a single dns provider is just asking for stuff such as this.
:-)
I'd love to hear how others are handling the overhead of managing two dns providers.
in my view of ( automated ) dns managment: Only on the one "master" dns server, make your DNS changes, update the serial number for example.com changes and reload the new update zone file ... notifications goes out to all known slave DNS servers .. For all the other authorized DNS servers, they should all automatically update itself ... magic all dns servers are in sync ... some folks don't like "master" DNS server vs slaves .. i donno why not .. but, you do have to configure your "master dns server" properly to only allow only authorized slaves access to their dns reccords similarly, slave DNS servers should only update from it's recognized master dns server there should be zero isues with managing 2 dns server or 100 dns servers before downloading new dns info, Man-in-the-Middle tests with OpenSSL certs should be done to confirm the other end is in fact who you think it is that you're going to be sending dns info to or receiving from c ya alvin http://DDoS-Mitigator.net
* Randy Bush:
anyone who relies on a single dns provider is just asking for stuff such as this.
Blaming the victim isn't helpful. And without end-user-visible changes, most of the victims would still depend on Verisign as a single provider for a critical part of their DNS service.
participants (41)
-
Alain Hebert -
Alexander Lyamin -
alvin nanog -
Andrew Fried -
Brandon Ross -
Brian Davies -
Chris Grundemann -
Chris Woodfield -
Daniel Ankers -
David Birdsong -
David Hubbard -
david raistrick -
Eitan Adler -
Florian Weimer -
George William Herbert -
Jared Mauch -
Jean-Francois Mezei -
joel jaeggli -
Josh Reynolds -
Keenan Tims -
LHC -
marcel.duregards＠yahoo.fr -
Mark Andrews -
Masood Ahmad Shah -
Matthew Black -
Mehmet Akcin -
Mike Hammett -
Måns Nilsson -
Nick Hilliard -
Niels Bakker -
Patrick W. Gilmore -
Randy Bush -
rar -
Rob Szarka -
Roland Dobbins -
Ronald F. Guilmette -
Seth Mattinen -
Steve Meuse -
Suzanne Woolf -
Wayne Bouchard -
Yang Yu