> http://www.slate.com/id/2166749/fr/podcast/ Downloading it now. John Markoff just called me for the NYT piece. Odd that it's just hitting the news now, two weeks later. -Bill
> > http://www.slate.com/id/2166749/fr/podcast/ > > Downloading it now. > > John Markoff just called me for the NYT piece. Odd that it's just hitting > the news now, two weeks later. http://www.washingtonpost.com/wp-dyn/content/article/2007/05/18/AR2007051802... -Bill
On Wed, 23 May 2007, Bill Woodcock wrote:
Downloading it now.
John Markoff just called me for the NYT piece. Odd that it's just hitting the news now, two weeks later.
I wonder, does this mean Estonia is now more likely to act/re-act to its own homegrown miscreants which attack systems in other countries after seeing the impact it had in their own country? Or is this going to remain a case of the "bad guys" are always in some other country, not mine.
On Wed, 23 May 2007, Sean Donelan wrote: > I wonder, does this mean Estonia is now more likely to act/re-act to its > own homegrown miscreants which attack systems in other countries after > seeing the impact it had in their own country? Or is this going to remain > a case of the "bad guys" are always in some other country, not mine. By "bad guys" do you mean the bots, or the C&C? I think in non-state-actor attacks, prosecution of C&C has been reasonably good. It's the botnets that I worry about. All those people still paying Microsoft to make their machines zombies. :-/ -Bill
On Wed, May 23, 2007 at 03:06:58PM -0400, Sean Donelan wrote:
On Wed, 23 May 2007, Bill Woodcock wrote:
Downloading it now.
John Markoff just called me for the NYT piece. Odd that it's just hitting the news now, two weeks later.
I wonder, does this mean Estonia is now more likely to act/re-act to its own homegrown miscreants which attack systems in other countries after seeing the impact it had in their own country? Or is this going to remain a case of the "bad guys" are always in some other country, not mine.
I just now got from a 6 hours beer fest with ISP/CERT/military/etc. guys who have been working on these attacks on Estonian infrastructure for the past 3 weeks here in Tallinn.. so if I make less sense than usual, please forgive me. Beer good. Sitting with these folks for the past week, I got so impressed with the abuse handling work they are doing that even I, who had a very negative opinion of Estonia and cyber-crime, completely changed my mind. Their CERT is *extremely* responsive, their ISPs are all talking and cooperating on abuse and security (and drinking beer). Things are very different from what they were even just a year ago. Even their Police force is clued. If anyone has issues in Estonia, I'd strongly urge you to contact the Estonian CERT at www.cert.ee, and you most likely won't get disappointed. A lot of good people over here. Gadi.
On 5/23/07, ge@linuxbox.org <ge@linuxbox.org> wrote:
I just now got from a 6 hours beer fest with ISP/CERT/military/etc. guys who have been working on these attacks on Estonian infrastructure for the past 3 weeks here in Tallinn.. so if I make less sense than usual, please forgive me. Beer good.
Sitting with these folks for the past week, I got so impressed with the abuse handling work they are doing that even I, who had a very negative opinion of Estonia and cyber-crime, completely changed my mind.
Their CERT is *extremely* responsive, their ISPs are all talking and cooperating on abuse and security (and drinking beer). Things are very different from what they were even just a year ago. Even their Police force is clued.
If anyone has issues in Estonia, I'd strongly urge you to contact the Estonian CERT at www.cert.ee, and you most likely won't get disappointed. A lot of good people over here.
Gadi.
How serious was the attack really? The national press reporting was either nonexistent or hysterical (Cyberwar! Woo!), but it didn't disturb anyone to post to NANOG at any point, and it does not seem to have had any measurable real-world consequences. Was this because a) it wasn't really that serious, b) it was serious but mitigation was successful, or c) being well-mitigated (BCP38 and the like) from the word go, its seriousness or otherwise wasn't obvious?
On Thu, 24 May 2007, Alexander Harrowell wrote: > a) it wasn't really that serious, b) it was serious > but mitigation was successful, or c) being well-mitigated (BCP38 and > the like) from the word go, its seriousness or otherwise wasn't > obvious? Definitely (b). The EE-CERT was remarkably well-prepared and effective, and their counterparts around the world cooperated dilligently and professionally. It was a very large attack. -Bill
On May 24, 2007, at 7:09 AM, Bill Woodcock wrote:
On Thu, 24 May 2007, Alexander Harrowell wrote:
a) it wasn't really that serious, b) it was serious but mitigation was successful, or c) being well-mitigated (BCP38 and the like) from the word go, its seriousness or otherwise wasn't obvious?
Definitely (b). The EE-CERT was remarkably well-prepared and effective, and their counterparts around the world cooperated dilligently and professionally. It was a very large attack.
-Bill
People might be interested in a military perspective on this : http://globalguerrillas.typepad.com/globalguerrillas/2007/05/ an_internet_ebo.html and a possible response http://globalguerrillas.typepad.com/globalguerrillas/2007/05/ journal_a_new_m.html Regards Marshall
Yes...definitely b. I was there.......and Bill speaks from experience as well since the RIPE meeting happened to be in Tallinn during the peak of the attack. I'm of estonian decent btw.......so I could keep up with local news (besides having personal contacts) and there were severe attacks to government sites and major banks as well as other facilities. Although nothing ended up being down for more than an hour that had been recognized ahead of time as being 'important'. Most people are not aware how much estonia relies on it's network infrastructure for government operations, banking and the daily life of ordinary people. They had the good fortune of starting 'fresh' in the early 1990's without legacy baggage :) They were prepared for the worst......technically the folks I've dealt with are some of the best.........and it's important to keep in mind that since the attack(s) were politically motivated, the timing of the worst of it was known and the banks, ISPs, police, government could coordinate a pretty tight plan of action. It is an unusual situation...or at least the first of its kind. - merike
It is an unusual situation...or at least the first of its kind.
Leaving aside the alleged political involvement of some government or other, this is far from true. Back in the days, when DOS attacks were delivered to mailboxes and USENET and IRC were the main tool of coordinating attacks, this was commonplace. A victim was identified, postings were made to newsgroups and IRC channels, and at the appointed time, the attack begins. What is fundamentally different here? Using web forums and IM instead of USENET/IRC is not fundamentally different. Using botnets to amplify the attack, is different from the mailbombing of the past, however, the botnets are often used in DDoS attacks, so I don't think we can consider this fundamentally different. What about the attackers? Is there something about Russians that would explain this? Yes, I think so. Over the past 20 years, economic and social problems have hit Russia hard and the people that lived through this time learned how to cooperate effectively and how to change tactics on short notice. At the same time, the Russian education system produces people who are very good at technical subjects, like networks, programming, etc. This has combined to create various criminal groups who can make a good living from net abuse by building and renting botnets or selling various spamming services or just plain phishing. The Russian mob does have a big market share of botnet C&C(Command and Control). IMHO, this is not about Estonia and this is not about the Russian government or military or intelligence agencies. This is all about free enterprise thinking which is more deeply embedded in Russia than in most of the developed world. Generally, these Russian hackers apply their skills to earning money or attacking each other, but Estonia accidentally raised the hackles of these people and they all pointed their firehoses in unison. It could have been any other country which does something that offends the sensibilities of ordinary Russians. On the other hand, if this attack had been directed at the USA, it would have had far less effect. The USA has its economic and government infrastructure scattered across many cities with lots of network capacity between. The target for the firehose is more diffuse and therefore harder to hit. Estonia is a little country with all its eggs in one basket in one city. It was an interesting coincidence that one of the more vulnerable countries just happened to get a large number of criminal hacker gangs upset enough to turn from earning money to attack them. Perhaps they haven't heard that people who live in glass houses shouldn't throw stones. There has been a lot of hyperbole over these incidents and little factual information. Some people want to point the finger of blame, but with botnets and diffuse C&C out there, this is not something that can be easily or quickly confirmed. If it was so easy, then we would have put the botnet operators out of business long ago. It's nice to hear that the Estonian CERT was prepared to respond to an attack and it's nice to hear that a lot of people helped mitigate the attack. But there is nothing new in that. There are a lot of accusations about attacks coming from a certain list of countries or from certain specific computers of certain government officials, but these sound like typical tabloid journalism explanations of any botnet-based DDoS. People say this was a BIG deal but then we hear that sites were down for only an hour. The Northeast blackout was a big deal, Katrina was a big deal, but a few hours of outage for a few data centres in one city doesn't seem to me like a big deal. A claim was made that 4 million packets per second were sent. I would like to hear more about this. How was it measured? Is this an aggregate or was this directed at the largest victim? Was it ingress into the network or packets delivered on the site's CPE router? How does this compare to other DDoS incidents. And, most importantly, does it indicate a growth in total DDoS capability (a bigger firehose than before) or was it simply the usual stuff all sent to the same victim at the same time, for a change. What can network operators learn from this? Do we need to beef up technical measures or will a well-run network already be prepared to mitigate this kind of thing? Is there some fundamental technical aspect of this attack that was different from the past? Did the mitigation of the attack do something fundamentally different from the past? --Michael Dillon
First of it's kind that it targeted a country. As far as technical details I'm pulling something together for nsp- sec BoF at NANOG. I saw the spike to 4m pps on their management station......so no 'claims' there. And yeah, OK, will need qualification. Basically that was seen by Estonian ISPs as traffic coming in.........technically there wasn't much difference to what people see today but the large scale coordination is unusual. Or maybe not since it's small country :) As far as the important sites being down for a short time.....that was because the mitigation techniques had been well thought out and they were prepared. And a LOT of money was spent to add equipment and enforce mitigation in the week before the worst was expected. There was a lot of pro-active activity which I do find to be unusual. Noone wants to spend money on security (said very tongue-in-cheek)....... I'll include answers to your last questions in my preso....... - merike As far as technical On May 24, 2007, at 9:35 AM, <michael.dillon@bt.com> <michael.dillon@bt.com> wrote:
It is an unusual situation...or at least the first of its kind.
Leaving aside the alleged political involvement of some government or other, this is far from true. Back in the days, when DOS attacks were delivered to mailboxes and USENET and IRC were the main tool of coordinating attacks, this was commonplace. A victim was identified, postings were made to newsgroups and IRC channels, and at the appointed time, the attack begins.
What is fundamentally different here?
Using web forums and IM instead of USENET/IRC is not fundamentally different. Using botnets to amplify the attack, is different from the mailbombing of the past, however, the botnets are often used in DDoS attacks, so I don't think we can consider this fundamentally different.
What about the attackers? Is there something about Russians that would explain this? Yes, I think so. Over the past 20 years, economic and social problems have hit Russia hard and the people that lived through this time learned how to cooperate effectively and how to change tactics on short notice. At the same time, the Russian education system produces people who are very good at technical subjects, like networks, programming, etc. This has combined to create various criminal groups who can make a good living from net abuse by building and renting botnets or selling various spamming services or just plain phishing. The Russian mob does have a big market share of botnet C&C(Command and Control).
IMHO, this is not about Estonia and this is not about the Russian government or military or intelligence agencies. This is all about free enterprise thinking which is more deeply embedded in Russia than in most of the developed world. Generally, these Russian hackers apply their skills to earning money or attacking each other, but Estonia accidentally raised the hackles of these people and they all pointed their firehoses in unison. It could have been any other country which does something that offends the sensibilities of ordinary Russians.
On the other hand, if this attack had been directed at the USA, it would have had far less effect. The USA has its economic and government infrastructure scattered across many cities with lots of network capacity between. The target for the firehose is more diffuse and therefore harder to hit. Estonia is a little country with all its eggs in one basket in one city.
It was an interesting coincidence that one of the more vulnerable countries just happened to get a large number of criminal hacker gangs upset enough to turn from earning money to attack them. Perhaps they haven't heard that people who live in glass houses shouldn't throw stones.
There has been a lot of hyperbole over these incidents and little factual information. Some people want to point the finger of blame, but with botnets and diffuse C&C out there, this is not something that can be easily or quickly confirmed. If it was so easy, then we would have put the botnet operators out of business long ago. It's nice to hear that the Estonian CERT was prepared to respond to an attack and it's nice to hear that a lot of people helped mitigate the attack. But there is nothing new in that. There are a lot of accusations about attacks coming from a certain list of countries or from certain specific computers of certain government officials, but these sound like typical tabloid journalism explanations of any botnet-based DDoS. People say this was a BIG deal but then we hear that sites were down for only an hour. The Northeast blackout was a big deal, Katrina was a big deal, but a few hours of outage for a few data centres in one city doesn't seem to me like a big deal.
A claim was made that 4 million packets per second were sent. I would like to hear more about this. How was it measured? Is this an aggregate or was this directed at the largest victim? Was it ingress into the network or packets delivered on the site's CPE router? How does this compare to other DDoS incidents. And, most importantly, does it indicate a growth in total DDoS capability (a bigger firehose than before) or was it simply the usual stuff all sent to the same victim at the same time, for a change.
What can network operators learn from this? Do we need to beef up technical measures or will a well-run network already be prepared to mitigate this kind of thing? Is there some fundamental technical aspect of this attack that was different from the past? Did the mitigation of the attack do something fundamentally different from the past?
--Michael Dillon
> First of it's kind that it targeted a country. No, at the very least, Moonlight Maze and Titan Rain came before. But by today's standards, Moonlight Maze would have been trivially small. I don't have any numbers for Titan Rain. Anyone know how it compared to the 4mpps of this attack? -Bill
On May 24, 2007, at 4:58 PM, Bill Woodcock wrote:
First of it's kind that it targeted a country.
No, at the very least, Moonlight Maze and Titan Rain came before. But by today's standards, Moonlight Maze would have been trivially small. I don't have any numbers for Titan Rain. Anyone know how it compared to the 4mpps of this attack?
A data point based on some information we have from looking at inter-domain traffic and attack attributes across ~40 ISPs (~1 Tbps) over ~250 days now (and rolling): Days seeing at least one attack exceeding a given threshold:
6 Mpps 1 5 Mpps 12 4 Mpps 33 3 Mpps 53 2 Mpps 91 1 Mpps 149
Total attacks exceeding a given threshold:
6 Mpps 1 5 Mpps 17 4 Mpps 82 3 Mpps 135 2 Mpps 352 1 Mpps 813
The above is from the perspective of *a single ISP*, so the aggregate of the attack is likely to be far greater (cross-ISP correlation of targets are NOT reflected in _this dataset). Mpps and greater attacks make up far less than 1% of the attacks we see (we've have data for ~142k known attacks over this period). More on this in the near future and note that none of the above is meant to marginalize the Estonian attacks in any way, 4 Mpps is a lot depending on where it's directed and how it's mitigated - it's ALL about perspective..... -danny
On Thu, 24 May 2007, Bill Woodcock wrote:
First of it's kind that it targeted a country. No, at the very least, Moonlight Maze and Titan Rain came before. But by today's standards, Moonlight Maze would have been trivially small. I don't have any numbers for Titan Rain. Anyone know how it compared to the 4mpps of this attack?
Don't forget the Pakistan/India cyber-skimishes. Substantial parts of their Internet infrastructure was successfully attacked for months at a time. Periodically, China, Japan and South Korea seem to have rotating grudge matches between their hacking groups. And in wars with bullets, there was Yugoslavia and Radio B92 all-media attacks; and pro-Chinese groups launched several attacks after the US bombed the Chincese embassy in Belgrade. There have been so many cyber-protests of many different US policies for many years even keeping a list would be a lot of work. And even some pro-US groups launching cyber-attacks to protest policies of other countries.
On Thu, 24 May 2007, Sean Donelan wrote:
On Thu, 24 May 2007, Bill Woodcock wrote:
First of it's kind that it targeted a country. No, at the very least, Moonlight Maze and Titan Rain came before. But by today's standards, Moonlight Maze would have been trivially small. I don't have any numbers for Titan Rain. Anyone know how it compared to the 4mpps of this attack?
Don't forget the Pakistan/India cyber-skimishes. Substantial parts of their Internet infrastructure was successfully attacked for months at a time.
may-day 2000? China/america hackers attack each other. most of 2001 hamas/isreali hackers battle it out, highlight: "www.hezbollah.org" attacked, which was hosted on inter.net.il (I think)... funny stuff :) certainly there have been other country-targetted attacks before estonia, yes. I do find it interesting that someone pointed the finger directly at the soverign nation instead of nationalist hacker groups or the like... I think the washingtonpost.com article last week about this said the finger pointed started because 'a kremlin minister's computer was identified as an attacker!' (or was bot'd and participated in the attack(s) ) anyway, fun stuff, I'll have to listen to the podcast eventually. -Chris
On Thu, 2007-05-24 at 10:06 -0700, Merike Kaeo wrote:
First of it's kind that it targeted a country.
Countries and govt infrastructure has been under attack before. As an example; The various parties in the Balkan conflict (former Yugoslavia) were fighting their "cyber-wars" back in the 90s. Attacks were of course at a different scale as national/regional ISPs at the time only had a few k of transit capacity. //per
On Thu, May 24, 2007 at 09:25:54AM +0100, Alexander Harrowell wrote:
On 5/23/07, ge@linuxbox.org <ge@linuxbox.org> wrote:
I just now got from a 6 hours beer fest with ISP/CERT/military/etc. guys who have been working on these attacks on Estonian infrastructure for the past 3 weeks here in Tallinn.. so if I make less sense than usual, please forgive me. Beer good.
Sitting with these folks for the past week, I got so impressed with the abuse handling work they are doing that even I, who had a very negative opinion of Estonia and cyber-crime, completely changed my mind.
Their CERT is *extremely* responsive, their ISPs are all talking and cooperating on abuse and security (and drinking beer). Things are very different from what they were even just a year ago. Even their Police force is clued.
If anyone has issues in Estonia, I'd strongly urge you to contact the Estonian CERT at www.cert.ee, and you most likely won't get disappointed. A lot of good people over here.
Gadi.
How serious was the attack really? The national press reporting was either nonexistent or hysterical (Cyberwar! Woo!), but it didn't disturb anyone to post to NANOG at any point, and it does not seem to have had any measurable real-world consequences.
Was this because a) it wasn't really that serious, b) it was serious but mitigation was successful, or c) being well-mitigated (BCP38 and the like) from the word go, its seriousness or otherwise wasn't obvious?
A lot of people had information to share and emotions to get out of the way, I sent my reply off-list. Also, it was really not my place reply on this - with all the work done by the Estonians, my contributions were secondary. My discussions with Mr. Harrowell are public on his blog. Information from Bill Wodcock was also sound. As to what actually happened over there, more information should become available soon and I will send it here. I keep getting stuck when trying to write the post-mortem and attack/defense analysis as I keep hitting a stone wall I did not expect: strategy. Suggestions for the future is also a part of that document, so I will speed it up with a more down-to-Earth technical analysis (which is what I promised CERT-EE). In the past I've been able to consider information warfare as a part of a larger strategy, utilizing it as a weapon. I was able to think of impact and tools, not to mention (mostly) disconnected attacks and defenses. I keep seeing strategy for the use IN information warfare battles as I write this document on what happened in Estonia, and I believe I need more time to explore this against my previous take on the issue, as well as take a look at some classics such as Clausewitz, as posh as it may sound. Thanks, Gadi.
On Thu, May 24, 2007 at 09:25:54AM +0100, Alexander Harrowell wrote:
On 5/23/07, ge@linuxbox.org <ge@linuxbox.org> wrote:
I just now got from a 6 hours beer fest with ISP/CERT/military/etc. guys who have been working on these attacks on Estonian infrastructure for the past 3 weeks here in Tallinn.. so if I make less sense than usual, please forgive me. Beer good.
Sitting with these folks for the past week, I got so impressed with the abuse handling work they are doing that even I, who had a very negative opinion of Estonia and cyber-crime, completely changed my mind.
Their CERT is *extremely* responsive, their ISPs are all talking and cooperating on abuse and security (and drinking beer). Things are very different from what they were even just a year ago. Even their Police force is clued.
If anyone has issues in Estonia, I'd strongly urge you to contact the Estonian CERT at www.cert.ee, and you most likely won't get disappointed. A lot of good people over here.
Gadi.
How serious was the attack really? The national press reporting was either nonexistent or hysterical (Cyberwar! Woo!), but it didn't disturb anyone to post to NANOG at any point, and it does not seem to have had any measurable real-world consequences.
Was this because a) it wasn't really that serious, b) it was serious but mitigation was successful, or c) being well-mitigated (BCP38 and the like) from the word go, its seriousness or otherwise wasn't obvious?
A lot of people had information to share and emotions to get out of the way, I sent my reply off-list. Also, it was really not my place reply on this - with all the work done by the Estonians, my contributions were secondary. My discussions with Mr. Harrowell are public on his blog. Information from Bill Wodcock was also sound. As to what actually happened over there, more information should become available soon and I will send it here. I keep getting stuck when trying to write the post-mortem and attack/defense analysis as I keep hitting a stone wall I did not expect: strategy. Suggestions for the future is also a part of that document, so I will speed it up with a more down-to-Earth technical analysis (which is what I promised CERT-EE). In the past I've been able to consider information warfare as a part of a larger strategy, utilizing it as a weapon. I was able to think of impact and tools, not to mention (mostly) disconnected attacks and defenses. I keep seeing strategy for the use IN information warfare battles as I write this document on what happened in Estonia, and I believe I need more time to explore this against my previous take on the issue, as well as take a look at some classics such as Clausewitz, as posh as it may sound. Thanks, Gadi.
participants (11)
-
Alexander Harrowell -
Bill Woodcock -
Chris L. Morrow -
Danny McPherson -
ge＠linuxbox.org -
Marshall Eubanks -
Merike Kaeo -
michael.dillon＠bt.com -
Per Heldal -
Sean Donelan -
Tom Vest