NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet
http://www.nytimes.com/2011/02/16/technology/16internet.html There has been intense debate both inside and outside Egypt on whether the
cutoff at 26 Ramses Street was accomplished by surgically tampering with the software mechanism that defines how networks at the core of the Internet communicate with one another, or by a blunt approach: simply cutting off the power to the router computers that connect Egypt to the outside world.
-- --------------------------------------------------------------- Joly MacFie 218 565 9365 Skype:punkcast WWWhatsup NYC - http://wwwhatsup.com http://pinstand.com - http://punkcast.com VP (Admin) - ISOC-NY - http://isoc-ny.org ---------------------------------------------------------------
On Feb 16, 2011, at 12:15 AM, Joly MacFie wrote:
http://www.nytimes.com/2011/02/16/technology/16internet.html
There has been intense debate both inside and outside Egypt on whether the
cutoff at 26 Ramses Street was accomplished by surgically tampering with the software mechanism that defines how networks at the core of the Internet communicate with one another, or by a blunt approach: simply cutting off the power to the router computers that connect Egypt to the outside world.
I do remember some intense debate, here and elsewhere, but I somehow don't remember those as being the primary debate parameters. Regards Marshall
-- --------------------------------------------------------------- Joly MacFie 218 565 9365 Skype:punkcast WWWhatsup NYC - http://wwwhatsup.com http://pinstand.com - http://punkcast.com VP (Admin) - ISOC-NY - http://isoc-ny.org ---------------------------------------------------------------
On Wed, Feb 16, 2011 at 9:09 AM, Marshall Eubanks <tme@americafree.tv> wrote:
On Feb 16, 2011, at 12:15 AM, Joly MacFie wrote:
http://www.nytimes.com/2011/02/16/technology/16internet.html
There has been intense debate both inside and outside Egypt on whether the
cutoff at 26 Ramses Street was accomplished by surgically tampering with the software mechanism that defines how networks at the core of the Internet communicate with one another, or by a blunt approach: simply cutting off the power to the router computers that connect Egypt to the outside world.
I do remember some intense debate, here and elsewhere, but I somehow don't remember those as being the primary debate parameters.
Interesting article though. There are several good pieces to take-away from that article (not really 'news', but still healthy with the occasional refresher): "Individual Internet service providers were also called on the carpet and ordered to shut down, as they are required to do by their licensing agreements if the government so decrees." "When he, too, noticed that domestic fiber-optic cables were open, he had a moment of exhilaration, remembering that he could link up servers directly and establish messaging using an older system called Internet Relay Chat. But then it dawned on him that he had always assumed he could download the necessary software via the Internet and had saved no copy." Operating local IRC networks is good, as is having local OS mirrors, such as Debian/Ubuntu and let's not forget, having a resilient DNS configuration (root zone copy hint 101: "dig @k.root-servers.net. . axfr"). A securely distributed "network-contingency-plan-autocrat-generic-all.deb" could be useful, as well. Cheers, Martin
----- Original Message -----
From: "Martin Millnert" <millnert@gmail.com> To: "Marshall Eubanks" <tme@americafree.tv> Cc: "North American Network Operators Group" <nanog@nanog.org> Sent: Thursday, 17 February, 2011 8:28:22 AM Subject: Re: NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet On Wed, Feb 16, 2011 at 9:09 AM, Marshall Eubanks <tme@americafree.tv> wrote:
On Feb 16, 2011, at 12:15 AM, Joly MacFie wrote:
"
Operating local IRC networks is good, as is having local OS mirrors, such as Debian/Ubuntu and let's not forget, having a resilient DNS configuration (root zone copy hint 101: "dig @k.root-servers.net. . axfr"). A securely distributed
Would it make sense for an ISP to "store" the root zone on their DNS servers instead of letting it be refreshed by the DNS cache? A cron job could refresh it from time to time. It would avoid entries from expiring and would always serve to clients entries with max ttl? A root server would be better, but that could be an intermediary step? Just speaking out loud here, so it may be total non-sense...
On 02/16/2011 11:50, Franck Martin wrote:
----- Original Message -----
From: "Martin Millnert"<millnert@gmail.com> To: "Marshall Eubanks"<tme@americafree.tv> Cc: "North American Network Operators Group"<nanog@nanog.org> Sent: Thursday, 17 February, 2011 8:28:22 AM Subject: Re: NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet On Wed, Feb 16, 2011 at 9:09 AM, Marshall Eubanks<tme@americafree.tv> wrote:
On Feb 16, 2011, at 12:15 AM, Joly MacFie wrote:
"
Operating local IRC networks is good, as is having local OS mirrors, such as Debian/Ubuntu and let's not forget, having a resilient DNS configuration (root zone copy hint 101: "dig @k.root-servers.net. . axfr"). A securely distributed
Would it make sense for an ISP to "store" the root zone on their DNS servers instead of letting it be refreshed by the DNS cache? A cron job could refresh it from time to time. It would avoid entries from expiring and would always serve to clients entries with max ttl?
A root server would be better, but that could be an intermediary step?
Just speaking out loud here, so it may be total non-sense...
This is a subject of intense debate amongst the DNS literati: CON: 1. Failure to pay attention to your setup could cause you to have a stale root zone. PRO: 1. Faster local resolution for your users, especially for malformed queries. 2. No spurious traffic will be sent from your network to the roots 3. Greater resilience to any potential root server failure/DDoS Personally I've been doing it for years, never had a problem. On larger sites where I have a lot of resolvers I make the hidden master a slave for the root zone, and also allow the local resolvers to slave it from the hidden master to be more net.friendly. For BIND, make sure you include "notify no;" in your zone{} statement. hth, Doug -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/
I don't think that the Egyptian shutdown of domain names had much effect; that's why the bgp prefixes were withdrawn. What was effective was the withdrawal of BGP prefixes. http://www.renesys.com/blog/2011/01/egypt-leaves-the-internet.shtml notes, for example, that routes *through* Egypt were operational, but routes through the same fiber and the same routers *to* Egypt were non-functional. https://labs.ripe.net/Members/akvadrako/live_eqyptian_internet_incident_anal... pretty clearly states that "prefixes associated with Egyptian ISPs were withdrawn". On Feb 16, 2011, at 11:50 AM, Franck Martin wrote:
----- Original Message -----
From: "Martin Millnert" <millnert@gmail.com> To: "Marshall Eubanks" <tme@americafree.tv> Cc: "North American Network Operators Group" <nanog@nanog.org> Sent: Thursday, 17 February, 2011 8:28:22 AM Subject: Re: NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet On Wed, Feb 16, 2011 at 9:09 AM, Marshall Eubanks <tme@americafree.tv> wrote:
On Feb 16, 2011, at 12:15 AM, Joly MacFie wrote:
"
Operating local IRC networks is good, as is having local OS mirrors, such as Debian/Ubuntu and let's not forget, having a resilient DNS configuration (root zone copy hint 101: "dig @k.root-servers.net. . axfr"). A securely distributed
Would it make sense for an ISP to "store" the root zone on their DNS servers instead of letting it be refreshed by the DNS cache? A cron job could refresh it from time to time. It would avoid entries from expiring and would always serve to clients entries with max ttl?
A root server would be better, but that could be an intermediary step?
Just speaking out loud here, so it may be total non-sense...
On 2/16/11 4:25 PM, Fred Baker wrote:
I don't think that the Egyptian shutdown of domain names had much effect ...
ditto. i'm not aware of any actions by the .eg registry operator, though i'll ask, coincidental to the prefix withdrawal. i suppose in the interests of completeness i should also ask about the (wicked recent) (مصر.) IDN ccTLD. these are both wicked small zones, relative to the density of names registered (registries other than .eg) by egyptian residents, and the larger number of network using egyptians. it is possible that as a preliminary step, the recursive resolver operators of each of the subsequently prefix withdrawing providers modified their cached data to provide policy-based resolution. -e
On Feb 16, 2011, at 4:25 13PM, Fred Baker wrote:
I don't think that the Egyptian shutdown of domain names had much effect; that's why the bgp prefixes were withdrawn. What was effective was the withdrawal of BGP prefixes.
Per the NYT article, the issue was the Egyptian "Intranet" -- people couldn't contact other sites within Egypt by host name, even though the routes were up, because they couldn't resolve .eg, .com, etc. --Steve Bellovin, http://www.cs.columbia.edu/~smb
On 2/16/11 6:10 PM, Steven Bellovin wrote:
On Feb 16, 2011, at 4:25 13PM, Fred Baker wrote:
I don't think that the Egyptian shutdown of domain names had much effect; that's why the bgp prefixes were withdrawn. What was effective was the withdrawal of BGP prefixes.
Per the NYT article, the issue was the Egyptian "Intranet" -- people couldn't contact other sites within Egypt by host name, even though the routes were up, because they couldn't resolve .eg, .com, etc.
i'll have to check if the .eg servers were ever taken off-line. resolution of .com (beyond local caches) would have been pointless post-prefix withdrawal, but if the claim is that local ix routing remained possible, so non-cached .eg resolution was successful from outside of the eun to any other egyptian provider net, then if there is data to support the claim, it will be interesting. i'll ask. -e
ah On Feb 16, 2011, at 3:10 PM, Steven Bellovin wrote:
On Feb 16, 2011, at 4:25 13PM, Fred Baker wrote:
I don't think that the Egyptian shutdown of domain names had much effect; that's why the bgp prefixes were withdrawn. What was effective was the withdrawal of BGP prefixes.
Per the NYT article, the issue was the Egyptian "Intranet" -- people couldn't contact other sites within Egypt by host name, even though the routes were up, because they couldn't resolve .eg, .com, etc.
--Steve Bellovin, http://www.cs.columbia.edu/~smb
On Feb 16, 2011, at 3:10 PM, Steven Bellovin wrote:
On Feb 16, 2011, at 4:25 13PM, Fred Baker wrote:
I don't think that the Egyptian shutdown of domain names had much effect; that's why the bgp prefixes were withdrawn. What was effective was the withdrawal of BGP prefixes.
Per the NYT article, the issue was the Egyptian "Intranet" -- people couldn't contact other sites within Egypt by host name, even though the routes were up, because they couldn't resolve .eg, .com, etc.
This is interesting, in that according to http://www.root-servers.org Cairo has two root servers (F and J). The presence of a Verisign-operated J Root leads me to assume there are probably also local .com and .net servers. One of the three name servers for .EG looks like it could plausibly be in Cairo (IP address space registered to an Egyptian postal address, 100 ms response time from London). If DNS look-ups at that level didn't work, it seems likely that there was some disruption of internal connectivity as well. Or, it may be that "the Internet" still mostly means foreign services. Being able to look up the addresses of Facebook's name servers isn't the same as being able to access Facebook. The Times article was a bit short of specifics on that, and I haven't seen other information on what it looked like internally. There's something important to keep in mind in cases like this, though. Having redundancy and local copies of things is very good for protecting against accidental disruptions or disruptions of services in other jurisdictions. Protecting things that local guys with guns want to have go away is a somewhat different story. It seems likely that if "the Internet" had still been working after the things the government did to shut it down, the government would have done more. If somebody had managed to put all the pieces together and provide wide access to content the government wanted gone, they would probably have been told to stop. I'm a bit skeptical that having more local copies of things would have helped much. -Steve
----- Original Message -----
From: "Randy Bush" <randy@psg.com> To: "Fred Baker" <fred@cisco.com> Cc: "Franck Martin" <franck@genius.com>, "North American Network Operators Group" <nanog@nanog.org> Sent: Thursday, 17 February, 2011 2:37:02 PM Subject: Re: Local root zone (Was NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet)
I don't think that the Egyptian shutdown of domain names had much effect
what shutdown of egyptian domain names?
randy, who has a server which serves them
The ASCII one .eg or the UTF8 one .xn--wgbh1c? xn--wgbh1c. 172800 IN NS ns1.dotmasr.eg. xn--wgbh1c. 172800 IN NS ns2.dotmasr.eg. xn--wgbh1c. 172800 IN NS ns3.dotmasr.eg. eg. 172800 IN NS ns5.univie.ac.at. eg. 172800 IN NS rip.psg.com. eg. 172800 IN NS frcu.eun.eg.
On 2/16/11 5:37 PM, Randy Bush wrote:
I don't think that the Egyptian shutdown of domain names had much effect
what shutdown of egyptian domain names?
randy, who has a server which serves them
there's an interesting point to be made about the geographic administrative and political distribution of secondaries being essential to insuring their survivability. Oddly your name is on bcp 16.
I don't think that the Egyptian shutdown of domain names had much effect what shutdown of egyptian domain names? randy, who has a server which serves them there's an interesting point to be made about the geographic administrative and political distribution of secondaries being essential to insuring their survivability. Oddly your name is on bcp 16.
clearly a forgery randy
It also seems like a question that could be decided empirically. Can anyone on here comment on whether or not the BGP session ended gracefully and the link lights remained lit? --Richard On Wed, Feb 16, 2011 at 9:09 AM, Marshall Eubanks <tme@americafree.tv> wrote:
On Feb 16, 2011, at 12:15 AM, Joly MacFie wrote:
http://www.nytimes.com/2011/02/16/technology/16internet.html
There has been intense debate both inside and outside Egypt on whether the
cutoff at 26 Ramses Street was accomplished by surgically tampering with the software mechanism that defines how networks at the core of the Internet communicate with one another, or by a blunt approach: simply cutting off the power to the router computers that connect Egypt to the outside world.
I do remember some intense debate, here and elsewhere, but I somehow don't remember those as being the primary debate parameters.
Regards Marshall
-- --------------------------------------------------------------- Joly MacFie 218 565 9365 Skype:punkcast WWWhatsup NYC - http://wwwhatsup.com http://pinstand.com - http://punkcast.com VP (Admin) - ISOC-NY - http://isoc-ny.org ---------------------------------------------------------------
Never mind, Messrs. Cowie and Baker answered my question: <http://mailman.nanog.org/pipermail/nanog/2011-February/033181.html> Couldn't have paths through Egypt if layer 2 were cut off. (Right?) --Richard On Wed, Feb 16, 2011 at 5:38 PM, Richard Barnes <richard.barnes@gmail.com> wrote:
It also seems like a question that could be decided empirically. Can anyone on here comment on whether or not the BGP session ended gracefully and the link lights remained lit?
--Richard
On Wed, Feb 16, 2011 at 9:09 AM, Marshall Eubanks <tme@americafree.tv> wrote:
On Feb 16, 2011, at 12:15 AM, Joly MacFie wrote:
http://www.nytimes.com/2011/02/16/technology/16internet.html
There has been intense debate both inside and outside Egypt on whether the
cutoff at 26 Ramses Street was accomplished by surgically tampering with the software mechanism that defines how networks at the core of the Internet communicate with one another, or by a blunt approach: simply cutting off the power to the router computers that connect Egypt to the outside world.
I do remember some intense debate, here and elsewhere, but I somehow don't remember those as being the primary debate parameters.
Regards Marshall
-- --------------------------------------------------------------- Joly MacFie 218 565 9365 Skype:punkcast WWWhatsup NYC - http://wwwhatsup.com http://pinstand.com - http://punkcast.com VP (Admin) - ISOC-NY - http://isoc-ny.org ---------------------------------------------------------------
No the BGP and the physical links were down. On Thu, Feb 17, 2011 at 12:38 AM, Richard Barnes <richard.barnes@gmail.com>wrote:
It also seems like a question that could be decided empirically. Can anyone on here comment on whether or not the BGP session ended gracefully and the link lights remained lit?
--Richard
On Wed, Feb 16, 2011 at 9:09 AM, Marshall Eubanks <tme@americafree.tv> wrote:
On Feb 16, 2011, at 12:15 AM, Joly MacFie wrote:
http://www.nytimes.com/2011/02/16/technology/16internet.html
There has been intense debate both inside and outside Egypt on whether
the
cutoff at 26 Ramses Street was accomplished by surgically tampering with the software mechanism that defines how networks at the core of the Internet communicate with one another, or by a blunt approach: simply cutting off the power to the router computers that connect Egypt to the outside world.
I do remember some intense debate, here and elsewhere, but I somehow don't remember those as being the primary debate parameters.
Regards Marshall
-- --------------------------------------------------------------- Joly MacFie 218 565 9365 Skype:punkcast WWWhatsup NYC - http://wwwhatsup.com http://pinstand.com - http://punkcast.com VP (Admin) - ISOC-NY - http://isoc-ny.org ---------------------------------------------------------------
-- Best Regards, Mounir Mohamed, CCIE#19573 (R&S/SP), JNCIS-M. Senior Network Engineer, Core Team. NOOR Data Networks, SAE Mobile# +2-010-2345-956 http://mounirmohamed.wordpress.com http://www.linkedin.com/in/mounirmohamed
Yes all sessions were operational during that period. On Thu, Feb 17, 2011 at 2:17 AM, Martin Millnert <millnert@gmail.com> wrote:
Mounir,
On Wed, Feb 16, 2011 at 6:58 PM, Mounir Mohamed <mounir.mohamed@gmail.com> wrote:
No the BGP and the physical links were down.
did you have any domestic BGP sessions up?
Regards, Martin
No the BGP and the physical links were down.
What about non-Internet layer 2 links? A number of companies have private IP networks extending into Egypt providing MPLS or other VPN services. In addition, there are often longlines into the Gulf states to provide the Egyptian sites with redundancy. Were these communications also cut? One way to find out would be to talk to the networking folks at any major international consumer brand that is in Egypt. I would expect that nowadays if a Coke or a Pepsi is in a country, they will have some kind of IP VPN crossing that country's borders. --Michael Dillon http://www.linkedin.com/profile/view?id=13566587
participants (14)
-
Doug Barton -
Eric Brunner-Williams -
Franck Martin -
Fred Baker -
Joel Jaeggli -
Joly MacFie -
Marshall Eubanks -
Martin Millnert -
Michael Dillon -
Mounir Mohamed -
Randy Bush -
Richard Barnes -
Steve Gibbard -
Steven Bellovin