We just lost Cogent across the country, along with several sister companies. Can't get through to a support person. Any idea what's going on? Joe Johnson Chief Information Officer Riverside Consulting Group, Ltd. Phone: 708.442.6033 x3456 Fax: 708.442.9722 joe@riversidecg.com www.riversidecg.com
Hello, Our 100Mb Metro-E link is up and running just fine... We are fed out of their San Francisco, CA facility. However, this is posted on the Cogent Status page: ****************************************************************************** ** Cogent Network Status Report Last Updated Thu Jan 14 10:38:30 2010 ** ****************************************************************************** ****************************************************************************** *Network Status:* Warning *DNS Servers Status:* Normal *Dialup/IPASS Status: *Normal *Mail Servers Status:* Normal *Webservers Status: *Normal *Cogent Network Status/DNS Server Status Description: * Welcome to Cogent Communications' Network Status Message. Customers with traffic going through or on the east coast may be experiencing packet loss and higher than normal latency due to a fiber cut in New Jersey. There is no ETR at this time. The Cogent ticket to reference for this issue is HD2082680. Sincerely, Bobby Glover Director of Information Services South Valley Internet On 1/14/2010 9:31 AM, Joe Johnson wrote:
We just lost Cogent across the country, along with several sister companies. Can't get through to a support person. Any idea what's going on?
Joe Johnson Chief Information Officer Riverside Consulting Group, Ltd. Phone: 708.442.6033 x3456 Fax: 708.442.9722 joe@riversidecg.com www.riversidecg.com
There is a fiber cut and it's located near Brunswick, NJ -- Raphaël Maunier NEO TELECOMS On Jan 14, 2010, at 6:36 PM, Robert Glover wrote:
Hello,
Our 100Mb Metro-E link is up and running just fine... We are fed out of their San Francisco, CA facility.
However, this is posted on the Cogent Status page:
****************************************************************************** ** Cogent Network Status Report Last Updated Thu Jan 14 10:38:30 2010 ** ****************************************************************************** ****************************************************************************** *Network Status:* Warning *DNS Servers Status:* Normal *Dialup/IPASS Status: *Normal *Mail Servers Status:* Normal *Webservers Status: *Normal
*Cogent Network Status/DNS Server Status Description: * Welcome to Cogent Communications' Network Status Message. Customers with traffic going through or on the east coast may be experiencing packet loss and higher than normal latency due to a fiber cut in New Jersey. There is no ETR at this time. The Cogent ticket to reference for this issue is HD2082680.
Sincerely, Bobby Glover Director of Information Services South Valley Internet
On 1/14/2010 9:31 AM, Joe Johnson wrote:
We just lost Cogent across the country, along with several sister companies. Can't get through to a support person. Any idea what's going on?
Joe Johnson Chief Information Officer Riverside Consulting Group, Ltd. Phone: 708.442.6033 x3456 Fax: 708.442.9722 joe@riversidecg.com www.riversidecg.com
Lost connectivity to a few things....Namely google. Kansas City ~J -----Original Message----- From: Robert Glover [mailto:robertg@garlic.com] Sent: Thursday, January 14, 2010 11:36 AM To: Joe Johnson Cc: nanog@nanog.org Subject: Re: Cogent Outage? Hello, Our 100Mb Metro-E link is up and running just fine... We are fed out of their San Francisco, CA facility. However, this is posted on the Cogent Status page: ****************************************************************************** ** Cogent Network Status Report Last Updated Thu Jan 14 10:38:30 2010 ** ****************************************************************************** ****************************************************************************** *Network Status:* Warning *DNS Servers Status:* Normal *Dialup/IPASS Status: *Normal *Mail Servers Status:* Normal *Webservers Status: *Normal *Cogent Network Status/DNS Server Status Description: * Welcome to Cogent Communications' Network Status Message. Customers with traffic going through or on the east coast may be experiencing packet loss and higher than normal latency due to a fiber cut in New Jersey. There is no ETR at this time. The Cogent ticket to reference for this issue is HD2082680. Sincerely, Bobby Glover Director of Information Services South Valley Internet On 1/14/2010 9:31 AM, Joe Johnson wrote:
We just lost Cogent across the country, along with several sister companies. Can't get through to a support person. Any idea what's going on?
Joe Johnson Chief Information Officer Riverside Consulting Group, Ltd. Phone: 708.442.6033 x3456 Fax: 708.442.9722 joe@riversidecg.com www.riversidecg.com
Sitting on hold with them now. We lost them completely in Kansas City for about 5 minutes. We're back but connectivity through them is spotty. Can't even resolve google.com. Same with other DCs in the area. -----Original Message----- From: Joe Johnson [mailto:joe@riversidecg.com] Sent: Thursday, January 14, 2010 11:31 AM To: nanog@nanog.org Subject: Cogent Outage? We just lost Cogent across the country, along with several sister companies. Can't get through to a support person. Any idea what's going on? Joe Johnson Chief Information Officer Riverside Consulting Group, Ltd. Phone: 708.442.6033 x3456 Fax: 708.442.9722 joe@riversidecg.com www.riversidecg.com No virus found in this incoming message. Checked by AVG - www.avg.com Version: 9.0.725 / Virus Database: 270.14.140/2621 - Release Date: 01/14/10 06:39:00
Hello, Internet Pulse shows some bad-mojo right now: http://www.internetpulse.com/Main.aspx?OriginValue=Cogent&OriginLevel=1 Sincerely, Bobby Glover Director of Information Services South Valley Internet On 1/14/2010 9:36 AM, Aaron Wendel wrote:
Sitting on hold with them now. We lost them completely in Kansas City for about 5 minutes. We're back but connectivity through them is spotty. Can't even resolve google.com. Same with other DCs in the area.
-----Original Message----- From: Joe Johnson [mailto:joe@riversidecg.com] Sent: Thursday, January 14, 2010 11:31 AM To: nanog@nanog.org Subject: Cogent Outage?
We just lost Cogent across the country, along with several sister companies. Can't get through to a support person. Any idea what's going on?
Joe Johnson Chief Information Officer Riverside Consulting Group, Ltd. Phone: 708.442.6033 x3456 Fax: 708.442.9722 joe@riversidecg.com www.riversidecg.com
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 9.0.725 / Virus Database: 270.14.140/2621 - Release Date: 01/14/10 06:39:00
Yes there is a Newyork to Philadelphia fiber cut is there It might not be an outage it might be high latency due to multiple routes going out via there buffalo POP. Ketan Mangal 2 Bloor St W, Ste 2004 Toronto, Ontario M4W 3E2 Toll Free: 1.866.353.9333 Fax: 416.855.7636 Client Services: 416.921.7000 Technical Support: 416.921.2221 Client Services e-mail: care@epiknetworks.com www.epiknetworks.com Toronto . Vancouver . Atlanta . Los Angeles -----Original Message----- From: Joe Johnson [mailto:joe@riversidecg.com] Sent: Thursday, January 14, 2010 12:31 PM To: nanog@nanog.org Subject: Cogent Outage? We just lost Cogent across the country, along with several sister companies. Can't get through to a support person. Any idea what's going on? Joe Johnson Chief Information Officer Riverside Consulting Group, Ltd. Phone: 708.442.6033 x3456 Fax: 708.442.9722 joe@riversidecg.com www.riversidecg.com No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.432 / Virus Database: 270.14.138/2618 - Release Date: 01/14/10 07:35:00
Ketan Mangal wrote:
Yes there is a Newyork to Philadelphia fiber cut is there It might not be an outage it might be high latency due to multiple routes going out via there buffalo POP.
That fiber cut was at 9:30EST this morning, the major Cogent internal routing problems started around 12:10 and lasted about 30 mins. - Kevin
On Jan 14, 2010, at 1:41 PM, Kevin Loch wrote:
Ketan Mangal wrote:
Yes there is a Newyork to Philadelphia fiber cut is there It might not be an outage it might be high latency due to multiple routes going out via there buffalo POP.
That fiber cut was at 9:30EST this morning, the major Cogent internal routing problems started around 12:10 and lasted about 30 mins.
Isn't this what the outages list is for?
On Jan 14, 2010, at 1:53 PM, John Payne wrote:
On Jan 14, 2010, at 1:41 PM, Kevin Loch wrote:
Ketan Mangal wrote:
Yes there is a Newyork to Philadelphia fiber cut is there It might not be an outage it might be high latency due to multiple routes going out via there buffalo POP.
That fiber cut was at 9:30EST this morning, the major Cogent internal routing problems started around 12:10 and lasted about 30 mins.
Isn't this what the outages list is for?
Indeed. Please keep NANOG clear for inane pontification and religious wars regarding firewalls, for non-lawyers arguing the finer points of the law, and for most anything else that is non-operational in nature.
-----Original Message----- From: Daniel Senie [mailto:dts@senie.com] Sent: Thursday, January 14, 2010 1:16 PM To: NANOG list Subject: Re: Cogent Outage?
<SNIP>
Indeed. Please keep NANOG clear for inane pontification and religious wars regarding firewalls, for non-lawyers arguing the finer points of the law, and for most anything else that is non-operational in nature.
Daniel, I may not understand the NANOG list charter, but I think operational issues would not be just YOUR operational issues. I would agree that not everything discussed here is operational for you. :) - Brian CONFIDENTIALITY NOTICE: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, copying, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Thank you.
On 1/14/2010 2:00 PM, Burns, Kevin wrote:
Unsubscribe
No! I just re-subscribed for the fist time in years. Who are you to tell me to unsubscribe? -- "Government big enough to supply everything you need is big enough to take everything you have." Remember: The Ark was built by amateurs, the Titanic by professionals. Requiescas in pace o email Ex turpi causa non oritur actio Eppure si rinfresca ICBM Targeting Information: http://tinyurl.com/4sqczs http://tinyurl.com/7tp8ml
Hello: As many of you are aware right now, Haiti is in Chaos. This website has been put up to provide a central repository where relatives from the States can register information about their loved ones. Looking at a report that 1And1 has turned-off the site because it is suspected as a "spam" site, due to the influx of posts. Please, this site is critical in this very fluid, and horrible situation. If anyone can help, please get 1And1 to take action here. Here is a post regarding the issue: "Koneksyon.com will be right back We are sorry but koneksyon.com will be right back. We are experiencing some technical difficulties and we cannot get in touch with 1and1.com our hosting company. It seems that our website has been reported for spamming when all we are trying to do is help people find information about their loved ones in Haiti. Published on January 14, 2010 10:43 pm. Filed under: News Tags: 1and1, haiti, help, hosting, koneksyon.com" This message is intended only for the personal and confidential use of the recipients named above. If the reader of this email is not the intended recipient, you have received this email in error and any review, dissemination, distribution or copying is strictly prohibited. If you have received this email in error, please notify the sender immediately by return email and permanently delete the copy you received. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. Wolverine is not responsible for any recommendation, solicitation, offer or agreement or any information about any transaction, customer account or account activity that may be attached to or contained in this communication. Wolverine accepts no liability for any content contained in the email, or any errors or omissions arising as a result of email transmission. Any opinions contained in this email constitute the sender's best judgment at this time and are subject to change without notice.
No offense but let the red cross do its job. Sent via BlackBerry from T-Mobile -----Original Message----- From: "Forestal, Andre Jr." <aforestal@wolve.com> Date: Thu, 14 Jan 2010 22:09:50 To: NANOG list<nanog@nanog.org> Subject: Contact @ 1And1 hosting re: http://www.koneksyon.com/ - URGENT Hello: As many of you are aware right now, Haiti is in Chaos. This website has been put up to provide a central repository where relatives from the States can register information about their loved ones. Looking at a report that 1And1 has turned-off the site because it is suspected as a "spam" site, due to the influx of posts. Please, this site is critical in this very fluid, and horrible situation. If anyone can help, please get 1And1 to take action here. Here is a post regarding the issue: "Koneksyon.com will be right back We are sorry but koneksyon.com will be right back. We are experiencing some technical difficulties and we cannot get in touch with 1and1.com our hosting company. It seems that our website has been reported for spamming when all we are trying to do is help people find information about their loved ones in Haiti. Published on January 14, 2010 10:43 pm. Filed under: News Tags: 1and1, haiti, help, hosting, koneksyon.com" This message is intended only for the personal and confidential use of the recipients named above. If the reader of this email is not the intended recipient, you have received this email in error and any review, dissemination, distribution or copying is strictly prohibited. If you have received this email in error, please notify the sender immediately by return email and permanently delete the copy you received. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. Wolverine is not responsible for any recommendation, solicitation, offer or agreement or any information about any transaction, customer account or account activity that may be attached to or contained in this communication. Wolverine accepts no liability for any content contained in the email, or any errors or omissions arising as a result of email transmission. Any opinions contained in this email constitute the sender's best judgment at this time and are subject to change without notice.
I completely understand, however this website went up since yesterday morning, way before anything else of its kind went up, and there were already thousands of entries and folks were already relying on it as a mean to locate loved ones. It is certainly not a substitute to the registry that the Redcross has in place, but it has already helped relatives track each other. I am simply appealing to this community as a way to get through to 1And1. That's all. Thx -----Original Message----- From: chaim.rieger@gmail.com [mailto:chaim.rieger@gmail.com] Sent: Thursday, January 14, 2010 11:13 PM To: Forestal, Andre Jr.; NANOG list Subject: Re: Contact @ 1And1 hosting re: http://www.koneksyon.com/ - URGENT No offense but let the red cross do its job. Sent via BlackBerry from T-Mobile -----Original Message----- From: "Forestal, Andre Jr." <aforestal@wolve.com> Date: Thu, 14 Jan 2010 22:09:50 To: NANOG list<nanog@nanog.org> Subject: Contact @ 1And1 hosting re: http://www.koneksyon.com/ - URGENT Hello: As many of you are aware right now, Haiti is in Chaos. This website has been put up to provide a central repository where relatives from the States can register information about their loved ones. Looking at a report that 1And1 has turned-off the site because it is suspected as a "spam" site, due to the influx of posts. Please, this site is critical in this very fluid, and horrible situation. If anyone can help, please get 1And1 to take action here. Here is a post regarding the issue: "Koneksyon.com will be right back We are sorry but koneksyon.com will be right back. We are experiencing some technical difficulties and we cannot get in touch with 1and1.com our hosting company. It seems that our website has been reported for spamming when all we are trying to do is help people find information about their loved ones in Haiti. Published on January 14, 2010 10:43 pm. Filed under: News Tags: 1and1, haiti, help, hosting, koneksyon.com" This message is intended only for the personal and confidential use of the recipients named above. If the reader of this email is not the intended recipient, you have received this email in error and any review, dissemination, distribution or copying is strictly prohibited. If you have received this email in error, please notify the sender immediately by return email and permanently delete the copy you received. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. Wolverine is not responsible for any recommendation, solicitation, offer or agreement or any information about any transaction, customer account or account activity that may be attached to or contained in this communication. Wolverine accepts no liability for any content contained in the email, or any errors or omissions arising as a result of email transmission. Any opinions contained in this email constitute the sender's best judgment at this time and are subject to change without notice. This message is intended only for the personal and confidential use of the recipients named above. If the reader of this email is not the intended recipient, you have received this email in error and any review, dissemination, distribution or copying is strictly prohibited. If you have received this email in error, please notify the sender immediately by return email and permanently delete the copy you received. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. Wolverine is not responsible for any recommendation, solicitation, offer or agreement or any information about any transaction, customer account or account activity that may be attached to or contained in this communication. Wolverine accepts no liability for any content contained in the email, or any errors or omissions arising as a result of email transmission. Any opinions contained in this email constitute the sender's best judgment at this time and are subject to change without notice.
I'm not sure if it relates to this site, you didn't provide a link to the report about 1and1 actions but the FBI is very active tracking many sites and hundreds of domains that are popping up in several places after the quake and that are a scam. I'm not saying this site is legit, but just starting with the name it sounds very suspicious. I second the recommendation to support the Red Cross or other well established relief organizations. If you are willing to provide a contribution another option is http://www.unfoundation.org/. UNDP is also providing updates via youtube http://www.youtube.com/user/undp Regards Jorge On Thu, Jan 14, 2010 at 10:09 PM, Forestal, Andre Jr. <aforestal@wolve.com> wrote:
Hello:
As many of you are aware right now, Haiti is in Chaos. This website has been put up to provide a central repository where relatives from the States can register information about their loved ones.
Looking at a report that 1And1 has turned-off the site because it is suspected as a "spam" site, due to the influx of posts. Please, this site is critical in this very fluid, and horrible situation.
If anyone can help, please get 1And1 to take action here.
Here is a post regarding the issue:
"Koneksyon.com will be right back We are sorry but koneksyon.com will be right back. We are experiencing some technical difficulties and we cannot get in touch with 1and1.com our hosting company. It seems that our website has been reported for spamming when all we are trying to do is help people find information about their loved ones in Haiti.
Published on January 14, 2010 10:43 pm. Filed under: News Tags: 1and1, haiti, help, hosting, koneksyon.com"
Thanks all for the quick responses. Got what we needed. for clarification: This website does not involve any donations at all. It is simply a bulletin board application that allows US based relatives to post name, last known location of people in Haiti whom they can not get in touch with. Since there is still internet access available in Haiti via satellite, there are people in Haiti who would frequent the list and post a response if they happen to know of the whereabouts of someone listed on the site. once again thank you all. ________________________________ From: Jorge Amodio [mailto:jmamodio@gmail.com] Sent: Thu 1/14/2010 11:34 PM To: Forestal, Andre Jr. Cc: NANOG list Subject: Re: Contact @ 1And1 hosting re: http://www.koneksyon.com/ - URGENT I'm not sure if it relates to this site, you didn't provide a link to the report about 1and1 actions but the FBI is very active tracking many sites and hundreds of domains that are popping up in several places after the quake and that are a scam. I'm not saying this site is legit, but just starting with the name it sounds very suspicious. I second the recommendation to support the Red Cross or other well established relief organizations. If you are willing to provide a contribution another option is http://www.unfoundation.org/. UNDP is also providing updates via youtube http://www.youtube.com/user/undp Regards Jorge On Thu, Jan 14, 2010 at 10:09 PM, Forestal, Andre Jr. <aforestal@wolve.com> wrote:
Hello:
As many of you are aware right now, Haiti is in Chaos. This website has been put up to provide a central repository where relatives from the States can register information about their loved ones.
Looking at a report that 1And1 has turned-off the site because it is suspected as a "spam" site, due to the influx of posts. Please, this site is critical in this very fluid, and horrible situation.
If anyone can help, please get 1And1 to take action here.
Here is a post regarding the issue:
"Koneksyon.com will be right back We are sorry but koneksyon.com will be right back. We are experiencing some technical difficulties and we cannot get in touch with 1and1.com our hosting company. It seems that our website has been reported for spamming when all we are trying to do is help people find information about their loved ones in Haiti.
Published on January 14, 2010 10:43 pm. Filed under: News Tags: 1and1, haiti, help, hosting, koneksyon.com"
This message is intended only for the personal and confidential use of the recipients named above. If the reader of this email is not the intended recipient, you have received this email in error and any review, dissemination, distribution or copying is strictly prohibited. If you have received this email in error, please notify the sender immediately by return email and permanently delete the copy you received. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. Wolverine is not responsible for any recommendation, solicitation, offer or agreement or any information about any transaction, customer account or account activity that may be attached to or contained in this communication. Wolverine accepts no liability for any content contained in the email, or any errors or omissions arising as a result of email transmission. Any opinions contained in this email constitute the sender's best judgment at this time and are subject to change without notice.
Lots of these web sites popping up http://apps.facebook.com/haiti_survivors
Fiber cut in New Jersey, affecting most of the easy coast (per their support number). I didn't jot the master ticket number down though. Our gear in Chicago seems partially affected though. On Thu, Jan 14, 2010 at 11:31 AM, Joe Johnson <joe@riversidecg.com> wrote:
We just lost Cogent across the country, along with several sister companies. Can't get through to a support person. Any idea what's going on?
Joe Johnson Chief Information Officer Riverside Consulting Group, Ltd. Phone: 708.442.6033 x3456 Fax: 708.442.9722 joe@riversidecg.com www.riversidecg.com
-- Brandon Galbraith Mobile: 630.400.6992 FNAL: 630.840.2141
Seems like there may an issue in Chicago, lots of traceroutes bombing out there. -----Original Message----- From: Joe Johnson [mailto:joe@riversidecg.com] Sent: Thursday, January 14, 2010 11:33 AM To: nanog@nanog.org Subject: Cogent Outage? We just lost Cogent across the country, along with several sister companies. Can't get through to a support person. Any idea what's going on? Joe Johnson Chief Information Officer Riverside Consulting Group, Ltd. Phone: 708.442.6033 x3456 Fax: 708.442.9722 joe@riversidecg.com www.riversidecg.com
-----Original Message----- From: Joe Johnson [mailto:joe@riversidecg.com] Sent: Thursday, January 14, 2010 11:31 AM To: nanog@nanog.org Subject: Cogent Outage?
We just lost Cogent across the country, along with several sister companies. Can't get through to a support person. Any idea what's going on?
We lost access through Cogent in the Chicago area for a small number of sites. Some partners also reported problems getting to us. Cogent's updated status page includes the Jersey fiber cut and a routing problem: "Customers with traffic going through or on the east coast may be experiencing packet loss and higher than normal latency due to a fiber cut in New Jersey. Splice crews are at the site. There is no ETR at this time. The Cogent ticket to reference for this issue is HD2082680. A separate issue has been identified for customers in Toronto and Chicago. This appears to be a routing issue and the Cogent NOC is looking into the problem. There is no ETR at this time. The Cogent ticket to reference for this issue is HD2082851 "
Joe Johnson Chief Information Officer Riverside Consulting Group, Ltd. Phone: 708.442.6033 x3456 Fax: 708.442.9722 joe@riversidecg.com www.riversidecg.com
-Brian Knight Sr. Network Engineer Mizuho Securities USA Inc http://www.mizuho-sc.com/ * Please note that I do not speak for my employer - only for myself. ** The below disclaimer is standard; the reader may consider addressees, mailing list members, and archive viewers to be intended recipients. CONFIDENTIAL: This e-mail, including its contents and attachments, if any, are confidential. It is neither an offer to buy or sell, nor a solicitation of an offer to buy or sell, any securities or any related financial instruments mentioned in it. If you are not the named recipient please notify the sender and immediately delete it. You may not disseminate, distribute, or forward this e-mail message or disclose its contents to anybody else. Unless otherwise indicated, copyright and any other intellectual property rights in its contents are the sole property of Mizuho Securities USA Inc. E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. Although we routinely screen for viruses, addressees should check this e-mail and any attachments for viruses. We make no representation or warranty as to the absence of viruses in this e-mail or any attachments. Please note that to ensure regulatory compliance and for the protection of our customers and business, we may monitor and read e-mails sent to and from our server(s). #####################################################################################
Joe Johnson said:
We just lost Cogent across the country, along with several sister companies. Can't get through to a support person. Any idea what's going on?
We're back up in both Chicago locations, Troy, LA, Dallas, and NY now. Chicago, Troy, and LA were complete losses for close to 35 minutes and Dallas and NY were 50-60% packet loss, so they were practically worthless. Thanks! Joe
participants (19)
-
Aaron Wendel -
Brandon Galbraith -
Brian Johnson -
Burns, Kevin -
chaim.rieger＠gmail.com -
Daniel Senie -
Forestal, Andre Jr. -
Joe Johnson -
John Payne -
Jorge Amodio -
Justin Horstman -
Ketan Mangal -
Kevin Loch -
Knight, Brian -
Larry Sheldon -
Raphael Maunier -
Robert Glover -
Roy -
Todd Mueller