SaidCom disconnected by Level 3 (former Telcove property)
http://www.phillyburbs.com/pb-dyn/articlePrint.cfm?id=1310151 Is this a normal thing for Level 3 to do, cut off small, responsive providers? Frank
On Wed, 14 Mar 2007, Frank Bulk wrote:
http://www.phillyburbs.com/pb-dyn/articlePrint.cfm?id=1310151
Is this a normal thing for Level 3 to do, cut off small, responsive providers?
I heard from a few folks working for TelCove that they were under order to do whatever it took to disconnect customers under certian levels in my local area (not philly). Related? Dunno. Level3 recently stiffed us on a colo contract though, 3 weeks past the "no later than" install date and we finally canceled it. (mind you that was 7 weeks after the contract was signed). ~shrug~ ..d --- david raistrick http://www.netmeister.org/news/learn2quote.html drais@atlasta.net http://www.expita.com/nomime.html
On 3/14/07, david raistrick <drais@atlasta.net> wrote:
I heard from a few folks working for TelCove that they were under order to do whatever it took to disconnect customers under certian levels in my local area (not philly).
Related? Dunno.
Level3 recently stiffed us on a colo contract though, 3 weeks past the "no later than" install date and we finally canceled it. (mind you that was 7 weeks after the contract was signed).
As a former Broadwing customer (now a Level3 customer), things like this worry me a bit... (and tend to keep my up at night). -brandon
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Frank Bulk wrote:
http://www.phillyburbs.com/pb-dyn/articlePrint.cfm?id=1310151
Is this a normal thing for Level 3 to do, cut off small, responsive providers?
Frank
Just curious, should "small responsive providers" should be multi-homed? regards, /virendra -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF+XOApbZvCIJx1bcRAtkwAJ9vNak3F8FlCf9VDycf6IlAr445nACg59kB w2OWAGdchd2XQyxxgZWQaug= =Yb1+ -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 virendra rode // wrote:
Frank Bulk wrote:
http://www.phillyburbs.com/pb-dyn/articlePrint.cfm?id=1310151
Is this a normal thing for Level 3 to do, cut off small, responsive providers?
Frank
Just curious, should "small responsive providers" should be multi-homed?
regards, /virendra
Sorry my keyboard came before coffee. I meant to say, shouldn't "small responsive providers" be multi-homed? regards, /virendra -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF+YL3pbZvCIJx1bcRApL+AKCzWJWtAu2L8zVtjUTOMSzELpTMBQCgyomG YdP85Tq6jO77NM5YTsnFQ/Y= =UR8g -----END PGP SIGNATURE-----
Almost ALL providers should be multihomed. --Mike -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of virendra rode // Sent: Thursday, March 15, 2007 11:26 AM To: NANOG Subject: Re: SaidCom disconnected by Level 3 (former Telcove property) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Frank Bulk wrote:
http://www.phillyburbs.com/pb-dyn/articlePrint.cfm?id=1310151
Is this a normal thing for Level 3 to do, cut off small, responsive providers?
Frank
Just curious, should "small responsive providers" should be multi-homed? regards, /virendra -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF+XOApbZvCIJx1bcRAtkwAJ9vNak3F8FlCf9VDycf6IlAr445nACg59kB w2OWAGdchd2XQyxxgZWQaug= =Yb1+ -----END PGP SIGNATURE-----
Almost ALL? Any company, or any person for that matter, that relies on their Internet connectivity for their lively hood should be multihomed. -wil On Mar 16, 2007, at 4:42 PM, Mike Hammett wrote:
Almost ALL providers should be multihomed.
--Mike
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of virendra rode // Sent: Thursday, March 15, 2007 11:26 AM To: NANOG Subject: Re: SaidCom disconnected by Level 3 (former Telcove property)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Frank Bulk wrote:
http://www.phillyburbs.com/pb-dyn/articlePrint.cfm?id=1310151
Is this a normal thing for Level 3 to do, cut off small, responsive providers?
Frank
Just curious, should "small responsive providers" should be multi- homed?
regards, /virendra
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFF+XOApbZvCIJx1bcRAtkwAJ9vNak3F8FlCf9VDycf6IlAr445nACg59kB w2OWAGdchd2XQyxxgZWQaug= =Yb1+ -----END PGP SIGNATURE-----
Some locations are just too cost prohibitive to multihome, but that really is a select few. Few places are out of the reach of a couple wireless hops back to civilization. --Mike -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Wil Schultz Sent: Friday, March 16, 2007 6:56 PM To: nanog@merit.edu Subject: Re: SaidCom disconnected by Level 3 (former Telcove property) Almost ALL? Any company, or any person for that matter, that relies on their Internet connectivity for their lively hood should be multihomed. -wil On Mar 16, 2007, at 4:42 PM, Mike Hammett wrote:
Almost ALL providers should be multihomed.
--Mike
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of virendra rode // Sent: Thursday, March 15, 2007 11:26 AM To: NANOG Subject: Re: SaidCom disconnected by Level 3 (former Telcove property)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Frank Bulk wrote:
http://www.phillyburbs.com/pb-dyn/articlePrint.cfm?id=1310151
Is this a normal thing for Level 3 to do, cut off small, responsive providers?
Frank
Just curious, should "small responsive providers" should be multi- homed?
regards, /virendra
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFF+XOApbZvCIJx1bcRAtkwAJ9vNak3F8FlCf9VDycf6IlAr445nACg59kB w2OWAGdchd2XQyxxgZWQaug= =Yb1+ -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 16, 2007, at 6:59 PM, Mike Hammett wrote:
Some locations are just too cost prohibitive to multihome, but that really is a select few.
It isn't just cost but can be path diversity (or lack thereof). We used to be headquartered 210 miles from civilization. We had a choice of providers and could have "multihomed". However, the only realistic way for any of those providers to get to us would have been Bell frame relay. Since by far the most likely point of failure was the "last mile" (which was 210 miles), we made a decision that actually multihoming wasn't a good use of resources. We instead went with a good quality regional provider who was themselves multihomed. Now clearly there were cases where that wouldn't have any good but given the remoteness it just seems most likely that anything that took out one provider would have taken the other one as well. Now this case we are discussing is probably the exception to our assumptions but we had a much better provider at the time than Level3 ;-] From the sounds of the original post I wouldn't be too surprised if it was also fairly remote. Chris -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iD8DBQFF+zOWElUlCLUT2d0RAtwLAJ9esOECOSbeXOpPhPbEL3A9vmbJ5wCfWgnU Dd4lEmIoaMtPCRU9WXJRSVo= =wxdX -----END PGP SIGNATURE-----
On Fri, 16 Mar 2007, Joe Abley wrote:
Almost ALL?
Surely all those except those who are competing with you for the same customers should multi-home. :-)
True :) I'd also think (read: hope) if an organization was located in an area where multi-homing was not possible, then that organization and its customers would not be doing things that are mission critical, i.e. business stops if there is no Internet connectivity. jms
On 3/16/07, Justin M. Streiner <streiner@cluebyfour.org> wrote:
On Fri, 16 Mar 2007, Joe Abley wrote:
Almost ALL?
Surely all those except those who are competing with you for the same customers should multi-home. :-)
True :) I'd also think (read: hope) if an organization was located in an area where multi-homing was not possible, then that organization and its customers would not be doing things that are mission critical, i.e. business stops if there is no Internet connectivity.
"Mission critical" seems to be quite subjective these days. -brandon
On Sat, 17 Mar 2007, Brandon Galbraith wrote:
True :) I'd also think (read: hope) if an organization was located in an area where multi-homing was not possible, then that organization and its customers would not be doing things that are mission critical, i.e. business stops if there is no Internet connectivity.
"Mission critical" seems to be quite subjective these days.
Sure. That's why I qualified my remarks. jms
All companies in all industries have a policy of stopping to provide their services if a customer stops paying or violates the contract, I really don't see this as a big/little provider argument. Yes, the small provider should be multi-homed, otherwise a fiber cut or outage can have this same effect. To me the only part of this that is up for argument is did SaidCom actually violate the contract and/or terms of use, and I certainly don't have enough information beyond that one article to make that decision. If someone else does please share with the group. -Scott (As always just my 2 cents) -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Frank Bulk Sent: Wednesday, March 14, 2007 10:30 PM To: nanog@merit.edu Subject: SaidCom disconnected by Level 3 (former Telcove property) http://www.phillyburbs.com/pb-dyn/articlePrint.cfm?id=1310151 Is this a normal thing for Level 3 to do, cut off small, responsive providers? Frank
On Thu, Mar 15, 2007 at 11:39:49AM -0700, Berkman, Scott wrote:
To me the only part of this that is up for argument is did SaidCom actually violate the contract and/or terms of use, and I certainly don't have enough information beyond that one article to make that decision. If someone else does please share with the group.
I will say that I've used Level(3) as a transit provider (not the only one, of course) at 2 different companies (both of which were / are pretty responsive to abuse complaints, in one case because I was the one responding to them), and I have never had an issue like this. However, both of these were content / hosting type setups, not ISPs who provide local Internet access, so the type of complaints (and probably the volume) were different than your typical ISP. I don't think there's any way to know for sure who screwed up here... maybe the small provider wasn't as responsive as they say they are, but are trying to shift the blame to (3). Or maybe (3) was trigger-happy. I don't think anyone from the outside has the information to tell for sure which happened. But Level(3) is a pretty big provider, and if they were just shutting off small competitors willy-nilly, I imagine we'd be hearing about it more. w
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Frank Bulk Sent: Wednesday, March 14, 2007 10:30 PM To: nanog@merit.edu Subject: SaidCom disconnected by Level 3 (former Telcove property)
http://www.phillyburbs.com/pb-dyn/articlePrint.cfm?id=1310151
Is this a normal thing for Level 3 to do, cut off small, responsive providers?
Frank
Not knowing anything about the case other than what I read in the article, my hang up is that a transit provider can make a phone call and destroy a customer's business with 30 minutes notice. On a DS3 that has actual real lead time to replace, that's a business killer. The argument of "should be multi homed" holds some water, but I've never considered multihoming as a typical remedy for a 30-90 day outage. And then it only works if lines are underutilized to the point of loosing one will constantly have zero affect on network performance, even during peak use, and if there's still some level of extra redundancy remaining. (Multiple contingency situations aside) My opinion is probably somewhat influenced by the fact that I'm a small ISP with customers that want the internet to NOT be slow, facing that same DS3 lead time problem. I ordered a DS3 in early December, (who's local loop was to ride on a preexisting OC3, sounds easy, right?) and with dates slipping over and over again, and with no firm install date in site provided from the company last week, I had to finally cancel and order with a different company last week. For the last month, the last thing I wanted to do was "punt" and start the process over again, but at some point, one starts to feel "choiceless." "Do you think I placed that order in December just for fun?" I see talk over and over again on NANOG about "Maybe some provider will come in with [insert new technology here] and compete with the cable/DSL providers" but as a small provider doing fix broadband wireless, I just don't see how even an army of small providers can compete against the likes of TENS OF BILLIONS OF DOLLARS of cable/telco market capitalization. After fighting Qwest for ten years, maybe I'm starting to feel a little hopeless.
On Wed, 14 Mar 2007, Frank Bulk wrote:
http://www.phillyburbs.com/pb-dyn/articlePrint.cfm?id=1310151
Is this a normal thing for Level 3 to do, cut off small, responsive providers?
Even from that one-sided account, I have serious problems with: Siwert said the Colorado-based Level 3 "cited several Internet abuses" by SaidCom customers as the reason for the shutdown, including spam problems. "Some customers abuse the system", but when that happens, SaidCom contacts the authorities, said Siwert. When we have a customer spamming, we don't call the police. We either talk to, ACL, or shut off the customer. The above suggests to me that SaidCom had spam issues that they were either unable or unwilling to remedy themselves. I also doubt that L3 shut them off without multiple prior warnings, though anything's possible. We had an issue many years ago where a leased line provider (coincidentally borged indirectly into L3) shut off all of our services with no warning at all on a Friday evening. Only after wasting some time with their NOC troubleshooting the circuits were we told we'd been shut off for non-payment. When we eventually got to the bottom of it (many hours later), it turned out they had another customer with a similar name that was way past due...and when billing told their NOC shut off "Atlantic" they shut off everything that looked like "Atlantic" even though we were two totally separate and unrelated customers. ---------------------------------------------------------------------- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On Mar 15, 2007, at 8:25 PM, Jon Lewis wrote:
On Wed, 14 Mar 2007, Frank Bulk wrote:
http://www.phillyburbs.com/pb-dyn/articlePrint.cfm?id=1310151
Is this a normal thing for Level 3 to do, cut off small, responsive providers?
Even from that one-sided account, I have serious problems with:
Siwert said the Colorado-based Level 3 "cited several Internet abuses" by SaidCom customers as the reason for the shutdown, including spam problems. "Some customers abuse the system", but when that happens, SaidCom contacts the authorities, said Siwert.
When we have a customer spamming, we don't call the police. We either talk to, ACL, or shut off the customer. The above suggests to me that SaidCom had spam issues that they were either unable or unwilling to remedy themselves.
ACL's are your friends with non-responsive customers. ;-), But maybe SaidCom did not know better. How many Abuse tickets had they received from TelCove/(3) over what time frame? I may be way off base here: Only knowing the "facts" presented in the above article, the "Abuse" complaints may have also included DMCA complaints, which, if not responded to in a timely manner, could also have resulted in liability for (3).* As per the quote above, the abuses included spam, he did not say they "were exclusively spam". *I am not a lawyer, nor do I play one on TV. Just reading between the lines here... G
On Thu, 15 Mar 2007, Jon Lewis wrote:
When we have a customer spamming, we don't call the police. We either talk to, ACL, or shut off the customer. The above suggests to me that SaidCom had spam issues that they were either unable or unwilling to remedy themselves.
Probably no one besides SaidCom and Level3 knows the whole story. If you do some more research you will find allegations, besides spam, that would involve law enforcement. I don't repeat them here because I don't know if they are true or not. Skipping the faces (or lack of facts ) in this particular case. People criticize ISPs for not shutting down customers. And people criticize ISPs when they do shut down customers. Whether it is a 10Gig link or a DSL/Cable link, do bad guys get more leeway just because they pay for more expensive circuits? Or should an ISP shutdown customers as soon as possible when they violate the AUP/TOS regardless who it is. Three warnings, six warnings, six hundred warnings, how many is too many? Would you be more careful about your computers being compromised, copyright infringement, signing up downstream customers if you knew violations of your ISP's AUP would result in disconnection? Everyone always wants the other guy's circuit terminated when something bad happens, but never wants their own circuit terminated when they screw up. How many people thank the police officer for stopping them and giving them a ticket for violating traffic rules?
--- Sean Donelan <sean@donelan.com> wrote:
How many people thank the police officer for stopping them and giving them a ticket for violating traffic rules?
I do, but perhaps I'm uncommon in this regard. Your larger point, however, is completely valid: there is a relatively normal desire to have rules enforced on other people with more zeal than one would choose for oneself. Perhaps more transparency is a tonic for this? If ToS and the AUP are more clearly written and enforced as consistently as possible, I would expect customers to be less horked off by AUP/ToS shutdowns. It does surprise me that no enterprising person/group has turned this into a salable feature: "we're the network which shuts down spammers/infected/baddies." I could imagine that there would be customers who would rather give their business to providers who are more active in this regard than less, and that would be a way for a service provider to differentiate themself from the rest of the pack. -David David Barak Need Geek Rock? Try The Franchise: http://www.listentothefranchise.com ____________________________________________________________________________________ Need Mail bonding? Go to the Yahoo! Mail Q&A for great tips from Yahoo! Answers users. http://answers.yahoo.com/dir/?link=list&sid=396546091
On Fri, Mar 16, 2007, David Barak wrote:
It does surprise me that no enterprising person/group has turned this into a salable feature: "we're the network which shuts down spammers/infected/baddies." I could imagine that there would be customers who would rather give their business to providers who are more active in this regard than less, and that would be a way for a service provider to differentiate themself from the rest of the pack.
People try that. They then get DDoS'ed. Then they stop. (Thats the people that try to do it by providing internet-based services. People who sell products probably fare slightly better.) Adrian
On Fri, 16 Mar 2007, David Barak wrote:
It does surprise me that no enterprising person/group has turned this into a salable feature: "we're the network which shuts down spammers/infected/baddies."
IMHO being the good cop has never been a mass-marketable feature, whether we're talking spam, botnets, phising, cracking attempts, whatever... -- Steve Sobol, Professional Geek ** Java/VB/VC/PHP/Perl ** Linux/*BSD/Windows Victorville, California PGP:0xE3AE35ED It's all fun and games until someone starts a bonfire in the living room.
On 3/17/07, Steve Sobol <sjsobol@justthe.net> wrote:
On Fri, 16 Mar 2007, David Barak wrote:
It does surprise me that no enterprising person/group has turned this into a salable feature: "we're the network which shuts down spammers/infected/baddies."
IMHO being the good cop has never been a mass-marketable feature, whether we're talking spam, botnets, phising, cracking attempts, whatever...
There are a few in that racket as well ... RSA Cyota for example. With some real big name customers. There's of course services like Markmonitor that go around looking for trademark violations on registered domains .. -- Suresh Ramasubramanian (ops.lists@gmail.com)
participants (20)
-
Adrian Chadd
-
Berkman, Scott
-
Brandon Galbraith
-
Chris Owen
-
David Barak
-
david raistrick
-
Frank Bulk
-
Gerry Boudreaux
-
Howard C. Berkowitz
-
Jerry Pasker
-
Joe Abley
-
Jon Lewis
-
Justin M. Streiner
-
Mike Hammett
-
Sean Donelan
-
Steve Sobol
-
Suresh Ramasubramanian
-
virendra rode //
-
Wil Schultz
-
William Yardley