Level 3's side of the story
http://www.prnewswire.com/cgi-bin/stories.pl?ACCT=104&STORY=/www/story/10-07-2005/0004164041&EDATE= "On October 6, Level 3, as it had repeatedly advised Cogent it would, terminated free traffic exchange with Cogent. Because Internet users, apparently without notice from Cogent and through no fault of their own, have been impacted, Level 3 has, effective immediately, re-established a free connection to Cogent. In order to allow Internet users to make alternative arrangements, we will maintain this connection until 6:00 a.m. ET, November 9, 2005. The effectiveness of this arrangement of course depends on Cogent's willingness to maintain their side of the traffic exchange. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
Richard A Steenbergen wrote:
http://www.prnewswire.com/cgi-bin/stories.pl?ACCT=104&STORY=/www/story/10-07-2005/0004164041&EDATE=
"On October 6, Level 3, as it had repeatedly advised Cogent it would, terminated free traffic exchange with Cogent. Because Internet users, apparently without notice from Cogent and through no fault of their own,
I don't remember seeing this public notice from Level(3) posted.... Wouldn't that be "without notice from Level(3)"?
have been impacted, Level 3 has, effective immediately, re-established a free connection to Cogent. In order to allow Internet users to make alternative arrangements,
Splendid, that gives the world sufficient time to accept Cogent's offer of 1 year free service.
we will maintain this connection until 6:00 a.m. ET, November 9, 2005. The effectiveness of this arrangement of course depends on Cogent's willingness to maintain their side of the traffic exchange.
At which time Level(3) will have had time to purchase transit, as it will be a "tier 2" hoisted on its own petard. -- William Allen Simpson Key fingerprint = 17 40 5E 67 15 6F 31 26 DD 0D B9 9B 6A 15 2C 32
On Oct 7, 2005, at 7:13 PM, William Allen Simpson wrote:
http://www.prnewswire.com/cgi-bin/stories.pl?ACCT=104&STORY=/www/ story/10-07-2005/0004164041&EDATE= "On October 6, Level 3, as it had repeatedly advised Cogent it would, terminated free traffic exchange with Cogent. Because Internet users, apparently without notice from Cogent and through no fault of their own,
I don't remember seeing this public notice from Level(3) posted.... Wouldn't that be "without notice from Level(3)"?
Every peering agreement I have seen includes an NDA. Most certainly notified Cogent, and almost certainly had no legal right to issue a press release.
have been impacted, Level 3 has, effective immediately, re- established a free connection to Cogent. In order to allow Internet users to make alternative arrangements,
Splendid, that gives the world sufficient time to accept Cogent's offer of 1 year free service.
I note that connecting to Cogent after November 9th will still give you only partial transit to the Internet - in fact, far less of the Internet than connecting to Level 3. (Despite Cogent's claims otherwise.)
we will maintain this connection until 6:00 a.m. ET, November 9, 2005. The effectiveness of this arrangement of course depends on Cogent's willingness to maintain their side of the traffic exchange.
At which time Level(3) will have had time to purchase transit, as it will be a "tier 2" hoisted on its own petard.
Is there any particular reason you think Level 3 will buy transit? Or that they give a rat's ass what tier _you_ think they are? Perhaps Cogent will have had time to buy transit? Or perhaps - just perhaps - the Internet will be bifurcated for the time being. There is nothing intrinsically wrong with an Internet which isn't fully connected. Each business will work out its own business needs and use the transit provider(s) they see fit. I am a bit confused about why so many people think this is such a bad thing. Personally, I believe there is room in the market for partial transit providers. I guess that theory is about to be tested. -- TTFN, patrick
On Fri, Oct 07, 2005 at 11:27:09PM -0400, Patrick W. Gilmore wrote:
I am a bit confused about why so many people think this is such a bad thing. Personally, I believe there is room in the market for partial transit providers. I guess that theory is about to be tested.
Agree that there is room in the market for attactively priced partial transit providers. Now where does that leave Level 3 after Nov 9th? Possibly with a second strategy reconsidertion. Ken Leland Monmouth Telecom
On Fri, Oct 07, 2005 at 11:27:09PM -0400, Patrick W. Gilmore wrote:
Is there any particular reason you think Level 3 will buy transit? Or that they give a rat's ass what tier _you_ think they are?
Perhaps Cogent will have had time to buy transit?
Or perhaps - just perhaps - the Internet will be bifurcated for the time being. There is nothing intrinsically wrong with an Internet which isn't fully connected. Each business will work out its own business needs and use the transit provider(s) they see fit.
I am a bit confused about why so many people think this is such a bad thing. Personally, I believe there is room in the market for partial transit providers. I guess that theory is about to be tested.
My advice to these people is, if you don't like the connectivity being provided, don't buy from that provider. Closing your wallet is a lot more powerful than flapping your lips. Personally I suspect there will still be an extensive market of people who are multihomed to Cogent and someone else, and that these people will continue to buy the product even without the 2.5% of the Internet that is single homed to (3). The biggest complainers so far seem to be from the people who didn't understand what they were signing on for, and a lot of those are the Verio customers who Cogent purchased. Maybe Cogent is willing to part with those customers, maybe their contracts are coming due and they won't be renewing, or maybe the customers should have thought better before signing a contract which allowed the transfer of the contract to a completely different entity without an opportunity to cancel. Certainly these are high-margin but low-bandwidth customers, maybe with enough complaints Cogent will be willing to stick them on a smaller seperate ASN which is willing to buy transit. Or, if you really care about it that much, buy service from someone else, unplug your port, stop paying your bill, and maybe send back a nastygram which includes suggestions as to where Cogent can relocate the router card that you used to be attached to, along with helpful advice as to how it might be made to fit there. Cogent will not waste time or money on lawsuits for disgruntled ex-Verio T1 customers. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
Richard A Steenbergen wrote:
Certainly these are high-margin but low-bandwidth customers, maybe with enough complaints Cogent will be willing to stick them on a smaller seperate ASN which is willing to buy transit.
Does anyone have reachability data for c-root during this episode? I wonder if they made separate arrangements for that or are planning to make arrangements for phase 2. - Kevin
Kevin Loch <kloch@hotnic.net> writes:
Richard A Steenbergen wrote:
Certainly these are high-margin but low-bandwidth customers, maybe with enough complaints Cogent will be willing to stick them on a smaller seperate ASN which is willing to buy transit.
Does anyone have reachability data for c-root during this episode?
I wonder if they made separate arrangements for that or are planning to make arrangements for phase 2.
The nice thing about having 13 root nameservers is that it really doesn't matter whether they do or not. If it starts to become trendy to have root nameservers on intentionally-partially-connected networks, (say, maybe four of them) *then* we can start thinking about the consequences. ---rob
Kevin Loch writes:
Does anyone have reachability data for c-root during this episode?
The RIPE NCC "DNSMON" service has some: http://dnsmon.ripe.net/dns-servmon/server/plot?server=c.root-servers.net&type=drops&tstart=1128246543&tstop=1128972253 According to BGPlay for that particular prefix from Route-Views data (for some reason the RIPE RIS server used by BGPlay seems to be down at the moment), the "episode" seems to be between these times (UTC): 2005-10-05 09:49:03 Route Withdrawal ( 3356 174 2149 ) 2005-10-07 19:24:13 Route Announcement 3356 174 2149 The interval in the URL above starts 72 hours before the start of the episode and ends 72 hours after its end. I cannot see any particular problems that would coincide with the episode, from that set of probes (RIPE TTM). Because we rely on default routes to our three transit providers, and Level(3) is one of them, some of our customers must have had connectivity issues to Cogent for a few hours, until we noticed (thanks to Adam Rothschild and NANOG) and implemented a workaround. But our RIPE TTM boxes (tt85 as well as the currently broken tt86) aren't in those parts of our network.
I wonder if they made separate arrangements for that or are planning to make arrangements for phase 2.
As someone else said, partial unreachability of a particular root nameserver isn't that much of an issue. But it's an interesting question nevertheless. -- Simon.
On 16.10 16:04, Simon Leinen wrote:
Kevin Loch writes:
Does anyone have reachability data for c-root during this episode?
The RIPE NCC "DNSMON" service has some:
If there is anyone with more comprehensive data I'd like to hear about it ;-) !
According to BGPlay for that particular prefix from Route-Views data (for some reason the RIPE RIS server used by BGPlay seems to be down at the moment), the "episode" seems to be between these times (UTC):
It is up now. Of the two routes to c.root-servers.net via Level 3 in that data set one stayed up during the period and the other got healed immediateely via AS286. It flipped back later.
...
The interval in the URL above starts 72 hours before the start of the episode and ends 72 hours after its end. I cannot see any particular problems that would coincide with the episode, from that set of probes (RIPE TTM).
I agree that there is nothing really significant here. It is mostly noise. What one is looking for in these graphs is strong vertical patterns. dnsmon did not detect anything significant concerning reachability of c.root-servers.net.
As someone else said, partial unreachability of a particular root nameserver isn't that much of an issue.
Indeed.
But it's an interesting question nevertheless.
A red instance of a fish from the northern seas ? Daniel
--On October 7, 2005 7:13:45 PM -0400 William Allen Simpson <wsimpson@greendragon.com> wrote:
Richard A Steenbergen wrote:
http://www.prnewswire.com/cgi-bin/stories.pl?ACCT=104&STORY=/www/story/1 0-07-2005/0004164041&EDATE=
"On October 6, Level 3, as it had repeatedly advised Cogent it would, terminated free traffic exchange with Cogent. Because Internet users, apparently without notice from Cogent and through no fault of their own,
I don't remember seeing this public notice from Level(3) posted.... Wouldn't that be "without notice from Level(3)"?
Why is it (3)'s responsibility to handle Cogent's customers? It isn't. If that was the case we'd be required to notify all downstream's when we terminate on default of contract or other reasons (as an ISP). No I think if (3) told Cogent then they did their job. It's absurd to say that (3) was responsible for public notice of Cogent's customers or anything of that nature. That's a Cogent internal matter that they screwed up, or intentionally withheld.
And after November 9, what is then? Cogent/L3 depeering part 2? On Fri, 7 Oct 2005, Richard A Steenbergen wrote:
http://www.prnewswire.com/cgi-bin/stories.pl?ACCT=104&STORY=/www/story/10-07-2005/0004164041&EDATE=
"On October 6, Level 3, as it had repeatedly advised Cogent it would, terminated free traffic exchange with Cogent. Because Internet users, apparently without notice from Cogent and through no fault of their own, have been impacted, Level 3 has, effective immediately, re-established a free connection to Cogent. In order to allow Internet users to make alternative arrangements, we will maintain this connection until 6:00 a.m. ET, November 9, 2005. The effectiveness of this arrangement of course depends on Cogent's willingness to maintain their side of the traffic exchange.
BTW - it sounds like maybe somebody was required to give 30 days notice of service changes to certain customers with good laywers.... On Fri, 7 Oct 2005, william(at)elan.net wrote:
And after November 9, what is then? Cogent/L3 depeering part 2?
On Fri, 7 Oct 2005, Richard A Steenbergen wrote:
http://www.prnewswire.com/cgi-bin/stories.pl?ACCT=104&STORY=/www/story/10-07-2005/0004164041&EDATE=
"On October 6, Level 3, as it had repeatedly advised Cogent it would, terminated free traffic exchange with Cogent. Because Internet users, apparently without notice from Cogent and through no fault of their own, have been impacted, Level 3 has, effective immediately, re-established a free connection to Cogent. In order to allow Internet users to make alternative arrangements, we will maintain this connection until 6:00 a.m. ET, November 9, 2005. The effectiveness of this arrangement of course depends on Cogent's willingness to maintain their side of the traffic exchange.
william(at)elan.net wrote:
BTW - it sounds like maybe somebody was required to give 30 days notice of service changes to certain customers with good laywers....
This is my bet. Let's see. Peering went down October 6th. Then a fedexed letter with nasty threats arrives today [with a fax copy arriving on the 6th]. Then a letter from Level3 fedexed back on the 7th for delivery on the 9th saying "We've cured the default and here is 30 days notice that we are terminating your account or changing your service." Then 30 days from the 10th to November 9th... Yup. Sounds about right. Dave (I'm merging your comment in here too): One question [referring to customers being used as pawns] -- what does ANY network have besides their customers to use in a peering negotiation? Engineers often try to use things like prefix counts, # of routers, # of countries, blah blah blah but really these are all oblique references to the size and import of the single homed customers behind that network. Level3 thought their customers were more desirable than Cogent's. Level3's customers seem to have enough respect for Cogent's customers to make Level3 blink. I seem to remember the same thing happening with [another network] trying to depeer Cogent a few months back. What I find interesting is that in this case, Cogent apparently allowed the sessions (whichever ones Level3 decided to fire back up) to come up cleanly. I think in that earlier instance Cogent kept the peering sessions down until some point [one speculates until the legal paperwork came through] -- notice I said "I think" I have no specific facts, just a vague recollection. Taking this back to pricing (I suddenly found about 30 minutes I didn't expect to have), if Cogent terminated their most recent $6/mb/s or $8/mb/s promotion because of this concern with the Level3 peering and Level3 does indeed blink [comes back to the table and agrees to an indefinite term of SFI] Cogent might re-establish the promotion and with it grow by another 20 or 30 Gb/s or more.. possibly at the direct expense of Level3 and others and force more growth of their interconnections. Another question (wow, 3 or 4 separate threads in one email). Instead of a "hard" de-peering. Wouldn't it be more effective if Level3 just stopped adding interconnection capacity to Cogent? That is an effective way of limiting the growth of losses between two networks, and if some discreet situation can be used [say a balanced traffic ratio] no new capacity will be added until that situation is resolved? Cogent [in this case] could solve the problem by adding the appropriate # and type of customers or by buying transit to add capacity [and possibly forfeiting SFI]. Then instead of using your peering partner's customers as pawns, you are using your own to drive the business case either way. That is a much "nicer" way to put pressure on a peer without it getting blasted all over the headlines... Especially if you are the network doing the depeering. And if your customer contracts don't cover (SLA-wise) interconnections, you don't care either way. Just a few ideas... Deepak
At 4:41 PM -0700 10/7/05, william(at)elan.net wrote:
BTW - it sounds like maybe somebody was required to give 30 days notice of service changes to certain customers with good laywers....
In any industry, it only takes a handful of consumers impacted through no fault of their own to generate significant political & regulatory attention. Reasonable-sounding explanations cannot stand if they imply that thousands of innocent parties can be impacted without notice. /John
On Fri, Oct 07, 2005 at 04:23:44PM -0700, william(at)elan.net wrote:
And after November 9, what is then? Cogent/L3 depeering part 2?
Part III: Level3 Strikes Back :) Depeering World Series now at: Level3: 2, Cogent: 14 -- James Jun Infrastructure and Technology Services TowardEX Technologies Office +1-617-459-4051 x179 | Mobile +1-978-394-2867 james@towardex.com | www.towardex.com
participants (13)
-
Daniel Karrenberg
-
Deepak Jain
-
James
-
John Curran
-
Ken Leland
-
Kevin Loch
-
Michael Loftis
-
Patrick W. Gilmore
-
Richard A Steenbergen
-
Robert E.Seastrom
-
Simon Leinen
-
William Allen Simpson
-
william(at)elan.net