Well, another round of the depeering battles. We received notice this morning that Exodus is depeering at all US public exchanges on Friday ( gotta love that notice by the way ). They are also not accepting any requests for private peering ( despite meeting the requirements still listed on the peering page ): http://bengi.exodus.net/external/peering.html They will happily continue to sell transit at said exchanges though, and all C&W peering contacts forward to sales ( ain't that cute! ). Should be interesting to see how this impacts the ability to reach sites hosted at Exodus. -Chris -- \\\|||/// \ StarNet Inc. \ Chris Parker \ ~ ~ / \ WX *is* Wireless! \ Director, Engineering | @ @ | \ http://www.starnetwx.net \ (847) 963-0116 oOo---(_)---oOo--\------------------------------------------------------ \ Wholesale Internet Services - http://www.megapop.net
I'm presuming that Exodus is planning to get the transit they need after this depeering via C&W's peering points? If so, this makes a certain amount of sense - no need to maintain separate peering circuits; this is probably just a step in the eventual assimilation of Exodus' IP backbone into C&W's. -C On Tue, Mar 26, 2002 at 11:12:12AM -0600, Chris Parker wrote:
Well, another round of the depeering battles.
We received notice this morning that Exodus is depeering at all US public exchanges on Friday ( gotta love that notice by the way ). They are also not accepting any requests for private peering ( despite meeting the requirements still listed on the peering page ):
http://bengi.exodus.net/external/peering.html
They will happily continue to sell transit at said exchanges though, and all C&W peering contacts forward to sales ( ain't that cute! ).
Should be interesting to see how this impacts the ability to reach sites hosted at Exodus.
-Chris -- \\\|||/// \ StarNet Inc. \ Chris Parker \ ~ ~ / \ WX *is* Wireless! \ Director, Engineering | @ @ | \ http://www.starnetwx.net \ (847) 963-0116 oOo---(_)---oOo--\------------------------------------------------------ \ Wholesale Internet Services - http://www.megapop.net
Chris, You are right. On Tue, 26 Mar 2002, Chris Woodfield wrote:
I'm presuming that Exodus is planning to get the transit they need after this depeering via C&W's peering points? If so, this makes a certain amount of sense - no
Looking at Exodus Route Server you will see that they are now getting transit from C&W. Probably using as you state their current peering circuits (it makes sense from an operational point of view, when you are consolidating an AS into a single one). route-server.exodus.net>sh ip bgp regexp _3561_ BGP table version is 15604957, local router ID is 209.1.220.234 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path * i3.0.0.0 209.1.40.148 1000 0 3561 1239 80 i * i 209.1.220.242 1000 0 3561 1239 80 i * i 209.1.220.102 1000 0 3561 1239 80 i * i 209.1.220.9 1000 0 3561 1239 80 i * i3.18.135.0/24 209.1.220.102 1000 0 3561 7018 ? * i 209.1.220.9 1000 0 3561 7018 ? * i4.0.0.0 209.1.40.148 1000 0 3561 1 i * i 209.1.220.174 1000 0 3561 1 i * i 209.1.220.102 1000 0 3561 1 i * i 209.1.220.242 1000 0 3561 1 i * i 209.1.220.133 1000 0 3561 1 i * i 209.1.40.72 1000 0 3561 1 i * i 209.1.40.141 1000 0 3561 1 i * i 209.1.220.9 1000 0 3561 1 i * i 209.1.220.102 1000 0 3561 1 i * i 209.1.220.9 1000 0 3561 1 i * i6.0.0.0/20 209.1.40.148 1000 0 3561 3549 i * i 209.1.220.156 1000 0 3561 3549 i * i 209.1.220.242 1000 0 3561 3549 i * i 209.1.40.72 1000 0 3561 3549 i * i 209.1.40.141 1000 0 3561 3549 i * i 209.1.220.174 1000 0 3561 3549 i * i9.2.0.0/16 209.1.40.148 1000 0 3561 701 i * i 209.1.220.174 1000 0 3561 701 i
need to maintain separate peering circuits; this is probably just a step in the eventual assimilation of Exodus' IP backbone into C&W's.
-C
What I don't know is what they are going to do with their private peers ? Does somebody has a clue on this ?
It is a free market and they can do anything they want. If you have 5000 routes, and OC48c backbone and 3 OC3s worth of traffic at a 2:1 ratio; peering with C&W is a snap. It clearly improved the ability of new players to enter the market for the FCC to aprove the transfer of MCI Internet assests to C&W. It clearly resulted in the market conditions the federal goverment desired. --On Tuesday, 26 March 2002 12:35 -0500 German Martinez <gmartine@mafalda.opentransit.net> wrote:
Chris, You are right.
On Tue, 26 Mar 2002, Chris Woodfield wrote:
I'm presuming that Exodus is planning to get the transit they need after this depeering via C&W's peering points? If so, this makes a certain amount of sense - no
Looking at Exodus Route Server you will see that they are now getting transit from C&W. Probably using as you state their current peering circuits (it makes sense from an operational point of view, when you are consolidating an AS into a single one).
route-server.exodus.net>sh ip bgp regexp _3561_ BGP table version is 15604957, local router ID is 209.1.220.234 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path * i3.0.0.0 209.1.40.148 1000 0 3561 1239 80 i * i 209.1.220.242 1000 0 3561 1239 80 i * i 209.1.220.102 1000 0 3561 1239 80 i * i 209.1.220.9 1000 0 3561 1239 80 i * i3.18.135.0/24 209.1.220.102 1000 0 3561 7018 ? * i 209.1.220.9 1000 0 3561 7018 ? * i4.0.0.0 209.1.40.148 1000 0 3561 1 i * i 209.1.220.174 1000 0 3561 1 i * i 209.1.220.102 1000 0 3561 1 i * i 209.1.220.242 1000 0 3561 1 i * i 209.1.220.133 1000 0 3561 1 i * i 209.1.40.72 1000 0 3561 1 i * i 209.1.40.141 1000 0 3561 1 i * i 209.1.220.9 1000 0 3561 1 i * i 209.1.220.102 1000 0 3561 1 i * i 209.1.220.9 1000 0 3561 1 i * i6.0.0.0/20 209.1.40.148 1000 0 3561 3549 i * i 209.1.220.156 1000 0 3561 3549 i * i 209.1.220.242 1000 0 3561 3549 i * i 209.1.40.72 1000 0 3561 3549 i * i 209.1.40.141 1000 0 3561 3549 i * i 209.1.220.174 1000 0 3561 3549 i * i9.2.0.0/16 209.1.40.148 1000 0 3561 701 i * i 209.1.220.174 1000 0 3561 701 i
need to maintain separate peering circuits; this is probably just a step in the eventual assimilation of Exodus' IP backbone into C&W's.
-C
What I don't know is what they are going to do with their private peers ? Does somebody has a clue on this ?
-- Joseph T. Klein
On Tue, 26 Mar 2002, Chris Woodfield wrote: > I'm presuming that Exodus is planning to get the transit they need after this > depeering via C&W's peering points? If so, this makes a certain amount of sense - no > need to maintain separate peering circuits. The point isn't that merging the networks doesn't make technical sense. Of course there's little point in maintaining an overlay network with the same AS and separate peering. The point is that since Exodus had a broader, flatter peering mesh than C&W, even if C&W expands their peering proportionately to accommodate the increased demand which Exodus' traffic will place on their network, it'll still be a net loss in global connectivity, since C&W's peering topology is much narrower. Average path lengths increase, the consumer loses. -Bill
On Tue, 26 Mar 2002, Bill Woodcock wrote:
Average path lengths increase, the consumer loses.
Not to mention Exodus customers. allan -- Allan Liska allan@allan.org http://www.allan.org
I wrote: > Of course there's little point in maintaining an overlay network with the > same AS and separate peering. ^^^^^^^ I meant "different AS". -Bill
You mean Exodus are well connected and C&W limit themselves which gives longer paths and increased latency. I guess its obvious to us this is bad, but the thing the C&W bosses are relying on is that it wont be bad enough for Joe Public to notice, and I very much doubt they will notice :/ Wonder what C&W long term plan is.. at some point they will have very few peers, a global network and transit customers. Will they then depeer with someone like Global Crossing in an attempt to force them to buy transit? We've seen similar upsets in the past but not to attain a business goal, but I do wonder here what they plan to achieve... ! Steve On Tue, 26 Mar 2002, Bill Woodcock wrote:
On Tue, 26 Mar 2002, Chris Woodfield wrote: > I'm presuming that Exodus is planning to get the transit they need after this > depeering via C&W's peering points? If so, this makes a certain amount of sense - no > need to maintain separate peering circuits.
The point isn't that merging the networks doesn't make technical sense. Of course there's little point in maintaining an overlay network with the same AS and separate peering. The point is that since Exodus had a broader, flatter peering mesh than C&W, even if C&W expands their peering proportionately to accommodate the increased demand which Exodus' traffic will place on their network, it'll still be a net loss in global connectivity, since C&W's peering topology is much narrower. Average path lengths increase, the consumer loses.
-Bill
-- Stephen J. Wilcox IP Services Manager, Opal Telecom http://www.opaltelecom.co.uk/ Tel: 0161 222 2000 Fax: 0161 222 2008
On Tue, 26 Mar 2002, Stephen J. Wilcox wrote: > You mean Exodus are well connected and C&W limit themselves which gives > longer paths and increased latency. Longer paths definitely, increased jitter probably, increased latency probably, increased loss possibly. C&W obviously have to have a lot of peering as well, since it's all they have to sell to their customers. However, their peering tends to be limited to a small number of peers to whom they have large connections, whereas Exodus had a large number of peers to whom they had medium-sized connections. So the average hop-count and as-path length for the Internet as a whole are both increased by this action, and nearly all paths increase in length for Exodus customers. So yes, Exodus customers are the big losers in the wake of this. -Bill
From the sound of things, it seems that C&W might have been better off migrating AS3561 into AS3967, not the other way around ;)
I am assuming that the reasons it's not happening like this are much more political than technical. -C On Tue, Mar 26, 2002 at 10:18:04AM -0800, Bill Woodcock wrote:
On Tue, 26 Mar 2002, Stephen J. Wilcox wrote: > You mean Exodus are well connected and C&W limit themselves which gives > longer paths and increased latency.
Longer paths definitely, increased jitter probably, increased latency probably, increased loss possibly.
C&W obviously have to have a lot of peering as well, since it's all they have to sell to their customers. However, their peering tends to be limited to a small number of peers to whom they have large connections, whereas Exodus had a large number of peers to whom they had medium-sized connections. So the average hop-count and as-path length for the Internet as a whole are both increased by this action, and nearly all paths increase in length for Exodus customers. So yes, Exodus customers are the big losers in the wake of this.
-Bill
> From the sound of things, it seems that C&W might have been better off migrating > AS3561 into AS3967, not the other way around ;) I think that's what C&W's engineering group thinks is happening. :-/ I will say that C&W maintains a good backbone internally, even if it's pretty constricted at the edges. Be sad to see that expertise subsumed or driven away. -Bill
On Tue, Mar 26, 2002 at 01:40:49PM -0500, Chris Woodfield wrote:
From the sound of things, it seems that C&W might have been better off migrating AS3561 into AS3967, not the other way around ;)
I'm sure the C&W money people think othervise ... /Jesper -- Jesper Skriver, jesper(at)skriver(dot)dk - CCIE #5456 Work: Network manager @ AS3292 (Tele Danmark DataNetworks) Private: FreeBSD committer @ AS2109 (A much smaller network ;-) One Unix to rule them all, One Resolver to find them, One IP to bring them all and in the zone to bind them.
At 10:18 AM 26-03-02 -0800, Bill Woodcock wrote:
On Tue, 26 Mar 2002, Stephen J. Wilcox wrote: > You mean Exodus are well connected and C&W limit themselves which gives > longer paths and increased latency.
Longer paths definitely, increased jitter probably, increased latency probably, increased loss possibly.
In general, as companies and backbones merge and eliminate "old" ASNs, that would reduce the overall AS path length. That in general should not affect latency but as tier-1 ASNs grow in size, and control more of the path end to end, the latency should improve. The majors/tier1s like AT&T, UUnet, Genuity and C&W provide SLAs "end-to-end" *within* their ASN. They control the pipes, they know what they can take and they don't have to worry about some overloaded peering link. So as consolidation takes place, we should see better latencies and better SLAs. -Hank
C&W obviously have to have a lot of peering as well, since it's all they have to sell to their customers. However, their peering tends to be limited to a small number of peers to whom they have large connections, whereas Exodus had a large number of peers to whom they had medium-sized connections. So the average hop-count and as-path length for the Internet as a whole are both increased by this action, and nearly all paths increase in length for Exodus customers. So yes, Exodus customers are the big losers in the wake of this.
-Bill
to end, the latency should improve. The majors/tier1s like AT&T, UUnet, Genuity and C&W provide SLAs "end-to-end" *within* their ASN. They control the pipes, they know what they can take and they don't have to worry about some overloaded peering link. So as consolidation takes place, we should see better latencies and better SLAs. --- One could also make the argument, that if the number of major players consolidates to say <=4 [like ages past], the pressure the market will be able to place will be significantly reduced. The trend in oligopolies is managed competition, or competition limited to a few markets and no competition in smaller/less interesting markets. In otherwords, there won't be much reason for these companies to make significant or even progressive improvements to their SLAs. This is, of course, the devil's advocate position. Regards, Deepak Jain AiNET
On Tue, 26 Mar 2002, Hank Nussbacher wrote: > In general, as companies and backbones merge and eliminate "old" ASNs, that > would reduce the overall AS path length. This isn't something I really care to make a big argument of, but my point was that for many ISPs, the path will go from: SELF - EXODUS to: SELF - OTHER BACKBONE - C&W for a net increase in average path length. That is, of course, a gross generalization. And not anything I'm trying to make a big point of. -Bill
<snip> Should be interesting to see how this impacts the ability to reach sites hosted at Exodus. </snip> nothing complicated. just means you will utilize a transit provider to reach Exodus hosted sites instead of direct public peer. unless you privately peer with C&W. the bottom line - it will now cost you more to reach Exodus hosted sites... /chris
On Tue, 26 Mar 2002, Chris Flores wrote:
<snip> Should be interesting to see how this impacts the ability to reach sites hosted at Exodus. </snip>
nothing complicated. just means you will utilize a transit provider to reach Exodus hosted sites instead of direct public peer. unless you privately peer with C&W. the bottom line - it will now cost you more to reach Exodus hosted sites...
Since Exodus is mostly a webhoster, do they have an asymetric traffic flow. Isn't bulk of the bandwidth is outbound from Exodus. Won't this just increase the distance and AS count for Exodus outbound traffic, making Exodus hosting even less desirable?
IIRC, Exodus had arrangements with at least some of their peering partners where in exchange for the toleration of the asymetric traffic flow at peering points, they would honor MEDs sent to them by said peering partners. I'm assuming that C&W's pering points do not do this. So, it appears that these carriers are going to find a lot more Exodus-orignated traffic on their networks come Friday. I just hope for C&W's sake that their peering points are up to for the challenge of carrying that traffic. -C On Tue, Mar 26, 2002 at 12:47:57PM -0500, Sean Donelan wrote:
On Tue, 26 Mar 2002, Chris Flores wrote:
<snip> Should be interesting to see how this impacts the ability to reach sites hosted at Exodus. </snip>
nothing complicated. just means you will utilize a transit provider to reach Exodus hosted sites instead of direct public peer. unless you privately peer with C&W. the bottom line - it will now cost you more to reach Exodus hosted sites...
Since Exodus is mostly a webhoster, do they have an asymetric traffic flow. Isn't bulk of the bandwidth is outbound from Exodus. Won't this just increase the distance and AS count for Exodus outbound traffic, making Exodus hosting even less desirable?
On Tue, 26 Mar 2002, Chris Woodfield wrote:
IIRC, Exodus had arrangements with at least some of their peering partners where in exchange for the toleration of the asymetric traffic flow at peering points, they would honor MEDs sent to them by said peering partners.
They did? Christian ------------------------------------- i am me, i dont write/speak for them
On 2002-03-26-12:58:09, Chris Woodfield <rekoil@semihuman.com> wrote:
IIRC, Exodus had arrangements with at least some of their peering partners where in exchange for the toleration of the asymetric traffic flow at peering points, they would honor MEDs sent to them by said peering partners.
'peering partners' == (paid) transit providers? -a
On Tue, Mar 26, 2002 at 12:47:57PM -0500, Sean Donelan wrote:
Since Exodus is mostly a webhoster, do they have an asymetric traffic flow. Isn't bulk of the bandwidth is outbound from Exodus. Won't this just increase the distance and AS count for Exodus outbound traffic, making Exodus hosting even less desirable?
I don't think Exodus hosting can get any less desirable at this point. On the other hand, this could help balance traffic ratios, and make more people qualify for peering with CW. Well probably not, considering their requirements include winners like this: A. The applicant shall consistently announce at least 5000 routes to AS3561 (way to encourage aggregation guys, no really, good job) With luck it will throw their peering ratios out of balance and get THEM depeered by someone. Oh well, I guess there is a reason the entire industry calls them Clueless & Witless. :) -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras PGP Key ID: 0x138EA177 (67 29 D7 BC E8 18 3E DA B2 46 B3 D8 14 36 FE B6)
On the other hand, this could help balance traffic ratios, and make more people qualify for peering with CW. Well probably not, considering their requirements include winners like this:
A. The applicant shall consistently announce at least 5000 routes to AS3561 (way to encourage aggregation guys, no really, good job)
Hmm based on my current peers this would exclude the likes of Telstra, Level3, HKT... Bye bye Asia Pacific!
With luck it will throw their peering ratios out of balance and get THEM depeered by someone. Oh well, I guess there is a reason the entire industry calls them Clueless & Witless. :)
On another angle, if enough people refuse to take C&W routes from transit preferring only peering.... nar, thats a conspiracy! Good plan tho. Steve
Date: Tue, 26 Mar 2002 18:20:02 +0000 (GMT) From: Stephen J. Wilcox <steve@opaltelecom.co.uk>
On another angle, if enough people refuse to take C&W routes from transit preferring only peering.... nar, thats a conspiracy! Good plan tho.
But if provider X becomes undesirable, I'd expect people to adjust local-pref on learned routes. That reduces the amount of traffic _to_ the provider in question, which certainly affects symmetry. If you _really_ want to get nasty, think frac-DS1, ^AS$ on inbound, and ^$ on outbound. :-P Oh, wait... except for the filter lists being a tish off, that's how peering between certain providers used to be in the mid, even late, 1990s. ;-) [Stretching the truth, but certain inter-AS hops sure made me wonder...] Eddy Brotsman & Dreger, Inc. - EverQuick Internet Division Phone: +1 (316) 794-8922 Wichita/(Inter)national Phone: +1 (785) 865-5885 Lawrence -- Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature. These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
Date: Tue, 26 Mar 2002 12:47:57 -0500 (EST) From: Sean Donelan <sean@donelan.com>
Since Exodus is mostly a webhoster, do they have an asymetric traffic flow. Isn't bulk of the bandwidth is outbound from Exodus. Won't this just increase the distance and AS count for Exodus outbound traffic, making Exodus hosting even less desirable?
Everyone who peer(s|ed) with AS3967 is supposed to purchase transit from AS3561. ;-) First 174, now 3967. Yes, they're different scenarios, but it seems that AS3561 has strong intentions to depeer. Sure, they can do as they please... but in whose best interest is it? As networks continue to amalgamate and assimilate, it'll be interesting to see what happens to carrier-neutral colo. Will the remaining "big players" pull out, telling customers "you come to us"? Some consolidation is inevitable -- and good. But I wonder if, come 2005, the Internet backbone will resemble how it was in 1995. Will the members of the oligopoly lose interest in maintaining decent peering, simply saying "you could reach us fine if you bought transit from us"? Eddy Brotsman & Dreger, Inc. - EverQuick Internet Division Phone: +1 (316) 794-8922 Wichita/(Inter)national Phone: +1 (785) 865-5885 Lawrence -- Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature. These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
Will the members of the oligopoly lose interest in maintaining decent peering, simply saying "you could reach us fine if you bought transit from us"?
There's an FCC report on "The Digital Handshake: Connecting Internet Backbones" at http://www.fcc.gov/Bureaus/OPP/working_papers/oppwp32.pdf that talks a bit about this very issue... -- Jen
Since Exodus is mostly a webhoster, do they have an asymmetric traffic flow. Isn't bulk of the bandwidth is outbound from Exodus. Won't this just increase the distance and AS count for Exodus outbound traffic, making Exodus hosting even less desirable?
Well spotted. Everybody sees this de-peering as a bad thing when in actual fact its creating a huge opportunity for the competition to hoover up some much needed additional business. A large proportion of Exodus customers bought service from them because of their peer with anyone policy and those customers are going to be very annoyed by this change and also by the large number of network outages and connectivity issues that will arise from this well thought out plan. If this is handled as well as the original 3561 migration then the value of the traffic within AS-EXODUS will be zero as there probably won't be any customers left. So prep your sales folks, dig up all the alleged old tales of woe from the 3561 migration, call up the bids that you lost to Exodus, make up special offers for these customers and help turn this into a non-issue. If there are big ISPs behind 3967 then try setting up direct peering with those guys, I do this for all intellectually challenged organisations who don't wish to peer with organisations that I have represented because of some policy that changes everytime the sun rises. Do some work with netflow identify the larger ASes and give do that funny thing with the phone and give them a call! These Things Do Work if you put the effort in. Oh and then tell the sales guy from the stupid provider your plan thats the icing on the cake :-) Regards, Neil. -- Neil J. McRae - Alive and Kicking neil@DOMINO.ORG
participants (18)
-
Adam Rothschild -
Allan Liska -
Bill Woodcock -
Chris Flores -
Chris Parker -
Chris Woodfield -
Christian Nielsen -
Deepak Jain -
E.B. Dreger -
German Martinez -
Hank Nussbacher -
Jennifer Rexford -
Jesper Skriver -
Joseph T. Klein -
neil＠DOMINO.ORG -
Richard A Steenbergen -
Sean Donelan -
Stephen J. Wilcox