On Wed, Oct 05, 2005 at 04:51:12PM -0700, JC Dill wrote:
Depeering never makes sense to me. Customers of both companies are expecting their vendor to connect them to the customers of the other company. These customers are each paying their respective vendor for this service. Why should one vendor pay the other for this traffic that is mutually beneficial to them while the other pays nothing? Why does the amount of bandwidth or the direction it travels make any difference? The customers are PAYING for the bandwidth.
Point being, they don't both feel that the traffic is mutually beneficial. Peering relationships end when either one of the parties decided they don't want to swap bits for free any more, at which point notice is given, and the two parties make arrangements to deliver their traffic via alternate means.
If each vendor pays their own costs to a peering point then they should be passing that cost onto > their respective customers as part of the *customer's* bandwidth bill. ... "Even though you pay us for internet access, we want this other network to pay us TOO so that we get paid double for transmitting your traffic to and from that other network rather than doing a mutual exchange like we do with other big networks."
At the risk of wasting everyone's time by explaining how and why peering and transit works on a mailing list which is supposed to be loaded with people who are experts on the subject (HAH), I think you're missing quite a few concepts that are central to the debate. First of all, do you have any idea how much it costs to operate a sizable network? Empty routers alone cost hundreds of thousands of dollars, then each OC192 or 10GE card costs hundreds of thousands of dollars more. You need routers in pairs for reliability, interconnected, and you need an aggregation layer to efficiently terminate customers. You need to interconnect individual nodes across a diverse backbone, if you're operating a fiber network you need to pay for LH DWDM equipment which costs hundreds of thousands of dollars for every card used along the path, which must be repeated many times, equaling hundreds of cards. You then need to operate metro fiber networks to deliver the bits to the specific buildings where the customer is located, which means more fiber costs, WDM costs, possibly telco circuit costs. We're already in the hundreds of millions of dollars on the cheap side, billions on the expensive side, of capex, plus millions in operational expenses. And we haven't even covered colocation, powered, techs, crossconnects, peering interfaces, traffic management, network engineers, corporate overhead, sales staff, etc. And Cogent sells transit for $10/Mbps. How much traffic do you think is OUT there to pay for this? Yes they lose money hand over fist doing it (though not nearly as high a burn rate in absolute dollars as some other networks *cough*), and it can only be sustained by investors pumping money into the system over and over again, under the delusion that somehow they'll come out of it making money, but it does happen. Despite the delicious irony that Level 3 was the original disruptive pricing carrier many years ago, Cogent is the disruptive pricing carrier now. They actually run a pretty lean and efficient shop, which is how they're able to keep the losses as low as they do, but Cogent makes what little money it does make on being able to deliver massive amounts of bandwidth to carriers all over the Internet for as cheap as possible. Much of its traffic is hosting based and outbound only, which means that they closest-exit ("hot potato") bits onto other networks, and leave those networks responsible for an extremely disproportionate amount of the longhaul, costing millions of dollars alone on an ongoing basis. All the while, Cogent undercuts the market of every other carrier who isn't as efficient as they are, leading to massive losses, bankruptcy filing after bankruptcy filing, out of court reorganizations and purchases for foreign companies, etc. In case you missed it, this industry is in a state of utter financial disaster on almost all fronts right now. Billion dollar companies are surviving on scraps and what they can sell from previously purchased infrastructure at any price, with no ability to handle the need for upgrades when the existing infrastructure runs out. Networks everywhere are running out of money, pissed off, and realizing that the access to their customers which they provide for free to Cogent is what is enabling Cogent to operate and offer the pricing that it does. They don't feel that it is of mutual benefit any more, and they want it to stop. Level 3 also delivers much more "expensive" traffic, is a real Tier 1, has almost 6x the customer base of Cogent by prefix count (though probably roughly the same or even less overall traffic). It is not unreasonable to expect them to want to capture as much settlement as possible from networks putting traffic into theirs, infact this is a critical requirement in order for them to compete. As with any Tier 1 network, a huge amount of the traffic stays on-net. Of course reality is far more complex than even that little generic description above. Peering is highly political, affecting sales, potential customers, and industry goodwill positively and negatively whether you do or don't peer, and whether you do so with the same policy ("fairly") for all networks. If you look at Cogent's previous routing, it is blatantly clear that they were never going for ratio balance with Level 3. It is entirely possible that Level 3 was asking them for traffic with which they could bill their customers, rather than delivering that traffic via the competition. Of course the only thing you're going to see on this list about why L3 felt the need to depeer Cogent in this particular instance is pure speculation, but there ARE very real and very steep costs associated with peering and peering policies at all levels. These costs do need to be taken into account, and L3 certainly needs all the money it can get its hand on (which is why it has depeered many more networks than just Cogent recently). If anything, I'm personally surprised they didn't do this sooner. Cogent is one of the few networks out there with the balls to put their customer base on the line in this way to fight such a depeering (possibly because their customer base consists so largely of multihomed and/or extremely cheap networks who are willing to endure this and other issues to save a buck). They certainly don't want to look weak on this issue, or other networks with whom they may have less than perfect peering relationships will certainly move on taking similar steps. What happens here could set the tone for many more future depeerings, or lack there of, and in some ways the survival of both companies and the course of the IP transit industry at large. Both networks are highly motivated and have perfectly reasonable positions which make sense to them, hence they are at an impass which can only be resolved through suffering. To force either one to peer by law or regulation would be even more disruptive to the industry as a whole, which may even lead to a complete collapse of the existing peering system as we know it. After all, if anyone can become or retain an "undeserved" peer by the BGP equivalent of saying "you can't make me route packets to you any other way", we could see this kind of thing happening a lot more often, trading short term customer satisfaction for long term economical gain. If forced to keep "undeserved" peers in the long term, peering may vanish completely, to be replaced by some other means of settlement (at least without government funded networks at the cost of billions). Some of the few insights on this list on the subject of depeering have been provided by folks like Vijay Gill and Dan Golding, if you want to get a better feel for why these large networks are motivated to depeer others I highly recommend reading their posts before jumping to any conclusions that the way you see peering as a small network operator has ANY relationship to the way a large network operator has to see it. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)