The Attitude (was: the Internet Backbone)
Catching up on my email after tax week (did anyone else notice that there were a lot fewer postings on this and other lists this past week?):
From: Sean Doran <smd@icp.net> It's because I'm an evil asshole determined to protect my employer's interests and make our shareholders rich.
This is somewhat incompatible with protecting the interests of our competitors and enriching their shareholders.
Gentlefolk, while I agree that Sean has been "mostly" right on many technical issues, I am seriously unhappy with this attitude! Nor is this attitude unique to Sean. Since my local ISP and NSP is Merit/MichNet, and I am an advocate of more regional exchanges, I have been trying to get them to interconnect with other local providers, so that local traffic wouldn't have to go through MAE-East (and suffer 30%-50% losses) to travel 3 blocks across town. They never managed to do anything in 3 years, despite the willingness of others (specifically MSEN) to interconnect. Recently, I asked why they aren't connected to the Detroit NAP. The response was "everyone else should connect to MichNet, and pay our affiliate fee". I noted that the others consider themselves competitors, and taking the same attitude would expect Merit to instead pay THEM, since MichNet generates the most traffic. Likewise, a lot of traffic from Ann Arbor Michigan to Columbus Ohio travels via MAE-East, despite the fact that Merit is already connected to CICnet, which is in turn connected to Columbus (both OSU and OARnet). The problem is, Merit has no "bi-lateral peering" with CICnet. Merit doesn't think there is a "cost benefit" to have regional interconnection and peering relationships. In some respects, they are right. The benefit is not to Merit itself, but rather to its customers (lower delays), and the rest of the Internet (less congestion at other exchanges). The problem is that ISPs are allowed to shove their regional connectivity out to others on the Internet. In effect, the _rest_ of the Internet is _paying_ for the regional underprovisioning.
Personally, I have little patience for the small and not-very-clueful who want to be direct competitors with a multibillion dollar company with lots of talent and who are taken to whining about my policies and those of my colleagues and associates, and even those of our competitors. This uncharitable attitude obviously does not endear me to them.
But your attitude that the whiners are "small and not-very-clueful" is less than useless. There are some quite clueful folks that don't agree with your policies, particularly with the failure to peer (and exchange traffic) with everyone else (even small folks) at an exchange. (Sprint is not the only perpetrator of this poor policy.) The fact is, whether you like it or not, they _ARE_ your competitors in their specific regions. But, to thrive, the Internet has a long tradition of _cooperation_ among competitors. Kinda misses the meaning of "exchange". That hurts everyone else on the net, by increased delay and more congestion elsewhere. In short, you are asking _others_ to bear the costs of _your_ making money. We've seen this time and again, such as the UK provider who sends all their traffic to the US, which then uses the congested US to Europe links. It only saves them money because others were unknowingly bearing the cost. Sounds like a form of fraud to me.
I would hope, though, that the bulk of our customers would be much happier with us driving towards a network reliable enough that they don't have to worry about their customers screaming (not to mention not having to worry about facing some very difficult scaling problems we are already staring at), than with us being the Department of Warm and Fuzzy Feelings.
All of them know full well that the drive ain't easy.
True. But there are some particular bones to pick with Sprint, like the underprovisioned Texas links that kept dropping out, just when Apple released its 7.5.3 MacOS Update to developers from Texas.... So, let's see some of that vaunted reliability first, please. WSimpson@UMich.edu Key fingerprint = 17 40 5E 67 15 6F 31 26 DD 0D B9 9B 6A 15 2C 32 BSimpson@MorningStar.com Key fingerprint = 2E 07 23 03 C5 62 70 D3 59 B1 4F 5E 1D C2 C1 A2
On Sun, 14 Apr 1996, William Allen Simpson wrote:
Likewise, a lot of traffic from Ann Arbor Michigan to Columbus Ohio travels via MAE-East, despite the fact that Merit is already connected to CICnet, which is in turn connected to Columbus (both OSU and OARnet). The problem is, Merit has no "bi-lateral peering" with CICnet.
This is not exactly correct. CICNet Michigan(AS266) peers with Merit over a FDDI ring in Ann Arbor. I suspect that the problem you are running into is that CICNet Primary region (AS1225) which does not have physical connectivity to AS266 or Merit provides connectivity for OSU, and peers with OARnet. This problems has nothing to do with who has what bilaterals with whom, or who is willing to peer with whomever else, but rather a simple constraint based on topology. So please choose a more apropos example next time. Thanks. -dorian
From: Sean Doran <smd@icp.net> It's because I'm an evil asshole determined to protect my employer's interests and make our shareholders rich.
This is somewhat incompatible with protecting the interests of our competitors and enriching their shareholders.
Gentlefolk, while I agree that Sean has been "mostly" right on many technical issues, I am seriously unhappy with this attitude!
Bill, here is why you will never be taken seriously be any ISP/NSP. If networks can't find ways to cooperaate without prodecting there own interests, we all might as well give up now. THE ONLY WAY TO BUILD A SCALABLE INTERNET IS FOR PROVIDERS TO BE MOTIVATED BY SELF INTEREST AND ECONOMIC FACTORS. You cannot build a multi billion/trillion dollar system based on happy thoughts. Sean is "mostly" right on techincal issues, because that is what he is good at, and when REAL WORLD technical issues, dictate policy, there isn't much choice for folks like Sean. You would do just as good slamming Cisco for limiting AGS+'s memory size, or processor speed in 7000 series routers, which BTW where a lot of Seans technical issues that cause these policies too happen.
Recently, I asked why they aren't connected to the Detroit NAP. The response was "everyone else should connect to MichNet, and pay our affiliate fee". I noted that the others consider themselves competitors, and taking the same attitude would expect Merit to instead pay THEM, since MichNet generates the most traffic.
This is typical of the NSFNet regional attituded. The more cluefull ones DO connect to Naps (Sesquinet is at Mae-Houston, for example).
Likewise, a lot of traffic from Ann Arbor Michigan to Columbus Ohio [ example delete ] The problem is that ISPs are allowed to shove their regional connectivity out to others on the Internet. In effect, the _rest_ of the Internet is _paying_ for the regional underprovisioning.
This is exaclty an issue of customer education, If customers wanted good connectity they CAN find it.
Personally, I have little patience for the small and not-very-clueful who want to be direct competitors with a multibillion dollar company with lots of talent and who are taken to whining about my policies and those of my colleagues and associates, and even those of our competitors. This uncharitable attitude obviously does not endear me to them.
But your attitude that the whiners are "small and not-very-clueful" is less than useless. There are some quite clueful folks that don't agree with your policies, particularly with the failure to peer (and exchange traffic) with everyone else (even small folks) at an exchange. (Sprint is not the only perpetrator of this poor policy.)
The fact is, that vast majority of the whiners are SMALL, and have virtually no experience operating large scale regional or national backbones. Sprint and others have VERY GOOD operational reasons for their policy, largely some network is better than no network. Six months ago, if Sprint had peered with everyone that asked, their entire network would have melted down. (Sean correct me if I'm wrong here.)
The fact is, whether you like it or not, they _ARE_ your competitors in their specific regions. But, to thrive, the Internet has a long tradition of _cooperation_ among competitors.
Kinda misses the meaning of "exchange". That hurts everyone else on the net, by increased delay and more congestion elsewhere.
In short, you are asking _others_ to bear the costs of _your_ making money. We've seen this time and again, such as the UK provider who sends all their traffic to the US, which then uses the congested US to Europe links. It only saves them money because others were unknowingly bearing the cost. Sounds like a form of fraud to me.
We if this is the case whoever is paying for those congested links is not getting their monies worth. If they were paying enough for thier bandwidth this wouldn't even been an issue, and largely is an issue of legacy subsdies.
I would hope, though, that the bulk of our customers would be much happier with us driving towards a network reliable enough that they don't have to worry about their customers screaming (not to mention not having to worry about facing some very difficult scaling problems we are already staring at), than with us being the Department of Warm and Fuzzy Feelings.
All of them know full well that the drive ain't easy.
True. But there are some particular bones to pick with Sprint, like the underprovisioned Texas links that kept dropping out, just when Apple released its 7.5.3 MacOS Update to developers from Texas....
Hm.. maybe Apple should have thought a little more about their little T-1 link to a single provider when they had about 3 T-1s worth of data to send. I offer to sell more connectivity to Apple, but they WEREN'T interested. They didn't care that there application/distribution model was broken, and breaking the net, they didn't want to fork out the extra bucks, or deal with the internal politics to put the release out on the West coast where they had a much higher bandwidth connection.
So, let's see some of that vaunted reliability first, please.
WSimpson@UMich.edu BSimpson@MorningStar.com
I haven't seen many reports of Sprint problems since Sprint put its policies in places and they had time work.o MCI on the other hand has been bleeding about 30% packets out of San Francisco, on its OWN INTERAL network for several MONTHS. Strangely about the same time as the problems started showing up, MCI changed some of its polcies too... So it looks like to me, Sprint 3, WSimpson 0. For those remain NETWORK OPERATORS, I propose a mailing list for certified network operators, say must be peering at 2 NAPs with at least 3 peers, or multihomed, to post to the list. This should completely elimiate the problem with whining and people trying to "define" the internet backbone. -- Jeremy Porter, Freeside Communications, Inc. jerry@fc.net PO BOX 80315 Austin, Tx 78708 | 1-800-968-8750 | 512-339-6094 http://www.fc.net
In reply to your message of Sun, 14 Apr 1996 12:28:49 CDT: | >True. But there are some particular bones to pick with Sprint, like the | >underprovisioned Texas links that kept dropping out, just when Apple | >released its 7.5.3 MacOS Update to developers from Texas.... | | Hm.. maybe Apple should have thought a little more | about their little T-1 link to a single provider when they | had about 3 T-1s worth of data to send. I offer to sell | more connectivity to Apple, but they WEREN'T interested. | They didn't care that there application/distribution model was | broken, and breaking the net, they didn't want to fork | out the extra bucks, or deal with the internal politics | to put the release out on the West coast where they | had a much higher bandwidth connection. In an attempt to squash distractors, it makes no sense to me to hold an ISP accountable for failure of a customer to pay for services adequate to their needs, and I also fail to see the relevance of a customer's burying themselves due to internal politics (which you indicate is the case). Also, in an allegedly politically divided house, it doesn't make sense to ascribe the lack of perspicacity to the entire organization; Apple has some great people running it's Internet stuff, and apparently some not so great people. None of this has anything do with a discussion of Internet routing policy issues. What is relevant to a discussion of routing policy issues ;-) is that if free market economics are not a viable means of producing a stable and happy Net (and I'm not saying they aren't! :-), then the alternative is to have regulated services where such things as equal and fair access can be defined and then inflicted on the participants in the industry. The current trend is away from regulation, and towards market economics. As such, Sprint is fully within their rights, and one could argue their charter to their stockholders, to act as they best see fit to meet their service commitments to their customers. Yeah, it makes a mess, as do a lot of other decisions which are economically motivated rather than philosophically motivated. But that's both a consequence and a challenge of transitioning from a Federally funded model to a market funded model. In theory, the market dynamics will support the ISP's offering the best connectivity, and as that dynamic comes more into play things will begin to sort themselves out. History also tells us that market dynamics favor large organizations and monopolies over small ones, hence the predictions of the coming shakeout. However, logically the current state of congestion flux on the net is a near perpetual one; either the providers will continue to improve service to meet growing customer demand, which will in turn generate more congestion, and so on on a perpetual curve of near-stability. Or, service will degrade until customer load diminishes, at which point the same type of near-stability is achieved. I don't see this model changing much until there is enough bandwidth, and enough equipment to drive it, to handle full-motion video to every home and desktop. I could be wildly mistaken; however, Dave O'leary's recent detail of the Net's history does suggest that this is, in fact, the norm and not the exception as far as Net traffic behavior and loading goes. Which means that regardless of what Sprint and other ISP's do to stabilize their services, unless there is a sudden regulatory frenzy there's not much to be done except to ride the wave of expansion out as best as can be done. .02, Cheers, Paul Paul "Corwin" Frommeyer Work Internet Engineer, CCIE Play ISP Systems Engineer Network Sorcerer At Large Cisco Systems, Inc. Paul's Fone Company pfrommey@cisco.com corwin@palas.com *** Speaking solely for myself unless otherwise noted ***
Bill, A short answer to your refusal to peer objection in SPRINT's policies in the form of a question. Which is better: 1. peering with everyone at an exchange, including those ISPs who are clearly clueless, and whose cluelessness leads to operational problems, i.e. injection of bogus routes, black holes, routing loops, routing flaps, and BGP peer transitions? 2. qualifying each of your peers as being clueful, prior to peering with them, such that you (and your customers!) don't suffer from the cluelessness of others? Perhaps the easy way out is to suggest that educating the ISPs as to what constitutes good behavior at an exchange (routing system stability and reliable packet delivery) is the responsibility of the exchange operators, and it might even be possible to enforce some interesting policies in that regard in the route servers (e.g. if you have more than N routing or BGP peer transitions per time period, the route server will refuse to peer with you for 48 hours - think of it as the hold-down or damper from Hell). I certainly think that to the extent that the exchange operators can measure such things as routing and peer stability, it is in everyone's interest to see the numbers (except those ISPs who are unstable). Who knows? A series good reports from exchange operators about an ISP might lead to offers of private peering arrangements outside of the exchange, to the benefit of the ISP. Similar to the way that having a good credit record seems to lead to endless offers of more credit. Lucky me, I'm just a customer, and don't have to worry about such issues, except as they affect my ISPs. Of course, I do expect my ISPs to deliver on the goods I'm paying them for: routing system stability and reliable delivery of packets... Erik E. Fair apple!fair fair@apple.com
participants (5)
-
Dorian Kim -
Erik E. Fair -
Jeremy Porter -
Paul Frommeyer -
William Allen Simpson