- Test - lists.nanog.org

The road not taken (was Re: Ungodly packet loss rates)
by Sean Donelan 22 Oct '96

22 Oct '96

>You're transiting two NAPs (MAE-WEST and the DEC exchange (?)) which >both provide service that's essential on one hand (wide access to >the greater Internet) and lousy on the other hand (focussed access >to a particular site). This whole thread seems to have taken a wrong turn somewhere, or ignored the fact that this is really goofy routing. Best (TLG) and BBNplanet are both at mae-west, yet velvet.com packets traverse Alternet to get to cisco.com. Cisco also appears to have a direct connection to Alternet which is also at mae-west, yet the packets traverse BBNplanet. I would prefer to think there is an 'Oops' somewhere. I don't know why any provider would purposely do this to their customer's traffic. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

2 1

Re: Ungodly packet loss rates
by Vadim Antonov 22 Oct '96

22 Oct '96

Bill Manning wrote: In fact... :) To borrow from your FedEx model.. We can build bigger doors (MTU) but the conveyer belts are stuck at 100Mm/sec. We need faster conveyer belts! (waiting for HPPI-64 or G-Ether technologies w/ baited breath :) And then watch out 2-4Gbps backplanes... (that is aggregate right?) Oh, well. You do not need 2 or whatever Gbps backplanes to build terabit per second networks. --vadim

2 1

Re: Ungodly packet loss rates
by Peter Kline 22 Oct '96

22 Oct '96

At 07:15 PM 10/21/96 -0400, Jon Zeeff wrote: > >In other words, the big players don't like the "open" naps and >are deliberately not installing sufficient bandwidth to them? Not exactly. We're a couple of weeks away from OC3 to both mae-east and mae-west, but when there are multiple providers there with 80+% utilized ports, I'm not really sure just how much we can do with that bandwidth except drop packets a whole lot faster. AGIS has had OC3 connections to both Chicago and PacBell NAPs on order for months, but WorldCom had tariff concerns to address before we could get the connections into our colocate space. Even when the will and the money are there, tariff prevails. Peter _Lucem Diffundo_

1 0

RE: Cisco Memory.
by Chris A. Icide 22 Oct '96

22 Oct '96

> Na, ram is ram. It is just like FDDI cables. I spent $250 on FDDI cables, > and then found out that I can get the exact same cable for $50 each. > Yep, it is... However, just to prevent any possible misconceptions.. In the SIMM world you still have different types of SIMM, however, I've found that most commercial router platforms that use SIMMs, use 72 pin parity SIMMs. Non Parity and EDO tend to give you a bad day. Chris

1 0

RE: Ungodly packet loss rates
by Chris A. Icide 22 Oct '96

22 Oct '96

>I phoned Sprint asking them to look into it - and I do have to say they >were very responsive and conducted extensive testing - and the bottleneck >turned out to be, surprise surprise, MAE-WEST. > Actually, I'd be more than willing to bet money that it's NOT actually MAE-WEST itself that is the problem. What has been happening is that the major NSP's have been redesigning thier networks to try and get a handle on the incredible growth rates. The "big boys" have gone to a private interconnection type of arrangement that allows the traffic to flow between them at multiple points with much less congestion. (Jeff and Jack from MCI and Hank or Kurt from Sprint could possibly elaborate upon this topology) The one outstanding problem that results from this is that the resources of thses companies are no longer directed at the meet points as an important (allow me some license here guys) part of thier network. So now all of the traffic originating from peers at the meet points moves happily across the meet point to the NSP's router, and then hops onto a DS-3 to travel from the meet point router to thier backbone. There are two achilles heels here. The port at the meet point (a switched FDDI port still can only handle 100Mbps simplex), and the link between the NSP's meet router and thier backbone. I think in most cases one will find that these are the locations of the problem, and it's not really a problem that can be fixed by adding more hardware or new technology at the meet points. It's more a policy and financial decision that must be made by the NSP's. > >Since all the discussion on what backbone (tier 1) providers can do to >improve interconnection seems to have done something - intra-provider traffic >seems to be much better nowadays than it was last year - can we start doing >the same with major exchange points such as MAE-WEST and MAE-EAST, etc? > Again, I think one of the guys I mentioned above could better express thier ideas here... > >If the problem is that the routers/gigaswitches are overloaded, perhaps >some sort of intelligent traffic analysis and division might be in order? > Now this definately has merit. Strong planning (a point that I'm weak in) by both the Exchange operators and the NSP's could prevent a large part of the congestion that shows up. But again, you must look at things from the other side. If I give peering to someone (say a regional ISP), I have to invest money into my network to handle the traffic load they send me. If I sell a circuit to that regional, then there is a revenue stream to maintain the network. Peering has become a financial decision in addition to a network engineering decision. This is really where some type of settlement method needs to be established if peering as we know it today is to remain a viable method of interconnecting the networks. Chris A. Icide Senior Engineer Nap.Net, L.L.C.

6 6

Re: Ungodly packet loss rates
by jbash＠velvet.com 22 Oct '96

22 Oct '96

> I think the original message that started it was a bit out of line, > but only because it was cc-ed to NANOG without giving providers > a chance to respond to the criticism. I apologize for that, but the point I was really trying to make was that this sort of thing seems to be pervasive, rather than to single out particular providers. I should have separated the specific trouble report from the discussion of the original problem. The two have now become almost inextricably linked. I'm sure that everybody will take the last message I sent as trashing Alternet, which isn't what I wanted to do at all. What I don't seem to see is much attention given to the very high level of service and cooperation needed to make the Net as a *whole* reasonably reliable. When I talk about "unacceptable levels of service", I mean service from the Net as a whole, not from any particular provider. For example, the problem I was originally complaining (or whining) about seems to have been caused by a reconfiguration by some third party, which knocked Alternet for a loop. If that's the case, it's not Alternet's fault, and it may be hard to really call it the other party's fault either... but, regardless, it's made my life difficult for several days. At a more obvious level, the loss probability for packets sent through a chain of providers is the product of the loss probabilities for the individual providers. 5 percent loss doesn't sound so bad, until you realize that chaining three networks with 5 percent loss gives you nearly 15 percent. When I talked about standards of service, I was thinking mostly of the loss probabilities... but you could extend it to, for example, defining how people would do various routing configuration things. I know that it's not going to be easy to find a way to make the Net as a whole work. Even from my position as an outdated outsider, I can see some of how tricky it is. I'm always amazed that the BGP infrastructure holds together at all. However, I think it's necessary if the network is really going to be useful. In addition, I really do think that maybe people need a "breather" from the constant growth. > To my mind, the right answer is to check with the folks involved! > Any other answer falls short. If there is loss somewhere, most > providers want to be told about it. In some cases, it's possible > to shift traffic around to deal with such problems. In other cases, > there may be planned upgrades in the works to deal with the problem. > In other cases, they may not even be aware of the problem. > To comdemn network providers without even asking them to justify and > investigate the loss represented by traceroute on their networks > is hardly fair. I didn't intend to condemn anybody. I'm sure people are fixing problems as fast as they can reasonably be expected to. The problem is that the environment creates problems faster than people can fix them. The whole thing is spinning out of control, and what I was really trying to suggest was that maybe people should slow down a little and try to develop the network at a rate that can really be managed. -- J. Bashinski

1 0

RE: Ungodly packet loss rates
by Chris A. Icide 21 Oct '96

21 Oct '96

Jeff, I think you may miss the point. In the commercial Internet world, I have to pay for my network. If I'm a big guy, my main value is hauling traffic long distances. How am I to make a profit (or even maintain my network) if 1) I can't exchange traffic with the other long haul providers without significant loss, and 2) I give free access to all of my most opportune customers? I doubt that the people who design, engineer, and install these networks feel at all that way about open naps. They have been driven to a business decision. In many peoples eyes it may not have been the best decision, but it happens to be the easiest and quickest method to quench the bleeding. We are at a point where the traffic growth is causing a lot of people to re-evaluate their network design philosophy. We haven't even begun to see the final shape of thinks (All IMHO) Chris ---------- From: Jon Zeeff[SMTP:jon@branch.net] Sent: Monday, October 21, 1996 2:15 PM To: Chris A. Icide Cc: dalvenjah(a)dal.net; emv(a)coast.net; nanog(a)merit.edu Subject: Re: Ungodly packet loss rates In other words, the big players don't like the "open" naps and are deliberately not installing sufficient bandwidth to them? -- (313) 741-4442 http://branch.com/ Jon Zeeff Branch Internet Services Inc. jon(a)branch.com *** WWW Hosting Services, WWW Site Development and the Branch Malls ***

1 0

Re: Ungodly packet loss rates
by John Hawkinson 21 Oct '96

21 Oct '96

This whole thread really disturbs me. I think the original message that started it was a bit out of line, but only because it was cc-ed to NANOG without giving providers a chance to respond to the criticism. Nevertheless, the posts that followed it were far worse. Ed Vielmetti have suggested interconnects are "to blame" and that if you transit any of them you're likely to see loss, or that if you even take multiple providers to get your destination, you might as well throw in the towel. Dalvenjah FoxFire complains about a similar amount of packet loss across MAE-WEST. Unfortunately there aren't enough details to see if the problem is related. But the implication is that MAE-WEST is at fault. This is pretty unlikely, go look at the graphs on http://www.mfsdatanet.com/MAE/west.stats.html, yourself. It is likely that at any given point some provider may have congestion on one of their links to an interconnect -- surely this does not mean the interconnect was to blame. Bill Bradford suggests you're lucky to get anything approximating a lossless connection. Derek Elder injects irrelevencies about holes in operating systems that make us wish NANOG was a moderated list (too bad the only qualified people don't have time to moderate). To my mind, the right answer is to check with the folks involved! Any other answer falls short. If there is loss somewhere, most providers want to be told about it. In some cases, it's possible to shift traffic around to deal with such problems. In other cases, there may be planned upgrades in the works to deal with the problem. In other cases, they may not even be aware of the problem. To comdemn network providers without even asking them to justify and investigate the loss represented by traceroute on their networks is hardly fair. Oh yes, the second time around, Ed suggests that network operators may have "no hesitation dropping traceroute or ping packets to low priority". No major network provider *drops* traceroutes or ping packets. Many vendors' code (cisco in particular) deprioritizes handling these packets such that round-trip-times are not meaningful; said vendors also rate-limit responses -- this is normally intrusive but might become so in the face of poor testing methodologies. --jhawk John Hawkinson

1 0

Re: Ungodly packet loss rates
by jbash＠velvet.com 21 Oct '96

21 Oct '96

> The wonderful, and possibly the only redeeming quality of the new commercial > Internet is that it is a free market system. You have your choice of which > ISP/NSP to connect to as a dial-up user. Cisco also has a choice as to > which provider to connect to. If either chooses poorly, you can always seek > out those providers which do not oversell their lines, provide UPS > and power generators in case of failure, which peer openly with all networks, > and offer guaranteed reliability. Good providers are available for a price. > However, if you are concerned with price, as a lot of users of certain ISPs > are, then I have no pity.. you simply pay what you get for. Well, I'd agree with you... if the providers I use had told me that I was going to get degraded service. They did not. They told me I was going to get ordinary, decent, usable commercial-quality service. It's true that they offered me a low price, but they offered me a low price for a product that they're not delivering to my satisfaction. Not only that, but the low price I get was explicitly justified by a lack of technical support, *not* by a lack of decent packet delivery. I don't think they were or are deliberately misleading me, or otherwise being dishonest. It seems as though they're following standard industry practice. At this point, the sort of thing I'm seeing seems dangerously close to becoming the accepted standard of service on the Net. Unfortunately, it's not adequate, and it's not what a reasonable person would expect when ordering a service. I make no apology for looking for the best price for Internet service, nor for complaining about it when I don't get that service. In fact, my immediate provider, TLGnet, doesn't seem to have any major internal capacity problems, and they do all the downtime avoidance things you suggest. I've never had a problem with service internal to TLGnet. It's the larger providers that seem to be the problem. TLGnet is at fault only insofar as they have a choice of which backbones to use... and such choice is hardly unlimited. If you want me to contract directly with somebody who has direct connections at all the various interconnect points, I suppose that's a possibility. It doesn't seem to have helped Cisco, though... BBN is pretty big, and there still seems to be lots of loss inside the BBN network. Same for Alternet. I haven't measured it, but I've heard that there are huge loss rates at MCI and Sprint (even if they'd deal with me, which I doubt they would). I'm sure there are others, but who's better? Who'm I supposed to call? How would I even *know* whether somebody had oversold capacity? Assuming that you're right, and that there are better providers, I don't have the dozens of hours it would take me to pore over people's network maps, and to read their technical information in hundreds of different formats. I also have no control over the provider choices of the people I communicate with... and large loss rates seem to be the norm, not the exception. Furthermore, I'm a bit of a rarity in that I actually understand (some of) the technical issues. Most users are no more capable of evaluating an ISP's network design than I am capable of flying to the moon. If they see poor service, they just assume that the Net is intrinsically flakey. I expect the providers I use to give me an appropriate level of service. Unless they've explicitly told me to expect low-quality service, I expect decent quality. From any provider. Regardless of price. That's the issue I'm raising for the providers I'm complaining about, and for other providers (apparently the majority) who are in the same boat. I don't really want to single out TLG, UUNET, or BBN, because as far as I can tell this kind of service is the norm. How much real-world traffic is there that *doesn't* cross at least one network with a high loss rate? For NANOG and the industry as a whole, the issue is the fact that the Internet is still, in some sense, "one thing". If one provider gives poor service, everybody's customers see the results. It does me as a customer little good to buy from a good provider if the people I communicate with are using poor providers. It's therefore in the industry's interest to educate customers, and to have easily applied have quality-of-service standards. I wouldn't make such standards mandatory, but I'd certainly think that they'd be useful to a lot of people, both in choosing services and in explaining to users what they could expect. > Let me stress again.. I seriously doubt this is the appropriate platform for > discussion of which providers are bad.. and therefore.. I'll stop right > here. So, what is the right forum? Is there a seal of approval somewhere? Should there be one? How about a standardized set of service grades? If I knew that I (or the other end of my connection) had ordered grade C service, I wouldn't complain if I didn't get grade A throughput. Right now, there's only one grade: "usable"... and I'm getting something that barely qualifies. -- J. Bashinski

2 1

Re: Ungodly packet loss rates
by jbash＠velvet.com 21 Oct '96

21 Oct '96

[Quotes reordered] > If you want good internet service to a particular point on the network, > you must shop for it specifically and be prepared to continue to shop > for it whenever the underlying network topology changes. My point is that I should be able to expect better than a 40-percent loss rate to *any* point on the Net, at *any* time, from *any* provider. In fact, I think I should be able to expect less than 5 percent, and probably less than 1 percent. Back when I actually touched routers, I used to recommend that people upgrade their internal networks when loss rates hit 1 percent. A situation where I have to "shop around" for connectivity depending on what site I want to talk to that day is just plain unacceptable. > You're transiting two NAPs (MAE-WEST and the DEC exchange (?)) which > both provide service that's essential on one hand (wide access to > the greater Internet) and lousy on the other hand (focussed access > to a particular site). It doesn't look to me as though the loss is being introduced at the NAPS. If you look at the trace, you'll see that significant loss starts to appear within Alternet, well after MAE-west. It looks as though more loss appears inside BBN's network, although it's difficult to tell because of the already large Alternet loss. > my suggestion would be to ask cisco what isp they recommend > for dialup access from your location, and dial in through them. > you should be able to lop off 60% of your hops and probably 99% > of your problems if you pick right. Thanks for the suggestion, but I work for Cisco, so ordinarily I have a Frame Relay connectivity into Cisco's network. This week the computer I usually use for work is in for service. The Cisco LAN at my house is air-gapped from the personal LAN, and it would be a real pain to reconfigure everything, so for the moment I'm coming in to Cisco over the Net, using my personal service. It's purely temporary. If I did interactive work under these conditions on a regular basis, I'd have gone insane long ago. -- John B.

3 2