- NANOG - lists.nanog.org

90% capacity loss - Business as usual?
by Sean Donelan 28 Nov '00

28 Nov '00

Perhaps the opposite of an 99.999% SLA, just how bad do things need to get before its a problem? Time after time, I've seen networks incapable of carrying 90%+ of its normal traffic, or unusable for its intended purpose. But the PR spokespeople never admit its an "outage." "Larry Plumb, Verizon spokesperson, said that as of Monday there had been no network outages in the Northeast region, just some slow throughput problems. This, despite a voice mail message at its main customer support number telling customers to remain patient until technicians resolved the problem. Repeated requests for information were ignored" http://www.internetnews.com/isp-news/article/0,,8_518561,00.html Ok, so the word "outage" is verboten for PR people. The electric utility industry has the same issue with the word "blackout." Instead there are sustained service anomalies. Which lets utilities say with a straight face they've had no blackouts. Instead of an SLA, do we need a SLG (Service Level Glossary)?

1 0

Re: Limits of reliability or is 99.999999999% realistic
by Toby_Williams＠enron.net 28 Nov '00

28 Nov '00

I think SLAs should always be supplied with a service and those that do not offer one, miss a vital point about the service infrastructure they've put in place. There's not one single, ideal level of service that everyone should be striving for with IP networks. We know this because no two applications have exactly the same network requirements, hence consumer A will have subtley different requirements from consumer B. End consumers *may* want a generalist service, that can do a bit of everything, but as an ASP would your applications have the same network requirements as a video/VoIP provider? Would someone doing backup services share these requirements? There's a vast (and growing) space for differentiation of services and this can only occur with valid SLAs. >From a legal perspective: If I sell someone transit for a month, then the service is "down" that month, and heh I didn't offer an SLA. Then you still have to pay me for the service and I pay you nothing back. OK I'd lose my customers, but: I'd be considerably richer having sold bandwidth for a month and not used any myself, than I should be considering I've just ripped my customers off. That's why the SLA exists. It is a necessary part of the contract that defines the quality of the product to be sold. Toby All-- In a perfect world, your provider says the service is always on, the service is always on. In the real world, we have to deal with outages -- some kids vandalise a phone box, new tech trips over some cables, some idiot telco misimplements MPLS and brings your service down for a day.... These things happen, and sometimes we all just have to suck it down and deal with it. But if it happens continuously, you have to ask your provider for some assurance that it won't keep happening. This is what SLAs are for. In my experience, a company that delivers reasonable levels of service has no need for SLAs with their clients. The service is up, everybody is happy. SLAs are like your parent telling you to do the dishes again "and get it right this time, or else you go to bed with no jell-o!" Which s why, in looking for a vendor, I ask for an SLA. If they have one as a standard offering, then I know that they've messed up a lot in the past, and will probably be messing up more than I like in the future. It's like the kid who never did his homework coming up to the teacher on the last day of school asking for extra credit. NO YOU FOOL! You should have done it right the last 180 days! Anyway, just my thoughts.

1 0

RE: I think I jinxed Sprint
by Mathew Butler 28 Nov '00

28 Nov '00

I thought that routers were supposed to send ICMP Source-Quench messages when they got congested? Or is this something that the proponents of QoS didn't decide on? -Mat -----Original Message----- From: Roeland Meyer [mailto:rmeyer@mhsc.com] Sent: Friday, November 24, 2000 8:58 AM To: 'Sean Donelan'; nanog(a)merit.edu Subject: RE: I think I jinxed Sprint The internet is a lot less forgiving wrt outages then the telco. The telco can have a circut outage, re-route to another circuit, and the customer never sees an availability gap. Also, a total outage, during reduced traffic times, and no customer ever misses a dial-tone because they aren't trying to get one, is not an outage in telco terms. The internet, on the other hand, may have similar issues, unless we start talking streaming video, streaming audio, and voice over IP. In those cases, packet losses can make a serious mess of things. Also, congestion is treated differently between the two systems. Telcos will actually return a fast-busy when a switch becomes congested. The internet simply starts dropping packets. You can actually hear the latter when using www.dialpad.com or MS-Netmeeting (both of which, I use extensively).

4 3

Internet access in Honolulu, HI ?
by Joe McGuckin 28 Nov '00

28 Nov '00

Anyone know of a public library (or a friendly ISP) that will allow me to drop by and plug in my laptop to check email, etc ? Thanks, Joe -- Joe McGuckin ViaNet Communications 994 San Antonio Road Palo Alto, CA 94303 Phone: 650-969-2203 Cell: 650-207-0372 Fax: 650-969-2124

1 0

ANN: New List: usdomreg
by jamie rishaw 28 Nov '00

28 Nov '00

Announcement: Mail list to discuss .US domain changes Everyone, In light of recent changes with ISI, VeriSign(NSI) and the .US TLD, and because of recent discussion that has come up on an (oops) unmoderated ISI mail distribution, I have created a mail list for those interested to discuss the changes to the .US domain name structure, privatization, and the like. *** To subscribe: echo subscribe | mail usdomreg-request(a)arpa.com (Translated: send a blank email to usdomreg-request(a)arpa.com) *** To post: (you must be a member) mail usdomreg(a)arpa.com

1 0

Re: Limits of reliability or is 99.999999999% realistic
by Sean Donelan 27 Nov '00

27 Nov '00

On Mon, 27 November 2000, Robert Cooper wrote: > On the other hand the data is somewhat conservative since it counts > total number of subscribers, not the total number of active or would-be > active users during the outage. The data does include overloads which > in the PSTN manifest themselves through call admission control > (e.g. network busy signal). Like most statistical studies, understanding the data collection process is key. Using the FCC outage reports has some good features. In theory all carriers are under the same obligation to report outages meeting the defined thresholds, so you should be able to compare the data across carriers. There are some data anomolies, which makes it difficult to compare the data more widely. The failure to report a reportable outages is difficult to detect in the FCC data. And even among carriers reporting, there is tremendous variation in the amount of information supplied. Outages in leased, rather than owned facilities are occasionally reported. Shared leased facilities can also result in a very wide-spread, but unreported outage because the owner may not have 30,000 direct customers although the downstream carriers may have far more customers impacted. The various Illuminet SS7 problems are an example of wide-spread, unreported problems. And finally, data or packet switched outages are only reportable in so far as they affect the voice network. For example, when AT&T's frame-relay network failed the analysis done was the impact of lots of ISDN dial-backup devices simultaneously attempting to re-establish the data connection across the switched telephone network. And finally, the FCC outage data excludes most outages less than 30 minutes in length. So it is impossible to claim 99.999% reliability based on just the FCC data because the resolution of the data isn't fine enough to detected outages of 5 minutes. At most you can claim is four-nines. > What data exists for the Internet? There are several sets of data for the Internet. As a starting point, I point people to Labovitz and Ahuja study "Experimental Study of Internet Stability and WideþArea Backbone Failures" http://www.eecs.umich.edu/techreports/cse/1998/CSE-TR-382-98.pdf One interesting piece of data they found is the mean time to repair an Internet problem is 20 minutes. If we followed the FCC outage reporting thresholds of thirty minutes or more, most (mean? half?) of the Internet problems wouldn't be counted as an outage. Also see the list of references at the end of the paper. Other sources of data include provider traffic reports, such as http://traffic.cw.net/ http://ipnetwork.bgtmo.ip.att.net/ provider status pages, such as http://www.noc.uu.net/ http://help.mindspring.com/netstatus/ and third-party performance reporting, such as http://average.miq.net/ http://internetpulse.com/ http://www.whatsdown.com/

2 1

Network Design and Planning Tools
by Bradley Dunn 27 Nov '00

27 Nov '00

Hi, Anyone who's using the network design and planning tools from WANDL or Make Systems care to comment on them? Any other tools people are using? http://www.wandl.com/ http://www.makesystems.com/ Thanks, Bradley

2 1

script kiddie probes
by Dan Hollis 27 Nov '00

27 Nov '00

http://www.dshield.org/top10.php http://www.dshield.org/topports.php Interesting stuff. -Dan

5 4

Limits of reliability or is 99.999999999% realistic
by Toby_Williams＠enron.net 27 Nov '00

27 Nov '00

SLAs are the point where commercial & operational worlds collide. The numbers being offered should: Reflect the targeted quality of the service Tie in with meaningful damages for not achieving them In a market where I can offer 99.7% or 100% and the difference is a whole day's service credit, I know what I'd be offering. No question. If on the other hand, the expectation is that I pay out a year's contract value in cash upfront every time I miss a target in a month, then I'd only want to offer a target that will *always* be achieved. In the market at the moment, the situation is much closer to the first scenario than the second - ie damages for SLAs do not mean anything to either: the buyer, as compensation for not receiving the service that they have contracted for; or the seller as a motivation to work within the targets, because capped service credit agreements do not touch the bottom line Today, in the IP market, it is irrelevant whether services come with 99.0 or 99.99999 SLAs & at some point the market needs to address the responsibility that if they are to offer a service of a certain "guaranteed" quality, then they should stand-by that guarantee with their money and give this guarantee meaning. I don't think this is the case at the moment, and that's why we even see 100% SLA in the market - because the level of pay-outs on SLAs don't matter to the seller. No. I'm not suggesting that sellers offer guarantees for consequential losses. Simply ones that: 1. give the buyer the peace of mind that if the service they've contracted to is below par, that they will have enough money put back in their pocket for them to have replaced their service, like-for-like in the market, to cover themselves over the period. 2. enable the market players to truly differentiate their service offerings by quality, rather than marketing. Toby [std. disclaimer of responsiblity here] Date: 25 Nov 2000 20:24:48 -0800 From: Sean Donelan <sean(a)donelan.com> Subject: Limits of reliability or is 99.999999999% realistic <snipped for brevity> But back to my question. What is the real requirement? Amazon.COM had system problems on Friday, and their site was unusuable for 30 minutes, definitely not 99.999%. But what did that really mean? The FAA loses its radar for several hours in various parts of the country. What did that really mean? Essentially every system given as an example of "high- availability, high-reliability" I've looked at, doesn't hold up under close examination. Is 99.999% just F.U.D. created by consultants? Instead of pretending we can build systems which will never fail, should we work on a realistic understanding of what can be delivered?

8 7

Re: Limits of reliability or is 99.999999999% realistic
by Robert Cooper 27 Nov '00

27 Nov '00

At 08:24 PM 11/25/00 -0800, Sean Donelan <sean(a)donelan.com> wrote: >But back to my question. What is the real requirement? Amazon.COM had >system problems on Friday, and their site was unusuable for 30 minutes, >definitely not 99.999%. But what did that really mean? The FAA loses >its radar for several hours in various parts of the country. What did >that really mean? Essentially every system given as an example of "high- >availability, high-reliability" I've looked at, doesn't hold up under >close examination. > >Is 99.999% just F.U.D. created by consultants? > >Instead of pretending we can build systems which will never fail, should >we work on a realistic understanding of what can be delivered? For some actual *data* on reliability in the US phone system take a look at: "Sources of Failure in the Public Switched Telephone Network" D. Richard Kuhn IEEE Computer Magazine, April 1997, Vol 30, No 4. pp31-36. OK, it's not actual data but a summary of it. It's culled from data that the phone companies have to supply to the feds for every outage that affects more than 30,000 subscribers -- about the number supported by a central office. Measured in user-outage-minutes (duration of outage * number of subscribers) he shows availability in the range of 99.999% for data from the 1992 and 1993. More interesting is the summary of the sources of outages and how much they contribute to the total picture. For instance, if you were one of the average of 250,000 customers who lost service for an average of 500 minutes due to vandalism, you might not think 99.999% means very much. On the other hand the data is somewhat conservative since it counts total number of subscribers, not the total number of active or would-be active users during the outage. The data does include overloads which in the PSTN manifest themselves through call admission control (e.g. network busy signal). What data exists for the Internet? [ Standard disclaimer ] Robert Cooper Ironbridge Networks 55 Hayden Ave, Lexington MA 02421 www.ironbridgenetworks.com

1 0