February 2002 - Test - lists.nanog.org

OT: Looking for advicein designing an private ATM network over a public carrier
by Rich Sena 21 Feb '02

21 Feb '02

I'm working a project in designing a private ATM network over a public provider. I have had a few options advanced to me and would like to discuss a few with some folk who have done a similar implementation. What I would like is to have a common virtual path with multiple nodes and set up my own virtual circuits between end nodes. Please reply off-list to me directly. Reply-To has been set... -- Rich Sena - ras(a)thick.net ThickNET Consulting "On the way to understanding; you understand, and forget."

1 1

best way to get to UUNet 'New York City' colo w/o car?
by bert hubert 21 Feb '02

21 Feb '02

Hi, We have half a rack in what UUNet like to call their 'New York City' colo but which is so far outside NY City that nobody there knows where it is. Now, because our people going there can't/don't want to rent a car, we find ourselves at a loss to figure out how to get there easily. The address is 401 Fieldcrest Drive, Elmsford, NY 10523-1026. We would like to know how to get their directly from JFK and from Manhattan, if possible. Last time we took a cab and that was very tricky - the cab driver proved to be very flexible and understood the output of Yahoo's driving instructions luckily, but you can't count on them being that good. It was also exceedingly expensive. Anything is fine with us, car service, public transport, whatever. Thanks for your time! Regards, bert -- http://www.PowerDNS.com Versatile DNS Software & Services http://www.tk the dot in .tk Netherlabs BV / Rent-a-Nerd.nl - Nerd Available - Linux Advanced Routing & Traffic Control: http://ds9a.nl/lartc

2 2

Cisco PPP DS-3 limitations - 42.9Mbpbs?
by Ehud Gavron 20 Feb '02

20 Feb '02

At a recent consulting opportunity, a client was only getting 40.9Mbps on their DS-3 (clearchannel DS3 over OC12 to the IP provider). (PA-2T3 to PA-2T3, unloaded CPU, drops occur when peak octets out per sec/8=40.9M) The client asked us why 40.9 was less than 44.210. The IP provider indicated that "Cisco told us the maximum theoretical rate of an interface doing DS-3 with PPP encapsulation is 42.9." I can't find documentation to the 42.9. I can't find documentation that indicates that the PPP overhead is somehow magically excluded from the IfOctetsIn and Out... I can't find documentation as to why 40.9 is the observed rate. Frankly, all my DS-3s (HDLC encap) go up to 44.210, so I was hoping I could share in the collective wisdom of NANOG readers who have DS-3s to IP providers, and can provide real observed feedback. Please reply privately. Please let me know privately if you want a summary. Thanks! Ehud Gavron gavron(a)wetwork.net

11 17

Re: Cisco PPP DS-3 limitations - 42.9Mbpbs?
by Dave Israel 20 Feb '02

20 Feb '02

The place where QOS always made sense to me is not where they want to put it (and it's very apropos to the current conversation.) Specifically, at the customer edge, as opposed to the provider core. Some folks are always going to have higher bandwidth needs than money. We can complain all we want about how, in a perfect world, everybody should buy more bandwidth before queue delay and congestion are a problem, but in the end, Generic Office X's communication budget is probably for a single T1, which can either run VOIP or be channelized into nxDS0 for phone and 24-nxDS0 for data. In either case, it doesn't take much legitimate traffic before you start seeing problems, especially if you're doing VOIP, video- conferencing, web serving, and Bob is sitting in his office listening streaming video and browsing alt.sex.toasters. And sure, it would be nice if people were responsible and didn't use bandwidth frivolously, but we're in the real world, and when people won't behave, the system should have the option of trying to do the right thing. So, at the edge, you prioritize voice over streaming media over web over news over mail, and everything behaves acceptably. *That* is where QOS really shines. For some reason, though, nobody wants to sell it there. Must be more profitable to sell 'em a bigger line. -Dave On 2/20/2002 at 14:11:01 -0800, Tony Rall said: > > On Wednesday, 2002/02/20 at 09:08 PST, Randy Bush <randy(a)psg.com> wrote: > > [0] - corollary: qos mechanisims decide which packets to drop. but isps > > are paid not to drop any packets. > > Exactly. I've been saying this to vendors for the last few years, as they > try to push their QOS mechanisms on me. > > I'm not in the business of trying to engineer some optimum packet discard > strategy. I would rather spend my time and money trying to minimize the > drop percentage. (I haven't been tested yet with the task of trying to > minimize or standardize latency for traffic like VOIP - might change my > tune if I was dealing with that.) > > Tony Rall

2 1

Re: Cisco PPP DS-3 limitations - 42.9Mbpbs?
by Tony Rall 20 Feb '02

20 Feb '02

On Wednesday, 2002/02/20 at 09:08 PST, Randy Bush <randy(a)psg.com> wrote: > [0] - corollary: qos mechanisims decide which packets to drop. but isps > are paid not to drop any packets. Exactly. I've been saying this to vendors for the last few years, as they try to push their QOS mechanisms on me. I'm not in the business of trying to engineer some optimum packet discard strategy. I would rather spend my time and money trying to minimize the drop percentage. (I haven't been tested yet with the task of trying to minimize or standardize latency for traffic like VOIP - might change my tune if I was dealing with that.) Tony Rall

2 1

RE: Cisco PPP DS-3 limitations - 42.9Mbpbs?
by Goodwin, Dustin T [IT] 20 Feb '02

20 Feb '02

I think that page 10 says it all in regards to how RED can improve circuit utilization. ftp://ftp.ee.lbl.gov/talks/vj-nanog-red.pdf - Dustin - -----Original Message----- From: Steve Naslund [mailto:snaslund@interaccess.com] Sent: Wednesday, February 20, 2002 4:24 PM To: nanog(a)merit.edu Subject: RE: Cisco PPP DS-3 limitations - 42.9Mbpbs? > On Wed, 20 Feb 2002, Jon Mansey wrote: > > > OMG! Arent we missing the point here? What about never running > links above > > 60% or so to allow for bursts against the 5 min average, and <shudder> > > upgrading or adding capacity when we get too little headroom. > > > And here we are, nickel and diming over a few MBps near to 45M > on a DS3... > > And why not? Obviously there is a reason why they're not upgrading, > because there is plenty of traffic to fill up a second or faster circuit > if packets are being dropped because of congestion. (Which has not been > confirmed so far.) > > There shouldn't be any problems pushing a DS3 well beyond 99% utilization, > by the way. With an average packet size of 500 bytes and 98 packets in the > output queue on average, 99% only introduces a 9 ms delay. The extra RTT > will also slow TCP down, but not in such a brutal way as significant > numbers of lost packets will. Just use a queue size of 500 or so, and > enable (W)RED to throttle back TCP when there are large bursts. > One problem you have here is how you are getting the utilization statistics. Since you are looking at 5 minute averages, chances are real good that your instantaneous figure is probably at the full capacity of the DS-3. As you approach the maximum capacity and start dropping packets, your throughput on the line will bounce around near the 45 megs figure but the "goodput" that the customer sees will drop dramatically. You will be sending retransmissions of the dropped packet, which then causes less bandwidth to be available for current traffic, which causes more drops, which causes more retransmissions and backoffs. My experience in the real world is that once you get over 40 mbps on a DS-3 you need to look at upgrading. Another thing I would question is where the Cisco is counting traffic. Since it is most likely looking at the real user traffic there is more likely some overhead to manage the DS-3 itself and some L2 protocol stuff also. This is definitely a factor when running ATM because the router only counts the input/output traffic after the ATM overhead has been stripped away unless you are looking at the controller and not the interface. Remember the interface is just that, an interface to the line itself. You may be able to get more accurate data by looking at the controller stats because they operate at a lower level than the interface. Steven Naslund Network Engineering Manager Hosting.com - Chicago

1 0

Re: Looking for ISP in Beirut
by Steven G. Huter 20 Feb '02

20 Feb '02

> Hello all, I'm looking for an ISP in Beirut that has 200ms or less to > New York City (a tall order I know). Hello Rishi, There is a listing of the ISPs in Lebanon, maintained at http://www.mpt.gov.lb/isplist.htm You might try checking with the engineers, cc:d above, for Fiberlink Networks and TerraNet - I believe those two have fiber connections to the East Coast. Steve Huter

1 0

red [was Re: Cisco PPP DS-3 limitations - 42.9Mbpbs?]
by smd＠clock.org 20 Feb '02

20 Feb '02

randy bush writes: | you can turn on [x]red. it will make a better choice of which packets | to drop. but packets will still be dropped [0]. fsvo "better". red drops packets randomly, a little earlier than packets will be tail-dropped by being unable to find buffer space in a queue. in most cases, tail-drop is worst for TCP connections in slow-start, and leads to such fun things as web-page-slowdown-syndrome, or ssh-stall-syndrome, with the attendant hitting of the "reload" button or frustrated banging on the keyboard trying to get one's keys to echo back, or trying to unstick the page of text being sent to /dev/tty on the remote end. "r" for random: this spreads out loss randomly, and in most cases, this means that several different TCPs each observe a single lost segment, which is quickly detected and causes an attendant slow-down, but not a stall. "e" for early: we have to keep the queue from filling, because that leads to tail-drop, so we have to drop packets when we could really send them. there is no way around this -- in virtually any implementation, the mechanism to turn on red can be translated into more fifo-queue space. the trade-off (early random dropping) is usually worth it. there are some corner cases where there is insufficient ability to spread drops across enough flows, where resulting behaviour can be alot like tail-drop (a huge difference between fast input speed and slow output speed, with only a few bulk-transfer flows), but these cases are rare, particularly where average_segment_size / bottleneck_kbps <= 0.4, with the 0.4 being a historical minimum target for queueing in the presence of flows which experience long RTTs (like ones across oceans). in general, you don't have to worry about that, though. | you can increase buffers blah blah blah. but twenty tomatoes will still | not fit in a fifteen tomato can. yup, the trick is to deploy a red control law that drops just before tail-dropping would start, so we don't lose packets unnecessarily, yet early enough before tail-dropping would start that the senders have time to back off in response to the detected lost segment, which is driven by the RTT. drop too early, and you get an underused line. drop too late and you get tail-drop, which also gives you an underused line. drop just right, and you can keep your line very very very close to 100% for sustained periods of time (hours!), assuming a large enough set of TCP flows that random dropping spreads the early dropping across different flows. | [0] - corollary: qos mechanisims decide which packets to drop. but isps | are paid not to drop any packets. it is impossible to get 0 drop and enjoy statistical multiplexing gain. statmux gain keeps the network cheap enough to use. red minimizes the liklihood that anyone will complain about performance during those inevitable periods when one's statistical bet goes wrong. Sean.

1 0

RE: Cisco PPP DS-3 limitations - 42.9Mbpbs?
by Goodwin, Dustin T [IT] 20 Feb '02

20 Feb '02

Assuming this utilization figure is correct the other way to milk a higher utilization out of the link is by using Random Early Detect (RED) or Weighted RED. This will eliminate jagged utilization pattern you see in fully consumed link which is caused by TCP window synchronization. RED will give you a smoother utilization line that is closer to maximum theoretical throughput of the link. - Dustin - -----Original Message----- From: Ehud Gavron [mailto:gavron@wetwork.net] Sent: Wednesday, February 20, 2002 9:24 AM To: nanog(a)merit.edu Cc: GAVRON(a)Opus1.COM Subject: Cisco PPP DS-3 limitations - 42.9Mbpbs? At a recent consulting opportunity, a client was only getting 40.9Mbps on their DS-3 (clearchannel DS3 over OC12 to the IP provider). (PA-2T3 to PA-2T3, unloaded CPU, drops occur when peak octets out per sec/8=40.9M) The client asked us why 40.9 was less than 44.210. The IP provider indicated that "Cisco told us the maximum theoretical rate of an interface doing DS-3 with PPP encapsulation is 42.9." I can't find documentation to the 42.9. I can't find documentation that indicates that the PPP overhead is somehow magically excluded from the IfOctetsIn and Out... I can't find documentation as to why 40.9 is the observed rate. Frankly, all my DS-3s (HDLC encap) go up to 44.210, so I was hoping I could share in the collective wisdom of NANOG readers who have DS-3s to IP providers, and can provide real observed feedback. Please reply privately. Please let me know privately if you want a summary. Thanks! Ehud Gavron gavron(a)wetwork.net

1 0

Reviewer Request: "The Art of Peering - The Peering Playbook"
by William B. Norton 20 Feb '02

20 Feb '02

Hi folks - I walked about 15 ISP Network Architects and Peering Coordinators through this latest draft of the white paper "The Art of Peering : The Peering Playbook", and I now have presentable draft that I'd like to share with a slightly broader set of interested folks (preferably Network Architects and Peering Coordinators). Here is the Abstract: ------------------------------------------------------- snip ------------------------------------------------------ Internet Service Provider (ISP) Peering Coordinators were interviewed over the past few years for the "Interconnection Strategies for ISPs", "Internet Service Providers and Peering", and "A Business Case for Peering" Internet Operations Research papers. In these previous works, we documented the commonly used terminology (Peering, Transit, Transport, etc.), the motivations, the financial justifications and the process of peering. In "The Art of Peering" we build upon this foundation and present tactics that Peering Coordinators have used to obtain peering where they otherwise might not have been able to obtain peering. We have identified 17 specific maneuvers that vary from mundane to the clever, from merely deceptive to manipulative. In sum, these tactics represent the "Peering Playbook", the current "Art" of Peering. ----------------------------------------------------- snip ------------------------------------------------------- If you can help and have the inclination to read it (about 10 pages) and comment, please send me e-mail: wbn(a)equinix.com Thanks - Bill PS - Thanks to those who helped with the last paper "A Business Case for Peering". If I didn't send you the latest (version 1.3) send me e-mail and I'd be glad to email you a copy. BTW - The latest version of "Internet Service Providers and Peering" is 2.4 and that is available as well. ---------------------------------------------------------------------------- ------------------------------------------------- William B. Norton Co-Founder and Chief Technical Liaison Equinix wbn(a)equinix.com 650.315.8635

1 0