- NANOG - lists.nanog.org

Re: not rewriting next-hop, pointing default, ...
by Sean M. Doran 13 Sep '97

13 Sep '97

Avi Freedman <freedman(a)netaxs.com> writes: > The danger with this approach is, obviously, that the router that you > try to do this to can go away. In which case you shoot yourself in the > foot. Some day, someone will send me a valid use of 'set ip next-hop' > but I haven't seen a good one yet. I believe initially it was dropped into route-maps to deal with situations involving having three different ASes share a broadcast LAN in Washington D.C. as an internal link. (AS 1239, AS 1790, both sharing a single IGP instance, AS 1800 with a different IGP instance, to be specific. Running two independent iIS-IS instances on a single wire was fun too.) FSVO "valid use", but it did make things work. Sean.

1 0

Re: not rewriting next-hop, pointing default, ...
by Sean M. Doran 13 Sep '97

13 Sep '97

"Alex.Bligh" <amb(a)xara.net> writes: > Failing this, the ability to disable responding to packets (*) > with source route set on the Cisco *host* TCP/IP stack > (and continue to forward them), Mourn the death of TUBA telnet... :) What you might want is to make sure that management functions can only happen over a separate private IP network. This has been a long-time engineering goal of one network at some priority or other. Then, some protection for routing protocols to make them both more robust and more secure, and life is a bit nicer. (Although taking an axe to all the routing protocols in use today has a strong appeal, actually, but that'll come later...) Unfortunately, though, in the absence of a method to query routers about their forwarding (i.e., "what would you do with this traffic profile?"), route calculation and NLRI redistribution policies, any tool which can help infer that from anywhere in the Internet is of use. I hate traceroute, I think it's a dreadful hack, and it is really hard to use it correctly for all sorts of reasons, lots of them having to do with the observer problem. LSRR helps enormously, and has been of critical use in the past. Killing it off to provide some warm fuzzies to operators who are still going to be exposed to lots of serious attacks on their routers and hosts strikes me as nearly as unreasonable as simply turning off routers and encasing them in concrete to make them safe. What would be REALLY nice is if lots of new hardware and software that doesn't keel over dead or use a really slow path to forward packets decorated with the LSRR option were deployed in everyone's networks. Sean.

1 0

Re: switching versus routing ATM redux
by Vadim Antonov 13 Sep '97

13 Sep '97

>> Who is Richard Irving? > >Tim Bass with a new spelling? Thanks, Sean. I'm curious if anybody knows why abundance of ellipsis in text is such a good indicator of TBism? You can make a five-liner bogon rejector based on the phenomenon, and in most cases it'll work wonderfully. --vadim PS Now, don't argue that bogon filtering is irrelevant to NANOG. Get a life :)

1 0

Re: switching versus routing ATM redux
by Sean M. Doran 13 Sep '97

13 Sep '97

Richard Irving <rirving(a)onecall.net> writes: > Who is Richard Irving? Tim Bass with a new spelling? > It is sort of an obsession of mine Wow, good guess, Sean. Sean.

1 0

Re: switching versus routing ATM redux
by Richard Irving 13 Sep '97

13 Sep '97

On Sat, 13 Sep 1997, William Allen Simpson wrote: > Wading thru the flood in the past two days, I was wondering why Carmen > Hillenburg <carmenh(a)onecall.net> is answering Richard Irving > <rirving(a)onecall.net> email? My significant other used my email while I was at work ;> > > Thus, ATM takes more bandwidth (is slower) and more cell switches (is > slower), cumulatively at every hop. Slower is slower is slower.... > Interesting note, I think all cross country timing data coming in is across ATM paths. Anyone have HSSI ? Regarding the deluge. It started when someone said, "Ok, what advantage does ATM have that IP doesn't" and I named 2. It started a Holy war. I had no Idea... Chris, I may never respond to you again.... ;) However, it is dragging to an end. *phew* We will know a fairly close representation of the ratio of overhead to cross the country, vs the speed of light ;). We have had input from some of the more interesting people in the field. And many interesting , yet *differing* views, yours is yet another. I would have stopped it long ago, but I felt obliged to at least answer. It is sort of an obsession of mine to make the data I *see*, correlate with the information the experts in the industry are offering...... Who is Richard Irving? No one! Just a tech like you, tasked with "The Net". Our company is an IXC, we run a hybrid-exchange (Giga-POP), and also carry our own ATM and IP circuits. (Routers, ATM Switches, IP Switches, and Telco:(ISDN/Pots) Switches.) We have a couple thousand T's around the country , (not all ip ;( ), Lots of "Hot glass" (up to OC48), and clean circuits. (Bandwidth Heaven) We buy/sell pipe from MFS, TWarner, Ameritech, etc.. And provide advanced communications to customers: U.S. based ATM, IP, LD, LDIP, and Frame. We function primarily as a bandwidth/advanced communications *wholesaler*, so you don't hear of us much. We are a C based shop, and also provide "Custom" Data services to the Carrier/Communications industry. Want a brochure? : mailto://techsale@onecall.net, or mailto://techsale@indyx.net as I am beginning to sound too much like a salesman for comfort. ;) No one, Just a tech who has to deal with so many technologies throughout the course of a week, that I begin to "bleed over" acronyms, and confuse everyone when I post ;) But, I am a member of NANOG, I think that counts! ;) Richard

1 0

Re: ATM (was Re: too many routes
by Richard Irving 13 Sep '97

13 Sep '97

On Fri, 12 Sep 1997, Vadim Antonov wrote: > Actually, 4000 miles is a very reasonable estimate for a cross-country > path physical length. That's why i took it as such without much > arguments. I am thinking it may be, we are getting together timings, and people are reporting the runs. The calculations are getting lower as we go. It is starting to look 30 ish. I agree this is not as significant as I thought it was going to be.... However, it is interesting to close in on. ;) > > > This implies that 39% of the actual timing is overhead. > > Hey, did you ever measure delay on a real cross-country fiber? > Did you compare that with pings between attached routers? > (Hint: i used to work for a long distance carrier). I do. And have for a while. That doesn't mean anything one way or another. Why don't we knock off the I am , I was, and I will be's and lets get the answer from data coming in. ;) This thread has been joined by some rather interesting people. All of which humble me, to be sure. (not that it is a difficult thing ;) The interesting thing is, most of these people are interested enough, that the data is coming in... I suspect, we may be able to get to a "reasonably" accurate ratio, but it should be interesting. Aren't you curious ? Or, do you have current data ? If so , post it. > > 74 ms you quoted is actally a nice RTT, for a loaded network > particularly. The first SprintLink's DS-3 between DC and > Stockton, CA had 80 ms (that's clearline, w/o any routers in the We have one down into the mid 60's now. Check the archive. > Now, would you care to explain how 0.3ms delay per router can > make things worse, considering that the average number of > _backbone_ hops for inter-provider trace is about 6? > At each juncture , layer 3, exists an opportunity for routing interaction. This is "for better" or "for worse". The better, is not what I am concerned with.... If indeed a router, participating in full tables, only introduces .3 ms delay, on a median, then the market bodes well.. But, I am concerned with the "distribution of the curve" so to speak. How -wide- is it.... How do we narrow it ? > --vadim >

1 0

The subject of being off the subject in the nanog list
by Bill Clark 13 Sep '97

13 Sep '97

With the thread that has been going around about being flamed for posting inappropriate subject matter on this list I believe I have an easy rule-of-thumb. Not saying by any means that I would try and impose my own beliefs on anyone or that mine are superior. Merely a suggestion to keep in mind. This list is not the following: Where I can I find this? Who can tell me what where this company peers? Why doesn't my router work? Anyone know if there is a patch for such and such IOS? Can you tell me how to add a route to my commodore 64? Politics What is the meaning of life? Now with that said, if someone were to ask hey anyone know where I can find gated? Then fine that's acceptable but 40 different people shouldn't post to address hey get it here at ftp.gated.com Why not send e-mail directly to that person. If somebody else says hey I am looking for that too, don't post it to nanog, send e-mail to the person who posed the question, hey if you get a good answer forward to me. By applying this practice we can eliminate the unnecessary e-mail and make life a little easier for yourself. It might be easier for you to post to the group and wait for some other people to post to the group but you are inconveniencing the rest of us, you want the benefits you bear the inconvenience of looking up that person's e-mail address and replying to them. Concept discussions are generally well received here, but hey my router keeps rebooting doesn't really need a thread of weeks to follow. Okay, I'll get off my soapbox now and prepare to get flamed myself. Bill Clark New Century Computer Solutions Network Engineer bclark(a)nccsinc.com

2 1

switching versus routing ATM redux
by William Allen Simpson 13 Sep '97

13 Sep '97

Wading thru the flood in the past two days, I was wondering why Carmen Hillenburg <carmenh(a)onecall.net> is answering Richard Irving <rirving(a)onecall.net> email? And why, with all the terminology being flung around, they don't realize that the term "Broadband ISDN" came years before "ATM"? The former is Engineering Speak, the latter Marketing Speak. FYI, they are confused. BRI ISDN is "baseband", not "narrowband". 10-base-2, 100-base-T, and gigabit ethernet are also "baseband". Cable TV channels (6 Mbps) are "broadband". Old 2400 bps satellite links are also "broadband". And why cannot either of them tell the difference between measured RTT of a ping and measured RTT using a real IP hop measurement tool, such as is available from LBL? And why there is any argument about forwarding latency, since by definition the last bit of the last ATM cell is not going to arrive any sooner than the last bit of a packet containing any IP datagram? Measuring the first bit arrival of cut-thru-switching is a waste of time. Processing of the IP datagram at the destination will not occur any sooner than the last bit.... And we all know that the task switch in the source and destination dominates the end-to-end timing (especially of non-fast-path ICMP messages like ping). A good ATM switch will buffer the entire packet anyway, to avoid wasting downstream bandwidth when one of the cells is dropped. No different than a router. By definition, there is much more link overhead (at least 2 ATM headers) and forwarding overhead (at least 2 ATM cell switches per SDH hop) for an IP/ATM/SDH datagram than an IP/PPP/SDH datagram (1 small header and 1 packet switch per SDH hop). Thus, ATM takes more bandwidth (is slower) and more cell switches (is slower), cumulatively at every hop. Slower is slower is slower.... And, on another thread, I was pretty sure that NetStar cum Ascend had a 622 Mbps OC-12c (STM-4) IP/PPP/SDH card long before Cisco. They were engineering it nearly 2 years ago. Did they not sell any? At the time, both Cisco and NetStar had very talented engineers on the OC-3 and OC-12 projects (and I'm not easily impressed), but the Cisco folks were hampered by the speed of their backplane and buffering on their card. NetStar had the backplane bandwidth and plenty of buffering, but had routing software instability. Reports on this list were that they were fixed. WSimpson(a)UMich.edu Key fingerprint = 17 40 5E 67 15 6F 31 26 DD 0D B9 9B 6A 15 2C 32 BSimpson(a)MorningStar.com Key fingerprint = 2E 07 23 03 C5 62 70 D3 59 B1 4F 5E 1D C2 C1 A2

1 0

Re: ATM (was Re: too many routes)
by Bill Woodcock 13 Sep '97

13 Sep '97

> hint 1 -- they took out the trash Ah, _cinematic_ references. :-) My mother? Let me tell you about my mother! And I have a friend at Apple who was responsible for performance tuning in the microkernel, which largely consisted of fixing other people's code, whose title (on his business card and everything) was Victor the Cleaner. -Bill

1 0

Re: ATM (was Re: too many routes)
by Richard Irving 12 Sep '97

12 Sep '97

Sean M. Doran wrote: > > Cool, I love being talked down to by old guys. It's > refreshing and doesn't happen nearly frequently enough. I am not that old. And I did not intentionally talk down, I was tired and grumpy, I am sorry if it came out that way. But, I would love to respond to the technical points in your extremely "learn-ed" email. ( That is a genuine compliment, not sarcasm. ) Now remember, I am argueing for ATM, not about the natural color of my hair. Or, the speed of light when chasing a south bound sparrow. ;) > PDH is dead. Uhhh.... > End-to-end POTS is already dying. Worldcom is making a > big deal over relatively simple technology which shuffles > fax traffic over the Internet. Yeah, and they are pushing it around over ATM. > However, there are neat > plans for SS7 and neater plans for doing clever things > with interpreting DTMF. Hey, did you ever notice that ATM is called broadband ISDN, and regular ISDN is called narrow band.. And that SS7 is designed to eventually intermesh these two technologies? This is still not an argument: CON ATM. This is not intended to be flippant.... ?sp? > In this model POTS and historical > telco voice and data schemes become services rather than > infrastructure. Yes, but the end user, in most cases, will still get POTS, as he knew it. And don't argue for ISDN... It is far easier to push voice across a clear channel T1 (Over ATM) using ISDN, than it is to map the RBS onto the stuff. But I agree completely. > The reason that you see "strange" or at least "unsmooth" > load balancing along parallel paths is that except with > fib and slow switching cisco had always forwarded packets > towards the same destination out the same interface, and > load balancing was performed by assigning (upon a cache > fault) particular destinations to particular interfaces. > > (Leon was invented to blow away cached entries so that > over time prefixes would slosh about from one interface to > another as they were re-demand-filled into the cache. > So ,you are saying, it is only Cisco who demonstrates this variability ? Can anyone back that ? > Points if you know who Leon and Viktor are. Hint: they're > both as dead as PDH.) This reminds me of Englands Queen. While "PDH may be dead", it is, and will be, emulated for a long time to come. So, "The Queen is dead, long live the Queen" ;) > > With CEF these days you can load-balance on a per-packet > basis. This has the side effect that you cannot guarantee > that packets will remain in sequence if the one way delay > across the load-balanced paths is off by more than about > half a packet transmission time. However, you also get > much more even link utilization and no ugly > cache/uncache/recache at frequent intervals (which really > sucks because unfortunately you have to push a packet > through the slow path at every recache). > I can't ever remember a sequencing issue over parallel ATM paths... Perhaps I am just not experienced enough ;> > So anyway, as I was saying, I'm ignorant about such > matters. > Hardly. > >Audio sounds great with lots of variability > > So if you aren't holding a full-duplex human-to-human > conversation you introduce delay on the receiver side > proportional to something like the 95th percentile and > throw away outliers. If you're holding a full-duplex > long-distance human-to-human conversation you can use POTS > (which is dying but which will live on in emulation) and > pay lots of money or you can use one of a number of rather > clever VON member packages and pay alot less money but put > up with little nagging problems. For local or toll-free > stuff, to expect better price-performance from an end-user > perspective now would require taking enormous doses of > reality-altering drugs. > All is quiet in the BoardRoom, the hostile merger is about to occur. All the Executives stand pensive, awaiting the last second instructions from the CEO, in germany. " And now, whatever you do, don't forget the *snap*, *crackle*, *pop*, or it will cost us millions, Oops! gotta run, ta. (click)" I am certainly not saying voice over IP is not viable... for the right situations. Matter of Fact, voice over IP, over ATM, is pretty solid ;) > Soft real time things can be implemented across a wide > variety of unpredictable media depending on the window > available to service the real time events and the slope of > the utility decay function. > > For instance, interactive voice and video have a number of > milliseconds leeway before a human audience will notice > lag. Inducing a delay to avoid missing the end of the > optimal window for receiving in-sequence frames or blobs > of compressed voice data is wise engineering, particularly > if the induced delay is adjusted to avoid it itself > leading to loss of data utility. > So , what do you think CDVT is all about ;) (I know , you already knew this, I just couldn't resist )I am still looking for the CON ATM in this point.... > > However, I have NEVER failed to get the bandwith > > "promised" in our nets. > > Sure, the problem is with mixing TCP and other window-based > congestion control schemes which rely on implicit feedback > with a rate-based congestion control scheme, particularly > when it relies on explicit feedback. The problem is > exacerbated when the former overlaps the latter, such that > only a part of the path between transmitter and receiver > is congestion controlled by the same rate-based explicit > feedback mechanism. > > What happens is that in the presence of transient > congestion unless timing is very tightly synchronized > (Van Jacobson has some really entertaining rants about > this) the "outer loop" will react by either hovering > around the equivalent of the CIR or by filling the pipe > until the rate based mechanism induces queue drops. > In easily observable pathological cases there is a stair > step or vacillation effect resembling an old TCP sawtooth > pattern rather than the much nicer patterns you get from a > modern TCP with FT/FR/1321 stamps/SACK. > > In other words your goodput suffers dramatically. Good point. However, FGCRA helps, graceful drops and all that... And yes, it will be a while before the entire mechanism is completely tied together. Oh, never mind, you make my point in the following. And remember, I specifically noted that legacy equipment does not count. > > > But, doesn't that same thing happen when you over-run the receiving > > router ????? > > Yes, and with OFRV's older equipment the lack of decent > buffering (where decent output buffering is, per port, > roughly the bandwidth x delay product across the network) > was obvious as bandwidth * delay products increased. > > With this now fixed in modern equipment and WRED > available, the implicit feedback is not so much dropped > packets as delayed ACKs, which leads to a much nicer > subtractive slow-down by the transmitter, rather than a > multiplicative backing off. > BYTW: I was arguing for ATM, not just ABR. But, the key here is to provide a reliable method whereby ATM can convey information in the same pro-active strategy. My original point is that in the ATM exchange environment, it is far less likely that an irresponsible neighbor will interrupt your service. This has scaling capacity .... And also a side effect I really like, only the ones who are abusing their lines, drop packets. (Usually...) > Finally another ABR demon is in the decay of the rate at > which a VS is allowed to send traffic, which in the face > of bursty traffic (as one tends to see with most TCP-based > protocols) throttles goodput rather dramatically. Having > to wait an RTT before an RM cell returns tends to produce > unfortunate effects, and the patch around this is to try > to adjust the scr contract to some decent but low value > and assure that there is enough buffering to allow a VS's > burst to wait to be serviced and hope that this doesn't > worsen the bursty pattern by bunching up alot of data > until an RM returns allowing the queue to drain suddenly. Or, just don't overaggregate to the Nth degree. ;\ Patient: Doctor, Doctor, it hurts when I do this..... Doctor: So, don't do that. That will be 50 dollars. ;) However, you have a very good point regarding the effect of delay awaiting the RM to indicate congestion. The solution does not exist, yet.... ATM is not without design considerations. *Nothing* is a substitute for good planning. It is easier to avoid bottlenecks when the top circuit is 622mbs, and you can run parallel paths. > > > Ahhh.. We await the completion, and proper interaction of RM, ILMI, > > and OAM. > > These will, (and in some cases already DO), provide that information > > back to the router/tag switch. > > Now do they use it well ????? > > That is a different story.... > > The problem is that you need the source to slow > transmission down, and the only mechanism to do that is to > delay ACKs or induce packet drops. Even translating > FECN/BECN into source quench or a drop close to the source > is unhelpful since the data in flight will already lead to > feedback which will slow down the source. Agreed, it is not currently as pro-active as an ideal situation would be. Weaknesses still exist providing layer 2 information to the layer 3 devices. The temporary fix is to provide large ingress and egress buffers, throughout the path, and provide as much end-to-end feedback as possible, preferably with the ability for all devices in a path to react to that feedback. OAM, RM, etc. Then to do as you say, set a reasonable SCR, and hope you bought enough buffers, and the CDVT is adequate to prevent loss. > > The congestion control schemes are essentially > fundamentally incompatible. > Lets just say "not optimal". > > > > Therefore, unless ABR > > > is deliberately inducing queueing delays, there is no way > > > your delay can be decreased when you send lots of traffic > > > unless the ATM people have found a way to accelerate > > > photons given enough pressure in the queues. > > > > > More available bandwidth = quicker transmission. > > I still say lets switch to tachyons ;) (Just for those who don't know already, I am not serious) > > 6 core2-fddi3-0.san-francisco.yourtransit.net (-.174.56.2) 567 ms > > 154 ms > > 292 ms > > > > >>>>>>>>>>>>>> Tell me this is a speed of light issue. > > >>>>>>>>>>>>>> From the FDDI to the HSSI on the same router. > > This has nothing to do with the router's switching or > route lookup mechanism. Router requirements allow routers > to be selective in generating ICMP messages, and cisco's > implementation on non-CEF routers will hand the task of > generating ICMP time exceededs, port unreachables and echo > replies to the main processor, which gets to the task as a > low priority when it's good and ready. If the processor > is doing anything else at the time you get rather long > delays in replies, and if it's busy enough to start doing > SPD you get nothing. I have noted the conversations regarding this. However, I should note that according to the data we collected, there is a direct correlation between the timings increasing, and the variability/latency/loss of packets travelling the path. My argument is based on our charting of timings of this nature, and the results from BoardWatch reviews. Our timing charts matched almost perfectly to boardwatches timing of major carriers. The lower the timings, and closer to minimal variability from these timings, the better the final score from boardwatch. Perhaps this relationship is merely demonstrating that, indeed, ICMP timing is "retarded" during periods of high router utilization, due to prioritizing of processes. However, the fact that this prioritization is being "kicked in" proves to be a good indicator that a router is experiencing a serious load. And routers experiencing heavy loads, are *usually* the cuplrit in packet delay/variance/loss. Not conclusive enough for the courts, but quite informational , none-the-less. > > Flow switching does a route determination once per flow, after that > > the packets are switched down a predetermined path "The Flow". Hence the > > term "flow switching". This reduces the variability of > > the entire flow. > > Um, no it doesn't. As with all demand-cached forwarding > schemes you have to process a packet heavily when you have > a cache miss. Darren Kerr did some really neat things to > make it less disgusting than previous demand-cached > switching schemes emanating out of OFRV, particularly > with respect to gleaning lots of useful information out of > the side-effects of a hand-tuned fast path that was > designed to account for all the header processing one > could expect. I have NO excuse for simplifying, other than a lack of need to get more detailed. I am now a little more versed in my intended audience.... (Postmortem) However, the concept still holds true. The flow switching is "in essence" demonstrating the advantage of minimizing interim processing. We have collected similar data, as well. (Hand tuned) BTW: Everyone seem to have missed that flow switching , in its current incarnation, does not scale well. Sort of like ethernet, there is a critical mass point. I was surprised no one attacked with that.. (not that it was important to the issue, or anything) > > Flow switching does magic matching of related packets to > cache entries which describe the disposition of the packet > that in previous caching schemes could only be determined > by processing individual packets to see if they matched > various access lists and the like. It's principal neat > feature is that less per-packet processing means more pps > throughput. > > MPLS is conceptually related, btw. > > Flow switching does not improve queueing delays or speed > up photons and electrons, however, nor does it worsen > them, therefore the effect of flow switching on > variability of normal traffic is nil. Hold it.... doesn't a better PPS throughput imply reduced que latency ? Not the physical process of entering the que, or the physical process of exiting the que, but the median time the data is "que homed". (again, full latency of ingress to egress) Kind of "by definition" ? The more packets that you can clear, the less congested the pipe. The less congested a pipe, lower the probability that a packet will need to experience delay. Or, have I over simplified ? Or, is your operational word "normal traffic"? That would make sense. > > Flow switching has mechanisms cleverer than Leon the > Cleaner to delete entries from the cache and consequently > there are much reduced odds of a cache fault during a > long-lived flow that is constantly sending at least > occasional traffic. You may see this as reducing > variability. I see it as fixing a openly-admitted design flaw. > Ok. But, it still works. Reduce the time spent processing the packet/or cell, and things get smoother... Why should it be otherwise? > > PS MAC Layer Switching, and ATM switching are apples and oranges. > > Although, one could be used to do the other. > > (Told you Dorian) > > Huh? Sorry, A response to another users mail. They referenced a holy war in the archives. I derived from that thread that it was an argument about traditional LAN switches, and their MAC based flow descisions -versus- Layer 3 Routing descisions. I guess technically speaking, it was related. But,it was not the immediate issue. I guess in the picture of things I created flame bait. I claim naivette' (sp ?) to the NANOG archives . However, I found the discussion refreshing. It was a pleasure ;) Richard

1 0