Date: Fri, 30 Aug 1996 15:59:23 -0700 (PDT) From: Michael Dillon <michael@memra.com> To: nanog@merit.edu Subject: Re: Agenda for next NANOG [...] What would it take to set up a couple of Alphas, routers and zero-mile T1's in a portable testing kit to go around from NAP to NAP and run tests? [...] Such machines could run a full suite of tests including FTP'ing large files, downloading complex web pages (i.e. multiple images) as well as the lower level things like ping tests.
This kind of test would provide useful numbers that customers can understand as well as point out problem areas that a NAP operator might need to investigate. And with co-operative NAP peers, the same test kit could be used with T1's out to various locations that feed into the NAP so as to run the same tests across the peer's routers and lines.
Would this kind of testing reveal any useful information that could not be gotten from examining router stats?
Actually, it would be nice if this "portable test kit" were actually an optional board for a router. You could, for example, stick the "portable test kit" board in a router and then test at will. We also tried to explain to our ATM tester vendors that we wanted ttcp implemented in the ATM tester. Some understood, some didn't... -tjs
In message <199608310000.TAA17929@uh.msc.edu>, Tim Salo writes:
Actually, it would be nice if this "portable test kit" were actually an optional board for a router. You could, for example, stick the "portable test kit" board in a router and then test at will.
Any of the ISPs losing packets like crazy need only look at the stats on their routers. We saw some creative lying at the last NANOG. I heard the statement "we have no loss on our links". No kidding. Now look on the routers. That is where congestion loss occurs and that is where loss due tolimitations in the router implementation will be recorded. The providers that are losing a lot of poackets can very easily look at these stats but they aren't about to put them on a viewgraph and bring them to NANOG.
We also tried to explain to our ATM tester vendors that we wanted ttcp implemented in the ATM tester. Some understood, some didn't...
-tjs
This is an interesting problem and actually close to solvable at the IP level. I'm sure you are familiar with PPP LQM. Take out the PPP part and keep the LQM packet format and local storage. Keep one LQM struct per ARP entry on a bcast or nbma interface (such as an ATM NAP). Count packets used by each ARP entry and update SNMP and the LQM entry as you would in PPP LQM. Occasionally (once a second is fine) send an LQM packet summarizing what has been sent. When you make two successive queries from the command interface one either side, you cat check the differences and get an exact (accurate to the packet) count of the packets sent during that period and the loss during that period. Then you just have to drive the network with whatever load you think appropriate. My understanding is that the ATM NAPs are not experiencing any level 2 loss at all so this is really not an issue for the NAPs. This clearly would be useful for ISPs using ATM or any switched service that doesn't absolutely guarentee no loss (ie: != VBR-V). Curtis
On Fri, 30 Aug 1996, Curtis Villamizar wrote:
Any of the ISPs losing packets like crazy need only look at the stats on their routers.
Maybe these ISP's don't know that there are stats on their routers or that there is packet loss there. As an NSP you would be doing your customers a service by telling them about such things. It doesn't help to call these people clueless idiots and suggest that they have no business running an ISP even if it is true. And even if an ISP monitors their router stats and sees peak traffic at around 60% of a T1 they could still have congestion problems elsewhere in their network such as their Ethernet or lines to POP's. All of this stuff contributes to public perceptions of a slow Internet and somebody has to take the first step to do end-to-end testing and point out where these trouble spots really are. Michael Dillon - ISP & Internet Consulting Memra Software Inc. - Fax: +1-604-546-3049 http://www.memra.com - E-mail: michael@memra.com
In message <Pine.BSI.3.93.960830205734.25896N-100000@sidhe.memra.com>, Michael Dillon writes:
And even if an ISP monitors their router stats and sees peak traffic at around 60% of a T1 they could still have congestion problems elsewhere in their network such as their Ethernet or lines to POP's. All of this stuff contributes to public perceptions of a slow Internet and somebody has to take the first step to do end-to-end testing and point out where these trouble spots really are.
I agree with you completely on this. I was talking about loss in T3 (and one OC3 that isn't all OC3) backbones. A congested serial link will drop packets but if it is the only bottleneck TCP will usually not push anything through the congested link twice. (Fast retransmit and fast recovery just retransmit what never went through the bottleneck in the first place). When you can't get anything anywhere near 28.8 on your 28.8, you get upset (or you don't get 56k on your 56k or T1 on your T1s). Curtis ps- I've copied as much as 600 MB and got about 160 KB/sec (1/2 T1) from Ann Arbor through the Elmsford T1 at mid day. That's reasonable performance. Not everyone can do that on their provider's backbone. Try it sometime. (Might want to go with less than 600 MB). I didn't tweak the code and open the TCP window or it might have been more (but folks in Elmsford might be slightly unhappy, even though it was TCP).
And with co-operative NAP peers, the same test kit could be used with T1's out to various locations that feed into the NAP so as to run the same tests across the peer's routers and lines.
Would this kind of testing reveal any useful information that could not be gotten from examining router stats?
Actually, it would be nice if this "portable test kit" were actually an optional board for a router. You could, for example, stick the "portable test kit" board in a router and then test at will.
It would be nice, but (IMHO) not likely unless alot of big customers of the router vendors asked for it. If I were an ISP engineer (*), I'd want to have my exchange provider provide a public router or diagnostic box (or both) connected to or near the level 2 switching fabric that would have the following features: router and diag boxes allow ICMP ping packets "I can't even ping _your_ box." router maintains peering sessions with each customer - Only back-end diagnostic net is advertised by the router. - Accepts all routes from all customers. - Connected and operator-owned nets are static. - Provides a BGP testing ground for the exchange point newbies. router allows loose-source routing For "traceroute -s" debuggung. router can have restricted logins for customers Look at your netowrk from outside your network just like CIX or route-server.cerf.net. This is very helpful for process of elimination for inter-provider routing problems. diag boxes (or even router) would be snmp-queryable "Wow, look at all the red around MAE-West. At least MFS's SJ router is still green, so we still have OK connectivity through our FDDI." diag boxes could monitor customers Ok, maybe not, unless perhaps the exchange operator had a NOC that cared if an ISP customer went down. diag boxes allow udp or tcp echo packets (real payload) "Real traffic through the exchange provider box doesn't seem to be lossy, perhaps that other ISP's router is out of CPU." This is all off the top of my head. Others could probably think up more. One diagram: Customer ISPs | | | +-+--+--+-+ | switch | +-+-------+ | | +-+----+ |router| +---+--+ | -+-+----+- switched ether or cddi/fddi | | +-+--+ +-+--+ |diag| |diag| +----+ +----+ Some might contend that the functions of the router and diagnostic box could be put into one (Unix?) box, possibly at relatively low cost. If an exchange-point had multiple switches connected by operator-owned lines (ala MAE-West or LA-NAP), a diagnostic router/box would be desired at each switching site. "See? There's no IP packet loss on the OC3 between the NAP ATM switches in LA and SF." Exchange operators have some really smart customers (ISP engineers) that could probably use more tools to diagnose inter-provider and exchange problems faster. Having the above-mentioned equipment at an exchange point not only helps dignose problems, they serve as a neutral test platform (ala Merit RA) for gathering private statistics about the health of the exchange point. BTW: Kudos go to CIX, CERFnet and Digex for allowing diagnostic access (more or less) to their routers. (*) Disclaimer: I'm not currently an IAP/ISP operator, just kibitzing. Many of you know better than I do what you want or need. I'm just offering an opinion that might be worth discussing. -- Eric Ziegast
participants (4)
-
Curtis Villamizar -
Eric Ziegast -
Michael Dillon -
salo＠msc.edu