Is it just me, or are the ANS commandos after me?
It's just you, really :-)
I don't think further discussion of ways of generating stats are terribly interesting. Needs are often defined in terms of available solutions, so you see a different need than I do.
That is in fact, the question. The ANS folks are curious about ways to generate stats, and because they've come from a position of having had a lot better stats than most of the rest of us, this is understandable. Further, they have a fair bit of experience on what can be done with a lot of data, and on interesting things to derive from that. It would behoove us NANOG participants to listen to what they have to say. There are really two discussions here that got merged together for a couple reasons, one being that of convenience: What to do about long term stats. What to do about short-term stats. The ping stuff is the short-term stats. It's certainly nice to be able to sample out packets when necessary, and can really help for tracking such things down. Others haven't mentioned it, but other ways to track down such flows (for those of us without wonderful instrumenting capabilities) include careful placement of host routes and/or tunnels to redirect traffic to points in the network where it can be more closely monitored, examination of IP ttls of such packets, etc., etc.
As for achieving some useful end result, several roads lead to Rome, or can be made to go there. The essential issue is not if knob X exists on box Y, but if there are enough knobs and instruments to let you do what you need.
It's true, but a discussion of what knobs HAVE EXISTED on boxes, and what knobs DO EXIST, and how to make those more standard across multiple vendors is a useful one, as long as it does not degenerate like it (almost) has.
One thing we have found much more useful than AS traffic matrices (which I have to admit has a certain trivia feel about them, although I'm the one who made the stuff) is live, X-based line load monitoring (humm; I made that too): each "interesting" (for some value of interesting) line has several 20-minute histograms for salient interface information, updated every 10 seconds, on one of four monitors, right here behind me.
This is the short-term, long-term issue. If you want to know who you exchange traffic with, who you need to consider peering with in more places or establishing additional links to, you need things like AS matrices. We certainly appreciate the modicum of data we have now, and would be happier with more of it. Knowing how hot each of your links is is nice, and may help you see short-term spikes, but it doesn't help long-term engineering of your network.
As noted, busy core routers are ill suited for collecting IP accounting. The fact that they may be border routers in BGP terms doesn't make them any less core routers from a network perspective. So you just have to rig things differently, then.
This is getting silly :-) It's relatively well-established that if you want to collect data somewhere, getting it right is going to be hard. The more you want to collect, the harder it is. Invariably it's useful to have stats on boxes at the borders of your network where you peer with other folks, and it's also useful inside your network. All of these things depend on what you're trying to engineer for, and almost all of them are useful. Balancing usefulness versus forwarding path performance is a tricky thing, but one should not assume it's impossible, and considering the possibilities is far more than a waste of time. --jhawk John Hawkinson