On Wed, Aug 22, 2007 at 04:10:35PM -0700, Bill Nash wrote:
If you're going to roll your own: Even if only a contract basis, I'd recommend tapping the skillset of a good DBA or a professional network toolsmith for advice on organizing the sheer dearth of data involved here. Getting off on a good footing is the single most important part of a task like this, to minimize how much time you spend going back and redoing things because they didn't scale or simply don't apply generally enough.
I'd like to echo this comment, except that instead of just "DBA" I'd suggest _really strongly_ that you find someone experienced in relational database design if you're going to build such tools using a relational database (and I'll bet lunch you'll eventually put this into some database, if you start to collect data). You can spend a great deal of money on data gurus later to try to make your poorly-designed original data model work without interruptions; or else you can design it with good normalisation at the outset. Someone who has built a lot of MySQL-backed web sites (and nothing else) does _not_ qualify as a database designer. Real databases: ask, for instance, how a new data source will fit into the model if its fields are wrong, or subtly different, and how new classes of data are to be accommodated. My experience suggests to me that four hours of real, solid database design up front is worth approximately two weeks of re-engineering and compatibility development after about a year. This is no exaggeration: the problems of badly normalized data are really bad once you have a pile of data that you want to be able to rely on. (Of course, my data geek colleagues are happy to take the consulting fees later. But save yourself grief up front.) This is as true for network tools as anything else. A -- Andrew Sullivan 204-4141 Yonge Street Afilias Canada Toronto, Ontario Canada <andrew@ca.afilias.info> M2P 2A8 jabber: ajsaf@jabber.org +1 416 646 3304 x4110