On Wed, Jan 31, 2001, Matthew Zito wrote:
No, no, don't do that. Given the scale of something like this, I'd expect you'd be running on something like Oracle that supports the concept of "hot backups". The table spaces are put into a quiesced state, and all writes are done to memory and to recovery logs. Once the backup is finished, you take it out of hot backup and it then writes all the pending transactions to the database files. That way, the database files are stable, and you also back up the recovery logs to something with real-time access (like another nfs server). In the event you have a catastrophic database failuser, you recover from tape (or if you have the space, you have a copy of the dbf files elsewhere), and run all the transaction logs - it takes about 5 minutes per hour of transactions. Then your database is brought up to the point where it was when it died. The worst case scenario is that there's a few transactions that don't get logged, which means that a few emails get dropped. If you had a stock smtp server that died, you could be looking at the same situation.
.. and then you have to make sure that you periodically garbage collect your local store, lest you end up with a whole bunch of files which are unreferenced and just take up space. :)
As far as backing up the actual mailboxes, there's no way to get around the fact that it'll take long enough to finish that stuff will be inaccurate by the time its finished. If you ever have to restore the mailboxes from tape without restoring the database, it'd be wise to have an application that builds a list of the messages that are on disk the database doesn't know about.
At least two commercial filesystems support "snapshots" - AdvFS and WALF (NetApp). I don't remember if XFS supports snapshots. Oh, FreeBSD's FFS has got snapshot capabilities but its not yet useful in a "real world" scenario. Adrian -- Adrian Chadd "Sex Change: a simple job of outside <adrian@creative.net.au> to inside plumbing." - Some random movie