I have a script that I’m reasonably happy with. It makes backups anytime the manager database changes and sends it off the server to a remote location. My test is less than ideal, but it works: I hash the database every half an hour. If the hash is different, send it off. Crude*, but effective. (I also keep a log of the hashes). The fact that the hash in the next half an hour is the same as the previous one has me assuming no changes occurred to the database while it was being copied (I hash both ends).
An Electrical Engineer friend of mine insists I have to shut manager down before making the backup, and whilst I would generally agree with him, that’s a bit of a PITA if I happen to be using it (and I want to keep it automated). But TBH, in this case, I don’t think I really need to stop manager, do I? Would anyone else bother shutting it down? Do you consider it should be?
My thinking is, it doesn’t need to be, but I just can’t satisfy myself of that. It will depend on the underlying filesystem and what happens when the copy (rsync in my case) command is issued, if the file is stored in memory or just on the drive, are transactions immediately written to disk, etc. I don’t believe the file is locked for rsync, but I’m not sure.
I think the fact the manager datafile is an archive, I assume the database may be kept in memory and that copying the file would in fact be ok. I believe that arbitrarily copying the database should be relatively fine. How can I be absolutely sure that what is being copied out is (for want of better terminology) “finalised and complete”? (I am only talking “to that point”, I don’t care if a change is made moments later, I’m concerned about file changes whilst being copied)
When the user presses the “Backup” button, manager supplies you a copy of the database as is, right now. However, is making a copy of the database while the server application is running is never going to be a “sure thing”, is it? (my belief here is the file that manager downloads in the backup is just a copy of the archive as it exists in the application directory, but is it locked while making that copy/backup?)
As mentioned, I’m reasonably confident I’m covered and safe.
@lubos is probably the best to answer this because he knows how the database transactions occur, disk writes, file locks etc but I’m curious as to anyone’s (especially a systems engineer’s) thoughts on this.
It would be easy enough to create a 4am cron job that does in fact stop the manager service, create the backup and restart the service. But I wouldn’t want to arbitrarily shut it down half a dozen or more times a day.
My file size is about 400MB. Bandwidth and disk space are not an issue.
* I really don’t like the fact I am consistently hashing the manager archive, but is there a better way? Do other automators just use time stamps?