NO-LONGER-MAINTAINED dsbackup v0.1 ============= $Id: README 47 2004-05-18 14:48:54Z john $ Intro ===== dsbackup consists of a number of bash scripts to make backups of remote hosts. It's mainly designed for backing up hosts situated over low capacity network links and therefore aims to use as little bandwidth as possible. It's designed to make daily rsync "live" backups and then weekly or monthly archives. Currently the only backup method uses rsync over ssh but other methods (such as tar over ssh and smb) are planned (along with an extensible backup method system that should allow others to be added relatively easily. The daily live backups are just an rsync mirror of the remote hosts filesystem. The monthly archives are tarballs of this mirror. Unchanged rsynced files are hard linked in from the previous backup. So you see a full tree for each day without wasting disk space storing duplicated files. Archives are full copies of the latest backup taken using tar. Config ====== I recommend creating a /home/backup directory to contain all the backup configs and scripts. For rsync to preserve uids,gids and permissions on the local filesystem it must unfortunately run as root. Also, if any uids/gids collide on the backup server, those users may be able to access some of the backed up files. If we were just storing the backups in a tar archive this wouldn't be necessary, but we choose rsync to save bandwidth. I recommend having a dedicated backup server, with no users other than root. The general config file specified the dsbackup "home" directory e.g: "HOMEDIR=/home/backup" All scripts take a path to this config file as a first parameter (usually /home/backup/config/general/CONFIG) The general config file also specifies the dsbackup ssh key. For automated backups this should be an unpassworded ssh key, the public half of which is installed on each server required to be backed up. e.g: SSHKEY=/home/backup/config/dsbackup.key I recommend configuring ssh on the backup hosts to only allow rsync to be executed using this key. This can be a bit tricky and can depend on the arguments used on the rsync client side, so you may need to experiment. A new host backup configuration is created using the "newhostconfig.sh" scripts. This script creates all the directories and files needed to start configuring backups from this host. syntax: newhostconfig.sh [configfilename] [backupname] e.g: newhostconfig.sh config/general/CONFIG mywebserver This will create the following files and directories config/hosts/mywebserver/HOST config/hosts/mywebserver/EXCLUDE config/hosts/mywebserver/INCLUDE live/hosts/mywebserver archive/hosts/mywebserver logs/hosts/mywebserver The HOST file specified the ip address, backup method (only rsync currently), backup period (e.g: daily) and archive period (e.g: monthly). The config/general/BINS config file specifies direct paths to all the tools used by dsbackup. These work for me on my RedHat system. dsbackup relies on the following tools: ls, ln, tar, time, unlink, rsync, grep, touch, echo, du, stat, cut and find The scripts =========== Use the dsbackup-host.sh or dsarchive-host.sh scripts to backup or archive a single host syntax: dsbackup-host.sh [configfilename] [backupname] syntax: dsarchive-host.sh [configfilename] [backupname] e.g: dsbackup-host.sh config/general/CONFIG mywebserver Use the dsbackup.sh or dsarchive.sh scripts to backup or archive all hosts for the specified backup/archive period syntax: dsbackup.sh [configfilename] [period] syntax: dsarchive.sh [configfilename] [period] e.g: dsarchive config/general/CONFIG monthly backup period and archive period are actually arbitrary strings you can use to group hosts. You could specify BACKUP_PERIOD as "thirtymins" for a number of hosts and crontab "dsbackup.sh [config] thirtymins" to run every 30 minutes. Dsbackup currently only supports a minimum backup granularity of 1 day, so an rsync every 30 mins will just update the same days backups. To disable the period based backups, set the period to "never" Use the dsstat-host.sh script to get information about a configured backup host, such as the last time a backup or archive ran. syntax: dsstat-host.sh [configfilename] [backupname] Use the dsstat.sh script to get info on all configured hosts syntax: dsstat.sh [configfilename] It's easy to use egrep to get summaries from dsstat.sh: e.g: bin/dsstat.sh config/general/CONFIG | egrep "name:|at:" output: INFO: Backup name: webserver INFO: Last backup at: 2004-02-20 02:06:22.000000000 +0000 INFO: Last archive at: 2004-02-18 13:43:34.000000000 +0000 INFO: Backup name: mailserver INFO: Last backup at: 2004-02-20 10:26:23.000000000 +0000 INFO: Last archive at: 2003-12-12 17:02:42.000000000 +0000 Automated backup ================ Run these scripts using a Cron daemon to make backups and archives outside of working hours. add to /etc/crontab 0 2 * * * root /home/backup/bin/dsbackup.sh /home/backup/config/general/CONFIG daily 0 5 * * 1 root /home/backup/bin/dsarchive.sh /home/backup/config/general/CONFIG monthly and if you have any hourly backup periods configured: 0 * * * * root /home/backup/bin/dsbackup.sh /home/backup/config/general/CONFIG hourly