Daily Remote Backups with Rsync

There are tons of pages on the web devoted to rsync scripts, but I had to pull together info from several sites to do what I wanted, so I'm adding another tutorial for a different basic use case.

My requirements:

• Hands-off/lights-out operation - no human interaction needed to run it.
• At least once a day, back up all changed files in a list of directories to a remote server.
• Be able to install it on multiple machines easily.
• Don't show a command window while backing up - it might scare my wife.
  
• Run on Windows XP Home.

My solution combines rsync, Cygwin/bash, a DOS batch file, and the Windows Task Scheduler. It runs whenever scheduled, in a minimized command window, and synchronizes an arbitrary list of configured directories to a specified root location on a remote server.

Install Cygwin from the above link. Install rsync and OpenSSH. I know, the installer is confusing, but it is free.

On your server, presumably running some flavor of Unix/Linux with rsync available (many online hosting sites that provide shell accounts make this part of their default setup), do the following:

ssh-keygen -t rsa #type Enter to accept the default file name/location
cd ~/.ssh
cat id_rsa.pub >> authorized_keys
chmod 600 authorized_keys

This allows any machine with the newly generated key to log into the current shell account on the remote server without a password. So guard the id_rsa and id_rsa.pub files carefully!

Copy the id_rsa and id_rsa.pub files to the "source" computer - the one with the files you want to back up. Put them in ~/.ssh. You can do this with scp, the ssh-based secure remote copy command in Bash/Cygwin:

mkdir ~/.ssh
scp [user]@[host.domain]:~/.ssh/id_rsa* ~/.ssh

Now you should be able to log in from the local computer to the remote server without entering the password:

ssh [user]@[host.domain]

Create a directory in the primary user's home directory (NOT My Documents, one level above, in c:\Documents and Settings\[username]).

In this directory, create a file backup.sh with the following contents:

#!/bin/bash
# DOS command: c:\cygwin\bin\bash.exe -c "~/backup/backup.sh"
export PATH=$PATH:/usr/bin/
rsync -arzve ssh --delete --no-p --no-g --chmod=u+rw,g+r,o-rwx,Dug+rwx --modify-window=1 --files-from ~/backup/files.txt ~/ [user]@[host.domain]:~/[server_backup_root_path]

then make it executable:
chmod a+x backup.sh

This is the heart of the backup. See the rsync man page for option descriptions. Hopefully the parts in green are self-explanatory - these are the pieces specific to your backup server.

I needed the chmod parameter for my hosting service - may not be needed for all servers. The --modify-window=1 parameter came from somewhere on the web, I can't find it now, and was important to seeing the right list of changed files.

The ~/backup/files.txt referenced above is just a plain text file with one relative path per line - relative to the path following the --files-from parameter, in this case the user home directory. The simplest case might be a files.txt simply containing:

My Documents/

If you want to back up your backup script, add "/backup" to the list. If you don't want all of your "My Documents" folder, you can prune the selections by specifying just specific subfolders:

My Documents/My Music/
My Documents/My Pictures/
My Documents/personal/

In all cases, the trailing slash is important - I didn't bother figuring out why, it just is, and that was all I needed to know.

Now the scheduling part. First, we need a DOS batch file we can run minimized. Create "backup.bat" with these contents:

title Backing up files...
C:\cygwin\bin\bash.exe -c "~/backup/backup.sh > ~/backup/backup.log 2>&1"

This is just a cryptic command line to start the bash script from a DOS command prompt, sending all output to backup.log.

Now create a shortcut in the backup directory to the backup.bat file. I called mine "backupShortcut". In it's properties, set it to run minimized.

Go to Control Panels > Scheduled Tasks. Right click the white space in the window, and select New > Scheduled Task. Rename it to "daily backup" or something.

In the properties for the new task, use the following on the "Task" tab:

Run: "c:\Documents and Settings\[user]\backup\backupShortcut.lnk"
Start in: "c:\Documents and Settings\[user]"

"Run as:" should be set to the current user. Click "Set Password..." to enter the user's login password. This lets the job run even if the user is not logged in or the computer is locked. Yes, this means you can't do this on a computer running Windows XP Home with no password on the primary account. But everyone should have a password, right?

Set the schedule to whatever you want. On my home computers, I do it at 2AM. On my work laptop, I do it at 2pm, because it is often suspended at night.

On the "Settings" tab, you may want to change the default length of time before the task is stopped, as the first backup may take a day or two, depending on your upload bandwidth. Comcast cable, with it's pokey 768K upload limit, can be really a dog the first time you back up pictures and music.

That should do it! Run the task manually to make sure a minimized window comes up and stays up. If there are problems, check the backup.log file for error messages.

I left out lots of DOS/bash basics, as this was a quick and dirty post to capture my work. I'll elaborate if anyone needs help.