Daily Remote Backups with Rsync

There are tons of pages on the web devoted to rsync scripts, but I had to pull together info from several sites to do what I wanted, so I'm adding another tutorial for a different basic use case.

My requirements:

• Hands-off/lights-out operation - no human interaction needed to run it.
• At least once a day, back up all changed files in a list of directories to a remote server.
• Be able to install it on multiple machines easily.
• Don't show a command window while backing up - it might scare my wife.
  
• Run on Windows XP Home.

My solution combines rsync, Cygwin/bash, a DOS batch file, and the Windows Task Scheduler. It runs whenever scheduled, in a minimized command window, and synchronizes an arbitrary list of configured directories to a specified root location on a remote server.

Install Cygwin from the above link. Install rsync and OpenSSH. I know, the installer is confusing, but it is free.

On your server, presumably running some flavor of Unix/Linux with rsync available (many online hosting sites that provide shell accounts make this part of their default setup), do the following:

ssh-keygen -t rsa #type Enter to accept the default file name/location
cd ~/.ssh
cat id_rsa.pub >> authorized_keys
chmod 600 authorized_keys

This allows any machine with the newly generated key to log into the current shell account on the remote server without a password. So guard the id_rsa and id_rsa.pub files carefully!

Copy the id_rsa and id_rsa.pub files to the "source" computer - the one with the files you want to back up. Put them in ~/.ssh. You can do this with scp, the ssh-based secure remote copy command in Bash/Cygwin:

mkdir ~/.ssh
scp [user]@[host.domain]:~/.ssh/id_rsa* ~/.ssh

Now you should be able to log in from the local computer to the remote server without entering the password:

ssh [user]@[host.domain]

Create a directory in the primary user's home directory (NOT My Documents, one level above, in c:\Documents and Settings\[username]).

In this directory, create a file backup.sh with the following contents:

#!/bin/bash
# DOS command: c:\cygwin\bin\bash.exe -c "~/backup/backup.sh"
export PATH=$PATH:/usr/bin/
rsync -arzve ssh --delete --no-p --no-g --chmod=u+rw,g+r,o-rwx,Dug+rwx --modify-window=1 --files-from ~/backup/files.txt ~/ [user]@[host.domain]:~/[server_backup_root_path]

then make it executable:
chmod a+x backup.sh

This is the heart of the backup. See the rsync man page for option descriptions. Hopefully the parts in green are self-explanatory - these are the pieces specific to your backup server.

I needed the chmod parameter for my hosting service - may not be needed for all servers. The --modify-window=1 parameter came from somewhere on the web, I can't find it now, and was important to seeing the right list of changed files.

The ~/backup/files.txt referenced above is just a plain text file with one relative path per line - relative to the path following the --files-from parameter, in this case the user home directory. The simplest case might be a files.txt simply containing:

My Documents/

If you want to back up your backup script, add "/backup" to the list. If you don't want all of your "My Documents" folder, you can prune the selections by specifying just specific subfolders:

My Documents/My Music/
My Documents/My Pictures/
My Documents/personal/

In all cases, the trailing slash is important - I didn't bother figuring out why, it just is, and that was all I needed to know.

Now the scheduling part. First, we need a DOS batch file we can run minimized. Create "backup.bat" with these contents:

title Backing up files...
C:\cygwin\bin\bash.exe -c "~/backup/backup.sh > ~/backup/backup.log 2>&1"

This is just a cryptic command line to start the bash script from a DOS command prompt, sending all output to backup.log.

Now create a shortcut in the backup directory to the backup.bat file. I called mine "backupShortcut". In it's properties, set it to run minimized.

Go to Control Panels > Scheduled Tasks. Right click the white space in the window, and select New > Scheduled Task. Rename it to "daily backup" or something.

In the properties for the new task, use the following on the "Task" tab:

Run: "c:\Documents and Settings\[user]\backup\backupShortcut.lnk"
Start in: "c:\Documents and Settings\[user]"

"Run as:" should be set to the current user. Click "Set Password..." to enter the user's login password. This lets the job run even if the user is not logged in or the computer is locked. Yes, this means you can't do this on a computer running Windows XP Home with no password on the primary account. But everyone should have a password, right?

Set the schedule to whatever you want. On my home computers, I do it at 2AM. On my work laptop, I do it at 2pm, because it is often suspended at night.

On the "Settings" tab, you may want to change the default length of time before the task is stopped, as the first backup may take a day or two, depending on your upload bandwidth. Comcast cable, with it's pokey 768K upload limit, can be really a dog the first time you back up pictures and music.

That should do it! Run the task manually to make sure a minimized window comes up and stays up. If there are problems, check the backup.log file for error messages.

I left out lots of DOS/bash basics, as this was a quick and dirty post to capture my work. I'll elaborate if anyone needs help.

Ehcache RMI Address Determination is Error Prone

I had to fix this problem with Ehcache at work. We use it with Hibernate in a clustered environment (more on that in another post). Recently we noticed that sometimes not all peers would discover each other.

The problem exists in all versions from 1.2.4 on, and is still unchanged in the current 1.5 release here, on line 166. Can you spot the problem?

return InetAddress.getLocalHost().getHostAddress()

Turns out Java doesn't specify exactly what this means on a machine with multiple network interfaces, and the result can vary between machines with similar hardware and operating system (as we found).

one of our cluster nodes had a setup that returned an IPv6 interface first, the loopback interface second, and an external IPv4 interface third. Java returned the loopback address from the above method.

Discussion of this Java issue can be found in many threads on the web. The solution wasn't available until java 1.4, but since almost no one has to use anything less any more, it should be available to most applications. You need to implement your own subclass of net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory overriding doCreateCachePeerListener to return a new instance of a subclass of net.sf.ehcache.distribution.RMICacheManagerPeerListener which overrides calculateHostAddress to discover the local host address in a more flexible manner. Register the factory class in your ehcache.xml configuration file.

The new code can look something like this:

Enumeration interfaces;
try {
interfaces = NetworkInterface.getNetworkInterfaces();
} catch (SocketException e) {
throw new UnknownHostException("Error getting network interfaces: ".concat(e.getLocalizedMessage()));
}

if (interfaces == null) {
throw new UnknownHostException("No network interfaces found");
}

InetAddress addrToUse = null;

while(interfaces.hasMoreElements()) {
NetworkInterface i = interfaces.nextElement();
Enumeration addresses = i.getInetAddresses();
if (addresses == null) continue;

while(addresses.hasMoreElements()) {
InetAddress a = addresses.nextElement();
if (addrToUse == null && ! a instanceof Inet6Address) addrToUse = a;
}
}
if (addrToUse == null) {
throw new UnknownHostException("No IPv4 non-loopback address found for any interface.");
}
return addrToUse.getHostAddress();

This returns the String representation of the first IPv4, non-loopback address found. The assumption here is that Java will always return the interfaces and addresses in the same order, given the machine network settings haven't changed.

If you want to fall back on the loopback address, if found, modify the above code accordingly. I'll leave that as an exercise for the reader (oops, my math background is showing).

With this updated host lookup code our app didn't need different configuration files for each instance, and all nodes automatically discovered each other, even between machines with multiple network interfaces, such as modern rack servers with multiple NICs and automatic failover between them.

Eventually I'll post about all the pieces we use to run a single Java web application in multiple Tomcat instances on multiple servers with Hibernate and an L2 Ehcache on each one, without the overhead of Tomcat clustering or a full-blown application server. All nodes sharing the same database automatically discover eachother and keep their cache contents in sync without any special configuration on individual nodes. Pretty cool.