Synchronize Your Data with rsync

As You Were

If you will be using rsync to create backups, it makes sense to keep the attributes of the original files. By attributes I mean permissions (read, write, execute, see the “Access Permissions” article) and timestamps – that is, information on the last access time (atime), the last status change (ctime), and the last modification (mtime).

Additionally, administrators can benefit from parameters that preserve owner and group data and support device files. To retain the permissions, just specify the ‑p option; ‑t handles the timestamps, and ‑g keeps the group membership.

Whereas any normal user can specify these parameters, the ‑o (keep the owner data) and ‑D (device attributes) flags are available only to root. The complete command line with all these options could look like this:

rsync ‑rlptgoD /home/huhn backup/

Don’t worry – you don’t have to remember all these options. Rsync offers a practical shortcut and a special option that combines these parameters for this case. Instead of ‑rlptgoD, just type ‑a.

Exclusive

Rsync has another practical option that allows you to exclude certain files from the synchronization process. To leverage this feature, specify the ‑ ‑exclude= option and a search pattern and define the files to exclude. With this option, you can use wildcards:

rsync ‑a ‑‑exclude=*.wav ~/music backup/

This example excludes large WAV files that end in .wav from the backup of a music collection. If you need to exclude MP3s as well, just append another exclude statement and a pattern:

rsync ‑a ‑‑exclude=*.wav ‑‑exclude=*.mp3 …

To save time, you can store your exclusions in a text file. To do this, you will need a separate line for each search pattern. Specify the ‑ ‑exclude‑from=file_with_exclusions parameter to parse the file.

Tidying Up

Rsync offers various parameters for deleting data that is no longer needed or wanted. To get rid of files in your backup that no longer exist in the source, type ‑ ‑delete. Rsync’s default behavior is to delete files before the transfer is finished. Alternatively, you can define ‑ ‑delete‑after to delete files of the target after all the syncing is done.

Additionally, you can tell rsync to delete files that you have excluded (see the previous section). For example, imagine you’ve decided that you no longer want the MP3s in the backup and you’ve started to exclude them with ‑ ‑exclude=*.mp3. Now you can define ‑ ‑delete‑excluded, and rsync will recognize that those files are no longer wanted.

All ‑ ‑delete options have basically the same goal: to keep an exact copy of the original. If you don’t use the switch, you will have to clean up manually; otherwise, the files that you’ve decided are useless will remain. Use these options with care (see the “Common Rsync Traps” box).

Tuning Rsync

Several options increase rsync’s performance. Often, I use the ‑z switch to compress data when I sync data over a network connection. Figure 1 shows this using Grsync, the graphical front end to rsync. If the connection is very slow, you can also define a bandwidth limit. To transfer data with only 20KBps, for example, use:

rsync ... ‑‑bwlimit=20