rsync: copying files between servers

rsync copies files over SSH and, when you run it again, transfers only what has changed. If the connection drops, run the same command again and it will pick up where it left off. That makes it handy for moving a website or data to a new server and for making copies.

Installation

rsync is needed on both servers — the one you copy from and the one you copy to. If the command isn’t there:

sudo apt install rsync    # Ubuntu, Debian
sudo dnf install rsync    # AlmaLinux
sudo pkg install rsync    # FreeBSD

Copying a folder to another server

rsync -avP /var/www/example.co.uk/ ubuntu@203.0.113.10:/var/www/example.co.uk/

• -a — copy whole folders, keeping permissions, dates and links;
• -v — show what is being copied;
• -P — show progress; partly transferred files aren’t deleted if the connection drops, and are resumed.

To pull files from a server to your own machine, do the same thing the other way round:

rsync -avP ubuntu@203.0.113.10:/var/www/example.co.uk/ ~/backup/example.co.uk/

Check what will happen first

The -n option shows what rsync would do, without changing anything:

rsync -avn /var/www/example.co.uk/ ubuntu@203.0.113.10:/var/www/example.co.uk/

An exact copy: --delete

Normally rsync only adds and updates files. With --delete it also deletes anything in the destination that is no longer in the source, giving you an exact copy. If you mix up the source and destination, rsync will delete your newest files. So with --delete, always do a run with -n first:

rsync -avn --delete /var/www/example.co.uk/ ubuntu@203.0.113.10:/var/www/example.co.uk/
rsync -av --delete /var/www/example.co.uk/ ubuntu@203.0.113.10:/var/www/example.co.uk/

To skip things you don’t need to copy, use --exclude='cache/' or --exclude='*.log'.

A different SSH port and your own key

rsync -avP -e 'ssh -p 2222' /data/ ubuntu@203.0.113.10:/data/
rsync -avP -e 'ssh -i ~/.ssh/backup_key' /data/ ubuntu@203.0.113.10:/data/

Moving from an old server

A convenient order for moving a website or data:

1. Copy everything in advance, while the old server is still running. This is the longest part, and your site isn’t interrupted.
2. Before you switch over, stop anything on the old server that changes files (or put the site in maintenance mode) and run the same command again — it will transfer only the changes, which takes minutes.
3. Move the database by exporting it, not by copying files: MySQL on your own server, PostgreSQL on a cloud server.

Files that only root can read (/etc, other users’ home folders) can’t be read by an account such as ubuntu. Run rsync on the remote side through sudo, using the --rsync-path option:

rsync -avP --rsync-path="sudo rsync" ubuntu@198.51.100.20:/etc/nginx/ ~/old-nginx/

This works if sudo on that server doesn’t ask for a password — which is how the administrator account in our images is set up. After copying, check the owners: users may have different names on the new server, so fix them with the chown command.

scp — for a single file

For copying a couple of files, scp is simpler. It also works over SSH and is available wherever ssh is, including Windows:

scp dump.sql.gz ubuntu@203.0.113.10:~/
scp ubuntu@203.0.113.10:/var/log/nginx/error.log .
scp -P 2222 file ubuntu@203.0.113.10:~/     # the port is a capital P

scp can’t resume a transfer and doesn’t skip files that have already been copied — for large amounts of data, use rsync.

If rsync reports an error

• rsync: command not found or remote command not found — rsync isn’t installed on one of the servers.
• Permission denied (publickey) — the key or user name is wrong. Check that a plain ssh with the same options lets you in.
• Permission denied on individual files — the account has no permission to read or write them: see the section on moving from an old server.
• Exit code 23 at the end — some files weren’t copied; the reason is in the lines above.