rsync-3-4-2

name: rsync-3-4-2 description: Complete toolkit for rsync 3.4.2, the fast file-copying tool using delta-transfer algorithm. Use when backing up files, mirroring directories, syncing data across hosts via SSH or rsync daemon, implementing incremental backups, configuring rsyncd.conf daemons, managing filter rules, or needing zstd compression and Y2038-safe timestamps.

rsync 3.4.2

Overview

Rsync is a fast and extraordinarily versatile file-copying tool. It can copy locally, to/from another host over any remote shell (typically SSH), or to/from a remote rsync daemon. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups, mirroring, and as an improved copy command for everyday use.

Rsync finds files that need to be transferred using a "quick check" algorithm (by default) that looks for files that have changed in size or in last-modified time. Any changes in other preserved attributes are made on the destination file directly when the quick check indicates that the file's data does not need to be updated.

Key features include:

Delta-transfer algorithm for efficient network usage
Support for copying links, devices, owners, groups, and permissions
Exclude and exclude-from options similar to GNU tar
CVS exclude mode for ignoring common build artifacts
Transparent remote shell support (SSH, rsh)
No super-user privileges required
Pipelining of file transfers to minimize latency
Anonymous or authenticated rsync daemons for mirroring
Incremental recursion for large file trees (reduced memory usage)
Batch mode for applying the same updates to many identical systems
Compression during transfer with configurable algorithms (zstd with multi-threading in 3.4.2)
ACL and extended attribute preservation

What's New in 3.4.2

Multi-threaded zstd compression — --compress=zstd now supports threading via --threads=N for faster compression on multi-core systems
Y2038-safe timestamps — fixed syscall timestamp handling to avoid overflow beyond 2038
C23 compatibility — bool keyword conflict resolved for C23 compilers
--dirs long option — added missing long-form alias for -d
Security fixes — reject negative token values in compressed stream, fix signed integer overflow in proxy protocol v2 header parsing, zero all new memory from alloc
ACL ID mapping — fixed ACL ID mapping for non-root users
xattr fixes — corrected count in qsort for xattrs, fixed handling of objects with many xattrs on FreeBSD
--open-noatime fix — option now works correctly on files (previously only directories)
Memory safety — fixed uninitialized pointer passed to memcpy, uninitialized mul_one in AVX2 checksum, invalid access to files array
Build improvements — reproducible builds, ia64 NonStop build fix, xattr support detection in both libc and -lattr

When to Use

Backing up files locally or across hosts
Mirroring directories (keeping source and destination in sync)
Syncing data over SSH between servers
Implementing incremental rotating backups with --link-dest or --backup-dir
Creating batch update files for bulk deployment to multiple identical systems
Setting up rsync daemons for anonymous or authenticated file distribution
Managing complex include/exclude filter rules for selective transfers
Moving files (using --remove-source-files)
Performing dry runs to preview what changes rsync would make

Core Concepts

Transfer Modes

Rsync operates in three modes:

Local copy — both source and destination are on the same machine:

rsync -av /src/dir/ /dest/dir/

Remote shell (pull) — pulling files from a remote host via SSH:

rsync -av user@remote:/src/dir/ /local/dest/

Remote shell (push) — pushing files to a remote host via SSH:

rsync -av /local/src/ user@remote:/dest/dir/

Rsync daemon (pull) — connecting directly to an rsync daemon:

rsync -av user@host::module/path /local/dest/
rsync -av rsync://user@host:873/module/path /local/dest/

Rsync daemon (push) — pushing to a daemon module:

rsync -av /local/src/ user@host::module/path

The Trailing Slash Rule

A trailing slash on the source changes behavior significantly:

rsync -av /src/foo /dest — copies the directory foo into /dest, creating /dest/foo
rsync -av /src/foo/ /dest — copies the contents of foo into /dest
Both set the attributes of the containing directory on the destination

For daemon connections, no trailing slash is needed to copy module contents:

rsync -av host::module /dest

Archive Mode (`-a`)

The -a (archive) flag is shorthand for -rlptgoD:

-r — recurse into directories
-l — copy symlinks as symlinks
-p — preserve permissions
-t — preserve modification times
-g — preserve group
-o — preserve owner (requires root on receiver)
-D — preserve devices and specials

Note: -a does not include -A (ACLs), -X (xattrs), -U (atimes), -N (crtimes), or -H (hard links). Add these explicitly when needed: rsync -aAXHN.

Delta-Transfer Algorithm

By default, rsync sends only the changed portions of files. This is controlled by the quick check algorithm, which compares file size and modification time. If both match, the file is skipped entirely. If they differ, rsync computes checksums on data blocks to identify exactly which portions need transferring.

Use --whole-file (-W) to disable delta-transfer and send files whole (faster for local copies or high-bandwidth connections where disk I/O is the bottleneck).

Quick Check vs. Checksum

Quick check (default): compares size and modification time — fast
Checksum (-c): computes full-file checksums — thorough but slow, uses lots of disk I/O

Always use -t (preserve times) or -a (archive mode) in your initial copy, otherwise rsync cannot effectively skip unchanged files on subsequent runs.

Usage Examples

Basic local copy

rsync -av /source/ /destination/

Remote sync over SSH

rsync -avz /local/data/ user@remote:/backup/data/

Mirror with deletions (use `--dry-run` first!)

rsync -av --delete /source/ /mirror/

Incremental rotating backup with hard links

DAY=$(date +%A)
rsync -a --link-dest=../current /source/ /backup/$DAY/
# Rotate:
mv /backup/current /backup/previous
ln -sfn "$DAY" /backup/current

7-day rotating incremental backup (from official examples)

#!/bin/sh
BDIR=/home/$USER
EXCLUDES=$HOME/cron/excludes
BSERVER=backup-host
export RSYNC_PASSWORD=XXXXXX

BACKUPDIR=$(date +%A)
OPTS="--force --ignore-errors --delete-excluded --exclude-from=$EXCLUDES \
      --delete --backup --backup-dir=/$BACKUPDIR -a"

# Clear last week's incremental directory
mkdir -p $HOME/emptydir
rsync --delete -a $HOME/emptydir/ $BSERVER::$USER/$BACKUPDIR/
rmdir $HOME/emptydir

# Actual transfer
rsync $OPTS $BDIR $BSERVER::$USER/current

Dry run to preview changes

rsync -ain /source/ /dest/

Bandwidth-limited transfer

rsync -avz --bwlimit=1000 /source/ user@remote:/dest/

Progress display with partial file resume

rsync -aP /large/source/ /dest/

Skip already-compressed files during compression

rsync -avz --skip-compress=gz/jpg/mp[34]/zip/bz2 /src/ /dst/

Multi-threaded zstd compression (new in 3.4.2)

rsync -av --compress=zstd --threads=4 /source/ user@remote:/dest/

Advanced Topics

Command-Line Options: Complete reference of all rsync options including transfer rules, delete modes, compression, and checksum algorithms → Command-Line Options

Filter Rules and Pattern Matching: Include/exclude patterns, merge files, per-directory filters, modifiers, anchoring, and interaction with deletions → Filter Rules

Rsync Daemon Configuration: rsyncd.conf parameters, module setup, authentication, access control, logging, SSL/TLS proxy, and daemon launch methods → Daemon Configuration

Batch Mode and Incremental Backups: Write-batch/read-batch for bulk deployment, --link-dest, --compare-dest, --copy-dest, --backup-dir, and atomic update patterns → Batch Mode and Backups

Symbolic Links, Security, and Troubleshooting: Symlink handling modes, multi-host security considerations, common error diagnostics, exit codes, and FAQ solutions → Security and Troubleshooting