10 practical examples of the rsync command in Linux

s2 0

Rsync (Remote synchronization) is the most commonly used command for copying and synchronizing files and directories remotely as well locally in Linux/Unix Systems.

With the help of rsync Command, you can copy and synchronize your data remotely and locally via directories, hard drives and networks, perform data backups and mirror them between two Linux computers.

Local and remote file synchronization with Rsync

This article explains 10 basic and advanced use of the rsync Command to transfer your files remotely and locally to Linux based Machinery. You don’t have to be one root User to run the rsync Command.

Some advantages and features of the rsync command

  • It efficiently copies and synchronizes files to or from a remote system.
  • Support copying links, devices, owners, groups and permissions.
  • It’s faster than scp (Safe copy) there rsync uses a remote update protocol that only transfers the differences between two sets of files. The first time it copies the entire contents of a file or directory from the source to the destination, but the next time it only copies the changed blocks and bytes to the destination.
  • Rsync uses less bandwidth because it uses the compression and decompression method while data is being sent and received at both ends.
The basic syntax of the rsync command
# rsync options source destination
Some common options used with rsync commands
  • -v : detailed
  • -R : copies data recursively (but does not retain timestamps and permissions when transferring data.
  • -a : Archive mode that enables recursive copying of files and also preserves symbolic links, file permissions, user and group ownership, and timestamp.
  • -With : Compress file data.
  • -H : human readable issue numbers in a human readable format.

[ You might also like: How to Sync Files/Directories Using Rsync with Non-standard SSH Port ]

Install Rsync in the Linux system

We can install them rsync Package into your Linux distribution using the following command.

$ sudo apt-get install rsync   [On Debian/Ubuntu & Mint] 
$ pacman -S rsync              [On Arch Linux]
$ emerge sys-apps/rsync        [On Gentoo]
$ sudo dnf install rsync       [On Fedora/CentOS/RHEL and Rocky Linux/AlmaLinux]
$ sudo zypper install rsync    [On openSUSE]

1. Copy / synchronize files and directory locally

Copy / sync a file on a local computer

The following command synchronizes a single file on a local computer from one location to another location. Here in this example a file name backup.tar must be copied or synced with / tmp / backups / Folder.

[[email protected]]# rsync -zvh backup.tar.gz /tmp/backups/

created directory /tmp/backups
backup.tar.gz

sent 224.54K bytes  received 70 bytes  449.21K bytes/sec
total size is 224.40K  speedup is 1.00

In the example above, you can see that rsync will automatically create a directory for the target if the target doesn’t already exist.

Local rsync files
Local rsync files
Copy / synchronize a directory on the local computer

The following command transfers or synchronizes all files from one directory to another directory on the same computer. Here in this example, / root / rpmpkgs contains some RPM package files and you want this directory to be copied into it / tmp / backups / Folder.

[[email protected]]# rsync -avzh /root/rpmpkgs /tmp/backups/

sending incremental file list
rpmpkgs/
rpmpkgs/httpd-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm
rpmpkgs/mod_ssl-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm
rpmpkgs/nagios-4.4.6-4.el8.x86_64.rpm
rpmpkgs/nagios-plugins-2.3.3-5.el8.x86_64.rpm

sent 3.47M bytes  received 96 bytes  2.32M bytes/sec
total size is 3.74M  speedup is 1.08
Local rsync directory
Local rsync directory

2. Copying / synchronizing files and directories on or from a server

Copy a directory from the local server to a remote server

This command synchronizes a directory on a local computer with a remote computer. For example, there is a folder on your local computer “rpmpkgs”That includes some rotational speed Packages and you want the contents of this local directory to be sent to a remote server, you can use the following command.

[[email protected]:~]# rsync -avzh /root/rpmpkgs [email protected]:/root/

The authenticity of host '192.168.0.141 (192.168.0.141)' can't be established.
ED25519 key fingerprint is SHA256:bH2tiWQn4S5o6qmZhmtXcBROV5TU5H4t2C42QDEMx1c.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.0.141' (ED25519) to the list of known hosts.
[email protected]'s password: 
sending incremental file list
rpmpkgs/
rpmpkgs/httpd-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm
rpmpkgs/mod_ssl-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm
rpmpkgs/nagios-4.4.6-4.el8.x86_64.rpm
rpmpkgs/nagios-plugins-2.3.3-5.el8.x86_64.rpm

sent 3.74M bytes  received 96 bytes  439.88K bytes/sec
total size is 3.74M  speedup is 1.00
Rsync directory remote system
Rsync directory remote system
Copy / synchronize a remote directory with a local computer

This command helps you synchronize a remote directory with a local directory. Here in this example a directory / root / rpmpkgs what is on a remote server is copied into to your local computer / tmp / myrpms.

[[email protected]:~]# rsync -avzh [email protected]:/root/rpmpkgs /tmp/myrpms

[email protected]'s password: 
receiving incremental file list
created directory /tmp/myrpms
rpmpkgs/
rpmpkgs/httpd-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm
rpmpkgs/mod_ssl-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm
rpmpkgs/nagios-4.4.6-4.el8.x86_64.rpm
rpmpkgs/nagios-plugins-2.3.3-5.el8.x86_64.rpm

sent 104 bytes  received 3.49M bytes  997.68K bytes/sec
total size is 3.74M  speedup is 1.07
Rsync remote directory to local
Rsync remote directory to local

3. Rsync over SSH

With rsync, we can use SSH (Safe shell) for data transfer with SSH Protocol while our data is being transmitted, you can be sure that your data is transmitted in a secure connection with encryption, so that no one can read your data while it is being transmitted over the Internet.

[ You might also like: How to Secure and Harden OpenSSH Server ]

Even if we use rsync we have to provide that user/root Password to do this particular task so use that SSH Option sends your logins encrypted so that your password will be safe.

Copy a file from a remote server to a local server using SSH

How to specify a protocol with rsync You have to “-e”Option with the protocol name you want to use. Here in this example we use the “ssh” with the “-e”Option and perform the data transfer.

[[email protected]:~]# rsync -avzhe ssh [email protected]:/root/anaconda-ks.cfg /tmp

[email protected]'s password: 
receiving incremental file list
anaconda-ks.cfg

sent 43 bytes  received 1.10K bytes  325.43 bytes/sec
total size is 1.90K  speedup is 1.67
Copy Rsync remote file to local
Copy Rsync remote file to local
Copy a file from a local server to a remote server using SSH
[[email protected]:~]# rsync -avzhe ssh backup.tar.gz [email protected]:/backups/

[email protected]'s password: 
sending incremental file list
created directory /backups
backup.tar.gz

sent 224.59K bytes  received 66 bytes  64.19K bytes/sec
total size is 224.40K  speedup is 1.00
Rsync copy local file to remote
Rsync copy local file to remote

[ You might also like: How to Use Rsync to Sync New or Changed/Modified Files in Linux ]

4. Show progress when transferring data with rsync

To see the progress in transferring the data from one computer to another, we can use the ‘-Progress’ Opportunity. It shows the files and the time remaining until the transfer is complete.

[[email protected]:/]# rsync -avzhe ssh --progress /root/rpmpkgs [email protected]:/root/rpmpkgs

[email protected]'s password: 
sending incremental file list
rpmpkgs/
rpmpkgs/httpd-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm
          1.47M 100%   31.80MB/s    0:00:00 (xfr#1, to-chk=3/5)
rpmpkgs/mod_ssl-2.4.37-40.module_el8.5.0+852+0aafc63b.x86_64.rpm
        138.01K 100%    2.69MB/s    0:00:00 (xfr#2, to-chk=2/5)
rpmpkgs/nagios-4.4.6-4.el8.x86_64.rpm
          2.01M 100%   18.45MB/s    0:00:00 (xfr#3, to-chk=1/5)
rpmpkgs/nagios-plugins-2.3.3-5.el8.x86_64.rpm
        120.48K 100%    1.04MB/s    0:00:00 (xfr#4, to-chk=0/5)

sent 3.74M bytes  received 96 bytes  1.50M bytes/sec
total size is 3.74M  speedup is 1.00
Rsync progress while copying files
Rsync progress while copying files

5. Use of –include and –exclude options

With these two options we can contain and exclude Files by specifying parameters with this option helps us identify the files or directories that you want to include in your synchronization and exclude files and folders that should not be transferred.

In this example, the rsync command only contains the files and directory that start with ‘R.‘and exclude all other files and directories.

[[email protected]:/]# rsync -avze ssh --include 'R*' --exclude '*' [email protected]:/var/lib/rpm/ /root/rpm

[email protected]'s password: 
receiving incremental file list
created directory /root/rpm
./
Requirename

sent 61 bytes  received 273,074 bytes  60,696.67 bytes/sec
total size is 761,856  speedup is 2.79
Rsync include and exclude files
Rsync include and exclude files

6. Using the delete option

If a file or directory does not exist at the source but already exists on the destination, you may want to delete that existing file / directory on the destination during synchronization.

We can the ‘-Clear‘to delete files that are not in the source directory.

Source and destination are synchronous. Now create a new file test.txt at the Finish.

[[email protected]:~]# cd /root/rpm/
[[email protected]:~/rpm]# touch test.txt
[[email protected]:~/rpm]# rsync -avz --delete [email protected]:/var/lib/rpm/ /root/rpm/

[email protected]'s password: 
receiving incremental file list
deleting test.txt
./
.dbenv.lock
.rpm.lock
Basenames
Conflictname
Dirnames
Enhancename
Filetriggername
Group
Installtid
Name
Obsoletename
Packages
Providename
Sha1header
Sigmd5
Suggestname
Supplementname
Transfiletriggername
Triggername
__db.001
__db.002
__db.003

sent 445 bytes  received 18,543,954 bytes  2,472,586.53 bytes/sec
total size is 71,151,616  speedup is 3.84

The destination is the new file named test.txt, when synchronizing with the source with the ‘-Clear‘Option, the file has been removed test.txt.

Rsync delete option
Rsync delete option

7. Set the maximum size of the files to be transferred

You can specify Max File size to be transferred or synchronized. You can do it with “-maximum size“Possibility. In this example, the maximum file size is 200k, so this command will only transfer files that are equal to or less than 200k.

[[email protected]:~]# rsync -avzhe ssh --max-size="200k" /var/lib/rpm/ [email protected]:/root/tmprpm

[email protected]'s password: 
sending incremental file list
created directory /root/tmprpm
./
.dbenv.lock
.rpm.lock
Conflictname
Enhancename
Filetriggername
Group
Installtid
Name
Obsoletename
Recommendname
Requirename
Sha1header
Sigmd5
Suggestname
Supplementname
Transfiletriggername
Triggername
__db.002

sent 129.52K bytes  received 396 bytes  28.87K bytes/sec
total size is 71.15M  speedup is 547.66
Rsync Set the maximum file transfer size
Rsync Set the maximum file transfer size

8. Automatically delete source files after successful transfer

Suppose you have the main web server and a data backup server, you have made a daily backup and synchronized with your backup server. Now you don’t want to keep that local copy of the backup on your web server.

So do you want to wait for the transfer to complete and then manually delete that local backup file? Of course not. This automatic deletion can be done with the ‘–Remove source files‘ Opportunity.

[[email protected]:~]# rsync --remove-source-files -zvh backup.tar.gz [email protected]:/tmp/backups/

[email protected]'s password: 
backup.tar.gz

sent 795 bytes  received 2.33K bytes  894.29 bytes/sec
total size is 267.30K  speedup is 85.40

[[email protected]:~]# ls -l backup.tar.gz

ls: cannot access 'backup.tar.gz': No such file or directory
Delete Rsync source file after transfer
Delete Rsync source file after transfer

9. Do a test run with rsync

If you are new to rsync and don’t know what exactly your command will do. Rsync could really mess things up in your destination folder and then undoing it can be a tedious job.

[ You might also like: How to Sync Two Apache Web Servers/Websites Using Rsync ]

When you use this option, no changes are made to the files and the output from the command is displayed. If the output shows exactly how you want it, you can use the ‘-Test run‘Option from your command and run it on the terminal.

[[email protected]:~]# rsync --dry-run --remove-source-files -zvh backup.tar.gz [email protected]:/tmp/backups/

[email protected]'s password: 
backup.tar.gz

sent 50 bytes  received 19 bytes  19.71 bytes/sec
total size is 267.30K  speedup is 3,873.97 (DRY RUN)
Rsync test run
Rsync test run

10. Set the Rsync bandwidth limit and transfer the file

You can limit the bandwidth when transferring data from one computer to another using ‘–Bulimite‘ Opportunity. This option helps us to limit I / O Bandwidth.

[[email protected]]# rsync --bwlimit=100 -avzhe ssh  /var/lib/rpm/  [email protected]:/root/tmprpm/
[email protected]'s password:
sending incremental file list
sent 324 bytes  received 12 bytes  61.09 bytes/sec
total size is 38.08M  speedup is 113347.05

In addition, rsync only synchronizes changed blocks and bytes by default. If you want to explicitly sync the entire file, use the ‘-W‘Option with it.

[[email protected]]# rsync -zvhW backup.tar /tmp/backups/backup.tar
backup.tar
sent 14.71M bytes  received 31 bytes  3.27M bytes/sec
total size is 16.18M  speedup is 1.10

That’s all with rsync now, you can see man pages for more options. Stay connected with Tecmint for more exciting and interesting tutorials in the future. Let your Comments and suggestions.

Tags: