Expert Tips for Efficiently Safeguarding Your Linux Data with Rsync Tools and Techniques

Expert Tips for Efficiently Safeguarding Your Linux Data with Rsync Tools and Techniques

Edward Lv13

Expert Tips for Efficiently Safeguarding Your Linux Data with Rsync Tools and Techniques

Don’t risk data loss. Back up your valuable data from the Linux command line. We’ll be using the rsync command for this, and we’ve even found some nice optional graphical interfaces for it.

rsync is the Best for Backups

There are many ways to accomplish making a backup copy of your files. We wanted to show you a robust, flexible, and reliable way to protect your data. We choose rsync because of its well-respected algorithms that calculate the differences between files in the source directory and the target directory. Only the differences between two versions of a file are transferred, not the whole file if that can be avoided.

When this efficiency is paired with its solid track record in performing file copies and directory synchronizations since the mid-1990’s, rsync is a perfect candidate for creating backups from the Linux command line.

Additionally, there are independent software programs that act as a front-end for rsync. They provide graphical user interfaces (GUIs) to rsync which some people may find easier to use.

The simpler and faster it is to make a backup, the more likely you are to do so.

Using rsync With an External Hard Drive

To make a backup copy of your data to an external hard drive, the hard drive must be mounted and accessible to you. If you can write to it, then so can rsync. In this example, an external USB hard drive called SILVERXHD (for “Silver eXternal Hard Drive”) is plugged into the Linux computer. It has been auto-mounted by the operating system.

You will need to know the path to the drive. In GNOME, open the Nautilus file browser and locate the name of the drive in the sidebar.

Hover the mouse pointer over the name of the external drive and a tooltip will show you the path to the drive.

tooltip for an external drive

In this example, the tooltip informs us that the mount point for the filesystem on the external drive is "/media/dave/SILVERXHD."

If your file browser does not do this, browse to the external drive and open a terminal window in that location. Use the pwd command to print the path to the terminal window.

Copying the Contents From the Source Directory

To use rsync to copy the contents of a directory to your backup destination, use the following command. The -r (recursive) option causes rsync to copy all nested subdirectories and their contents. Note that there is forward slash “/“ at the end of the word “SILVERXHD,” but it has wrapped round to the next line in the screenshot.

rsync -r /home/dave/Documents/ /media/dave/SILVERXHD/

rsync -r /home/dave/Documents/ /media/dave/SILVERXHD/ in a terminal window

The file copy takes place, and you are returned to the command line prompt.

If we look at the external USB drive, we see the directories that are in the Documents directory have been copied to the root of the external drive.

ls

ls in a terminal window

Copying the Source Directory and Its Contents

If you had wanted to have the Documents directory and its contents copied to the external drive, remove the “/“ from the end of “/home/dave/Documents” in the command line, like this:

rsync -r /home/dave/Documents /media/dave/SILVERXHD/

rsync -r /home/dave/Documents /media/dave/SILVERXHD/ in a terminal window

To avoid confusion, I removed the two previously copied directories from the external drive before this second command was executed.

If we let the second copy complete and take another look at the external drive, we see the Documents directory has been copied over. Its contents are within that directory. They are not in the root of the external drive.

ls in a terminal window

Copying to a Specific Target Directory

To copy to a specific directory on the target hard drive, add the name of the directory to the target path. Let’s suppose we want to copy the contents of the “/home/dave/Documents” directory to a directory called “backups” on the external drive.

We’d do this with the following command.

rsync -r /home/dave/Documents/ /media/dave/SILVERXHD/backups/

rsync -r /home/dave/Documents/ /media/dave/SILVERXHD/backups/ na terminal window

Checking on the external drive we can see the backups directory has been created, and within that directory are the contents of the “/home/dave/Documents” directory.

ls

ls backups

Output from ls in a terminal window

Power Tools add-on for Google Sheets, 12-month subscription

Preserving File Ownership and Permissions

Use the -a (archive) option to preserve file attributes such as modification dates, file ownership, access permissions, and more, for copied files, symlinks, and special block files.

rsync -ra /home/dave/Documents/ /media/dave/SILVERXHD/backups/

rsync -ra /home/dave/Documents/ /media/dave/SILVERXHD/backups/ in a terminal window

Using Verbose Mode

The -v (verbose) option forces rsync to list the files as they are being copied.

rsync -rav /home/dave/Documents/ /media/dave/SILVERXHD/backups/

rsync -rav /home/dave/Documents/ /media/dave/SILVERXHD/backups/ in a terminal window

A summary of the backup is presented when the copying is complete.

A summary of the output.

  • Sent: The bytes transferred to the target.
  • Received: The bytes received at the host.
  • Bytes/sec: is the effective transfer rate.
  • Total size: Represents the size of the data that would have been sent if you were not using rsync. On subsequent runs of rsync it will only transfer the file differences. This figure will represent the data that did not have to be transferred.
  • Speedup: This is the ratio between the amount of data that had to be sent and the total amount of data that there is. If rsync needs to copy all of the files in their entirety (the first time it is run, for example) the speedup will be 1.0. When rsync is next used, it will optimize the transfers. It will only send the differences between the files, not the entire files. FIles with no changes will be ignored. The speedup figure will represent the ratio between the small amount of data that was required to be transferred versus the total size of the files.

Using The Progress Option

The -P (progress) option causes rsync to generate a small progress report after each file is copied.

rsync -raP /home/dave/Documents/ /media/dave/SILVERXHD/backups/

Rsync with a progress report.

The information provided can be seen between each copied file.

Each line indicates what has been performed by rsync.

Parental Control Software

#1 Rated Parental Control Software.
Monitor & Control all PC Activity!
sentrypc.com/parental-controls/
The information provided is:
  • Byte size: Data transferred for this file.
  • Percentage: Percentage of the file transferred.
  • B/s: Data transfer rate.
  • Time remaining: Estimated time left to transfer this file.
  • xfr#: The number of files transferred so far.
  • to-chk: The number of files left to be checked and verified by the optimization algorithms.

Adding More Speed

To speed up transfers, use the -z (compression) option. This compresses the file in transfer, but the file is stored uncompressed in the target directory.

The compression option will not yield significant benefits for transfers involving many small files. For collections of larger files, it can reduce the transfer time in a meaningful way.

We’re also using the --partial option here. rsync will delete partially transferred files caused by network glitches or other interruptions. The --partial option forces rsync to leave the partially transferred files on the target. The net time rsync runs it will not have to re-transfer the portions of the partially transferred files.

Note that you might not want to use this option if there is a risk that someone will mistake the partially transferred files for completely transferred files.

rsync -ravz –partial /home/dave/Documents/ /media/dave/SILVERXHD/backups/

Running rsync with an argument to make it faster.

In our example, the benefits are marginal.

Rsync running slightly faster.

The speedup ratio has improved but by two-hundredths of a percent! In a real-world scenario, your speed improvements will be more impressive.

Using rsync Over A Network

So far we’ve been targetting an external USB drive. To use a network location as the target for the backup, use the path to that location on the command line. There is a network attached storage device (NAS) on the network that this article was researched on.

We can use the same trick we used earlier to identify the path to the NAS, by hovering the mouse over the connection to that device in Nautilus.

There are no special options to backup across a network; these are all options we have already used.

rsync -ravz –partial /home/dave/Documents/ /media/dave/NAS/dave/backups/

Rsync running to a network device.

There is no difference in the format of the output.

Rsync over a network looks the same as regular rsync.

Not surprisingly, there is a significant improvement in the Bytes/sec figure.

If we run rsync once again, we can see that there are no files to transfer because there have been no changes, but there are still some bytes transferred back and forth. This is the amount of data that needs to be transferred to compare the file list on the target with the file list on the source.

Rsync run again shows no files to copy.

The speedup ratio is an order of magnitude better in this instance. In practice, your performance ratios will be somewhere between our two pseudo-artificial readings.

Using rsync Over SSH

rsync supports backing up across an SSH connection. We need to provide the user account name and the SSH location on the command line. We’re using a network name here, but you can also use an IP address.

Note the “:” between the SSH connection details and the start of the network path on the remote target.

rsync -ravz –partial /home/dave/Documents/ dave@sulaco.local :/home/dave/Backups/

Running rsync over SSH.

You will be asked for the password of the user account on the remote machine. This isn’t your password on the source machine.

Rsync's output over SSH looks the same as any other output.

vMix Pro - Software based live production. vMix Pro includes everything in vMix 4K plus 8 channels of Replay and 8 vMix Call
This bundle includes Studio 200 for vMix from Virtualsetworks, HTTP Matrix 1.0 automation scheduler, and 4 introductory training videos from the Udemy vMix Basic to Amazing course.

The backup will complete as usual. The throughput isn’t as fast as a regular network connection, because of the encryption and decryption that takes place in the secure shell connection.

Automating Your Backups

We can easily create automated backups by adding entries to your crontab file.

crontab -e

Run 'crontab -e' to schedule rsync.

PDF application, powered by AI-based OCR, for unified workflows with both digital and scanned documents.

We’ll set up an automated backup to run each day at 04:30 (if the computer is on at that time, of course). The syntax for the rsync command doesn’t change at all.

Some crontab settings.

Ctrl+O will write your changes to the file, and Ctrl+X will close the nano editor.

Putting a Friendly Face on Rsync

People who are less comfortable with the command line can use one of a number of programs that put a graphical user interface (GUI) on rsync. Two good examples are luckyBackup and Grsync . Both of these programs allow many of the rsync options to be selected through the user interface.

The Grsync program concentrates on being a visual wrapper for rync. It provides easy access to the rsync options and adds only a limited set of new functionality.

The grsync UI.

The luckyBackup program is much more than a simple wrapper for rsync. It is a backup program that uses rsync behind the scenes. For example, luckyBackup can make multiple “snapshots” of your backup. You can then “roll back” to the versions of the files in any of the snapshots.

The LuckyBackup UI.

To install Grsync

To install Grsync in Ubuntu, use this command:

sudo apt install grsync

Installing grsync on Ubuntu.

To install Grsync in Fedora, use this command:

sudo dnf install grsync

Installing grsync on Fedora.

To install Grsync in Manaro use this command:

sudo pacman -Syu grsync

Installing grsync on Manaro.To Install luckyBackup

To install luckyBackup in Ubuntu, use the following command:

sudo apt install luckybackup

The command to install luckybackup on Ubuntu or most Debian distros.

To install `luckyBackup` in Fedora use the following command:

sudo dnf install luckybackup

sudo dnf install luckyback in a terminal window

In Manjaro you must install luckyBackup from the Arch User Repository (AUR). You can do this with the pamac package manager.

Searhc for 'lucky' in the Arch User Repository.Don’t Risk It, Back Up Your Data Often

Backups are absolutely vital. Back up frequently, back up to many locations, and back up to different media. Once it is set up, rsync can do all of that for you.

| | Linux Commands | | |
| —————– | ———————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————— | |
| Files | tar · pv · cat · tac · chmod · grep · diff · sed · ar · man · pushd · popd · fsck · testdisk · seq · fd · pandoc · cd · $PATH · awk · join · jq · fold · uniq · journalctl · tail · stat · ls · fstab · echo · less · chgrp · chown · rev · look · strings · type · rename · zip · unzip · mount · umount · install · fdisk · mkfs · rm · rmdir · rsync · df · gpg · vi · nano · mkdir · du · ln · patch · convert · rclone · shred · srm · scp · gzip · chattr · cut · find · umask · wc · tr | |
| Processes | alias · screen · top · nice · renice · progress · strace · systemd · tmux · chsh · history · at · batch · free · which · dmesg · chfn · usermod · ps · chroot · xargs · tty · pinky · lsof · vmstat · timeout · wall · yes · kill · sleep · sudo · su · time · groupadd · usermod · groups · lshw · shutdown · reboot · halt · poweroff · passwd · lscpu · crontab · date · bg · fg · pidof · nohup · pmap | |
| Networking | netstat · ping · traceroute · ip · ss · whois · fail2ban · bmon · dig · finger · nmap · ftp · curl · wget · who · whoami · w · iptables · ssh-keygen · ufw · arping · firewalld | |

  • Title: Expert Tips for Efficiently Safeguarding Your Linux Data with Rsync Tools and Techniques
  • Author: Edward
  • Created at : 2024-08-31 08:55:08
  • Updated at : 2024-09-01 08:55:08
  • Link: https://vp-tips.techidaily.com/expert-tips-for-efficiently-safeguarding-your-linux-data-with-rsync-tools-and-techniques/
  • License: This work is licensed under CC BY-NC-SA 4.0.