How to rsync two directories
Publisher: Psychz Networks, October 03,2015Rsync (Remote Sync) is commonly used for copying and synchronizing files and directories remotely as well as locally in Linux/Unix systems.
Rsync can be used to pull and send data over local and remote networks. Rsync is not secure unlike SCP and the risk of traffic being spoof is high, you should always make sure that both local and remote locations are secure prior to using this command.
Step 1. Check if "rsync" is installed using the following command
rsync --version
Step 2. Installing Rsync (Skip this step if "rsync" already exists)
RPM base OS yum -y install rsync
Debian / Ubuntu family apt-get install rsync -y
(NOTE: Before sending data across machines, make sure that the local firewall on both servers has the incoming IP address whitelisted to avoid being block by the firewall.)
Rsync has many flags which can be used to send files over the network, well demonstrate a series of live examples our engineers use to send large files when performing customer migrations.
(NOTE: We recommend to use the screen command to start a screen session when migrating large amounts of data across networks otherwise a disconnect can suddenly kill your migration.)
Example 1:
Sending the entire content of a directory using 10Mbps transfer rate. Replace the X with the destination IP address.
"file" is the example source folder, and "/home/new-file" is the destination folder.
rsync -apW --progress --bwlimit=10000 /file/* root@x.x.x.x:/home/new-file
Example 2:
Sending files and showing a progress bar of the amount of data being transfer and elapse time
rsync -pavW --progress /home/file/* root@x.x.x.x.:/home/new-file
Example 3:
For slow network connections, you can use the following command.
rsync -pavz --progress /home/file/* root@x.x.x.x.:/home/new-file
Flags of interest:
-v: verbose is optional to show logs in case errors occur when transferring data
-a: archive the files for data transfer to increase transfer speed
-W: This rule is considered as the whole option, its use to speed up the transfer speed by keeping the whole files intact instead. This is perfect when both sides don’t have limits
-z: this option is used to compress data for slow connections
-p: to retain permissions, this is used if bends end will have the same ownership and permissions
--progress: shows the progress of the download
example 4:
Sending files to remote machines who ssh port are not port 22
rsync -pavW --progress --inplace --rsh='ssh -p 2244' file/* root@x.x.x.x:/file
NOTE: when sending files using this syntax ; rsync -pavW --progress /home/file/* root@x.x.x.x.:/home/file/* you are telling rsync to create another file folder within the remote server ending with a /file/file. Never end the destination path with a wildcard only as the source path.
You do not need to pre-create the destination path prior to migrating the data, you can simply do;rsync -pavW --progress /home/file/* root@x.x.x.x.:/home/file if the file doesn't exist it will create it and move the data to the new directory.
last updated on July, 30th 2018