Psychz - Luis
Votes: 0Posted On: Sep 23, 2015 13:52:19
Rsync (Remote Sync) is commonly used for copying and synchronizing files and directories remotely as well as locally in Linux/Unix systems
Rsync can be used to pull and send data over local and remote networks. Rsync is not secure unlike SCP the risk of traffic being spoof is high, you should always make sure that both local and remote locations
are secure prior to using this command.
0. Login to the machine where you will be operating this command
1. Check if rsync is installed
rsync --version
2. Installing Rsync
RPM base OS - yum -y install rsync
Debian / Ubuntu family - apt-get install rsync -y
(NOTE: Before sending data across machines, make sure that the local firewall on both servers have the incoming IP address whitelisted to avoid being block by the firewall.)
Rsync Examples with Flags
Rsync has many flags which can be used to send files over the network, we will demonstrate a series of live examples our engineers use to send large files when performing customer migrations.
(NOTE: We recommend to use the screen command to start a screen session when migrating large amounts of data across networks otherwise a disconnect can suddenly kill your migration.)
Example 1: Sending the entire content of a directory using 10Mbps transfer rate. Replace the X with the destination IP address.
rsync -apW --progress --bwlimit=10000 /file/* root@x.x.x.x:/home/file
Example 2: Sending files and showing a progress bar of the amount of data being transfer and elapse time
rsync -pavW --progress /home/file/* root@x.x.x.x.:/home/file
Example 3: For slow network connections
rsync -pavz --progress /home/file/* root@x.x.x.x.:/home/file
Example 4: Sending files to remote machines who ssh port are not port 22
rsync -pavW --progress --inplace --rsh='ssh -p 2244' file/* root@x.x.x.x:/file
NOTE: When sending files using this syntax rsync -pavW --progress /home/file/* root@x.x.x.x.:/home/file/* you are telling rsync to create another file folder within the remote server ending with a /file/file. Never end the destination path with a wildcard only as the source path.
NOTE: You do not need to pre-create the destination path prior to migrating the data, you can simply do rsync -pavW --progress /home/file/* root@x.x.x.x.:/home/file ; if the file doesn't exist it will create it and move the data to the new directory.
Flags of interest:
-v: verbose is optional to show logs in case errors occur when transferring data
-a: archive the files for data transfer to increase transfer speed
-W: This rule is to consider the whole option, its use to speed up the transfer speed by keeping the whole files intact instead. This is perfect when both sides don’t have limits
-z: this option is used to compress data for slow connections
-p: to retain permissions, this is used if both ends will have the same ownership and permissions
--progress: shows the progress of the download