Set up Unison file synchronization between two servers on Debian 10 (Buster)
This tutorial shows how to use Unison to set up file synchronization between two Debian 10 servers. Unison is a file synchronization tool similar to rsync. The biggest difference is that it can track / synchronize changes in both directions, that is, copy the changed files on server1 to server2 and vice versa.
1 Preliminary note
In this tutorial, I will use the following two Debian servers:
- Server1.googlesyndication.com with IP address 192.168.0.100
- Server2.googlesyndication.com with IP address 192.168.0.101
I want to synchronize the directory / var / www between the two servers. In this tutorial, I will run Unison as the root user so that Unison has sufficient permissions to synchronize user and group permissions.
All commands in this tutorial are run as the root user. Log in to the two servers on the shell as the root user and start from step 2 Install Unison “.
2 install Unison
Server 1 / Server 2:
Unison must be installed on server1 and server2; since we use SSH to connect from server1 to server2, we also need the SSH package, I will install the nano editor on the shell for file editing. This can be achieved in the following ways:
apt-get -y install unison openssh-server ssh nano
3 Create a private / public key pair on server1
Now, we create a private / public key pair on server1.googlesyndication.com:
ssh-keygen -t dsa
[email protected]: ~ # Ssh-keygen -t dsa generates public / private key dsa key pair. Enter the file (/root/.ssh/id_dsa) in the file where the key is saved: <-ENTER to create the directory '/root/.ssh'. Enter password (empty (no password): <-ENTER Enter the same password: <-ENTER Your logo has been saved in /root/.ssh/id_dsa. Your public key has been saved in /root/.ssh/id_dsa.pub The fingerprint is: ba: 82: e1: a1: 42: 9b: d4: c8: 99: c8: bd: 8b: 7d: 4d: d4: 66 [email protected]The randomart picture of the key is: + ---[DSA 1024]---- + | || ||. ||. E || + *. S || .Ooo o || ooo +. + || oo = ... o || .. oo .. | + ----------------- +[email protected]: ~ #
It is important that you do not enter a password, otherwise mirroring will not be possible without human-computer interaction, so just press ENTER!
Next, we copy the public key to server2.googlesyndication.com:
ssh-copy-id -i $HOME/.ssh/id_dsa.pub [email protected]
# ssh-copy-id -i $HOME/.ssh/id_dsa.pub [email protected]
The authenticity of host '192.168.1.102 (192.168.0.101)' can't be established.ECDSA key fingerprint is 2b:3c:35:ad:3d:e2:fc:16:2f:55:5c:e1:2c:d7:3d:a9.Are you sure you want to continue connecting (yes/no)? <-- yes (you will see this only if this is the first time you connect to server2)/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys[email protected]'s password: <-- server2 root password
Number of key(s) added: 1
Now try logging into the machine, with: "ssh '[email protected]'"and check to make sure that only the key(s) you wanted were added.
Now check whether the public key of server1 is correctly transmitted on server2:
[email protected]:/home/administrator# cat $HOME/.ssh/authorized_keysssh-dss AAAAB3NzaC1kc3MAAACBAKHLdAztIr8muZIlQYuE/4f75kmgTwWqJRZJ1dTqHDnHWsy48emDU8v85hxAPg43k9aF7/zAwpA0MNNNk5T9Tx/DyUkK/KcyVP2f4p8tvovrkUvoxsZACkTUmFqKdq2x6/AGfjsCRmkpLhZuad7r5rKEXHRh8KYGHqD1Id8wcpy5AAAAFQCww3OekKcKMshMAwBK3XQmmYEGUwAAAIEAgjztlwh8OFYxwQve/RrhI2sceCXwS/yjQyH7q0zdWB9Fr4s/16T2PLBT+7M3vb+JlPDO3JRqgaYbel1kS2F2iKrY0EX0FI3/9fVDfWoz3mhCscPLriqy5AcsHitxQNfiZgA5wDiSjWpk1v+FbIC+VuqbKdQuE4MBKj19N9YALIUAAACABQ4NDsa2UBc8jsxvghjoLhUWF7HChaCksXQcL6i98VNRcemtPC6wpIri75iR4Uhv1666bDOBAdmIBX9Qf7A/+czPKPaj4CGI1hVy1pgYMa3btnEvoSnH/ONtjpOz9q+3up1OOOn+5fud7xjJn+Fq8WoGROgarBpCbQU3w2GUUnM= [email protected]er1
4 run Unison
Now we can run Unison for the first time to synchronize the / var / www directory on both servers. Run on server1:
unison /var/www ssh://192.168.0.101//var/www
The output will be similar to this-you may have to answer a few questions because this is the first time Unison runs:
[email protected]:/var/www# unison /var/www ssh://192.168.0.101//var/wwwContacting server...Connected [//server1//var/www -> //server2//var/www]Looking for changesWarning: No archive files were found for these roots, whose canonical names are:/var/www//server2//var/wwwThis can happen eitherbecause this is the first time you have synchronized these roots,or because you have upgraded Unison to a new version with a differentarchive format.
Update detection may take a while on this run if the replicas arelarge.
Unison will assume that the 'last synchronized state' of both replicaswas completely empty. This means that any files that are differentwill be reported as conflicts, and any files that exist only on onereplica will be judged as new and propagated to the other replica.If the two replicas are identical, then no changes will be reported.
If you see this message repeatedly, it may be because one of your machinesis getting its address from DHCP, which is causing its host name to changebetween synchronizations. See the documentation for the UNISONLOCALHOSTNAMEenvironment variable for advice on how to correct this.
Donations to the Unison project are gratefully accepted:https://www.cis.upenn.edu/~bcpierce/unison
Press return to continue.[
] <-- Press Enter
Waiting for changes from serverReconciling changes
local server2dir ----> googlesyndication.com [f] <-- Press Enterdir ----> example.de [f] <-- Press Enter
Proceed with propagating updates?  <-- Enter "y"Propagating updates
UNISON 2.48.4 started propagating changes at 13:24:01.10 on 05 May 2020[BGN] Copying googlesyndication.com from /var/www to //server2//var/www[BGN] Copying example.de from /var/www to //server2//var/wwwShortcut: copied /var/www/example.de/web/index.html from local file /var/www/.unison.googlesyndication.com.d3783bddaaf59b9ba4d2ed0433f9db63.unison.tmp/web/index.html[END] Copying example.de[END] Copying googlesyndication.comUNISON 2.48.4 finished propagating changes at 13:24:01.98 on 05 May 2020
Saving synchronizer stateSynchronization complete at 13:24:01 (2 items transferred, 0 skipped, 0 failed)
Now check the / var / www directory on server1 and server2, you should find that they are now synchronized.
Of course, we do n’t want to run Unison interactively, so we can create a preferences file (/root/.unison/default.prf) that contains all the settings we must specify on the command line:
# Roots of the synchronization root = /var/www root = ssh://192.168.0.101//var/www # Paths to synchronize #path = current #path = common #path = .netscape/bookmarks.html # Some regexps specifying names and paths to ignore #ignore = Path stats ## ignores /var/www/stats #ignore = Path stats/* ## ignores /var/www/stats/* #ignore = Path */stats ## ignores /var/www/somedir/stats, but not /var/www/a/b/c/stats #ignore = Name *stats ## ignores all files/directories that end with "stats" #ignore = Name stats* ## ignores all files/directories that begin with "stats" #ignore = Name *.tmp ## ignores all files with the extension .tmp # When set to true, this flag causes the user interface to skip # asking for confirmations on non-conflicting changes. (More # precisely, when the user interface is done setting the # propagation direction for one entry and is about to move to the # next, it will skip over all non-conflicting entries and go # directly to the next conflict.) auto=true # When this is set to true, the user interface will ask no # questions at all. Non-conflicting changes will be propagated; # conflicts will be skipped. batch=true # !When this is set to true, Unison will request an extra # confirmation if it appears that the entire replica has been # deleted, before propagating the change. If the batch flag is # also set, synchronization will be aborted. When the path # preference is used, the same confirmation will be requested for # top-level paths. (At the moment, this flag only affects the # text user interface.) See also the mountpoint preference. confirmbigdel=true # When this preference is set to true, Unison will use the # modification time and length of a file as a `pseudo inode # number' when scanning replicas for updates, instead of reading # the full contents of every file. Under Windows, this may cause # Unison to miss propagating an update if the modification time # and length of the file are both unchanged by the update. # However, Unison will never overwrite such an update with a # change from the other replica, since it always does a safe # check for updates just before propagating a change. Thus, it is # reasonable to use this switch under Windows most of the time # and occasionally run Unison once with fastcheck set to false, # if you are worried that Unison may have overlooked an update. # The default value of the preference is auto, which causes # Unison to use fast checking on Unix replicas (where it is safe) # and slow checking on Windows replicas. For backward # compatibility, yes, no, and default can be used in place of # true, false, and auto. See the section "Fast Checking" for more # information. fastcheck=true # When this flag is set to true, the group attributes of the # files are synchronized. Whether the group names or the group # identifiers are synchronizeddepends on the preference numerids. group=true # When this flag is set to true, the owner attributes of the # files are synchronized. Whether the owner names or the owner # identifiers are synchronizeddepends on the preference # extttnumerids. owner=true # Including the preference -prefer root causes Unison always to # resolve conflicts in favor of root, rather than asking for # guidance from the user. (The syntax of root is the same as for # the root preference, plus the special values newer and older.) # This preference is overridden by the preferpartial preference. # This preference should be used only if you are sure you know # what you are doing! prefer=newer # When this preference is set to true, the textual user interface # will print nothing at all, except in the case of errors. # Setting silent to true automatically sets the batch preference # to true. silent=true # When this flag is set to true, file modification times (but not # directory modtimes) are propagated. times=true
Comments should make the file easy to explain, except for path instructions. If you do not specify any path directives, the directories in the root directive will be synchronized. If you specify a path directive, the path is relative to the root path (for example, root = / var / www, and path = current is converted to / var / www / current), and only these subdirectories will be synchronized, not the entire directory The synchronization is in the root instruction.
You can find more information about the available options by checking the Unison man page:
Now that we have placed all settings in the preferences file (especially the root (and path) directive), we can run Unison without any parameters:
5 Create a Cron job for Unison
We want to automate synchronization, which is why we create a cron job for it on server1.googlesyndication.com:
*/5 * * * * /usr/bin/unison &> /dev/null
This will run Unison every 5 minutes; adjust it to your needs (see
man 5 crontab
). I use the full path here to unify (/ usr / bin / unison), just to make sure that cron knows where to find unification. Your consistent location may vary. Advertise
Find out where you are.
6 tests Unison
Now, I will test Unison's 2-way synchronization to see if the settings are completely normal.
Run the following command on server1 to create a test file with the content "Test 1":
echo "Test 1" > /var/www/test.txt
Now wait at least 5 minutes (because we created a cronjob that runs every 5 minutes). Then run on server2:
The content of the file test.txt is displayed on the screen. The output should be similar to this screenshot.
Now run the following command on server2, which will update the content of our test file to "Test 2":
echo "Test 2" > /var/www/test.txt
And wait at least 5 minutes. Then run the cat command on server1: Advertisements
The output should be: