msgbartop
A chronological documentation test project, nothing serious, really!
msgbarbottom

13 Jan 2012 Using Lsyncd to perform “live” syncronization of a local directory to a remote directory

This post is a short HOWTO and describes how you can install and run lsyncd to perform a rsync syncronization from local to a remote server using SSH.
Lsyncd is a daemon to continuously synchronize directory trees and relies on inotify. If you need real live syncronization DRBD might be a better alternative since it is a block level syncronization.

Installing Lsyncd 2.0 from source on CentOS 6
Lsyncd is not included as a package in CentOS 6, so you need to download the source file from http://code.google.com/p/lsyncd/downloads/list.
You should have rsync, GCC and lua-devel installed on your system before you continue installing Lsyncd.

# yum install rsync lua-devel

Unpack the lsyncd source file and run the following commands from the unpacked file

# configure 
# make
# make install

make install copies the compiled files and install them to the right directories in your system.

I need to configure a non password SSH communication between the two servers with a shared SSH key.
On the source server run the following command to generate a SSH key, if you have not done this already.
Remember to do this as the user you are going to perform the sync with.

# ssh-keygen

Secure copy the generated SSH key from the source server to your target server

# scp ~/.ssh/id_rsa.pub root@remoteserver:/tmp

On the target server you need to add the copied SSH key to your existing authorized keys file.
Also remember to do this with the user you are going to connect with from the source server.

# cat /tmp/id_rsa.pub >> ~/.ssh/authorized_keys

If you do not have this file, just create it using the touch command described below

# touch ~/.ssh/authorized_keys

Test if you can ssh without a password from your source server to the target server.

I have made a config file, /root/scripts/lsyncd.conf that tells Lsyncd where to put the log- and statusfile. That it should be running as a daemon in the background, and a sync should occur after 900 seconds (15 minutes) if there have not been any filesystem changes and there should not be more than 6 parallell Lsyncd processes.

settings = {
   logfile      = "/tmp/lsyncd.log",
   statusFile   = "/tmp/lsyncd.status",
   nodaemon     = false,
   maxDelays    = 900,
   maxProcesses = 6,
}

sync{default.rsyncssh, source="/path/on/source/", host="hostnam.target.server.tld", targetdir="/path/on/target/"}

To start lsyncd you run the command

# lsyncd /root/scripts/lsyncd.conf

You should now see a Lsyncd process running as a daemon on your system. It performs a sync when you start and then waits for any filesystem changes or sync after 900 seconds.

If you would like Lsyncd to start at boot, just add the following line to the bottom of file /etc/rc.local

lsyncd /root/scripts/lsyncd.conf

You do now have a working secure rsync syncronization between two servers.

What directories you are syncing

# tail -f /tmp/lsyncd.status

What is happening now

# tail -f /tmp/lsyncd.log

Tags: , , , , ,

Posted by

09 Jun 2007 Synchronize current directory with remote one using rsync

This is a simple way to synchronize all the files from the current computer to a remote computer using SSH as transportation protocol.

rsync -raz --progress --delete --bwlimit=4096 /synchronize/from/this-folder/ remote-server:/to/this-folder/

This example is using PUT as transfer method because the files are put from the local computer to the remote.

Options explained

  • -r synchronize directories recursive
  • a archive, preserves file attributes, ownership, timestamps…
  • z compress data, saves bandwidth but is CPU intensive
  • –progress shows the progress of all the files that are being syncronized
  • –delete delete files remotely that no longer exist locally
  • –bwlimit=4096 specify a maximum transfer rate of 4096 kilobytes per second.

Tags: ,

Posted by

10 Apr 2007 rsync backup on a local pc

#!/bin/bash
echo "Starting Daily rsync backup on `date`"

for FSYS in /home/username/rsync
do
  /usr/bin/rsync -aE --delete $FSYS /home/username/rsync_bck
  if [ $? != 0 ]; then
    echo " ==> Error during rsync of $FSYS"
  else
    echo "rsync of $FSYS OK on `date`"
  fi
done

echo "Finished Daily rsync backup at `date`"

Tags: ,

Posted by