CentOS / SL 6/7 with lsyncd and csync2
Configuring File Replication in RHEL / CentOS / SL 6/7 with lsyncd and csync2
If you want some directory of yours actively backed after every change, they are a lot of sync options to choose from.
Software like csync, rsync, csync2, lsyncd and many others will all do the job, but in this article we will only review one of them. We will introduce features of lsyncd, a synchronization daemon that enables you to mirror your designated directory with to any other directory on the network or locally.
To save network and disk bandwidth, it only mirror changes to your directory. So lets start.
Installing the necessary packages
To install lsyncd and csync2 in CentOS / RHEL 6 you only need the EPEL repository and it installs directly via yum as detailed below.
RHEL / CentOS / SL 6
Configure the EPEL repository
rpm -Uhv https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
Install the necessary packages
Yum install csync2 lsyncd xinetd
RHEL / CentOS / SL 7
For versions of RHEL 7 you need to download the version of csync2 that is for Fedora20,
it can be downloaded from the following URL http://rpm.pbone.net/index.php3/stat/3/limit/1/srodzaj/ 1 / dl / 40 / search / csync2 / field% 5B% 5D / 1 / field% 5B% 5D / 2
Once downloaded we installed with the command:
Yum install /ruta/del/rpm/que/download.rpm
The previous steps are the only ones that differ as to version 7 of version 6, once installed the steps are the same in both versions
Configure csync2
Enable the csync2 service under xinetd:
To enable the csync2 service through xinetd
sed -i.bak ‘s#yes#no#g’ /etc/xinetd.d/csync2
Then we lift the xinetd service
service xinetd start
Generate .key for authentication between servers
For the authentication between the servers it is necessary to create a .key file,
We can create it with the command csync2 and the -k option:
csync2 -k /etc/csync2/csync2.key
Configure csync2
To configure csync2 we must modify the file /etc/csync2/csync2.cfg, loading the following:
vim /etc/csync2/csync2.cfg
nossl * *;
group web
{
host node1;
host node2;
host node3;
key /etc/csync2/csync2.key;
include /home/website/public_html;
exclude *.log;
auto younger;
}
It should be noted that the host name (in this case node1) must be the hostname or fqdn of each server
Copy the csync files to the other nodes:
Before copying the settings to the other nodes, the file must be copied /etc/csync2/csync2.cfg a /etc/csync2/csync2_node1.cfg /etc/csync2/csync2_node2.cfg /etc/csync2/csync2_node3.cfg:
cp /etc/csync2/csync2.cfg /etc/csync2/csync2_node1.cfg
cp /etc/csync2/csync2.cfg /etc/csync2/csync2_node2.cfg
cp /etc/csync2/csync2.cfg /etc/csync2/csync2_node3.cfg
Finally we copy to the other nodes:
scp /etc/csync2/* node2:/etc/csync2
scp /etc/csync2/* node3:/etc/csync3
Start csync2 replication
For the initial synchronization it is necessary to run the following command on all the nodes
csyncs2 -xv
Configure lsyncd
Create configuration files
The following lines must be added to the file /etc/lsyncd.conf
vim /etc/lsyncd.conf
settings {
logident = “lsyncd”,
logfacility = “user”,
logfile = “/var/log/lsyncd.log”,
statusFile = “/var/log/lsyncd_status.log”,
statusInterval = 1
}
initSync = {
delay = 1,
maxProcesses = 1,
action = function(inlet)
local config = inlet.getConfig()
local elist = inlet.getEvents(function(event)
return event.etype ~= “Init”
end)
local directory = string.sub(config.source, 1, -2)
local paths = elist.getPaths(function(etype, path)
return “\t” .. config.syncid .. “:” .. directory .. path
end)
log(“Normal”, “Processing syncing list:\n”, table.concat(paths, “\n”))
spawn(elist, “/usr/sbin/csync2.sh”, “-C”, config.syncid, “-x”)
end,
collect = function(agent, exitcode)
local config = agent.config
if not agent.isList and agent.etype == “Init” then
if exitcode == 0 then
log(“Normal”, “Startup of ‘”, config.syncid, “‘ instance finished.”)
elseif config.exitcodes and config.exitcodes[exitcode] == “again” then
log(“Normal”, “Retrying startup of ‘”, config.syncid, “‘ instance.”)
return “again”
else
log(“Error”, “Failure on startup of ‘”, config.syncid, “‘ instance.”)
terminate(-1)
end
return
end
local rc = config.exitcodes and config.exitcodes[exitcode]
if rc == “die” then
return rc
end
if agent.isList then
if rc == “again” then
log(“Normal”, “Retrying events list on exitcode = “, exitcode)
else
log(“Normal”, “Finished events list = “, exitcode)
end
else
if rc == “again” then
log(“Normal”, “Retrying “, agent.etype, ” on “, agent.sourcePath, ” = “, exitcode)
else
log(“Normal”, “Finished “, agent.etype, ” on “, agent.sourcePath, ” = “, exitcode)
end
end
return rc
end,
init = function(event)
local inlet = event.inlet;
local config = inlet.getConfig();
log(“Normal”, “Recursive startup sync: “, config.syncid, “:”, config.source)
spawn(event, “/usr/sbin/csync2.sh”, “-C”, config.syncid, “-x”)
end,
prepare = function(config)
if not config.syncid then
error(“Missing ‘syncid’ parameter.”, 4)
end
local c = “csync2_” .. config.syncid .. “.cfg”
local f, err = io.open(“/etc/csync2/” .. c, “r”)
if not f then
error(“Invalid ‘syncid’ parameter: ” .. err, 4)
end
f:close()
end
}
local sources = {
— change the node1 value with respective host
[“/home/website/public_html”] = “node1”,
[“/otrodirectorio”] = “node1”
}
for key, value in pairs(sources) do
sync {initSync, source=key, syncid=value}
end
Do not forget to change the “node1″ in each node. For example node2, your /etc/lsyncd.conf file definition ‘local source’ should use ‘node2’.
Add the path of the lsync configuration file
We must add the path of the file created in the previous step in the file /etc/sysconfig/lsyncd, that we can do with the command sed:
sed -i.bak ‘s#^LSYNCD_OPTIONS=.*#LSYNCD_OPTIONS=” /etc/lsyncd.conf”#g’ /etc/sysconfig/lsyncd
Create the script that will run lsync
To synchronize the folders, lsync will try to execute the script /usr/sbin/csync2.sh, el cual debemos crear:
/usr/sbin/
cat <<EOF > csync2.sh
#!/bin/bash
/usr/sbin/csync2 -xXrv
exit 0
## Exit 0 is added so that lsyncd does not stop when it gives an error
EOF
Give Execution Permissions to Script
Chmod 755 /usr/sbin/csync2.sh
Enable and Start the lsyncd service:
We enable the service to interact with the Operating System
Chkconfig lsyncd on
Then we lift the service
Service lsyncd start
#!/bin/sh
for ((i=0; i<100; i++)); do
touch /home/website/${i}.html
done
#!/bin/bash
for ((i=0;i<10;i++))
do
number=$((1000000 + ($(od -An -N2 -i /dev/random)) % (10000 + 1000)))
for ((j=0; j<1000; j++))
do
touch /home/website/${i}-${j}.html
done
sleep ${number}
done
Recent Comments