GlusterFS servers on RHEL 7 and Centos 7
vi /etc/hosts
192.168.1.71 apache1.rmohan.com apache1
192.168.1.72 apache2.rmohan.com apache2
192.168.1.74 apache3.rmohan.com apache3
For this lab, we disabled both firewall and SELinux
vi /etc/selinux/config
egrep “^SELINUX=” /etc/sysconfig/selinux
SELINUX=disabled
systemctl is-enabled firewalld
disabled
systemctl disable firewalld
yum update
yum -y install wget openssh* net-tools ntp
ntpdate -u 0.centos.pool.ntp.org
Before proceeding with the installation, we need to enable both EPEL (Exta Packages for Enterprise Linux) and GlusterFS repositories on both the servers to resolve the external dependencies at the time of installation. If you have enabled only GlusterFS repository and not enabled EPEL repository then it is likely that you will hit the following error message while installing glusterfs-server:
–> Finished Dependency Resolution
Error: Package: glusterfs-server-3.7.11-1.el7.x86_64 (glusterfs-epel)
Requires: liburcu-cds.so.1()(64bit)
Error: Package: glusterfs-server-3.7.11-1.el7.x86_64 (glusterfs-epel)
Requires: liburcu-bp.so.1()(64bit)
You could try using –skip-broken to work around the problem
You could try running: rpm -Va –nofiles –nodigest
EPEL installation
Retrieve EPEL repository : all 3 servers
wget http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-6.noarch.rpm
Gluster installation :
Retrieve GlusterFS repository : all 3 servers
wget -P /etc/yum.repos.d http://download.gluster.org/pub/gluster/glusterfs/LATEST/RHEL/glusterfs-epel.repo
Download GlusterFS packages : all 3 servers
yum -y install glusterfs-server
Now, start the service and keep it started across reboot :
[root@apache3 ~]# systemctl start glusterd && systemctl enable glusterd
Created symlink from /etc/systemd/system/multi-user.target.wants/glusterd.service to /usr/lib/systemd/system/glusterd.service.
[root@apache3 ~]#
Update “/etc/hosts” :
192.168.1.71 apache1.rmohan.com apache1
192.168.1.72 apache2.rmohan.com apache2
192.168.1.74 apache3.rmohan.com apache3
Check the 2 GlusterFS servers
[root@apache1 ~]# gluster peer probe apache2
peer probe: success.
[root@apache1 ~]# gluster peer probe apache3
peer probe: success.
[root@apache1 ~]#
[root@apache2 ~]# gluster peer probe apache3
peer probe: success. Host apache3 port 24007 already in peer list
[root@apache2 ~]# gluster peer probe apache1
peer probe: success. Host apache1 port 24007 already in peer list
[root@apache2 ~]#
[root@apache3 ~]# gluster peer probe apache1
peer probe: success. Host apache1 port 24007 already in peer list
[root@apache3 ~]# gluster peer probe apache2
peer probe: success. Host apache2 port 24007 already in peer list
[root@apache3 ~]#
Check the Peers :
root@apache1 ~]# gluster peer status
Number of Peers: 2
Hostname: apache2
Uuid: 09db776a-ab96-47a3-ad77-6bc77906566b
State: Peer in Cluster (Connected)
Hostname: apache3
Uuid: 14db2875-378d-481d-968c-552a10f2fd7e
State: Peer in Cluster (Connected)
[root@apache1 ~]#
[root@apache2 ~]# gluster peer status
Number of Peers: 2
Hostname: apache1.rmohan.com
Uuid: 16612fd4-8b94-4006-8ed7-24ac33c564e0
State: Peer in Cluster (Connected)
Other names:
apache1
Hostname: apache3
Uuid: 14db2875-378d-481d-968c-552a10f2fd7e
State: Peer in Cluster (Connected)
[root@apache3 ~]# gluster peer status
Number of Peers: 2
Hostname: apache1.rmohan.com
Uuid: 16612fd4-8b94-4006-8ed7-24ac33c564e0
State: Peer in Cluster (Connected)
Other names:
apache1
Hostname: apache2
Uuid: 09db776a-ab96-47a3-ad77-6bc77906566b
State: Peer in Cluster (Connected)
Create GlusterFS volume
We add 2 new drives from VMware ESXi (in fact, the 3 GlusterFS servers are 3 virtual machines) : do the below commands on all 3 servers
[root@apache1 ~]# for HOST in $(ls /sys/class/scsi_host) ; do echo ‘- – -‘>”/sys/class/scsi_host/$HOST/scan” ; done
[root@apache1 ~]# lsblk -f
NAME FSTYPE LABEL UUID MOUNTPOINT
sda
??sda1 xfs be8c0a37-1ed9-4743-b4ef-03d59ba3d06d /boot
??sda2 LVM2_member YXOIJV-EPlD-dXwg-ePQX-D7av-jPdr-Grb4rp
??centos-root xfs d2c46dc9-6365-4345-9b60-021aabfd0525 /
??centos-swap swap 74c70324-6875-40e6-9d8f-591d3325c06e [SWAP]
??centos-home xfs bd8e9210-bc95-4af9-be87-a33da31f8dbb /home
sdb
sr0 iso9660 CentOS 7 x86_64 2016-04-01-13-21-01-00
[root@apache1 ~]#
[root@apache1 ~]# parted -s /dev/sdb mklabel gpt && parted -s /dev/sdb mkpart primary xfs 0% 100% && mkfs.xfs -f /dev/sdb
meta-data=/dev/sdb isize=256 agcount=4, agsize=1310720 blks
= sectsz=512 attr=2, projid32bit=1
= crc=0 finobt=0
data = bsize=4096 blocks=5242880, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=0
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
[root@apache1 ~]#
Create gluster and mount on all 3 nodes
[root@apache1 ~]# mkdir /gluster
[root@apache1 ~]# vi /etc/fstab
[root@apache1 ~]# grep -i gluster /etc/fstab
/dev/sdb /gluster xfs defaults 0 0
[root@apache1 ~]#
[root@apache1 ~]# mount -a
[root@apache1 ~]# df -Th
Filesystem Type Size Used Avail Use% Mounted on
/dev/mapper/centos-root xfs 38G 1.2G 37G 3% /
devtmpfs devtmpfs 1.8G 0 1.8G 0% /dev
tmpfs tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs tmpfs 1.9G 8.6M 1.9G 1% /run
tmpfs tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/sda1 xfs 497M 186M 312M 38% /boot
/dev/mapper/centos-home xfs 19G 33M 19G 1% /home
tmpfs tmpfs 371M 0 371M 0% /run/user/0
/dev/sdb xfs 20G 33M 20G 1% /gluster
[root@apache1 ~]#
Create a directory within the file system on the servers :
mkdir -p /gluster/briks
Create the GlusterFS volume based on those 3 replicas :
gluster volume create appgluster_vol replica 3 apache1:/gluster/briks apache2:/gluster/briks apache3:/gluster/briks
[root@apache1 ~]# gluster volume create appgluster_vol replica 3 apache1:/gluster/briks apache2:/gluster/briks apache3:/gluster/briks
volume create: appgluster_vol: success: please start the volume to access data
[root@apache1 ~]#
Then, start this volume :
[root@apache1 ~]# gluster volume start appgluster_vol
volume start: appgluster_vol: success
[root@apache1 ~]#
[root@apache1 ~]# gluster volume info
Volume Name: appgluster_vol
Type: Replicate
Volume ID: 856f1052-9d19-49ee-b0c1-1e53d32b86f4
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: apache1:/gluster/briks
Brick2: apache2:/gluster/briks
Brick3: apache3:/gluster/briks
Options Reconfigured:
performance.readdir-ahead: on
[root@apache1 ~]#
[root@apache1 ~]# gluster volume status
Status of volume: appgluster_vol
Gluster process TCP Port RDMA Port Online Pid
——————————————————————————
Brick apache1:/gluster/briks 49152 0 Y 3108
Brick apache2:/gluster/briks 49152 0 Y 2923
Brick apache3:/gluster/briks 49152 0 Y 2892
NFS Server on localhost 2049 0 Y 3128
Self-heal Daemon on localhost N/A N/A Y 3136
NFS Server on apache2 2049 0 Y 2943
Self-heal Daemon on apache2 N/A N/A Y 2951
NFS Server on apache3 2049 0 Y 2912
Self-heal Daemon on apache3 N/A N/A Y 2920
Task Status of Volume appgluster_vol
——————————————————————————
There are no active volume tasks
[root@apache1 ~]#
Client side
Both firewall and SELinux are disabled :
192.168.1.21 clusterserver2.rmohan.com clusterserver2
192.168.1.26 clusterserver3.rmohan.com clusterserver3
one the clinet side add the host file
192.168.1.21 clusterserver2.rmohan.com clusterserver2
192.168.1.26 clusterserver3.rmohan.com clusterserver3
192.168.1.71 apache1.rmohan.com apache1
192.168.1.72 apache2.rmohan.com apache2
192.168.1.74 apache3.rmohan.com apache3
[root@clusterserver3 ~]# mount.glusterfs apache1:/appgluster_vol /mnt/
[root@clusterserver3 ~]# df -Th
Filesystem Type Size Used Avail Use% Mounted on
/dev/mapper/centos-root xfs 50G 3.6G 47G 8% /
devtmpfs devtmpfs 1.8G 0 1.8G 0% /dev
tmpfs tmpfs 1.9G 16K 1.9G 1% /dev/shm
tmpfs tmpfs 1.9G 17M 1.8G 1% /run
tmpfs tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/loop0 ext4 1.9G 6.1M 1.7G 1% /srv/node/swiftloopback
/dev/mapper/centos-home xfs 46G 33M 46G 1% /home
/dev/sda1 xfs 497M 213M 285M 43% /boot
tmpfs tmpfs 371M 0 371M 0% /run/user/0
apache1:/appgluster_vol fuse.glusterfs 20G 33M 20G 1% /mnt
[root@clusterserver3 ~]#
[root@clusterserver3 ~]# echo “here is a GlusterFS file” > /mnt/file
[root@apache1 ~]# cat /gluster/briks/file
here is a GlusterFS file
[root@apache2 ~]# cat /gluster/briks/file
here is a GlusterFS file
[root@apache2 ~]#
[root@apache3 ~]# cat /gluster/briks/file
here is a GlusterFS file
[root@apache3 ~]#
Recent Comments