{"id":7240,"date":"2018-03-01T15:45:04","date_gmt":"2018-03-01T07:45:04","guid":{"rendered":"http:\/\/rmohan.com\/?p=7240"},"modified":"2018-03-01T15:45:04","modified_gmt":"2018-03-01T07:45:04","slug":"how-to-set-up-nginx-high-availability-with-pacemaker-and-corosync-on-centos-7","status":"publish","type":"post","link":"https:\/\/mohan.sg\/?p=7240","title":{"rendered":"How to Set up Nginx High Availability with Pacemaker and Corosync on CentOS 7"},"content":{"rendered":"

How to Set up Nginx High Availability with Pacemaker and Corosync on CentOS 7<\/strong><\/p>\n

We will create the Active-Passive Cluster or Failover-cluster Nginx web server using Pacemaker on a CentOS 7 system.
\nPacemaker is an open source cluster manager software that achieves maximum high availability of your services. It’s an advanced and scalable HA cluster manager distributed by ClusterLabs.
\nCorosync Cluster Engine is an open source project derived from the OpenAIS project under new BSD License. It’s a group communication system with additional features for implementing High Availability within applications.
\nThere are some applications for the Pacemaker interfaces. Pcsd is one of the Pacemaker command line interface and GUI for managing the Pacemaker. We can create, configure, or add a new node to the cluster with the pcsd command pcs.
\nPrerequisites<\/p>\n

[root@clusterserver1 ~]# cat \/etc\/hosts
\n127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
\n::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
\n192.168.1.20 clusterserver1.rmohan.com clusterserver1
\n192.168.1.21 clusterserver2.rmohan.com clusterserver2
\n192.168.1.22 clusterserver3.rmohan.com clusterserver3
\n[root@clusterserver1 ~]#<\/p>\n

Floating IP Address 192.168.1.25
\nRoot Privileges<\/p>\n

Now test the hosts’ mapping configuration.<\/p>\n

ping -c 3 clusterserver1
\nping -c 3 clusterserver2<\/p>\n

Install Epel Repository and Nginx
\nIn this step, we will install the epel repository and then install the Nginx web server. EPEL or Extra Packages for Enterprise Linux repository is needed for installing Nginx packages.<\/p>\n

Install EPEL Repository using the following\u00a0yum command.<\/p>\n

yum -y install epel-release<\/p>\n

Now install Nginx web server from the EPEL repository.<\/p>\n

yum -y install nginx<\/p>\n

systemctl start nginx
\nsystemctl enable nginx
\nsystemctl status nginx<\/p>\n

#Run Command on ‘clusterserver1’
\necho ‘<h1>clusterserver1 – TEST SERVER1<\/h1>’ > \/usr\/share\/nginx\/html\/index.html<\/p>\n

#Run Command on ‘clusterserver2’
\necho ‘<h1>clusterserver2 – TEST SERVER2<\/h1>’ > \/usr\/share\/nginx\/html\/index.html<\/p>\n

#Run Command on ‘clusterserver3’
\necho ‘<h1>clusterserver3 – TEST SERVER3<\/h1>’ > \/usr\/share\/nginx\/html\/index.html<\/p>\n

Install and configure Pacemaker, Corosync, and Pcsd
\nPacemaker, Corosync, and Pcsd are available in\u00a0the default system repository. So they\u00a0all can be installed from the CentOS repository using the following\u00a0yum command.
\nyum -y install corosync pacemaker pcs
\nAfter\u00a0the installation has been completed, enable all services to launch automatically at system boot\u00a0using the systemctl commands below.
\nsystemctl enable pcsd
\nsystemctl enable corosync
\nsystemctl enable pacemaker
\nNow start the pcsd Pacemaker command line interface on all servers.
\nsystemctl start pcsd
\nNext, create a new password for ‘hacluster’ user and use the same password for all servers. This user has been created automatically during software installation.
\nHere’s how you configure a password for the ‘hacluster’ user.
\npasswd hacluster
\nEnter new password:
\nHigh Availability software stack Pacemaker, Corosync, and Pcsd are installed on to the system.<\/p>\n

Create and Configure the Cluster<\/p>\n

In this step, we will create a new cluster with 3 centos servers. Then configure\u00a0the Floating IP address and add\u00a0new Nginx resources.
\nTo create the cluster, we need to authorize all servers using the pcs command and the hacluster user.
\nAuthorize all servers with the pcs command and hacluster user and password.
\npcs cluster auth clusterserver1 clusterserver2 clusterserver3
\nUsername: hacluster
\nPassword: test123<\/p>\n

[root@clusterserver1 ~]# pcs cluster auth clusterserver1 clusterserver2 clusterserver3
\nUsername: hacluster
\nPassword:
\nclusterserver3: Authorized
\nclusterserver2: Authorized
\nclusterserver1: Authorized
\n[root@clusterserver1 ~]#<\/p>\n

Now it’s time set up the cluster. Define the cluster name and all servers that will be part of the cluster.<\/p>\n

pcs cluster setup –name mohan_cluster clusterserver1 clusterserver2 clusterserver3<\/p>\n

[root@clusterserver1 ~]# pcs cluster setup –name mohan_cluster clusterserver1 clusterserver2 clusterserver3
\nDestroying cluster on nodes: clusterserver1, clusterserver2, clusterserver3…
\nclusterserver1: Stopping Cluster (pacemaker)…
\nclusterserver2: Stopping Cluster (pacemaker)…
\nclusterserver3: Stopping Cluster (pacemaker)…
\nclusterserver1: Successfully destroyed cluster
\nclusterserver3: Successfully destroyed cluster
\nclusterserver2: Successfully destroyed cluster<\/p>\n

Sending ‘pacemaker_remote authkey’ to ‘clusterserver1’, ‘clusterserver2’, ‘clusterserver3’
\nclusterserver3: successful distribution of the file ‘pacemaker_remote authkey’
\nclusterserver1: successful distribution of the file ‘pacemaker_remote authkey’
\nclusterserver2: successful distribution of the file ‘pacemaker_remote authkey’
\nSending cluster config files to the nodes…
\nclusterserver1: Succeeded
\nclusterserver2: Succeeded
\nclusterserver3: Succeeded<\/p>\n

Synchronizing pcsd certificates on nodes clusterserver1, clusterserver2, clusterserver3…
\nclusterserver3: Success
\nclusterserver2: Success
\nclusterserver1: Success
\nRestarting pcsd on the nodes in order to reload the certificates…
\nclusterserver3: Success
\nclusterserver2: Success<\/p>\n

[root@clusterserver1 ~]# pcs cluster start –all
\nclusterserver3: Starting Cluster…
\nclusterserver1: Starting Cluster…
\nclusterserver2: Starting Cluster…
\n[root@clusterserver1 ~]#<\/p>\n

[root@clusterserver1 ~]# pcs status cluster
\nCluster Status:
\nStack: unknown
\nCurrent DC: NONE
\nLast updated: Fri Mar 10 03:56:45 2017
\nLast change: Fri Mar 10 03:56:26 2017 by hacluster via crmd on clusterserver1
\n3 nodes configured
\n0 resources configured<\/p>\n

PCSD Status:
\nclusterserver1: Online
\nclusterserver2: Online
\nclusterserver3: Online
\n[root@clusterserver1 ~]#<\/p>\n

Disable STONITH and Ignore the Quorum Policy
\nSince we’re not using the fencing device, we will disable the STONITH. STONITH or Shoot The Other Node
\nIn The Head is the fencing implementation on Pacemaker. If you’re in production, it’s better to enable STONITH.
\nDisable STONITH with the following pcs command.<\/p>\n

pcs property set stonith-enabled=false
\nNext, for the Quorum policy, ignore it.
\npcs property set no-quorum-policy=ignore
\nCheck the property list and make sure stonith and the quorum policy are disabled.<\/p>\n

pcs property list<\/p>\n

[root@clusterserver1 ~]# pcs property list
\nCluster Properties:
\ncluster-infrastructure: corosync
\ncluster-name: mohan_cluster
\ndc-version: 1.1.16-12.el7_4.5-94ff4df
\nhave-watchdog: false
\nno-quorum-policy: ignore
\nstonith-enabled: false
\n[root@clusterserver1 ~]#<\/p>\n

The STONITH and Quorum Policy is disabled.<\/p>\n

Add the Floating-IP and Resources
\nFloating IP is the IP address that can be migrated\/moved automatically from one server to another server in\u00a0the same Data Center. And we’ve already defined the floating IP address for the Pacemaker High-Availability to be ‘10.0.15.15’. Now we want to add two resources, the Floating IP address resource with the name ‘virtual_ip’ and a new resource for the Nginx web server named ‘webserver’.
\nAdd the new resource floating IP address ‘virtual_ip’ using the\u00a0pcs command as shown below.<\/p>\n

pcs resource create virtual_ip ocf:heartbeat:IPaddr2 ip=192.168.1.25 cidr_netmask=32 op monitor interval=30s<\/p>\n

Next, add a new resource for the Nginx ‘webserver’.<\/p>\n

pcs resource create webserver ocf:heartbeat:nginx configfile=\/etc\/nginx\/nginx.conf op monitor timeout=”5s” interval=”5s”<\/p>\n

Make sure you got no error result, then check the resources available.<\/p>\n

pcs status
\npcs status resources<\/p>\n

You will see two resources ‘virtual_ip’ and a ‘webserver’. New resources for the Floating IP and Nginx web server have been added.<\/p>\n

Add Constraint Rules to the Cluster<\/p>\n

In this step, we will setup High Availability Rules, and will setup resource constraint with the pcs command line interface.
\nSet the collation constraint for webserver and virtual_ip resources with score ‘INFINITY’.
\nAlso, setup the webserver and virtual_ip resources as\u00a0same on all server nodes.<\/p>\n

pcs constraint colocation add webserver virtual_ip INFINITY<\/p>\n

Set the ‘virtual_ip’ and ‘webserver’ resources always on same node servers.<\/p>\n

pcs constraint order virtual_ip then the webserver<\/p>\n

pcs constraint colocation add webserver virtual_ip INFINITY<\/p>\n

Next, stop the cluster and then start again.<\/p>\n

pcs cluster stop –all
\npcs cluster start –all<\/p>\n

[root@clusterserver1 ~]# pcs cluster stop –all
\nclusterserver1: Stopping Cluster (pacemaker)…
\nclusterserver3: Stopping Cluster (pacemaker)…
\nclusterserver2: Stopping Cluster (pacemaker)…
\nclusterserver3: Stopping Cluster (corosync)…
\nclusterserver1: Stopping Cluster (corosync)…
\nclusterserver2: Stopping Cluster (corosync)…
\n[root@clusterserver1 ~]# pcs cluster start –all
\nclusterserver1: Starting Cluster…
\nclusterserver2: Starting Cluster…
\nclusterserver3: Starting Cluster…
\n[root@clusterserver1 ~]#
\n[root@clusterserver1 ~]#
\n[root@clusterserver1 ~]#<\/p>\n

Testing
\nIn this step, we’re gonna do some test for the cluster. Test the node status (‘Online’ or ‘Offline’), test the corosync members and status, and then test the high-availability of the Nginx webserver by accessing the Floating IP address.
\nTest node status with the following command.<\/p>\n

[root@clusterserver1 ~]# pcs status nodes
\nPacemaker Nodes:
\nOnline: clusterserver1
\nStandby:
\nMaintenance:
\nOffline: clusterserver2 clusterserver3
\nPacemaker Remote Nodes:
\nOnline:
\nStandby:
\nMaintenance:
\nOffline:
\n[root@clusterserver1 ~]#<\/p>\n

Test the corosync members.
\ncorosync-cmapctl | grep members
\nYou will get Corosync members IP address<\/p>\n

[root@clusterserver1 ~]# corosync-cmapctl | grep members
\nruntime.totem.pg.mrp.srp.members.1.config_version (u64) = 0
\nruntime.totem.pg.mrp.srp.members.1.ip (str) = r(0) ip(192.168.1.20)
\nruntime.totem.pg.mrp.srp.members.1.join_count (u32) = 1
\nruntime.totem.pg.mrp.srp.members.1.status (str) = joined<\/p>\n

[root@clusterserver1 ~]# pcs status
\nCluster name: mohan_cluster
\nStack: corosync
\nCurrent DC: clusterserver1 (version 1.1.16-12.el7_4.5-94ff4df) – partition WITHOUT quorum
\nLast updated: Fri Mar 10 04:12:59 2017
\nLast change: Fri Mar 10 04:11:59 2017 by root via cibadmin on clusterserver1<\/p>\n

3 nodes configured
\n2 resources configured<\/p>\n

Online: [ clusterserver1 ]
\nOFFLINE: [ clusterserver2 clusterserver3 ]<\/p>\n

Full list of resources:<\/p>\n

virtual_ip (ocf::heartbeat:IPaddr2): Started clusterserver1
\nwebserver (ocf::heartbeat:nginx): Started clusterserver1<\/p>\n

Daemon Status:
\ncorosync: active\/enabled
\npacemaker: active\/enabled
\npcsd: active\/enabled
\n[root@clusterserver1 ~]#
\n[root@clusterserver1 ~]#
\n[root@clusterserver1 ~]#
\n[root@clusterserver1 ~]#<\/p>\n","protected":false},"excerpt":{"rendered":"

How to Set up Nginx High Availability with Pacemaker and Corosync on CentOS 7<\/p>\n

We will create the Active-Passive Cluster or Failover-cluster Nginx web server using Pacemaker on a CentOS 7 system. Pacemaker is an open source cluster manager software that achieves maximum high availability of your services. It’s an advanced and scalable HA cluster […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[73,70],"tags":[],"_links":{"self":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/7240"}],"collection":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=7240"}],"version-history":[{"count":1,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/7240\/revisions"}],"predecessor-version":[{"id":7241,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/7240\/revisions\/7241"}],"wp:attachment":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=7240"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=7240"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=7240"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}