After installing Apache with Load Balancer for a failover backend infrastructure on a Linux CentOS 7.x server ( see my article: https://www.angioni.nl/2018/03/02/use-apache-load-balancer-with-session-affinity-for-domino-webservers/), we still have a single point of failure on the frontend infra part.

You can use PaceMaker tooling for creating a front-end cluster based on a floating ipaddress between 2 Apache servers installed on 2 servers.

Here is the procedure to install and configure PaceMaker on a CentOS 7.x Linux server:

Serverinformation:

LoadBalancer-1: 192.168.100.10, Clustername: lbpri
LoadBalancer-2: 192.168.100.20, Clustername: lbsec
Virtual ipaddress: 192.168.100.30

Add the 2 clusternames in both the load balancers’ hostfile:
192.168.100.10 lbpri
192.168.100.20 lbsec

Now try to ping to them by using the clustername.

Installation PaceMaker on both servers:
yum install pacemaker pcs psmisc policycoreutils-python
(yum install corosync pcs pacemaker)

Enable and start services on both servers:
systemctl enable pcsd
systemctl enable corosync
systemctl enable pacemaker

systemctl start pcsd.service

Configuration on both servers:
passwd hacluster (user was created during installation )
<set password of your choice>

Configuration on lbpri:
pcs cluster auth lbpri lbsec
pcs cluster setup –name frontendwebcluster lbpri lbsec

pcs cluster start –all
pcs cluster enable –all

Check the cluster:
pcs status cluster
pcs status nodes
corosync-cmapctl | grep members
pcs status corosync

Check status STONITH
STONITH (Shoot The Other Node In The Head), which is a mechanism to ensure that you don’t end up with two nodes that both think they are active and claim to be the service and virtual IP owner, also called a split brain situation. This is a simple cluster, so we can disable this functionality.

crm_verify -L -V
pcs property set stonith-enabled=false

Set quorum to ignore ( multiple nodes as active for computing power )
pcs property set no-quorum-policy=ignore

Configure Virtual IPAddress for Apache listener:
pcs resource create virtual_ip ocf:heartbeat:IPaddr2 ip=192.168.100.30 cidr_netmask=32 op monitor interval=30s

Check virtual_ip:
pcs status resources
pcs status|grep virtual_ip

BUG Workaround for starting CoroSync:
Redhat bugzilla bug #1030583
vi /usr/lib/systemd/system/corosync.service

Add the following bold line:

[Service]
ExecStartPre=/usr/bin/sleep 10
ExecStart=/usr/share/corosync/corosync start

Cluster Apache:

Now we have a virtual ipaddress, we have to connect Apache to it. When Apache crashes or stopped manually, the failover kicks in to the other server.

Server-status:
Create server-status configfile which PaceMaker can use to check on Apache server-status:
vi /etc/httpd/conf.d/status.conf
<Location /server-status>
SetHandler server-status
Order Deny,Allow
Deny from all
Allow from 127.0.0.1
</Location>

Create Apache resource in PaceMaker:
pcs resource create ClusterWebServer ocf:heartbeat:apache configfile=/etc/httpd/conf/httpd.conf statusurl=”http://127.0.0.1/server-status” op monitor interval=5s

Create contraint so both resources must run on the same server:
pcs constraint colocation add ClusterWebServer virtual_ip INFINITY

Disable and stop httpd service in systemctl:

PaceMaker cannot take control over Apache when it’s still managed by systemctl so we have to disable it in systemctl.

servicectl stop httpd.service on both servers
servicectl disable httpd.service on both servers

Restart PCS on both servers:
pcs cluster stop
pcs cluster start

Now virtual_ip and ClysterWebServer resources are controlled by PaceMaker and activated on the same server.

pcs resource create ClusterWebServer ocf:heartbeat:apache configfile=/etc/httpd/conf/httpd.conf statusurl=”http://127.0.0.1/server-status” op monitor interval=5s

pcs constraint colocation add ClusterWebServer ClusterIP INFINITY

servicectl stop httpd.service op beide servers
servicectl disable httpd.service op beide servers

pcs cluster stop –all
pcs cluster start –all

pcs resource create ClusterWebServer ocf:heartbeat:apache configfile=/etc/httpd/conf/httpd.conf statusurl=”http://127.0.0.1/server-status” op monitor interval=5s

pcs constraint colocation add ClusterWebServer ClusterIP INFINITY

servicectl stop httpd.service op beide servers
servicectl disable httpd.service op beide servers

pcs cluster stop –all
pcs cluster start –all

PaceMaker Cluster errors:

For unknown reasons to me, sometimes access to the file /var/lib/pacemaker/cib/cib.xml will change into something wrong and therefor the clusternode cannot start. PCS thinks that the node is not a part of the cluster.

When you start PCS or give “pcs status” you see the error:

Error: this node is not part of a cluster


Examening the log /var/log/cluster/corosync.log you see the line:

error: pcmk__daemon_can_write:  /var/lib/pacemaker/cib/cib.xml must be owned and writable by either user hacluster or group haclient | st_mode=0100600l


Examening the /var/log/messages you see the lines:

May 12 22:04:03 928137-app2 cib[21587]: notice: /var/lib/pacemaker/cib/cib.xml is not owned by user hacluster
May 12 22:04:03 928137-app2 cib[21587]: notice: /var/lib/pacemaker/cib/cib.xml is not readable and writable by group haclient


The solution is to change security of the file
/var/lib/pacemaker/cib/cib.xml to 777. When you start pcsd again, it will change the security itself to the correct settings.


Visits: 701

By angioni

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.