Tomcat clustering and failover

 

In this post I will describe how to implement a functional Tomcat cluster with load balancing and failover

I did a great deal of research trying to get this environment working. There are a number of good tutorials for simple Tomcat clustering and load balancing. I was unable to find an example that was complete which provided a load balanced solution with failover.

Example 1 shows load balancing with a single server.

Example 2 shows load balancing with a multiple Tomcat servers.

Example 3 shows load balancing with failover which is described in this post.

The first step in setting up a cluster like we have above is to insure that multicast works in your environment. In order to check for multicast support on Linux first check to see if your kernel was compiled with multicast support by trying the following:

cat proc/net/dev_mcast

If the file doesn’t exist a kernel recompile is in order, the instructions for this are outside the scope of this post. Assuming we have kernel multicast support we must check that it is working. This can be done with a simple java utility contained in the zipfile multicasttest.zip. There are 2 classes in the zipfile, TestServer and TestClient first start the test client on all nodes in the cluster, then run TestServer on one of the nodes with the number of messages to send as a count. This will send a message every 3 seconds until the message count is reached. All of the clients should report each message in the form “Received: Message 1 of 10? and the server will display “Sending message 1?. If each of the clients report the messages your clustering should work.

Next configure your Tomcat servers for clustering and failover. Samples of the configuration file shown below are contained in the zipfile SampleConfig.zip.

Add the following block in your [TOMCAT]/conf/server.xml:

 

 

<!– mod_jk ajp protocol connector –>

<Connector port=”8009? protocol=”AJP/1.3? redirectPort=”8443? server=”Apache” />

 

<!-- Engine with clustering support set jvmRoute to "jvm"+id unique to tomcat instance -->

 

<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm2">

 

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8">

 

<Manager className="org.apache.catalina.ha.session.DeltaManager" expireSessionsOnShutdown="false" notifyListenersOnReplication="true"/>

 

 

 

 

<Channel className="org.apache.catalina.tribes.group.GroupChannel">

<!--Membership address and port should be set to the proper

value for the environment -->

<Membership className="org.apache.catalina.tribes.membership.McastService"

address="228.0.0.4" port="45564" frequency="500" dropTime="3000"/>

<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">

<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>

</Sender>

<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="auto" port="4000" autoBind="100" selectorTimeout="5000" maxThreads="6"/>

<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>

<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>

</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>

 

 

 

 

Create an [APACHE]/conf/httpd.conf:
Include conf/mod_jk.conf

Save the following to [APACHE]/conf/mod_jk.conf and enter the following:

LoadModule jk_module modules/mod_jk.so

# Specify path to worker configuration file
JkWorkersFile conf/workers.properties

# Configure logging and memory
JkShmFile logs/mod_jk.shm
JkLogFile logs/mod_jk.log
JkLogLevel debug

# Configure monitoring
JkMount /jkmanager/* jkstatus

<Location /jkmanager>
Order deny,allow
Deny from all
Allow from localhost
</Location>

# Configure applications
JkMount /myApp/* loadbalancer
JkMount /myOtherApp/* loadbalancer

 

Create an [APACHE]/conf/workers.properties and enter the following:

# Define worker names
worker.list=worker1, worker2, jkstatus, loadbalancer

# Create virtual workers
worker.jkstatus.type=status
worker.loadbalancer.type=lb

# Declare Tomcat server worker 1 on the localhost
worker.worker1.type=ajp13
worker.worker1.distance=1
worker.worker1.port=8009
worker.worker1.host=localhost

# Declare Tomcat server worker 2 on the remote host
# setting the host value to the remote ip address
worker.worker2.type=ajp13
worker.worker2.distance=10
worker.worker2.port=8009
worker.worker2.host=XXX.XXX.XXX.XXX #IP address of remote server

# Associate real workers with virtual LoadBalancer worker
worker.loadbalancer.balanced_workers=worker1, worker2
worker.loadbalancer.sticky_session=1

You should now be able to start your Apache and Tomcat instances and have clustering and failover.

For this example Tomcat 6.0.29 was used with Apache 2.2 and mod_jk 1.2.31.

 

 

 

Clustering allows us to run an application on several parallel servers (i.e. cluster nodes). The load is distributed across different servers, and even if any of the servers fails the application is still accessible via other cluster nodes. Clustering is crucial for scalable enterprise applications, as you can improve performance by simply adding more nodes to the cluster.

This document provides step by step instructions for configuring Apache server which handles  
request for static contents and delegates request for dynamic pages (i.e. JSP or Servlet) to two  instances of Tomcat server using JK Connectors.

• Installation of Apache Server
• Setting up tomcat instances 
• Configuration of Clustering for tomcat instances with JK Connectors 

Installation of Apache Server

Download windows installer for Apache Server 2.0.63 from http://httpd.apache.org/download.cgi and install the same into C:\Program Files\Apache 
Software Foundation.

Setting up tomcat instances

Download apache-tomcat-6.0.20 from http://tomcat.apache.org. Create 3 instances of tomcat named tomcatA, tomcatB with following configuration in conf/server.xml.

Uncomment the following entry in server.xml in all the instances of tomcat.

<!--
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
-->

 

Define the HTTP, AJP Connector, Shutdown ports for all the tomcat instances as below,

 

tomcatA

HTTP	8081
Shutdown	8105
AJP Connector	8109
jvmRoute	tomcatA

 

tomcatB

HTTP	8082
Shutdown	8205
AJP Connector	8209
jvmRoute	tomcatB

 

Deploying web application in all tomcat instances:

 

Deploy the Hello world JSP web application into all the tomcat instances, Make your HelloWorld.jsp with following content to feel the load balancing and session replication,

 

tomcatA – hello.jsp

 

<%
session.setAttribute(“name” , “bala”);
%>
From tomcatA - Name from session - <%= session.getAttribute(“name”); %>
Session ID - <%= session.getID() %>

 

tomcatB – hello.jsp

 

From tomcatB - Name from session - <%= session.getAttribute(“name”); %>
Session ID - <%= session.getID() %>

 

<distributable/> tag should be defined in web.xml of your application, so that session replication will be supported by the web container.

 

web.xml

 

<webapp>
…
..
<distributable/>
..
..
</webapp>

 

Configuring JK Connector for routing request to Tomcat from Apache Server:

 

Download JK connector 1.2.8 from http://tomcat.apache.org/connectors-doc and place mod_jk-1.2.28-httpd-2.2.3.so in modules/ directory of Apache server.

 

Add following configuration to httpd.conf file of Apache server,

 

LoadModule jk_module modules/mod_jk-1.2.28-httpd-2.2.3.so
JkWorkersFile C:\cluster\workers.properties
JkLogFile "logs/mod_jk.log"
JkLogLevel trace
JkMount /cluster loadbalancer
JkMount /cluster/* loadbalancer

 

workers.properties should have the details about the tomcat instances which we have created as below,

 

workers.tomcat_home=/tomcatA
workers.java_home=$JAVA_HOME
ps=/
worker.list=tomcatA,tomcatB,loadbalancer
worker.tomcatA.port=8109
worker.tomcatA.host=localhost
worker.tomcatA.type=ajp13
worker.tomcatA.lbfactor=1
worker.tomcatB.port=8209
worker.tomcatB.host=localhost
worker.tomcatB.type=ajp13
worker.tomcatB.lbfactor=1
worker.loadbalancer.type=lb
worker.loadbalancer.balanced_workers=tomcatA,tomcatB
worker.loadbalancer.sticky_session=1

 

Now restart the Apache Server, TomcatA and TomcatB instances and access the HelloWorld application via URL, http://localhost/Helloworld/hello.jsp.

 

Now you can see the session created by tomcatA will available in tomcatB also when accessing

the above URL repeatedly.