Glassfish Cluster: LoadBalancer with Apache (ENG)

Glassfish Cluster: LoadBalancer with Apache

(valid for glassfish v2ur2)

The loadbalancer task is to distribute the traffic between different nodes in the cluster.

The load balancing between the various nodes in the cluster can be achieved in several ways:
LVS, mod_jk, plugins for Glassfish, Apache Rewrite Rule using dedicated equipment etc. etc.
The objective is to examine the different nodes in a particular algorithm in order to load the various nodes in accordance with the requirements of the system.

In subsequent steps, reference is made to configure a LoadBalancer, using the Glassfish plugin and apache webserver.

Using the solution plug-in you can take the following advantages:

- Integration with application servers
- Ability to configure loadbalancing from admin console:

-  Integrated Health Checking System, the module determines if a request is not able to respond in order to eliminate from balancing load.

- Configuring the algorithm of load distribution, with the possibility to load a user-defined one.

- Automatic balancing module, the plugin allows you to automatically or manually export the configuration file for the module so as to make it current, even in real-time, on the structure of the cluster (eg. Addition or deletion of nodes) .


The communication between the Web server that acts as loadbalancer and the DAS is via SSL.
It 'necessary to enable the apache server to support SSL.

The plugin for Glassfish is considered compatible with apache 2.0.x (32 bit version)
so do not use the newer 2.2 or higher.

The steps to be performed are divided into the following macro-steps:

  • Installing and Configuring Apache with SSL support
  • Install the plugin and configure Apache to communicate with the plugin
  • Create and enable the plugin in the DAS

Installing and Configuring Apache with SSL support

The following steps refer to the Windows platform:
For Linux or Solaris, you can follow one of several guides on the web, or alternatively refer to the guidance provided by the sun

Download a version of Apache with openssl:

  • Enable the module on httpd.conf by removing the comment '#'

Ssl_module LoadModule modules /

  • Ensure the existence of the following in httpd.conf lines

<IfModule Mod_ssl.c> 
Include conf / ssl.conf 
</ IfModule>

  • Generate the certificate and the key for apache

%% ApacheDIR bin openssl.exe req-new-x509-keyout newreq.pem-out newreq.pem-days 365-config "% ApacheDIR% confopenssl 
. CNF "
(substitute the appropriate path for the file openssl.cnf used) 
Note: the generation of the certificate creates problems with the environment variable that indicates the location of openssl configuration, best to use the switch-config as shown.

WARNING: the common name applied for must correspond to the hostname used on apache. This value will then during the configuration of Glassfish plugin on the host device.

  • Opening with a text editor file newreq.pem

  • copy the portion that goes from

----- END CERTIFICATE ----- 

ApacheDIR in file%% / conf / ssl.crt / server.crt 

Instead copy the portion that goes from 
----- END RSA PRIVATE KEY ----- 

ApacheDIR in file%% / conf / ssl.key / server.key 

Note: If you entered a passphrase when generating the key file server.key will have a version of the cryptographic key. Apache, have shown during the opening a dialog to enter your passphrase. 
This dialog box is NOT supported on windows, so you need to remove the passphrase from the key with the following command: 

ApacheDIR%% / bin / openssl rsa-in server.key-out serverNoPhrase.key 

The serverNoPhrase.key is obtained without passaphrase. Make sure the file ssl.conf it points to that file rather than the encrypted or copy serverNoPhrase.keyin server.key and delete serverNoPhrase.key. 

  • Start Apache in SSL mode. In windows the command is as follows:
apache-D SSL

  • Test the operation of apache with ssl not forgetting the browser to use https.

WARNING: There may be conflict in the door between Glassfish and apache. 
Reading the log (error.log in apache) or (in Glassfish server.log) to identify the conflict and act accordingly.

Install the plugin and configure Apache to communicate with the plugin

  • Download the plugin from the following link:


Note: surf to search for any newer versions and other platforms 

In this configuration has been used "aslb-9.1-MS4-b7.jar"

  • Unpack the file. Jar using the command jar-xvf nomefile.jar or using an archiver like WinRar.
  • Zip files obtained ( and should be unpacked in the folder lib / lbplugin server
then create the folder "lbplugin" on% GLASSFISH_HOME / lib / and unpack the contents of two files inside. zip 

Note: in linux assign permissions with the command: chmod-R 755 <GLASSFISH_HOME> / lib / lbplugin / lib 

  • Copy the file mod_balancer.dll in <GLASSFISH_HOME> / lib/lbplugin/lib/webserver-plugin/windows/apache2 /
within the modules directory of Apache. 

  • Create and copy within ApacheDir%% / modules / resource files. <GLASSFISH_HOME> In res / lib/lbplugin/lib/webserver-plugin/windows/apache2 /

  • Create and copy within ApacheDir%% / modules / errorpages files in <GLASSFISH_HOME> / lib/lbplugin/lib/webserver-plugin/windows/apache2/errorpages

  • Create and copy within ApacheDir%% / sec_db_files files. <GLASSFISH_HOME> In db / lib/lbplugin/lib/webserver-plugin/windows/apache2 /

  • Copy within ApacheDir%% / conf file loadbalancer.xml.example in <GLASSFISH_HOME> / lib / lbplugin / lib / install / templates / loadbalancer.xml.example
and rename loadbalancer.xml

  • Restart Apache in SSL mode

Create and enable the plugin in the DAS

E 'advisable at this stage of the setup of the open console can read in real time the contents of file error log (for Glassfish server.log and error.log for apache, also in the case of the cluster light not only on the server. log of the domain but also the hub and various forums)

You need to login the Admin console of the DAS, positioned on HTTP Load Balancers and click New to create a new reference to the load balancer.

  • Choose a name for the load balancer, for example apacheLB
  • How Device Host, the host of the same apache configuration in ssl.conf, typed in the same common name when creating the file newreq.pem (certificate + key)
  • How Devce Admin Port, the port number of apache ssl (default is 443)
  • Select the correct target in a way that the points on the cluster loadbalancer
  • The Apply Changes Automatically checkbox can be selected if you want to automatically notify the module to apache balancing with respect to changes in structure of the cluster (eg, adding or deleting nodes instance)
  • After saving the test configuration apacheLB selecting and clicking on Test Connection

Note: In the Export section of the LoadBalancer exists the possibility to export the configuration files necessary to loadbalancer.xml form of balancing apache or edit by clicking on "Apply Changes Now".
About the automatic generation of xml files to the directory of apache on linux you must correctly set the different permissions.

Make sure that the instances in the cluster have enabled load balancing (Click on "Enable Load Balancing" as per image)

The Plug In is configured.

Installing an application on the cluster and check if your browser meets one of the nodes by calling


Note: The configuration has been implemented using SSL for communication between the DAS and apache, the load balancing is the case using the normal http protocol.

Possible Problems

Trying to make repeated requests to the loadbalancer you may experience the following errors (see error.log for apache)

[warn] lb.runtime: RNTM2024: Daemon http://MOBISERVER:38081 is unhealthy. 
[warn] lb.runtime: RNTM2030: Daemon Monitor: http://MOBISERVER:38081: could be because daemon is down 
[warn] lb.runtime: RNTM2025: Daemon http://mobipc:38080 is healthy. 
[warn] lb.runtime: RNTM2025: Daemon http://mobipc:38080 is healthy. 
[warn] lb.runtime: RNTM2025: Daemon http://mobipc:38080 is healthy. 
[warn] lb.healthchecker: HLCK3003: Listener: http://MOBISERVER:38081 is still detected to be unhealthy in cluster: cluster 
[warn] lb.healthchecker: HLCK3003: Listener: http://MOBISERVER:38081 is still detected to be unhealthy in cluster: cluster 
[warn] lb.healthchecker: HLCK3003: Listener: http://MOBISERVER:38081 is still detected to be unhealthy in cluster: cluster 
[warn] lb.healthchecker: HLCK3003: Listener: http://mobipc:38080 is still detected to be unhealthy in cluster: cluster 
[warn] lb.healthchecker: HLCK3003: Listener: http://mobipc:38080 is still detected to be unhealthy in cluster: cluster

If instances of the cluster are running in the state and if I answer calls directly on their ip: port means you may be some problem nell'httpd. Conf, it is necessary to check that the virtual host name and server have the correct number name.

In addition, the Apache error.log file is updated by module mod_balancer, and then provide the appropriate automatic rotation of log files or to set a different log level.

Reference Link:

The article was translated from the italian version i'm wrote before.
I translate it with google translate service and after a fast review and some correction I think that it is a bad english too(I'm sorry).
The article that I wrote directly in english are a little bit better.
however I hope it is helpful.
If you want to correct any sentences please contact me through a comment, don't hesitate!

Clustering and EJB Timer (ENG)

Glassfish Cluster and EJB Timer

clicca qui per la versione in italiano

One of EJB features is "Timer Service".
It allows you to activate timer for specified periods of time or after certain dates.

This service relies on a embedded Glassfish database.
Go into the console between the connection pool (see JDBC resources) you can see is the connection pool __TimerPool "used by the resource __TimerPool.

This resource is available by default for the target server, then the cluster is not even aware of a db that supports the timer service.

Furthermore, the embedded database is not sufficient as instances of the cluster can not access the db and then know the status of management of the timer service.

Having to deploy on Glassfish cluster that uses an EJB timer service should set up an external db can support transactions and ensure that only one instance to handle the timeout without replicating the events more than necessary.

Steps to configure:

To configure the cluster so that it supports the EJB Timer Service is necessary to follow the following steps

- Create a db (eg TIMER_DB)

- Create in a table using sql scripts that are within $ GLASSFISH_HOME / lib / install / databases
The script mysql is not present you can copy it from here:


Note: The table name must be uppercase

- Make in "Resource-> Connection Pools" a connection pool that can communicates with the db created.

- Create a resource pointing to the jdbc connection pool you just created.
The resource must be created as the target cluster, to ensure that you can create the resource by clicking on
"Cluster-> name cluster-> Resources-> New-> JDBC Resources"

- Go on the configuration of the cluster (The default is called cluster-config)
select EJB container,
click EJB Timer Service
set in the Timer Datasource jdbc resource name created in previous step.

- Restart the cluster.

Driver XA and Possible problems

To make a good setup needs to be created, as I suggested earlier, a resource jdbc appropriate for the timer.
This choice, however, can create problems when the application that makes use of the timer service simultaneously access a resource jdbc different (for example through hibernate you log on to the connection pool of the db).

In this case the same bean needs to access two resources at once and ensure the ACID properties.
To do this you need a JDBC driver XA-capable

This driver obviously needs to be available in the classpath, eg copying it into the / lib of the domain (default domain1).
(Note: when you update a library, you must restart the application server)

In the case of mysql, the J connector (currently version 5.1.6) is XA compliant.

Besides having a driver with such characteristics is necessary to set the connection pool so that these features are used.
Then in the connection pool should be created as set Resource Type "java.sql.XADataSourcein the field instead DataSourceClassName must set the correct value based on the DBMS used

In the case of this class is mysql

Unfortunately, when the connector J creates problems due to bug:

Consequently when you restart the timer (For example, when you restart an instance of the cluster) will otter the following exception:

INFO: Exception thrown from bean; nested exception is: java.lang.IllegalArgumentException: null source jav ... 

Caused by: java.lang.IllegalArgumentException: null source 
at java.util.EventObject. <init> ( 38) 
at javax.sql.StatementEvent. <init> ( 39) 
com.mysql.jdbc.jdbc2.optional.JDBC4PreparedStatementWrapper.close (JDBC4PreparedStatementWr 70) 
at com.caucho.sql.UserStatement.close ( 127) 
at com.caucho.sql.UserPreparedStatement.close ( 450) 
at org.hibernate.jdbc.AbstractBatcher.closePreparedStatement ( 534) 

caused by the bug in question.

In order to resolve the issue, pending a new release you can use a nightly build of the connector j providing for the bug fix.
(Note: When you reinstall a library jar in Glassfish must restart).

I settled with that of September 20.


The article was translated from the italian version i'm wrote before.
I translate it with google translate service and after a fast review and some correction I think that it is a bad english too(I'm sorry).
The other article that I wrote directly in english are a little bit better.
however I hope it is helpful.
If you want to correct any sentences please contact me through a comment, don't hesitate!

Setup Glassfish Cluster (ENG)

Glassfish Cluster

A cluster usually consists of a machine that acts as an administrator and a set of nodes that perform the work of the applications installed on them.
The objective of a cluster is usually to improve the system scalability and high availability .

The machine acting as an administrator in Glassfish clustering is called DAS (Domain Administrator Server).

Each node is made up of 2 elements:

1) Node Agent: It manages the life cycle of the application to which it is associated.
2) Application: Application server ..

The applications are installed on the DAS and are automatically propagated in nodes.

Cluster Support

Support cluster must be installed only on the DAS machine.

The Glassfish can be installed directly with the active cluster support or you can activate after a standard installation.

  • To enable support during installation use the following command:% libantbinant-f setup-cluster.xml instead of% libantbinant-f setup.xml

  • To enable the support afterwards (see picture above):
    • - Open the admin console
    • - Select from the common task: Enable Cluster Support
    • - Click Ok

Now it is prepared in this way the machine will be the DAS.

Cluster setup with a node

Suppose you want to create a cluster minimal (no sense in real situations):

A single machine is the DAS and also it is the node(later we add the second node).

In The  DAS machine, which have enabled cluster support will create the first node.

Note: The nodes can be created only from the command line. From the panel you can add a node but it is just a placeholder .. or a predisposition to create real

Creating Node Agent

From the command line type:

asadmin create-node-agent nodoDAS

nodoDAS is the name that was chosen for the node agent on DAS machine.
This command creates a node agent on DAS machine.

After you create must be started with the following command:

asadmin start-node-agent nodoDAS

Cluster Creation

Panel will now proceed with the creation of the cluster.

Clusters go on and click the button to create a new cluster.

About configuring the cluster that will leave the options on "Configuration" is unchanged will create a configuration
default-config, the system will call the copy "cluster-config". From this moment you can configure differently as regards the cluster different from that which concerns the application stand-alone acting on the different configurations.

Instance creation

Once the cluster is carried by adding the first instance and associating the nodoDAS.
You call this instance istanzaDAS.
In order to proceed with the creation, click the cluster you just created (it is located inside the item Clusters, in the previous image there is already called a cluster), then click on the Instances tab and then click New.
Enter the name and associate it instanzaDAS to NodeAgent nodoDAS after click on ok.

We have just created an instance associated with the node created by default at each stop and start the node corresponding similar operation The application is created.

State of Cluster

Making a summary on what has been achieved so far, we have a machine that acts as a DAS which is the main controlling nodeAgent one instance (instanceDAS).

Adding Second Node

To give meaning to the cluster will add a second node.

Note: You do not need the media cluster on the second machine.

The steps needed to achieve this objective are:

1) create a node agent

use the command asadmin on the machine where we want to create the second node: 

asadmin create-node-agent - host macchinaDAS nodo1 

command is equivalent to the one used to create nodoDAS but this time you must add the parameters to indicate the function of creation 
(create-node-agent) of the node that the agent machine where resides the DAS is not localhost (default value) but it is another machine.
The command used is assumed that the machine on which you installed the DAS has the hostname macchinaDAS. 

NOTE: E 'advisable to use a hostname rather than the ip, the JMX API can create some problems. 
Do not forget about linux to edit the file / etc / hosts / hostname with the choice. 

NOTE: When prompted admin admin user name and password when creating the node, enter the credentials for the login to DAS. 

The node has been created called nodo1. 

2) start the node

run from the console of the machine with the command nodo1 

asadmin start-node-agent nodo1 

in order to start the new node agent. 

3) create the istanza1 on the machine with nodo1

This can be done by hand panel. 
Going on the cluster and adding as done previously for instanzaDAS a new instance associating the node agent nodo1. 

the application created the call istanza1 

4) start istanza1

Click on the cluster, click istanza1, click on "start instance". 

Now we have two machines

DAS machine with the DAS, an instance (istanzaDAS), the main node Agent (nodeAgentDAS).
1 car with an instance (instanza1), the corresponding node Agent nodo1.

Start the Cluster

Feasible operation panel of DAS.

Click on the cluster and on the "Start Cluster".

The Cluster Test

To test the cluster, you can use a sample application provided with the same Glassfish.
Ear file should reside in the folder samples / quickstart / clusterjsp.

Effettauare deploy the panel by paying attention to the target on which you install the application.
We must remove the target server and instead add the target cluster.

Select the Availability flag if the application makes use of sessions and these sessions must be replicated in different nodes in the cluster.

If all went well, the application should be installed on both machines.
To check this in your browser, type the following url:


DAS is the machine that the machine 1.

NOTE: As you can see the URL of the cluster instances listen on port 38080 and not 8080. 8080 is about listening to standalone application server that despite the cluster is still running. Obviously not necessary if you can stop it and use only those instances configured in the cluster.

The article was translated from the italian version i'm wrote before.
I translate it with google translate service and after a fast review and some correction I think that it is a bad english too(I'm sorry).
The article that I wrote directly in english are a little bit better.
however I hope it is helpful.
If you want to correct any sentences please contact me through a comment, don't hesitate!