Enportal/5.5/admin/enPortal installation/installation/clustering and failover: Difference between revisions

imported>Jason.nicholls
m (1 revision)
 
imported>Jason.nicholls
 
(6 intermediate revisions by the same user not shown)
Line 1: Line 1:
[[Category:enPortal 5.5]]
[[Category:enPortal 5.5]]
{{DISPLAYTITLE:Clustering & Failover}}
{{DISPLAYTITLE:Clustering & Failover}}
== Architecture ==
== Overview ==


The enPortal/AppBoard system is implemented as a web application architecture. It runs as a web application inside an Apache Tomcat Server, and accesses a JDBC accessible RDBMS or clustered RDBMS.   The enPortal/AppBoard web application server scales horizontally by replicating on additional servers/platforms.
enPortal is implemented using a highly scalable web application architecture. As a Java application running inside an Apache Tomcat server, enPortal is able to make use of multi-core and multi-processor systems with large amounts of RAM on 64-bit operating systems. In addition to scaling vertically on a single system, enPortal supports horizontal scaling to handle even greater loads and/or to provide for high availability environments through the use of a shared configuration database.
enPortal can be used in the following configurations:


Horizontal scaling can be accomplished by adding new enPortal/AppBoard servers to the system in order to grow the capacity of the enPortal/AppBoard cluster. The diagram below shows a load-balanced cluster.
#<b>Load Balanced</b>: Two or more nodes are fully operational at all times. The load balancer directs traffic to nodes based on standard load balancing techniques such as round-robin, fewest sessions, smallest load, etc... If a server is detected as down it is removed from the active pool.
#<b>Failover</b>: A two-node configuration with both nodes running but all traffic is routed to the primary node unless it is detected to be down. At this point the load balancer re-directs traffic to the secondary node.
#<b>Cold Standby</b>: A two-node configuration where the secondary node is offline in normal operation. If the primary node is detected to be down the secondary node is brought online and traffic re-directed.


In cases where high-availability is required then regardless of the load a cluster configuration is recommended. In cases where load is a concern refer to the [[enportal/5.5/admin/enPortal_installation/performance_and_sizing|Performance Tuning & Sizing]] documentation for more information.


[[Image:Simple_Cluster.png|thumb|center|700px|n-node Cluster of enPortal/AppBoard]]
== Architecture & Licensing ==


[[File:Simple_Cluster.png|frameless|756px|center|Two Node Cluster Architecture Diagram]]


In the above example, assuming n=2, there are three options for configuring the availability of non-primary nodes:
Whether running a load-balanced, failover, or cold-standby configuration the following components are required:


#<b>Load Balanced</b> - Node 1 and Node 2 are fully operational at all times.  The Load Balancer directs traffic across the two nodes based on standard load balancing techniques such as round-robin, fewest active sessions, smallest load, etc.
* enPortal installation per node, this requires a separate license for each node.
#<b>Failover</b> - All traffic is routed to the AppBoard server on node 1, unless node 1 is detected to be down, in which case traffic is automatically re-routed to node 2. Node 1 and Node 2 are powered up and available at all times.
* External (shared) configuration database. This database is not provided by Edge and is recommended to reside on a different host to the enPortal servers. In high availability environments the database itself should also highly available. See the [[enportal/5.5/overview/system_requirements|System Requirements]] for supported external configuration databases.
#<b>Cold Standby</b> - All traffic is routed to the AppBoard server on node 1. If there is an outage on node 1, node 2 is brought online and traffic is re-directed.
* Load Balancer. This component is not provided by Edge but is required in cluster configurations.


== Cluster Configuration ==


== Load Distribution ==
The overall cluster configuration is made up of separate parts that follow the cluster architecture:


The Load Balancer can distribute sessions to one or more enPortal/AppBoard nodes using any standard load balancing algorithm (e.g. Round-Robin, smallest load, fewest sessions, etc.).  The only requirement is that the session affinity is maintained such that a single user is always routed to the same enPortal/AppBoard node during the full duration of the session.  enPortal/AppBoard manages the session handling for all of the Tomcat processes and integrated applications within the enPortal/AppBoard component of the solution.  Since enPortal/AppBoard is managing session state with integrated back-end web aplications, switching between enPortal/AppBoard servers during a session can cause unexpected session time-outs and additional licensing costs.
# Load Balancer configuration
# Shared enPortal configuration: via an external shared configuration database.
# per-node enPortal configuration and filesystem assets.


The two session cookies used by enPortal/AppBoard are JSESSIONID and enPortal_sessionid.  When configuring the Load Balancer for session affinity, it is recommended to use enPortal_sessionid to avoid any conflicts with other applications that may also have a JSESSIONID cookie.  A test page (check.jsp) is available from Edge for purposes of testing the Load Balancer configuration.  You can drop this test page into the /enportal/ directory of each server node in the system.  If you then direct the Load Balancer to hit that page directly (e.g. https://<ipaddress>:<port>/enportal/check.jsp), it will return a 200 code if all segments of the enPortal system are running properly (including database access), and will return a 500 code if there is a failure.
Also consider that establishing a new cluster and maintaining the cluster may have different approaches as outlined below.


=== Shared Configuration Database ===
In simple single-server enPortal configurations it is recommended to use the built-in, in-memory, H2 configuration database. However, in cluster configurations the configuration needs to be shared and kept in sync across two or more nodes so an external configuration database is required. Setting up an external database for redundancy operation has two main steps:


== Designing your enPortal/AppBoard system ==
# Configure enPortal to use an external database (versus the build-in H2). This process is documented in isolation on the [[enportal/5.5/admin/configuration_database|Configuration Database]] page.
# Configure enPortal to operate in redundancy mode.


The physical design of your enPortal/AppBoard system requires an examination of each of the following interrelated
{{Note|The above points out the main steps in isolation, please refer to the ''Establishing a Cluster'' for more details.}}
factors:


*Initial load (users) and expected growth
=== Per-Node Configuration & Assets ===
*Host hardware and operating system selection
While the shared configuration database takes care of enPortal content and provisioning information, there is other configuration and filesystem assets that need to be maintained per node:
*Network layer requirements
*Security


* license file
* configuration database connection details
* all other filesystem assets such as login pages, images for look and feel, enPortal PIM files, and other miscellaneous pieces that have been built into the solution such as custom JSPs, CGIs, HTML/CSS/JS, etc...


Of all of these factors, the “initial load (users) and expected growth” is typically in the forefront of the customer’s mind when designing an enPortal/AppBoard system. The total number of users having access to and the number of users logged in to an enPortal/AppBoard system at one time are the two user load metrics that they are most concerned with.
The recommended approach, which also serves to ensure full-backups are made of the system, is to configure the [[enportal/5.5/admin/enPortal_installation/backup_and_recovery#Customizing_the_Export|Backup export list]] to include all filesystem assets. Then when establishing, or updating a cluster, the archive can be used to maintain the filesystem components. Also, other custom approaches may also be suitable such as filesystem synchronization tools.  


The amount of load induced on a system by a single user is certainly not a constant. There are many factors
The license and database configuration will need to be handled when first establishing the cluster but will not need to be changed after that point.
that affect this load including: the network bandwidth between the user’s machine and the enPortal/AppBoard
system, the performance of communications between elements in your integrated solution, the activities the user is performing, and the activity level of the user. However, the following guidelines will enable you to proceed with
the design of your enPortal/AppBoard system:


*Use the general rule of “100 concurrent active end-users per node” to get an initial assessment of cluster size. System administrators can incur additional overhead, so if a system is going to have a large number of administrators this should also be taken into consideration, especially if they will be making concurrent changes to the system configuration.
=== Load Balancer ===
*If your system needs to support over 200 concurrent users or if high availability is a key requirement for your system, then design your enPortal/AppBoard system as a cluster from the start (rather than a single
The Load Balancer can distribute sessions to one or more enPortal nodes using any standard load balancing algorithm (e.g. Round-Robin, smallest load, fewest sessions, etc.). The only requirement is that the '''session affinity is maintained''' such that a single user is always routed to the same enPortal node during the full duration of the session.
box solution).
*A more accurate assessment of the scalability of your particular enPortal/AppBoard system can be determined by simulating end-user usage against a single enPortal/AppBoard node. A linear extrapolation
of this value can then be used to determine the number of nodes in your cluster. However, because the addition of new nodes to an enPortal/AppBoard cluster is quite simple, it is often better to start with a single box or 2-node cluster to establish a baseline, and grow the cluster as dictated by real world needs.


The following general guidelines should also be followed when designing your enPortal/AppBoard system:
The two session cookies used by enPortal are <tt>JSESSIONID</tt> and <tt>enPortal_sessionid</tt>. When configuring the Load Balancer for session affinity, it is recommended to use '''<tt>enPortal_sessionid</tt>''' to avoid any conflicts with other applications that may also have a JSESSIONID cookie.  
*Memory selection: be sure to allocate enough physical memory to enPortal/AppBoard. A fully loaded enPortal/AppBoard node can require 4 GB or more of RAM
*Database location: for security and availability purposes, the enPortal/AppBoard database is often hosted on its own server. In this case, minimize the latency of network communications between the application server and your database. A simple ping between each portal server and the database server will determine this latency. (Recommendation: <= 20ms latency)
*IP Load Balancer: a load balancer is necessary for clustered enPortal/AppBoard systems. The CITRIX NetScaler Server Appliance is a recommended hardware solution. The critical configuration that is necessary for any IP Load Balancer solution is to establish a client-to-server affinity, in order to make sure that web browser talks to the same enportal server for the length of each user’s session.


The host hardware and operating system selection is often dictated by external influences rather than enPortal/
The following URL can be used by the load balancer as a means of testing enPortal availability:
AppBoard specific benefits. This is not a concern due to enPortal/AppBoard’s platform independence. Support for scalability through clustering of many nodes makes Solaris, Windows, and Linux based solutions all
viable for large deployments.


"Network layer requirements” and “Security” are often the same issue. Since enPortal/AppBoard is often used as
<tt>http://''server:port''/enportal/check.jsp</tt>
an intermediary between a public audience and private resources, it needs to safely bridge the security gap between
these two entities. The multi-tiered architecture allows an enPortal/AppBoard system to be segmented across
multiple firewall-protected networks.


This script returns a HTTP status code 200 (success) if all components of enPortal are running properly, otherwise it returns a 500 (internal error) if there is an issue. And in the case the enPortal server isn't running, then of course there will be no response.


== Licensing Requirements of a Multiple Node Solution ==
=== Virtualized Environments ===
Whether running on the bare metal or within virtualized environments the clustering configuration remains the same.


The higher availability and greater capacity of a multiple node solution have already been established. However, it should be noted and planned ahead that there may be additional licensing requirements. The following costs related to designing a multiple node solution should be considered before departing from a typical single-node solution:
Some virtualization environments may offer their own layer of fault tolerance although this is usually targeted at reducing/eliminating the impact of hardware failure - e.g. VMware Fault Tolerance to transparently failover a guest from a failed physical host to a different physical host such that everything continues un-interrupted. This type of system is useful on it's own but may not be aware of application-level failures that can also occur.


*Each enPortal/AppBoard server node is a separate portal installation. Any filesystem changes (patches, configuration changes, etc.) that are required to maintain the solution must be performed on each node.
== Establishing a Cluster ==
*An enPortal/AppBoard server license must be purchased for each server node.
*The load balancing device (not included with enPortal/AppBoard) must be acquired.


The following process can be used to establish a cluster environment. If you're skipped straight here, please go back and read over the previous sections to understand the overall architecture and configuration components.


== enPortal/Appboard Replication for Clustering ==
The following process assumes an existing environment, although a clean install environment can be used too.
The enPortal/AppBoard system is implemented on top of a Apache Tomcat application container, with its more dynamic configurations stored in a standalone database.  It also contains design time file system artifacts that are used to store customizations such as Content Retrieval Service rule sets and configuration information for AppBoard Stacks.  Thus, it is important to consider that both the database and the file system artifacts are replicated and managed properly in a clustered system.


For a clustered system to work properly, it is essential that all of the enPortal/appBoard instances remain in sync with one another.  Because the same database is shared, it always remains in sync. In addition, however, the filesystems must also remain in sync. Any changes or customizations made to one instance need to be replicated across the other instance(s). This can be achieved by using the "filesexport" and "filesimport" commands, which are provided to migrate a set of enPortal/AppBoard customizations from one server to another. The filesexport command is configured, by default, to bundle up all of the stock configuration files. However, if you have any custom configuration files, you may need to modify the definitions of the file bundle that is generated by this command.
=== Initial Preparation ===
# Setup an external database for the shared configuration database, you will need the access details. Do not actually configure enPortal to use the external database at this stage.
# Set enPortal to operate in redundancy mode. This is enabled using the following setting in <tt>[INSTALL_HOME]/server/webapps/enportal/WEB-INF/config/custom.properties</tt>:
#: <tt>hosts.redundant=true</tt>
# Create a full backup archive of your existing system. Ensure this backup has been configured to include all custom filesystem assets. Refer to the [[enportal/5.5/admin/enPortal_installation/backup_and_recovery#Customizing_the_Export|Backup & Recovery]] page for more information. It's recommended this archive also include all required JDBC database drivers, including the JDBC driver required for the external configuration database.
# Have two or more systems ready to be configured for clustering. Note it's possible to also use the existing system without having to re-install.
# Have the enPortal turnkey distribution and license files for each node.


The configuration steps detailed below assume that an instance of enPortal/AppBoard is already up and running in the development environment. You are now ready to deploy enPortal/AppBoard in the production environment that uses a load balancer and two instances of enPortal/AppBoard (primary and secondary servers) which are connected to the load balancer.
{{Note|If re-establishing a cluster using the existing external database then '''all cluster nodes must be shutdown''' before proceeding.}}


=== Setup Process ===
The following applies to the '''primary node'''. Even in a purely load-balanced configuration pick one of the nodes as the primary for purposes of establishing the cluster:


[[Image:Failover_Configuration.png|thumb|center|700px|Summary of Steps for Initial Failover Configuration]]
# If converting the existing deployment into a cluster deployment, first shutdown enPortal.
# Otherwise, deploy the enPortal turnkey and ensure a valid environment. See the [[enportal/5.5/admin/enPortal_installation/installation|Installation]] documentation for more info.
# Configure enPortal to use an external configuration database. Follow the instructions to use <tt>dbsetup</tt> on the [[enportal/5.5/admin/configuration_database|Configuration Database]] page, but '''do not''' restore any archives or perform a dbreset.
# Load the complete backup archive created in the initial preparation:
#: <tt>portal '''Apply''' -jar ''archive.jar''</tt>
# Install the license file if not already - remember this is node specific.
# Start enPortal. At this point you should have a working cluster of one node!
# Verify in the enPortal <tt>catalina.out</tt> log file the following lines:
#: Establishing a connection to the external configuration database:
#: <tt>Connecting to the ''DB_TYPE'' database at ''jdbc:mysql://HOST:PORT/DB_NAME''...Connected.</tt>
#: Redundancy mode is enabled:
#: <tt>''date@timestamp'' Redundancy support enabled</tt>
# Also verify you can log into this instance of enPortal and all content is as expected.


The following applies to all '''other nodes''' in the cluster:


#Preparation work
# Deploy the enPortal turnkey and ensure a valid environment. See the [[enportal/5.5/admin/enPortal_installation/installation|Installation]] documentation for more info.
##In the development instance of enPortal/AppBoard, create an archive from the dev environment by running the [[Filesexport|filesexport]] command.
# Load the complete backup archive created in the initial preparation. Note this is '''different''' than for the primary node:
##Transfer the generated archive to the primary server's box.
#: <tt>portal '''FilesImport -skiplist apply''' -jar ''archive.jar''</tt>
#Importing the archive to the primary server.
# Ensure database configuration is '''not''' loaded on enPortal startup by renaming the following files by adding <tt>.disabled</tt> to their filenames:
##[[Database_Configuration|Configure the primary server's database connectivity]] using the "portal dbsetup" command in ${TomcatHome}/bin directory.
#* <tt>[INSTALL_HOME]/server/webapps/enportal/WEB-INF/xmlroot/server/load_*.txt</tt>
##In the primary server, import the archive by running the "portal filesimport" command in ${TomcatHome}/bin directory.
# Configure enPortal to use an external configuration database. Follow the instructions to use <tt>dbsetup</tt> on the [[enportal/5.5/admin/configuration_database|Configuration Database]] page, but '''do not''' load any archives or perform a dbreset.
#Synchronizing content between the primary and the secondary enPortal/AppBoard instances.
# Install the license file if not already - remember this is node specific.
##In the primary server, create an archive by running the [[Filesexport|filesexport]] command.
# Start enPortal. At this point you should now have a cluster of two or more nodes!
##Transfer the generated archive to the secondary server's box.
# Verify this instance the same as the primary instance.
##Validate the secondary server's database connectivity using the "portal dbsetup" command in ${TomcatHome}/bin directory.
##In the secondary server, import the archive by running the "portal filesimport" command in ${TomcatHome}/bin directory.
#Modify the database configuration setting to support redundancy
##Edit the file <tt>[<i>AppBoardHome</i>]\server\webapps\enportal\WEB-INF\config\hosts.properties</tt>
##Locate the line: <tt>hosts.redundant=false</tt>
##When running in a clustered configuration, change the setting to <tt>hosts.redundant=true</tt>


Additional verification of the whole cluster can be performed by logging into the separate nodes from different locations and viewing the [[enportal/5.5/admin/system_administration/manage_sessions|Session Management]] system administration page and verifying that all sessions are visible regardless of the node logged into.


Notes:
== Maintaining a Cluster ==
*To verify that the above process is complete and worked correctly, you must do a full file comparison of the dev, primary, and secondary file systems to confirm that they are mirrored properly.  If any custom files from the dev or primary system are missing from the secondary system, then you must modify the configuration of the filesexport command to include the additional customization files that did not get archived.  For more information on this, see the [[Filesexport|filesexport]] page.
*The database connector replication will occur automatically by the process above, because the database connection settings are stored in the file: persist.properites.  Each time an enPortal/AppBoard instance is started, any load files in /server/webapps/enportal/WEB-INF/xmlroot/server/ which do not have the extension ".disabled" will be executed and the default XML content will be loaded into the database.  Once the default content has been initially loaded into the database, you can remove the load files from the other instances before starting them, to prevent the XML content from being auto-loaded again into the database when that instance is started. This step would only be necessary if a server node has a new installation which is being started for the first time.  The load files to remove are the following:
**load_reloadContent.txt
**load_restore.txt
**load_stock.txt
*Verify that the primary and secondary instances are running against a shared database by logging in to the admin UI, adding a temporary user on each instance, and confirming that the user is shown in the other instances.  Note: due to database caching, there may be a delay before changes made to the database from one instance will be directly observed in another instance.
*The default filesexport bundle includes the license.properties file.  You typically do not want to copy the license file from the primary server to the secondary server(s), since it will not work on the server which has a different IP address.  You can modify the name of the license.properties file to something like license-prod2.properties so that it would not be overwritten.  Or you can allow the file to be over-written and then manually reinstate the license.properties file from a backup after executing the filesimport command.
*The default filesexport bundle includes the server.xml file, which includes some of the basic Tomcat settings (such as port number).  If you have different settings for server.xml on different servers, you can allow the file to be over-written and then manually reinstate the server.xml file from a backup after executing the filesimport command.
*The default filesexport bundle includes the persist.properties file, which manages the database connection. Procedures for managing this file may vary.  Typically this file is the same across all production servers, but some care may be needed when migrating from a dev server to a prod server, which may use different databases and thus have different persist.properties settings.


== See Also ==
=== Product Updates ===
For full product updates a process similar to establishing a new cluster is required. A full archive should be performed. This should be tested in isolation with the new build to ensure there are no upgrade issues. If changes need to be made due to the upgrade then produce a new full archive on the new version of the product for use when establishing the cluster. As the entire cluster will need to be taken down, and re-established, this process will require some scheduled downtime to complete.


CITRIX NetScaler Server Appliance
In high-availability environments if scheduled downtime is not possible then another approach would be to take down one node, re-establish a new cluster with a different external configuration database, then failover all clients to the new node. Other nodes can then be re-established pointing to the new configuration database, and then update the load balancer configuration to bring all nods back online.
http://www.citrix.com/English/ps2/products/product.asp?contentID=21679&ntref=hp_nav_US


<!-- Commented out link because it duplicates appboard/2.4/admin/backup_and_recovery and enPortal has its own backup page -->
For patch updates these are usually overlaid onto an existing install and do not modify configuration. This can be done in-place and typically just require the enPortal server to be restarted.
<!-- [[Backup_and_Restore|Backup and Restore]] -->
 
[[Database_Failover|Database Failover]]
=== Content/Configuration Updates ===
 
Content and provisioning changes made directly on the cluster are immediately available to all cluster nodes and nothing further needs to be done.
 
For content and provisioning changes made in a separate development or staging environment then a full archive should be made and then deployed to production. This would essentially require a re-establishment of the cluster with the shortcut that enPortal is already installed, licensed, and the database configuration is in place.

Latest revision as of 18:10, 17 November 2014


Overview

enPortal is implemented using a highly scalable web application architecture. As a Java application running inside an Apache Tomcat server, enPortal is able to make use of multi-core and multi-processor systems with large amounts of RAM on 64-bit operating systems. In addition to scaling vertically on a single system, enPortal supports horizontal scaling to handle even greater loads and/or to provide for high availability environments through the use of a shared configuration database. enPortal can be used in the following configurations:

  1. Load Balanced: Two or more nodes are fully operational at all times. The load balancer directs traffic to nodes based on standard load balancing techniques such as round-robin, fewest sessions, smallest load, etc... If a server is detected as down it is removed from the active pool.
  2. Failover: A two-node configuration with both nodes running but all traffic is routed to the primary node unless it is detected to be down. At this point the load balancer re-directs traffic to the secondary node.
  3. Cold Standby: A two-node configuration where the secondary node is offline in normal operation. If the primary node is detected to be down the secondary node is brought online and traffic re-directed.

In cases where high-availability is required then regardless of the load a cluster configuration is recommended. In cases where load is a concern refer to the Performance Tuning & Sizing documentation for more information.

Architecture & Licensing

Two Node Cluster Architecture Diagram

Whether running a load-balanced, failover, or cold-standby configuration the following components are required:

  • enPortal installation per node, this requires a separate license for each node.
  • External (shared) configuration database. This database is not provided by Edge and is recommended to reside on a different host to the enPortal servers. In high availability environments the database itself should also highly available. See the System Requirements for supported external configuration databases.
  • Load Balancer. This component is not provided by Edge but is required in cluster configurations.

Cluster Configuration

The overall cluster configuration is made up of separate parts that follow the cluster architecture:

  1. Load Balancer configuration
  2. Shared enPortal configuration: via an external shared configuration database.
  3. per-node enPortal configuration and filesystem assets.

Also consider that establishing a new cluster and maintaining the cluster may have different approaches as outlined below.

Shared Configuration Database

In simple single-server enPortal configurations it is recommended to use the built-in, in-memory, H2 configuration database. However, in cluster configurations the configuration needs to be shared and kept in sync across two or more nodes so an external configuration database is required. Setting up an external database for redundancy operation has two main steps:

  1. Configure enPortal to use an external database (versus the build-in H2). This process is documented in isolation on the Configuration Database page.
  2. Configure enPortal to operate in redundancy mode.
Template-note.png
The above points out the main steps in isolation, please refer to the Establishing a Cluster for more details.

Per-Node Configuration & Assets

While the shared configuration database takes care of enPortal content and provisioning information, there is other configuration and filesystem assets that need to be maintained per node:

  • license file
  • configuration database connection details
  • all other filesystem assets such as login pages, images for look and feel, enPortal PIM files, and other miscellaneous pieces that have been built into the solution such as custom JSPs, CGIs, HTML/CSS/JS, etc...

The recommended approach, which also serves to ensure full-backups are made of the system, is to configure the Backup export list to include all filesystem assets. Then when establishing, or updating a cluster, the archive can be used to maintain the filesystem components. Also, other custom approaches may also be suitable such as filesystem synchronization tools.

The license and database configuration will need to be handled when first establishing the cluster but will not need to be changed after that point.

Load Balancer

The Load Balancer can distribute sessions to one or more enPortal nodes using any standard load balancing algorithm (e.g. Round-Robin, smallest load, fewest sessions, etc.). The only requirement is that the session affinity is maintained such that a single user is always routed to the same enPortal node during the full duration of the session.

The two session cookies used by enPortal are JSESSIONID and enPortal_sessionid. When configuring the Load Balancer for session affinity, it is recommended to use enPortal_sessionid to avoid any conflicts with other applications that may also have a JSESSIONID cookie.

The following URL can be used by the load balancer as a means of testing enPortal availability:

http://server:port/enportal/check.jsp

This script returns a HTTP status code 200 (success) if all components of enPortal are running properly, otherwise it returns a 500 (internal error) if there is an issue. And in the case the enPortal server isn't running, then of course there will be no response.

Virtualized Environments

Whether running on the bare metal or within virtualized environments the clustering configuration remains the same.

Some virtualization environments may offer their own layer of fault tolerance although this is usually targeted at reducing/eliminating the impact of hardware failure - e.g. VMware Fault Tolerance to transparently failover a guest from a failed physical host to a different physical host such that everything continues un-interrupted. This type of system is useful on it's own but may not be aware of application-level failures that can also occur.

Establishing a Cluster

The following process can be used to establish a cluster environment. If you're skipped straight here, please go back and read over the previous sections to understand the overall architecture and configuration components.

The following process assumes an existing environment, although a clean install environment can be used too.

Initial Preparation

  1. Setup an external database for the shared configuration database, you will need the access details. Do not actually configure enPortal to use the external database at this stage.
  2. Set enPortal to operate in redundancy mode. This is enabled using the following setting in [INSTALL_HOME]/server/webapps/enportal/WEB-INF/config/custom.properties:
    hosts.redundant=true
  3. Create a full backup archive of your existing system. Ensure this backup has been configured to include all custom filesystem assets. Refer to the Backup & Recovery page for more information. It's recommended this archive also include all required JDBC database drivers, including the JDBC driver required for the external configuration database.
  4. Have two or more systems ready to be configured for clustering. Note it's possible to also use the existing system without having to re-install.
  5. Have the enPortal turnkey distribution and license files for each node.
Template-note.png
If re-establishing a cluster using the existing external database then all cluster nodes must be shutdown before proceeding.

Setup Process

The following applies to the primary node. Even in a purely load-balanced configuration pick one of the nodes as the primary for purposes of establishing the cluster:

  1. If converting the existing deployment into a cluster deployment, first shutdown enPortal.
  2. Otherwise, deploy the enPortal turnkey and ensure a valid environment. See the Installation documentation for more info.
  3. Configure enPortal to use an external configuration database. Follow the instructions to use dbsetup on the Configuration Database page, but do not restore any archives or perform a dbreset.
  4. Load the complete backup archive created in the initial preparation:
    portal Apply -jar archive.jar
  5. Install the license file if not already - remember this is node specific.
  6. Start enPortal. At this point you should have a working cluster of one node!
  7. Verify in the enPortal catalina.out log file the following lines:
    Establishing a connection to the external configuration database:
    Connecting to the DB_TYPE database at jdbc:mysql://HOST:PORT/DB_NAME...Connected.
    Redundancy mode is enabled:
    date@timestamp Redundancy support enabled
  8. Also verify you can log into this instance of enPortal and all content is as expected.

The following applies to all other nodes in the cluster:

  1. Deploy the enPortal turnkey and ensure a valid environment. See the Installation documentation for more info.
  2. Load the complete backup archive created in the initial preparation. Note this is different than for the primary node:
    portal FilesImport -skiplist apply -jar archive.jar
  3. Ensure database configuration is not loaded on enPortal startup by renaming the following files by adding .disabled to their filenames:
    • [INSTALL_HOME]/server/webapps/enportal/WEB-INF/xmlroot/server/load_*.txt
  4. Configure enPortal to use an external configuration database. Follow the instructions to use dbsetup on the Configuration Database page, but do not load any archives or perform a dbreset.
  5. Install the license file if not already - remember this is node specific.
  6. Start enPortal. At this point you should now have a cluster of two or more nodes!
  7. Verify this instance the same as the primary instance.

Additional verification of the whole cluster can be performed by logging into the separate nodes from different locations and viewing the Session Management system administration page and verifying that all sessions are visible regardless of the node logged into.

Maintaining a Cluster

Product Updates

For full product updates a process similar to establishing a new cluster is required. A full archive should be performed. This should be tested in isolation with the new build to ensure there are no upgrade issues. If changes need to be made due to the upgrade then produce a new full archive on the new version of the product for use when establishing the cluster. As the entire cluster will need to be taken down, and re-established, this process will require some scheduled downtime to complete.

In high-availability environments if scheduled downtime is not possible then another approach would be to take down one node, re-establish a new cluster with a different external configuration database, then failover all clients to the new node. Other nodes can then be re-established pointing to the new configuration database, and then update the load balancer configuration to bring all nods back online.

For patch updates these are usually overlaid onto an existing install and do not modify configuration. This can be done in-place and typically just require the enPortal server to be restarted.

Content/Configuration Updates

Content and provisioning changes made directly on the cluster are immediately available to all cluster nodes and nothing further needs to be done.

For content and provisioning changes made in a separate development or staging environment then a full archive should be made and then deployed to production. This would essentially require a re-establishment of the cluster with the shortcut that enPortal is already installed, licensed, and the database configuration is in place.

Retrieved from "http://ab.edge-technologies.com/docs/index.php?title=Enportal/5.5/admin/enPortal_installation/installation/clustering_and_failover&oldid=7182"