Loading TOC...
Scalability, Availability, and Failover Guide (PDF)

MarkLogic 10 Product Documentation
Scalability, Availability, and Failover Guide — Chapter 7

« Previous chapter
Next chapter »

Configuring Local-Disk Failover for a Forest

This chapter describes the procedure for configuring local-disk failover for a forest. For details about how failover works and the requirements for failover, see High Availability of Data Nodes With Failover. For details on configuring shared-disk failover, see Configuring Shared-Disk Failover for a Forest. This chapter includes the following sections:

For other failover administrative procedures that apply to both local-disk and shared-disk failover, see Other Failover Configuration Tasks.

Setting Up Local-Disk Failover for a Forest

Setting up failover for a forest is a relatively simple administrative process. This section describes this procedure. There are two basic parts to the procedure:

Enabling Failover in a Group

For each group in which you want to host a failover forest, perform the following steps:

  1. Before setting up failover, ensure that you have met all the requirements for failover, as described in Requirements for Local-Disk Failover.
  2. In the groups configuration page for the group in which the failover host belongs, make sure the failover enable button is set to true.

    This group-level failover enable button provides global control, at the group level, for enabling and disabling failover for all forests in that group.

You can enable or disable failover at the group level at any time. It does not stop you from configuring failover for forests; it only stops failover from actually occurring.

Configuring Local-Disk Failover For a Forest

To set up local-disk failover on a forest and set up one or more replica forests, perform the following steps:

  1. Before setting up failover, ensure that you have met all the requirements for failover, as described in Requirements for Local-Disk Failover and enable failover for the group, as described in Enabling Failover in a Group.
  2. Create one or more forests on one or more hosts to use as a replica (or use existing forests). If you add a replica forest to an existing master forest, it will take the master forest offline for a period ranging between a few seconds to a few minutes.
  3. Either create a new forest or use an existing forest on a different host from the replica forest(s). This forest will be the primary forest. If you are modifying an existing forest, skip to step 6. To create a new forest, first click the Forests link in the left tree menu, then click the Create tab. The Create Forest page appears. Note the failover enable button and the failover hosts section at the bottom.

  4. Enter a name for the forest.
  5. Specify a data directory.
  6. Select true for failover enable. Note that failover enable must be set to true at both the forest and the group level for failover to be active.
  7. Select the replica forest from the drop down menu for forest replicas. You can set one or more replica forests.
  8. Click OK to create or modify the forest.

The forest is now configured with the specified replica forests. You must attach the primary forest to a database before you can use the forest, but it is ready and set up for failover. You cannot attach the replica forests to a database; they will automatically be kept up-to-date as you update content in the primary forest.

Reverting a Failed Over Forest Back to the Primary Host

If a forest fails over to a failover host, causing a replica forest to take the role of the primary forest, the replica forest will remain in the open state until the host unmounts the forest. If you have a failed over forest and want to revert it back to the original primary host (unfailover the forest), you must either restart the forest that is open or restart the host in which the forest open. You should only do this if the original primary forest has a state of sync replicating, which indicates that it is up-to-date and ready to take over. After restarting the forest that is currently open, the forest will automatically open on the primary host (if the original primary forest is in the state sync replicating). Make sure the primary host is back online and corrected before attempting to unfailover the forest. To check the status of the hosts in the cluster, see the Cluster Status Page in the Admin Interface. To check the status of the forest, see the Forest Status Pages in the Admin Interface.

To restart the forest, perform the following steps:

  1. Navigate to the Status page for the forest that has failed over. For example, if the forest name is myFailoverForest, click Forests > myFailoverForest in the left tree menu, then click the Status tab.
  2. On the Forest Status page, click the restart button.

  3. Click OK on the Restart Forest confirmation page.
  4. When the Forest Status page returns, if the Mount State is unmounted, the forest might not have completed mounting. Refresh the page and the Mount State should indicate that the forest is open.

The forest is restarted, and if the primary host is available, the primary host will mount the forest. If the primary host is not available, the first failover host will try to mount the forest, and so on until there are no more failover hosts to try. If you look in the ErrorLog.txt log file for the primary host, you will see messages similar to the following:

2010-09-13 20:16:47.751 Info: Mounted forest myFailoverForest locally on /space/marklogic/Forests/myFailoverForest
2010-09-13 20:16:47.751 Info: Forest replica accepts forest myFailoverForest as the master with timestamp 2564905551526239330

If you look at the ErrorLog.txt log file for any other host in the cluster, you will see messages similar to the following:

2010-09-13 20:16:47.751 Info: Forest failover1 accepts forest myFailover as the master with timestamp 2564905551526239330
2010-09-14 17:01:29.651 Info: Forest replica1 starting synchronization to forest failover1
2010-09-14 17:01:29.666 Info: Forest replica1 starting bulk replication to forest failover1
2010-09-14 17:01:29.776 Info: Forest replica1 needs to replicate 0 fragments to forest failover1
2010-09-14 17:01:29.776 Info: Forest replica1 finished bulk replicated 0 fragments to forest failover1
2010-09-14 17:01:29.807 Info: Forest replica1 finished bulk replication to forest failover1
2010-09-14 17:01:29.822 Info: Forest replica1 finished synchronizing to replica forest failover1
2010-09-14 17:09:26.638 Info: Forest replica1 accepts forest failover1 as the master with precise time 12845094147890000
« Previous chapter
Next chapter »