Database Replication Guide — Chapter 3

Database Replication Security

The admin role is required to configure and couple local and foreign clusters and to configure Database Replication.

When coupling clusters, you can configure SSL to encrypt the XDQP data passed between them. For details on configuring SSL, see Coupling the Local and Foreign Clusters. If SSL is enabled for XDQP, you must set the protocol to HTTPS in the procedures described in Coupling the Local and Foreign Clusters and Configuring Database Replication.

Avoid Replicating the App-Services Database

The App-Services database is used to store information used in Query Console and for other internal applications. Therefore, it needs to be writable on all hosts, otherwise those applications might not work correctly. Therefore, MarkLogic recommends that you do not set up database replication on the App-Services database.

Identifying the Local Cluster to Foreign Clusters and Designating the Bootstrap Hosts

By default, the name of the cluster is that of the bootstrap host. You must edit the cluster name in the Local Cluster Configuration on the each local cluster to be coupled with foreign clusters.

1. On the local host, select Local Cluster under Clusters at the bottom of the left-hand menu:

   

2. Select the Configure tab to display the Edit Local Cluster Configuration page. Enter the cluster name:

   

3. As described in Understanding Database Replication, each cluster to participate in Database Replication must have one or more bootstrap hosts that stores the configuration information needed to establish an initial connection to foreign clusters. You must identify the bootstrap hosts in each cluster before attempting any of the configuration procedures described in this chapter.

   A production Database Replication scheme will typically have more than one bootstrap host to ensure availability. When establishing an initial connection with a local cluster, a foreign cluster will connect to the first available bootstrap host.

   In the Local Cluster Configuration page, select one or more hosts to serve as the bootstrap hosts for this cluster.

   It is best to choose the host that hosts your Security forest as your bootstrap host. If you have configured your Security forest for local disk failover, then also choose the host that hosts your Replica Security forest as a bootstrap host.

   

4. Click OK to save the Local Cluster Configuration.

Coupling the Local and Foreign Clusters

This section describes how to 'couple' a foreign cluster configuration to the bootstrap host on your local cluster. If you have designated more than one bootstrap host on your local cluster, pick any one of them.

The foreign cluster must be running the same version of MarkLogic as the local cluster.

The procedure described in this section must be repeated for every Replica cluster.

1. On a bootstrap host on the Master cluster, select Local Cluster under Clusters at the bottom of the left-hand menu:

   

2. In the Local Cluster Configuration page, select the Couple tab.

   

3. In the Foreign Cluster portion of the Local Cluster Configuration page, enter the Host Name for any host in the foreign cluster to be coupled. You can also specify the Admin Port (if necessary) and the communication protocol to be used between the clusters (HTTP or HTTPS). When you have SSL enabled on the Admin App Server on the bootstrap host in the foreign cluster, set the protocol to https.

   

4. In the Foreign Cluster Configuration page, if you are using SSL for inter-cluster communication, configure the SSL security settings and timeout values.

1. In the Verify Add Foreign Cluster page confirm all of the settings are correct and click OK.

   

2. The Summary window appears and displays the summary for the Foreign Cluster configuration. Bootstrapped indicates whether the foreign cluster configuration has been received by the local cluster. Last Bootstrap indicates the last time the foreign cluster configuration was received by the local cluster. Initially the status of Bootstrapped may appear as false and Last Bootstrap as never. Refresh your browser page to see the current status.

   

Configuring Database Replication

This section describes how to configure Database Replication on the cluster that hosts the Master database. Before starting this procedure, you must have identified all of the foreign clusters you plan to replicate to by following the procedure described in Identifying the Local Cluster to Foreign Clusters and Designating the Bootstrap Hosts.

This section describes how to configure Database Replication from the Master cluster. You can also configure Database Replication on a Replica cluster. However, if you are replicating to multiple Replica clusters, it is more convenient to do all of the configuration from the Master cluster.

When replicating a Master schema database, you must create a second empty schema database for the Replica schema database on the Replica cluster. For details see Master and Replica Database Index Settings.

Any documents with URIs in a Replica forest that are not in its respective Master forest will be automatically cleared when Database Replication is configured.

Repeat the following procedure for each replicated database and for each Replica cluster.

1. On the bootstrap host in the Master cluster, navigate to the database to be replicated in the left-hand menu and select Database Replication:

   

2. Select the Configure tab and then the Replica cluster in the Foreign Cluster pull-down menu:

   

3. The Configure Database Replication page will change. For Local Database as, select Master. Select the database to be replicated to in the Foreign Database pull-down menu.

   

   The other settings on the Configure Database Replication page are:

   Database Replication Setting	Description
   Lag Limit	The Replica cluster sends an acknowledgement to the Master cluster each time it receives a replicated journal frame. You can set a Lag Limit in your configuration that specifies that transactions on the Master will be stalled if the Master does not receive an acknowledgement from the Replica within the number of seconds specified by the Lag Limit. By default the Lag Limit is 15 seconds. For more information on Lag Limit, see Replication Lag.
   Connect Forests by Name	If the forest names are the same on the Master and Replica clusters, set Connect Forests by Name to true. If new forests of the same name are later added to the Master and Replica databases, they will be automatically configured for Database Replication. If you set Connect Forests by Name to false, you must manually connect the Master forests on the local cluster to the Replica forests on the foreign cluster, as described in Connecting Master and Replica Forests with Different Names.
   Foreign Admin UI Protocol	The communication protocol to be used to connect to the foreign Admin App Server. Select https if SSL is enabled on the Admin App Server on the bootstrap host in the foreign cluster. Otherwise, select http.
   Foreign Admin UI Port	If the Admin App Server for the bootstrap host on the foreign cluster uses a port other than 8001, specify the port number.
   Host in Foreign Cluster	The name of a host on the foreign cluster.

1. When you have finished, click OK. The final configuration page appears to confirm the Database Replication configuration. Click OK.

   

2. The Verify add of foreign Replica/Master page appears. Click OK.

   

3. The Summary page appears with the Database Replication settings.

   

Connecting Master and Replica Forests with Different Names

If you are replicating between databases that contain forests with different names, you can manually connect Master to Replica forests.

The Replica databases must have at least as many forests as the Master database. Otherwise, not all of the data on the Master database will be replicated.

For example, if you have a database named Master on the Master cluster and a database named Replica on the Replica cluster and each database has unique forest names, use the following procedure.

1. On the bootstrap host in the Master cluster, navigate to the Master database in the left-hand menu and select Database Replication:

   

2. In the Configure Database Replication Step 1 page, select the Foreign Cluster that contains the Replica Database. Click OK.

   

3. In the Configure Database Replication Step 2 page, select Replica for the Foreign Database.

   

4. Select false for Connect Forests by Name and configure the other settings as described in Configuring Database Replication.

   

5. In the Connect Forests by Name table, use the pull-down menus to connect the Replica forests to the local Master forests. Click OK.

   

6. In the Confirm to Add Foreign Replica page, confirm the configuration and click OK.

   

Deleting a Database Replication Configuration

You can delete a Database Replication configuration from a bootstrap host on either the Master or Replica cluster. The procedure described in this section describes how to delete a Database Replication configuration on the Master cluster.

1. On the bootstrap host in the Master cluster, navigate to the Master database in the left-hand menu and select Database Replication:

   

2. In the Foreign Replicas for Database section of the Summary page, click Delete.

   

3. In the Delete Database Replication page, select https as the UI Protocol if SSL is enabled on the Admin App Server on the bootstrap host in the Replica cluster. Click OK.

   

4. In the Confirm to Delete Database Replication page, click OK.

   

Decoupling the Local and Foreign Clusters

This section describes how to decouple two clusters.

Before decoupling clusters, you must remove any Database Replication configurations between the local cluster and the cluster to be decoupled.

For example, to decouple a cluster named Replica from the local cluster, do the following:

1. On the local host, select Clusters at the bottom of the left-hand menu:

   

2. Locate the foreign cluster to be decoupled and click Delete:

   

3. Click OK in the Decouple Clusters page:

   

4. Click OK in the Decouple Clusters- Validated page:

   

Configuring App Servers on the Replica Cluster

As described in Reducing Blocking with Multi-Version Concurrency Control in the Application Developer's Guide, setting multi version concurrency control to 'nonblocking' on an App Server minimizes transaction blocking, which is useful if the App Server makes use of a replica database that significantly lags its master database.

It is recommended that, at a minimum, you set multi version concurrency control to 'nonblocking' on the Admin App Server because the Security database is typically not frequently updated. The 'nonblocking' multi version concurrency control option minimizes transaction blocking at the cost of queries potentially seeing a less timely view of the database, so you should weigh these two factors when determining whether to set this options on other App Servers on your replica cluster.

Changing the Foreign Bind Port

As described in Inter-cluster Communication, communication between clusters is done using the intra-cluster XDQP protocol on the foreign bind port. By default, the foreign bind port is port 7998. This section describes how to change the foreign bind port.

The procedure in this section must be repeated for each host in the cluster that is involved in inter-cluster replication.

1. Select a host in the local cluster under Hosts in the left-hand menu:

   

2. In the Host Configuration page, change the port number in the foreign bind port field:

   

TCP Tuning For High-Latency Environments

On Linux systems, if you have configured database replication where there is high-latency between the master and the replica environments (for example, if your master is in San Francisco and your replica is in Tokyo), you might need to tune the Linux TCP settings to increase throughput. The following are examples of the tuned Linux TCP setting (these settings are tuned with values greater than the typical defaults):

# sysctl -w net.core.rmem_max=8388608
net.core.rmem_max = 8388608
# sysctl -w net.core.wmem_max=8388608
net.core.wmem_max = 8388608
# sysctl -w net.ipv4.tcp_mem='8388608 8388608 8388608'
net.ipv4.tcp_mem = 8388608 8388608 8388608
# sysctl -w net.ipv4.tcp_rmem='4096 87380 8388608'
net.ipv4.tcp_rmem = 4096 87380 8388608
# sysctl -w net.ipv4.tcp_wmem='4096 87380 8388608'
net.ipv4.tcp_wmem = 4096 87380 8388608
# sysctl -w net.ipv4.route.flush=1
net.ipv4.route.flush = 1 

To see your current TCP settings, run the following Unix command:

# sysctl -a | grep mem

The setting shown above will change the running system but will not survive a reboot. Once you have tuned your system to your satisfaction, you need to add these settings to your startup environment to persist them through system reboots.

For details on your TCP settings, see your operating system documentation. If you have questions about how these operating system parameters might behave with your MarkLogic environment, contact MarkLogic Technical Support.