Loading TOC...
Ops Director Guide (PDF)

Ops Director Guide — Chapter 2

Installing and Configuring Ops Director

Ops Director provides you with system monitoring, management, analysis, and support capabilities, as well as license auditing and role-based access control (RBAC) management. The Ops Director web-based interface offers graphical and tabular data representation intended to highlight irregular performance or alerts on a given cluster, host, database, or application server. This chapter contains instructions for installing and configuring Ops Director.

This chapter covers the following topics:

System Requirements

The following table shows installation compatibility requirements for MarkLogic Server and Ops Director:

Installed Version of MarkLogic Server Can Install Ops Director Version(s)
9.0-6 and earlier Can install Ops Director 1.0 and 1.1
9.0-7 or 9.0-8 Can install Ops Director 2.0 and 2.0.1
9.0-9 + Can install Ops Director 2.0.1 only.

Ops Director 1.1 requires that you have MarkLogic Server Version 9.0-3 or later running on the Application Cluster and MarkLogic Server Version 9.0-2 or later on the Managed Clusters. For definitions of the Application Cluster and Managed Cluster terms, see Terms.

Ops Director 2.0 requires the following software versions:

  • MarkLogic Server 9.0-7 or greater
  • Internet Explorer 11 on Windows 10
  • Firefox 62 on Windows 10 and Mac OS
  • Chrome 63 on Windows 10 and Mac OS

When a cluster is configured to be managed by Ops Director, a SecureManage App Server is created with SSL enabled.

In general, the hardware requirements for Ops Director are the same as those described in Memory, Disk Space, and Swap Space Requirements in the Installation Guide. However, it is recommended that you have at least 32 GB of memory on a host where Ops Director application runs.

Installing Ops Director

The installation procedure has changed from Ops Director 1.1 to Ops Director 2.0. Please follow the instructions for the version of Ops Director you are installing.

The following installations are available:

Installing Ops Director version 1.1 on an Application Cluster

Ops Director is an application built on top of MarkLogic Server and runs on a designated MarkLogic cluster called Ops Director Application Cluster. Managed Clusters are connected to the Ops Director Application Cluster. An Ops Director Application Cluster can serve as both an Application Cluster and a Managed Cluster.

The following procedure describes how to install Ops Director 1.1 on an application cluster.

  1. Log into the Admin Interface.
  2. Navigate to Configure > Clusters in the left tree menu.
  3. Select the local cluster. The Edit Local Cluster Configuration page displays.

  4. Select the Ops Director tab at the top of the page.

  5. On the Ops Director Setup page, click install.

  6. On the Install Ops Director page, select Generate new MarkLogic Ops Director certificate from the Ops Director Certificate Authority menu. Click ok.

    This procedure secures inter-cluster communication by means of an internally-generated, self-signed certificate. If you plan to use externally signed certificates, see Securing Ops Director with Externally Signed Certificates.

  7. A Certificate Authority configuration page displays. Fill in the blank fields with all of your relevant information. Click ok.

    In most cases, you may leave the default values for all other settings. However, in some cases you have to specify values different from the default ones. In particular, for installation on AWS, it is important to change the forest data directory.

  8. The Ops Director Setup page displays. If you want this cluster to serve as both the Ops Director Application Cluster and as a Managed Cluster, select manage this cluster. Otherwise click ok.

  9. If you opted to have the cluster serve as both the Ops Director Application Cluster and as a Managed Cluster, the Configure as a Managed Cluster page displays. Enter the name of the local host in this cluster that contains the OpsDirectorApplication and OpsDirectorSystem application servers. Set the port number for the OpsDirectorSystem server (8009, be default) and select MarkLogic Ops Director Certificate Authority from the Ops Director Certificate Authority pulldown menu.

    Set the level for log messages sent to Ops Director, as well as the frequency at which the metering data is collected. For details on the log levels, see Understanding the Log Levels in the Administrator's Guide.

    To set the log interval filter in Ops Director to Raw, as described in Date and Time Filters, you must also set the Ops Director Metering to raw here in the Admin Interface.

    When finished, Click ok.

  10. The Ops Director Setup page displays, with the settings configured during the installation of Ops Director.

    Click ok.

  11. The Summary page for the local cluster displays.

    Select the Ops Director tab again. Notice that the About Ops Director description specifies Ops Director is currently installed on this cluster.

    Also, if you opted to have the cluster serve as both the Ops Director Application Cluster and as a Managed Cluster, the description specifies Managing 1 cluster.

  12. Click Configure on the left tree menu. In the right page, select the Summary tab. The Configuration Summary page displays all of the resources created by installing Ops Director.

Installing Ops Director 2.0 and Greater

Follow these instructions to install Ops Director and greater.

Before You Install Ops Director

Make sure the following conditions are in place before you begin the installation process:

  • Internet access is required to install Ops Director 2.0. Make sure you can connect to the Internet.
  • Make sure you are running MarkLogic Server 9.0-7 or greater.
  • Make sure you have access to ports 8000, 8001, 8002, 8008, and 8009 on the managed cluster.
  • Start MarkLogic Server and log into the Admin UI (port 8001).
  • If you have not already uninstalled a previous version of Ops Director, follow the instructions in Removing Ops Director.

    The Ops Director installer is written in ML Gradle. A version of Gradle is included in the Ops Director 2.0 installation directory. For more information about Gradle, visit https://docs.gradle.org/current/userguide/userguide.html.

Ops Director 2.0 Installation

To install or upgrade to Ops Director 2.0 or greater, follow these instructions:

  1. Download the Ops Director zip file from marklogic.com at:

    https://developer.marklogic.com/products/opsdirector

  2. Unzip the file, and cd to the directory containing the files.
  3. Either edit the Ops Director settings in the gradle.properties file or enter them as options on the command line. The following table contains the host settings.
    Setting Description
    mlHost Host where Ops Director will be installed. Default value: localhost.
    mlUsername Username for admin access to that host. Default value: admin.
    mlPassword Password for admin access to that host. Default value: admin. MarkLogic strongly that recommends you change this value.
    opsdirDataExpires An xs:dayTimeDuration specifying how long to keep historical data in the Ops Director database. Default value: P60D

    The following settings determine how the Ops Director cluster communicates with the managed clusters. Choose one of the following Certificate Authority options:

    • opsdirCa=generate

      If you set this value, you also need to add the following values:

      Setting Definition
      opsdirCaCountry The two-letter code for the country in which the server is located (for example, US)
      opsdirCaProvince The two-letter code for the state or province in which the server is located (for example, CA)
      opsdirCaLocality The city in which your server is located (for example, San Carlos)
      opsdirCaOrgName The organization or company to which the server belongs (for example, MarkLogic)
      opsdirCaOrgUnit The organizational unit to which the server belongs (for example, Engineering)
      opsdirCaEmail The email address to contact regarding this server (for example, webmaster@yourcompany.com)
    • opsdirCa=external

      Use this setting to use an external certificate for Ops Director.

    • opsdirCa=ML_ID

      Use this setting to use a certificate that is already installed where ML_ID is the MarkLogic ID number for the certificate.

  4. Log into the MarkLogic Server Admin UI.
  5. From the same Ops Director 2.0 installation directory, run the installer:

    ./gradlew mlDeploy

    If this is the first time you are installing Ops Director 2.0, a lot of periods will display, followed by files being downloaded.

Ops Director Resources

The following resources are created on the host where your Ops Director application runs.

Resource Type Resource Name and Function
Certificate Authority The Certificate Authority (MarkLogic Ops Director in this example) is used to generate the secure credentials required for the Ops Director application and Managed Clusters to securely communicate.
Certificate Templates

OpsDirector-SSL-Template for SSL on the Application Cluster.

OpsDirector Template for SSL, if also configured as a Managed Cluster.

Roles

opsdir-guest role is used for a default user of the Ops Director application.

opsdir-user role is required for any user to be able to access the Ops Director application.

opsdir-license-admin role grants a user rights to manage licenses for the Ops Director application.

opsdir-admin role grants a user administrative rights to the Ops Director application.

Execute Privileges

opsdir-admin protects administrative functions.

opsdir-license-admin is required to access license information.

opsdir-user is required to access the browser application.

Database and Forests The OpsDirector database and forests hold the Ops Director configuration data.
App Servers

OpsDirectorApplication (port 8008) provides the Ops Director browser application and consumes the data stored in the OpsDirector database.

OpsDirectorSystem (port 8009) receives data transmitted from Managed Clusters and stores it in the OpsDirector database.

External Security Configurations

OpsDirector-External-Security for authentication of clients.

OpsDirectorSystem for secure communication with OpsDirectorSystem server.

Secure Credentials

OpsDirector-Credential to sign OpsDirector-SSL-Template.

MarkLogic-OpsDirector for accessing OpsDirectorSystem server.

Connecting a Managed Cluster to Ops Director

The following procedure describes how to configure a cluster to be managed by Ops Director.

Do not connect new managed clusters to Ops Director if you started the process of Ops Director upgrade and have not completed it. Adding managed clusters in the middle of Ops Director upgrade can lead to error conditions. For more details, see Upgrade Error Prevention and Troubleshooting.

Before you log into the cluster to be managed, you must copy the value of the Ops Director Certificate Authority field in the Admin Interface from the host where Ops Director is installed to a host of your Managed Cluster. You can either use the REST Management API to script the operation, or you can follow these steps:

  1. On the host where Ops Director is installed, make sure that MarkLogic Server is running, open the Admin Interface, and click Configure > Clusters in the left tree menu.
  2. Select the local cluster. The Edit Local Cluster Configuration page displays. Select the Ops Director tab to display the Ops Director Setup page.
  3. Scroll down to the Ops Director Certificate Authority section and copy the certificate by selecting everything between and including -----BEGIN CERTIFICATE----- and -----END CERTIFICATE-----.

Log on to the Admin Interface of a host on the cluster to be managed by Ops Director and do the following:

  1. Select Configure > Security > Certificate Authorities on the left menu.
  2. On the right page, select the Import tab at the top of the Summary page.
  3. Paste the certificate copied from the host where Ops Director is installed.
  4. Click ok.
  1. Select Configure > Clusters on the left tree menu.
  2. Select the local cluster. The Edit Local Cluster Configuration page displays.
  3. Select the Ops Director tab at the top of the page.
  4. The Ops Director Setup page displays. Select manage this cluster.
  1. The Configure as a Managed Cluster page displays. Enter the name of the host where your Ops Director application runs. This is the host you set up in Installing Ops Director.
  2. Select MarkLogic Ops Director Certificate Authority from the Ops Director Certificate Authority menu.
  3. Set the level for log messages sent to Ops Director, as well as the frequency at which the metering data is collected. For details on the log levels, see Understanding the Log Levels in the Administrator's Guide.

    When finished, click ok.

  4. The Ops Director Setup page displays. Click ok.

    When you designate a Managed Cluster, a SecureManage App Server is created in each group on the managed cluster to provide secure connections with the Ops Director Application Cluster. For more detail, see Security and Database Dependencies of Managed Clusters.

Launching Ops Director and Logging In

In a browser window, enter the hostname of the host where Ops Director runs and the port number of the OpsDirectorApplication server (by default, 8008). For example:

https://englab.marklogic.com:8008

You are asked for credentials with a standard authentication dialog box. To log into Ops Director, enter a valid user name and password an existing MarkLogic user with the opsdir-admin role and click Log in.

User credentials can come from LDAP or internal security. If internal security is used, the user must exist on the Ops Director Application Cluster.

By default, Ops Director is configured with digest-based authentication. To enable application-level authentication, perform the steps described in Switching from Digest to Application-Level Authentication.

When you use digest-based authentication, make sure to refresh the Ops Director application in the browser after you log into the application with different credentials, which may have different access to resources than the previous user that was logged-in. If you do not refresh the Ops Director application in the browser, the application may show stale information.

Switching from Digest to Application-Level Authentication

To enable application-level authentication, perform the following steps:

  1. Install SSL certificate on the Ops Director application server, as described in General Procedure for Setting up SSL for an App Server in the Security Guide.
  2. Log into the Admin Interface on the host where the Ops Director application is running.
  3. In the left menu, select Configure > Groups > Default > App Servers > OpsDirectorApplication. On the right page, select the Configure tab.
  4. Scroll down to the authentication menu and change the selection from digest (default) to application-level.

  5. Click ok and wait for the operation to complete.
  6. In a browser window, enter the hostname of the host where Ops Director runs and the port number of the OpsDirectorApplication server (by default, 8008). For example:
    https://englab.marklogic.com:8008
  7. The Ops Director login screen displays. Enter a valid user name and password and click LOGIN.

    Switching from digest authentication to application-level authentication without first installing an SSL certificate to enable HTTPS on port 8008 presents a security problem because credentials are sent over HTTP as clear text.

Configuring the Ops Director Data Retention Policy

The following data accumulates in the OpsDirector database:

  • Meters data
  • Log files
  • Configuration data of managed clusters
  • System alerts

Ops Director has a built-in auto-cleanup mechanism to delete accumulated data periodically.

Once a day, at 11:00 PM local time, a task runs to delete accumulated Ops Director data based on the defined data retention policy. By default, all types of data--meters, logs, cluster configuration data, and system alerts--is retained for 60 days.

The default data retention period for each type of data is defined in the configuration file /config/opsdirector.xml stored in the OpsDirector database. Follow these instructions to retrieve the contents of this file.

  1. Open Query Console on the host where Ops Director runs:
    http://<hostname>:8000/qconsole/
  2. On the Query Console, set Query Type to XQuery and Database to OpsDirector.
  3. Execute the following code:
    xquery version "1.0-ml";
    doc("/config/opsdirector.xml")
In the response, look for the following elements:
  • max-meters-age -- specifies the maximum age for meters data;
  • max-logs-age -- specifies the maximum age for log files;
  • max-resource-age -- specifies the maximum age for cluster configuration data and system alerts.

By default, the maximum age for each class of data is set to 60 days, as in the following example:

<config xmlns="http://marklogic.com/v1/opsdirector/config">
  <max-meters-age>P60D</max-meters-age>
  <max-logs-age>P60D</max-logs-age>
  <max-resource-age>P60D</max-resource-age>
</config>

You can modify the default data retention period as follows: edit the configuration file /config/opsdirector.xml and replace default P60D values with other values of type xs:dayTimeDuration to define your preferred data retention periods for corresponding data classes.

For details on xs:dayTimeDurationdata type, see.

Ops Director includes a utility library to safely change values in the configuration file. For example, to retain meters data and log files for 30 days only, run the following query on the OpsDirector database:

xquery version "1.0-ml";

declare namespace
   cfg="http://marklogic.com/v1/opsdirector/config";
declare default function namespace
   "http://www.w3.org/2005/xpath-functions";
declare option xdmp:mapping "false";

let $config := doc("/config/opsdirector.xml")/cfg:config
let $_ := if (empty($config))
          then error((), "No config document?")
          else ()

let $expire := xs:dayTimeDuration("P30D")

for $property in ("max-meters-age", "max-logs-age")
let $qname := 
   fn:QName("http://marklogic.com/v1/opsdirector/config", $property)
let $node := $config/*[node-name(.) = $qname]

return
   if (empty($node))
   then xdmp:node-insert-child($config, element { $qname } { $expire })
   else xdmp:node-replace($node/text(), text { $expire })

If you want a specific class of data to be stored forever, set the data retention period for that class to infinity in the configuration file /config/opsdirector.xml. Note that this setting will cause the OpsDirector database to grow as data accumulates. Make sure that you have sufficient capacity to handle this growth.

Disconnecting a Managed Cluster from Ops Director

A Managed Cluster can be disconnected from Ops Director. Once disconnected, the Managed Cluster stops sending configuration, metrics, and log data to the Ops Director cluster.

Ops Director will continue to make Management API calls to assess the cluster health as long as there is a valid session for that cluster. After the session expires, Ops Director will stop making calls and the disconnect will be complete.

The procedure for disconnecting a Managed Cluster from Ops Director is as follows:

  1. Log into the Admin Interface on the Managed Cluster.
  2. Select Configure > Clusters on the left tree menu.
  3. Select the local cluster. The Edit Local Cluster Configuration page displays.
  4. Select the Ops Director tab at the top of the page.
  5. In the Ops Director Setup page, click stop managing this cluster.

  6. On the Stop Managing this Cluster page, select the Remove Ops Director Certificate Authority box and click ok.

    If you plan to have this cluster managed again by the same instance of Ops Director at some point in the future, you may leave the Remove Ops Director Certificate Authority box unchecked.

Removing Ops Director

This section describes how to remove the Ops Director application.

Before you remove Ops Director, disconnect all Managed Clusters, as described in Disconnecting a Managed Cluster from Ops Director.

Removing Ops Director 1.1 and Earlier

The following procedure describes how to remove Ops Director 1.1 and earlier from the MarkLogic host.

  1. Log into the Admin Interface.
  2. Select Configure > Clusters on the left tree menu.
  3. Select the local cluster. The Edit Local Cluster Configuration page displays.
  4. Select the Ops Director tab at the top of the page.
  5. In the Install Ops Director section, click uninstall.

  6. The Uninstall Ops Director page displays and asks for confirmation: This change will cause a cluster restart. Do you wish to continue? Click ok.

    When you use this method of uninstalling Ops Director, the Ops Director database is not uninstalled.

Removing Ops Director 2.0 and Later

The following procedure describes how to remove Ops Director 2.0 and later from the MarkLogic host.

  1. Change to the Ops Director installation directory (for example, opsdirector-2-0-0).
  2. If you used the default value for mlGroupName in the gradle.properties file, enter the command:

    gradlew mlUndeploy -Pconfirm=true -PconfirmManaged=true

  3. If you used a different value for mlGroupName in the gradle.properties file, enter the command (on one line):

    gradlew mlUndeploy -PmlGroupName=mlGroupName -Pconfirm=true -PconfirmManaged=true

    where mlGroupName is the value from the gradle.properties file.

Running Ops Director on Amazon Web Services (AWS)

When AWS hosts are stopped and restarted, they are assigned new host names and new IP addresses. They do not get new MarkLogic host IDs, as those are persistent. Ops Director will recover from AWS host renaming under the following circumstances:

  • If a Managed Cluster host is renamed, when Ops Director receives information from it, the fact that an existing host id has appeared with a new host name will be used to update the Ops Director configuration. At that point, things return to normal.
  • If an Ops Director host is renamed, the Managed Cluster will be unable to communicate with it. But the next time Ops Director polls that Managed Cluster, it will detect that out-of-date session endpoint and update it. At that point, things return to normal.

However, if you stop both the Ops Director host and the Managed Cluster hosts at the same time, then all hosts will come back with new host names. There is no way to fix the problem automatically, but you can fix the problem with a script. In this example, Ops Director used to be AMAZONOLD and now is AMAZONNEW. On each Managed Cluster that is still trying to talk to AMAZONOLD, use the following script on the Query Console:

xquery version "1.0-ml";
import module namespace admin = "http://marklogic.com/xdmp/admin"
      at "/MarkLogic/admin.xqy";

let $config := admin:get-configuration() 
let $config := admin:cluster-set-opsdirector-session-endpoint($config,
               "https://AMAZONNEW:8009/v1/opsdirector/session")

return
    admin:save-configuration($config)

Securing Ops Director with Externally Signed Certificates

The procedures described in Installing Ops Director make use of an internally-generated, self-signed certificate.

To use externally signed certificates with any version of Ops Director, you must get the Certificate Signing Requests (CSRs) from MarkLogic, pass the CSRs to the Certificate Authority and receive signed certificates, and then install the signed certificates on the cluster. You must ensure that you have secured a certificate authority (CA) and can get signed certificates issued by that CA for each of the nodes involved.

In this example, we will be working with a two-node Ops Director cluster with hosts named node1 and node2, and a two-node managed cluster with hosts named node3 and node4. In each step, a code example is presented showing how the REST Management API (RMA) can be used to accomplish the task. Where the RMA cannot be used, XQuery scripts are shown.

Your MarkLogic server hosts must have fully-qualified names that can be resolved by DNS.

If the signing authority is Verisign, Thawte, or another well established authority, MarkLogic will already have the appropriate Certificate Authority installed. If you are using a less well-known authority, you must install the appropriate Certificate Authority on each MarkLogic cluster.

Before you begin this procedure, make sure you have a way of obtaining certificates.

The following sections contain instructions for configuring Ops Director to use externally signed certificates.

Using Externally Signed Certificates with MarkLogic Server 9.0-9 and Later and Ops Director 2.0.1 and Later

The steps involved for configuring Ops Director to use externally signed certificates are:

Configure the Ops Director Cluster

Once you have obtained the CA file (in this example, it is /tmp/opsdir.ca), follow these steps to configure the Ops Director cluster:

  1. If necessary, use curl to install the Certificate Authority on the cluster, replacing /tmp/opsdir.ca with the location and name of your CA file:
    curl --anyauth -u admin:admin -X POST -H "content-type:text/plain" \
       --data-binary @/tmp/opsdir.ca \
       http://node1:8002/manage/v2/certificate-authorities
  2. Install Ops Director:
    ./gradlew -PopsdirCa=external mlDeploy

    You may need to specify additional options, such as mlHost, depending on your configuration.

  3. Create a certificate template for the hosts on the cluster:
    {
      "template-name": "OpsDirector-SSL-Template",
      "template-description": "Ops Director SSL Template",
        "key-type": "rsa",
        "key-options": {
        "key-length": "2048"
      },
      "req": {
      "version": "0",
      "subject": {
        "countryName": "US",
        "stateOrProvinceName": "California",
        "localityName": "San Carlos",
        "organizationName": "MarkLogic Corporation"
      }
     }
    }

    Our examples use MarkLogic-specific values. You may use other values, but you must use them consistently.

  4. Use curl to post this certificate template to the REST Management API, replacing /tmp/od-template.json with the location and name of your template filename:
    curl --anyauth -u admin:admin -X POST -H \
       "content-type:application/json" \
       -d @/tmp/od-template.json \
       http://node1:8002/manage/v2/certificate-templates
  5. Obtain the ID of the Certificate Authority from Step <HyperlinkSerif>1, replacing OpsDirector CA with the common name of your certificate of the Certifcate Authority. The following XQuery code accomplishes this:
    xquery version "1.0-ml";
    import module namespace pki = "http://marklogic.com/xdmp/pki"
      at "/MarkLogic/pki.xqy";
    xdmp:invoke-function(function () {
     /pki:certificate[pki:authority=true()][pki:host-name='OpsDirector CA']/pki:certificate-id/data()
     }, map:entry("database", xdmp:security-database()))

    If you prefer, the following curl command will do it with the REST Management API. Note that this command relies on the https://stedolan.github.io/jq/ tool, which must be on your system.

    curl -s --anyauth -u admin:admin \
       "http://localhost:8002/manage/v2/certificate-authorities?format=json" \
       | jq ".\"certificate-authorities-default-list\".\"list-items\".\"list-item\"[]\ 
       | select(.nameref==\"OpsDirector CA\") | .idref" \
       | tr -d '"'

    Either of these techniques will give you the ID of the certificate authority.

  6. Create a JSON file (in this example, we call it /tmp/od-extsec.json) that creates an external authentication configuration object, replacing CA_ID with the CA ID you obtained in the previous step:
    {
      "external-security-name": "OpsDirectorSystem",
      "description": "Ops Director System",
      "authentication": "certificate",
      "cache-timeout": 300,
      "authorization": "internal",
      "ssl-client-certificate-authority": [ "CA_ID" ]
    }
  7. Use curl to POST this file on the REST Management API (port 8002):
    curl --anyauth -u admin:admin -X POST -H \
         "content-type:application/json" \
         -d @/tmp/od-extsec.json \
         http://node1:8002/manage/v2/external-security
  8. Create an XML file (in this example, we call it /tmp/od-server.xml) to update the server configuration to modify some of the properties of the OpsDirectorSystem application server. In this example, replace Insert_the_certificate_here with your certificate:
    <http-server-properties xmlns="http://marklogic.com/manage">
      <enabled>true</enabled>
      <authentication>application-level</authentication>
      <external-securities>
      <external-security>OpsDirectorSystem</external-security>
      </external-securities>
      <ssl-certificate-template>OpsDirector-SSL-Template</ssl-certificate-template>
      <ssl-client-certificate-pems>
      <ssl-client-certificate-pem>Insert_the_certificate_here</ssl-client-certificate-pem>
      </ssl-client-certificate-pems>
    </http-server-properties>
  9. Use curl to PUT this file on the REST Management API (port 8002):
    curl --anyauth -u admin:admin -X PUT -H \
      "content-type:application/xml" \
      --data-binary @/tmp/od-server.xml \
      http://node1:8002/manage/v2/servers/OpsDirectorSystem/properties?group-id=Default

    If you are not using the Default group for the Ops Director servers, update group-id accordingly.

  10. Use the following shell script to generate certificate requests for each of the hosts, replacing host_list with the names of the hosts in your cluster separated by spaces:
    for host in host_list; do
      echo "Generating CSR for $host to /tmp/$host.csr ..."
      curl -s --anyauth -X POST -u admin:admin -H \
        "content-type:application/json" -d "{\"operation\": \
        \"generate-certificate-request\", \"common-name\": \
        \"$host\", \"dns-name\": \"$host\" }" \
        //node1:8002/manage/v2/certificate-templates/OpsDirector-SSL-Template \
        > /tmp/$host.csr
    done
  11. Get a signed server certificate for each CSR from the CA in step <HyperlinkSerif>1.
  12. To install the certificates, run the following code, replacing host_list with the names of the hosts in your cluster separated by spaces, and store the certificate in /tmp/hostname.crt for each host:
    for host in host_list; do
      "Installing server certificate for $host from /tmp/$host.crt ..."
      curl -s --anyauth -X POST -u admin:admin -H \
        "content-type:text/plain" --data-binary @/tmp/$host.crt \
        http://node1:8002/manage/v2/certificates
    done

    If you prefer, you can put all of the certificates in a single ZIP file and post that to the /manage/v2/certificates endpoint.

    Uploading a ZIP file of certificates fails if the ZIP file contains an empty directory.

  13. Verify that the certificates have been installed. In the Admin UI, navigate to Configure > Security > Certificate Templates > OpsDirector-SSL-Template. In the right pane, select the Status tab. Make sure each node has a signed certificate and that it is not Temporary.
  14. Create a JSON file (in this example, we call it od-user.json) that creates a user called opsdirector-system-user:
    {
      "user-name": "opsdirector-system-user",
      "description": "Ops Director system user",
      "role": [ "opsdir-admin-internal" ],
      "external-name": [ "C=US,O=MarkLogic Corporation,CN=OpsDirector" ]
    }

    Make sure the external name exactly matches the client certificates generated in later steps.

  15. Use curl to POST this to the REST Management API (port 8002):
    curl -s --anyauth -X POST -u admin:admin -H \
       "content-type:application/json" \
       -d @/tmp/od-user.json \
       http://node1:8002/manage/v2/users
Configure the Managed Cluster

When using external certificates, configuring a new managed cluster requires changes on both the managed cluster and the Ops Director Cluster.

Setting up the Managed Cluster

Follow these instructions on the managed cluster to set up the managed cluster to work with Ops Director.

  1. If necessary, use the following curl command to install the CA on the cluster, replacing /tmp/opsdir.ca with the location and name of your CA:
    curl --anyauth -u admin:admin -X POST -H "content-type:text/plain" \
       --data-binary @/tmp/opsdir.ca \
       http://node3:8002/manage/v2/certificate-authorities
  2. Use one of the following commands to obtain the ID of the managed cluster:
    • XQuery:
      xquery version "1.0-ml";
      xdmp:cluster()
    • REST (via curl):
      curl -s --anyauth -u admin:admin http://builder:8002/manage/v2 \
         | grep "<id>" | sed "s/<id>//" | sed "s/<.*>//"
  3. Create the following JSON file to create a certificate template for the hosts on the cluster (in this example, we call it /tmp/md-template.json), replacing cluster_ID with the managed cluster ID from the previous step:
    {
      "template-name": "OpsDirector Template cluster_ID",
      "template-description": "Ops Director SSL Template",
      "key-type": "rsa",
      "key-options": {
      "key-length": "2048"
      },
      "req": {
      "version": "0",
      "subject": {
       "countryName": "US",
       "stateOrProvinceName": "California",
       "localityName": "San Carlos",
       "organizationName": "MarkLogic Corporation"
      }
      }
    }
  4. Use curl to POST this JSON file to the REST Management API (port 8002):
    curl --anyauth -u admin:admin -X POST -H \
       "content-type:application/json" \
       -d @/tmp/md-template.json \
       http://node3:8002/manage/v2/certificate-templates
  5. Run the following code, substituting the names of each of the hosts in your cluster, separated by spaces, for host_list:
    for host in host_list; do
      echo "Generating CSR for $host to /tmp/$host.csr ..."
      curl -s --anyauth -X POST -u admin:admin -H \
       "content-type:application/json" \
        -d "{\"operation\": \"generate-certificate-request\", \
       \"common-name\": \"$host\", \"dns-name\": \"$host\" }" \
       "http://node3:8002/manage/v2/certificate-templates/OpsDirector Template cluster_ID" > /tmp/$host.csr
    done
  6. Get a signed server certificate for each CSR from the CA in step <HyperlinkSerif>1.
  7. To install these certificates, run the following code, substituting the names of each of the hosts in your cluster, separated by spaces, for host_list, and replacing /tmp/host.crt with the location and name of your saved certificate:
    for host in host_list; do
      echo "Installing server certificate for $host from /tmp/$host.crt ..."
      curl -s --anyauth -X POST -u admin:admin -H \
      "content-type:text/plain" --data-binary @/tmp/$host.crt \
      http://node3:8002/manage/v2/certificates
    done

    If you prefer, you can put all of the certificates in a single ZIP file and post that to the /manage/v2/certificates endpoint.

    Uploading a ZIP file of certificates fails if the ZIP file contains an empty directory.

  8. Verify that the certificates have been installed. Navigate to Configure > Security > Certificate Templates > OpsDirector-SSL-Template. In the right pane, select the Status tab. Make sure each node has a signed certificate and that it is not Temporary.
  9. Use one of the following commands to obtain the ID of the CA from step 1, replacing OpsDirector CA with the name of your CA:
    • XQuery:
      xquery version "1.0-ml";
      import module namespace pki = "http://marklogic.com/xdmp/pki" at "/MarkLogic/pki.xqy";
      xdmp:invoke-function(function () {
      /pki:certificate[pki:authority=true()][pki:host-name='OpsDirector CA']/pki:certificate-id/data()
      }, map:entry("database", xdmp:security-database()))
    • REST (via curl):
      curl -s --anyauth -u admin:admin "http://localhost:8002/manage/v2/certificate-authorities?format=json" | jq ".\"certificate-authorities-default-list\".\"list-items\".\"list-item\"[] | select(.nameref==\"OpsDirector CA\") | .idref" | tr -d '"'

      The https://stedolan.github.io/jq/ tool must be on your system for the REST command to work.

      In this example, the Certificate Authority is 5955670036362913372.

  10. Create the following JSON file (in this example, we use /tmp/md-extsec.json) to create an external security module, replacing cluster_ID with the name of your cluster and 5955670036362913372 with your CA:
    {
      "external-security-name": "OpsDirectorSystem-cluster_ID",
      "description": "Ops Director System",
      "authentication": "certificate",
      "cache-timeout": 300,
      "authorization": "internal",
      "ssl-client-certificate-authority": [ "5955670036362913372" ]
    }
  11. Use curl to POST this certificate template to the REST Management API (port 8002):
    curl --anyauth -u admin:admin -X POST -H \
       "content-type:application/json" \
       -d @/tmp/md-extsec.json \
       http://node3:8002/manage/v2/external-security
  12. Create the following JSON file (in this example, we use /tmp/md-client.json) to generate a CSR for the Ops Director client certificate, matching the parameters to those from step 14 in the previous section:
    {
      "operation": "client-certificate-request",
      "countryName": "US",
      "organizationName": "MarkLogic Corporation",
      "commonName": "OpsDirector"
    }
  13. Use curl to POST this certificate template to the REST Management API (port 8002):
    curl -s --anyauth -X POST -u admin:admin -H \
       "content-type:application/json" \
       -d @/tmp/md-client.json \
       http://node3:8002/manage/v2/credentials/secure \
       > /tmp/md-client.csr

    The server responds by generating a private/public key pair, storing the relevant private key in the Security database and returning a client certificate request.

  14. Get a signed client certificate for the CSR from the CA in step <HyperlinkSerif>1.
  15. Create an XML file (in this example we use /tmp/md-secure-credential.xml) that constructs a secure credential for accessing the Ops Director cluster, replacingcluster_ID with the cluster ID, certificate with the certificate you obtained in the previous step, and 8009 with the Ops Director system port:
    <secure-credential-properties xmlns="http://marklogic.com/manage/secure-credential/properties">
      <name>MarkLogic-OpsDirector-cluster_ID</name>
      <description>Secure credential to access the Ops Director cluster</description>
      <certificate>certificate</certificate>
      <targets>
      <target>
       <uri-pattern>https://.*:8009/.*</uri-pattern>
       <authentication>basic</authentication>
      </target>
      </targets>
      <signing>false</signing>
      <permissions>
      <permission>
       <role-name>admin</role-name>
       <capability>read</capability>
      </permission>
      </permissions>
    </secure-credential-properties>
  16. Use curl to POST this credential to the REST Management API (port 8002):
    curl -s --anyauth -X POST -u admin:admin -H \
       "content-type:application/xml" \
       --data-binary @/tmp/md-secure-credential.xml \
       http://node3:8002/manage/v2/credentials/secure
  17. Create the following JSON file (in this example, we use /tmp/md-user.json) to construct a user called opsdirector-system-user-cluster_ID, replacing cluster_ID with the ID of the cluster, and replacing the parameters in the external-name field with the external name from step 12:
    {
      "user-name": "opsdirector-system-user-cluster_ID",
      "description": "Ops Director system user",
      "role": [ "manage-admin", "security" ],
      "external-name": [ "C=US,O=MarkLogic Corporation,CN=OpsDirector" ]
    }

    The external name must exactly match what you put in the client certificate!

  18. Use curl to POST this certificate template to the REST Management API (port 8002):
    curl -s --anyauth -X POST -u admin:admin -H \
       "content-type:application/json" \
       -d @/tmp/md-user.json \
       http://node3:8002/manage/v2/users

Setting Up the Ops Director Cluster

Follow these instructions on the Ops Director cluster to set up the managed cluster to work with Ops Director.

  1. Create a JSON file (in this example, we use /tmp/od-client.json) to generate a CSR for the Ops Director managed cluster client certificate, making sure to match the parameters in step 12:
    {
      "operation": "client-certificate-request",
      "countryName": "US",
      "organizationName": "MarkLogic Corporation",
      "commonName": "OpsDirector"
    }
  2. Use curl to POST the request to the REST Management API on the Ops Director cluster:
    curl -s --anyauth -X POST -u admin:admin -H \
       "content-type:application/json" \
       -d @/tmp/od-client.json \
       http://node1:8002/manage/v2/credentials/secure \
       > /tmp/od-client.csr

    The server responds by generating a private/public key pair, storing the relevant private key in the Security database, and returning a client certificate request.

  3. Get a signed client certificate for that CSR from the CA.
  4. Create an XML file (in this example, we use /tmp/od-secure-credential.xml) that constructs a secure credential for accessing the managed cluster, replacing cluster_ID with the name of your cluster ID and certificate with the certificate obtained in the previous step:
    <secure-credential-properties xmlns="http://marklogic.com/manage/secure-credential/properties">
      <name>opsdir-cluster_ID</name>
      <description>Secure credential to access the cluster_ID cluster</description>
      <certificate>certificate</certificate>
      <targets>
      <target>
       <uri-pattern>https://.*:8003/.*</uri-pattern>
       <authentication>basic</authentication>
      </target>
      </targets>
      <signing>false</signing>
      <permissions>
      <permission>
       <role-name>admin</role-name>
       <capability>read</capability>
      </permission>
      </permissions>
    </secure-credential-properties>
  5. Install the CA on the cluster:
    curl --anyauth -u admin:admin -X POST -H "content-type:text/plain" \
       --data-binary @/tmp/opsdir.ca \
       http://node1:8002/manage/v2/certificate-authorities
  6. Use curl to POST this to the REST Management API (port 8002):
    curl -s --anyauth -X POST -u admin:admin -H \
       "content-type:application/xml" \
       --data-binary @/tmp/md-secure-credential.xml \
       http://node1:8002/manage/v2/credentials/secure

Now you can manage the cluster using external certificates.

Using Externally Signed Certificates with MarkLogic 9.0-8 and earlier or Ops Director 2.0 and Earlier

The following sections describe how to configure MarkLogic Server 9.0-8 and earlier and Ops Director 2.0 and earlier to use externally signed certificates.

The steps involved for configuring Ops Director to use externally signed certificates are:

Configure the Ops Director 2.0 and Earlier Cluster

To configure the Ops Director 2.0 and earlier cluster:

  1. Use curl to install the Certificate Authority on the cluster:
    curl --anyauth -u admin:admin -X POST -H "content-type:text/plain" \
       --data-binary @/tmp/opsdir.ca \
       http://node1:8002/manage/v2/certificate-authorities
  2. Install Ops Director.
    • If you are using MarkLogic 9.0-7 or earlier, use the Admin UI. See Installing Ops Director for details.
    • If you are using MarkLogic 9.0-8 or later, use the Gradle installer for Ops Director 2.0-0:
      ./gradlew -PopsdirCa=external mlDeploy

      You may need to specify additional options, such as mlHost, depending on your configuration.

  3. In the Admin UI, navigate to Configure > Groups > Default. On the Scheduled Tasks tab, disable the task /common/tasks/running.xqy:

  4. Create the following JSON file (in this example, we use /tmp/od-template.json) to create a certificate template for the hosts on the cluster:
    {
      "template-name": "OpsDirector-SSL-Template",
      "template-description": "Ops Director SSL Template",
      "key-type": "rsa",
      "key-options": {
      "key-length": "2048"
      },
      "req": {
      "version": "0",
      "subject": {
       "countryName": "US",
       "stateOrProvinceName": "California",
       "localityName": "San Carlos",
       "organizationName": "MarkLogic Corporation"
      }
      }
    }
  5. Use curl to POST this to the REST Management API (port 8002):
    curl --anyauth -u admin:admin -X POST -H \
       "content-type:application/json" \
       -d @/tmp/od-template.json \
       http://node1:8002/manage/v2/certificate-templates
  6. Obtain the ID of the Certificate Authority from Step <HyperlinkSerif>1. If you are using a different certificate, replace OpsDirector CA with the common name of your certificate. The following XQuery code accomplishes this:
    xquery version "1.0-ml";
    import module namespace pki = "http://marklogic.com/xdmp/pki" at "/MarkLogic/pki.xqy";
    xdmp:invoke-function(function () {
     /pki:certificate[pki:authority=true()][pki:host-name='OpsDirector CA']/pki:certificate-id/data()
    }, map:entry("database", xdmp:security-database()))

    If you prefer, the following curl command will do it with the REST Management API. Note that this command relies on the https://stedolan.github.io/jq/ tool, which must be on your system.

    curl -s --anyauth -u admin:admin \
       "http://localhost:8002/manage/v2/certificate-authorities?format=json" \
       | jq ".\"certificate-authorities-default-list\".\"list-items\".\"list-item\"[]\ 
       | select(.nameref==\"OpsDirector CA\") | .idref" \
       | tr -d '"'

    Either of these techniques will give you the ID of the OpsDirector CA certificate authority (in this example, it is 5955670036362913372).

  7. Create a JSON file (in this example, we call it /tmp/od-extsec.json) where the external security name is called OpsDirectorSystem, replacing the 5955670036362913372 with the number you got when running the previous step:
    {
      "external-security-name": "OpsDirectorSystem",
      "description": "Ops Director System",
      "authentication": "certificate",
      "cache-timeout": 300,
      "authorization": "internal",
      "ssl-client-certificate-authority": [ "5955670036362913372" ]
    }
  8. Use curl to post the JSON code to the external security endpoint using the REST Management API (in this example, http://node1:8002/manage/v2/external-security):
    curl --anyauth -u admin:admin -X POST -H \
       "content-type:application/json" \
       -d @/tmp/od-extsec.json \
       http://node1:8002/manage/v2/external-security
  9. To update the server configuration, create an XML file (in this example, we call it /tmp/od-server.xml) to update the server configuration to modify some of the properties of the OpsDirectorSystem application server. In this example, the CA is ssl-client-certificate-pem, and Insert_the_certificate_here gets replaced with your certificate (this example uses the certificate from the beginning of this section):
    <http-server-properties xmlns="http://marklogic.com/manage">
      <enabled>true</enabled>
      <authentication>application-level</authentication>
      <external-securities>
      <external-security>OpsDirectorSystem</external-security>
      </external-securities>
      <ssl-certificate-template>OpsDirector-SSL-Template</ssl-certificate-template>
      <ssl-client-certificate-pems>
      <ssl-client-certificate-pem>-----BEGIN CERTIFICATE-----
    Insert_the_certificate_here
    -----END CERTIFICATE-----</ssl-client-certificate-pem>
      </ssl-client-certificate-pems>
    </http-server-properties>
  10. Use curl to PUT this file to the REST Management API (port 8002):
    curl --anyauth -u admin:admin -X PUT -H \
      "content-type:application/xml" \
      --data-binary @/tmp/od-server.xml \
      http://node1:8002/manage/v2/servers/OpsDirectorSystem/properties?group-id=Default

    If you are not using the Default group for the Ops Director servers, update the group-id accordingly.

  11. Generate server certificate requests for each of the hosts:
    1. Navigate to Configure > Security > Certificate Templates > OpsDirector-SSL-Template. In the right pane, select the Request tab.
    2. Generate CSRs for all hosts.
    3. On the Status tab, download the Pending Certificate Requests.
    4. Get them signed as server certificates, and put all of the signed certificates in a ZIP file at /tmp/od-certs.zip.
  12. Use curl to upload the certificates:
    curl --anyauth -u admin:admin -X POST -H \
      "content-type:application/xml" \
      --data-binary @/tmp/od-certs.zip \
      http://node1:8002/manage/v2/certificates
  13. Verify that the certificates have been installed. Navigate to Configure > Security > Certificate Templates > OpsDirector-SSL-Template. In the right pane, select the Status tab. Make sure each node has a signed certificate and that it is not Temporary.

    If any nodes still show a pending certificate request or a temporary certificate, go back to step <HyperlinkSerif>11. Make sure you select to generate requests for All nodes and repeat the steps to download, sign, and upload the certificates.

  14. In the Admin UI, navigate to Configure > Groups > Default. On the Scheduled Tasks tab, enable the task /common/tasks/running.xqy:
  15. Create a JSON file (in this example, we call it od-user.json) that creates a user called opsdirector-system-user, replacing password with a password of your choice:
    {
      "user-name": "opsdirector-system-user",
      "password": "password",
      "description": "Ops Director system user",
      "role": [ "opsdir-admin-internal" ],
      "external-name": [ "C=US,O=MarkLogic Corporation,CN=OpsDirector" ]
    }
  16. Use curl to POST this to the REST Management API (port 8002):
    curl -s --anyauth -X POST -u admin:admin -H \
       "content-type:application/json" \
       -d @/tmp/od-user.json \
       http://node1:8002/manage/v2/users

    The external name must exactly match the subject of the client certificates that you generated when setting up a managed cluster.

Manage a Cluster using Ops Director 2.0 and Earlier

When using external certificates, configuring a new managed cluster requires changes on both the managed cluster and the Ops Director Cluster.

Setting Up the Managed Cluster

Follow these instructions on the managed cluster to set up the managed cluster to work with Ops Director 2.0 and earlier.

  1. Use the following curl command to install the CA on the cluster, replacing /tmp/opsdir.ca with your CA location and name:
    curl --anyauth -u admin:admin -X POST -H "content-type:text/plain" \
       --data-binary @/tmp/opsdir.ca \
       http://node3:8002/manage/v2/certificate-authorities
  2. Use one of the following commands to obtain the ID of the managed cluster:
    • XQuery:
      xquery version "1.0-ml";
      xdmp:cluster()
    • REST (via curl):
      curl -s --anyauth -u admin:admin http://builder:8002/manage/v2 \
         | grep "<id>" | sed "s/<id>//" | sed "s/<.*>//"
  3. Create the following JSON file to create a certificate template for the hosts on the cluster (in this example, we call it md-template.json), replacing cluster_ID with the managed cluster ID from the previous step:
    {
      "template-name": "OpsDirector Template cluster_ID",
      "template-description": "Ops Director SSL Template",
      "key-type": "rsa",
      "key-options": {
      "key-length": "2048"
      },
      "req": {
      "version": "0",
      "subject": {
       "countryName": "US",
       "stateOrProvinceName": "California",
       "localityName": "San Carlos",
       "organizationName": "MarkLogic Corporation"
      }
      }
    }
  4. Use curl to POST this JSON file to the REST Management API (port 8002):
    curl --anyauth -u admin:admin -X POST -H \
       "content-type:application/json" \
       -d @/tmp/md-template.json \
       http://node3:8002/manage/v2/certificate-templates
  5. Use one of the following commands to obtain the ID of the CA from step 1:
    • XQuery:
      xquery version "1.0-ml";
      import module namespace pki = "http://marklogic.com/xdmp/pki" at "/MarkLogic/pki.xqy";
      xdmp:invoke-function(function () {
      /pki:certificate[pki:authority=true()][pki:host-name='OpsDirector CA']/pki:certificate-id/data()
      }, map:entry("database", xdmp:security-database()))
    • REST (via curl):
      curl -s --anyauth -u admin:admin "http://localhost:8002/manage/v2/certificate-authorities?format=json" | jq ".\"certificate-authorities-default-list\".\"list-items\".\"list-item\"[] | select(.nameref==\"OpsDirector CA\") | .idref" | tr -d '"'

      The https://stedolan.github.io/jq/ tool must be on your system for the REST command to work.

      Either of these techniques will give you the ID of the OpsDirector CA certificate authority.

  6. Create the following JSON file (in this example, we use /tmp/md-extsec.json) to create an external security module, replacing cluster_ID with the ID of the Ops Director cluster and CA_ID with your CA ID:
    {
      "external-security-name": "OpsDirectorSystem-cluster_ID",
      "description": "Ops Director System",
      "authentication": "certificate",
      "cache-timeout": 300,
      "authorization": "internal",
      "ssl-client-certificate-authority": [ "CA_ID" ]
    }
  7. Use curl to POST this certificate template to the REST Management API (port 8002):
    curl --anyauth -u admin:admin -X POST -H \
       "content-type:application/json" \
       -d @/tmp/md-extsec.json \
       http://node3:8002/manage/v2/external-security
  8. Run the following XQuery code to generate server certificate requests for each of the hosts:
    xquery version "1.0-ml";
     
    import module namespace pki="http://marklogic.com/xdmp/pki"
      at "/MarkLogic/pki.xqy";
     
    declare namespace x509="http://marklogic.com/xdmp/x509";
     
    let $tid := pki:get-template-by-name("OpsDirector Template")/pki:template-id
    for $hid in xdmp:hosts()
      let $hostname := xdmp:host-name($hid)
      let $csr := pki:generate-certificate-request($tid, $hostname, (), ())
    return
      $csr
  9. Get a signed server certificate for each CSR from the CA in step .
  10. Put all of the signed certificates in a ZIP file (in this example, we use /tmp/mc-certs.zip).
  11. Use curl to upload the certificates:
    curl --anyauth -u admin:admin -X POST -H \
       "content-type:application/zip" \
       --data-binary @/tmp/mc-certs.zip \
       http://node3:8002/manage/v2/certificates
  12. Verify that the certificates have been installed. Navigate to Configure > Security > Certificate Templates > OpsDirector-SSL-Template. In the right pane, select the Status tab. Make sure each node has a signed certificate and that it is not Temporary.
  13. Run the following XQuery code to generate a client CSR:

    The subject must exactly match the external name of the opsdirector-system-user on the Ops Director cluster.

    xquery version "1.0-ml";
     
    declare namespace x509 = "http://marklogic.com/xdmp/x509";
     
    let $country := "US"
    let $state := ()
    let $city := ()
    let $org := "MarkLogic Corporation"
    let $unit := ()
    let $email := ()
    let $common := "OpsDirector"
     
    let $keys := xdmp:rsa-generate()
    let $privkey := $keys[1]
    let $pubkey := $keys[2]
    let $req := <x509:req>
           <x509:version>0</x509:version>
           <x509:subject>
            { $country ! <x509:countryName>{.}</x509:countryName> }
            { $state   ! <x509:stateOrProvinceName>{.}</x509:stateOrProvinceName> }
            { $city  ! <x509:localityName>{.}</x509:localityName> }
            { $org   ! <x509:organizationName>{.}</x509:organizationName> }
            { $unit  ! <x509:organizationalUnitName>{.}</x509:organizationalUnitName> }
            { $email   ! <x509:emailAddress>{.}</x509:emailAddress> }
            { $common  ! <x509:commonName>{.}</x509:commonName> }
           </x509:subject>
           <x509:publicKey>{$pubkey}</x509:publicKey>
          </x509:req>
    let $csr   := xdmp:x509-request-generate($req, $privkey)
    return
      ($csr, $privkey)
  14. Get the CSR signed as a client certificate.
  15. Run the following XQuery code to create a credential using the client certificate and its private key, replacing certificate with the client certificate and private_key with the private key for the certificate:
    xquery version "1.0-ml";
     
    import module namespace sec="http://marklogic.com/xdmp/security"
      at "/MarkLogic/security.xqy";
     
     
    let $credname := "MarkLogic-OpsDirector"
    let $opts   := <options xmlns="xdmp:document-get"><format>text</format></options>
    let $cert   := "certificate"
    let $privkey  := "private_key"
    let $uri-pattern := "https://.*:8009/.*"
    return
      xdmp:invoke-function(function() {
      sec:create-credential($credname,
        "A credential for accessing .*:8009",
        (), (), $cert, $privkey, fn:false(),
        sec:uri-credential-target($uri-pattern, "basic"),
        xdmp:permission("security", "read"))
    
      }, map:entry("database", xdmp:security-database()))
  16. Create a JSON file (in this example, we call it od-user.json) that creates a user called opsdirector-system-user, replacing password with a password of your choice:
    {
      "user-name": "opsdirector-system-user",
      "password": "password",
      "description": "Ops Director system user",
      "role": [ "opsdir-admin-internal" ],
      "external-name": [ "C=US,O=MarkLogic Corporation,CN=OpsDirector" ]
    }

    The external name must exactly match the subject of the client certificates that you generated when setting up a managed cluster.

  17. Use curl to POST this to the REST Management API (port 8002):
    curl -s --anyauth -X POST -u admin:admin -H \
       "content-type:application/json" \
       -d @/tmp/od-user.json \
       http://node3:8002/manage/v2/users

Setting Up the Ops Director Cluster

Follow these instructions on the Ops Director cluster to set up the managed cluster to work with Ops Director 2.0-0 and earlier.

  1. Run the following XQuery code to generate a client CSR:
    xquery version "1.0-ml";
     
    declare namespace x509 = "http://marklogic.com/xdmp/x509";
     
    let $country := "US"
    let $state := ()
    let $city := ()
    let $org := "MarkLogic Corporation"
    let $unit := ()
    let $email := ()
    let $common := "OpsDirector"
     
    let $keys := xdmp:rsa-generate()
    let $privkey := $keys[1]
    let $pubkey := $keys[2]
    let $req := <x509:req>
      <x509:version>0</x509:version>
      <x509:subject>
        { $country ! <x509:countryName>{.}</x509:countryName> }
        { $state   ! <x509:stateOrProvinceName>{.}</x509:stateOrProvinceName> }
        { $city  ! <x509:localityName>{.}</x509:localityName> }
        { $org   ! <x509:organizationName>{.}</x509:organizationName> }
        { $unit  ! <x509:organizationalUnitName>{.}</x509:organizationalUnitName> }
        { $email   ! <x509:emailAddress>{.}</x509:emailAddress> }
        { $common  ! <x509:commonName>{.}</x509:commonName> }
       </x509:subject>
       <x509:publicKey>{$pubkey}</x509:publicKey>
     </x509:req>
    let $csr   := xdmp:x509-request-generate($req, $privkey)
    return
      ($csr, $privkey)

    The subject must exactly match external name of the opsdirector-system-user on the managed cluster.

  2. Get the CSR signed as a client certificate.
  3. Run the following XQuery code to create a credential using the client certificate and its private key, replacing cluster_ID with the cluster ID, certificate with the client certificate, and private_key with the private key:
    xquery version "1.0-ml";
     
    import module namespace sec="http://marklogic.com/xdmp/security"
        at "/MarkLogic/security.xqy";
     
     
    let $credname := "opsdir-cluster_ID"
    let $opts     := <options xmlns="xdmp:document-get"><format>text</format></options>
    let $cert     := "certificate"
    let $privkey  := "private_key"
    let $uri-pattern := "https://.*:8003/.*"
    return
      xdmp:invoke-function(function() {
        sec:create-credential($credname,
           "A credential for accessing SecureManage on the cluster",
           (), (), $cert, $privkey, fn:false(),
           sec:uri-credential-target($uri-pattern, "basic"),
           xdmp:permission("security", "read"))
     
      }, map:entry("database", xdmp:security-database()))

You can now manage the cluster from the Admin UI using Ops Director 2.0-0 and earlier.

Upgrading Ops Director

This section describes the upgrade process for the Ops Director application. It covers the following topics:

Upgrade Process Overview

Ops Director is shipped with the MarkLogic Server. This means that a server upgrade necessarily upgrades the version of Ops Director, from the perspective of executable code. However, the server upgrade process updates neither the databases that Ops Director uses nor the configurations of Ops Director application servers.

The Ops Director upgrade process provides a seamless way to update the application, databases, and configuration from the Ops Director UI. For details on the possible upgrade workflows, see Upgrade Scenarios and Workflows.

Version Compatibility and Order of Upgrade

The Ops Director application is part of the MarkLogic Server release; its version depends on the version of the server installed. For example:

  • Ops Director version 1.0-1 is installed with MarkLogic Server releases 9.0-3 and 9.0-4;
  • Ops Director version 1.1-1 is installed with MarkLogic Server release 9.0-5.

Ops Director can manage only clusters that run either the same version of MarkLogic Server as the version it was shipped with or one of the older versions starting with MarkLogic Server 9.0-2. The version compatibility tables follow.

Ops Director Version Can manage clusters with MarkLogic releases
1.0-1, 1.1-1 9.0-2 to 9.0-6
2.0 9.0-7 to 9.0-8
2.0-1 9.0-7 and later
If Running MarkLogic Server Version Can Install Ops Director Version If Running Ops Director Version Can Upgrade To Ops Director Version
9.0-6 and earlier n/a 1.0.0 n/a
9.0-7 2.0, 2.0.1 1.1.0 2.0, 2.0.1
9.0-7 and later 2.0.1 1.1.1, 2.0 2.0.1

This version compatibility defines the order of upgrade: first, the Ops Director Application Cluster must be upgraded to the latest MarkLogic Server release. Then you can upgrade managed clusters to the latest MarkLogic Server release.

Upgrade Scenarios and Workflows

Possible scenarios and workflows of upgradingOps Director are described in this section.

The Ops Director upgrade procedure requires that you have the admin role. It is not sufficient to login as opsdir-admin. You must login with administrator credentials of the MarkLogic Server.

Scenario 1: Single host, MarkLogic Server upgraded, Ops Director databases need update.

In this scenario, the Ops Director Application Cluster is a single-node cluster. The single host has already been upgraded to the latest MarkLogic Server release. The Ops Director databases and configuration need to be updated.

Perform the following steps:

  1. Launch the Ops Director application on the Ops Director host: in a browser window, enter the hostname and the port number of the OpsDirectorApplication server (by default, 8008).
  2. Login with admin credentials. The following message displays:

    Ops Director update required. Ops Director is being updated on this cluster. Please wait....

    The waiting indicator shows that the update is in progress.

  3. Once the update has completed and the Ops Director application is ready to run, the following message displays:

    Ops Director update completed. Ops Director has been updated to version version.

  4. Click ok to launch the Ops Director application.

    If you encounter errors in the Ops Director application after upgrade, clear the browser cache and reload the Ops Director application.

Scenario 2: Multiple hosts, one or more need upgrade of MarkLogic Server release.

In this scenario, the Ops Director Application Cluster is a multi-node cluster. Some of the hosts have already been upgraded to the latest MarkLogic Server release, but one or more hosts still have to be upgraded.

Perform the following steps:

  1. Launch the Ops Director application on a host of the Ops Director cluster: in a browser window, enter the hostname and the port number of the OpsDirectorApplication server (by default, 8008).
  2. Login with admin credentials. The following message displays:

    Cluster upgrade must be completed. The following host(s) must be upgraded before Ops Director can run on this host: [hostname1], [hostname2], [hostname3]. Retry when upgrades have been completed.

  3. For each host from the list of hosts provided in the message on the previous step, upgrade to the latest MarkLogic Server release. Follow the steps described in Upgrading from Previous Releases in the Installation Guide.

    After you installed the latest MarkLogic Server release on the host and started MarkLogic Server, make sure to open the Admin Interface in a browser (http://hostname:8001/), and, when the Admin Interface prompts you to upgrade the databases and the configuration files, click the button to confirm the upgrade. Also, make sure to reload Ops Director application in the browser (http://hostname:8008/) after the server is upgraded.

  4. While hosts are being upgraded, you may check the status of the upgrade by clicking the Retry button on the message box from step 2.

    Once all hosts have been upgraded, the following message displays: Ops Director update required. Ops Director is being updated on this cluster. Please wait....

    The waiting indicator shows that the update is in progress.

    Once the update is completed and the Ops Director application is ready to run, the following message displays:

    Ops Director update completed. Ops Director has been updated to version version.

  5. Click ok to launch the Ops Director application.

    If you encounter errors in the Ops Director application after the upgrade, clear the browser cache and reload the Ops Director application.

Scenario 3: Multiple hosts, all upgraded, Ops Director databases' update is in progress.

In this scenario, the Ops Director Application Cluster is a multi-node cluster. All hosts in the cluster have already been upgraded to the latest MarkLogic Server release. The Ops Director databases update is currently in progress (due to being initiated from a different host).

Perform the following steps:

  1. Launch the Ops Director application on a host of the Ops Director cluster: in a browser window, enter the hostname and the port number of the OpsDirectorApplication server (by default, 8008).
  2. Login with admin credentials. The following message displays:

    Ops Director update in progress. Ops Director is being updated on this cluster. Please wait....

    The waiting indicator shows that the update is in progress. Once the update is completed and the Ops Director application is ready to run, the following message displays:

    Ops Director update completed. Ops Director has been updated to version [version].

  3. Click ok to launch the Ops Director application.

    If you encounter errors in the Ops Director application after upgrade, clear the browser cache and reload the Ops Director application.

Upgrade Error Prevention and Troubleshooting

Before you being to upgrade the Ops Director Application Cluster hosts to the latest MarkLogic Server release, make sure to close your browser containing the Ops Director application. If you do not close your browser, you may see errors in the browser window.

If you have started an Ops Director upgrade, do not connect new managed clusters until the upgrade process has completed. Adding managed clusters in the middle of an upgrade can cause error conditions.

If upgrading any host in the cluster to the latest MarkLogic Server release fails, address the problem before trying to launch Ops Director.

Undelivered data from managed clusters might be lost during the upgrade process, just like during network outage.

« Previous chapter
Next chapter »