Scripting Administrative Tasks Guide (PDF)

Scripting Administrative Tasks Guide — Chapter 2

« Previous chapter
Next chapter »

Using the Admin API

MarkLogic Server includes an Admin Library Module that provides XQuery functions that enable you to perform many administrative tasks on MarkLogic Server. With the Admin API, you can write XQuery programs to create or modify databases, forests, App Servers, and perform all kinds of administrative and configuration tasks on MarkLogic Server.

The Admin API is different from the other administrative APIs listed in The Administration APIs in that the Admin API functions operate on an in-memory cluster configuration that is extracted from and then saved to the configuration files on the server's file system. This design impacts what configuration tasks you can accomplish within a single transaction, as described in Transactional Considerations When Using the Admin API and Techniques for Making Multiple Changes to a Configuration.

This chapter describes how you can use the Admin API to perform administrative tasks for MarkLogic Server, and includes the following sections:

Tasks You Can Perform Using the Admin API

The Admin API provides an alternative way to make configuration changes to MarkLogic Server compared to using the Admin Interface. The Admin API is a large library of XQuery functions, providing a programmatic interface to do most things that the Admin Interface does, including:

  • Creating and modifying databases
  • Creating and modifying forests
  • Creating and modifying groups
  • Creating and modifying app servers
  • Modifying host configurations

Transactional Considerations When Using the Admin API

The Admin API modifies MarkLogic Server configuration files that are stored on the file system, not in a database. Because the configuration files are not stored in a database, database-level locking does not occur on the configuration files and the transactional semantics are different from updates to a database. For this reason, only one process should update the configuration files at a time:

  • Only run one Admin API script that modifies the configuration at a time; do not have concurrent users simultaneously modifying the configuration.
  • No other session should modify the configuration with the Admin Interface concurrent to modifying the configuration with the Admin API.

Additionally, avoid doing multiple save operations to the configuration files in the same transaction. Instead, perform each step in memory before passing the changed configuration to the next function, as described in Techniques for Making Multiple Changes to a Configuration.

Privileges Required For Running Admin APIs

The APIs in the Admin Library module are protected by the following two execute privileges:

  • http://marklogic.com/xdmp/privileges/admin-module-read
  • http://marklogic.com/xdmp/privileges/admin-module-write

Any user who runs XQuery code containing the Admin API functions needs the to have the first privilege (via a role) for reading anything in the configuration and needs both privileges for writing anything to the configuration. This allows you control access to your configuration information. For details about security in MarkLogic Server, see Understanding and Using Security Guide.

General Steps for Scripting Administrative Tasks

Because the Admin API is implemented as an XQuery module, you can write XQuery programs which will modify your MarkLogic Server configuration. Without using the Admin API, you must use the Admin Interface to perform most administrative tasks.

This section outlines the general steps needed to execute functions in the Admin API. These steps assume you are running against either an HTTP App Server or an XDBC App Server using XCC to issue the XQuery requests. This section includes the following parts:

Importing the Admin Library Module

Because the Admin API is an XQuery library module, you must import it into your XQuery program. The Admin API is installed in the <marklogic-dir>/Modules/MarkLogic/admin.xqy file. To import the Admin API module, include the following in your XQuery prolog:

xquery version "1.0-ml";
import module namespace admin = "http://marklogic.com/xdmp/admin" 
   at "/MarkLogic/admin.xqy";

Getting the Current Configuration in Memory

The admin:get-configuration function gets an in-memory representation of the configuration for the purpose of modifying the configuration. It is designed to only be used with the Admin API, not to get the configuration for other purposes. To optimize performance, the admin:get-configuration function gets only the configuration information it needs to process configuration information in a given XQuery request. The configuration information is an XML structure, and it is designed to be used in conjunction with the other Admin API functions; do not try and use the admin:get-configuration function outside of the scope of the Admin API.

The typical design pattern to get the configuration is to get the configuration once per XQuery request and bind its value to a variable (for example, in the let clause of a FLWOR expression). Then you can use that variable through the query to refer to the configuration. Do not try and change the XML representation returned from the admin:get-configuration function.

Because it is optimized to only get the parts of the configuration it needs to make a set of changes, you might find that the admin:get-configuration function sometimes returns an empty element if you run it without performing any other functions. It is designed to be used only with the Admin API functions.

Saving the Configuration Changes

Once you use the various APIs to modify the configuration, you must save the configuration to make it take effect in the MarkLogic Server cluster. There are two APIs to save the configuration:

Both functions take a configuration element, and then save that element to the configuration files. The configuration file changes are propagated to the entire cluster. The difference between the two functions is that the first one will automatically restart any MarkLogic Server instances when the configuration changes are 'cold' and require a restart. The admin:save-configuration-without-restart function does not automatically restart MarkLogic Server; if a restart is needed for the changes to take effect, it returns the host IDs of the hosts needing a restart; if no restart is required, it returns the empty sequence.

You should only call these functions (one or the other of two) once per XQuery program (that is, once per transaction).

Techniques for Making Multiple Changes to a Configuration

When writing a program that creates an entire configuration, it is most efficient to write modules that create and configure multiple server objects. This section describes some useful techniques to use when writing modules that make multiple changes to a configuration. The topics are:

Saving versus Passing the Configuration

There are a number of ways to manage multiple changes to the configuration. The simplest, but most cumbersome, approach is to get and save the configuration for each admin function in a separate transaction. For example:

xquery version "1.0-ml";
import module namespace admin = "http://marklogic.com/xdmp/admin" 
         at "/MarkLogic/admin.xqy";
let $config := admin:get-configuration()
return admin:save-configuration(admin:function-call-1($config, ....));
xquery version "1.0-ml";
import module namespace admin = "http://marklogic.com/xdmp/admin" 
         at "/MarkLogic/admin.xqy";
let $config := admin:get-configuration()
return admin:save-configuration(admin:function-call-2($config, ....));

Another approach is to pass the configuration returned from a previous admin function to the next admin function. For example, in the following pseudo-code example, the $config variable returned from admin:get-configuration holds the initial configuration of MarkLogic Server. The $config variable is progressively updated as it is returned from each admin function and passed as input to the next admin function. The final $config variable holds the configuration from all of the admin function calls and is saved by admin:save-configuration.

xquery version "1.0-ml";
import module namespace admin = "http://marklogic.com/xdmp/admin" 
         at "/MarkLogic/admin.xqy";
(: Get the configuration :)
let $config := admin:get-configuration()
(: Update the configuration :)
let $config := admin:function-call-1($config, ....)
let $config := admin:function-call-2($config, ....)
(: etc....... :)
(: Save the configuration :)
return admin:save-configuration($config5)

Creating and Configuring Objects in a Single Transaction

The admin:object-get-id functions (admin:forest-get-id, admin:database-get-id, admin:group-get-id, admin:appserver-get-id, admin:host-get-id) allow you to create and configure a server object in a single transaction. The main difference between the admin:object-get-id functions and their xdmp:object counterparts is that the admin:object-get-id functions return object IDs from the configuration, whether it has been saved or not, whereas their xdmp:object counterparts return the object IDs of existing objects in MarkLogic Server. In this way, the admin:object-get-id functions provide a faster alternative to what would otherwise be a need to create the object, save the configuration, configure the object, and save the configuration.

For example, the following is an abbreviated version of the code shown in Creating and Configuring Databases in a Single Transaction :

xquery version "1.0-ml";
import module namespace admin = "http://marklogic.com/xdmp/admin" 
         at "/MarkLogic/admin.xqy";
(: Get the configuration :)
let $config := admin:get-configuration()
(: Add new database to the configuration :)
let $config := admin:database-create(
    $config, 
    "Sample-Database",
    xdmp:database("Security"),
    xdmp:database("Schemas"))
(: Obtain the database ID of 'Sample-Database' :)
let $Sample-Database := admin:database-get-id(
    $config, 
    "Sample-Database")
(: Attach the "SampleDB-Forest" forest to 'Sample-Database' :)
let $config := admin:database-attach-forest(
    $config,
    $Sample-Database,
    xdmp:forest("SampleDB-Forest"))
(: Add an index to 'Sample-Database' :)
let $rangespec := admin:database-range-element-index(
    "string", 
    "http://marklogic.com/wikipedia",
    "name", 
    "http://marklogic.com/collation/",
    fn:false() )
let $config := admin:database-add-range-element-index(
    $config, 
    $Sample-Database, 
    $rangespec)
return
   admin:save-configuration($config)

The IDs returned by the admin:object-get-id functions should only be passed to other functions in the Admin library because they operate on the configuration of objects, rather than the existing objects in MarkLogic Server. Functions outside the Admin library only understand the existing objects in MarkLogic Server.

Making Transactions Idempotent

There may be circumstances in which you want to modify a configuration without the need to uninstall it first. For example, you may want to update your configuration program to change an App Server setting or add an index to a database without removing and reinstalling the entire configuration. To enable this option, you need to write idempotent transactions, so that the transaction doesn't fail if some of the objects already exist or have already been deleted.

For example, the transaction that creates and configures a database shown above in Creating and Configuring Objects in a Single Transaction will fail if any of the objects it attempts to create already exist. To make this transaction idempotent, you need to check for the existence of objects before creating them. If an object exists, then no change is made to the configuration. This allows you to modify the transaction later to create, modify, or delete individual objects without the entire transaction failing.

xquery version "1.0-ml";
import module namespace admin = "http://marklogic.com/xdmp/admin" 
         at "/MarkLogic/admin.xqy";
declare namespace xdmpdb = "http://marklogic.com/xdmp/database";
(: Get the configuration :)
let $config := admin:get-configuration()
(: Get all of the existing databases :)
let $ExistingDatabases :=
    for $id in admin:get-database-ids($config)
       return admin:database-get-name($config, $id)
(: Check to see if "Sample-Database" already exists. 
   If not, create new database :)
let $config :=
    if ("Sample-Database" = $ExistingDatabases) 
    then $config
    else  
      admin:database-create(
        $config, 
        "Sample-Database",
        xdmp:database("Security"),
        xdmp:database("Schemas"))
(: Obtain the database ID of 'Sample-Database' :)
let $Sample-Database := admin:database-get-id(
    $config, 
    "Sample-Database")
(: Get all of the forests attached to 'Sample-Database' :)
let $AttachedForests :=
    admin:forest-get-name( 
       $config, 
       (admin:database-get-attached-forests(
          $config, 
          $Sample-Database) ))
(: Check to see if the "SampleDB-Forest" forest is already attached 
   to 'Sample-Database'. If not, attach the forest to the database :)
let $config :=
    if ("SampleDB-Forest" = $AttachedForests) 
    then $config
    else
       admin:database-attach-forest(
          $config,
          $Sample-Database,
          xdmp:forest("SampleDB-Forest"))
(: Define a new range element index :)
let $rangespec := admin:database-range-element-index(
    "string", 
    "http://marklogic.com/wikipedia",
    "name", 
    "http://marklogic.com/collation/",
    fn:false() )
(: Get the existing range element indexes for 'Sample-Database' :)
let $ExistingREindexes :=
    fn:data(admin:database-get-range-element-indexes(
      $config, 
      $Sample-Database)/xdmpdb:localname)
(: Check to see if the 'name' range element index already exists 
   for "Sample-Database". If not, add the index :)
let $config :=
    if ("name" = $ExistingREindexes) 
    then $config
    else 
       admin:database-add-range-element-index(
          $config, 
          $Sample-Database, 
          $rangespec)
return
   admin:save-configuration($config)

When Separate Transactions are Needed

There are cases where you must make configuration changes in separate transactions. If you are creating a new object that is to be used by another object, each task must be done in a separate transaction. For example, to create a forest and attach it to a database, you must first create the forest and save the configuration in one transaction, then attach the forest to the database and save the configuration in a separate transaction. Similarly, if you are deleting an object and then deleting an object that uses that object, the deletes must occur and the configuration must be saved in separate transactions.

You can define separate transactions in a single module by separating the transactions with semi-colons. Alternatively you can create separate modules for each transaction or execute each admin create/delete function using a different-transaction isolation of an xdmp:eval or xdmp:invoke.

Specifically, for the following functions, the forest/database/group must first exist or be deleted/reassigned before calling the function or an exception is thrown:

Function Object Dependency
admin:database-attach-forest Forest must exist.
admin:appserver-set-database Content database must exist.
admin:appserver-set-modules-database Modules database must exist.
admin:host-set-group Group must exist.
admin:http-server-create Group and databases must exist.
admin:xdbc-server-create Group and databases must exist.
admin:webdav-server-create Group and databases must exist.
admin:group-delete All hosts must be reassigned to another group.
admin:database-delete All app servers that use the database must be deleted or reassigned another database.
admin:forest-delete All databases that attach the forest must be deleted or attached to another forest.

See Sequence for Creating Server and Security Objects and Sequence for Deleting Server and Security Objects for the specific order in which objects should be created and deleted. For details on transactions, see Understanding Transactions in MarkLogic Server in the Application Developer's Guide.

Sequence for Creating Server and Security Objects

When configuring a complete MarkLogic application environment, certain server objects must exist before others can be created. The sequence for creating server objects is shown in the configure-server function in the configure-server.xqy module in the sample configuration program. The sequence for creating security objects is shown in the install.xqy module.

In general, the sequence for creating new server objects is:

  1. Create and configure forests
  2. Save configuration
  3. Create and configure databases
  4. Save configuration
  5. Create and configure App Servers
  6. Save configuration
You can create the security objects before, after or anything in between creating the server objects.

If you are creating new roles and assigning them to new users, the sequence for creating the new security objects is:

  1. Create roles
  2. Create users

Sequence for Deleting Server and Security Objects

The sequence for deleting server objects is demonstrated in the remove-server function in the configure-server.xqy module. The security objects, roles and users, can be deleted in any order.

In general, the sequence for deleting server objects is the reverse order in which they were created.

Deleting an App Server automatically restarts MarkLogic Server. The remove-server function uses the admin:save-configuration-without-restart function to defer the server restart until all of the operations in the remove-server have completed.

  1. Delete App Servers
  2. Save configuration without restarting server
  3. Delete Databases
  4. Save configuration without restarting server
  5. Delete forests
  6. Save configuration and restart server

Executing Queries in Select Databases

By default, queries are executed in the database set for your App Server. For example, if you configure your App Server as shown in How to Execute the Sample Configuration Program, your queries are executed in the Documents database.

You can use the xdmp:eval or xdmp:invoke function to execute queries in a database other than the database set for your App Server. The use of the xdmp:eval function to execute queries in the Security and Sample-Modules databases is demonstrated in the install.xqy module in the sample configuration program.

For example, you can create a module, named create-role.xqy, with the contents:

xquery version "1.0-ml";
module namespace   create-role="http://marklogic.com/sampleConfig/create-role";
import module namespace sec="http://marklogic.com/xdmp/security" 
       at "/MarkLogic/security.xqy";
declare function configure-roles()
{
  sec:create-role(
    "Temporary",
    "Temporary worker access",
    ("filesystem-access"),
    (),
    ("testDocument"))
};

You can then call the create-role.xqy module from the main module within an xdmp:eval function with the <database> option that specifies the Security database.

xdmp:eval(
  'xquery version "1.0-ml";
  import module namespace create-role=     "http://marklogic.com/sampleConfig/create-role" 
      at "create-role.xqy";
  create-role:configure-roles()',
  (),
  <options xmlns="xdmp:eval">
    <database>{xdmp:database("Security")}</database>
  </options>)

« Previous chapter
Next chapter »
Powered by MarkLogic Server | Terms of Use | Privacy Policy