MarkLogic 9 Product Documentation

Java Application Developer's Guide — Chapter 1

Required Software

The Java Client API is supported on the same platforms as MarkLogic Server. For a complete list of platforms see Supported Platforms in the Installation Guide.

The Java Client API requires the following software:

• MarkLogic 9
• Oracle/Sun Java Runtime Environment (JRE) 1.8 or later

The Java Client API also requires access to a MarkLogic Server installation configured with a REST Client API instance. When you install MarkLogic 8 or later, a pre-configured REST API instance is available on port 8000. For more details, see Administering REST Client API Instances in the REST Application Developer's Guide.

For information specific to rolling upgrades, see Java Client API in the Administrator's Guide.

Make the Libraries Available to Your Application

You can make the Java Client API libraries available to your project in one of the following ways:

• ZIP File
• Maven
• Gradle

For more details, see the following page:

http://developer.marklogic.com/products/java

The Java Client API is an open-source project, so you can also access the sources and build your own library. For details, see Downloading the Library Source Code.

http://developer.marklogic.com/products/java

<dependency>
    <groupId>com.marklogic</groupId>
    <artifactId>marklogic-client-api</artifactId>
    <version>4.0.3</version>
</dependency>

<repository>
    <id>jcenter</id>
    <url>http://jcenter.bintray.com</url>
</repository>

compile group: 'com.marklogic', 
name: 'marklogic-client-api', 
version: '4.0.3'

jcenter()

Choose a REST API Instance

The Java API implementation interacts with MarkLogic Server using the MarkLogic REST Client API. Therefore you must have access to a REST API instance in MarkLogic Server before you can run an application that uses the Java Client API.

A REST API instance includes a specially configured HTTP App Server capable of handling REST Client API requests, a content database, and a modules database. MarkLogic Server comes with a suitable REST API instance attached to the Documents database, listening on port 8000.

The examples in this guide assume you're using the pre-configured REST API instance on port 8000 of localhost. If you want to create and use a different REST instance, see , see Administering REST Client API Instances in the REST Application Developer's Guide.

Each application must use a separate modules database and REST API instance.

Create Users

You might need to create MarkLogic Server users with appropriate security roles, or give additional privileges to existing users.

Any user who reads data will need at least the rest-reader role and any user that writes data will need at least the rest-writer role.

REST instance configuration operations, such as setting instance properties require the rest-admin role. For details, see REST Server Configuration.

Some operations require additional privileges. For example, a DatabaseClient that connects to a database other than the default database associated with the REST instance must have the http://marklogic.com/xdmp/privileges/xdmp-eval-in privilege. Using the ServerEvaluationCall interface also requires special privileges; for details, see Evaluating an Ad-Hoc Query or Server-Side Module.

Note that MarkLogic Server Administration is not exposed in Java, so operations such as creating indices, creating users, creating databases, etc. must be done via the Admin Interface, REST Management API, or other MarkLogic Server administration tool. The server configuration component of the Java API is restricted to configuration operations on the REST instance.

For details, see Security Requirements in the REST Application Developer's Guide.

Explore the Examples

The Java Client API distribution includes several examples in the examples/ directory. The examples include the following packages:

• com.marklogic.client.example.cookbook: A collection of small examples of using the core features of the API, such as document operations and search. Most of the example code in this guide is drawn from the Cookbook examples.
• com.marklogic.client.example.handle: Examples of using handles based on open source document models, such as JDOM or Jackson. Examples of handle extensions that read or write database documents in a new way.
• com.marklogic.client.example.extension: A collection of extension classes and examples for manipulating documents in batches.

For instructions on building and running the examples, see the project wiki on GitHub:

http://github.com/marklogic/java-client-api/wiki/Running-the-Examples

The Role of a Database Client

A DatabaseClient object encapsulates the information needed to connect to MarkLogic, such as the host and port of a REST API instance, the database to operate on, and the authentication context. Internally, each DatabaseClient object is associated with a connection pool, as described in Connection Management and Configuration.

Most tasks you perform using the Java Client API are handled by a manager object. For example, you use a QueryManager to search the database and a DocumentManager to read, update, and delete documents. You create manager objects using factory methods on DatabaseClient, such as newQueryManager and newDocumentManager.

Expected Database Client Lifetime

Best practice is to maintain a single, shared reference to a DatabaseClient object for the lifetime of your application's interaction MarkLogic, rather than frequently creating and destroying client objects.

You need multiple DatabaseClient objects if you need to connect to multiple databases or to connect to MarkLogic as multiple users. You must create a different DatabaseClient instance for each combination of (host, port, database, authentication context). Again, it is best to keep these instances around throughout their potential useful lifetime, rather than repeatedly recreating them.

You can one DatabaseClient object across multiple threads. After initial configuration, a DatabaseClient object is thread safe.

Connection Management and Configuration

Internally, the Java Client API maintains an OkHttpClient connection pool that is shared by all DatabaseClient objects. The connection pool efficiently re-uses connections whether you use a single DatabaseClient instance throughout the lifetime of your application or create and discard DatabaseClient objects on demand.

Whenever a DatabaseClient object makes a request to MarkLogic, an available connection is drawn from the connection pool. New connections are created on demand, as needed.

A DatabaseClient object returns its connection to the pool once it receives and processes the HTTP request on whose behalf it claimed the connection. A connection in the pool persists until it is explicitly released or times out due to idleness. The default maximum idle time is 5 minutes.

No state information is maintained with a connection. All cookies are discarded unless a multi-statement (multi-request) transaction is in use. The cookies associated with a multi-statement transaction are cached on the transaction object rather than with the connection.

You can adjust the connection pool configuration by implementing OkHttpClientConfigurator and calling its configure method. However, such adjustments depend on Java Client API internals and will be ignored if a future version of the API uses a different HTTP client implementation.

Creating a Database Client

To create a database client, use the com.marklogic.client.DatabaseClientFactory.newClient() method. For example, the following client connects to the default content database associated with the REST instance on port 8000 of localhost using digest authentication.

DatabaseClient client = 
  DatabaseClientFactory.newClient(
    "localhost", 8000,
    new DatabaseClientFactory.DigestAuthContext("myuser", "mypassword"));

You can also create clients that connect to a specific content database. For example, the following client also connects to the REST instance on port 8000 of localhost, but all operations are performed against the database MyDatabase:

DatabaseClient client = 
  DatabaseClientFactory.newClient(
    "localhost", 8000, "MyDatabase",
    new DatabaseClientFactory.DigestAuthContext("myuser", "mypassword"));

To use a database other than the default database associated with the REST instance requires a user with the following privilege or the equivalent role: http://marklogic.com/xdmp/privileges/xdmp-eval-in.

The host and port values must be those of a REST API instance. When you install MarkLogic, a REST API instance associated with the Documents database is pre-configured for port 8000. You can also create your own instance.

The authentication context object should match the configuration of the REST API instance. For more details, see Authentication and Connection Security.

Connecting Through a Load Balancer

When your application connects to MarkLogic through a load balancer, you should follow these guidelines:

• Configure your DatabaseClient objects to make a GATEWAY type connection. This tells the Java Client API that direct connections to hosts in your MarkLogic cluster are not available.
• Configure your load balancer and MarkLogic cluster timeouts to be consistent with each other. Unavailable hosts should be invalidated by the load balancer only after the MarkLogic host timeout. Cookies should expire only after the MarkLogic session timeout.

For most Java Client API operations, the connection type is transparent. However, features such as the Data Movement SDK need to know whether or not all traffic must go through a gateway host.

The default connection type for a DatabaseClient is DIRECT, meaning that the Java Client API can make direct connections to hosts in your MarkLogic cluster if necessary.

To configure a DatabaseClient for a gateway connection, pass a DatabaseClient.ConnectionType value of GATEWAY as the last paramter to DatabaseClientFactory.newClient. For example:

DatabaseClient client = 
  DatabaseClientFactory.newClient(
    "localhost", 8000, "MyDatabase",
    new DatabaseClientFactory.DigestAuthContext("myuser", "mypassword"),
    DatabaseClient.ConnectionType.GATEWAY);

For additional, context-specific load balancer guidelines, see the following topics:

• Multi-statement transactions: Transaction Management When Using a Load Balancer.
• Asynchronous batch-oriented document operations: Working with a Load Balancer.

Releasing a Database Client

When you no longer need a client and want to release connection resources, use the DatabaseClient object's release() method.

client.release();

DatabaseClient objects efficiently manage connection resources and are expected to be long lived. You do not need to release and re-create client objects just because your application might not require a connection for an extended time. For more details, see Expected Database Client Lifetime and Connection Management and Configuration.

Creating a SecurityContext Object

One of the inputs to DatabaseClientFactory.newClient is a SecurityContext object. This object tells the API what credentials to use to authenticate with MarkLogic. You can select from authentication methods such as Kerberos, digest, and basic.

For example, the database client created by the following statement uses digest authentication. The username and password are those of a user configured into MarkLogic.

import com.marklogic.client.DatabaseClientFactor.DigestAuthContext;
...
DatabaseClient client = DatabaseClientFactory.newClient(
    "localhost", 8000, new DigestAuthContext(username, password));

The authentication context object should match the configuration of the REST API instance. Kerberos based authentication is most secure. Basic authentication sends the password in obfuscated, but not encrypted, mode. Digest authentication encrypts passwords sent over the network.

You can connect to MarkLogic using SSL by attaching SSL configuration information to the security context. For details, see Connecting to MarkLogic with SSL.

For more information about user authentication, see Authenticating Users in the Security Guide.

Using Kerberos Authentication

Use the following steps to configure your MarkLogic installation and client application environment for Kerberos authentication:

• Configuring MarkLogic to Use Kerberos
• Configuring Your Client Host for Kerberos
• Creating a Database Client that Uses Kerberos

Your client host must be running Linux in order to use Kerberos with the Java Client API.

import com.marklogic.client.DatabaseClientFactory.KerberosAuthContext;
...
DatabaseClient client = DatabaseClientFactory.newClient(
    "localhost", 8000, new KerberosAuthContext());

https://github.com/marklogic/java-client-api/blob/master/marklogic-client-api/src/main/java/com/marklogic/client/example/cookbook/KerberosSSLClientCreator.java

Connecting to MarkLogic with SSL

You can use the security context to specify whether or not to use a secure SSL connection to communicate with MarkLogic. The App Server you connect to must also be configured to accept SSL connections. By default, the Java Client API does not use SSL.

For example, the database client created by the following statement uses digest authentication and an SSL connection:

import com.marklogic.client.DatabaseClientFactor.DigestAuthContext;
import javax.net.ssl.SSLContext;
...
SSLContext scontext = SSLContext.getInstance(...).init(...);
DatabaseClient client = DatabaseClientFactory.newClient(
    "localhost", 8000, 
    new DigestAuthContext(username, password).withSSLContext(scontext);

The SSLContext object represents a secure socket protocol implementation which acts as a factory for secure socket factories. For more information about creating and working with SSLContext objects, see Accessing SSL-Enabled XDBC App Servers in the XCC Developer's Guide.

For even more security, you can also include a DatabaseClientFactory.SSLHostnameVerifier object to check if a hostname is acceptable.

For a working example, see the project on GitHub:

https://github.com/marklogic/java-client-api/blob/master/marklogic-client-api/src/main/java/com/marklogic/client/example/cookbook/SSLClientCreator.java

For more information about secure communication with MarkLogic, see the Security Guide.

Using SAML Authentication

Your client application is responsible for acquiring a SAML assertions token from the SAML Identity Provider (IDP). You can then use the SAML assertions token to make requests to the MarkLogic App Server with the MarkLogic Client Java API. That division of responsibility makes it possible for your application to adapt to a wide variety of possible SAML scenarios and IDPs.

After configuring the MarkLogic App Server to authenticate with the SAML IDP, specify a SAMLAuthContext as the SecurityContext when calling DatabaseClientFactory to create a new DatabaseClient.

You can construct a SAMLAuthContext in any of three ways, depending on your approach to authorization:

• If you plan to finish using the DatabaseClient before the SAML assertions token expires, you can call the SAMLAuthContext constructor with the SAML assertions token.
• If you need to extend the expiration of the SAML assertions token before you finish using the DatabaseClient, you can call the SAMLAuthContext constructor with an ExpiringSAMLAuth object and a callback that renews the SAML assertions token with the SAML IDP.

  The ExpiringSAMLAuth object provides getters for the SAML assertions token and the expiration timestamp. Your client application can construct an ExpiringSAMLAuth object by calling the SAMLAuthContext.newExpiringSAMLAuth factory method.

  The renewer callback conforms to the SAMLAuthContext.RenewerCallback functional interface by taking the initial ExpiringSAMLAuth object as input and returning an Instant with the new expiration timestamp for the renewed SAML assertions token, as in the following example:

  class MyClass {
     Instant renewer(ExpiringSAMLAuth authorization) {
       .... call to IDP ....
     }
  }

• If you need to get a new SAML assertions token before you finish using the DatabaseClient, you can call the SAMLAuthContext constructor with a callback that authorizes with the SAML IDP by getting a new SAML assertions token.

  The authorizer callback conforms to the SAMLAuthContext.AuthorizerCallback functional interface that takes an ExpiringSAMLAuth object as input and returns an ExpiringSAMLAuth object with the new SAML assertions token and an expiration timestamp, as shown in the following example:

  class MyClass {
     ExpiringSAMLAuth authorizer(ExpiringSAMLAuth previous) {
       .... call to IDP ....
     }
  }

  On the first call, the ExpiringSAMLAuth parameter is null because no existing authorization exists. Your callback can construct an ExpiringSAMLAuth object by calling the SAMLAuthContext.newExpiringSAMLAuth factory method.

Tradeoffs to consider when choosing whether to renew or reauthorize include the following:

• The renewer callback executes in a background thread, allowing continued requests to the MarkLogic appserver while renewing the SAML assertions token, improving utilization and performance. Extending the life of the SAML assertions token, however, could increase vulnerability in less secure environments.
• The authorizer callback blocks requests to the MarkLogic appserver while getting a new SAML assertions token, reducing utilization and performance but maintaining the highest level of security.

You can reduce the expiration time to allow for network latency and the IDP response generation. Renewer and authorizer callbacks are called in advance of the stated expiration time to reduce the possibility that the SAML assertions token expires as a request is sent to the MarkLogic appserver.

If you need to maintain state between calls to a renewer or authorizer callback, you can implement the ExpiringSAMLAuth interface with your own class instead of calling the SAMLAuthContext.newExpiringSAMLAuth factory method to construct a default instance.

Apart from the specifics of acquiring the SAML assertions token, the use of a DatabaseClient remains the same:

• Multiple threads can use the same DatabaseClient object.
• DatabaseClient objects can be created with different authorizations (including different SAML assertions tokens).

Handle Overview

Content handles are key to working with the Java Client API. Handles make use of the Adapter design pattern to enable strongly typed reading and writing of a diverse and extensible set of content formats. For example, you can create a com.marklogic.client.io.DOMHandle to read or write XML DOM data.

// reading
XMLDocumentManager docMgr = client.newXMLDocumentManager();
Document doc = docMgr.read(docURI, new DOMHandle()).get();

// writing
docMgr.write(docURI, new DOMHandle(someDocument));

The Java Client API pre-defines many handle implementations. The following packages contain handle classes:

• com.marklogic.client.io - Handles classes for standard representations such as String, File, and DOM.
• com.marklogic.extra - Handle classes for 3rd party formats such as DOM4J and GSON. Using these handle classes requires 3rd party libraries that are not included in the Java Client API distribution.

Some handles support both read and write operations. For example, you can use a FileHandle for reading and writing files. Some handles have a special purpose. For example, you use SearchHandle for processing the results of a search operation. For a complete list of handles and what they do, see the com.marklogic.client.io package in the Java Client API Documentation.

Handles are not thread safe. Whenever you create a new thread, you will have to also create new handle objects to use while in that thread.

Some Java Client API methods enable you to use I/O short cuts that do not require explicit creation of a handle. These shortcut methods always have an As suffix, such as readAs. For example, the XMLDocumentManager.read method shown above has an XMLDocumentManager.readAs counterpart that implicitly creates the handle for you. For example:

// reading
Document doc = docMgr.readAs(docURI, Document.class);

// writing
docMgr.writeAs(docURI, someDocument);

These shortcut methods are not more efficient, but they can improve the readability of your code. For more details, see Shortcut Methods as an Alternative to Creating Handles.

Specifying Content Format

Some handles can be used with multiple document formats. For example, an InputStream can provide content in any format, so InputStreamHandle can be used for any document format. Where content format is not explicit in the handle type, use the handle's setFormat method to specify it. For example, the following call tells the Java Client API that the handle can be used with JSON content:

new InputStreamHandle().setFormat(Format.JSON);

You cannot set a format for all handle types. For example, a DOMHandle can only be used for reading and writing XML, so you cannot specify a format.

Handle Type Quick Reference

Not all handles support all content types. In addition, though most handles can be used for either reading or writing, some are more limited. This section provides a quick guide to the content formats, operations, and data types supported by each handle class. Special purpose handle classes, such as SearchHandle, are not included.

BytesHandle

DocumentMetadataHandle

DOMHandle

FileHandle

InputSourceHandle

InputStreamHandle

JacksonHandle

JacksonDatabindHandle

JacksonParserHandle

JAXBHandle

OutputStreamHandle

ReaderHandle

SourceHandle

StringHandle

XMLEventReaderHandle

XMLStreamReaderHandle

Handle Example

The following code uses a DOMHandle to read an XML document from the server into an in-memory DOM object:

XMLDocumentManager docMgr = client.newXMLDocumentManager();
DOMHandle handle = new DOMHandle();
docMgr.read(docURI, handle);
org.w3c.dom.Document document = handle.get();

The following code uses a DOMHandle to write an XML document to MarkLogic. Assume document is some previously initialized in-memory XML DOM document.

XMLDocumentManager docMgr = client.newXMLDocumentManager();
DOMHandle handle = new DOMHandle();
handle.set(document);
docMgr.write(docId, handle);

For additional examples, see the examples in the following packages. The source is available on GitHub. For details, see Downloading the Library Source Code.

• com.marklogic.client.example.cookbook
• com.marklogic.client.example.handle

Understanding Shortcut Methods

Many Java Client API classes and interfaces include shortcut methods of the form operationAs, such as readAs or writeAs. These methods enable you to bypass the equivalent, more strongly typed methods that require you to pass in a handle. Using shortcut methods instead of handles can make your code more readable.

For example, the XMLDocumentManager interface includes both read and readAs methods such as the following:

// strongly typed, handle based
read(String docId, T contentHanlde)

// shortcut equivalent
readAs(String docId, Class<T> as)

This means you can read a document from the database using a call of either of the following forms:

// strongly typed, returns the populated DOMHandle object
DOMHandle handle = docMgr.read(docURI, new DOMHandle());

// shortcut, returns a DOM Document
Document doc = docMgr.read(docURI, Document.class);

Similarly, you can use XMLDocumentManager to write a document to the database using either of the following calls:

// strongly typed
docMgr.write(docURI, new DOMHandle(theDocument));

// shortcut
docMgr.writeAs(docURI, theDocument);

Shortcut methods are not limited to document read and write operations. For example, you can use either QueryManager.newRawCombinedQueryDefinition or QueryManager.newRawCombinedQueryDefinitionAs to create a RawCombinedQueryDefinition.

When to Choose Strongly Typed Over Shortcut

Shortcut methods are the best choice in most cases because they improve the readability and maintainability of your code. However, you should keep the following points in mind:

• A shortcut method is not more efficient than the equivalent strongly typed method. Internally, a Handle is still created to manage the data.
• Using a shortcut method introduces a small risk because you're side-stepping the strong typing provided by a handle. For example, an exception is thrown if there is no handle type corresponding to class type you provide to the shortcut method.

The typing exposure is limited since the Java Client API pre-defines Handle classes for a broad range of types. You can register your own class-to-handle pairings to extend the supported classes. For details, see Extending Shortcuts by Registering Handle Factories.

Consider the strongly typed call form in the following cases:

• You want compile-time checking of input and output types.
• You want a slight increase in efficiency over a large number of requests.
• You need to control the MIME type or format of a handle.

Extending Shortcuts by Registering Handle Factories

Though you do not have to create a handle when using a shortcut method, the shortcut implementation still creates a Handle to manage the data.

For example, when you issue a shortcut call such as the following, the implementation creates a DOMHandle to receive the document read from the database.

docMgr.read(docURI, Document.class);

This means that a shortcut method must be able to create a handle capable of handling the targeted class type. This capability is provided by a registry for handle factories. The shortcut method can query the registry for a handle factory that can process a particular class type. For details, see DatabaseClientFactory.HandleFactoryRegistry in the Java Client API Documentation.

The Java Client API automatically registers factories for the following handle classes. For details on the data types supported by each handle type, see the handle class documentation in the Java Client API Documentation.

If you create your own handle class, you can register a handle factory for it so that you can use shortcut methods on the classes supported by your handle class.

Handle factory registration must be completed before you create a DatabaseClient.

You can use the same mechansim with a JAXBHandle factory to enable shortcut methods on POJOs. For example, if you have a POJO class named Product, then you can add it to the registry as follows:

DatabaseClientFactory.getHandleRegistry().register(
    JAXBHandle.newFactory(Product.class);

Then you can subsequently write Product POJOs to MarkLogic and read them back as follows:

XMLDocumentManager docMgr = client.newXMLDocumentManager();
Product product = // ...create a Product

docMgr.writeAs(docURI, product);
// ...
product = docMgr.readAs(docURI, Product.class);

Note that the Java Client API also includes a POJO data binding capability as an alternative to managing your own POJOs with JAXB. Using this feature eliminates the need for the above registration. For more details, see POJO Data Binding Interface.