Loading TOC...
Java Application Developer's Guide (PDF)

Java Application Developer's Guide — Chapter 15

Extending the Java API

You can extend the Java API in a variety of ways, including resource service extensions and evaluation of ad-hoc queries and server-side modules. This chapter covers the following topics:

Available Extension Points

The Java API offers several ways to extend and customize the capabilities using user-defined code that is either pre-installed on MarkLogic Server or supplied at request time. The following extension points are available:

  • Content transformations: A user-defined transform function can be applied when documents are written to the database or read from the database; for details, see Content Transformations. You can also define custom replacement content generators for the patch feature; for details, see Construct Replacement Data on the Server.
  • Search result customization: Customization opportunities include constraint parsers for string queries, search result snippet generation, and search result customization. For details, see Searching and the Search Developer's Guide.
  • Resource service extensions: Define your own REST endpoints, accessible from Java using the ResourceExtensionsManager interface. Resource service extensions are covered in detail in this chapter. To get started, see Introduction to Resource Service Extensions.
  • Ad-hoc query execution: Send an arbitrary block of XQuery or JavaScript code to MarkLogic Server for evaluation. For details, see Evaluating an Ad-Hoc Query or Server-Side Module.
  • Server-side module evaluation: Evaluate user-defined XQuery or JavaScript modules after installing them on MarkLogic Server. For details, see Evaluating an Ad-Hoc Query or Server-Side Module.

Introduction to Resource Service Extensions

Resource service extensions extend the MarkLogic Java API by making XQuery and server-side JavaScript modules available for use from Java. A resource extension implements services for a server-side resource. For example, you can create a dictionary program resource extension that looks up words, checks spelling, and makes suggestions for not found words. The individual operations an application programmer may call, for example, lookUpWords(), spellCheck(), and so on, are the services that make up the resource extension.

The following are the basic steps to create and use a resource extension using the Java API:

  1. Create an XQuery or JavaScript module that implements the services for the resource.
  2. Install the resource service extension implementation in the modules database associated with the REST API instance using com.marklogic.client.admin.ResourceExtensionsManager.
  3. Make your resource extension available to Java applications by creating a wrapper class that is a subclass of com.marklogic.client.extensions.ResourceManager. Inside this class, access the resource extension methods using a com.marklogic.client.extensions.ResourceServices object obtained through the ResourceManager.getServices() method.
  4. Use the methods of your ResourceManager subclass to access the services provided by the extension from the rest of your application.

The key classes for resource extentions in the Java API are:

  • ResourceExtensionsManager, which manages creation, modification, and deletion of resource service service extension implementations on the REST Server. You must connect to MarkLogic as a user with the rest-admin role to create and work with ResourceExtensionsManager.
  • ResourceManager, the base class for classes that you write to provide client interfaces to resource services.
  • ResourceServices, which supports calling the services for a resource. The resource services extension implementation must already be installed on the server via the ResourceExtensionsManager before ResourceServices can access it.

These objects are created via a ServerConfigManager, since resource services are associated with the server, not the database.

For a complete example of implementing and using a resource service extension, see com.marklogic.client.example.cookbook.ResourceExtension in the example/ directory of your Java API installation.

Creating a Resource Extension

You can implement a resource service Extension using server-side JavaScript or XQuery. The interface is shared across multiple MarkLogic client APIs, so you can use the same extensions with the Java Client API, Node.js Client API, and the REST Client API. For the interface definition, authoring guidelines, and example implementations, see Extending the REST API in the REST Application Developer's Guide.

Installing Resource Extensions

Before you can use a resource extension, you must install the implementation on MarkLogic Server as follows:

  1. If your resource extension depends on additional library modules, install these dependent libraries on MarkLogic Server. For details, see Managing Dependent Libraries and Other Assets.
  2. If you have not already done so, create a DatabaseClient for connecting to the database. For example, if using digest authentication:
    DatabaseClient client = DatabaseClientFactory.newClient(
        host, port, new DigestAuthContext(username, password));
  3. If you have not already done so, create a ResourceExtensionsManager using ServerConfigManager.
    ResourceExtensionsManager resourceMgr =
        client.newServerConfigManager().newResourceExtensionsManager();
  4. Create a com.marklogic.client.admin.ExtensionMetadata object to hold the implementation language of your extension.
    ExtensionMetadata metadata = new ExtensionMetadata();
    metadata.setScriptLanguage(ExtensionMetadata.JAVASCRIPT);
  5. Optionally, populate the ExtensionMetadataObject with your resource extension's metadata. You can set title, description, provider name, version, and expected parameters. For example:
    metadata.setTitle("Spelling Dictionary Resource Services");
    metadata.setDescription("This plugin supports spelling dictionaries");
    metadata.setProvider("MarkLogic");
    metadata.setVersion("0.1");
  6. Optionally, define one or more objects containing method interface metadata using com.marklogic.client.admin.ResourceExtensionsManager.MethodParameters. The following example creates metadata for a GET method expecting one string parameter:
    MethodParameters getParams = new MethodParameters(MethodType.GET);
    getParams.add("my-uri", "xs:string?");
  7. Create a handle (such as an input stream and a handle associated with it) to the extension's source code. For example:
    FileInputStream myStream = new FileInputStream("sourcefile.sjs");
    InputStreamHandle handle = new InputStreamHandle(myStream);
    handle.set(myStream);
  8. Install the extension by calling the ResourceExtensionManager.writeServices() method, supplying the extension name, the handle to the implementation, and any metadata objects. For example:
    resourceMgr.writeServices(DictionaryManager.NAME,handle,metadata,getParams);
  9. Release the client if you no longer need the database connection.
    client.release();

The following code sample demonstrates the above steps. For a complete example, see com.marklogic.client.example.cookbook.ResourceExtension in the example/ directory of your Java API distribution.

// create a manager for resource extensions
ResourceExtensionsManager resourceMgr =
  client.newServerConfigManager().newResourceExtensionsManager();

// specify metadata about the resource extension
ExtensionMetadata metadata = new ExtensionMetadata();
metadata.setScriptLanguage(ExtensionMetadata.XQUERY);
metadata.setTitle("Spelling Dictionary Resource Services");
metadata.setDescription("This plugin supports spelling dictionaries");
metadata.setProvider("MarkLogic");
metadata.setVersion("0.1");

// specify metadata about method interfaces
MethodParameters getParams = new MethodParameters(MethodType.GET);
getParams.add("my-uri", "xs:string?");

// acquire the resource extension source code
InputStream sourceStream = new FileInputStream("dictionary.xqy");

// create a handle on the extension source code
InputStreamHandle handle = new InputStreamHandle();
handle.set(sourceStream);

// write the resource extension to the database
resourceMgr.writeServices(DictionaryManager.NAME, handle,
                          metadata, getParams);

Deleting Resource Extensions

To delete a resource extension, call the deleteServices() method of com.marklogic.client.admin.ResourceExtensionManager. For example, assuming you have already obtained a ResourceExtensionsManager object, do the following:

resourceMgr.deleteServices(resourceName);

Listing Resource Extensions

To list all the installed extensions, use a handle as in the following example, which gets the extensions list in XML format:

String result = resourceMgr.listServices(
   new StringHandle().withFormat(Format.XML)).get();

By default, calling listServices() rebuilds the extension metadata to ensure the metadata is up to date. If you find this refresh makes discovery take too long, you can disable the refresh by setting the refresh parameter to false:

String result = resourceMgr.listServices(
   new StringHandle().withFormat(Format.XML), false).get();

Disabling the refresh can result in this request returning inaccurate information, but it does not affect the freshness or availability of the implementation of any extensions.

Using Resource Extensions

After you install the extension as described in Installing Resource Extensions, create a wrapper class that exposes the functionality of the extension to your application. The wrapper class should be a subclass of com.marklogic.client.extensions.ResourceManager. In the implementation of your wrapper class, use com.marklogic.client.extensions.ResourceServices to invoke the GET, PUT, POST and/or DELETE methods of the resource extension.

Use these guidelines in implementing your wrapper subclass:

  1. Before using any services, initialize your ResourceManager subclass by passing it to com.marklogic.client.DatabaseClient.init(). For example:
    public class DictionaryManager extends ResourceManager {
        static final public String NAME = "dictionary";
        ...
    
        public DictionaryManager(DatabaseClient client) {
             super();
     
            // Initialize the Resource Manager via the Database Client
           client.init(NAME, this);
        }
        ...
    }
  2. To pass parameters to a resource extension method, create a com.marklogic.client.util.RequestParameters object and add parameters to it. Each parameter is represented by a parameter name and value. Use the parameter names defined by the resource extension. For example:
    //Build up the set of parameters for the service call
    RequestParameters params = new RequestParameters();
    params.add("service", "dictionary");
    params.add("uris", uris);
  3. Obtain a com.marklogic.com.extensions.ResourceServices object through the inherited protected method getServices(). For example:
    public class DictionaryManager extends ResourceManager {
        ...
        public Document[] checkDictionaries(String. . . uris) {
    
            ...
            // get the initialized service object from the base class
            ResourceServices services = getServices();
            ...
        }
    }
  4. Use the get(), put(), post(), and delete() methods of ResourceServices to invoke methods of the resource extension on the server. For example:
    ResourceServices services = getServices();
    ServiceResultIterator resultItr = services.get(params, mimetypes);ServiceResultIterator resultItr = services.post(params, mimetypes);ServiceResultIterator resultItr = services.put(params, mimetypes);ServiceResultIterator resultItr = services.delete(params, mimetypes);

The results from calling a resource extension method are returned as either a com.marklogic.client.extensions.ResourceServices.ServiceResultIterator or a handle on the appropriate content type. Use a ServiceResultIterator when a method can return multiple items; use a handle when it returns only one. Resources associated with the results are not released until the associated handle is discarded or the iterator is closed or discarded.

The following code combines all the guidelines together in a sample application that exposes dictionary operations. For the complete example, see the Cookbook example com.marklogic.client.example.cookbook.ResourceExtension in the example/ directory of your Java API distribution.

public class DictionaryManager extends ResourceManager {
    static final public String NAME = "dictionary";
    private XMLDocumentManager docMgr;

    public DictionaryManager(DatabaseClient client) {
         super();
 
        // Initialize the Resource Manager via the Database Client
       client.init(NAME, this);
   }

    // Our first Java implementation of a specific service from 
    // the extension
    public Document[] checkDictionaries(String. . . uris) {
        //Build up the set of parameters for the service call
        RequestParameters params = new RequestParameters();
        // Add the dictionary service parameter
        params.add("service", "dictionary");
        params.add("uris", uris);
        String[] mimetypes = new String[uris.length];
        for (int i=0; i < uris.length; i++) {
            mimetypes[i] = "application/xml";
        }

        // get the initialized service object from the base class
        ResourceServices services = getServices();
        // call the service implementation on the REST Server, 
        // returning a ResourceServices object
        ServiceResultIterator resultItr = 
            services.get(params, mimetypes);
        //iterate over results, get content
        ...
        // release resources
        resultItr.close();
    }
    ...
}

Managing Dependent Libraries and Other Assets

This section covers installation and maintenance of XQuery libraries and other server-side assets used by your application. This includes dependent libraries needed by resource extensions and transformations, and replacement content generation functions usable for partially updates to documents and metadata.

The following topics are covered:

You can also manage assets using the MarkLogic REST API. For details, see Managing Dependent Libraries and Other Assets in the REST Application Developer's Guide.

Maintenance of Dependent Libraries and Other Assets

When you install or update a dependent library module or other asset as described in this section, the asset is replicated across your cluster automatically. There can be a delay of up to one minute between updating and availability.

MarkLogic Server does not automatically remove dependent assets when you delete the related extension or transform.

Since dependent assets are installed in the modules database, they are removed when you remove the REST API instance if you include the modules database in the instance teardown.

If you installed assets in a REST API instance using MarkLogic 6, they cannot be managed using the /ext service unless you re-install them using /ext. Reinstalling the assets may require additional changes because the asset URIs will change. If you choose not to migrate such assets, continue to maintain them according to the documentation for MarkLogic 6.

Installing or Updating Assets

Follow this procedure to install or update a library module or other asset in the modules database associated with your REST Server. If the REST Server is part of a cluster, the asset is automatically propagated throughout the cluster.

The modules database path under which you install an asset must begin with /ext/.

  1. If you have not already done so, connect to the database, storing the connection in a com.marklogic.client.DatabaseClient object. For example, if using digest authentication:
    DatabaseClient client = DatabaseClientFactory.newClient(
        host, port, new DigestAuthContext(username, password));
  2. If you have not already done so, create a com.marklogic.client.admin.ExtensionLibrariesManager. Note that the method for doing so is associated with a ServerConfigManager.
    ExtensionLibrariesManager libMgr =
        client.newServerConfigManager().newExtensionLibrariesManager();
  3. Associate a handle with the asset. The following example associates a FileHandle with the text file containing an XQuery module.
    FileHandle handle = 
        new FileHandle(new File("module.xqy")).withFormat(Format.TEXT));
  4. Install the module in the modules database by calling ExtensionLibrariesManager.write(). For example:
    libMgr.write("/ext/my/path/to/my/module.xqy", handle);

You can also specify asset-specific permissions by passing an ExtensionLibraryDescriptor instead of a simple path string to ExtensionLibrariesManager.write(). The following example uses an descriptor:

ExtensionLibraryDescriptor desc = new ExtensionLibraryDescriptor();
desc.setPath("/ext/my/path/to/my/module.xqy");
desc.addPermission("my-role", "my-capability");
...
libMgr.write(desc, handle);

To use a dependent library installed with /ext in your extension or transform module, use the same URI under which you installed the dependent library. For example, if a dependent library is installed with the URI /ext/my/domain/my-lib.xqy, then the extension module using this library should include an import of the form:

import module namespace dep="mylib" at "/ext/my/domain/my-lib.xqy";

Removing an Asset

To remove an asset from the modules database associated with the REST Server, call com.marklogic.client.admin.ExtensionLibrariesManager.delete(). For example:

DatabaseClient client = DatabaseClientFactory.newClient(...);
ExtensionLibrariesManager libMgr =
    client.newServerConfigManager().newExtensionLibrariesManager();

libMgr.delete("/ext/my/path/to/my/module.xqy");

You can also call delete() with a ExtensionLibraryDescriptor.

If the path passed to delete(), whether by String or descriptor, is a database directory path, all assets in the directory are deleted. If the path is a single asset, just that asset is deleted.

Retrieving an Asset List

You can retrieve a list of all the assets installed in the modules database associated with the REST Server by calling com.marklogic.client.admin.ExtensionsLibraryManager.list(). If you call list() with no parameters, you get a list of ExtensionLibraryDescriptor objects for all assets. If you call list() with a path, you get a similar list of descriptors for all assets installed in that database directory.

The following code snippet retrieves descriptors for all installed assets and prints the path of each one to stdout.

DatabaseClient client = DatabaseClientFactory.newClient(...);
ExtensionLibrariesManager libMgr =
    client.newServerConfigManager().newExtensionLibrariesManager();

ExtensionLibraryDescriptor[] descriptors = libMgr.list();
for (ExtensionLibraryDescriptor descriptor : descriptors) {
    System.out.println(descriptor.getPath());
}

Retrieving an Asset

To retrieve the contents of an asset installed in the modules database associated with a REST Server, call com.marklogic.client.admin.LibrariesExtensionManager.read(). You must first create a handle to receive the contents.

The following code snippet reads the contents of an XQuery library module into a string:

DatabaseClient client = DatabaseClientFactory.newClient(...);
ExtensionLibrariesManager libMgr =
    client.newServerConfigManager().newExtensionLibrariesManager();

StringHandle handle =
    libMgr.read("/ext/my/path/to/my/module.xqy", new StringHandle());

Evaluating an Ad-Hoc Query or Server-Side Module

The com.marklogic.client.eval.ServerEvaluationCall enables you to send blocks of JavaScript and XQuery to MarkLogic Server for evaluation or to invoke an XQuery or JavaScript module installed in the modules database. This is equivalent to calling the builtin server functions xdmp:eval or xdmp:invoke (XQuery), or xdmp.eval or xdmp.invoke (JavaScript).

This section covers the following related topics:

Security Requirements

Evaluating an ad-hoc query on MarkLogic Server requires the following privileges or the equivalent:

  • http://marklogic.com/xdmp/privileges/xdmp-eval
  • http://marklogic.com/xdmp/privileges/xdmp-eval-in
  • http://marklogic.com/xdmp/privileges/xdbc-eval
  • http://marklogic.com/xdmp/privileges/xdbc-eval-in

Invoking a module on MarkLogic Server requires the following privileges or the equivalent:

  • http://marklogic.com/xdmp/privileges/xdmp-invoke
  • http://marklogic.com/xdmp/privileges/xdmp-invoke-in
  • http://marklogic.com/xdmp/privileges/xdbc-invoke
  • http://marklogic.com/xdmp/privileges/xdbc-invoke-in

Basic Step for Ad-Hoc Query Evaluation

Follow this procedure to evaluate an Ad-Hoc XQuery or JavaScript query on MarkLogic Server. You must use a user that has the privileges listed in Security Requirements.

  1. If you have not already done so, create a DatabaseClient for connecting to the database. For example, if using digest authentication:
    DatabaseClient client = DatabaseClientFactory.newClient(
        host, port, new DigestAuthContext(username, password));
  2. Create a ServerEvaluationCall object.
    ServerEvaluationCall theCall = client.newServerEval();
  3. Associate your ad-hoc query with the call object. You can specify the query using a String or a TextWriteHandle.
    1. For a JavaScript query, pass in the query text using ServerEvaluationCall.javascript:
      String query = "word1 \" \" + word2";
      theCall.javascript(query);
    2. For an XQuery query, pass in the query text using ServerEvaluationCall.xquery.
      String query = 
        "xquery version '1.0-ml';" +
        "declare variable $word1 as xs:string external;" +
        "declare variable $word2 as xs:string external;" +
        "fn:concat($word1, ' ', $word2)";
      theCall.xquery(query);
  4. If the query expects input variable values, supply them using ServerEvaluationCall.addVariable. For details, see Specifying External Variable Values.
    theCall.addVariable("word1", "hello");
    theCall.addVariable("word2", "world");
  5. Send the query to MarkLogic Server for evaluation by calling ServerEvaluationCall.eval or ServerEvaluationCall.evalAs. For details, see Interpreting the Results of Eval or Invoke.
    String response = theCall.evalAs(String.class);
  6. Release the client if you no longer need the database connection.
    client.release();

The following code puts these steps together into a single block.

DatabaseClient client = DatabaseClientFactory.newClient(
   host, port, new DigestAuthContext(username, password));
ServerEvaluationCall theCall = client.newServerEval();
String query = "word1 \" \" + word2";

String result = theCall.javascript(query)
                  .addVariable("word1", "hello")
                  .addVariable("word2", "world")
                  .evalAs(String.class);

Basic Steps for Module Invocation

You can invoke an arbitrary JavaScript or XQuery module installed in the modules database associated with the REST API instance by setting a module path on a ServerEvaluationCall object and then calling ServerEvaluationCall.eval or ServerEvaluationCall.evalAs. The module path is resolved using the rules described in Rules for Resolving Import, Invoke, and Spawn Paths in the Application Developer's Guide.

You can install your module is using com.marklogic.client.admin.ExtensionLibrariesManager. For details, see Installing or Updating Assets. If you install your module using the ExtensionLibrariesManager interface, your module path will always being with /ext/.

Follow this procedure to invoke an XQuery or JavaScript module pre-installed on MarkLogic Server. You must use a user that has the privileges listed in Security Requirements.

  1. If you have not already done so, create a DatabaseClient for connecting to the database. For example, if using digest authentication:
    DatabaseClient client = DatabaseClientFactory.newClient(
        host, port, new DigestAuthContext(username, password));
  2. Create a ServerEvaluationCall object.
    ServerEvaluationCall invoker = client.newServerEval();
  3. Associate your module with the call object by setting the module path.
    invoker.modulePath("/my/module/path.sjs");
  4. If the query expects input variable values, supply them using ServerEvaluationCall.addVariable. For details, see Specifying External Variable Values.
    invoker.addVariable("word1", "hello");
    invoker.addVariable("word2", "world");
  5. Invoke the module on MarkLogic Server by calling ServerEvaluationCall.eval or ServerEvaluationCall.evalAs. For details, see Interpreting the Results of Eval or Invoke.
    String response = invoker.evalAs(String.class);
  6. Release the client if you no longer need the database connection.
    client.release();

The following code puts these steps together into a single block.

DatabaseClient client = DatabaseClientFactory.newClient(
   host, port, new DigestAuthContext(username, password));
ServerEvaluationCall invoker = client.newServerEval();

String result = invoker.modulePath("/ext/invoke/example.sjs")
                  .addVariable("word1", "hello")
                  .addVariable("word2", "world")
                  .evalAs(String.class);

Specifying External Variable Values

You can pass values to an ad-hoc query or invoked module at runtime using external variables. Specify the variable values using ServerEvaluationCall.addVariable. or ServerEvaluationCall.addVariableAs.

Use addVariable for simple value types, such as String, Number, and Boolean and values with a suitable AbstractWriteHandle, such as DOMHandle for XML. For example:

ServerEvaluationCall theCall = client.newServerEval();
...
theCall.addVariable("aString", "hello")
       .addVariable("aBool", true)
       .addVariable("aNumber", 3.14);

Use addVariableAs for other complex value types such as objects. For example, the following code uses a Jackson object mapper to set an external variable value to a JSON object that can be used as a JavaScript object by the server-side code:

theCall.addVariableAs("anObj", 
    new ObjectMapper().createObjectNode().put("key", "value"))

If you're evaluating or invoking XQuery code, you must declare the variables explicitly in your ad-hoc query or module. For example, the following XQuery prolog declares two external string-valued variables whose values can be supplied at runtime.

xquery version "1.0-ml";
declare variable $word1 as xs:string external;
declare variable $word2 as xs:string external;
...

If your XQuery external variables are in a namespace, use ServerEvaluationCall.addNamespace to associate a prefix with the namespace, and then use the namespace prefix in the variable name passed to ServerEvaluationCall.addVariable. For example, given the following ad-hoc query:

xquery version "1.0-ml";
declare namespace my = "http://example.com";
declare variable $my:who as xs:string external;
fn:concat("hello", " ", $my:who)

Set the variable values as follows:

theCall.addNamespace("my", "http://example.com")
       .addVariable("my:who", "me")
       ...

Interpreting the Results of Eval or Invoke

You can request results in the following ways:

  • If you know the ad-hoc query or invoked module returns a single value of a simple known type, use ServerEvaluationCall.evalAs. For example, if you know an ad-hoc query returns a single String value, you can evaluate it as follows:
    String result = theCall.evalAs(String.class);
  • Pass an AbstractReadHandle to ServerEvaluationCall.eval to process a single result through a handle. For example:
    DOMHandle result = theCall.eval(new DOMHandle());
  • If the query or invoked module can return multiple values or you do not know the return type, use ServerEvaluationCall.eval with no parameters to return an EvalResultIterator. For example:
    EvalResultIterator result = theCall.eval();

When you use an EvalResultIterator, each value is encapsulated in a com.marklogic.client.eval.EvalResult that provides type information and accessors for the value. The EvalResult.format method provides abstract type information, such as text, binary, json, or xml. The EvalResult.getType method provides more detailed type information, when available, such as string, integer, decimal, or date. Detailed type information is not always available.

The table below maps common server-side value types to the values you can expect to their corresponding com.marklogic.client.io.Format (from EvalResult.format) and EvalResult.Type (from EvalResult.getType).

Value Type Format Type
document-node[object-node()]
Format.JSON
Type.JSON
object-node()
Format.JSON
Type.JSON
document-node[array-node()]
Format.JSON
Type.JSON
array-node()
Format.JSON
Type.JSON
map:map
Format.JSON
Type.JSON
json:array
Format.JSON
Type.JSON
document-node[element()]
Format.XML
Type.XML
element()
Format.XML
Type.XML
document-node[binary()]
Format.BINARY
Type.BINARY
binary()
Format.BINARY
Type.BINARY
document-node[text()]
Format.TEXT
Format.TEXT
text()
Format.TEXT
Format.TEXT
any atomic value
Format.TEXT
corresponding type, such as Format.BOOLEAN or Format.INTEGER.
JavaScript string
Format.TEXT
Format.STRING
JavaScript number
Format.TEXT
Format.DECIMAL, a derived type such as FORMAT.INTEGER, or Format.STRING (for infinity)
JavaScript boolean
Format.TEXT
Format.BOOLEAN
« Previous chapter
Next chapter »