You can extend the Java API in a variety of ways, including resource service extensions and evaluation of ad-hoc queries and server-side modules. This chapter covers the following topics:
The Java API offers several ways to extend and customize the capabilities using user-defined code that is either pre-installed on MarkLogic Server or supplied at request time. The following extension points are available:
ResourceExtensionsManager
interface. Resource service extensions are covered in detail in this chapter. To get started, see Introduction to Resource Service Extensions.Resource service extensions extend the MarkLogic Java API by making XQuery and server-side JavaScript modules available for use from Java. A resource extension implements services for a server-side resource. For example, you can create a dictionary program resource extension that looks up words, checks spelling, and makes suggestions for not found words. The individual operations an application programmer may call, for example, lookUpWords()
, spellCheck()
, and so on, are the services that make up the resource extension.
The following are the basic steps to create and use a resource extension using the Java API:
com.marklogic.client.admin.ResourceExtensionsManager
.com.marklogic.client.extensions.ResourceManager
. Inside this class, access the resource extension methods using a com.marklogic.client.extensions.ResourceServices
object obtained through the ResourceManager.getServices()
method.ResourceManager
subclass to access the services provided by the extension from the rest of your application.The key classes for resource extensions in the Java API are:
ResourceExtensionsManager
, which manages creation, modification, and deletion of resource service service extension implementations on the REST Server. You must connect to MarkLogic as a user with the rest-admin
role to create and work with ResourceExtensionsManager
.ResourceManager
, the base class for classes that you write to provide client interfaces to resource services. ResourceServices
, which supports calling the services for a resource. The resource services extension implementation must already be installed on the server via the ResourceExtensionsManager
before ResourceServices
can access it.These objects are created via a ServerConfigManager
, since resource services are associated with the server, not the database.
For a complete example of implementing and using a resource service extension, see com.marklogic.client.example.cookbook.ResourceExtension
in the example/
directory of your Java API installation.
You can implement a resource service Extension using server-side JavaScript or XQuery. The interface is shared across multiple MarkLogic client APIs, so you can use the same extensions with the Java Client API, Node.js Client API, and the REST Client API. For the interface definition, authoring guidelines, and example implementations, see Extending the REST API in the REST Application Developer's Guide.
Resource service extensions, transforms, row mappers and reducers, and other hooks cannot be implemented as JavaScript MJS modules.
Before you can use a resource extension, you must install the implementation on MarkLogic Server as follows:
DatabaseClient
for connecting to the database. For example, if using digest authentication:DatabaseClient client = DatabaseClientFactory.newClient( host, port, new DigestAuthContext(username, password));
ResourceExtensionsManager
using ServerConfigManager
.ResourceExtensionsManager resourceMgr =
client.newServerConfigManager().newResourceExtensionsManager();
com.marklogic.client.admin.ExtensionMetadata
object to hold the implementation language of your extension.ExtensionMetadata metadata = new ExtensionMetadata();
metadata.setScriptLanguage(ExtensionMetadata.JAVASCRIPT);
metadata.setTitle("Spelling Dictionary Resource Services"); metadata.setDescription("This plugin supports spelling dictionaries"); metadata.setProvider("MarkLogic"); metadata.setVersion("0.1");
com.marklogic.client.admin.ResourceExtensionsManager.MethodParameters
. The following example creates metadata for a GET method expecting one string parameter:MethodParameters getParams = new MethodParameters(MethodType.GET); getParams.add("my-uri", "xs:string?");
FileInputStream myStream = new FileInputStream("sourcefile.sjs"); InputStreamHandle handle = new InputStreamHandle(myStream); handle.set(myStream);
ResourceExtensionManager.writeServices()
method, supplying the extension name, the handle to the implementation, and any metadata objects. For example:resourceMgr.writeServices(DictionaryManager.NAME,handle,metadata,getParams);
client.release();
The following code sample demonstrates the above steps. For a complete example, see com.marklogic.client.example.cookbook.ResourceExtension
in the example/
directory of your Java API distribution.
// create a manager for resource extensions
ResourceExtensionsManager resourceMgr =
client.newServerConfigManager().newResourceExtensionsManager();
// specify metadata about the resource extension
ExtensionMetadata metadata = new ExtensionMetadata();
metadata.setScriptLanguage(ExtensionMetadata.XQUERY);
metadata.setTitle("Spelling Dictionary Resource Services");
metadata.setDescription("This plugin supports spelling dictionaries");
metadata.setProvider("MarkLogic");
metadata.setVersion("0.1");
// specify metadata about method interfaces
MethodParameters getParams = new MethodParameters(MethodType.GET);
getParams.add("my-uri", "xs:string?");
// acquire the resource extension source code
InputStream sourceStream = new FileInputStream("dictionary.xqy");
// create a handle on the extension source code
InputStreamHandle handle = new InputStreamHandle();
handle.set(sourceStream);
// write the resource extension to the database
resourceMgr.writeServices(DictionaryManager.NAME, handle,
metadata, getParams);
To delete a resource extension, call the deleteServices()
method of com.marklogic.client.admin.ResourceExtensionManager
. For example, assuming you have already obtained a ResourceExtensionsManager
object, do the following:
resourceMgr.deleteServices(resourceName);
To list all the installed extensions, use a handle as in the following example, which gets the extensions list in XML or JSON format:
String result = resourceMgr.listServices( new StringHandle().withFormat(Format.XML)).get(); //Or String result = resourceMgr.listServices( new StringHandle().withFormat(Format.JSON)).get();
By default, calling listServices()
rebuilds the extension metadata to ensure the metadata is up to date. If you find this refresh makes discovery take too long, you can disable the refresh by setting the refresh
parameter to false:
String result = resourceMgr.listServices( new StringHandle().withFormat(Format.XML), false).get(); //Or String result = resourceMgr.listServices( new StringHandle().withFormat(Format.JSON), false).get();
Disabling the refresh can result in this request returning inaccurate information, but it does not affect the freshness or availability of the implementation of any extensions.
After you install the extension as described in Installing Resource Extensions, create a wrapper class that exposes the functionality of the extension to your application. The wrapper class should be a subclass of com.marklogic.client.extensions.ResourceManager
. In the implementation of your wrapper class, use com.marklogic.client.extensions.ResourceServices
to invoke the GET, PUT, POST and/or DELETE methods of the resource extension.
Use these guidelines in implementing your wrapper subclass:
ResourceManager
subclass by passing it to com.marklogic.client.DatabaseClient.init()
. For example:public class DictionaryManager extends ResourceManager { static final public String NAME = "dictionary"; ... public DictionaryManager(DatabaseClient client) { super(); // Initialize the Resource Manager via the Database Client client.init(NAME, this); } ... }
com.marklogic.client.util.RequestParameters
object and add parameters to it. Each parameter is represented by a parameter name and value. Use the parameter names defined by the resource extension. For example://Build up the set of parameters for the service call RequestParameters params = new RequestParameters(); params.add("service", "dictionary"); params.add("uris", uris);
com.marklogic.com.extensions.ResourceServices
object through the inherited protected method getServices()
. For example:public class DictionaryManager extends ResourceManager { ... public Document[] checkDictionaries(String. . . uris) { ... // get the initialized service object from the base class ResourceServices services = getServices(); ... } }
get()
, put()
, post()
, and delete()
methods of ResourceServices
to invoke methods of the resource extension on the server. For example:ResourceServices services = getServices(); ServiceResultIterator resultItr = services.get(params, mimetypes); ServiceResultIterator resultItr = services.post(params, mimetypes); ServiceResultIterator resultItr = services.put(params, mimetypes); ServiceResultIterator resultItr = services.delete(params, mimetypes);
The results from calling a resource extension method are returned as either a com.marklogic.client.extensions.ResourceServices.ServiceResultIterator
or a handle on the appropriate content type. Use a ServiceResultIterator
when a method can return multiple items; use a handle when it returns only one. Resources associated with the results are not released until the associated handle is discarded or the iterator is closed or discarded.
The following code combines all the guidelines together in a sample application that exposes dictionary operations. For the complete example, see the Cookbook example com.marklogic.client.example.cookbook.ResourceExtension
in the example/
directory of your Java API distribution.
public class DictionaryManager extends ResourceManager { static final public String NAME = "dictionary"; private XMLDocumentManager docMgr; public DictionaryManager(DatabaseClient client) { super(); // Initialize the Resource Manager via the Database Client client.init(NAME, this); } // Our first Java implementation of a specific service from // the extension public Document[] checkDictionaries(String. . . uris) { //Build up the set of parameters for the service call RequestParameters params = new RequestParameters(); // Add the dictionary service parameter params.add("service", "dictionary"); params.add("uris", uris); String[] mimetypes = new String[uris.length]; for (int i=0; i < uris.length; i++) { mimetypes[i] = "application/xml"; } // get the initialized service object from the base class ResourceServices services = getServices(); // call the service implementation on the REST Server, // returning a ResourceServices object ServiceResultIterator resultItr = services.get(params, mimetypes); //iterate over results, get content ... // release resources resultItr.close(); } ... }
This section covers installation and maintenance of XQuery libraries and other server-side assets used by your application. This includes dependent libraries needed by resource extensions and transformations, and replacement content generation functions usable for partially updates to documents and metadata.
The following topics are covered:
You can also manage assets using the MarkLogic REST API. For details, see Managing Dependent Libraries and Other Assets in the REST Application Developer's Guide.
When you install or update a dependent library module or other asset as described in this section, the asset is replicated across your cluster automatically. There can be a delay of up to one minute between updating and availability.
MarkLogic Server does not automatically remove dependent assets when you delete the related extension or transform.
Since dependent assets are installed in the modules database, they are removed when you remove the REST API instance if you include the modules database in the instance teardown.
If you installed assets in a REST API instance using MarkLogic 6, they cannot be managed using the /ext
service unless you re-install them using /ext
. Reinstalling the assets may require additional changes because the asset URIs will change. If you choose not to migrate such assets, continue to maintain them according to the documentation for MarkLogic 6.
Follow this procedure to install or update a library module or other asset in the modules database associated with your REST Server. If the REST Server is part of a cluster, the asset is automatically propagated throughout the cluster.
The modules database path under which you install an asset must begin with /ext/
.
com.marklogic.client.DatabaseClient
object. For example, if using digest authentication:DatabaseClient client = DatabaseClientFactory.newClient( host, port, new DigestAuthContext(username, password));
com.marklogic.client.admin.ExtensionLibrariesManager
. Note that the method for doing so is associated with a ServerConfigManager
.ExtensionLibrariesManager libMgr = client.newServerConfigManager().newExtensionLibrariesManager();
FileHandle handle = new FileHandle(new File("module.xqy")).withFormat(Format.TEXT));
FileHandle handle = new FileHandle(new File("module.sjs")).withFormat(Format.TEXT));
ExtensionLibrariesManager.write()
. For example:libMgr.write("/ext/my/path/to/my/module.xqy", handle); //Or libMgr.write("/ext/my/path/to/my/module.sjs", handle);
You can also specify asset-specific permissions by passing an ExtensionLibraryDescriptor
instead of a simple path string to ExtensionLibrariesManager.write()
. The following example uses an descriptor:
ExtensionLibraryDescriptor desc = new ExtensionLibraryDescriptor(); desc.setPath("/ext/my/path/to/my/module.xqy"); //Or desc.setPath("/ext/my/path/to/my/module.sjs"); desc.addPermission("my-role", "my-capability"); ... libMgr.write(desc, handle);
To use a dependent library installed with /ext
in your extension or transform module, use the same URI under which you installed the dependent library. For example, if a dependent library is installed with the URI /ext/my/domain/my-lib.xqy
, then the extension module using this library should include an import of the form:
import module namespace dep="mylib" at "/ext/my/domain/my-lib.xqy";
const dep = require("/ext/my/domain/my-lib.sjs");
To remove an asset from the modules database associated with the REST Server, call com.marklogic.client.admin.ExtensionLibrariesManager.delete()
. For example:
DatabaseClient client = DatabaseClientFactory.newClient(...); ExtensionLibrariesManager libMgr = client.newServerConfigManager().newExtensionLibrariesManager(); libMgr.delete("/ext/my/path/to/my/module.xqy"); //Or libMgr.delete("/ext/my/path/to/my/module.sjs");
You can also call delete()
with a ExtensionLibraryDescriptor
.
If the path passed to delete()
, whether by String
or descriptor, is a database directory path, all assets in the directory are deleted. If the path is a single asset, just that asset is deleted.
You can retrieve a list of all the assets installed in the modules database associated with the REST Server by calling com.marklogic.client.admin.ExtensionsLibraryManager.list
(). If you call list()
with no parameters, you get a list of ExtensionLibraryDescriptor
objects for all assets. If you call list()
with a path, you get a similar list of descriptors for all assets installed in that database directory.
The following code snippet retrieves descriptors for all installed assets and prints the path of each one to stdout.
DatabaseClient client = DatabaseClientFactory.newClient(...); ExtensionLibrariesManager libMgr = client.newServerConfigManager().newExtensionLibrariesManager(); ExtensionLibraryDescriptor[] descriptors = libMgr.list(); for (ExtensionLibraryDescriptor descriptor : descriptors) { System.out.println(descriptor.getPath()); }
To retrieve the contents of an asset installed in the modules database associated with a REST Server, call com.marklogic.client.admin.LibrariesExtensionManager.read()
. You must first create a handle to receive the contents.
The following code snippet reads the contents of an XQuery library module into a string:
DatabaseClient client = DatabaseClientFactory.newClient(...); ExtensionLibrariesManager libMgr = client.newServerConfigManager().newExtensionLibrariesManager(); StringHandle handle = libMgr.read("/ext/my/path/to/my/module.xqy", new StringHandle());
The following code snippet reads the contents of an Javascript library module into a string:
DatabaseClient client = DatabaseClientFactory.newClient(...); ExtensionLibrariesManager libMgr = client.newServerConfigManager().newExtensionLibrariesManager(); StringHandle handle = libMgr.read("/ext/my/path/to/my/module.sjs", new StringHandle());
The com.marklogic.client.eval.ServerEvaluationCall
enables you to send blocks of JavaScript and XQuery to MarkLogic Server for evaluation or to invoke an XQuery or JavaScript module installed in the modules database. This is equivalent to calling the builtin server functions xdmp:eval or xdmp:invoke (XQuery), or xdmp.eval or xdmp.invoke (JavaScript).
This section covers the following related topics:
Evaluating an ad-hoc query on MarkLogic Server requires the following privileges or the equivalent:
http://marklogic.com/xdmp/privileges/xdmp-eval
http://marklogic.com/xdmp/privileges/xdmp-eval-in
http://marklogic.com/xdmp/privileges/xdbc-eval
http://marklogic.com/xdmp/privileges/xdbc-eval-in
Invoking a module on MarkLogic Server requires the following privileges or the equivalent:
Follow this procedure to evaluate an Ad-Hoc XQuery or JavaScript query on MarkLogic Server. You must use a user that has the privileges listed in Security Requirements.
DatabaseClient
for connecting to the database. For example, if using digest authentication:DatabaseClient client = DatabaseClientFactory.newClient( host, port, new DigestAuthContext(username, password));
ServerEvaluationCall
object.ServerEvaluationCall theCall = client.newServerEval();
String
or a TextWriteHandle
. ServerEvaluationCall.javascript
:String query = "word1 \" \" + word2"; theCall.javascript(query);
ServerEvaluationCall.xquery
.String query = "xquery version '1.0-ml';" + "declare variable $word1 as xs:string external;" + "declare variable $word2 as xs:string external;" + "fn:concat($word1, ' ', $word2)"; theCall.xquery(query);
ServerEvaluationCall.addVariable
. For details, see Specifying External Variable Values.theCall.addVariable("word1", "hello"); theCall.addVariable("word2", "world");
ServerEvaluationCall.eval
or ServerEvaluationCall.evalAs
. For details, see Interpreting the Results of Eval or Invoke.String response = theCall.evalAs(String.class);
client.release();
The following code puts these steps together into a single block.
DatabaseClient client = DatabaseClientFactory.newClient( host, port, new DigestAuthContext(username, password)); ServerEvaluationCall theCall = client.newServerEval(); String query = "word1 \" \" + word2"; String result = theCall.javascript(query) .addVariable("word1", "hello") .addVariable("word2", "world") .evalAs(String.class);
A JavaScript MJS module can be invoked through the /v1/invoke
endpoint. This is the preferred method.
A data service endpoint can be implemented as a JavaScript MJS module. This is the preferred method.
You can invoke an arbitrary JavaScript or XQuery module installed in the modules database associated with the REST API instance by setting a module path on a ServerEvaluationCall object and then calling ServerEvaluationCall.eval
or ServerEvaluationCall.evalAs
. The module path is resolved using the rules described in Rules for Resolving Import, Invoke, and Spawn Paths in the Application Developer's Guide.
You can install your module is using com.marklogic.client.admin.ExtensionLibrariesManager
. For details, see Installing or Updating Assets. If you install your module using the ExtensionLibrariesManager
interface, your module path will always being with /ext/.
Follow this procedure to invoke an XQuery or JavaScript module pre-installed on MarkLogic Server. You must use a user that has the privileges listed in Security Requirements.
DatabaseClient
for connecting to the database. For example, if using digest authentication:DatabaseClient client = DatabaseClientFactory.newClient( host, port, new DigestAuthContext(username, password));
ServerEvaluationCall
object.ServerEvaluationCall invoker = client.newServerEval();
invoker.modulePath("/my/module/path.sjs");
ServerEvaluationCall.addVariable
. For details, see Specifying External Variable Values.invoker.addVariable("word1", "hello"); invoker.addVariable("word2", "world");
ServerEvaluationCall.eval
or ServerEvaluationCall.evalAs
. For details, see Interpreting the Results of Eval or Invoke.String response = invoker.evalAs(String.class);
client.release();
The following code puts these steps together into a single block.
DatabaseClient client = DatabaseClientFactory.newClient( host, port, new DigestAuthContext(username, password)); ServerEvaluationCall invoker = client.newServerEval(); String result = invoker.modulePath("/ext/invoke/example.sjs") .addVariable("word1", "hello") .addVariable("word2", "world") .evalAs(String.class);
You can pass values to an ad-hoc query or invoked module at runtime using external variables. Specify the variable values using ServerEvaluationCall.addVariable
. or ServerEvaluationCall.addVariableAs
.
Use addVariable
for simple value types, such as String
, Number
, and Boolean
and values with a suitable AbstractWriteHandle
, such as DOMHandle
for XML and JacksonHandle
for JSON. For example:
ServerEvaluationCall theCall = client.newServerEval(); ... theCall.addVariable("aString", "hello") .addVariable("aBool", true) .addVariable("aNumber", 3.14);
Use addVariableAs
for other complex value types such as objects. For example, the following code uses a Jackson object mapper to set an external variable value to a JSON object that can be used as a JavaScript object by the server-side code:
theCall.addVariableAs("anObj", new ObjectMapper().createObjectNode().put("key", "value"))
If you're evaluating or invoking XQuery code, you must declare the variables explicitly in your ad-hoc query or module. For example, the following XQuery prolog declares two external string-valued variables whose values can be supplied at runtime.
xquery version "1.0-ml"; declare variable $word1 as xs:string external; declare variable $word2 as xs:string external; ...
If your XQuery external variables are in a namespace, use ServerEvaluationCall.addNamespac
e to associate a prefix with the namespace, and then use the namespace prefix in the variable name passed to ServerEvaluationCall.addVariable
. For example, given the following ad-hoc query:
xquery version "1.0-ml"; declare namespace my = "http://example.com"; declare variable $my:who as xs:string external; fn:concat("hello", " ", $my:who)
Set the variable values as follows:
theCall.addNamespace("my", "http://example.com") .addVariable("my:who", "me") ...
You can request results in the following ways:
ServerEvaluationCall.evalAs
. For example, if you know an ad-hoc query returns a single String value, you can evaluate it as follows:String result = theCall.evalAs(String.class);
AbstractReadHandle
to ServerEvaluationCall.eval
to process a single result through a handle. For example:DOMHandle result = theCall.eval(new DOMHandle()); //Or JacksonHandle result = theCall.eval(new JacksonHandle());
ServerEvaluationCall.eval
with no parameters to return an EvalResultIterator
. For example:EvalResultIterator result = theCall.eval();
When you use an EvalResultIterator
, each value is encapsulated in a com.marklogic.client.eval.EvalResult
that provides type information and accessors for the value.
The EvalResult.format
method provides abstract type information, such as text, binary, json, or xml. The EvalResult.getType
method provides more detailed type information, when available, such as string, integer, decimal, or date. Detailed type information is not always available.
The table below maps common server-side value types to the values you can expect to their corresponding com.marklogic.client.io.Format
(from EvalResult.format
) and EvalResult.Type
(from EvalResult.getType
).