MarkLogic 10 Product Documentation

Java Application Developer's Guide — Chapter 16

How it Works

You declare a function signature for each endpoint that implements a data service.

From a set of such declarations, the development tools generate a Java proxy service class that encapsulates the execution of the endpoints including the marshalling and transport of the request and response data. The middle-tier business logic can then call the methods of the generated class.

A MarkLogic data service consists of three main components:

• Endpoint Declaration: This is a JSON document used to specify the name of the service as well as the names and data types of the inputs and outputs.
• Endpoint Proxy: Code that exposes the service definition in Java, automatically invoking the services remotely against a MarkLogic cluster for the caller.
• Endpoint Module: This is the implementation of a data service in MarkLogic as a JavaScript or XQuery module.

By declaring the data tier functions needed by the middle-tier business logic, the endpoint declaration establishes a division of responsibility between the Java middle-tier developer and the data service developer. The endpoint declaration acts as a contract for collaboration between the two roles.

It is the responsibility of the end point service developer to limit access to the Data Services assets by adding the necessary security asserts (using xdmp.securityAssert or xdmp:security-assert functions ) to test for privileges.

Prerequisites

To create a proxy service, you need a Java JDK environment with Gradle and the following MarkLogic software components:

• Current version of MarkLogic Server
• Current version of MarkLogic Client Java API
• Current version of ml-gradle

The MarkLogic Java development tools are available as a Gradle plugin.

This document assumes that you are familiar with Java and Gradle.

If you are unfamiliar with Gradle, the ml-gradle project lists some resources for getting started:

Installing and learning Gradle

Typically, you create one Gradle project directory for all of the work on proxy services for one content database.

Relation to the Java Client API

The MarkLogic Java Client API includes development tooling and runtime proxies so that a Java application can access custom data services in a MarkLogic cluster. The Java application calls strongly typed services running in the databases as if they were "out of the box" Java methods. The API handles the underlying network protocol and data marshalling.

Setting Up an App Server for the Proxy Service

Typically, you set up a single App Server for all of the proxy services for a content database.

The App Server configuration must have the following characteristics:

• Must have a modules database.
• Must have a root of /.

You cannot use the following App Servers, created by default when you install MarkLogic:

• The REST/HTTP/XDBC App Server on port 8000
• The Admin API App Server on port 8001
• The REST Management API App Server on port 8002

As noted above, you are also able to use a REST server (that is, an App Server created for the Client REST API).

Data services can reside on REST and non-REST App Servers.

To make creation and configuration of the App Server and its modules database, you should manage a repeatable operation in a version control system. You can also put resources in the Gradle project directory and use ml-gradle to operate on those resources.

See Getting started for a step-by-step guide to this Gradle procedure.

As an easy expedient when learning about MarkLogic, you can instead configure the App Server and modules database manually. As a long-term practice, however, we recommend a repeatable approach using Gradle.

Creating the Proxy Service Directory

For each proxy service, you create a separate subdirectory under the Gradle project directory.

Each proxy service directory holds all of the resources required to support the proxy service, including:

• The service declaration
• The endpoint proxy declarations
• The module called by each endpoint proxy
• Any server-side libraries to support the endpoint modules

For easier deployment to the modules database using ml-gradle, you should create the proxy service directory under the src/main/ml-modules/root project subdirectory. If you are working under a MarkLogic ReST server application, you should use the following proxy service directory: src/main/ml-modules/root/ds.

For instance, a project might choose to provide the priceDynamically service in the following proxy service directory:

src/main/ml-modules/root/inventory/priceDynamically

Declaring the Proxy Service

The proxy service directory must contain exactly one service declaration file. The service declaration file must have the name service.json

The service declaration consists of a JSON object with the following properties:

The following example declares the /inventory/priceDynamically/ directory as the address of the endpoints in the modules database and declares com.some.business.inventory.DynamicPricer as the generated Java class:

{	
   "endpointDirectory" : "/inventory/priceDynamically/",
   "$javaClass"        : "com.some.business.inventory.DynamicPricer"
}

Conventionally, the value of the endpointDirectory property should be the same as the path of the proxy service directory under the special ml-gradle src/main/ml-modules/root directory (so, the service directory for this service.json file would conventionally be src/main/ml-modules/root/inventory/priceDynamically).

The endpoint directory value should include the leading / and should resemble a Linux path.

After declaring the service, you populate it with endpoint proxy declarations

Declaring the Endpoint

The name, parameters, and return value for each endpoint is declared in a file with the .api extension in the service directory. The file contains a JSON data structure with the following properties:

The endpoint declaration is used both to generate a method in a Java class to call on the middle-tier and to unmarshal the request and marshal the response when the App Server executes the endpoint module.

The .api file for proxy endpoint must be loaded into the modules database with the endpoint module.

The following sections provide more detail about the params and return declarations

{
  "functionName" : "lookupPricingFactors",
  "params" : [ {
    "name" : "productCode",
    "datatype" : "string"
  }, {
    "name" : "customerId",
    "datatype" : "unsignedLong"
  } ],
  "return" : {
    "datatype" : "jsonDocument"
  }}

{
  "functionName" : "produceReport",
  "params":[ {
    "name":"id", "datatype":"int"
  }, {
    "name":"occurred", "datatype":"date",
      "$javaClass":"java.time.LocalDate"
  } ],
  "return" : {
    "datatype":"jsonDocument",
      "$javaClass":"com.fasterxml.jackson.databind.JsonNode"}
  }
}

// A simple example of the use of "session" in an .api declaration:
{
  "functionName" : "SessionChecks",
  "params" : [ {
    "name" : "api_session",
    "datatype" : "session",
    "desc" : "Holds the session object"    
  },
...
}

Providing the Module for an Endpoint Proxy

A JavaScript MJS module can be invoked through the /v1/invoke endpoint. This is the preferred method.

A data service endpoint can be implemented as a JavaScript MJS module. This is the preferred method.

You implement the data operations for an endpoint proxy in an XQuery or Server-Side JavaScript endpoint module. The proxy service directory of your project must contain exactly one endpoint module for each endpoint declaration in your service.

An endpoint module must have the same base name as the endpoint declaration. In addition, it must have either an .xqy (XQuery) or .sjs (JavaScript) extension, depending on the implementation language.

The App Server handles marshalling and unmarshalling for the endpoint. That is, the endpoint does not interact directly with the transport layer (which, internally, is currently HTTP).

The endpoint module must define an external variable for each parameter in the endpoint declaration. In an SJS endpoint, use a var statement at the top of the module with no initialization of the variable. In an XQuery endpoint, use an external variable with the server data type corresponding to the parameter data type.

The endpoint module must also return a value with the appropriate data type.

For the lookupPricingFactors endpoint whose declaration appears earlier, the SJS endpoint module would resemble the following fragment:

'use strict';      
var productCode; // an xs.string value
var customerId;  // an xs.unsignedLong value
... /* the code that produces a JSON document as output */

The equivalent XQuery endpoint module would resemble the following fragment:

xquery version "1.0-ml";
 declare variable $productCode as xs:string       external;
 declare variable $customerId as xs:unsignedLong external;
 declare option xdmp:mapping "false";
 ... (: the code that produces a JSON document as output :)

As a convenience, you can use the initializeModule Gradle task to create the skeleton for an endpoint module from an endpoint declaration. You specify the path (relative to the project directory) for the endpoint declaration with the endpointDeclarationFile property and the module extension (which can be either sjs or xqy) with the moduleExtension property.

Your Gradle build script should apply the com.marklogic.ml-development-tools plugin. You can execute the Gradle task using any of the following techniques:

• By setting the properties in the gradle.properties file and specifying the initializeModule task on the gradle command line
• By specifying the properties with the -P option as well as the initializeModule task on the gradle command line
• By supplying a build script with a custom task of the com.marklogic.client.tools.gradle.ModuleInitTask type

For the command-line approach, the Gradle build script would resemble the following example:

plugins {
    id 'com.marklogic.ml-development-tools' version '4.1.1'
}

On Linux, the command-line for initializing the lookupPricingFactors.sjs SJS endpoint module from the lookupPricingFactors.api endpoint declaration might resemble the following example:

gradle \
-PendpointDeclarationFile=src/main/ml-modules/root/inventory/priceDynamically/lookupPricingFactors.api \
    -PmoduleExtension=sjs \
    initializeModule

Once each .api endpoint declaration file has an equivalent endpoint module to implement the endpoint, you can load the proxy service directory into the modules database and generate the proxy service Java class. (The Java code generation checks the endpoint module in the service directory to determine how to invoke the endpoint.)

Deploying a Proxy Service

You must load the resources from the proxy service directory into the module database of the App Server. Deploy your resources to the same database directory as the value of the endpointDirectory property of the service declaration file (service.json).

To load a directory into the modules database, you can use either of the mlLoadModules or mlReloadModules tasks provided by ml-gradle. You supply the properties required for deployment including the following:

• mlHost - required
• mlAppServicesUsername - required if not admin and mlPassword not set
• mlAppServicesPassword - required if not admin and mlUsername not set
• ml|AppServicesPort - required if not 8000
• mlModulesDatabaseName - required
• mlModulePermissions - required
• mlNoRestServer - required to be true, so that mlDeploy will not create a REST API server by default.
• mlReplaceTokensInModules - typically false

If you did not create the proxy service directory under the src/main/ml-modules/root project subdirectory, you must specify the parent directory for the root directory with the mlModulePaths property.

You can supply properties using a gradle.properties file or a task.

After you have configured the properties, the command to load the modules would resemble the following example (or the equivalent with mlReloadModules):

gradle mlLoadModules

For more information, see How modules are loaded.

Generating the Proxy Service Class

A proxy service class is a Java interface for calling the endpoint modules for your service on the MarkLogic e-node. You generate the proxy service class from the resources in the proxy service directory.

The proxy service class has the name specified by the $javaClass property of the service declaration file (service.json). The class has one method for each endpoint declaration with an associated endpoint module in the proxy service directory.

To generate the class, you use the generateEndpointProxies Gradle task. You specify the path (relative to the project directory) of the service declaration file (service.json) with the serviceDeclarationFile property. You can also specify the output directory with the javaBaseDirectory property or omit the property to use the default (which is the src/main/java subdirectory of the project directory).

Your Gradle build script should apply the com.marklogic.ml-development-tools plugin. You can execute the task using any of the following techniques:

• By setting the properties in the gradle.properties file and specifying the generateEndpointProxies task on the gradle command line
• By specifying the properties with the -P option as well as the generateEndpointProxies task on the gradle command line
• By supplying a build script with custom task of the com.marklogic.client.tools.gradle.EndpointProxiesGenTask type
• By supplying a build script with the endpointProxiesConfig extension configuration and specifying the generateEndpointProxies task on the gradle command line

For the custom task approach, the Gradle build script for generating a class with a method for each endpoint in the priceDynamically service might resemble the following example:

plugins {
    id 'com.marklogic.ml-development-tools' version '4.1.1'
}
task generateDynamicPricer(type: com.marklogic.client.tools.gradle.EndpointProxiesGenTask) {
    serviceDeclarationFile = 'src/main/ml-modules/root/inventory/priceDynamically/service.json'
}

The command-line to execute the custom task would resemble the following example:

gradle generateDynamicPricer

You only need to regenerate the proxy service class when the list of endpoints or the name, parameters, or return value for an endpoint changes. You do not need the regenerate the proxy service class after changing the module that implements the endpoint.

Using a Proxy Service Class

In general, you can work with your generated proxy service Java class in the same way as with manually written Java source files.

The generated class has an on() static method that is a factory for constructing the class. The on() method requires a DatabaseClient for the App Server. You construct the database client by using the DatabaseClientFactory class of the Java API.

Note: You cannot specify the database explicitly when creating the DatabaseClient but, instead, must use the default database associated with the App Server.

Modifying the Source project to Enable Publication

The procedure is to modify the build.gradle file for the source project to publish the Data Services implementation to a Maven repository, as in:

plugins {
    ...
    id 'maven-publish'
    ...
}
configurations {
    ...
    myDataServiceBundle
}
task myDataServiceJar(type: Jar) {
   baseName = 'myDataService'
   description = "..."
   from("src/test/ml-modules/root/ds/myDataService") {
       into("myDataService/ml-modules/root/ds/myDataService")
   }
   destinationDir file("build/libs")
}
publishing {
    publications {
        ...
        MainMyDataService(MavenPublication) {
            artifactId "myDataService"
            artifact myDataServiceJar
        }
        ...
    }
}

Using the Maven Bundle in Other Projects

After the bundle for the Data Service endpoint implementation has been published to a Maven repository, other projects can use the bundle by configuring build.gradle to use the mlBundle task provided by the ml-gradle tool:

plugins {
    ...
    id "com.marklogic.ml-gradle" version "..."
    ...
}
dependencies {
    ...
    mlBundle group: '...', name: 'myDataService', version: '...'
    ...
}

For more information, see the Bundles section of our ml-gradle documentation:

Following the standard approach for Gradle and Maven repositories, the client interface can be published and consumed as a Java jar.