MarkLogic 9 Product Documentation

Node.js Application Developer's Guide — Chapter 8

What is a Resource Service Extension?

Resource service extensions extend the Node.js Client API by creating a RESTful interface to XQuery and server-side JavaScript modules. The server-side extension implements functions to handle GET, PUT, POST, and DELETE HTTP requests received on the extension by the REST Client API. The Node.js Client API enables you to invoke these methods via the DatabaseClient.resources interface. You can wrap your own Node.js interface around the DatabaseClient.resources operations to expose the service in a domain-specific way.

For example, you can create a dictionary program resource extension that looks up words, checks spelling, and makes suggestions for unknown words on MarkLogic Server. The individual operations an application programmer may call, for example, lookUpWords(), spellCheck(), and so on, are the domain-specific services that expose the resource extension.

The following are the basic steps to create and use a resource extension using the Node.js Client API:

1. Create an XQuery or JavaScript module that implements the services for the resource.
2. Install the resource service extension implementation in the modules database associated with the REST API instance using DatabaseClient.config.resources.write.
3. Access the resource extension methods using DatabaseClient.resources operations such as DatabaseClient.resources.get.

The DatabaseClient.config.resources interface also supports dynamic discovery of installed extensions. When you install an extension, you can specify metadata, including method parameter name and type information to make it easier to use dynamically discovered extensions. The metadata is purely informational.

If your extension depends on other modules or assets, you can install them in the modules database using DatabaseClient.extlibs interface. For details, see Managing Assets in the Modules Database.

For a complete example, see Example: Installing and Using a Resource Service Extension.

Creating a Resource Service Extension

You can implement a resource service Extension using server-side JavaScript or XQuery. The interface is shared across multiple MarkLogic client APIs, so you can use the same extensions with the Java Client API, Node.js Client API, and the REST Client API.

You can install an extension with one client API and use it with all them. For example, you can use a resource service extension installed using the REST Client API with an application implemented using the Node.js Client API and the Java Client API.

For the interface definition, authoring guidelines, and example implementations, see Extending the REST API in the REST Application Developer's Guide.

Installing a Resource Service Extension

Before you can use a resource extension, you must install the implementation on MarkLogic Server. You must have the rest-admin role or equivalent privileges to install a resource service extension.

Use the following procedure to install your extension implementation:

1. If your resource extension depends on additional library modules, install these dependent libraries on MarkLogic Server. For details, see Managing Assets in the Modules Database.
2. Optionally, define metadata for your extension that describes attributes such as provider, description, and version.
3. Call DatabaseClient.config.resources.write to install your extension into the modules database of the REST API instance associated with your DatabaseClient object. Your call must provide a name for the extension, the implementation language (XQuery or JavaScript), and the implementation source code. You may include optional metadata.

   For an XQuery extension, the extension must be installed under the same name as the name in the extension module namespace declaration. For example, an XQuery extension with the following module namespace must be installed as example.

   xquery version "1.0-ml";
   module namespace yourNSPrefix = 
     "http://marklogic.com/rest-api/resource/example";
   ...

For example, the following code installs a JavaScript extension under the name js-example, without metadata. The extension implementation is streamed from the file js-example.sjs.

const fs = require('fs');
const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.config.resources.write(
  'js-example', 'javascript',
  fs.createReadStream('./js-example.sjs')
).result(function(response) {
  console.log('Installed extension: ' + response.name);
}, function(error) {
  console.log(JSON.stringify(error, null, 2));
});

The following code installs the same extension with a full set of metadata. You need not provide all metadata properties. You must include name, format, and source.

const fs = require('fs');
const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.config.resources.write({
  name: 'js-example',
  format: 'javascript',
  source: fs.createReadStream('./js-example.sjs'),
  // everything below this is optional metadata
  title: 'Example JavaScript Extension',
  description: 'An example of implementing resource extensions in SJS',
  provider: 'MarkLogic',
  version: 1.0
}).result(function(response) {
  console.log('Installed extension: ' + response.name);
}, function(error) {
  console.log(JSON.stringify(error, null, 2));
});

For a complete example, see Example: Installing and Using a Resource Service Extension.

Using a Resource Service Extension

To invoke the HTTP methods of a resource service extension, use the DatabaseClient.resources interface. The interface has a function corresponding to each HTTP verb: get, put, post, and remove (DELETE). For example, you can invoke the get method of your extension with no parameters as follows:

db.resources.get('js-example')

You can also pass in an object that encapsulates parameters expected by the implementation, and a transaction id.

The result of the invocation depends on the method. For example, the GET extension interface enables you return one or more documents, so the result of calling resources.get is an object whose stream function can be used to incrementally process the documents in the response.

The following example invokes the get function of the resource service extension from Example: JavaScript Resource Service Extension in the REST Application Developer's Guide. Three parameters are passed to the implementation, named a, b, and c. The GET implementation of this extension simply echos back the supplied parameters as a JSON document, one document per parameter.

const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.resources.get({
  name: 'js-example', 
  params: { a: 1, b: 2, c: 'three'}
}).result(function(response) {
  console.log(response);
}, function(error) {
  console.log(JSON.stringify(error, null, 2));
});

If the call is successful, the output is similar to the following. The value of the content property in each array item is a document returned by the extensions GET method implementation.

[ { contentType: 'application/json',
    format: 'json',
    contentLength: '29',
    content: { name: 'c', value: 'three' } },
  { contentType: 'application/json',
    format: 'json',
    contentLength: '25',
    content: { name: 'b', value: '2' } },
  { contentType: 'application/json',
    format: 'json',
    contentLength: '25',
    content: { name: 'a', value: '1' } } ]

For details, see Example: Installing and Using a Resource Service Extension.

Example: Installing and Using a Resource Service Extension

This example demonstrates how to install and exercise the GET and PUT methods of the JavaScript extension from Example: JavaScript Resource Service Extension in the REST Application Developer's Guide. The extension is usable with any of the MarkLogic client APIs (Node.js, Java, REST).

Use the following procedure to install the extension and exercise the GET method. The GET method of this extension accepts one or more caller-defined parameters and returns a JSON document of the following form for each parameter passed in: { "name": param-name, "value": param-value }.

1. Copy the extension implementation to a file named js-example.sjs. For the implementation, see JavaScript Extension Implementation in the REST Application Developer's Guide.
2. Install the extension under the name js-example by running the following script. You must have the rest-admin role or equivalent privileges to install an extension. For details, see Installing a Resource Service Extension.

   const fs = require('fs');
   const marklogic = require('marklogic');
   const my = require('./my-connection.js');
   const db = marklogic.createDatabaseClient(my.connInfo);
   
   db.config.resources.write({
     name: 'js-example',
     format: 'javascript',
     source: fs.createReadStream('./js-example.sjs'),
     // everything below this is optional metadata
     title: 'Example JavaScript Extension',
     description: 'An example of implementing resource extensions in SJS',
     provider: 'MarkLogic',
     version: 1.0
   }).result(function(response) {
     console.log('Installed extension: ' + response.name);
   }, function(error) {
     console.log(JSON.stringify(error, null, 2));
   });

3. Optionally, retrieve metadata about the extension using the following script. For details, see Discovering Resource Service Extensions.

   const marklogic = require('marklogic');
   const my = require('./my-connection.js');
   const db = marklogic.createDatabaseClient(my.connInfo);
   
   db.config.resources.read('js-example').result(
     function(response) {
       console.log(response);
     }, 
     function(error) {
       console.log(JSON.stringify(error, null, 2));
     });

4. Exercise the GET method of the extension by running the following script. For details, see Using a Resource Service Extension.

   const marklogic = require('marklogic');
   const my = require('./my-connection.js');
   const db = marklogic.createDatabaseClient(my.connInfo);
   
   db.resources.get({
     name: 'js-example', 
     params: { a: 1, b: 2, c: 'three'}
   }).result(function(response) {
     console.log(response);
   }, function(error) {
     console.log(JSON.stringify(error, null, 2));
   });

   The GET method generates a JSON document of the form { name: pName, value: pValue} for each parameter passed in. Thus, the invocation above should generate three documents. The expected output from invoking the GET method is similar to the following:

   [ { contentType: 'application/json',
       format: 'json',
       contentLength: '29',
       content: { name: 'c', value: 'three' } },
     { contentType: 'application/json',
       format: 'json',
       contentLength: '25',
       content: { name: 'b', value: '2' } },
     { contentType: 'application/json',
       format: 'json',
       contentLength: '25',
       content: { name: 'a', value: '1' } } ]

5. Exercise the PUT method of the extension by running the following script.

   const marklogic = require('marklogic');
   const my = require('./my-connection.js');
   const db = marklogic.createDatabaseClient(my.connInfo);
   
   db.resources.put({
     name: 'js-example', 
     params: {
       basename: ['one', 'two']},
     documents: [
       { contentType: 'application/json',
       content: {key1:'value1'} },
       { contentType: 'application/json',
       content: {key2:'value2'} },
     ]
   }).result(function(response) {
     console.log(JSON.stringify(response, null, 2));
   }, function(error) {
     console.log(JSON.stringify(error, null, 2));
   });

   The PUT method of this extensions accepts JSON and XML documents as input. For each input JSON document, a written property is added to the document before it is inserted into the database. XML documents are inserted into the database unchanged. The document URIs are derived from a basename parameter supplied by the caller. The following is the expected output from invoking PUT method.

   { "written": [
       "/extensions/one.json",
       "/extensions/two.json"
   ] } 

If you examine the two documents created by the PUT exercise, you can see that a written property has been added. For example, /extensions/one.json has contents similar to the following (the timestamp value will vary):

{
  "key1": "value1",
  "written": "08:35:54-08:00"
}

To report errors from your implementation to the client, you must use the convention described in Error Reporting in Extensions and Transformations. For example, if you do not pass a basename parameter value for each input document, the extension reports an error in the following way:

if (docs.count > basenames.length) {
  returnErrToClient(400, 'Bad Request', 
    'Insufficient number of uri basenames. Expected ' +
    docs.count + ' got ' + basenames.length + '.');
    // unreachable - control does not return from fn.error
}

The error reaches the client application in the following form:

{
  "message": "js-example: response with invalid 400 status",
  "statusCode": 400,
  "body": {
    "errorResponse": {
      "statusCode": 400,
      "status": "Bad Request",
      "messageCode": "RESTAPI-SRVEXERR",
      "message": "Insufficient number of uri basenames. Expected 2 got 1."
    }
  }
}

Retrieving the Implementation of a Resource Service Extension

Use DatabaseClient.config.resources.read to retrieve the implementation of a resource service extension. You must have the rest-admin role or equivalent privileges to use this interface.

For example, the following call retrieves the implementation of the resource service extension installed as js-example:

const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.config.resources.read('js-example').result(
  function(response) {
    console.log(response);
  }, 
  function(error) {
    console.log(JSON.stringify(error, null, 2));
  });

Discovering Resource Service Extensions

You can use DatabaseClient.config.resources.list to retrieve the name, interface, and other metadata about installed resource extensions. You must have the rest-reader role or equivalent privileges to use this interface.

The amount of information available about a given extension depends on the amount of metadata provided during installation of the extension. The name and methods are always available. Details such as provider, version, and method parameter information are optional.

By default, this request rebuilds the extension metadata each time it is called to ensure the metadata is up to date.

The following example retrieves data about the installed extensions:

const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.config.resources.list().result(
function(response) {
  console.log('Installed extensions: ');
  console.log(JSON.stringify(response, null, 2));
}, function(error) {
  console.log(JSON.stringify(error, null, 2));
});

If you installed a single extension named js-example with metadata, as shown in Installing a Resource Service Extension, then the output of the above script is similar to the following.

{ "resources": {
    "resource": [ {
      "name": "js-example",
      "source-format": "javascript",
      "provider-name": "MarkLogic",
      "title": "Example JavaScript Extension",
      "version": "1",
      "description": "An example of implementing resource extensions in SJS",
      "methods": {
        "method": [
          { "method-name": "get" },
          { "method-name": "post" },
          { "method-name": "put" },
          { "method-name": "delete" }
        ]
      },
      "resource-source": "/v1/resources/js-example"
    }
  ] }
}

Deleting Resource Service Extensions

Use DatabaseClient.config.resources.remove to remove a resource service extension. You must supply the same name that you used in installing the extension. To remove an extension, you must have the rest-admin role or the equivalent privileges.

Deleting an extension is an idempotent operations. That is, you will receive the same response whether the named extension exists or not.

The following code snippet removes the resource service extension named js-example.

const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.config.resources.remove('js-example').result(
  function(response) {
    console.log('Removed extension: ', response.name);
  }, 
  function(error) {
    console.log(JSON.stringify(error, null, 2));
  });

What is a Content Transformation?

The Node.js Client API enables you to create custom content transformations and apply them during operations such as document ingestion and retrieval. For example, you can create a write transform that adds or modifies a JSON property or XML element for each document as it is inserted into the database. The API has hooks for applying the following kinds of transform:

• Write transform: Applied before inserting documents into the database.
• Read transform: Applied when reading documents from the database. You can configure both default and per-request read transforms.
• Search result transform: Applied to the search result summary when you make queries that include a summary instead of just returning matching documents and metadata.

You implement a transform as a server-side JavaScript function, XQuery function, or XSLT stylesheet that accepts a document as input and produces documents as output. Your transform must conform to the interface and guidelines described Writing Transformations in the REST Application Developer's Guide. Your transforms can accept transform-specific parameters.

Transforms must be installed in the modules database associated with the REST API instance before you can use them. Use the DatabaseClient.config.transforms interface to install and manage your transforms using Node.js. For details, see Installing a Transformation.

You apply a transform by passing its name to supporting operations, such as DatabaseClient.documents.write, DatabaseClient.documents.read and DatabaseClient.documents.query. For details, see Using a Transformation.

Creating a Transformation

You can implement a transform function using server-side JavaScript or XQuery. The interface is shared across multiple MarkLogic client APIs, so you can use the same transforms with the Java Client API, Node.js Client API, and the REST Client API.

Your transform module must include an export named transform. For example:

function insertTimestamp(context, params, content)
{...}
exports.transform = insertTimestamp;

You can install a transform with one client API and use it with all of them. For example, you can use transform installed using the Node.js Client API with an application implemented using the Node.js Client API or the Java Client API.

For the interface definition, authoring guidelines, and example implementations, see Writing Transformations in the REST Application Developer's Guide. To return errors from your transform to the client, use the conventions described in Error Reporting in Extensions and Transformations.

For a complete Node.js and JavaScript example, see Example: Read, Write, and Query Transforms.

Installing a Transformation

Use DatabaseClient.config.transforms.write to install a transform in the modules database associated with your REST API instance. Using this interface ensures your transform is installed according to the conventions expected by the API, enabling you to subsequently apply and manage the transform with the Node.js Client API.

You must have the rest-admin role or equivalent privileges to install a transform.

You can include optional metadata about your transform during installation. The metadata is purely informational. You can retrieve it using DatabaseClient.config.transforms.list.

The following script installs a transform under the name js-transform. The transform implementation is read from a file name transform.sjs. Only name, format, and source are required. Everything else is optional metadata.

const fs = require('fs');
const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.config.transforms.write({
  name: 'js-transform',
  format: 'javascript',
  source: fs.createReadStream('./transform.sjs'),
  // everything below this is optional metadata
  title: 'Example JavaScript Transform',
  description: 'An example of an SJS read/write transform',
  provider: 'MarkLogic',
  version: 1.0
}).result(function(response) {
  console.log('Installed transform: ' + response.name);
}, function(error) {
  console.log(JSON.stringify(error, null, 2));
});

For a complete example, see Example: Read, Write, and Query Transforms.

You can retrieve a list of installed transforms and their metadata using DatabaseClient.transforms.list. For details, see Discovering Installed Transforms.

For a complete example, see Example: Read, Write, and Query Transforms.

Using a Transformation

You can specify a transform on document read, write, and query operations such as the following:

• DatabaseClient.documents.read
• DatabaseClient.documents.write and DatabaseClient.documents.createWriteStream
• DatabaseClient.documents.query, using queryBuilder.slice
• DatabaseClient.values.read, using valuesBuilder.slice

In all cases, you supply the name of a transform previously installed using DatabaseClient.config.transforms.write or the equivalent operation through one of the other client APIs.

For a complete example, see Example: Read, Write, and Query Transforms.

You can only specify one transform per operation. The transform applies to all inputs (write) or outputs (read or query).

To specify a transform to documents.read or documents.write, add a transform property to your call object with one of the following forms. The first two forms are equivalent. Use the third form to pass parameters expected by the transform.

transform: transformName

transform: [ transformName ]

transform: [ transformName, {paramName: paramValue, ...} ]

For read and write, include the transform property as an immediate child of the input call object. For example, if you pass a single document descriptor to documents.write, you can include the transform in the descriptor:

db.documents.write({ 
  uri: '/doc/example.json',
  contentType: 'application/json',
  content: { some: 'data' },
  transform: ['js-write-transform']
})

By contrast, if you use the multi-document form of input to documents.write, include the transform descriptor in the top level object, not inside each document descriptor. For example:

db.documents.write({
  documents: [
    { uri: '/transforms/example1.json',
      contentType: 'application/json',
      content: { some: 'data' },
    },
    { uri: '/transforms/example2.json',
      contentType: 'application/json',
      content: { some: 'more data' },
    }
  ],
  transform: ['js-write-transform']
})

To apply a transform to the results from DatabaseClient.documents.query or DatabaseClient.values.read, use a transform builder to create a descriptor, and then attach the descriptor to the query through the slice clause. For example, the following call applies a transform to a content query:

db.documents.query( 
  qb.where(
    qb.byExample({writeTimestamp: {'$exists': {}}})
  ).slice(qb.transform('js-query-transform', {a: 1, b: 'two'}))
)

The following call applies the same transform (without extra parameters) to a values query:

db.values.read(
  vb.fromIndexes('reputation')
    .slice(3,5, vb.transform('js-query-transform'))
)

Example: Read, Write, and Query Transforms

This example demonstrates installing and using transforms for read, write, and query operations. The example is designed for you to exercise all three types of transform in sequence, as follows:

1. Install the Transforms
2. Use the Write Transform
3. Use the Read Transform
4. Use the Query Transform

The source code for each transform is provided in the following sections:

• Read Transform Source Code
• Write Transform Source Code
• Query Transform Source Code

$ node install-transform.js
Installed transform: js-write-transform
Installed transform: js-query-transform
Installed transform: js-read-transform

const marklogic = require('marklogic');
const my = require('./my-connection.js');

const db = marklogic.createDatabaseClient(my.connInfo);

db.documents.write({
  documents: [
    { uri: '/transforms/example1.json',
      contentType: 'application/json',
      content: { some: 'data' },
    },
    { uri: '/transforms/example2.json',
      contentType: 'application/json',
      content: { some: 'more data' },
    }
  ],
  transform: ['js-write-transform']
}).result(function(response) {
  response.documents.forEach(function(document) {
    console.log(document.uri);
  });
}, function(error) {
  console.log(JSON.stringify(error));
});

$ node write.js
/transforms/example1.json
/transforms/example2.json

{
  "some": "data",
  "writeTimestamp": "2015-01-02T10:33:39.330483-08:00"
}

db.documents.write({
  uri: '/transforms/example3.json',
  contentType: 'application/json',
  content: { some: 'even more data' },
  transform: ['js-write-transform']
})...

transform: ['js-read-transform', {extra1: 1, extra2: 'two'}]

const marklogic = require('marklogic');
const my = require('./my-connection.js');

const db = marklogic.createDatabaseClient(my.connInfo);

db.documents.read({ 
  uris: ['/transforms/example1.json', '/transforms/example2.json'],
  transform: ['js-read-transform', {extra1: 1, extra2: 'two'}]
}).stream().on('data', function(document) {
  console.log("URI: " + document.uri);
  console.log(JSON.stringify(document.content, null, 2) + '\n');
}).on('end', function() {
  console.log('Finished');
})

URI: /transforms/example1.json
{
  "some": "data",
  "writeTimestamp": "2015-01-02T10:33:39.330483-08:00",
  "readTimestamp": "2015-01-02T10:44:18.538343-08:00",
  "extra2": "two",
  "extra1": "1"
}

URI: /transforms/example2.json
{
  "some": "more data",
  "writeTimestamp": "2015-01-02T10:33:39.355913-08:00",
  "readTimestamp": "2015-01-02T10:44:18.5632-08:00",
  "extra2": "two",
  "extra1": "1"
}

const marklogic = require('marklogic');
const my = require('./my-connection.js');

const db = marklogic.createDatabaseClient(my.connInfo);
const qb = marklogic.queryBuilder;

// Retrieve just search result summary by setting slice to 0
db.documents.query( 
  qb.where(
    qb.byExample({writeTimestamp: {'$exists': {}}})
  ).slice(qb.transform('js-query-transform'))
   .withOptions({categories: 'none'})
).result(function(response) {
  console.log(JSON.stringify(response, null, 2));
}, function(error) {
  console.log(JSON.stringify(error, null, 2));
});

// Retrieve matching documents instead of summary
db.documents.query( 
  qb.where(
    qb.byExample({writeTimestamp: {'$exists': {}}})
  ).slice(qb.transform('js-query-transform', {a: 1, b: 'two'}))
).stream().on('data', function(document) {
  console.log("URI: " + document.uri);
  console.log(JSON.stringify(document.content, null, 2) + '\n');
}).on('end', function() {
  console.log('Finished');
});

$ node query.js
URI: /transforms/example1.json
{
  "some": "data",
  "writeTimestamp": "2015-01-02T10:33:39.330483-08:00",
  "readTimestamp": "2015-01-02T11:09:58.410351-08:00",
  "b": "two",
  "a": "1"
}

URI: /transforms/example2.json
{
  "some": "more data",
  "writeTimestamp": "2015-01-02T10:33:39.355913-08:00",
  "readTimestamp": "2015-01-02T11:09:58.431676-08:00",
  "b": "two",
  "a": "1"
}

Finished
[
  {
    "snippet-format": "snippet",
    "total": 2,
    "start": 1,
    "page-length": 0,
    "results": [],
    "metrics": {
      "query-resolution-time": "PT0.001552S",
      "facet-resolution-time": "PT0.000141S",
      "snippet-resolution-time": "PT0S",
      "total-time": "PT0.165583S"
    },
    "queryTimestamp": "2015-01-02T11:10:00.208708-08:00"
  }
]

slice(qb.transform('js-query-transform', {a: 1, b: 'two'}))

if (result.hasOwnProperty('snippet-format')) {
  // search result summary
  result.queryTimestamp = fn.currentDateTime();
} 

// Example Read Transform
//
// If the input is a JSON document:
// - Add a readTimestamp to the result document.
// - For each parameter passed in by the client, add a
//   property of the form: propName: propValue.
// Other document types are unchanged.
function readTimestamp(context, params, content)
{
  //if (context.inputType.search('json') >= 0) {
  if (context.inputType.search('json') >= 0) {
    const result = content.toObject();

    result.readTimestamp = fn.currentDateTime();

    // Add a property for each caller-supplied request param
    for (const pname in params) {
      if (params.hasOwnProperty(pname)) {
        result[pname] = params[pname];
      }
    }
    return result;
  } else {
    // Pass thru for non-JSON documents
    return content;
  }
};

exports.transform = readTimestamp;

// Example Write Transform
//
// If the input is a JSON document:
// - Add a writeTimestamp to the document.
// - For each parameter passed in by the client, add a property
//   of the form "propName: propValue" to the document.
// Non-JSON documents are returned unmodified.
function writeTimestamp(context, params, content)
{
  if (context.inputType.search('json') >= 0) {
    const result = content.toObject();
    result.writeTimestamp = fn.currentDateTime();

    // Add a property for each caller-supplied request param
    for (const pname in params) {
      if (params.hasOwnProperty(pname)) {
        result[pname] = params[pname];
      }
    }
    return result;
  } else {
    // Pass thru for non-JSON documents
    return content;
  }
};

exports.transform = writeTimestamp;

// When applied to a query operation, a transform is invoked on
// both the search result summary and the matching documents (when
// used as a multi-document read).
//
// The transform does the following:
// - For a JSON search result summary (determined by the presence
//   of a search-snippet property), add a queryTimestamp property.
// - For a JSON document, add a readTimestamp property.
// - For all other input, pass it through unchanged.
function queryTimestamp(context, params, content)
{
  if (context.inputType.search('json') >= 0) {
    const result = content.toObject();
    if (result.hasOwnProperty('snippet-format')) {
      // search result summary
      result.queryTimestamp = fn.currentDateTime();
    } else {
      // JSON document. Add readTimestamp property plus a property
      // for each param passed in by the client.
      result.readTimestamp = fn.currentDateTime();
      for (const pname in params) {
        if (params.hasOwnProperty(pname)) {
          result[pname] = params[pname];
        }
      }
    }
    return result;
  } else {
    // Pass thru for non-JSON documents or XML search summary
    return content;
  }
};

exports.transform = queryTimestamp;

Discovering Installed Transforms

You can retrieve the names and metadata for installed transforms using DatabaseClient.transforms.list. You must have the read-reader role or equivalent privileges to retrieve the list of installed transforms.

The following example retrieves the list of installed transforms and displays the response on the console.

const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.config.transforms.list().result(
function(response) {
  console.log('Installed transforms: ');
  console.log(JSON.stringify(response, null, 2));
}, function(error) {
  console.log(JSON.stringify(error, null, 2));
});

If you have installed the transform from Installing a Transformation, then running the above script produces output similar to the following:

{ "transforms": {
    "transform": [ {
        "name": "js-transform",
        "source-format": "javascript",
        "title": "Example JavaScript Transform",
        "version": "1",
        "provider-name": "MarkLogic",
        "description": "An example of an SJS read/write transform",
        "transform-parameters": "",
        "transform-source": "/v1/config/transforms/js-transform"
    } ]
} }

For additional examples, see test-basic/documents-transform.js in the Node.js Client API GitHub project.

Deleting a Transformation

Use DatabaseClient.transforms.remove to uninstall a transform on MarkLogic Server. The uninstall operation is idempotent. That is, the results are the same whether or not the named transform is installed.

You must have the rest-admin role or equivalent privileges to uninstall a transform.

The following script uninstalls a transform named js-transform.

const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.config.transforms.remove('js-transform').result(
  function(response) {
    console.log('Removed transform: ', response.name);
  }, 
  function(error) {
    console.log(JSON.stringify(error, null, 2));
  });

Example: Reporting Errors in JavaScript

To return an error to the client application from a JavaScript extension or transform, use fn.error to report a RESTAPI-SRVEXERR error and provide additional information in the data parameter of fn.error. You can control the response status code and status message, and provide an additional error reporting response payload. For example, you can return an error to the client in the following way:

fn.error(null, 'RESTAPI-SRVEXERR', 
  Sequence.from([400, 'Bad Request', 
                 'Insufficient number of uri basenames.']));
  // unreachable - control does not return from fn.error

The 3rd parameter to fn.error should be a sequence of the form (status-code, 'status-message', 'payload-format', 'response-payload'). That is, when using fn.error to raise RESTAPI-SRVEXERR, the data parameter to fn.error is sequence containing the following items, all optional:

• HTTP status code. Default: 400.
• HTTP status message. Default: Bad Request.
• Response payload. It is best to restrict this to text as the payload may be in JSON or XML, depending on the REST API instance configuration.

  Best practice is to use RESTAPI-SRVEXERR. If you report any other error or raise any other exception, it is reported to the calling application as a 500 Server Internal Error.

You can use xdmp.arrayValues or Sequence.from to construct a sequence from a JavaScript array.

Control does not return from fn.error. You should perform any necessary cleanup or other tasks prior to calling it.

You can use a utility function similar to the following to abstract most of the details away from your extension implementation:

function returnErrToClient(statusCode, statusMsg, body)
{
  fn.error(null, 'RESTAPI-SRVEXERR', 
           Sequence.from([statusCode, statusMsg, body]));
  // unreachable
};

The following is an example of using this function:

returnErrToClient(400, 'Bad Request', 
  'Insufficient number of uri basenames.');

If errors from an extension invocation are trapped as follows using the Node.js API:

db.resources.put({
  ...
}).result(function(response) {
  console.log(JSON.stringify(response, null, 2));
}, function(error) {
  console.log(JSON.stringify(error, null, 2));
});

Then the output is similar to the following:

{
  "message": "js-example: response with invalid 400 status",
  "statusCode": 400,
  "body": {
    "errorResponse": {
      "statusCode": 400,
      "status": "Bad Request",
      "messageCode": "RESTAPI-SRVEXERR",
      "message": "Insufficient number of uri basenames."
    }
  }
}

For a working example, see Example: Installing and Using a Resource Service Extension.

Example: Reporting Errors in XQuery

Use fn:error to report a RESTAPI-SRVEXERR error, and provide additional information in the $data parameter of fn:error. You can control the response status code, status message, and provide an additional error reporting response payload. For example, you can return an error to the client in the following way:

fn:error((),"RESTAPI-SRVEXERR", 
  (415, "Unsupported Input Type", 
   "Only application/xml is supported"))

The 3rd parameter to fn:error should be a sequence of the form ("status-code", "status-message", "response-payload"). That is, when using fn:error to raise RESTAPI-SRVEXERR, the $data parameter to fn:error is a sequence with the following members, all optional:

• HTTP status code. Default: 400.
• HTTP status message. Default: Bad Request.
• Response payload. It best to limit this to text as the payload can be either JSON or XML, depending on the REST API instance configuration.

  Best practice is to use RESTAPI-SRVEXERR. If you report any other error or raise any other exception, it is reported to the calling application as a 500 Server Internal Error.

For example, this resource extension function raises RESTAPI-SRVEXERR if the input content type is not as expected:

declare function example:put(
    $context as map:map,
    $params  as map:map,
    $input   as document-node()
) as document-node()
{
    (: get 'input-types' to use in content negotiation :)
    let $input-types := map:get($context,"input-types")
    let $negotiate :=
        if ($input-types = "application/xml")
        then () (: process, insert/update :)
        else fn:error((),"RESTAPI-SRVEXERR",
          ("415", "Raven", "nevermore"))
    return document { "Done"}  (: may return a document node :)
};

If a PUT request is made to the extension with an unexpected content type, the fn:error call causes the request to fail with a status 415 and to include the additional error description in the response body:

HTTP/1.1 415 Raven
Content-type: application/xml
Server: MarkLogic
Set-Cookie: SessionID=714070bdf4076536; path=/
Content-Length: 62
Connection: close
<?xml version="1.0" encoding="UTF-8"?>
<word>nevermore</word>

Required Privileges

Using DatabaseClient.eval, DatabaseClient.xqueryEval, and DatabaseClient.invoke requires additional privileges, beyond those required for normal read/write/query operations using the Node.js Client API.

To use DatabaseClient.eval or DatabaseClient.xqueryEval, you must have at least the following privileges or their equivalent:

• http://marklogic.com/xdmp/privileges/xdmp-eval
• http://marklogic.com/xdmp/privileges/xdmp-eval-in
• http://marklogic.com/xdmp/privileges/xdbc-eval
• http://marklogic.com/xdmp/privileges/xdbc-eval-in

To use DatabaseClient.invoke, you must have at least the following privileges or their equivalent:

• http://marklogic.com/xdmp/privileges/xdmp-invoke
• http://marklogic.com/xdmp/privileges/xdmp-invoke-in
• http://marklogic.com/xdmp/privileges/xdbc-invoke
• http://marklogic.com/xdmp/privileges/xdbc-invoke-in

The privileges listed above merely make it possible to eval/invoke server-side code. The operations performed by that code may require additional privileges.

Evaluating a Ad-Hoc Query

Use DatabaseClient.eval to evaluate an ad-hoc block of JavaScript on MarkLogic Server. You must use the MarkLogic server-side JavaScript dialect described in the JavaScript Reference Guide. To evaluate an ad-hoc block of XQuery, use DatabaseClient.xqueryEval. The calling and response conventions are the same for both eval and xqueryEval. These operations are equivalent to using the xdmp.eval (JavaScript) or xdmp:eval (XQuery) builtin function. The code is evaluated in the context of the database associated with the DatabaseClient object.

Using eval or xqueryEval requires extra security privileges; for details, see Required Privileges.

You can call eval and xqueryEval using one of the following forms. The code is the only required parameter/property.

db.eval(codeAsString, externalVarsObj)
db.eval({source: codeAsString, variables: externalVarsObj, txid:...})

db.xqueryEval(codeAsString, externalVarsObj)
db.xqueryEval({
  source: codeAsString, 
  variables: externalVarsObj, 
  txid:...})

External variables enable you to pass variable values to MarkLogic Server, where they're substituted into your ad-hoc code. For details, see Specifying External Variable Values.

For example, suppose you want to evaluate the following JavaScript code, where word1 and word2 are external variable values supplied by your application:

word1 + " " + word2

Then the following call evaluates the code on MarkLogic Server. The values for word1 and word2 are passed to MarkLogic Server through the second parameter.

db.eval('word1 + " " + word2', {word1: 'hello', word2: 'world'})

The response from calling eval is an array containing an item for each value returned by the code block. Each item contains the returned value, plus type information to help you interpret the value. For details, see Interpreting the Results of Eval or Invoke.

For example, the above call returns the following response.

[{
  "format": "text",
  "datatype": "string",
  "value": "hello world"
}]

You can return documents, objects, and arrays as well as atomic values. To return multiple items, you must return either a Sequence (JavaScript only) or a sequence. You can construct a Sequence from an array-like or generator, and many builtin functions return multiple values return a Sequence. To construct a sequence in server-side JavaScript, apply xdmp.arrayValues or Sequence.from to a JavaScript array.

For example, to extend the previous example to return the combined lenght of the two input values as well as the concatenated string, accumulate the results in an array and then apply Sequence.from to the array:

Sequence.from([word1.length + word2.length, word1 + " " + word2])

The following script evaluates the above code. The response contains 2 array items: One for the length and one for concatenated string.

const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.eval(
   'Sequence.from([word1.length + word2.length, word1 + " " + word2])',
   {word1: 'hello', word2: 'world'}
).result(function(response) {
    console.log(JSON.stringify(response, null, 2));
}, function(error) {
    console.log(JSON.stringify(error, null, 2));
});

Running the script produces the following output:

[
  {
    "format": "text",
    "datatype": "integer",
    "value": 10
  },
  {
    "format": "text",
    "datatype": "string",
    "value": "hello world"
  }
]

The following script uses DatabaseClient.xqueryEval to evaluates a block XQuery that performs the same operations as the previous JavaScript eval. The output is exactly as before. Note that in XQuery you must explicitly declare the external variables in your ad-hoc code.

const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.xqueryEval(
  'xquery version "1.0-ml";' +
  'declare variable $word1 as xs:string external;' +
  'declare variable $word2 as xs:string external;' +
  '(fn:string-length($word1) + fn:string-length($word2),' +
  ' concat($word1, " ", $word2))',
  {word1: 'hello', word2: 'world'}
).result(function(response) {
    console.log(JSON.stringify(response, null, 2));
}, function(error) {
    console.log(JSON.stringify(error, null, 2));
});

For more examples, see test-basic/server-exec.js in the Node.js Client API source project on GitHub.

Invoking a Module Installed on MarkLogic Server

You can use DatabaseClient.invoke to an XQuery or server-side JavaScript module installed on MarkLogic Server. This is equivalent to calling the builtin server function xdmp.invoke (JavaScript) or xdmp:invoke (XQuery). Using invoke requires extra security privileges; for details, see Required Privileges.

The module you invoke must already be installed on MarkLogic Server. You can install your module in the modules database associated with your REST API instance using DatabaseClient.config.extlibs.write or an equivalent operation. For details, see Managing Assets in the Modules Database.

Installing a module using DatabaseClient.config.extlibs.write adds a /ext/. prefix to the path. Omit the prefix when using the config.extlibs interface, but include it in your module path when calling invoke.

When installing the module, you must include the module path, content type, and source code. For a JavaScript module, set the content type to application/vnd.marklogic-javascript and set the file extension in your module path to .sjs. For an XQuery module, set the content type to application/xquery and set the file extension in your module path to .xqy. See the example below.

You can use external variables to pass arbitrary values to your module at runtime. For details, see Specifying External Variable Values.

The response to invoke is an array containing one item for each value returned by the invoked module. For details, see Interpreting the Results of Eval or Invoke.

The following example installs a JavaScript module on MarkLogic Server and then uses DatabaseClient.invoke to evaluate it.

const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

// (1) Install the module in the modules database
//     Note: You do not need to install on every invocation.
//     It is included here to make the example self-contained.
db.config.extlibs.write({
  path: '/invoke/example.sjs',
  contentType: 'application/vnd.marklogic-javascript',
  source: 'Sequence.from([word1, word2, word1 + " " + word2])'
}).result().then(function(response) {
  console.log('Installed module: ' + response.path);

  // (2) Invoke the module
  return db.invoke({
    path: '/ext/' + response.path, 
    variables: {word1: 'hello', word2: 'world'}
  }).result(function(response) {
    console.log(JSON.stringify(response, null, 2));
  }, function(error) {
    console.log(JSON.stringify(error, null, 2));
  });
}, function(error) {
    console.log(JSON.stringify(error, null, 2));
});

If you save the script to a file and run it, you should see results similar to the following:

[
  {
    "format": "text",
    "datatype": "string",
    "value": "hello"
  },
  {
    "format": "text",
    "datatype": "string",
    "value": "world"
  },
  {
    "format": "text",
    "datatype": "string",
    "value": "hello world"
  }
]

To install an equivalent XQuery module, use a call similar to the following:

db.config.extlibs.write({
  path: '/invoke/example.xqy',
  contentType: 'application/xquery',
  source: 
    'xquery version "1.0-ml";' +
    'declare variable $word1 as xs:string external;' +
    'declare variable $word2 as xs:string external;' +
    '($word1, $word2, fn:concat($word1, " ", $word2))'
})

Interpreting the Results of Eval or Invoke

When you evaluate or invoke server-side code using DatabaseClient.eval, DatabaseClient.xqueryEval, or DatabaseClient.invoke, the response is always an array containing an item for each value returned by the server.

Each item contains information that helps your application interpret the value. Each item has the following form, where format and value are always present, but datatype is not.

{ 
  format: 'text' | 'json' | 'xml' | 'binary'
  datatype: string
  value: ...
}

The datatype property can be a node type, an XSD datatype, or any other server type, such as cts:query. The reported type may be more general than the actual type. Types derived from anyAtomicType include anyURI, boolean, dateTime, double, and string. For details, see http://www.w3.org/TR/xpath-functions/#datatypes.

The table below summarizes how the representation of the data in the value property is determined.

json

node()

text

xml

node()

binary

For example, an atomic value (anyAtomicType, a type derived from anyAtomicType, or an equivalent JavaScript type) has a datatype property that can specify an explicit type such as integer, string, or date.

If your code or module returns JSON (or a Javascript object or array), then value is a parsed JavaScript object or array. For example:

db.eval('const result = {number: 42, phrase: "hello"}; result;')

==>
[ { format: 'json',
    datatype: 'node()',
    value: { number: 42, phrase: 'hello' } 
} ]

Specifying External Variable Values

You can pass values to an ad-hoc query (or invoked module) at runtime using external variables. Specify external variables to your eval and invoke calls using a JavaScript object of the following form. The values must be JavaScript primitives.

{ varName1: varValue1, varName2: varValue2, ... }

For example, the following object supplies values for two external variables, named word1 and word2:

{ word1: 'hello', word2: 'world' }

If you're evaluating or invoking XQuery code, you must declare the variables explicitly in the ad-hoc query or module. For example, the folloiwng prolog declares two external variables whose values can be supplied by the above parameter object:

xquery version "1.0-ml";
declare variable $word1 as xs:string external;
declare variable $word2 as xs:string external;
...

If you're evaluating or invoking XQuery code that depends on variables in a namespace, use Clark notation on the variable name. That is, specify the name using notation of the form {namespaceURI}name.

For example, the following script uses a namspace qualified external variable, $my:who. The external variable input parameter uses the fully qualified variable in Clark notation: {'{http://example.com}who':'world'}.

const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.xqueryEval(
  'xquery version "1.0-ml";' +
  'declare namespace my = "http://example.com";' +
  'declare variable $my:who as xs:string external;' +
  'fn:concat("hello ", $my:who)',
  {'{http://example.com}who' : 'world'}
).result(function(response) {
    console.log(JSON.stringify(response, null, 2));
}, function(error) {
    console.log(JSON.stringify(error, null, 2));
});

Overview of Asset Management

Your Node.js Client API application can use several kinds of user-defined code that is stored in the modules database associated with your REST API instance, including transforms, resource service extension implementations, constraint binding parsers, custom snippet generators, and patch content generators.

Most of these asset classes have specialized management interfaces, such as DatabaseClient.config.resources and DatabaseClient.config.query.snippet. These interfaces abstract away the details of where and how the API manages the assets. You generally should not manage such assets through another, more general interface. Assets which do not have a specialized interface can be managed using the DatabaseClient.config.extlibs interface.

The table below summarizes the asset management interfaces available through the Node.js Client API.

DatabaseClient.config.extlibs

DatabaseClient.config.patch.replace

DatabaseClient.config.query.custom

DatabaseClient.config.query.snippet

DatabaseClient.config.resources

DatabaseClient.config.transforms

All the asset management interfaces offer the same basic set of methods, customized to suit a given asset classet:

• write: Install an asset in the modules database.
• read: Retrieve an asset from the modules database.
• list: Retrieve a list of all assets of a given class from the modules database, such as all resource service extensions or all facet generators.
• remove: Remove an asset from the modules database.

You should not mix and match interfaces among asset classes. For example, you should not install a snippeter using DatabaseClient.config.query.snippet.write and then delete it using DatabaseClient.config.extlibs.remove. You can manage assets through the equivalent interfaces of the other client APIs, such as the Java Client API and the REST Client API.

When you install or update an asset in the modules database, the asset is replicated across your cluster automatically. There can be a delay of up to one minute between update and availability.

MarkLogic Server does not automatically remove dependent assets when you delete the related extension or transform.

Since dependent assets are installed in the modules database, they are removed when you remove the REST API instance if you include the modules database in the instance teardown. For details, see Removing an Instance in the REST Application Developer's Guide.

Installing or Updating an Asset

This section describes how to install or update an asset that is not covered by a specialized asset management interface, such as a dependent library or a module to be invoked using DatabaseClient.invoke. For other asset classes, use the write method of the specialized interface. For a list of the specialized interfaces, see Overview of Asset Management.

Use DatabaseClient.config.extlibs.write to install or update an asset in the modules database associated with your REST API instance. You must provide a module path, content type, and the asset contents. You can insert assets into the modules database as JSON, XML, text, or binary documents. MarkLogic Server determines the document format. The document type is determined by the content type or the module path URI file extension and the server MIME type mappings.

The module path you provide is prepended with /ext/ during installation. You can omit the prefix when manipulating the asset using the extlibs interface, but you should include when you reference the module elsewhere, such as in a resource service extension require statement that uses an absolute path or when invoking a module with using DatabaseClient.invoke.

The following example installs a module whose contents are read in from a file. The module is installed in the modules database with the URI /ext/extlibs/example.sjs.

const fs = require('fs');
const marklogic = require('marklogic');
const db = marklogic.createDatabaseClient(my.connInfo);

...
db.config.extlibs.write({
  path: '/extlibs/example.sjs',
  contentType: 'application/vnd.marklogic-javascript',
  source: fs.createReadStream('./example.sjs')
})...

For additional examples, see Invoking a Module Installed on MarkLogic Server or test-basic/extlibs.js in the marklogic/node-client-api project on GitHub.

Referencing an Asset from Server-Side Code

To use a dependent library installed with DatabaseClient.extlibs.write from an extension, transform, or invoked module, use the same URI under which you installed the dependent library, including the /ext/ prefix.

For example, if a dependent asset is installed with using db.config.extlibs.write({path: '/my/domain/lib/myasset', ...}), then its URI in the modules database is /ext/my/domain/myasset.

A JavaScript extension, transform, or invoked module using this asset can refer to it as follows:

const myDep = require('/ext/my/domain/lib/myasset');

An XQuery extension, transform, or invoked module using this library can include an import of the following form:

import module namespace dep="mylib" at "/ext/my/domain/lib/myasset";

Removing an Asset

Use DatabaseClient.config.extlibs.remove to delete an asset from the modules database if it was installed using DatabaseClient.config.extlibs.write. For assets with specialized interfaces, such as extensions and tranforms, use the remove method of the specialized interface, such as DatabaseClient.config.resources.remove.

Removing an asset is an idempotent operation. That is, it returns the same response whether the asset exists or not.

To remove all the assets in a given directory, supply the containing directory name instead of a specific asset path.

For example, if an asset is installed as follows:

db.config.extlibs.write({
  path: '/invoke/example.sjs',
  contentType: 'application/vnd.marklogic-javascript',
  source: ...
})

Then you can remove that single asset with a call similar to the following:

db.config.extlibs.remove('/invoke/example.sjs');

To remove all the assets installed under /ext/invoke/ instead, use a call similar to the following:

db.config.extlibs.remove('/invoke/');

Retrieving an Asset List

Use DatabaseClient.config.extlibs.list to retrieve a list of assets installed using DatabaseClient.config.extlibs.write. For assets with specialized interfaces, such as extensions and tranforms, use the list method of the specialized interface, such as DatabaseClient.config.transforms.list.

The response has the following format:

{ "assets": [
    { "asset": "/ext/invoke/example.sjs" },
    { "asset": "/ext/util/dep.sjs" },
    { "asset": assetModulePath }, ...
]}

Retrieving an Asset

Use DatabaseClient.config.extlibs.read to retrieve an asset installed using DatabaseClient.config.extlibs.write. For assets with specialized interfaces, such as extensions and tranforms, use the read method of the specialized interface, such as DatabaseClient.config.transforms.read.

Retrieve the asset using the same module path you used to install it. For example:

db.config.extlibs.read('/invoke/example.sjs')