Loading TOC...
REST Application Developer's Guide (PDF)

REST Application Developer's Guide — Chapter 6

Working With Content Transformations

The MarkLogic REST API allows you to create custom content transformations and apply them during operations such as document ingestion and retrieval. Transforms can be implemented as XQuery library modules or XSLT stylesheets, and can accept transform-specific parameters. Transforms can be applied on any MarkLogic REST API service that accepts a transform parameter.

This chapter covers the following topics:

Writing Transformations

This section covers the following topics:

Guidelines for Writing Transforms

Keep the following guidelines in mind when authoring a transform:

  • A transform is one step in a document processing pipeline controlled by the MarkLogic REST API framework. As such, transforms typically should not have side-effects.
  • A transform operates on an in-memory input document. MarkLogic Server builtin functions that operate on in-database documents, such as xdmp:node-replace, cannot be applied directly to the input document. To modify the structure of the input document, perform the modifications during construction of a copy of the input content.
  • A transform executes in the transaction context of the HTTP request on whose behalf it is called.
  • A transform should not alter the HTTP request context. For example, you should not set the response code or headers directly. The MarkLogic REST API framework manages the request context.
  • Report errors by throwing one of the exceptions provided by the MarkLogic REST API. For details, see Reporting Errors.

Writing XQuery Transformations

To create an XQuery transformation, implement an XQuery library module in the namespace

http://marklogic.com/rest-api/transform/yourTransformName

Where yourTransformName is the name used to refer to the transform in subsequent REST requests, such as installing and applying the transform.

The module must implement a public function called transform with the following interface:

declare function yourNS:transform(
  $context as map:map,
  $params as map:map,
  $content as document-node())
as document-node()

The table below describes the parameters:

Parameter Description
$context Service request context information such as input document type and URI, and output types accepted by the caller. For details, see Context Map Keys.
$params Transformation specific parameters. The map contains a key for each transform parameter passed to the request for which the transform is applied. Define parameters when installing the transformation. For details, see Installing Transformations.
$content The input document to which to apply the transformation.

The map:map type is a MarkLogic Server extended XQuery type representing a key-value map. For details, see Using the map Functions to Create Name-Value Maps in the Application Developer's Guide.

For a complete example, see XQuery Example: Adding an Attribute During Ingestion.

Writing XSLT Transformations

To create an XSLT transformation, implement an XSLT stylesheet that accepts the following parameters:

Parameter Type Description
$context map:map Service request context information such as input document type and URI, and output types accepted by the caller. For details, see Context Map Keys.
$params map:map Transformation specific parameters. The map contains a key for each transform parameter passed to the request for which the transform is applied. Define any parameters when installing the transformation. For details, see Installing Transformations.

The map:map type is a MarkLogic Server extended XQuery type representing a key-value map. For details, see Using the map Functions to Create Name-Value Maps in the Application Developer's Guide.

To use the map type in your stylesheet, include the namespace http://marklogic.com/xdmp/map. Reference the map contents using the map:get XQuery function.

The following example illustrates declaring the map namespace and $context and $params parameters, and referencing values in the $params map:

<xsl:stylesheet version="2.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:example="http://marklogic.com/rest-api/example/transform"
    xmlns:map="http://marklogic.com/xdmp/map">
<xsl:param name="context" as="map:map"/>
<xsl:param name="params"  as="map:map"/>
<xsl:template match="/*">
  <xsl:copy>
    <xsl:attribute name='{(map:get($params,"name"),"transformed")[1]}'
      select='(map:get($params,"value"),"UNDEFINED")[1]'/>
...

For a complete example, see XSLT Example: Adding an Attribute During Ingestion.

Reporting Errors

This section discusses proper error reporting in your transform or service resource extension implementation. Use the approach described here for both XQuery and XSLT implementations. For more information about error reporting conventions, see Error Reporting.

To report an error from your implementation, use the fn:error XQuery function with the RESTAPI-EXTNERR error code, and provide the HTTP response code and additional error information in the 3rd parameter to fn:error.

The 3rd parameter to fn:error should be a sequence of the form ("status-code", "status-message", "payload-format", "response-payload"). That is, when using fn:error to raise REST-EXTNERR, the $data parameter to fn:error is a sequence with the following members, all optional:

  • HTTP status code. Default: 400.
  • HTTP status message. Default: Bad Request.
  • Payload Format. Allowed values: xml or json. If there is a payload format, there must be a payload.
  • Response payload.

    Best practice is to use RESTAPI-EXTNERR. If you report any other error or raise any other exception, it is reported to the calling application as a 500 Server Internal Error.

For example, this resource extension function raises RESTAPI-EXTNERR if the input content type is not as expected:

declare function example:put(
    $context as map:map,
    $params  as map:map,
    $input   as document-node()*
) as document-node()?
{
    (: get 'input-types' to use in content negotiation :)
    let $input-types := map:get($context,"input-types")
    let $negotiate :=
        if ($input-types = "application/xml")
        then () (: process, insert/update :)
        else fn:error((),"RESTAPI-EXTNERR",
          ("415","Raven","xml",xdmp:quote(<word>nevermore</word>)))
    return document { "Done"}  (: may return a document node :)
};

If a PUT request is made to the extension with an unexpected content type, the fn:error call causes the request to fail with a status 415 and to include the additional error description in the response body:

HTTP/1.1 415 Raven
Content-type: application/xml
Server: MarkLogic
Set-Cookie: SessionID=714070bdf4076536; path=/
Content-Length: 62
Connection: close
<?xml version="1.0" encoding="UTF-8"?>
<word>nevermore</word>

Context Map Keys

The context map parameter made available to XQuery and XSLT transformations can contain the following keys:

Key Description
input-type The MIME type of the input document.
uri The URI of the input document.
accept-types The MIME types accepted by the requestor. Multiple MIME types are separated by whitespace.
output-type The expected output document MIME type. If the transform supports content negotiation, the transform function can set/modify this value.

Installing Transformations

This section covers the following topics:

Basic Installation

To install or update a transformation, send a PUT request with a URL of the following form, with the transform code in the request body:

http://host:port/version/config/transforms/yourTransformName

Where yourTransformName is the name used when administering and applying the transformation.

If you are installing an XQuery transform, yourTransformName must match the leaf name in the transform module namespace declaration. For example:

xquery version "1.0-ml";
module namespace example = 
  "http://marklogic.com/rest-api/transform/yourTransformName";
...

Set the Content-type of the request to application/xquery for an XQuery transformation and application/xslt+xml for an XSLT transformation.

If your transform requires additional user-defined XQuery library modules, you must install them separately before installing your transform. For details, see Installing Dependent XQuery Library Modules.

For a complete example, see XSLT Example: Adding an Attribute During Ingestion or XQuery Example: Adding an Attribute During Ingestion.

Defining Transform-Specific Parameters

If the transform expects parameters, include the parameters as request parameters of the following form:

trans:paramName=paramTypeAndCardinality

Where paramName is the name of the parameter and paramTypeAndCardinality reflects the type and cardinality the parameter would have in an XQuery function declaration.

For example, 'string?' means a string type parameter which may appear 0 or 1 time. The following URL illustrates a transform called 'example' with an optional string parameter named 'reviewer':

http://host:port/version/config/transforms/example?trans:reviewer=string\?

Including Transform Metadata

You can include metadata for provider, version, title, and description as request parameters. For example:

http://host:port/version/config/transforms/example?trans:reviewer=string\?&provider=MarkLogic%20Corporation

Metadata can be retrieved by sending a GET request to the /config/transforms service. See Discovering Transformations.

Applying Transformations

All MarkLogic REST API services that support transformations use the following mechanism for specifying application of a transformation:

  1. Specify the name of an installed transform as the value of the transform request parameter:
    transform=transformName
  2. If the transform expects parameters, specify each parameter with a request parameter of the form:
    trans:paramName=value

For example, to apply a transform during document insertion, you can send a PUT request to the /documents service with a transform request parameter. Assuming the transform called 'example' with one parameter named 'reviewer', the PUT request URL might look like the following:

http://host:port/version/documents?uri=/doc/theDoc.xml&transform=example&trans:reviewer=me

You could apply the same transformation during document retrieval by sending a GET request to the /documents service with the same URL.

For a complete example, see XSLT Example: Adding an Attribute During Ingestion or XQuery Example: Adding an Attribute During Ingestion.

Discovering Transformations

To discover the name, interface, and other metadata about installed transformations, send a GET request to the /config/transforms service of the form:

http://host:port/version/config/transforms

MarkLogic Server returns a summary of the installed transforms in XML or JSON. The default summary format is XML. Use the Accept headers or the format request parameter to explicitly specify expected content type, as described in Controlling Output Content Type.

The following example requests an XML report:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    http://localhost:8003/v1/config/transforms?format=xml
...
HTTP/1.1 200 Transform List Retrieved
Content-type: application/xml
Server: MarkLogic
Content-Length: 348
Connection: close

<?xml version="1.0" encoding="UTF-8"?>
<rapi:transforms xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:transform>
    <rapi:name>add-attr-xsl</rapi:name>
    <rapi:title/>
    <rapi:version/>
    <rapi:provider-name>MarkLogic Corporation</rapi:provider-name>
    <rapi:description/>
    <rapi:transform-parameters>
      <rapi:parameter>
        <rapi:parameter-name>value</rapi:parameter-name>
        <rapi:parameter-type>string?</rapi:parameter-type>
      </rapi:parameter>
      <rapi:parameter>
        <rapi:parameter-name>name</rapi:parameter-name>
        <rapi:parameter-type>string?</rapi:parameter-type>
      </rapi:parameter>
    </rapi:transform-parameters>
    <rapi:transform-source>
      /v1/transforms/add-attr
    </rapi:transform-source>
  </rapi:transform>
</rapi:transforms>

Retrieving the Implementation of a Transformation

To retrieve the XQuery code or XSLT stylesheet that implements a transformation, send a GET request to the /config/transforms/{name} service of the form:

http://host:port/version/config/transforms/transformName

Where transformName is the name under which the extension is installed.

Set the request Accept header to application/xquery for an XQuery transform. Set the request Accept header to application/xslt+xml for an XSLT transform.

MarkLogic Server returns the XQuery library module or XSLT stylesheet that implements the transform in the response body. For example:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/xquery" \
    http://localhost:8003/v1/config/transforms/example
...
xquery version "1.0-ml";
module namespace example="http://marklogic.com/rest-api/transform/example";
declare function example:transform( ...

XQuery Example: Adding an Attribute During Ingestion

This example demonstrates using a transformation to add an attribute to the root node of an XML document during ingestion. The name and value of the added attribute are passed in as parameters to the transform function. You can use the same transform for document retrieval by applying it to a GET request instead of a PUT.

The transformation XQuery module shown below reads the attribute name and value from parameters, and then makes a copy of the input document, adding the requested attribute to the root node. The transformation cannot perform direct transformations on the input XML, such as calling xdmp:node-replace, because the document has not yet been inserted into the database when the transform function runs.

xquery version "1.0-ml";
module namespace example =
  "http://marklogic.com/rest-api/transform/add-attr";

declare function example:transform(
  $context as map:map,
  $params as map:map,
  $content as document-node()
) as document-node()
{
  if (fn:empty($content/*)) then $content
  else
    let $value := (map:get($params,"value"),"UNDEFINED")[1]
    let $name := (map:get($params, "name"), "transformed")[1]
    let $root  := $content/*
    return document {
      $root/preceding-sibling::node(),
      element {fn:name($root)} {
        attribute { fn:QName("", $name) } {$value},
        $root/@*,
        $root/node()
      },
      $root/following-sibling::node()
    }
};

To exercise this example:

  1. Create a MarkLogic REST API instance on port 8003. For instructions, see Creating an Instance.
  2. Cut and paste the code above into a file called 'add-attr.xqy'.
  3. Make a PUT request to the /config/transforms/{name} service to install the transform as 'add-attr', accepting 2 string parameters called 'name' and 'value'. For example:
    # Windows users, see Modifying the Example Commands for Windows 
    $ curl --anyauth --user user:password -X PUT -d@"./add-attr.xqy" \
        -H "Content-type: application/xquery" \
        'http://localhost:8003/v1/config/transforms/add-attr?trans:name=string\?&trans:value=string\?'
  4. Optionally, verify installation of the transformation by making a GET request to the /config/transforms service. For example:
    $ curl --anyauth --user user:password -X GET \
        http://localhost:8003/v1/config/transforms?format=xml
    ...
    HTTP/1.1 200 Transform List Retrieved
    Content-type: application/xml
    Server: MarkLogic
    Content-Length: 348
    Connection: close
    
    <?xml version="1.0" encoding="UTF-8"?>
    <rapi:transforms xmlns:rapi="http://marklogic.com/rest-api">
      <rapi:transform>
        <rapi:name>add-attr</rapi:name>
        <rapi:title/>
        <rapi:version/>
        <rapi:provider-name/>
        <rapi:description/>
        <rapi:transform-parameters>
          <rapi:parameter>
            <rapi:parameter-name>value</rapi:parameter-name>
            <rapi:parameter-type>string?</rapi:parameter-type>
          </rapi:parameter>
          <rapi:parameter>
            <rapi:parameter-name>name</rapi:parameter-name>
            <rapi:parameter-type>string?</rapi:parameter-type>
          </rapi:parameter>
        </rapi:transform-parameters>
        <rapi:transform-source>
          /v1/transforms/add-attr
        </rapi:transform-source>
      </rapi:transform>
    </rapi:transforms>
    
  5. Make a PUT request to the /documents service to insert a document into the database, using the 'add-attr' transform to add the attribute 'example' with a value of 'new'. For example:
    $ curl --anyauth --user user:password -X PUT -d"<data/>" \
        'http://localhost:8003/v1/documents?uri=/doc/transformed.xml&transform=add-attr&trans:name=example&trans:value=new'
  6. Make a GET request to the /documents service to retrieve the inserted document. Note the added 'example' attribute. For example:
    $ curl --anyauth --user user:password -X GET \
        http://localhost:8003/v1/documents?uri=/doc/transformed.xml
    <?xml version="1.0" encoding="UTF-8"?>
    <data example="new"/>

XSLT Example: Adding an Attribute During Ingestion

This example demonstrates using a transformation to add an attribute to the root node of an XML document during ingestion. The name and value of the added attribute are passed in as parameters to the stylesheet. You can use the same transform for document retrieval by applying it to a GET request instead of a PUT.

The transformation stylesheet shown below reads the attribute name and value from parameters, and then makes a copy of the input document, adding the requested attribute to the root node.

<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:example="http://marklogic.com/rest-api/example/transform"
    xmlns:map="http://marklogic.com/xdmp/map">
<xsl:param name="context" as="map:map"/>
<xsl:param name="params"  as="map:map"/>
<xsl:template match="/*">
    <xsl:copy>
        <xsl:attribute name='{(map:get($params,"name"),"transformed")[1]}'
            select='(map:get($params,"value"),"UNDEFINED")[1]'/>
        <xsl:copy-of select="@*"/>
        <xsl:copy-of select="node()"/>
    </xsl:copy>
</xsl:template>
</xsl:stylesheet>

To exercise this example:

  1. Create a MarkLogic REST API instance on port 8003. For instructions, see Creating an Instance.
  2. Cut and paste the stylesheet above into a file called 'add-attr.xsl'.
  3. Make a PUT request to the /config/transforms/{name} service to install the transform as 'add-attr-xsl', accepting 2 string parameters called 'name' and 'value'. For example:
    # Windows users, see Modifying the Example Commands for Windows 
    $ curl --anyauth --user user:password -X PUT -d@"./add-attr.xsl" \
        -H "Content-type: application/xslt+xml" \
        'http://localhost:8003/v1/config/transforms/add-attr-xsl?trans:name=string\?&trans:value=string\?'
  4. Optionally, verify installation of the transformation by making a GET request to the /config/transforms service. For example:
    $ curl --anyauth --user user:password -X GET \
        http://localhost:8003/v1/config/transforms?format=xml
    ...
    HTTP/1.1 200 Transform List Retrieved
    Content-type: application/xml
    Server: MarkLogic
    Content-Length: 348
    Connection: close
    
    <?xml version="1.0" encoding="UTF-8"?>
    <rapi:transforms xmlns:rapi="http://marklogic.com/rest-api">
      <rapi:transform>
        <rapi:name>add-attr-xsl</rapi:name>
        <rapi:title/>
        <rapi:version/>
        <rapi:provider-name/>
        <rapi:description/>
        <rapi:transform-parameters>
          <rapi:parameter>
            <rapi:parameter-name>value</rapi:parameter-name>
            <rapi:parameter-type>string?</rapi:parameter-type>
          </rapi:parameter>
          <rapi:parameter>
            <rapi:parameter-name>name</rapi:parameter-name>
            <rapi:parameter-type>string?</rapi:parameter-type>
          </rapi:parameter>
        </rapi:transform-parameters>
        <rapi:transform-source>
          /v1/transforms/add-attr
        </rapi:transform-source>
      </rapi:transform>
    </rapi:transforms>
  5. Make a PUT request to the /documents service to insert a document into the database, using the 'add-attr-xsl' transform to add the attribute 'example' with a value of 'new'. For example:
    $ curl --anyauth --user user:password -X PUT -d"<data/>" \
        'http://localhost:8003/v1/documents?uri=/doc/transformed.xml&transform=add-attr-xsl&trans:name=example&trans:value=new'
  6. Make a GET request to the /documents service to retrieve the inserted document. Note the added 'example' attribute. For example:
    $ curl --anyauth --user user:password -X GET \
        http://localhost:8003/v1/documents?uri=/doc/transformed.xml
    <?xml version="1.0" encoding="UTF-8"?>
    <data example="new"/>

« Previous chapter
Next chapter »