Loading TOC...
REST Application Developer's Guide (PDF)

REST Application Developer's Guide — Chapter 3

Manipulating Documents

This chapter discusses the following topics related to using the REST Client API to create, read, update and delete documents and metadata:

Summary of Document Management Services

This section gives a brief summary of the REST Client API services available for creating, reading, updating, and delete documents. This section covers the following topics:

Summary of the /documents Service

Use the /documents service to create, read, update, and delete document content and metadata for XML, JSON, text, and binary documents, including XML documents containing embedded semantic RDF triple data. To load documents containing only RDF triples, use the /graph service; for details, see Summary of the /graphs Service.

The following table summarizes the supported operations:

OperationMethodDescription
Create/UpdatePUTCreate or update the content or metadata of a document.
CreatePOSTCreate a document with a URI automatically generated by MarkLogic Server.
Partial UpdatePOSTUpdate a portion of the content of an XML or JSON document, or update metadata of any document.
Multi-Document WritePOSTCreate or update multiple the content and/or metadata of multiple documents in a single request.
RetrieveGETRetrieve content and/or metadata for one or more documents.
DeleteDELETERemove a document, or remove or reset document metadata.
TestHEADTest for the existence of a document or determine the size.

The service supports the following additional document features through request parameters:

  • Transaction control
  • Transformation of content during ingestion
  • Content repair during ingestion
  • Transformation of results during document insertion or retrieval
  • Conditional document insertion using optimistic locking
  • Conditional reads based on content versioning
  • Multi-request reads at fixed point-in-time

    XML, JSON and text documents must use UTF-8 encoding.

    The service does not support content conversions except through custom transformations. For example, you cannot retrieve an XML document as JSON.

You can also use POST /v1/documents/protection to protect temporal documents from various levels of update. For details, see Working with Temporal Documents and the MarkLogic REST API Reference.

This chapter only covers single-document operations. For multi-document operations, see Reading and Writing Multiple Documents.

Summary of the /graphs Service

Use the /graphs service to create, read, update, and delete documents containing RDF triples. For triples embedded in XML documents, use the /documents service.

The following table summarizes the supported operations:

OperationMethodDescription
Create/UpdatePUTCreate or replace triples in a named graph or the default graph.
UpdatePOSTMerge triples into a named graph or the default graph.
RetrieveGETRetrieve a named graph or the default graph.
DeleteDELETERemove triples in a graph.
TestHEADTest for the existence of a graph in the database or retrieve the headers that would be returned by a GET request.

Though a graph can include both triples embedded in XML documents and triples from a pure triples document (one with a <sem:triples> root element), the /graphs service does not operate on embedded triples. For example, if you make a DELETE request on a graph that includes embedded triples, the embedded triples and their containing document are unaffected, and therefore, the graph continues to exist.

For details, see Loading Triples Using the REST API in Semantics Developer's Guide.

Loading Content into the Database

This section focuses on ingesting whole documents and their metadata into the database. To learn about modifying just a portion of a document, see Partially Updating Document Content or Metadata.

To insert content or metadata into the database, make a PUT or POST request to the /documents service. This section covers the following topics:

Loading Content

This section describes how to insert or update an entire document at a user-defined database URI. If you want to have MarkLogic Server generate document URIs automatically, see Automatically Generating Document URIs. You can also modify only a portion of a document; for details, see Partially Updating Document Content or Metadata. For loading multiple documents in a single request, see Reading and Writing Multiple Documents.

To insert or update an XML, JSON, text, or binary document, make a PUT request to a URL of the form:

http://host:port/version/documents?uri=document_uri

When constructing the request:

  1. Set the uri parameter to the URI of the destination document in the database.
  2. Place the content in the request body.
  3. Specify the MIME type of the content in the Content-type HTTP header.

    XML, JSON and text documents must be use UTF-8 encoding.

Documents you create with the REST Client API have a read permission for the rest-reader role and an update permission for the rest-writer role. To restrict access, use custom roles. For details, see Controlling Access to Documents and Other Artifacts.

The following example command sends a request to insert the contents of the file ./my.xml into the database as an XML document with URI /xml/example.xml:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT -T ./my.xml \
    -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/documents?uri=/xml/example.xml

If the MIME type is not set in the HTTP Content-type header, MarkLogic Server uses the file extension on the document URI to determine the content format, based on the MIME type mappings defined for your installation.

You can also set metadata such as collections, permissions, and named properties when loading content. See Loading Content and Adding Metadata in the Same Request.

You can have MarkLogic Server generate document URIs for you. For details, see Automatically Generating Document URIs.

Adding Metadata

This section describes inserting or updating metadata using a PUT request to /documents. The following topics are covered:

Inserting or Updating Metadata

To insert or update only metadata for a document, make a PUT request to a URL of the form:

http://host:port/version/documents?uri=document_uri&category=metadata_category

Where category can appear multiple times, with the values described in Metadata Categories.

You cannot supply metadata via request parameters when there is no document content. In this case, you must place the XML or JSON metadata in the request body.

When constructing the request:

  1. Set the category parameter to the type of metadata to insert or replace. Specify category multiple times to include more than one type of metadata.
  2. Place the metadata in the request body. For format details, see Working with Metadata.
  3. Specify the metadata format in the HTTP Content-type header or the format parameter. You may supply either JSON or XML metadata.

Metadata category merging is not available. A PUT request for metadata completely replaces each category of metadata specified in the request. For example, a PUT request for collections replaces all existing collections. When the category is metadata, all metadata is replaced or reset to default values.

When setting permissions, at least one update permission must be included.

Any explicitly specified permissions are combined with the default permissions for the role of the current user.

Metadata can be supplied as either XML or JSON. Use the format parameter or the Content-type header to specify the format of the metadata. If both format and Content-type are given, format takes precedence. If neither format nor Content-type is specified, XML is assumed. For details on formats, see Working with Metadata.

Metadata for categories other than those named by the category parameter(s) are ignored. For example, if the request body contains metadata for both collections and properties, but only category=collections is given in the URL, then only the collections are updated.

Example: Replacing One Metadata Category Using XML

The following example places the document with URI /xml/example.xml into the 'interesting' collection by specifying category=collections. The document is removed from any other collections. The metadata XML in the request body defines the name of the collection(s).

$ cat metadata.xml
<rapi:metadata xmlns:rapi="http://marklogic.com/rest-api"
               xmlns:prop="http://marklogic.com/xdmp/property">
  <rapi:collections>
    <rapi:collection>interesting</rapi:collection>
  </rapi:collections>
</rapi:metadata>
# Windows users, see Modifying the Example Commands for Windows 
$ curl -X PUT -T ./metadata.xml -H "Content-type: application/xml" \
    --anyauth --user user:password \
    'http://localhost:8000/LATEST/documents?uri=/xml/example.xml&category=collections'
Example: Replacing Multiple Metadata Categories Using XML

This example replaces multiple types of metadata on the document with URI /xml/example.xml by specifying multiple category parameters. The metadata in the request body defines a collection name ('interesting') and a property ('my-property'). The request URL includes category=collections and category=properties. Any collections or properties previously set for the document /xml/example.xml are replaced with the new values.

$ cat > metadata.xml
<rapi:metadata xmlns:rapi="http://marklogic.com/rest-api"
               xmlns:prop="http://marklogic.com/xdmp/property">
  <rapi:collections>
    <rapi:collection>interesting</rapi:collection>
  </rapi:collections>
  <prop:properties>
    <my-property>value</my-property>
  </prop:properties>
</rapi:metadata>
# Windows users, see Modifying the Example Commands for Windows 
$ curl -X PUT -T ./metadata.xml -H "Content-type: application/xml" \
    --anyauth --user user:password \
    'http://localhost:8000/LATEST/documents?uri=/xml/example.xml&category=collections&category=properties'
Example: Replacing Multiple Metadata Categories Using JSON

This example replaces multiple types of metadata on the document with URI /xml/example.xml by specifying multiple category parameters. The JSON metadata in the request body defines a collection name ('interesting') and a property ('my-property'). The request URL includes category=collections and category=properties. Any collections or properties previously set for the document /xml/example.xml are replaced with the new values.

$ cat > metadata.json
{
  "collections":["interesting"],
  "properties": {
    "my-property":"name"
  }
}
# Windows users, see Modifying the Example Commands for Windows 
$ curl -X PUT -T ./metadata.json --anyauth --user user:password \
    -H "Content-type: application/json" \
    'http://localhost:8000/LATEST/documents?uri=/xml/example.xml&category=collections&category=properties'

The example uses the Content-type header to communicate the metadata content type to MarkLogic Server. The content type can also be specified using the format request parameter.

Loading Content and Adding Metadata in the Same Request

You can update content and metadata for a document in a single request using two methods:

For loading multiple content and/or metadata for multiple documents in a single request, see Reading and Writing Multiple Documents.

Loading Content and Metadata Using Request Parameters

Use this method when you want to specify metadata using request parameters. To load content and include metadata in the request parameters, send a PUT request of the following form to the/documents service:

http://host:port/version/documents?uri=doc_uri&metadata_param=value

Where metadata_param is one of collection, perm:role, prop:name, quality, or value:key. For example, to set a document property named 'color' to red, include prop:color=red in the URL.

When constructing the request:

  1. Set the uri parameter to the URI of the destination document in the database.
  2. Place the content in the request body.
  3. Specify the MIME type of the content in the Content-type HTTP header.
  4. Specify the value of one or more metadata categories through request parameters, such as collection or prop.

If the MIME type is not set in the Content-type header, MarkLogic Server uses the file extension on the document URI to determine the content format, based on the MIME type mapping defined for the database. MIME type mappings for file suffixes are defined in the Admin Interface.

The following example inserts a binary document with the URI /images/critter.jpg into the database, adds it to the 'animals' collection, and sets a 'species' property:

# Windows users, see Modifying the Example Commands for Windows 
$ curl -X PUT -T ./critter.jpg --anyauth --user user:password \
    -H "Content-type: image/jpeg" \
    'http://localhost:8000/LATEST/documents?uri=/images/critter.jpg&collection=animals&prop:species="canus lupus"'

You can also create a block of metadata as JSON or XML and pass it in the request body with the content. See Loading Content and Metadata Using a Multipart Message.

Loading Content and Metadata Using a Multipart Message

Use this method when you want to insert or update both content and metadata for a document in a single request, and you want to specify the metadata as JSON or XML in the request body. You can also specify metadata using request parameters. For details, see Loading Content and Metadata Using Request Parameters.

Construct a PUT request with a multipart/mixed message body where the metadata is in the first part and the document content is in the second part of the request body. The request URL is of the form:

http://host:port/version/documents?uri=doc_uri&category=content&category=metadata_category

Where category can appear multiple times, with the values described in Metadata Categories.

When constructing the request:

  1. Set the uri parameter to the URI of the destination document in the database.
  2. Set category=content in the request URL to indicate content is included in the body.
  3. Set additional category parameters to indicate the type(s) of metadata to add or update.
  4. Specify multipart/mixed in the HTTP Content-type header for the request.
  5. Set the part boundary string in the HTTP Content-type header to a string of your choosing.
  6. Set the Content-type of the first part to either application/xml or application/json and place the XML or JSON metadata in the part body.
  7. Set the Content-type of the second part to the MIME type of the content and place the content in the part body.

For details on metadata formats, see Working with Metadata.

Metadata must always be the first part of the multipart body.

The following example inserts an XML document with the URI /xml/box.xml into the database and adds it to the 'shapes' and 'squares' collection. The collection metadata is provided in XML format in the first part of the body, and the content is provided as XML in the second part.

$ cat ./the-body
--BOUNDARY
Content-Type: application/xml

<?xml version="1.0" encoding="UTF-8"?>
<metadata xmlns="http://marklogic.com/rest-api">
  <collections>
    <collection>shapes</collection>
    <collection>squares</collection>
  </collections>
</metadata>
--BOUNDARY
Content-Type: text/xml

<?xml version="1.0" encoding="UTF-8"?>
<data>
  <mybox>
    This is my box. There are many like it, but this one is mine.
  </mybox>
</data>
--BOUNDARY--
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    --data-binary @./the-body \
    -H "Content-type: multipart/mixed; boundary=BOUNDARY" \
    'http://localhost:8000/LATEST/documents?uri=/xml/box.xml&category=collections&category=content'

You can also pass metadata in URL parameters instead of in the request body. See Loading Content and Metadata Using Request Parameters.

Automatically Generating Document URIs

MarkLogic Server can automatically generate databse URIs for documents inserted with the REST API. You can only use this feature to create new documents. To update an existing document, you must know the URI; for details, see Loading Content.

To use this feature, send a POST request to /documents that includes the extension request parameter and does not include the uri request parameter. That is, a POST request with a URL of the following form:

http://host:port/version/documents?extension=file-extension

Where extension specifies the document URI file prefix, such as xml or txt. Optionally, you can also include a database directory prefix with the directory request parameter.

The directory prefix should end in a forward slash (/).

MarkLogic Server returns the auto-generated URI in the Location header of the response.

The following example inserts a document into the database with a URI of the form /my/directory/auto-generated.xml:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X POST -d@'./my-content' -i \
  -H "Content-type: application/xml" \
  'http://localhost:8000/LATEST/documents?extension=xml&directory=/my/directory/'
HTTP/1.1 201 Document Created
Location: /my/directory/6041706572796142832.xml
...

You can use the same features with generated URIs as when loading documents with user-defined URIs. For example, you can insert both documents and metadata and apply content transformations. For details, see Loading Content into the Database and the MarkLogic REST API Reference.

Loading Triples

You can use the /graphs service to load semantic triples into the database in several formats. The service implements the W3C Graph Store Protocol, described by the following specification:

http://www.w3.org/TR/2013/REC-sparql11-http-rdf-update-20130321/

The collection lexicon must be enabled on your database when using the semantics REST services or use the GRAPH '?g' construct in a SPARQL query. For details, see Configuring the Database to Work with Triples in the Semantics Developer's Guide.

To load triples, send a PUT or POST request to the /graphs service with one of the following forms of URL, depending upon whether you wish to address a specific named graph or the default graph:

http://host:port/version/graphs?graph=graph-uri 

http://host:port/version/documents?default

For details, see Loading Triples Using the REST API and Supported RDF Triple Formats in Semantics Developer's Guide.

You can also create, update, and delete triples and graphs with SPARQL Update by sending a POST request to the /graphs/sparql service. For example, the following SPARQL update request inserts a triple into the default graph:

PREFIX dc: <http://purl.org/dc/elements/1.1/>
INSERT DATA
{ <http://example/book0> dc:title "A default book"  }

If you place the above SPARQL Update request into a file named update.sparql, then the following request to the /graphs/sparql performs the requested update:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X POST -i 
  --data-binary @./update.sparql \
  -H "Content-type:application/sparql-update" \
  -H "Accept:application/sparql-results+xml" \
  'http://localhost:8000/LATEST/graphs/sparql'

When you put your SPARQL Update request in the HTTP request body, the request Content-type must be application/sparql-update. You can also specify your update using URL encoded form data. For more details and examples, see SPARQL Update with the REST Client API in the Semantics Developer's Guide.

Controlling Access to Documents Created with the REST API

Documents you create with the REST Client API have a read permission for the rest-reader role and an update permission for the rest-writer role by default. To restrict access to particular users, create custom roles rather than assigning users to the default rest-* roles. For example, you can use a custom role to restrict users in one group from seeing documents created by another.

For details, see Controlling Access to Documents and Other Artifacts.

Transforming Content During Ingestion

You can transform content during ingestion by applying custom transform. A transform is an XQuery module or XSLT stylesheet you write and install using /config/transforms/{name}. For details, see Working With Content Transformations.

To apply a transform when creating or updating a document, add the transform parameter to your request. If the transform expects parameters, specify them using trans:paramName parameters. That is, your request should be of the form:

http://host:port/version/documents?...&transform=name&trans:arg=value

The following example applies a transform installed under the name 'example' that expects a parameter named 'reviewer':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    -d@./the-body -H "Content-type: application/xml" \
    'http://localhost:8000/LATEST/documents?uri=/doc/theDoc.xml&transform=example&trans:reviewer=me'

For a complete example, see XQuery Example: Adding an Attribute During Ingestion or XSLT Example: Adding an Attribute During Ingestion.

Retrieving Documents from the Database

To a retrieve document from the database, make a GET request to the /documents service. You can retrieve just the contents, just the metadata, or both contents and metadata. This section covers the following topics:

To retrieve multiple documents in a single request, see Reading and Writing Multiple Documents.

Retrieving the Contents of a Document

To retrieve a document from the database, construct a GET request of the following form:

http://host:port/version/documents?uri=doc_uri

HTTP content type negotiation is not supported. If the HTTP Accept header is not set, MarkLogic Server uses the file extension on the document URI to determine the response content type, based on the server-wide MIME type mapping definitions. See 'Mimetypes' in the Admin Interface.

Though content negotiation is not supported, you can use the transform feature to apply server-side transformations to the content before the response is constructed. For details, see Working With Content Transformations.

To retrieve multiple documents in a single request, see Reading and Writing Multiple Documents.

Retrieving Metadata About a Document

To retrieve metadata about a document without retrieving the contents, construct a GET request of the following form:

http://host:port/version/documents?uri=doc_uri&category=metadata_category

Where category can appear multiple times, with the values described in Metadata Categories.

When constructing the request:

  1. Set the category parameter to the type of metadata to retrieve. Specify category multiple times to request more than one type of metadata.
  2. Specify the metadata content type (XML or JSON) in the HTTP Accept header or the format parameter.

For details on metadata categories and formats, see Working with Metadata.

Use the format parameter or the Accept header to specify the format of the metadata. If both format and Accept are set, format takes precedence. If neither format nor Accept is specified, XML is assumed.

To retrieve metadata as XML, set format to 'xml' or set the Accept header to application/xml. To retrieve metadata as JSON, set format to 'json' or set the Accept header to application/json.

Retrieving Content and Metadata in a Single Request

To retrieve content and metadata for a document in a single request, construct GET request to a URL of the form:

http://host:port/version/documents?uri=doc_uri&category=content&category=metadata_category

Where category can appear multiple times, with the values described in Metadata Categories.

The request response is a multipart/mixed message, with the metadata in the first body part and content in the second body part. The Content-Type headers for the parts are determined as follows:

  • The MIME type of the metadata part is determined by the format parameter, which you can set to either xml or json; the default is xml. For details on metadata format, see Working with Metadata.
  • The MIME type of the content part is determined by the server-wide MIME type mapping for the document URI extension. See 'Mimetypes' in the Admin Interface.

The following example command retrieves a document and its metadata in a single request:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    -H "Accept: multipart/mixed;boundary=BOUNDARY" \
    'http://localhost:8000/LATEST/documents?uri=/xml/box.xml&category=metadata&category=content&format=xml'
--BOUNDARY
Content-Type: application/xml
Content-Length: 518

<?xml version="1.0" encoding="UTF-8"?>
<rapi:metadata uri="/xml/box.xml"
    xsi:schemaLocation="http://marklogic.com/rest-api/database dbmeta.xsd" 
    xmlns:rapi="http://marklogic.com/rest-api"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <rapi:collections>
    <rapi:collection>shapes</rapi:collection>
    <rapi:collection>squares</rapi:collection>
  </rapi:collections>
  <rapi:permissions/>
  <prop:properties xmlns:prop="http://marklogic.com/xdmp/property"/>
  <rapi:quality>0</rapi:quality>
  <rapi:metadata-values/>
</rapi:metadata>
--BOUNDARY
Content-Type: text/xml
Content-Length: 128

<?xml version="1.0" encoding="UTF-8"?>
<data><mybox>This is my box. There are many like it, but this one is mine.</mybox></data>
--BOUNDARY--

HTTP content negotiation is not supported, but custom server-side content transformations can be applied using the transform parameter. For details, see Retrieving the Contents of a Document and Working With Content Transformations.

Transforming Content During Retrieval

You can apply custom transforms to a document before returning it to the requestor. A transform is a JavaScript module, XQuery module, or XSLT stylesheet you write and install using /config/transforms/{name}. For details, see Working With Content Transformations.

You can configure a default transform that is automatically applied whenever a document is retrieved. You can also specify a per-request transform using the transform request parameter. If there is both a default transform and a per-request transform, the transforms are chained together, with the default transform running first. Thus, the output of the default transform is the input to the per-request transform:

To configure a default transformation, set the document-transform-out configuration parameter for the REST Client API instance. Instance-wide parameters are set using /config/properties. For details, see Configuring Instance Properties.

To specify a per-request transform, add the transform parameter to your request. If the transform expects parameters, specify them using trans:paramName parameters. That is, your request should be of the form:

http://host:port/version/documents?...&transform=name&trans:arg=value

The following example applies a transform installed under the name 'example' that expects a parameter named 'reviewer':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/xml" \
    'http://localhost:8000/LATEST/documents?uri=/doc/theDoc.xml&transform=example&trans:reviewer=me'

Partially Updating Document Content or Metadata

This section focuses on modifying a portion of an existing XML or JSON document or its metadata. To learn about inserting or updating an entire document or its metadata, see Loading Content into the Database.

This section covers the following topics:

Introduction to Content and Metadata Patching

A partial update is an update you apply to a portion of a document or metadata. For example, inserting an XML element or attribute or changing the value associated with a JSON property. You can only apply partial content updates to XML and JSON documents. You can apply partial metadata updates to any document type.

A patch is a partial update descriptor, expressed in XML or JSON, that tells MarkLogic Server where to apply an update and what update to apply. A patch is a wrapper XML element or JSON object that encapsulates one or more update operations.

Use a partial update to do the following operations:

  • Add, replace, or delete an XML element, XML attribute, or JSON sub-object of an existing document.
  • Add, replace, or delete a subset of the metadata of an existing document. For example, modify a permission or insert a document property.
  • Dynamically generate replacement content or metadata on MarkLogic Server using builtin or user-defined functions. For details, see Constructing Replacement Data on the Server.

To perform a partial update, send a POST or PATCH request to the /documents service with a patch in the request body. For example, the following patch descriptor inserts a new <child/> element as the last child of the element located through the XPath expression /data:

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/data" position="last-child">
    <child>the last one</child>
  </rapi:insert>
</rapi:patch>

Assuming the above patch is saved to the file patch.xml, the following command uses a POST request to apply the patch to the document with the URI /doc/example.xml.

$ curl --anyauth --user user:password -X POST -d @./patch.xml -i \
    -H "Content-type: application/xml" \
    -H "X-HTTP-Method-Override: PATCH" \
    http://localhost:8000/LATEST/documents?uri=/doc/example.xml

The following example is an equivalent request using PATCH instead of POST:

$ curl --anyauth --user user:password -X PATCH -d @./patch.xml -i \
    -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/documents?uri=/doc/example.xml

For details, see Basic Steps for Patching Documents.

You can apply multiple updates in a single request. You can patch both content and metadata in a single request.

Content negotiation is not supported: A content patch must match the content type of the target document. That is, you cannot submit a JSON patch for an XML document or an XML patch for a JSON document.

If a patch contains multiple operations, they are applied independently to the target document. That is, within the same patch, one operation does not affect the context or select results or the content changes of another. Each operation in a patch is applied independently to every matched node. If any operation in a patch fails with an error, the entire patch fails.

Content transformations are not directly supported in a partial update. However, you can implement a custom replacement content generation function to achieve the same effect. For details, see Constructing Replacement Data on the Server.

Basic Steps for Patching Documents

To perform a partial update of an XML or JSON document, or of document metadata, send a POST or PATCH request to the /documents service with a URL of the following form:

http://host:port/version/documents?uri=doc_uri

The body of the request must contain a content patch, as described in XML Patch Reference and JSON Patch Reference. You can patch metadata and content in the same request by using the category request parameter; for details, see Patching Metadata.

When constructing your request, follow these guidelines:

  1. Set the uri parameter to the URI of the target document in the database.
  2. Set the category request parameter to content, metadata, and/or one of the metadata subcategories. The default is content.
  3. Set the Content-type to either application/xml or application/json. When patching content, you must use the content type that matches the target document; only XML and JSON are supported.
  4. Place the XML or JSON patch in the request body.
  5. If you POST, set the request header X-HTTP-Method-Override to PATCH to tell MarkLogic Server this is a partial update request.

New content that is added or replaced can be specified directly in the patch, or generated on MarkLogic Server by a replacement content generator function. For details, see Constructing Replacement Data on the Server.

XML Patch Reference

This section summarizes the structure of the XML patch structure used to describe partial updates to XML documents and document metatdata. Each section includes a syntax summary, a description of the components, and an example.

The following elements, all in the namespace http://marklogic.com/rest-api, are covered. Start wit the <patch/> wrapper.

patch

The top level wrapper element around a partial update descriptor. A <patch/> has the following structure:

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert />
  <rapi:replace />
  <rapi:replace-insert />
  <rapi:delete />
  <rapi:replace-library />
</rapi:patch>

All elements are optional. The operations can occur multiple times. The <replace-library/> can occur at most once and is only required if you use user-defined functions to generate replacement content server-side; for details, see Constructing Replacement Data on the Server.

For example, the following patch includes an insertion, deletion, and replacement:

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/inventory/history" position="last-child">
    <modified>2012-11-5</modified>
  </rapi:insert>
  <rapi:delete select="saleExpirationDate"/>
  <rapi:replace select="price" apply="ml.multiply">1.1</rapi:replace>
</rapi:patch>

Where a patch operation includes new XML content, such as an element insertion, the new content must use namespaces appropriate to the target document. Namespaces declared in the root <patch/> node are in scope for this content. For details, see Managing XML Namespaces in a Patch.

For more details, see the individual operations.

insert

Insert a new element, attribute, text node, comment, or processing instruction in an XML document or in document metadata. An <insert/> element has the following structure:

<insert context=xpath-expr position=pos-selector
        cardinality=occurrence>
  content-to-insert
</insert>

For example, the following patch adds a <modified/> element as the last child of every <history/> node that matches the XPath expression /inventory/history:

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/inventory/history" position="last-child">
    <modified>2012-11-5</modified>
  </rapi:insert>
</rapi:patch>

For additional examples, see XML Examples of Partial Updates.

The following table summarizes the parts of an <insert/> element.

ComponentReq'dDescription
@context=xpath-exprY

An XPath expression that selects an existing node(s) or attribute on which to operate. Namespaces declared on the root <patch/> node are in scope for this path.

When inserting an attribute, @context can identify either the containing element(s) or another attribute of the containing element(s). In all other cases, @context should identify one or more nodes.

If no matches are found for the @context expression, the operation is silently ignored.

The path expression is restricted to the subset of XPath that can be used to define an index. For details, see Path Expressions Usable in Index Definitions.

@position=pos-selectorYWhere to insert the content, relative to the node(s) selected by @context. The attribute value must be one of before, after, or last-child.
@cardinality=occurrenceN

The required occurrence of matches to @position. If the number of matches does not meet the expectation, the operation fails and an error is returned. Allowed values:

  • Zero or one matches required: ? (question mark)
  • Exactly one match required: . (period)
  • Zero or more matches required: * (asterisk)
  • One or more matches required: + (plus)

Default: * (The occurrence requirement is always met).

content-to-insertY

When inserting elements, text, comments, and processing directives, specify the new nodes as they should appear in the target document.

When inserting attributes, specify the attribute(s) on a <rapi:attribute-list/> element. The following example specifies 2 attributes to insert:

<rapi:attribute-list a1="v1" a2="v2"/>
replace

Replace an existing element or attribute. If no matching element or attribute exists, the operation is silently ignored. A <replace/> element has the following structure:

<replace select=xpath-expr cardinality=occurrence apply=func-name>
  replacement-content
</replace>

For example, the following patch replaces the first <child/> element of <parent/>:

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace select="/parent/child[1]">
    <child>REPLACED</child>
  </rapi:replace>
</rapi:patch>

For additional examples, see XML Examples of Partial Updates.

You can also supply a content generation builtin or user-defined function using @apply, using the format below. For example, you can use the builtin ml.multiply function to multiply the current value of an element that contains numeric data, without needing to know the current value.

<rapi:replace select="my-elem" apply=ml.multiply>5</rapi:replace>

For details, see Constructing Replacement Data on the Server.

The following table summarizes the parts of a <replace/> element.

ComponentReq'dDescription
@select=xpath-exprY

An XPath expression that selects an existing node(s) or attribute to replace. Namespaces declared on the root <patch/> node are in scope for this path.

The path expression is restricted to the subset of XPath that can be used to define an index. For details, see Path Expressions Usable in Index Definitions.

The selected node cannot be the target of any other operation in the patch. The ancestor of the selected node may not be modified by a delete, replace, or replace-insert operation in the same patch.

@cardinality=occurrenceN

The required occurrence of matches to @select. If the number of matches does not meet the expectation, the operation fails and an error is returned. Allowed values:

  • Zero or one matches required: ? (question mark)
  • Exactly one match required: . (period)
  • Zero or more matches required: * (asterisk)
  • One or more matches required: + (plus)

Default: * (The occurrence requirement is always met).

replacement-contentN

The content with which to replace the selected node or attribute. If present, this must be a single node. If there is no replacement-content, you must specify a content generation function using @apply.

To replace an element, specify either a replacement element or text. When you use text, the text replaces just the content of the target element.

To replace an element with a text node, wrap the text in a <text/> element in the namespace http://marklogic.com/rest-api. For example:

<rapi:text>replacement text</rapi:text>

To replace an attribute, use either a <text/> or <attribute-list/> wrapper in the namespace http://marklogic.com/rest-api. For example:

<rapi:attribute-list my-attr="new-value" />
<rapi:text>new-value</rapi:text>
@apply=func-nameN

The localname of a replacement content generation function. If you do not specify a function, the operation must include replacement-content.

If you name a user-defined function, the <patch/> must also contain a <replace-library/> element.

For details, see Constructing Replacement Data on the Server.

replace-insert

Replace an element or attribute; if there are no existing matching elements or attributes, perform an insertion operation instead. A <replace-insert/> element has the following structure:

<replace-insert select=xpath-expr context=xpath-expr position=pos
    cardinality=occurrence apply=func-name>
  replacement-content
</replace-insert>

For example, the following patch replaces the first <grandchild/> element of a <child/> of <parent/> elements. If no <child/> element has a <grandchild/> element, one is inserted in every <child/> of a <parent/> element.

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-insert context="/parent/child" select="grandchild[1]"
      position="last-child">
    <grandchild>CHANGED</grandchild>
  </rapi:replace-insert>
</rapi:patch>

For additional examples, see XML Examples of Partial Updates.

You can also use @apply to specify a content generation builtin or user-defined function for generating dynamic content. For details, see Constructing Replacement Data on the Server.

The following table summarizes the parts of a <replace-insert/> element.

ComponentReq'dDescription
@select=xpath-exprY

An XPath expression that selects an existing node(s) or attribute to replace. Namespaces declared on the root <patch/> node are in scope for this path. The path may be absolute or relative to the @context path.

The path expression is restricted to the subset of XPath that can be used to define an index. For details, see Path Expressions Usable in Index Definitions.

If no matches are found for the @select expression, @context and @position are used to attempt an insert. If no match is then found for @context, the behavior depends on the value of @cardinality.

The selected node cannot be the target of any other operation in the patch. The ancestor of the selected node may not be modified by a delete, replace, or replace-insert operation in the same patch.

replacement-contentN

The content with which to replace the selected node or attribute. If there is no replacement-content, you must specify a content generation function using @apply.

To replace an element, specify either a replacement element or text. When you use text, the text replaces just the content of the target element.

To replace an element with a text node, wrap the text in a <text/> element in the namespace http://marklogic.com/rest-api. For example:

<rapi:text>replacement text</rapi:text>

To replace an attribute, use either a <text/> or <attribute-list/> wrapper in the namespace http://marklogic.com/rest-api. For example:

<rapi:attribute-list my-attr="new-value" />
<rapi:text>new-value</rapi:text>
@context=xpath-exprY

An XPath expression that selects an existing node(s) or attribute on which to operate if the @select target does not exist. @context is also used to resolve @select if the select expression is a relative path. Namespaces declared on the root <patch/> node are in scope for this path.

The path expression is restricted to the subset of XPath that can be used to define an index. That is, path expressions for which cts:valid-index-path returns true.

When inserting an attribute, @context can identify either the containing element(s) or another attribute of the containing element(s). In all other cases, @context should identify one or more nodes.

If no matches are found for either the @context or @select expressions, the operation is silently ignored.

The ancestor of the selected node may not be modified by a delete, replace, or replace-insert operation in the same patch.

@position=pos-selectorYIf @select does not match anything, where to insert the content, relative to the node(s) selected by @context. The attribute value must be one of before, after, or last-child. Ignored for attributes.
@cardinality=occurrenceN

The required occurrence of matches to @position. If the number of matches does not meet the expectation, the operation fails and an error is returned. Allowed values:

  • Zero or one matches required: ? (question mark)
  • Exactly one match required: . (period)
  • Zero or more matches required: * (asterisk)
  • One or more matches required: + (plus)

Default: * (The occurrence requirement is always met).

@apply=func-nameN

The localname of a replacement content generation function. If you do not specify a function, the operation must include replacement-content.

If you name a user-defined function, the <patch/> must also contain a <replace-library/> element.

For details, see Constructing Replacement Data on the Server.

delete

Remove an element or attribute from an existing node. A <delete/> element has the following structure:

<delete select=xpath-expr cardinality=occurrence />

For example, the following patch deletes the first <child/> element of a <parent/> and removes the my-attr attribute from the <parent/> element.

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:delete select="/parent/child[1]" />
  <rapi:delete select="/parent/@my-attr" />
</rapi:patch>

For additional examples, see XML Examples of Partial Updates.

The following table summarizes the parts of a <delete/> element.

ComponentReq'dDescription
@select=xpath-exprY

An XPath expression that selects the node(s) or attribute(s) to delete. Namespaces declared on the root <patch/> node are in scope for this path.

The path expression is restricted to the subset of XPath that can be used to define an index. For details, see Path Expressions Usable in Index Definitions.

The selected node cannot be the target of any other operation in the patch. The ancestor of the selected node may not be modified by a delete, replace, or replace-insert operation in the same patch.

@cardinality=occurrenceN

The required occurrence of matches to @select. If the number of matches does not meet the expectation, the operation fails and an error is returned. Allowed values:

  • Zero or one matches required: ? (question mark)
  • Exactly one match required: . (period)
  • Zero or more matches required: * (asterisk)
  • One or more matches required: + (plus)

Default: * (The occurrence requirement is always met).

replace-library

Specify an XQuery library module that contains user-defined replacement content generation functions. These functions can be used in @apply on <replace/> and <replace-insert/> operations. For details, see Constructing Replacement Data on the Server.

For example, the following patch uses the function my-ns:my-func in the library module in the with the URI /my.domain/my-module.xqy.

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-library
    at="/my.domain/my-module.xqy" ns="my-ns" />
  <rapi:replace select="/parent/child[1]" apply="my-func" />
</rapi:patch>

For additional examples, see Constructing Replacement Data on the Server.

The following table summarizes the parts of a <replace-library/> element.

ComponentReq'dDescription
@at=lib-module-pathNThe path to the XQuery library module containing user-defined replacement content generation functions. The module must be installed in the modules database associated with the REST API instance.
@ns=func-namespaceNThe module namespace alias defined by the @at module.

Managing XML Namespaces in a Patch

The patch descriptor must be in the namespace http://marklogic.com/rest-api. To distinguish the <patch/> elements from elements in your document, you should include an alias for the patch descriptor namespace. For example:

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
    ...
</rapi:patch>

Declare additional namespace aliases for the content described by the patch. Namespaces declared on the <patch/> root element are in scope in @context and @select path expressions and in the content portion of <insert/>, <replace/> and <replace-insert/> elements.

For example, if you patch a document with the following contents:

<parent xmlns="http://my/namespace">
  <child>first</child>
</parent>

Then your <patch/> should either define http://my/namespace as the default namespace or define a namespace alias for it. For example, the following patch inserts a <child/> element when you define a default namespace:

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api"
    xmlns="http://my/namespace">
  <rapi:insert context="/parent" position="last-child">
    <child>last</child>
  </rapi:insert>
</rapi:patch>

The following is the same patch, using an explicit namespace alias of my-ns:

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api"
    xmlns:my-ns="http://my/namespace">
  <rapi:insert context="/my-ns:parent" position="last-child">
    <my-ns:child>last</my-ns:child>
  </rapi:insert>
</rapi:patch>

Failure to use namespace declarations properly in a patch can lead to surprising results. For example, if you did not include a namespace declaration for http://my/namespace, the patch would not work because the context XPath expression would not match any elements in the target document. Similarly, if you include a namespace alias but do not use it in the definition of the new child, the new child element would be inserted in no namespace: <child xmlns="">...</child>.

XML Examples of Partial Updates

This section includes the following examples of applying a partial update to an XML document or metadata:

Example cURL Commands

The examples in this section can be applied using commands similar to the following, assuming you copy the sample input document to a file called example.xml and the patch to a file called patch.xml.

# Windows users, see Modifying the Example Commands for Windows 
# Create the example document
$ curl --anyauth --user user:password -X PUT -d @./example.xml \
    -i -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.xml
# Apply the patch
$ curl --anyauth --user user:password -X POST -d @./patch.xml \
    -i -H "Content-type: application/xml" \
    -H "X-HTTP-Method-Override: PATCH" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.xml
# Retrieve the results
curl --anyauth --user user:password -X GET \
    -i -H "Accept: application/xml" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.xml
Example: Inserting an Element

The following patch inserts an <INSERTED/> element before each <child/> element matched by the context expression.

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/parent/child" position="before">
    <INSERTED/>
  </rapi:insert>
</rapi:patch>

The table below shows how applying the patch changes the target document.

Before UpdateAfter Update
<parent>
  <child>one</child>
  <child>two</child>
  <child>three</child>
</parent>
<parent>
  <INSERTED/>
  <child>one</child>
  <INSERTED/>
  <child>two</child>
  <INSERTED/>
  <child>three</child>
</parent>
Example: Inserting an Attribute

The following patch adds two attributes, my-attr1 and my-attr2, on the second <child/> element.

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/parent/child[2]">
    <rapi:attribute-list my-attr1="val1" my-attr2="val2" />
  </rapi:insert>
</rapi:patch>

The table below shows how applying the patch changes the target document.

Before UpdateAfter Update
<parent>
  <child>one</child>
  <child>two</child>
  <child>three</child>
</parent>
<parent>
  <child>one</child>
  <child my-attr1="val1" my-attr2="val2">
    two
  </child>
  <child>three</child>
</parent>
Example: Inserting a Text Node

The following patch inserts a text node before the second <child/> element matched by the context expression.

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/parent/child[2]"
     position="before">INSERTED</rapi:insert>
</rapi:patch>

The table below shows how applying the patch changes the target document.

Before UpdateAfter Update
<parent>
  <child>one</child>
  <child>two</child>
  <child>three</child>
</parent>
<parent>
  <child>one</child>
  INSERTED
  <child>two</child>
  <child>three</child>
</parent>
Example: Inserting a Comment or Processing Instruction

The following patch inserts a comment as the first child of every <parent/> element matched by the context expression.

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/parent/*[1]" position="before">
    <!-- INSERTED -->
  </rapi:insert>
</rapi:patch>

The table below shows how applying the patch changes the target document.

Before UpdateAfter Update
<parent>
  <child>one</child>
  <child>two</child>
  <child>three</child>
</parent>
<parent>
  <!-- INSERTED -->
  <child>one</child>
  <child>two</child>
  <child>three</child>
</parent>
Example: Multiple Inserts in One Patch

The following patch performs several insert operations on the target document:

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/parent/child" position="before">
    <INSERTED/>
  </rapi:insert>
  <rapi:insert context="/parent/child[2]"
      position="before">INSERTED</rapi:insert>
  <rapi:insert context="/parent/child[2]">
    <rapi:attribute-list my-attr="val" />
  </rapi:insert>
  <rapi:insert context="/parent/*[1]" position="before">
    <!-- INSERTED -->
  </rapi:insert>
</rapi:patch>

The table below shows how applying the patch changes the target document. Notice that the insert operations act independently on the target document, rather than interacting with each other. For example, the comment insertion operation, which uses the context expression /parent/*[1], inserts the comment before the first original <child/> element, not before the <INSERTED/> element added by one of the other operations in the patch.

Before UpdateAfter Update
<parent>
  <child>one</child>
  <child>two</child>
  <child>three</child>
</parent>
<parent>
  <INSERTED/>
  <!-- INSERTED -->
  <child>one</child>
  <INSERTED/>
  INSERTED
  <child my-attr="val">two</child>
  <INSERTED/>
  <child>three</child>
</parent>
Example: Replacing an Element or Element Contents

The following patch replaces the first <child/> node with a new element, replaces the contents of the second <child/>, and replaces the third <child/> with a text node.

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <!-- replace an entire element -->
  <rapi:replace select="/parent/child[1]">
    <REPLACED />
  </rapi:replace>

  <!-- replace the element contents -->
  <rapi:replace select="/parent/child[2]">
    REPLACED
  </rapi:replace>

  <!-- replace an element with a text node -->
  <rapi:replace select="/parent/child[3]">
    <rapi:text>REPLACED</rapi:text>
  </rapi:replace>
</rapi:patch>

The table below shows how applying the patch changes the target document.

Before UpdateAfter Update
<parent>
  <child>one</child>
  <child>two</child>
  <child>three</child>
</parent>
<parent>
  <REPLACED />
  <child>REPLACED</child>
  REPLACED
</parent>
Example: Replacing an Attribute Value

The following patch demonstrates two methods of replacing the value of an attribute. The first <replace/> uses a <text/> wrapper and the second uses an <attribute-list> container element.

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace select="/parent/@attr1">
    <rapi:text>REPLACE-1</rapi:text>
  </rapi:replace>
  <rapi:replace select="/parent/child/@attr2">
    <rapi:attribute-list attr2="REPLACE-2"/>
  </rapi:replace>
</rapi:patch>

The table below shows how applying the patch changes the target document.

Before UpdateAfter Update
<parent attr1="val1">
  <child attr2="val2">one</child>
  <child attr2="val2">two</child>
</parent>
<parent attr1="REPLACE-1">
  <child attr2="REPLACE-2">one</child>
  <child attr2="REPLACE-2">two</child>
</parent>
Example: Replacing or Inserting an Element in One Operation

The following patch replaces the first <grandchild/> element of a <child/> of <parent/> elements. If no <child/> element has a <grandchild/> element, one is inserted in every <child/> of a <parent/> element.

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-insert context="/parent/child" select="grandchild[1]"
    position="last-child">
    <grandchild>CHANGED</grandchild>
  </rapi:replace-insert>
</rapi:patch>

The following table illustrates applying this patch to two documents, one for which there is a match to the replacement target, and one for which there is no match. In the first case, the node matching the @select expression (relative to @context) is replaced with the patch contents. In the second case, there are no matching nodes, so the content in the patch is inserted into all nodes that match the @context expression.

Before UpdateAfter Update
Document contains matching elements.
<parent>
  <child/>
  <child>
    <grandchild/>
    <grandchild>
      unchanged
    </grandchild>
  </child>
</parent>
Matching elements replaced.
<parent>
  <child/>
  <child>
   <grandchild>CHANGED</grandchild>
   <grandchild>
      unchanged
    </grandchild>
  </child>
</parent>
Document contains no matching elements.
<parent>
  <child/>
  <child/>
</parent>
New elements inserted using @context.
<parent>
  <child>
   <grandchild>CHANGED</grandchild>
  </child>
  <child>
   <grandchild>CHANGED</grandchild>
  </child>
</parent>

The following patch is equivalent, except that the <grandchild/> elements are replaced with a text node:

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-insert context="/parent/child" select="grandchild[1]"
      position="last-child">
    <rapi:text>CHANGED</rapi:text>
  </rapi:replace-insert>
</rapi:patch>
Example: Replacing or Inserting an Attribute in One Operation

The following patch replaces /parent/child/@my-attr and inserts /parent/child/@new-attr.

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-insert select="@my-attr" context="/parent/child">
    REPLACED
  </rapi:replace-insert>
  <rapi:replace-insert select="@new-attr" context="/parent/child">
    <rapi:attribute-list new-attr="INSERTED" />
  </rapi:replace-insert>
</rapi:patch>

The following table illustrates the results of applying this patch:

Before UpdateAfter Update
<parent>
  <child />
  <child my-attr="val2" />
  <child />
</parent>
<parent>
  <child new-attr="INSERTED"/>
  <child my-attr="REPLACED"          new-attr="INSERTED"/>
  <child new-attr="INSERTED"/>
</parent>
Example: Deleting an Element or Attribute

The following patch deletes all <grandchild/> elements and deletes an attribute from the <parent/> element.

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:delete select="/parent/child/grandchild"/>
  <rapi:delete select="/parent/@my-attr"/>
</rapi:patch>

The table below shows how applying the patch changes the target document.

Before UpdateAfter Update
<parent my-attr="val">
  <child/>
  <child>
    <grandchild/>
    <grandchild/>
  </child>
</parent>
<parent>
  <child/>
  <child></child>
</parent>

JSON Patch Reference

This section summarizes the structure of the JSON patch structured used to describe partial updates to JSON documents. Each section includes a syntax summary, a description of component JSON properties, and an example.

This topic covers the following properties of a patch:

The insert, replace, replace-insert, and delete patch operations all require constructing one or more path expressions that identify the content to be changed. To accurately address JSON with XPath, you need to understand the MarkLogic JSON document model. For details, see Traversing JSON Documents Using XPath in the Application Developer's Guide.

pathlang

Specify the path language (JSONPath or XPath) with which the patch specifies the document components targeted by an patch operation, such as the value of the select and context properties of a replace-insert operation.

The default value is "xpath", so you only need to include this property in your patch when you want to use JSONPath.

The use of JSONPath is deprecated.

The following example sets the path language to JSONPath:

{ "pathlang": "jsonpath",
  "patch": [
    { "insert": {
        "context": "$.parent.child",
        "position": "last-child",
        "content": { "new-key": "new-value" }
  }}
]}

To learn more about addressing JSON with XPath, see Limitations of JSON Path Expressions and Traversing JSON Documents Using XPath in the Application Developer's Guide.

patch

A patch is the top level wrapper object for applying partial updates to JSON documents. A JSON patch has the following structure. A patch object can contain multiple operations.

{"patch": [
    insert,
    replace,
    replace-insert,
    delete,
    replace-library 
] }

All subobjects are optional. The operations (insert, replace, replace-insert) can occur multiple times. You can only have one replace-library, and it is only required if you use user-defined functions to generate replacement content.

For example, the following patch describes an insert and a replace operation. For more examples, see JSON Examples of Partial Update.

{"patch": [
  { "insert": {
      "context": "/parent/child",
      "position": "last-child",
      "content": { "new-key": "new-value" }
  }},
  { "replace": {
      "select": "/parent/child[0]",
      "content": "new-value"
    }}
]}
insert

Insert new JSON properties or array elements. An insert operation has the following structure:

{
  "insert": {
    "context": path-expr,
    "position": pos-selector,
    "content": new-content,
    "cardinality": occurrence
  }
}

For example, if /parent/child is the path to an array valued JSON property, then the following patch inserts the value "INSERTED" after each element of the array:

{"insert": {
    "context": "/parent/child",
    "position": "after",
    "content": "INSERTED"
}}

The following table summarizes the components of an insert operation.

JSON PropertyReq'dDescription
"context": path-exprY

An XPath or JSONPath expression that selects an existing JSON property or array element on which to operate. The expression can select multiple items.

If no matches are found for the context expression, the operation is silently ignored.

The path expression is restricted to the subset of XPath (or its JSON Path equivalent) that can be used to define an index. For details, see Path Expressions Usable in Index Definitions.

"position" : pos-selectorYWhere to insert the content, relative to the JSON property or value selected by context. The pos-selector must be one of "before", "after", or "last-child". For details, see Specifying Position in JSON.
"content": new-contentYThe new content to be inserted.
cardinality: occurrenceN

The required occurrence of matches to position. If the number of matches does not meet the expectation, the operation fails and an error is returned. Allowed values:

  • Zero or one matches required: ? (question mark)
  • Exactly one match required: . (period)
  • Zero or more matches required: * (asterisk)
  • One or more matches required: + (plus)

Default: * (The occurrence requirement is always met).

replace

Replace an existing JSON property or array element. If no matching JSON property or array element exists, the operation is silently ignored. A replace operation has the following structure:

{ "replace": {
    "select" : path-expr,
    "content" : new-value,
    "cardinality": occurrence,
    "apply" : func-name
  }
}

For example, the following patch replaces the value of the first array element in the JSON property named "child" that is a subobject of the JSON property named "parent":

{"patch": [
  {"replace": {
      "select": "/parent/child[1]",
      "content": "REPLACED"
  }}
]}

For additional examples, see JSON Examples of Partial Update.

You can use apply to specify a content generation builtin or user-defined function for generating dynamic content. For details, see Constructing Replacement Data on the Server.

The following table summarizes the parts of a replace operation.

ComponentReq'dDescription
"select": path-exprY

An XPath or JSONPath expression that selects the JSON property or array element to replace. If no matches are found for the select expression, the operation is silently ignored.

The path expression is restricted to the subset of XPath that can be used to define an index (or its JSONPath equivalent). For details, see Path Expressions Usable in Index Definitions.

The selected item(s) cannot be the target of any other operation in the patch. The ancestor of the selected item may not be modified by a delete, replace, or replace-insert operation in the same patch.

"content" : new-valueN

The replacement value. If there is no new-value, you must specify a content generation function using apply.

If select targets a property and new-value is an object, the operation replaces the entire target property. Otherwise, the operation replaces just the value.

cardinality: occurrenceN

The required occurrence of matches to select. If the number of matches does not meet the expectation, the operation fails and an error is returned. Allowed values:

  • Zero or one matches required: ? (question mark)
  • Exactly one match required: . (period)
  • Zero or more matches required: * (asterisk)
  • One or more matches required: + (plus)

Default: * (The occurrence requirement is always met).

"apply": func-nameN

The localname of a replacement content generation function. If you do not specify a function, the operation must include a content property.

If you name a user-defined function, the patch must include a replace-library property that describes the XQuery library module containing the function implementation.

For details, see Constructing Replacement Data on the Server.

replace-insert

Replace a property or array item; if there are no existing matching properties, perform an insertion operation instead. A replace-insert operation has the following structure:

{ "replace-insert": {
    "select" : path-expr,
    "context" : path-expr,
    "position": pos-selector,
    "cardinality": occurrence,
    "content" : replacement-content,
    "apply" : func-name
  }
}

Optionally, you can supply a builtin or user-defined content generation function name using apply. If the function does not expect parameters, you can omit content when using apply. For details, see Constructing Replacement Data on the Server.

For example, the following patch replaces the value of "target" with "new-value" in the array value of /parent/child, if it exists. If no such value is found, then "new-value" is inserted as the last element in the array addressable as /parent/array-node('child').

{ "patch": [
    { "replace-insert" : {
        "select": "/parent/child[. = 'target']",
        "context": "/parent/array-node('child')",
        "position": "last-child",
        "content": "new-value"
    }}
]}

For additional examples, see JSON Examples of Partial Update.

The following table summarizes the parts of a replace-insert operation.

ComponentReq'dDescription
"select" : path-exprY

An XPath or JSONPath expression that selects the property or array element to replace. If no matches are found for the select expression, context and position are used to attempt an insert. If no match is found for context, the operation does nothing.

The path expression is restricted to the subset of XPath that can be used to define an index, or its JSONPath equivalent. For details, see Path Expressions Usable in Index Definitions.

The selected item(s) cannot be the target of any other operation in the patch. The ancestor of the selected item may not be modified by a delete, replace, or replace-insert operation in the same patch.

replacement-contentN

The content with which to replace the selected value. If there is no replacement-content, you must specify a content generation function using apply.

If select targets a property and replacement-content is an object, the operation replaces the entire target property. Otherwise, the operation replaces just the value.

"context" : path-exprY

An XPath or JSONPath expression that selects an existing property or array element on which to operate. The expression can select multiple items.

If no matches are found for the either the select or context expression, the operation is silently ignored.

The path expression is restricted to the subset of XPath that can be used to define an index, or its JSONPath equivalent. For details, see Path Expressions Usable in Index Definitions.

The ancestor of the selected node may not be modified by a delete, replace, or replace-insert operation in the same patch.

"position" : pos-selectorN

If select does not match anything, where to insert the content, relative to the property or value selected by context. The pos-selector must be one of "before", "after", or "last-child". For details, see Specifying Position in JSON.

Default: last-child.

cardinality: occurrenceN

The required occurrence of matches to position. If the number of matches does not meet the expectation, the operation fails and an error is returned. Allowed values:

  • Zero or one matches required: ? (question mark)
  • Exactly one match required: . (period)
  • Zero or more matches required: * (asterisk)
  • One or more matches required: + (plus)

Default: * (The occurrence requirement is always met).

"apply" : func-nameN

The localname of a replacement content generation function. If you do not specify a function, the operation must include a content property.

If you name a user-defined function, the patch must include a replace-library property that describes the XQuery library module containing the function implementation.

For details, see Constructing Replacement Data on the Server.

delete

Remove a property or array element. A delete operation has the following structure:

{ "delete": { 
    "select" : path-expr,    "cardinality": occurrence
} }

For example, the following patch deletes any JSON property named child that is contained in the property named parent.

{"patch": [
  { "delete" : { "select": "/parent//child" } }
]}

For additional examples, see JSON Examples of Partial Update.

The following table summarizes the parts of a delete operation.

ComponentReq'dDescription
select : path-exprY

An XPath or JSONPath expression that selects the JSON property or array element to remove. If no matches are found for the select expression, the operation is silently ignored.

The path expression is restricted to the subset of XPath that can be used to define an index, or its JSONPath equivalent. For details, see Path Expressions Usable in Index Definitions.

The selected item(s) cannot be the target of any other operation in the patch. The ancestor of the selected item may not be modified by a delete, replace, or replace-insert operation in the same patch.

cardinality: occurrenceN

The required occurrence of matches to select. If the number of matches does not meet the expectation, the operation fails and an error is returned. Allowed values:

  • Zero or one matches required: ? (question mark)
  • Exactly one match required: . (period)
  • Zero or more matches required: * (asterisk)
  • One or more matches required: + (plus)

Default: * (The occurrence requirement is always met).

replace-library

Specify an XQuery library module that contains user-defined replacement content generation functions. These functions can be used in the apply property of a replace or replace-insert operation. A patch can contain at most one replace-library. For details, see Constructing Replacement Data on the Server.

For example, the following patch uses the function my-ns:my-func in the library module in the with the URI /my.domain/my-module.xqy.

{ "patch": [
   {"replace-library": {
     "at": "/my.domain/my-module.xqy",
     "ns": "my-ns"
   }},
   {"replace": {
     "select": "/inventory[name eq 'orange']/price",
     "apply": "my-func"
   }}
]}

For additional examples, see Constructing Replacement Data on the Server.

The following table summarizes the parts of a <replace-library/> element.

ComponentReq'dDescription
"at": lib-module-pathNThe path to the XQuery library module containing user-defined replacement content generation functions. The module must be installed in the modules database associated with the REST API instance.
"ns": func-namespaceNThe module namespace alias defined by the at module.

JSON Examples of Partial Update

This section includes the examples of applying a partial update with a JSON patch. The following topics are covered:

To understand these examples, it is important that you understand how to use XPath to address JSON document components. For details, see Working With JSON in the Application Developer's Guide.

Example cURL Commands

The examples in this section can be applied using commands similar to the following, assuming you copy the sample input document to a file called example.json and the patch to a file called patch.json.

# Windows users, see Modifying the Example Commands for Windows 

# Create the example document
$ curl --anyauth --user user:password -X PUT -d @./example.json \
    -i -H "Content-type: application/json" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.json
# Apply the patch
$ curl --anyauth --user user:password -X POST -d @./patch.json \
    -i -H "Content-type: application/json" \
    -H "X-HTTP-Method-Override: PATCH" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.json
# Retrieve the results
curl --anyauth --user user:password -X GET \
    -i -H "Accept: application/json" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.json
Example: Insert

The following patch inserts a new property in the root object in the first child position, inserts a new array element value in the child3 subject object, and inserts a new property in the root object in first child position.

{ "patch": [
    { "insert": {
          "context": "/parent/child1",
          "position": "before",
          "content": { "INSERT1": "INSERTED1" }
    }},
    { "insert": {
          "context": "/parent/child3[1]",
          "position": "after",
          "content": "INSERT2"
    }},
    { "insert": {
          "context": "/parent",
          "position": "last-child",
          "content": { "INSERT3": "INSERTED3" }
    }}
  ] }

The following table shows how applying the patch changes the target document.

Before UpdateAfter Update
{ "parent": {
    "child1": { 
      "grandchild": "value" 
    },
    "child2": "simple",
    "child3": [ "av1", "av2" ],
    "child4": [ 
      { "a1" : "v1"}, 
      { "a2": "v2" } 
    ]
} }
{ "parent":{
    "INSERT1": "INSERTED1", 
    "child1": {
      "grandchild":"value"
    }, 
    "child2": "simple", 
    "child3": ["av1","av2","INSERT2"],
    "child4": [
      {"a1":"v1"},
      {"a2":"v2"}
    ], 
    "INSERT3":"INSERTED3"
} }

For more details on the JSON document model and traversing JSON documents with XPath, see Working With JSON in the Application Developer's Guide.

Example: Replace

The following patch replaces one property with another, replaces the simple value of a property, and replaces the array value of a property.

{ "patch": [
    { "replace": {
        "select": "/parent/child1",
        "content": { "REPLACE1": "REPLACED1" }
    }},
    { "replace": {
        "select": "/parent/child2",
        "content": "REPLACED2"
    }},
    { "replace": {
        "select": "/parent/array-node('child3')",
        "content": [ "REPLACED3a", "REPLACED3b" ]
    }}
]}

The following table shows how applying the patch changes the target document.

Before UpdateAfter Update
{ "parent": {
    "child1": {
      "grandchild": "value"
    },
    "child2": "simple",
    "child3": [ "av1", "av2" ]
} }
{ "parent": {
    "child1": {
      "REPLACE1": "REPLACED1"
    },
    "child2": "REPLACED2",
    "child3": [
      "REPLACED3a",
      "REPLACED3b"
    ]
} }

For more details on the JSON document model and traversing JSON documents with XPath, see Working With JSON in the Application Developer's Guide.

Example: Replace-Insert

The following patch demonstrates replace-insert on array values. The patch includes two operations for each child of parent, one that results in a replace and one that results in an insert.

{ "patch": [
    { "replace-insert": {
        "select": "/parent/child1[1]",
        "context": "/parent/array-node('child1')",
        "position": "last-child",
        "content": "REPLACED1"
    }},
    { "replace-insert": {
        "select": "/parent/child1[3]",
        "context": "/parent/child1[2]",
        "position": "after",
        "content": "INSERTED1"
    }},
    { "replace-insert": {
        "select": "/parent/child2[. = 'c2_v1']",
        "context": "/parent/node('child2')",
        "position": "last-child",
        "content": "REPLACED2"
    }},
    { "replace-insert": {
        "select": "/parent/child2[. = 'INSERTED2']",
        "context": "/parent/node('child2')",
        "position": "last-child",
        "content": "INSERTED2"
    }},
    { "replace-insert": {
        "select": "/parent/child3[c3_a]",
        "context": "/parent/node('child3')",
        "position": "last-child",
        "content": { "REPLACED3" : "REPLACED_V" }
    }},
    { "replace-insert": {
        "select": "/parent/child3[INSERTED3]",
        "context": "/parent/node('child3')",
        "position": "last-child",
        "content": { "INSERTED3" : "INSERTED_V" }
    }}
]}

The following table shows how applying the patch changes the target document.

Before UpdateAfter Update
{"parent": {
  "child1": [ "c1_v1", "c1_v2" ],
  "child2": [ "c2_v1", "c2_v2" ],
  "child3": [
    { "c3_a": "c3_v1" },
    { "c3_b": "c3_v2" }
  ]
} }
{ "parent": {
    "child1": [
      "REPLACED1",
      "c1_v2",
      "INSERTED1"
    ],
    "child2": [
      "REPLACED2",
      "c2_v2",
      "INSERTED2"
    ],
    "child3": [
      { "REPLACED3": "REPLACED_V" },
      { "c3_b": "c3_v2" },
      { "INSERTED3": "INSERTED_V" }
    ]
} }

Recall that the select path identifies the content to replace. When working with an array item, an absolute path is usually required. For example, consider the following patch operation:

{ "replace-insert": {
    "select": "/parent/child1[1]",
    "context": "/parent/array-node('child1')",
    "position": "last-child",
    "content": "REPLACED1"
}}

The goal is to replace the value of the first item in the array value of /parent/child1 if it exists. If the array is empty, insert the new value. That is, one of these two transformations takes place:

{"parent"{"child1": ["c1_v1", "c1_v2"], ... }
    ==> {"parent"{"child1": ["REPLACED1", "c1_v2"], ... }
{"parent"{"child1": [], ... }
    ==> {"parent"{"child1": ["REPLACED1"], ... }

The select expression, /parent/child1[1], must target an array item value, while the context expression must target the containing array node by referencing /parent/array-node("child1"). You cannot make the select expression relative to the context expression in this case.

Note that while you can target an entire array item value with replace-insert, you cannot target just the value of a property. For example, consider the following array:

"child3": [ 
  { "c3_a": "c3_v1" },
  { "c3_b": "c3_v2" } 
]

You can use replace-insert on the entire object-valued item { "c3a": "c3v1" }, as is done in the example. However, you cannot construct an operation that targets just the value of the property in the object ("c3_v1"). The replacement of the property value is fundamentally different from inserting a new object in the array. A property (as opposed to the containing object) can only be replaced by deleting it and then inserting a new item.

For more details on the JSON document model and traversing JSON documents with XPath, see Working With JSON in the Application Developer's Guide.

Example: Delete

The following patch removes properties and array elements at various levels of a document. See the table below for how the patch affects the example content.

{{ "patch": [
    { "delete" : {
        "select": "/props/node('anyType')"
    }},
    { "delete" : {
        "select": "/props/objOrLiteral"
    }},
    { "delete" : {
        "select": "/props/array-node('arrayVal')"
    }},
    { "delete" : {
        "select": "/arrayItems/all"
    }},
    { "delete" : {
        "select": "/arrayItems/byPos[1]"
    }},
    { "delete" : {
        "select": "/arrayItems/byVal[. = 'DELETE']"
    }},
    { "delete" : {
        "select": "/arrayItems/byName[DELETE]"
    }}
] }

The following table shows how applying the patch changes the target document.

Before UpdateAfter Update
{ "props": {
    "anyType": [1, 2],
    "objOrLiteral": "anything",
    "arrayVal": [3, 4]
  },
  "arrayItems": {
    "byPos": ["DELETE", "PRESERVE"],
    "byVal": ["DELETE", "PRESERVE"],
    "byName": [ 
      {"DELETE": 5}, 
      {"PRESERVE": 6} 
    ],
    "all": ["DELETE1", "DELETE2"]
  }
}
{ "props": { },
  "arrayItems": {
    "byPos": [ "PRESERVE" ],
    "byVal": [ "PRESERVE" ],
    "byPropName": [
      { "PRESERVE": 6 }
    ],
    "all": [ ]
  }
}

Note that when removing properties, you must either use a named node step to identify the target property or be aware of the value type. Consider these 3 select path expressions from the example program:

/props/node('anyType')
/props/objOrLiteral'
/props/array-node('arrayVal')

The first path is type agnostic: /props/node("anyType"). This expression selects any nodes named anyType, regardless of the type of value. The second path, /props/objOrLiteral, deletes the entire name-value pair only if the value is an object or literal (string, number, boolean). If the value of the property is an array, it deletes the items in the array. That is, this operation applied to the original document will delete the contents of the array, not the arrayVal property:

pb.remove('/props/arrayVal')
==> arrayVal: [ ]

The third form, /props/array-node('arrayVal'), deletes the arrayVal property, but it will only work on properties with array type. Therefore, if you need to delete a property by name without regard to its type, use path of the form /path/to/parent/node('propName').

For more details on the JSON document model and traversing JSON documents with XPath, see Working With JSON in the Application Developer's Guide.

Patching Metadata

You can partially update metadata for any document type, including XML, JSON, text, and binary. To update a portion of the metadata for a document, send a POST request to the /documents service with a URI of the following form.

http://host:port/version/documents?uri=document-uri&category=category

The category parameter should either be metadata or one of the metadata sub-categories of collections, properties, permissions, metadata-values, and quality. The category affects what pieces of metadata MarkLogic Server considers for updating. For example, if you use category=permissions, but your patch only contains an operation applied to properties, the patch has no effect.

Category does not affect how you construct path expressions in a patch. That is, context and select paths within the patch are not relative to the specified metadata category.

The request body must contain an XML or JSON patch descriptor. Construct your metadata patch to apply to the structure described in Working with Metadata, using the same syntax and semantics as for a content patch. For details, see in XML Patch Reference or JSON Patch Reference.

Document quality (the <quality/> XML element or quality JSON property) can only be the target of a replace operation.

You can update content and metadata in the same request by including operations on both inyour patch descriptor and setting the category request parameter appropriately. For example:

http://localhost:8000/LATEST/documents?uri=/my.xml&category=content&category=metadata

When patching content and metadata in the same request, the usual content type restrictions apply: You can only apply content patches to XML and JSON documents, and you cannot use a JSON patch descriptor on XML content, or vice versa.

Since JSON does not use qualified names, you can construct a context or select path that addresses both metadata and content items. If your content contains JSON properties with the same name as the metadata properties (properties, collections, permissions, metadataValues, and quality) and you cannot construct an unambiguous path, do not combine content and metadata partial updates in the same request.

Paths for context and select can be relative or absolute with respect to the metadata structure. For example, both of the following operations insert a document into a new collection, using different context paths.

XML JSON
<rapi:insert
   context="/rapi:metadata/rapi:collections"
   position="last-child">
  <rapi:collection>INSERTED</rapi:collection>
</rapi:insert>
{"insert": {
   "context":"$.collections",
   "position":"last-child",
   "content":"INSERTED"
}}
<rapi:insert 
    context="rapi:collections"
    position="last-child">
  <rapi:collection>INSERTED</rapi:collection>
</rapi:insert>
{"insert": {
   "context":"collections",
   "position":"last-child",
   "content":"INSERTED"
}}

The following example is a patch that performs multiple metadata updates: Add the target document to a new collection, add or replace a property, and replace the document quality:

FormatPatch
XML
<rapi:patch xmlns:rapi="http://marklogic.com/rest-api"
     xmlns:prop="http://marklogic.com/xdmp/property">
  <rapi:insert context="/rapi:metadata/rapi:collections"
      position="last-child">
    <rapi:collection>INSERTED</rapi:collection>
  </rapi:insert>
  <rapi:replace-insert select="my-property"
      context="/rapi:metadata/prop:properties"
      position="last-child">
    <my-property>REPLACE_OR_INSERT</my-property>
  </rapi:replace-insert>
  <rapi:replace select="/rapi:metadata/rapi:quality">2</rapi:replace>
</rapi:patch>
JSON
{ "patch" : [
  { "insert": {
      "context": "$.collections",
      "position": "last-child",
      "content" : "INSERTED"
  }},
  { "replace-insert": {
      "select": "my-property",
      "context": "$.properties",
      "position": "last-child",
      "content" : { "my-property": "REPLACE_OR_INSERT" }
  }},
  { "replace": {
      "select": "$.quality",
      "content" : 2
  }}
  ], 
  "pathlang": "jsonpath" 
}

When patching properties using XML, you must explicitly declare the namespace http://marklogic.com/xdmp/property on the root <patch/> element because the <properties/> metadata element is in a different namespace from the other metadata elements.

Values metadata (category=metadata-values) is represented differently in XML and JSON. The equivalent of the XML metadata-values element is a JSON property named metadataValues, and where XML has metadata-value child elements to represent each key-value pair, JSON has key:value child properties. For details, see Working with Metadata.

Constructing Replacement Data on the Server

You can use builtin or user-defined replacement functions to generate the content for a partial update operation dynamically on MarkLogic Server. The builtin functions support simple arithmetic and string manipulation. For example, you can use a builtin function to increment the current value of numeric data or concatenate strings. For more complex operations, create and install a user-defined function.

The following topics are covered:

Using a Replacement Constructor Function

You can only use replacement generator functions with the replace and replace-insert operations. The REST Client API ships with a set of builtin arithmetic and string manipulation functions. If your patch includes a replace-library specification, you can also create user-defined functions.

The following example increases every price in an inventory element or object by 10% by using the builtin ml.multiply function to multiply the current value by 1.1.

XMLJSON
<rapi:patch
    xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace 
    select="/inventory/price"
    apply="ml.multiply">1.1</rapi:replace>
</rapi:patch>
{"patch": [
  {"replace": {
    "select": "/inventory/price",
    "apply": "ml.multiply",
    "content": 1.1
  } }
]}

The following example uses a user-defined function to double the price in every inventory element or object. The dbl function is implemented by the XQuery module /my.domain/my-lib.xqy, which uses the module namespace http://my/ns. The dbl function does not expect any argument values, so there is no content included in the replace operation.

XMLJSON
<rapi:patch
    xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-library
    at="/my.domain/my-lib.xqy" 
    ns="http://my/ns" />
  <rapi:replace 
    select="/inventory/price"
    apply="dbl"/>
</rapi:patch>
{"patch": [
  {"replace-library": {
    "at": "/my.domain/my-lib.xqy",
    "ns": "http://my/ns"
  } },
  {"replace": {
    "select": "/inventory/price",
    "apply": "dbl"
  } }
] }

Use the following guidelines to specify a replacement generator function in a patch descriptor:

  1. Set the value of the apply XML attribute or JSON property to the function localname.
  2. If the function is a user-defined function, include a replace-library XML element or JSON property in the patch to specify the XQuery module containing the function implementation. The implementation module must be installed in the modules database of the REST API instance.
  3. If the function expects arguments, specify them as child nodes of the operation element in XML or in the value of the content JSON property. See the example below.

If a function expects a single argument, you can place the value directly in the content or in a <rapi:value/> XML element or $value JSON property. The previous example places the value (1.1) directly in the content section of the patch.

If a function expects multiple arguments, enclose each value in a <rapi:value/> XML element or $value JSON property. For example, the following patch uses the builtin function ml.concat-between which expects two string parameters.

XMLJSON
<rapi:patch
    xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace select="/my/data"
      apply="ml.concat-between">
    <rapi:value>my prefix</rapi:value>
    <rapi:value>my suffix</rapi:value>
  </rapi:replace>
</rapi:patch>
{"patch": [
  {"replace": {
    "select": "/my/data",
    "apply": "ml.concat-between",
    "content": [
      { "$value" : "my prefix" },
      { "$value" : "my suffix" }
    ]
  } }
] }

In XML, the implicit datatype of arguments is xs:untypedAtomic, though a generator function can cast the value to the expected parameter type. For example, the builtin arithmetic functions cast to xs:double when possible. Use @xsi:type to explicitly specify a type on a value.

The following example specifies an explicit xs:date datatype. Note that you must declare the xs and xsi namespace aliases on the root of your patch:

<rapi:patch
    xmlns:rapi="http://marklogic.com/rest-api"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <rapi:replace select="/elem" apply="my-func">
    <rapi:value xsi:type="xs:date">2013-03-15</rapi:value>
  </rapi:replace>
</rapi:patch>

In JSON, use $datatype to explicitly specify an xsi:type for a $value. Omit the xs: namespace prefix on the type name. The following example explicitly specifies xs:date as the datatype for the replace function input parameter:

{"patch": [
  {"replace": {
    "select": "/my/data",
    "apply": "ml.concat-between",
    "content": [
      { "$value" : "2013-03-15", $datatype: "date"}
    ]
  } }
] }
Using Builtin Replacement Constructors

The REST Client API includes several builtin server-side functions you can use to dynamically generate the content for a replace or replace-insert operation. For example, you can use a builtin function to increment the current value of a data item.

All the builtin function names have a 'ml.' prefix. You cannot use this prefix on user-defined replacement generator functions.

The builtin arithmetic functions are equivalent to the XQuery +, -, *, and div operators, and accept values castable to the same datatypes. That is, numeric, date, dateTime, duration, and Gregorian (xs:gMonth, xs:gYearMonth, etc.) values. The operand type combinations are as supported by XQuery; for details, see http://www.w3.org/TR/xquery/#mapping. All other functions expect values castable to string.

The table below lists the builtin replacement generator functions provided by the REST Client API. In the table, $current represents the current value of the target of the replace operation; $arg and $argN represent argument values passed in by the patch.

Function NameNum ArgsEffect
ml.add1$current + $arg
ml.subtract1$current - $arg
ml.multiply1$current * $arg
ml.divide1$current div $arg
ml.concat-before1fn:concat($arg, $current)
ml.concat-after1fn:concat($current, $arg)
ml.concat-between2fn:concat($arg1, $current, $arg2)
ml.substring-before1fn:substring-before($current, $arg)
ml.substring-after1fn:substring-after($current, $arg)
ml.replace-regex2 or 3fn:replace($current, $arg1, $arg2, $arg3)
Writing a User-Defined Replacement Constructor

You can create your own XQuery functions to generate content for the replace and replace-insert operations. To make your functions available to partial update operations, install it in the modules database associated with your REST API instance; for details, see Installing or Updating a User-Defined Replace Library.

A user-defined replacement generator function has the following interface:

declare function module-ns:func-name(
  $current as node()?,
  $args as item()*
) as node()*

The current node ($current) is empty only when the function is invoked as an insert on behalf of a replace-insert.

The argument list supplied by the operation is passed through $args. You are responsible for validating the argument values. If the content supplied by the patch operation is JSON array or a sequence of XML <rapi:value/> elements, then $args is the result of calling the fn:data function to each value. If an explicit datatype is specified by the patch operation, the cast is applied before invoking your function.

Your function should report errors using fn:error and RESTAPI-SRVEXERR. For details, see Reporting Errors.

The following example XQuery library module implements two replacement content constructors, mylib:dbl to double the value in a target node, and my-lib:min to replace the value of a node with the minimum value of the current node and a sequence of values passed in by the patch. For simplicity, this example skips most of the input data validation that a production implementation should include.

xquery version "1.0-ml";

module namespace my-lib = "http://marklogic.com/example/my-lib";

(: Double the value of a node :)
declare function my-lib:dbl(
  $current as node()?,
  $args as item()*
) as node()*
{
  if ($current/data() castable as xs:decimal)
  then
    let $new-value := xs:decimal($current) * 2
    return
      typeswitch($current)
      case number-node()       (: JSON :)
        return number-node {$new-value}
      case element()           (: XML :)
        return element {fn:node-name($current)} {$new-value}
      default return fn:error((), "RESTAPI-SRVEXERR",
         ("400", "Bad Request", 
          fn:concat("Not an element or number node: ",
                    xdmp:path($current))
          ))
  else fn:error((), "RESTAPI-SRVEXERR",
    ("400", "Bad Request", fn:concat("Non-decimal data: ", $current)))
};

(: Find the minimum value in a sequence of value composed of :)
(: the current node and a set of input values. :)
declare function my-lib:min(
  $current as node()?,
  $args as item()*
) as node()*
{
  if ($current/data() castable as xs:decimal)
  then
    let $new-value := fn:min(($current, $args))
    return
      typeswitch($current)
      case element()           (: XML :)
        return element {fn:node-name($current)} {$new-value}
      case number-node()       (: JSON :)
        return number-node {$new-value}
      default return fn:error((), "RESTAPI-SRVEXERR",
        ("400", "Bad Request", 
           fn:concat("Not an element or number node: ",
                     xdmp:path($current))))
  else fn:error((), "RESTAPI-SRVEXERR", ("400", "Bad Request",
         fn:concat("Non-decimal data: ", $current)))
};

The following patch snippet uses the above module, assuming the module is installed in the modules database with the URI /ext/replace/my-lib.xqy:

Format

Patch
XML
<rapi:patch
    xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-library at="/ext/replace/my-lib.xqy" 
      ns="http://marklogic.com/example/my-lib" />
  <rapi:replace select="/parent/child" apply="dbl"/>
</rapi:patch>
JSON
{"patch": [
  {"replace-library": {
    "at": "/ext/replace/my-lib.xqy",
    "ns": "http://marklogic.com/example/my-lib"
  } },
  {"replace": {
    "select": "/parent/child",
    "apply": "dbl"
  } }
] }
Installing or Updating a User-Defined Replace Library

To install user-defined replacement constructor functions, place your function(s) into an XQuery library module and install the module in the modules database associated with your REST API instance.

Your implementation, including any dependent libraries, must be installed in the modules database associated with your REST API instance. The simplest way to achieve this is to use the /ext service that is part of the REST Client API. For details, see Managing Dependent Libraries and Other Assets.

To install your library using the /ext service, send a PUT request to a URL of the following form, with your library module implementation in the request body:

http://host:port/version/ext/your/path/your-lib.xqy?perm:perm

If you do not specify any permissions, the module will only be executable by users with the rest-admin role.

When you use /ext to install your module, MarkLogic Server installs the XQuery library module in the request body into the modules database associated with your REST API instance, at a URI derived from the above URL, beginning with /ext. For example, the following command installs the library module with the URI /ext/replace/my-lib.xqy:

$ curl --anyauth --user user:password -X PUT -d @./my-lib.xqy \
    -i -H "Content-type: application/xquery" \
  http://localhost:8000/LATEST/ext/replace/my-lib.xqy?perm:my-user:execute

The following patch snippet uses the module URI in the value of the at XML attribute or JSON property in a replace-lib directive:

<rapi:replace-library at="/ext/replace/my-lib.xqy"
   ns="http://marklogic.com/example/my-lib" />

If your library module requires dependent libraries, you can install them in the same database directory (/ext/replace/, in the above example). The /ext service allows you to manage at both the directory and file level. For details, see Managing Dependent Libraries and Other Assets.

Use the same procedure to update your implementation after initial installation.

How Position Affects the Insertion Point

The insert and replace-insert patch operations include a position member that defines the point of insertion when coupled with a context expression. This section describes the details of how position affects the insertion point. The following topics are covered:

Specifying Position in XML

The @position of an <insert/> or <replace-insert/> operation specifies where to insert new content, relative to the context node or attribute. Position can be one of the following values:

  • before: Insert before the element or attribute identified by @context. The @context must not target the root element.
  • after: Insert after the element or attribute identified by @context. The @context must not target the root element.
  • last-child: Insert as the last child of the element identified by @context. The @context must target a node, not an attribute.

The last-child is not meaningful for operations on attributes.

The following table shows example combinations of @context and @position and the resulting insertion point for new content. The insertion point is indicated by ***.

Context

Position

Example Insertion Point
/parent/child[1]before
<parent>
  ***
  <child>
    <grandchild/>
  </child>
  <child a1="val" a2="val"/>
</parent>
/parent/childafter
<parent>
  <child>
    <grandchild/>
  </child>
  ***
  <child a1="val" a2="val"/>
  ***
</parent>
/parent/child[1]last-child
<parent>
  <child>
    <grandchild/>
    ***
  </child>
  <child a1="val" a2="val"/>
</parent>
/parent/child/@a1before
<parent>
  <child>
    <grandchild/>
  </child>
  <child *** a1="val" a2="val"/>
</parent>
/parent/child/@a1after
<parent>
  <child>
    <grandchild/>
  </child>
  <child a1="val" *** a2="val"/>
</parent>
/parent/childlast-child
<parent>
  <child ***>
    <grandchild/>
  </child>
  <child a1="val" a2="val" *** />
</parent>
/parent/child/@a1last-childError. You cannot insert into an attribute.
Specifying Position in JSON

The position property in a patch specifies where to insert new content relative to the context item. Position can be one of the following values:

  • before: Insert before the property or array element value selected by context.
  • after: Insert after the property or array element value selected by context.
  • last-child: Insert into the value of the target property or array, in the position of last child. The value selected by context must be an object or array.

The same usage applies whether inserting on behalf of an insert operation or a replace-insert operation.

You cannot use last-child to insert a property as an immediate child of the root node of a document. Use before or after instead. For details, see Limitations of JSON Path Expressions.

The following table shows example combinations of context and position and the resulting insertion point for new content. The insertion point is indicated by ***.

ContextPositionExample Insertion Point

/theTop/node("child1")

a property

after
{"theTop" : {
    "child1" : [ "val1", "val2" ],
    ***
    "child2" : [ 
      {"one": "val1"}, 
      {"two": "val2"} 
    ]
} }

/theTop/child1[1]

an array item

before
{"theTop" : {
    "child1" : [ *** "val1", "val2" ],
    "child2" : [ 
      {"one": "val1"}, 
      {"two": "val2"} 
    ]
} }

/theTop/array-node("child1")

an array

last-child
{"theTop" : {
    "child1" : [ "val1", "val2" ***],
    "child2" : [ 
      {"one": "val1"}, 
      {"two": "val2"} 
    ]
} }

/node("theTop")

an object

last-child
{"theTop" : {
    "child1" : [ "val1", "val2"],
    "child2" : [ 
      {"one": "val1"}, 
      {"two": "val2"} 
    ]
    ***
} }

/theTop/child1

a value

last-child

Error because the selected value is not an object or array. For example, an error results if the target document has the following contents:

{"theTop" : {     "child1" : "val1", "child2" : [...] } }

Path Expressions Usable in Index Definitions

A patch operation uses one or more path expressions to identify the target of the operation.

For performance reasons, the path expression used in the context and select specifications of a patch operation is restricted to an XPath expression that can be used to define a path range index. For details, see Understanding Path Range Indexes in Administrator's Guide.

JSONPath expressions are constrained by equivalent limitations. For example, the only operators you can use in predicate expressions are comparison operators.

The supported subset of XPath does not enable addressing anonymous nodes in JSON data. For details, see Limitations of JSON Path Expressions.

Limitations of JSON Path Expressions

The limitations on path range index expressions and the JSON docment model impose some special limitations on constructing path expressions for JSON that do not arise when addressing XML.

  • You can only address nodes by name, which means you cannot include anonymous node path steps, such as node() or array-node().
  • You can only address items in an array with a bare-name step. That is, /my/a[1] is allowed, but /my/array-node("a")[1] is not. Thus, you can only use array names that are valid XML NCNames in path expressions.
  • You cannot address an array nested inside another array.

The rest of this section explores some of the cases which thus become unsupported:

To learn more about path expressions over JSON, see Traversing JSON Documents Using XPath in the Application Developer's Guide.

Inserting a Child Under the Root Node

You cannot insert a property immediately under the root of a JSON document using 'last-child' position because 'last-child' position requires you to select the parent node as the insertion context, and the root object node is an anonymous node. For example, you cannot use 'last-child' to insert in the following position:

{ "one": 1, "two": 2 }
                    ^insert here

You can still insert a property in the desired position using the 'after' or 'before' positions because these positions use the (named) siblings as the insertion context.

Addressing Items in Nested Arrays

You must address items in a nested array as if they're part of the outermost enclosing array because nested arrays are flattened. For example, if your data contains the following array value:

{"a": [1, [2, 3], 4]}

You must treat the data as if the array value is [1, 2, 3, 4]. The path expression /a[2] addresses only the value 2, not the nested array [2, 3]. It is not possible to address the nested array at all because it is an unnamed node.

Replacing the Name and Value of a Property

You cannot replace an entire property (name-value pair). You must delete the existing property and then insert the new one. However, you can replace the just value of a property.

For example, if your data contains the following object:

{ "a": 1, "c": 3}

You cannot use a single replace patch operation to change it into {"a": 1, "b": 2} by replacing just the "c":3 property. This is because "c" is the name of value 3. The name-value pair is anonymous and cannot be addressed as a unit.

One implication of the anonymity of properties is that you cannot use a replace-insert operation to perform an operation like 'insert this property if it doesn't exist; otherwise, replace it.' Nor can you use separate insert and replace operations to achieve the same effect because the supported subset of XPath for JSON does not enable testing for the non-existence.

Introduction to JSONPath

When applying partial updates to JSON documents, you can specify the target of a patch operation using JSONPath expressions in the context and select properties of a patch operation. This section gives a brief overview of JSONPath syntax. For a complete description of JSONPath, see http://goessner.net/articles/JsonPath/.

The default path expression language is XPath. Using JSONPath is deprecated. To use JSONPath, you must set the pathlang property to jsonpath. For details, see pathlang.

  • The anonymous root of a JSON object is identified by a dollar sign ($).
  • Delimit path steps by either periods (.) or square brackets ([]).
  • Use square brackets to enclose predicate expressions and array element selectors.
  • The first element in an array has position 0, which is different from XPath.
  • Predicates must be enclosed in ?(), as in [?(@.value < 10)].

The set of JSONPath expressions usable for document operations like partial update is limited to JSONPath equivalent to XPath expressions that can be used to define a path range index. For details, see Path Expressions Usable in Index Definitions.

The following table contains some simple JSONPath examples that satisfy the restrictions. The bold text indicates what is selected by each expression.

JSONPathSelection
$.parent
$['parent']
{ "parent": [
    {"child": [ "one", "two" ] },
    {"child": {"grandchild": "three" }}
] }
$.parent[0]
$['parent'][0]
{ "parent": [
    {"child": [ "one", "two" ] },
    {"child": {"grandchild": "three" }}
] }
$.parent['child']
{ "parent": [
    {"child": [ "one", "two" ] },
    {"child": {"grandchild": "three" }}
] }
$.parent.child[*]['one']
{ "parent": [
    {"child": [ "one", "two" ] },
    {"child": {"grandchild": "three" }}
] }

Performing a Lightweight Document Check

Use this method to:

  • Test for the existence of a document in the database.
  • Retrieve a document identifier without fetching content or metadata when content versioning is enabled.
  • Determine the total length of a document for setting the end boundary when iterating over content ranges in binary documents.

To perform a document check, construct a HEAD request to a URL of the form:

http://host:port/version/documents?uri=document_uri

The following example sends a HEAD request for an XML document:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X HEAD \
    -H "Accept: application/xml" \
    http://localhost:8000/LATEST/documents?uri=/xml/box.xml
...
Content-type: application/xml
Server: MarkLogic
Connection: close

HTTP/1.1 200 Document Retrieved
vnd.marklogic.document-format: xml
Content-type: application/xml
Server: MarkLogic
Connection: close

Issuing the same request on a non-existent document returns status 404:

$ curl --anyauth --user user:password -X HEAD \
    -H "Accept: application/xml" \
    http://localhost:8000/LATEST/documents?uri=/xml/dne.xml
...
Content-type: application/xml
Server: MarkLogic
Connection: close

HTTP/1.1 404 Not Found
Content-type: application/xml
Server: MarkLogic
Connection: close

Removing Documents from the Database

This section covers using the /documents and /search services to remove documents from the database. The following topics are covered:

Removing a Document or Metadata

To remove a document and its metadata from the database, construct a DELETE request with a URL of the form:

http://host:port/version/documents?uri=document_uri

When you delete a document, its metadata is also deleted.

You can remove multiple documents (or reset their metadata) by specifying multiple URIs. Optimistic locking is not supported when you specify multiple URIs.

To remove or reset just metadata for a document, construct a DELETE request with a URL of the form:

http://host:port/version/documents?uri=document_uri&category=metadata_category

The category parameter can appear multiple times, with the values described in Metadata Categories. Resetting permissions resets the document permissions to the default permissions for the current user. Resetting quality resets the document quality to the default (0).

Deleting a binary document with extracted metadata stored in a separate XHTML document also deletes the XHTML metadata document. For more information, see Working with Binary Documents.

Removing Multiple Documents

In addition to removing multiple documents by URI as described in Removing a Document or Metadata, you can remove all documents in a collection or in a database directory.

To remove all documents in a collection, send a DELETE request with a URL of the following form:

http://host:port/version/search?collection=collection_name

Similarly, to remove all documents in a directory, send a DELETE request with a URL of the following form:

http://host:port/version/search?directory=directory_name

Where directory_name is the name of a directory in the database. The directory name must include a trailing '/'.

You can specify only one collection or one directory in a single request.

Failing to specify either a directory or a collection removes all documents in the database.

Removing a document also removes its metadata. Deleting a binary document with extracted metadata stored in a separate XHTML document also deletes the XHTML metadata document. For more information, see Working with Binary Documents.

The following example remove all documents in the '/plays' directory:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -i -X DELETE \
    http://localhost:8000/LATEST/search?directory=/plays/
...
HTTP/1.1 204 Updated
Server: MarkLogic
Content-Length: 0
Connection: close

Removing All Documents

To remove all documents in the database, send a DELETE request with a URL of the following form:

http://host:port/version/search

Clearing the database requires the rest-admin role or equivalent.

There is no confirmation or other safety net when you clear the database in this way. Creating a backup is advised.

The following example removes all documents and metadata from the content database:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -i -X DELETE \
    http://localhost:8000/LATEST/search

Removing a Semantic Graph

To remove the triples in a graph, send a DELETE to the /graphs service with a URL of one of the following forms:

http://host:port/version/graphs?graph=graph-urihttp://host:port/version/graphs?default

The /graphs service only affects triples stored in a triple document (a document with a <sem:triples> root element). Therefore, if the graph includes triples embedded in normal XML documents, the embedded triples are unaffected and the graph will continue to exist.

You can also delete a graph by using an empty graph in a PUT request to the /graphs service.

For details, see Semantics Developer's Guide.

Using Optimistic Locking to Update Documents

An application using optimistic locking creates a document only when the document does not exist and updates or deletes a document only when the document has not changed since this application last changed it. However, optimistic locking does not actually involve placing a lock on document.

Optimistic locking is useful in environments where integrity is important, but contention is rare enough that it is useful to minimize server load by avoiding unnecessary multi-statement transactions.

This section covers the following topics:

Understanding Optimistic Locking

Consider an application that reads a document, makes modifications, and then updates the document in the database with the changes. The traditional approach to ensuring document integrity is to perform the read, modification, and update in a multi-statement transaction. This holds a lock on the document from the point when the document is read until the update is committed. However, this pessimistic locking blocks access to the document and incurs more overhead on the App Server.

With optimistic locking, the application does not hold a lock on a document between read and update. Instead, the application saves the document state on read, and then checks for changes at the time of update. The update fails if the document has changed between read and update. This is a conditional update.

Optimistic locking is useful in environments where integrity is important, but contention is rare enough that it is useful to minimize server load by avoiding unnecessary multi-statement transactions. The REST Client API also supports multi-statement transactions. For details, see Managing Transactions.

The REST Client API uses content versioning to implement optimistic locking. When content versioning is enabled, MarkLogic Server associates an opaque version id with a document. The version id changes each time you update the document. The version id is returned when you read a document, and you can pass it back in an update or delete operation to test for changes prior to commit.

Content versioning in the REST Client API does not implement document versioning. When content versioning is enabled, MarkLogic Server does not keep multiple versions of a document or track what changes occur. The version id can only be used to detect that a change occurred.

Using optimistic locking in your application requires the following steps:

  1. Enabling Optimistic Locking by setting the update-policy REST instance configuration property.
  2. Obtaining a Version Id for documents you wish to conditionally update.
  3. Applying a Conditional Update by supplying the version id in your update operations.

Enabling Optimistic Locking

Enable optimistic locking using the update-policy instance configuration property, as described in Configuring Instance Properties. For example, the following command sets update-policy to version-optional:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    -d'{"update-policy":"version-optional"}' \
    -H "Content-type: application/json" \
    http://localhost:8000/LATEST/config/properties

The update-policy property can be set to merge-metadata (the default), version-required, or version-optional. Set the property to version-required if you want every document update or delete operation to use optimistic locking. Set the property to version-optional to allow selective use of optimistic locking.

The update-policy property replaces the older content-versions policy; content-versions is deprecated. Setting update-policy to version-optional is equivalent to setting content-versions to optional. Setting update-policy to version-required is equivalent to setting content-versions to required.

The table below describes how each setting for this property affects document operations.

SettingEffect
merge-metadataThis is the default setting. If you insert, update, or delete a document that does not exist, the operation succeeds. If a version id is present, it is ignored.
version-optionalIf you insert, update or delete a document that does not exist, the operation succeeds. If a version id is present, the operation fails if the document exists and the current version id does not match the supplied version id.
version-requiredIf you update or delete a document without supplying a version id and the document does not exist, then the operation succeeds; if the document exists, the operation fails. If a version id is present, the operation fails if the document exists and the current version id does not match the version in the header.
overwrite-metadataThe behavior is the same as merge-metadata, except that metadata in the request overwrites any pre-existing metadata, rather than being merged into it. This setting disables optimistic locking.

Obtaining a Version Id

When optimistic locking is enabled, a version id is included in the response when you read documents. You can only obtain a version id if optimistic locking is enabled by setting update-policy; for details, see Enabling Optimistic Locking.

You can obtain a version id for use in conditional updates in the following ways:

  • Perform a single document GET or HEAD request to /documents. The version id is returned in the ETag header of the response.
  • Perform a multi-document read request, such as GET /documents with multiple URIs. The version id is returned in the Content-Disposition header of each content part in the multipart response.

You can mix and match these methods for obtaining a version id.

The following example command returns the version id for a single document:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -i -X HEAD \
    -H "Accept: application/xml" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.xml
...
HTTP/1.1 200 Document Retrieved
Content-type: application/xml
ETag: "13473172834878540"
Server: MarkLogic
Connection: close

The following example command returns the version id for multiple documents in a single request:

curl --anyauth --user user:password -X GET -i \
  -H "Accept: multipart/mixed; boundary=BOUNDARY" \
  http://localhost:8000/LATEST/documents?uri=doc1.xml\&uri=doc2.json
...
--BOUNDARY
Content-Type: application/xml
Content-Disposition: attachment; filename="doc1.xml"; category=content; format=xml; versionId=14075140367230760
Content-Length: 87

...document contents...
--BOUNDARY
Content-Type: application/json
Content-Disposition: attachment; filename="doc2.json"; category=content; format=json; versionId=14075140367230760
Content-Length: 15

...document contents...

For details on multi-document requests, see Reading and Writing Multiple Documents.

Applying a Conditional Update

To apply a conditional update, supply a version id on your update operation. The version id is ignored if optimistic locking is not enabled; for details, see Enabling Optimistic Locking.

When a document update includes a version id, MarkLogic Server checks for changes to the version id before committing the update (or delete) operation. If the version id has changed, the update fails. In a multi-document update, the entire batch of updates is rejected if any conditional update fails.

You can supply a version id on an update in the following ways:

  • Pass the version id in the If-Match header of a single document PUT, POST, PATCH, or DELETE request to /documents.
  • Pass the version id in the Content-Disposition header of a content part in a multi-document POST request to /documents.

The following example performs a conditional update of a single document by passing a version id in the If-Match request header:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -i -X PUT -d"<modified-data/>"  \
    -H "Content-type: application/xml" \
    -H "If-Match: 13473769780393030" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.xml

The following POST body excerpt for a multi-document update unconditionally updates the first document (doc1.xml) and conditionally updates the second document (doc2.xml). Note that if update-policy is set to version-required, the request will fail if doc1.xml already exists because no version id is supplied for it.

--BOUNDARY
Content-Type: application/xml
Content-Disposition: attachment; filename="doc1.xml"
Content-Length: 87

...document contents...
--BOUNDARY
Content-Type: application/json
Content-Disposition: attachment; filename="doc2.json"; versionId=14075140367230760
Content-Length: 15

...document contents...
--BOUNDARY--

The following command applies the above update, if the complete POST body is in the file /example/optimistic-locking.

$ curl --anyauth --user user:password -X POST -i \
  --data-binary @/example/optimistic-locking \
  -H "Content-type: multipart/mixed; boundary=BOUNDARY" \
  -H "Accept: application/xml" \
  http://localhost:8000/LATEST/documents

For details on multi-document requests, see Reading and Writing Multiple Documents.

Example: Updating a Document Using Optimistic Locking

The following example demonstrates using optimistic locking to conditionally update a single document. Both a successful and a failed update are shown.

This example uses a PUT request to demonstrate a conditional update. For a POST, PATCH, or DELETE operation, follow a similar procedure, passing the version id in the If-Match header of your update or delete request. For an equivalent multi-document update, pass the version id in the Content-Disposition header of each content part.

  1. If content versioning is not already enabled, enable it. The following command sets update-policy to version-optional:
    # Windows users, see Modifying the Example Commands for Windows 
    $ curl --anyauth --user user:password -X PUT \
        -d'{"update-policy":"version-optional"}' \
        -H "Content-type: application/json" \
        http://localhost:8000/LATEST/config/properties
  2. Insert an example document into the database to initialize the example:
    $ curl --anyauth --user user:password -i -X PUT -d"<data/>" \
        -H "Content-type: application/xml" \
        http://localhost:8000/LATEST/documents?uri=/docs/example.xml
  3. Unconditionally retrieve a local copy of the document. Note the version id is returned in the ETag header:
    $ curl --anyauth --user user:password -i -X GET \
        -H "Accept: application/xml" \
        http://localhost:8000/LATEST/documents?uri=/docs/example.xml
    ...
    HTTP/1.1 200 Document Retrieved
    vnd.marklogic.document-format: xml
    Content-type: application/xml
    ETag: "13473769780393030"
    Server: MarkLogic
    Content-Length: 47
    Connection: close
    
    <?xml version="1.0" encoding="UTF-8"?>
    <data/>
  4. Conditionally update the document by sending new contents with the version id from Step 3 in the If-Match header. Since the document has not changed since Step 3, the update succeeds. The document version id is changed by this operation.
    $ curl --anyauth --user user:password -i -X PUT -d"<modified-data/>"  \
        -H "Content-type: application/xml" \
        -H "If-Match: 13473769780393030" \
        http://localhost:8000/LATEST/documents?uri=/docs/example.xml
    ...
    HTTP/1.1 204 Content Updated
    Server: MarkLogic
    Content-Length: 0
    Connection: close
  5. To illustrate update failure when the version ids do not match, attempt to update the document again, using the version id from Step 3, which is no longer current. The update fails.
    $ curl --anyauth --user user:password -i -X PUT -d"<data/>"  \
        -H "Content-type: application/xml" \
        -H "If-Match: 13473769780393030" \
        http://localhost:8000/LATEST/documents?uri=/docs/example.xml
    ...
    HTTP/1.1 412 Precondition Failed
    Content-type: application/xml
    Server: MarkLogic
    Content-Length: 370
    Connection: close
    
    <?xml version="1.0"?>
    <rapi:error xmlns:rapi="http://marklogic.com/rest-api">
      <rapi:status-code>412</rapi:status-code>
      <rapi:status>Precondition Failed</rapi:status>
      <rapi:message-code>RESTAPI-CONTENTWRONGVERSION</rapi:message-code>
      <rapi:message>RESTAPI-CONTENTWRONGVERSION: (err:FOER0000) Content version mismatch:  uri: /docs/example.xml version: 13473788748796580</rapi:message>
    </rapi:error>

Client-Side Cache Management Using Content Versioning

You can use content versioning to refresh a copy of a document stored on the client only if the document in the database has been modified. This section covers the following topics:

Enabling Content Versioning

Enable content versioning using the update-policy instance configuration property, as described in Configuring Instance Properties.

The default update policy is merge-metadata. If you set update-policy to version-required or version-optional, content versioning is enabled, and a GET or HEAD request to /documents returns a version id in the ETag response header.

Enabling content versioning can affects document insertion, update, and deletion. For details, see Enabling Optimistic Locking.

Content versioning in the REST Client API does not implement document versioning. When content versioning is enabled, MarkLogic Server does not keep multiple versions of a document or track what changes occur. The version id can only be used to detect that a change occurred.

Using Content Versioning for Cache Refreshing

When content versioning is enabled, sending a GET request to /documents with a version id in the If-None-Match HTTP header only retrieves a new copy of the document if it has changed relative to the version id in the header.

The update-policy property replaces the older content-versions property; content-versions is deprecated. Setting update-policy to version-optional is equivalent to setting content-versions to optional. Setting update-policy to version-required is equivalent to setting content-versions to required.

Follow the procedure below to use content version for cache refreshing. For a complete example, see Example: Refreshing a Cached Document.

  1. If content versioning is not already enabled, set the update-policy instance configuration property to version-optional or version-required; see Enabling Content Versioning.
  2. Read a document by making a GET request to /documents. The response includes the version id in the ETag header.
  3. Save the version id returned in the ETag response header.
  4. When you want to refresh the cache, send another GET request to /documents with the version id from Step 2 in the If-None-Match HTTP header.

    If the current version id matches the one in the If-None-Match header, no document is retrieved and MarkLogic Server returns status 304. If the current version id differs from the one in the If-None-Match header, the document is returned, along with the new version id in the ETag response header.

Example: Refreshing a Cached Document

The following example demonstrates using content versioning to refresh a client-side document cache. The example includes includes a case where the document is unchanged in the database, as well as the case where the local cache is out of date.

  1. If content versioning is not already enabled, enable it by setting update-policy to version-optional:
    # Windows users, see Modifying the Example Commands for Windows 
    $ curl --anyauth --user user:password -X PUT \
        -d'{"update-policy":"version-optional"}' \
        -H "Content-type: application/json" \
        http://localhost:8000/LATEST/config/properties
  2. Insert a sample document into the database to initialize the example:
    $ curl --anyauth --user user:password -i -X PUT -d"<data/>" \
        -H "Content-type: application/xml" \
        http://localhost:8000/LATEST/documents?uri=/docs/example.xml
  3. Unconditionally retrieve a local copy of the document, as if to cache it. Note the version id is returned in the ETag header:
    $ curl --anyauth --user user:password -i -X GET \
        -H "Accept: application/xml" \
        http://localhost:8000/LATEST/documents?uri=/docs/example.xml
    ...
    HTTP/1.1 200 Document Retrieved
    vnd.marklogic.document-format: xml
    Content-type: application/xml
    ETag: "13473769780393030"
    Server: MarkLogic
    Content-Length: 47
    Connection: close
    
    <?xml version="1.0" encoding="UTF-8"?>
    <data/>
  4. Conditionally retrieve the document, as if to refresh the cache. Supply the version id from Step 3 in the If-None-Match header. Since the document has not changed, no content is retrieved.
    $ curl --anyauth --user user:password -i -X GET \
        -H "Accept: application/xml" \
        -H "If-None-Match: 13473769780393030" \
        http://localhost:8000/LATEST/documents?uri=/docs/example.xml
    ...
    HTTP/1.1 304 Content Version Not Modified
    ETag: "13473769780393030"
    Server: MarkLogic
    Content-Length: 0
    Connection: close
  5. Update the document in the database, which changes the version id.
    $ curl --anyauth --user user:password -i -X PUT -d"<modified-data/>"  \
        -H "Content-type: application/xml" \
        http://localhost:8000/LATEST/documents?uri=/docs/example.xml
    ...
    HTTP/1.1 204 Content Updated
    Server: MarkLogic
    Content-Length: 0
    Connection: close
  6. Conditionally retrieve the document again, as if to refresh the cache. Supply the version id from Step 3 in the If-None-Match header. Since the document has changed, the content is retrieved. The new version id is also returned via the ETag header.
    $ curl --anyauth --user user:password -i -X GET \
        -H "Accept: application/xml" \
        -H "If-None-Match: 13473769780393030" \
        http://localhost:8000/LATEST/documents?uri=/docs/example.xml
    ...
    HTTP/1.1 200 Document Retrieved
    vnd.marklogic.document-format: xml
    Content-type: application/xml
    ETag: "13473770707201670"
    Server: MarkLogic
    Content-Length: 56
    Connection: close
    
    <?xml version="1.0" encoding="UTF-8"?>
    <modified-data/>

Working with Binary Documents

This section covers the following topics:

Types of Binary Documents

This section provides a brief summary of binary document types. For details, see Working With Binary Documents in the Application Developer's Guide.

MarkLogic Server can store binary documents in three representations:

  • Small binary documents are stored entirely in the database.
  • Large binary documents are stored on disk with a small reference fragment in the database. The on-disk content is managed by MarkLogic Server.
  • External binary documents are stored on disk with a small reference fragment in the database. However, the on-disk content is not managed by MarkLogic Server.

Small and large binary documents are created automatically for you, depending on the document size. External binary documents cannot be created using the REST Client API.

Large binary documents can be streamed out of the database using Range requests. For details, see Streaming Binary Content.

Streaming Binary Content

Streaming binary content out of the database avoids loading the entire document into memory. You can stream binary documents by sending GET requests to /documents that include range requests under following conditions:

  • The size of the binary content returned is over the large binary size threshold. For details, see Working With Binary Documents in the Application Developer's Guide.
  • The request is for content only. That is, no metadata is requested.
  • The MIME type of the content is determinable from the Accept header or the document URI file extension.

The following example requests the first 500K of the binary document with URI /binaries/large.jpg:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -i -o piece.jpg -X GET \
    -H "Accept: application/jpg" -r "0-511999" \
    http://localhost:8000/LATEST/documents?uri=/binaries/large.jpg
...
HTTP/1.1 206 Binary Part Retrieved
Content-type: application/jpeg
Content-Range: bytes 0-511999/533817
Server: MarkLogic
Content-Length: 511999
Connection: close

Working with Temporal Documents

Most document write operations, such as PUT, POST, and PATCH on the /documents service enable you to work with temporal documents by exposing the following request parameters:

  • temporal-collection: The URI of the temporal collection into which the new document should be inserted, or the name of the temporal collection that contains the document being updated.
  • temporal-document: The 'logical' URI of the document in the temporal collection; the temporal collection document URI. This is equivalent to the first parameter of the temporal:statement-set-document-version-uri XQuery function or of the temporal.statementSetDocumentVersionUri Server-Side JavaScript function.
  • source-document: The temporal collection document URI of the document being operated on. This parameter is only applicable when updating existing documents. This parameter facilitates working with documents with user-maintained version URIs.
  • system-time: The system start time for an update or insert.

To create temporal documents, you must use an endpoint that accepts a caller-specified URI. You cannot use POST /v1/documents?extension={ext} to create a temporal document because MarkLogic generates the document URI when you use this method.

During an update operation, if you do not specify source-document or temporal-document, then the uri request parameter indicates the source document. If you specify temporal-document, but do not specify source-document, then the temporal-document URI identifies the source document.

The uri request parameter always refers to the output document URI. When the MarkLogic manages the version URIs, the document URI and temporal document collection URI have the same value. When the user manages version URIs, they can be different.

Use POST /v1/documents/protection to protect a temporal document from operations such as update, delete, and wipe for a specified period of time. This method is equivalent to calling the temporal:document-protect XQuery function or the temporal.documentProtect Server-Side JavaScript function.

Use POST /v1/temporal/collections/{name} to advance LSQT on a temporal collection. This method is equivalent to calling the temporal:advance-lsqt XQuery function or the temporal.advanceLsqt Server-Side JavaScript function.

For more details, see the Temporal Developer's Guide and specific methods in the MarkLogic REST API Reference.

Working with Metadata

The /documents service supports inserting, updating, and retrieving document metadata. Metadata is the properties, collections, permissions, quality and key-value metadata of content.

Metadata manipulation is most often exposed by the REST Client API through a category URL parameter. Metadata can be passed around as either XML or JSON, usually controlled through either an HTTP header or a format URL parameter. For specifics, see the MarkLogic REST API Reference.

This section covers the following topics related to metadata storage and retrieval:

Metadata Categories

Where the REST Client API accepts specification of metadata categories, the following categories are recognized:

  • collections
  • permissions
  • properties
  • quality
  • metadata-values
  • metadata

The metadata category is shorthand for all the other categories. That is, metadata includes collections, permissions, properties, key-value metadata and quality.

Some requests also support a content category as a convenience for requesting or updating both metadata and document content in a single request.

The metadata-values category represents a 'metadata field'. This type of metadata is expressed as simple key-value pairs and you must configure a database field on a key before you can search it. While similar in some ways to document properties, this type of metadata is stored separately from the associated document and can be operated on like any other field. For more details, see Metadata Fields in the Administrator's Guide.

XML Metadata Format

Metadata contains information about document collections, permissions, properties, quality, and key-value metadata. The format is fully described by the schema file:

MARKLOGIC_INSTALL_DIR/Config/restapi.xsd

The following is a summary of the structure of the metadata. All elements are in the namespace http://marklogic.com/rest-api. You can have 0 or more <collection/>, <permission/>, <metdata-values>, <metadata-value/> or property child elements. There can be only one <quality/> element. The element name and contents of each document property element depends on the property.

<metadata xmlns="http://marklogic.com/rest-api">
  <collections>
    <collection>collection-name</collection>
  </collections>
  <permissions>
    <permission>
      <role-name>name</role-name>
      <capability>capability</capability>
    </permission>
  </permissions>
  <properties>
    <property-element/>
  </properties>
  <quality>integer</quality>
  <metadata-values>
    <metadata-value key="keyName">value</metadata-value>
  </metdata-values>
</metadata>

The following example shows a document in two collections, with one permission, two properties, and one key-value metadata item.

<rapi:metadata xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:collections>
    <rapi:collection>shapes</rapi:collection>
    <rapi:collection>squares</rapi:collection>
  </rapi:collections>
  <rapi:permissions>
    <rapi:permission>
      <rapi:role-name>hadoop-user-read</rapi:role-name>
      <rapi:capability>read</rapi:capability>
    </rapi:permission>
  </rapi:permissions>
  <prop:properties xmlns:prop="http://marklogic.com/xdmp/property">
    <myprop>this is my prop</myprop>
    <myotherprop>this is my other prop</myotherprop>
  </prop:properties>
  <rapi:quality>0</rapi:quality>
  <rapi:metadata-values>
    <rapi:metadata-value key="level">high</rapi:metadata-value>
  </rapi:metadata-values>
</rapi:metadata>

JSON Metadata Format

Metadata contains information about document collections, permissions, properties, quality, and key-value metadata. A block of metadata can contain multiple collections, permissions and properties, but only 1 quality. The structure of each child of the properties JSON property depends on the property contents.

{
  "collections" : [ string ],
  "permissions" : [
    { 
      "role-name" : string,
      "capabilities" : [ string ]
    }
  ],
  "properties" : {
    property-name : property-value
  },
  "quality" : integer
  "metadataValues": {
    key: value
  }
}

The following example shows a document in two collections, with one permission, two document properties, and two key-value metadata items.

{
  "collections": [
    "shapes",
    "squares"
  ],
  "permissions": [
    {
      "role-name": "hadoop-user-read",
      "capabilities": [
        "read"
      ]
    }
  ],
  "properties": {
    "myprop": "this is my prop",
    "myotherprop": "this is my other prop"
  },
  "quality": 0,
  "metadataValues": {
    "level": "high",
    "rating": 5
  }
}

Working With Document Properties Using JSON

In most cases, your application should work with properties in a consistent format. That is, if you insert or update properties as XML, then you should retrieve them in XML. If you insert or update properties as JSON, you should retrieve them in JSON.

You should only insert document property data as JSON when you can specify the content type, such as through the request or part header of POST /v1/documents. Do not use the prop request parameter of PUT /v1/documents to insert document properties with JSON values as such input is untyped.

Internally, document properties are always stored as XML. When you insert a user-defined property using JSON, the XML representation is an XML element in the namespace http://marklogic.com/xdmp/json/basic with a localname that matches the property name in JSON. When you retrieve the property data as JSON, it is converted from XML back to JSON.

As long as you handle the data consistently, the conversion to and from XML is largely transparent to your application. However, you need to be aware of the XML representation when searching properties. Also, the implementation of transforms and extensions only sees the XML representation.

If you configure an index based on a user-defined property inserted using JSON, treat them as XML elements and use the http://marklogic.com/xdmp/json/basic in your configuration.

Protected system properties such as last-updated cannot be modified by your application. When retrieved as JSON, such protected properties are wrapped in an object with the JSON property name $ml.prop. For example:

{ "properties": {
    "$ml.prop": {
      "last-updated": "2013-11-06T10:01:11-08:00"
    }
} }

The following example code adds a document property expressed as JSON to /my/doc.json:

$ curl --anyauth --user user:password -X PUT -i \
  -H "Content-type: application/json" \
  -d '{"properties": {"pname":"pvalue"}}' \
  'http://localhost:8000/LATEST/documents?uri=/doc/my.json&category=properties'

If you retrieve the properties of /doc/my.json as JSON, you get back what was inserted. The internal representation as XML is hidden from your application. For example:

$ curl --anyauth --user user:password -X GET \
  -H "Accept: application/json" \
  'http://localhost:8000/LATEST/documents?uri=/doc/my.json&category=properties'
{"properties":{"pname":"pvalue"}}

However, if you retrieve the properties as XML or examine them using the Explore feature of Query Console, you will see the XML representation. For example:

$ curl --anyauth --user user:password -X GET \
  -H "Accept: application/xml" \
  'http://localhost:8000/LATEST/documents?uri=/doc/my.json&category=properties'
<rapi:metadata uri="/doc/my.json" ...>
  <prop:properties xmlns:prop="http://marklogic.com/xdmp/property">
    <pname type="string" xmlns="http://marklogic.com/xdmp/json/basic">
      pvalue
    </pname>
  </prop:properties>
</rapi:metadata>

Disabling Metadata Merging

If you use the REST Client API to ingest a large number of documents at one time and you find the performance unacceptable, you might see a small performance improvement by disabling metadata merging. This topic explains the tradeoff and how to disable metadata merging.

When to Consider Disabling Metadata Merging

The performance gain from disabling metadata merging is modest, so you are unlikely to see significant performance improvement from it unless you ingest a large number of documents. You might see a performance gain under one of the following circumstances:

  • Ingesting a large number of documents, one at a time.
  • Updating a large number of documents that share the same metadata or that use the default metadata.

You cannot disable metadata merging in conjunction with update policies version-optional or version-required.

Metadata merging is disabled by default for multi-document write requests, as long as the request includes content for a given document. For details, see Understanding When Metadata is Preserved or Replaced.

Understanding Metadata Merging

Disabling metadata merging effectively eliminates the distinction between inserting a new document and updating an existing document, with respect to metadata handling.

Metadata merging is enabled by default. When you update metadata, the metadata in your update is merged with existing metadata. You can extend or update metadata without re-specifying unmodified document properties, collections, permissions, document quality, or key-value metadata. Similarly, when you update document content without including any metadata updates, the existing metadata is preserved.

When metadata merging is disabled, updating any metadata on a document overwrites all metadata for the document with the metadata provided in the request (plus default values for unspecified metadata categories). A content update that does not include any metadata resets the document metadata to the default values.

For example, if you set permissions on a document when you initially insert it into the database, and then later add the document to a collection while metadata merging is disabled, the permissions will be reset to default document permissions. In order to preserve the pre-existing permissions, you must specify them explicitly in your update.

How to Disable Metadata Merging

Metadata merging is controlled by the update-policy instance configuration property. The default value is merge-metadata.

To disable metadata merging, set update-policy to overwrite-metadata using the procedure described in Configuring Instance Properties. For example:

$ cat props.xml
<properties xmlns="http://marklogic.com/rest-api">
  <update-policy>overwrite-metadata</update-policy>
</properties>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT -d@"./props.xml" \
    -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/config/properties

The equivalent JSON input is shown below:

{
  "update-policy" : "overwrite-metadata"
}

« Previous chapter
Next chapter »
Powered by MarkLogic Server 7.0-4.1 and rundmc | Terms of Use | Privacy Policy