MarkLogic 9 Product Documentation

REST Application Developer's Guide — Chapter 3

Summary of the /documents Service

Use the /documents service to create, read, update, and delete document content and metadata for XML, JSON, text, and binary documents, including XML documents containing embedded semantic RDF triple data. To load documents containing only RDF triples, use the /graph service; for details, see Summary of the /graphs Service.

The following table summarizes the supported operations:

The service supports the following additional document features through request parameters:

• Transaction control
• Transformation of content during ingestion
• Content repair during ingestion
• Transformation of results during document insertion or retrieval
• Conditional document insertion using optimistic locking
• Conditional reads based on content versioning
• Multi-request reads at fixed point-in-time

  XML, JSON and text documents must use UTF-8 encoding.

  The service does not support content conversions except through custom transformations. For example, you cannot retrieve an XML document as JSON.

You can also use POST /v1/documents/protection to protect temporal documents from various levels of update. For details, see Working with Temporal Documents and the MarkLogic REST API Reference.

This chapter only covers single-document operations. For multi-document operations, see Reading and Writing Multiple Documents.

Summary of the /graphs Service

Use the /graphs service to create, read, update, and delete documents containing RDF triples. For triples embedded in XML documents, use the /documents service.

The following table summarizes the supported operations:

Though a graph can include both triples embedded in XML documents and triples from a pure triples document (one with a <sem:triples> root element), the /graphs service does not operate on embedded triples. For example, if you make a DELETE request on a graph that includes embedded triples, the embedded triples and their containing document are unaffected, and therefore, the graph continues to exist.

For details, see Loading Triples Using the REST API in Semantics Developer's Guide.

Loading Content

This section describes how to insert or update an entire document at a user-defined database URI. If you want to have MarkLogic Server generate document URIs automatically, see Automatically Generating Document URIs. You can also modify only a portion of a document; for details, see Partially Updating Document Content or Metadata. For loading multiple documents in a single request, see Reading and Writing Multiple Documents.

To insert or update an XML, JSON, text, or binary document, make a PUT request to a URL of the form:

http://host:port/version/documents?uri=document_uri

When constructing the request:

1. Set the uri parameter to the URI of the destination document in the database.
2. Place the content in the request body.
3. Specify the MIME type of the content in the Content-type HTTP header. The Content-type header does not necessarily determine the document type; for details, see Controlling Input and Output Content Type.

   XML, JSON and text documents must use UTF-8 encoding.

Documents you create with the REST Client API have a read permission for the rest-reader role and an update permission for the rest-writer role. To restrict access, use custom roles. For details, see Controlling Access to Documents and Other Artifacts.

The following example command sends a request to insert the contents of the file ./my.xml into the database as an XML document with URI /xml/example.xml:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT -T ./my.xml \
    -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/documents?uri=/xml/example.xml

If the MIME type is not set in the HTTP Content-type header, MarkLogic Server uses the file extension on the document URI to determine the content format, based on the MIME type mappings defined for your installation. For details, see Controlling Input and Output Content Type.

You can also set metadata such as collections, permissions, and named properties when loading content. See Loading Content and Adding Metadata in the Same Request.

You can have MarkLogic Server generate document URIs for you. For details, see Automatically Generating Document URIs.

Adding Metadata

This section describes inserting or updating metadata using a PUT request to /documents. The following topics are covered:

• Inserting or Updating Metadata
• Example: Replacing One Metadata Category Using XML
• Example: Replacing Multiple Metadata Categories Using XML
• Example: Replacing Multiple Metadata Categories Using JSON

http://host:port/version/documents?uri=document_uri&category=metadata_category

$ cat metadata.xml
<rapi:metadata xmlns:rapi="http://marklogic.com/rest-api"
               xmlns:prop="http://marklogic.com/xdmp/property">
  <rapi:collections>
    <rapi:collection>interesting</rapi:collection>
  </rapi:collections>
</rapi:metadata>
# Windows users, see Modifying the Example Commands for Windows 
$ curl -X PUT -T ./metadata.xml -H "Content-type: application/xml" \
    --anyauth --user user:password \
    'http://localhost:8000/LATEST/documents?uri=/xml/example.xml&category=collections'

$ cat > metadata.xml
<rapi:metadata xmlns:rapi="http://marklogic.com/rest-api"
               xmlns:prop="http://marklogic.com/xdmp/property">
  <rapi:collections>
    <rapi:collection>interesting</rapi:collection>
  </rapi:collections>
  <prop:properties>
    <my-property>value</my-property>
  </prop:properties>
</rapi:metadata>
# Windows users, see Modifying the Example Commands for Windows 
$ curl -X PUT -T ./metadata.xml -H "Content-type: application/xml" \
    --anyauth --user user:password \
    'http://localhost:8000/LATEST/documents?uri=/xml/example.xml&category=collections&category=properties'

$ cat > metadata.json
{
  "collections":["interesting"],
  "properties": {
    "my-property":"name"
  }
}
# Windows users, see Modifying the Example Commands for Windows 
$ curl -X PUT -T ./metadata.json --anyauth --user user:password \
    -H "Content-type: application/json" \
    'http://localhost:8000/LATEST/documents?uri=/xml/example.xml&category=collections&category=properties'

Loading Content and Adding Metadata in the Same Request

You can update content and metadata for a document in a single request using the following methods. You must choose one or the other; they cannot be combined.

• Loading Content and Metadata Using Request Parameters
• Loading Content and Metadata Using a Multipart Message

For loading multiple content and/or metadata for multiple documents in a single request, see Reading and Writing Multiple Documents.

http://host:port/version/documents?uri=doc_uri&metadata_param=value

# Windows users, see Modifying the Example Commands for Windows 
$ curl -X PUT -T ./critter.jpg --anyauth --user user:password \
    -H "Content-type: image/jpeg" \
    'http://localhost:8000/LATEST/documents?uri=/images/critter.jpg&collection=animals&prop:species="canus lupus"'

http://host:port/version/documents?uri=doc_uri&category=content&category=metadata_category

$ cat ./the-body
--BOUNDARY
Content-Type: application/xml

<?xml version="1.0" encoding="UTF-8"?>
<metadata xmlns="http://marklogic.com/rest-api">
  <collections>
    <collection>shapes</collection>
    <collection>squares</collection>
  </collections>
</metadata>
--BOUNDARY
Content-Type: text/xml

<?xml version="1.0" encoding="UTF-8"?>
<data>
  <mybox>
    This is my box. There are many like it, but this one is mine.
  </mybox>
</data>
--BOUNDARY--
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    --data-binary @./the-body \
    -H "Content-type: multipart/mixed; boundary=BOUNDARY" \
    'http://localhost:8000/LATEST/documents?uri=/xml/box.xml&category=collections&category=content'

Automatically Generating Document URIs

MarkLogic Server can automatically generate databse URIs for documents inserted with the REST API. You can only use this feature to create new documents. To update an existing document, you must know the URI; for details, see Loading Content.

To use this feature, send a POST request to /documents that includes the extension request parameter and does not include the uri request parameter. That is, a POST request with a URL of the following form:

http://host:port/version/documents?extension=file-extension

Where extension specifies the document URI file prefix, such as xml or txt. Optionally, you can also include a database directory prefix with the directory request parameter.

The directory prefix should end in a forward slash (/).

MarkLogic Server returns the auto-generated URI in the Location header of the response.

The following example inserts a document into the database with a URI of the form /my/directory/auto-generated.xml:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X POST -d@'./my-content' -i \
  -H "Content-type: application/xml" \
  'http://localhost:8000/LATEST/documents?extension=xml&directory=/my/directory/'
HTTP/1.1 201 Document Created
Location: /my/directory/6041706572796142832.xml
...

You can use the same features with generated URIs as when loading documents with user-defined URIs. For example, you can insert both documents and metadata and apply content transformations. For details, see Loading Content into the Database and the MarkLogic REST API Reference.

Loading Triples

You can use the /graphs service to load semantic triples into the database in several formats. The service implements the W3C Graph Store Protocol, described by the following specification:

http://www.w3.org/TR/2013/REC-sparql11-http-rdf-update-20130321/

The collection lexicon must be enabled on your database when using the semantics REST services or use the GRAPH '?g' construct in a SPARQL query. For details, see Configuring the Database to Work with Triples in the Semantics Developer's Guide.

To load triples, send a PUT or POST request to the /graphs service with one of the following forms of URL, depending upon whether you wish to address a specific named graph or the default graph:

http://host:port/version/graphs?graph=graph-uri 

http://host:port/version/documents?default

For details, see Loading Triples Using the REST API and Supported RDF Triple Formats in Semantics Developer's Guide.

You can also create, update, and delete triples and graphs with SPARQL Update by sending a POST request to the /graphs/sparql service. For example, the following SPARQL update request inserts a triple into the default graph:

PREFIX dc: <http://purl.org/dc/elements/1.1/>
INSERT DATA
{ <http://example/book0> dc:title "A default book"  }

If you place the above SPARQL Update request into a file named update.sparql, then the following request to the /graphs/sparql performs the requested update:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X POST -i 
  --data-binary @./update.sparql \
  -H "Content-type:application/sparql-update" \
  -H "Accept:application/sparql-results+xml" \
  'http://localhost:8000/LATEST/graphs/sparql'

When you put your SPARQL Update request in the HTTP request body, the request Content-type must be application/sparql-update. You can also specify your update using URL encoded form data. For more details and examples, see SPARQL Update with the REST Client API in the Semantics Developer's Guide.

Controlling Access to Documents Created with the REST API

Documents you create with the REST Client API have a read permission for the rest-reader role and an update permission for the rest-writer role by default. To restrict access to particular users, create custom roles rather than assigning users to the default rest-* roles. For example, you can use a custom role to restrict users in one group from seeing documents created by another.

For details, see Controlling Access to Documents and Other Artifacts.

Transforming Content During Ingestion

You can transform content during ingestion by applying custom transform. A transform is an XQuery module or XSLT stylesheet you write and install using /config/transforms/{name}. For details, see Working With Content Transformations.

To apply a transform when creating or updating a document, add the transform parameter to your request. If the transform expects parameters, specify them using trans:paramName parameters. That is, your request should be of the form:

http://host:port/version/documents?...&transform=name&trans:arg=value

The following example applies a transform installed under the name example that expects a parameter named reviewer:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    -d@./the-body -H "Content-type: application/xml" \
    'http://localhost:8000/LATEST/documents?uri=/doc/theDoc.xml&transform=example&trans:reviewer=me'

For a complete example, see XQuery Example: Adding an Attribute During Ingestion or XSLT Example: Adding an Attribute During Ingestion.

Retrieving the Contents of a Document

To retrieve a document from the database, construct a GET request of the following form:

http://host:port/version/documents?uri=doc_uri

HTTP content type negotiation is not supported. If the HTTP Accept header is not set, MarkLogic Server uses the file extension on the document URI to determine the response content type, based on the server-wide MIME type mapping definitions. See Mimetypes in the Admin Interface.

Though content negotiation is not supported, you can use the transform feature to apply server-side transformations to the content before the response is constructed. For details, see Working With Content Transformations.

To retrieve multiple documents in a single request, see Reading and Writing Multiple Documents.

Retrieving Metadata About a Document

To retrieve metadata about a document without retrieving the contents, construct a GET request of the following form:

http://host:port/version/documents?uri=doc_uri&category=metadata_category

Where category can appear multiple times, with the values described in Metadata Categories.

When constructing the request:

1. Set the category parameter to the type of metadata to retrieve. Specify category multiple times to request more than one type of metadata.
2. Specify the metadata content type (XML or JSON) in the HTTP Accept header or the format parameter.

For details on metadata categories and formats, see Working with Metadata.

Use the format parameter or the Accept header to specify the format of the metadata. If both format and Accept are set, format takes precedence. If neither format nor Accept is specified, XML is assumed. For details, see Controlling Input and Output Content Type.

To retrieve metadata as XML, set format to xml or set the Accept header to application/xml. To retrieve metadata as JSON, set format to json or set the Accept header to application/json.

Retrieving Content and Metadata in a Single Request

To retrieve content and metadata for a document in a single request, construct GET request to a URL of the form:

http://host:port/version/documents?uri=doc_uri&category=content&category=metadata_category

Where category can appear multiple times, with the values described in Metadata Categories.

The request response is a multipart/mixed message, with the metadata in the first body part and content in the second body part. The Content-Type headers for the parts are determined as follows:

• The MIME type of the metadata part is determined by the format parameter, which you can set to either xml or json; the default is xml. For details on metadata format, see Working with Metadata.
• The MIME type of the content part is determined by the server-wide MIME type mapping for the document URI extension. See Mimetypes in the Admin Interface.

The following example command retrieves a document and its metadata in a single request:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    -H "Accept: multipart/mixed;boundary=BOUNDARY" \
    'http://localhost:8000/LATEST/documents?uri=/xml/box.xml&category=metadata&category=content&format=xml'
--BOUNDARY
Content-Type: application/xml
Content-Length: 518

<?xml version="1.0" encoding="UTF-8"?>
<rapi:metadata uri="/xml/box.xml"
    xsi:schemaLocation="http://marklogic.com/rest-api/database dbmeta.xsd" 
    xmlns:rapi="http://marklogic.com/rest-api"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <rapi:collections>
    <rapi:collection>shapes</rapi:collection>
    <rapi:collection>squares</rapi:collection>
  </rapi:collections>
  <rapi:permissions/>
  <prop:properties xmlns:prop="http://marklogic.com/xdmp/property"/>
  <rapi:quality>0</rapi:quality>
  <rapi:metadata-values/>
</rapi:metadata>
--BOUNDARY
Content-Type: text/xml
Content-Length: 128

<?xml version="1.0" encoding="UTF-8"?>
<data><mybox>This is my box. There are many like it, but this one is mine.</mybox></data>
--BOUNDARY--

HTTP content negotiation is not supported, but custom server-side content transformations can be applied using the transform parameter. For details, see Retrieving the Contents of a Document and Working With Content Transformations.

Transforming Content During Retrieval

You can apply custom transforms to a document before returning it to the requestor. A transform is a JavaScript module, XQuery module, or XSLT stylesheet you write and install using /config/transforms/{name}. For details, see Working With Content Transformations.

You can configure a default transform that is automatically applied whenever a document is retrieved. You can also specify a per-request transform using the transform request parameter. If there is both a default transform and a per-request transform, the transforms are chained together, with the default transform running first. Thus, the output of the default transform is the input to the per-request transform:

To configure a default transformation, set the document-transform-out configuration parameter for the REST Client API instance. Instance-wide parameters are set using /config/properties. For details, see Configuring Instance Properties.

To specify a per-request transform, add the transform parameter to your request. If the transform expects parameters, specify them using trans:paramName parameters. That is, your request should be of the form:

http://host:port/version/documents?...&transform=name&trans:arg=value

The following example applies a transform installed under the name example that expects a parameter named reviewer:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/xml" \
    'http://localhost:8000/LATEST/documents?uri=/doc/theDoc.xml&transform=example&trans:reviewer=me'

Introduction to Content and Metadata Patching

A partial update is an update you apply to a portion of a document or metadata. For example, inserting an XML element or attribute or changing the value associated with a JSON property. You can only apply partial content updates to XML and JSON documents. You can apply partial metadata updates to any document type.

A patch is a partial update descriptor, expressed in XML or JSON, that tells MarkLogic Server where to apply an update and what update to apply. A patch is a wrapper XML element or JSON object that encapsulates one or more update operations.

Use a partial update to do the following operations:

• Add, replace, or delete an XML element, XML attribute, or JSON sub-object of an existing document.
• Add, replace, or delete a subset of the metadata of an existing document. For example, modify a permission or insert a document property.
• Dynamically generate replacement content or metadata on MarkLogic Server using builtin or user-defined functions. For details, see Constructing Replacement Data on the Server.

To perform a partial update, send a POST or PATCH request to the /documents service with a patch in the request body. For example, the following patch descriptor inserts a new <child/> element as the last child of the element located through the XPath expression /data:

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/data" position="last-child">
    <child>the last one</child>
  </rapi:insert>
</rapi:patch>

Assuming the above patch is saved to the file patch.xml, the following command uses a POST request to apply the patch to the document with the URI /doc/example.xml.

$ curl --anyauth --user user:password -X POST -d @./patch.xml -i \
    -H "Content-type: application/xml" \
    -H "X-HTTP-Method-Override: PATCH" \
    http://localhost:8000/LATEST/documents?uri=/doc/example.xml

The following example is an equivalent request using PATCH instead of POST:

$ curl --anyauth --user user:password -X PATCH -d @./patch.xml -i \
    -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/documents?uri=/doc/example.xml

For details, see Basic Steps for Patching Documents.

You can apply multiple updates in a single request. You can patch both content and metadata in a single request.

Content negotiation is not supported: A content patch must match the content type of the target document. That is, you cannot submit a JSON patch for an XML document or an XML patch for a JSON document.

If a patch contains multiple operations, they are applied independently to the target document. That is, within the same patch, one operation does not affect the context or select results or the content changes of another. Each operation in a patch is applied independently to every matched node. If any operation in a patch fails with an error, the entire patch fails.

Content transformations are not directly supported in a partial update. However, you can implement a custom replacement content generation function to achieve the same effect. For details, see Constructing Replacement Data on the Server.

Basic Steps for Patching Documents

To perform a partial update of an XML or JSON document, or of document metadata, send a POST or PATCH request to the /documents service with a URL of the following form:

http://host:port/version/documents?uri=doc_uri

The body of the request must contain a content patch, as described in XML Patch Reference and JSON Patch Reference. You can patch metadata and content in the same request by using the category request parameter; for details, see Patching Metadata.

When constructing your request, follow these guidelines:

1. Set the uri parameter to the URI of the target document in the database.
2. Set the category request parameter to content, metadata, and/or one of the metadata subcategories. The default is content.
3. Set the Content-type to either application/xml or application/json. When patching content, you must use the content type that matches the target document; only XML and JSON are supported.
4. Place the XML or JSON patch in the request body.
5. If you POST, set the request header X-HTTP-Method-Override to PATCH to tell MarkLogic Server this is a partial update request.

New content that is added or replaced can be specified directly in the patch, or generated on MarkLogic Server by a replacement content generator function. For details, see Constructing Replacement Data on the Server.

XML Patch Reference

This section summarizes the structure of the XML patch structure used to describe partial updates to XML documents and document metatdata. Each section includes a syntax summary, a description of the components, and an example.

The following elements, all in the namespace http://marklogic.com/rest-api, are covered. Start wit the <patch/> wrapper.

• patch
• insert
• replace
• replace-insert
• delete
• replace-library

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert />
  <rapi:replace />
  <rapi:replace-insert />
  <rapi:delete />
  <rapi:replace-library />
</rapi:patch>

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/inventory/history" position="last-child">
    <modified>2012-11-5</modified>
  </rapi:insert>
  <rapi:delete select="saleExpirationDate"/>
  <rapi:replace select="price" apply="ml.multiply">1.1</rapi:replace>
</rapi:patch>

<insert context=xpath-expr position=pos-selector
        cardinality=occurrence>
  content-to-insert
</insert>

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/inventory/history" position="last-child">
    <modified>2012-11-5</modified>
  </rapi:insert>
</rapi:patch>

<rapi:attribute-list a1="v1" a2="v2"/>

<replace select=xpath-expr cardinality=occurrence apply=func-name>
  replacement-content
</replace>

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace select="/parent/child[1]">
    <child>REPLACED</child>
  </rapi:replace>
</rapi:patch>

<rapi:replace select="my-elem" apply=ml.multiply>5</rapi:replace>

<rapi:text>replacement text</rapi:text>

<rapi:attribute-list my-attr="new-value" />
<rapi:text>new-value</rapi:text>

<replace-insert select=xpath-expr context=xpath-expr position=pos
    cardinality=occurrence apply=func-name>
  replacement-content
</replace-insert>

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-insert context="/parent/child" select="grandchild[1]"
      position="last-child">
    <grandchild>CHANGED</grandchild>
  </rapi:replace-insert>
</rapi:patch>

<rapi:text>replacement text</rapi:text>

<rapi:attribute-list my-attr="new-value" />
<rapi:text>new-value</rapi:text>

<delete select=xpath-expr cardinality=occurrence />

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:delete select="/parent/child[1]" />
  <rapi:delete select="/parent/@my-attr" />
</rapi:patch>

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-library
    at="/my.domain/my-module.xqy" ns="my-ns" />
  <rapi:replace select="/parent/child[1]" apply="my-func" />
</rapi:patch>

Managing XML Namespaces in a Patch

The patch descriptor must be in the namespace http://marklogic.com/rest-api. To distinguish the <patch/> elements from elements in your document, you should include an alias for the patch descriptor namespace. For example:

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
    ...
</rapi:patch>

Declare additional namespace aliases for the content described by the patch. Namespaces declared on the <patch/> root element are in scope in @context and @select path expressions and in the content portion of <insert/>, <replace/> and <replace-insert/> elements.

For example, if you patch a document with the following contents:

<parent xmlns="http://my/namespace">
  <child>first</child>
</parent>

Then your <patch/> should either define http://my/namespace as the default namespace or define a namespace alias for it. For example, the following patch inserts a <child/> element when you define a default namespace:

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api"
    xmlns="http://my/namespace">
  <rapi:insert context="/parent" position="last-child">
    <child>last</child>
  </rapi:insert>
</rapi:patch>

The following is the same patch, using an explicit namespace alias of my-ns:

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api"
    xmlns:my-ns="http://my/namespace">
  <rapi:insert context="/my-ns:parent" position="last-child">
    <my-ns:child>last</my-ns:child>
  </rapi:insert>
</rapi:patch>

Failure to use namespace declarations properly in a patch can lead to surprising results. For example, if you did not include a namespace declaration for http://my/namespace, the patch would not work because the context XPath expression would not match any elements in the target document. Similarly, if you include a namespace alias but do not use it in the definition of the new child, the new child element would be inserted in no namespace: <child xmlns="">...</child>.

XML Examples of Partial Updates

This section includes the following examples of applying a partial update to an XML document or metadata:

• Example cURL Commands
• Example: Inserting an Element
• Example: Inserting an Attribute
• Example: Inserting a Text Node
• Example: Inserting a Comment or Processing Instruction
• Example: Multiple Inserts in One Patch
• Example: Replacing an Element or Element Contents
• Example: Replacing an Attribute Value
• Example: Replacing or Inserting an Element in One Operation
• Example: Replacing or Inserting an Attribute in One Operation
• Example: Deleting an Element or Attribute

  Many of the examples include extra whitespace to improve readability. You should not do likewise unless you want extra whitespace in your target document. Whitespace in the body of an <insert/>, <replace/>, or <replace-insert/> operation is usually significant.

# Windows users, see Modifying the Example Commands for Windows 
# Create the example document
$ curl --anyauth --user user:password -X PUT -d @./example.xml \
    -i -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.xml
# Apply the patch
$ curl --anyauth --user user:password -X POST -d @./patch.xml \
    -i -H "Content-type: application/xml" \
    -H "X-HTTP-Method-Override: PATCH" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.xml
# Retrieve the results
curl --anyauth --user user:password -X GET \
    -i -H "Accept: application/xml" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.xml

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/parent/child" position="before">
    <INSERTED/>
  </rapi:insert>
</rapi:patch>

<parent>
  <child>one</child>
  <child>two</child>
  <child>three</child>
</parent>

<parent>
  <INSERTED/>
  <child>one</child>
  <INSERTED/>
  <child>two</child>
  <INSERTED/>
  <child>three</child>
</parent>

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/parent/child[2]">
    <rapi:attribute-list my-attr1="val1" my-attr2="val2" />
  </rapi:insert>
</rapi:patch>

<parent>
  <child>one</child>
  <child>two</child>
  <child>three</child>
</parent>

<parent>
  <child>one</child>
  <child my-attr1="val1" my-attr2="val2">
    two
  </child>
  <child>three</child>
</parent>

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/parent/child[2]"
     position="before">INSERTED</rapi:insert>
</rapi:patch>

<parent>
  <child>one</child>
  <child>two</child>
  <child>three</child>
</parent>

<parent>
  <child>one</child>
  INSERTED
  <child>two</child>
  <child>three</child>
</parent>

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/parent/*[1]" position="before">
    <!-- INSERTED -->
  </rapi:insert>
</rapi:patch>

<parent>
  <child>one</child>
  <child>two</child>
  <child>three</child>
</parent>

<parent>
  <!-- INSERTED -->
  <child>one</child>
  <child>two</child>
  <child>three</child>
</parent>

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:insert context="/parent/child" position="before">
    <INSERTED/>
  </rapi:insert>
  <rapi:insert context="/parent/child[2]"
      position="before">INSERTED</rapi:insert>
  <rapi:insert context="/parent/child[2]">
    <rapi:attribute-list my-attr="val" />
  </rapi:insert>
  <rapi:insert context="/parent/*[1]" position="before">
    <!-- INSERTED -->
  </rapi:insert>
</rapi:patch>

<parent>
  <child>one</child>
  <child>two</child>
  <child>three</child>
</parent>

<parent>
  <INSERTED/>
  <!-- INSERTED -->
  <child>one</child>
  <INSERTED/>
  INSERTED
  <child my-attr="val">two</child>
  <INSERTED/>
  <child>three</child>
</parent>

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <!-- replace an entire element -->
  <rapi:replace select="/parent/child[1]">
    <REPLACED />
  </rapi:replace>

  <!-- replace the element contents -->
  <rapi:replace select="/parent/child[2]">
    REPLACED
  </rapi:replace>

  <!-- replace an element with a text node -->
  <rapi:replace select="/parent/child[3]">
    <rapi:text>REPLACED</rapi:text>
  </rapi:replace>
</rapi:patch>

<parent>
  <child>one</child>
  <child>two</child>
  <child>three</child>
</parent>

<parent>
  <REPLACED />
  <child>REPLACED</child>
  REPLACED
</parent>

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace select="/parent/@attr1">
    <rapi:text>REPLACE-1</rapi:text>
  </rapi:replace>
  <rapi:replace select="/parent/child/@attr2">
    <rapi:attribute-list attr2="REPLACE-2"/>
  </rapi:replace>
</rapi:patch>

<parent attr1="val1">
  <child attr2="val2">one</child>
  <child attr2="val2">two</child>
</parent>

<parent attr1="REPLACE-1">
  <child attr2="REPLACE-2">one</child>
  <child attr2="REPLACE-2">two</child>
</parent>

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-insert context="/parent/child" select="grandchild[1]"
    position="last-child">
    <grandchild>CHANGED</grandchild>
  </rapi:replace-insert>
</rapi:patch>

<parent>
  <child/>
  <child>
    <grandchild/>
    <grandchild>
      unchanged
    </grandchild>
  </child>
</parent>

<parent>
  <child/>
  <child>
   <grandchild>CHANGED</grandchild>
   <grandchild>
      unchanged
    </grandchild>
  </child>
</parent>

<parent>
  <child/>
  <child/>
</parent>

<parent>
  <child>
   <grandchild>CHANGED</grandchild>
  </child>
  <child>
   <grandchild>CHANGED</grandchild>
  </child>
</parent>

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-insert context="/parent/child" select="grandchild[1]"
      position="last-child">
    <rapi:text>CHANGED</rapi:text>
  </rapi:replace-insert>
</rapi:patch>

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-insert select="@my-attr" context="/parent/child">
    REPLACED
  </rapi:replace-insert>
  <rapi:replace-insert select="@new-attr" context="/parent/child">
    <rapi:attribute-list new-attr="INSERTED" />
  </rapi:replace-insert>
</rapi:patch>

<parent>
  <child />
  <child my-attr="val2" />
  <child />
</parent>

<parent>
  <child new-attr="INSERTED"/>
  <child my-attr="REPLACED"          new-attr="INSERTED"/>
  <child new-attr="INSERTED"/>
</parent>

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:delete select="/parent/child/grandchild"/>
  <rapi:delete select="/parent/@my-attr"/>
</rapi:patch>

<parent my-attr="val">
  <child/>
  <child>
    <grandchild/>
    <grandchild/>
  </child>
</parent>

<parent>
  <child/>
  <child></child>
</parent>

JSON Patch Reference

This section summarizes the structure of the JSON patch structured used to describe partial updates to JSON documents. Each section includes a syntax summary, a description of component JSON properties, and an example.

This topic covers the following properties of a patch:

• pathlang
• patch
• insert
• replace
• replace-insert
• delete
• replace-library

The insert, replace, replace-insert, and delete patch operations all require constructing one or more path expressions that identify the content to be changed. To accurately address JSON with XPath, you need to understand the MarkLogic JSON document model. For details, see Traversing JSON Documents Using XPath in the Application Developer's Guide.

{ "pathlang": "jsonpath",
  "patch": [
    { "insert": {
        "context": "$.parent.child",
        "position": "last-child",
        "content": { "new-key": "new-value" }
  }}
]}

{"patch": [
    insert,
    replace,
    replace-insert,
    delete,
    replace-library 
] }

{"patch": [
  { "insert": {
      "context": "/parent/child",
      "position": "last-child",
      "content": { "new-key": "new-value" }
  }},
  { "replace": {
      "select": "/parent/child[0]",
      "content": "new-value"
    }}
]}

{
  "insert": {
    "context": path-expr,
    "position": pos-selector,
    "content": new-content,
    "cardinality": occurrence
  }
}

{"insert": {
    "context": "/parent/child",
    "position": "after",
    "content": "INSERTED"
}}

{ "replace": {
    "select" : path-expr,
    "content" : new-value,
    "cardinality": occurrence,
    "apply" : func-name
  }
}

{"patch": [
  {"replace": {
      "select": "/parent/child[1]",
      "content": "REPLACED"
  }}
]}

{ "replace-insert": {
    "select" : path-expr,
    "context" : path-expr,
    "position": pos-selector,
    "cardinality": occurrence,
    "content" : replacement-content,
    "apply" : func-name
  }
}

{ "patch": [
    { "replace-insert" : {
        "select": "/parent/child[. = 'target']",
        "context": "/parent/array-node('child')",
        "position": "last-child",
        "content": "new-value"
    }}
]}

{ "delete": { 
    "select" : path-expr,    "cardinality": occurrence
} }

{"patch": [
  { "delete" : { "select": "/parent//child" } }
]}

{ "patch": [
   {"replace-library": {
     "at": "/my.domain/my-module.xqy",
     "ns": "my-ns"
   }},
   {"replace": {
     "select": "/inventory[name eq 'orange']/price",
     "apply": "my-func"
   }}
]}

JSON Examples of Partial Update

This section includes the examples of applying a partial update with a JSON patch. The following topics are covered:

• Example cURL Commands
• Example: Insert
• Example: Replace
• Example: Replace-Insert
• Example: Delete

To understand these examples, it is important that you understand how to use XPath to address JSON document components. For details, see Working With JSON in the Application Developer's Guide.

# Windows users, see Modifying the Example Commands for Windows 

# Create the example document
$ curl --anyauth --user user:password -X PUT -d @./example.json \
    -i -H "Content-type: application/json" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.json
# Apply the patch
$ curl --anyauth --user user:password -X POST -d @./patch.json \
    -i -H "Content-type: application/json" \
    -H "X-HTTP-Method-Override: PATCH" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.json
# Retrieve the results
curl --anyauth --user user:password -X GET \
    -i -H "Accept: application/json" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.json

{ "patch": [
    { "insert": {
          "context": "/parent/child1",
          "position": "before",
          "content": { "INSERT1": "INSERTED1" }
    }},
    { "insert": {
          "context": "/parent/child3[1]",
          "position": "after",
          "content": "INSERTED2"
    }},
    { "insert": {
          "context": "/node()",
          "position": "last-child",
          "content": { "INSERT3": "INSERTED3" }
    }}
  ] }

{ "parent": {
    "child1": { 
      "grandchild": "value" 
    },
    "child2": "simple",
    "child3": [ "av1", "av2" ],
    "child4": [ 
      { "a1" : "v1"}, 
      { "a2": "v2" } 
    ]
} }

{ "parent": {
    "INSERT1": "INSERTED1",
    "child1": { "grandchild": "value" },
    "child2": "simple",
    "child3": ["av1", "INSERTED2", "av2"],
    "child4": [
      { "a1": "v1" },
      { "a2": "v2" }
    ]
  },
  "INSERT3": "INSERTED3"
}

{ "patch": [
  { "insert": {
          "context": "/parent[fn:empty(TARGET)]",
          "position": "last-child",
          "content": { "TARGET": "INSERTED" }
  }}
]}

{ "patch": [
    { "replace": {
        "select": "/parent/child1",
        "content": { "REPLACE1": "REPLACED1" }
    }},
    { "replace": {
        "select": "/parent/child2",
        "content": "REPLACED2"
    }},
    { "replace": {
        "select": "/parent/child3[1]",
        "content": "REPLACED3"
    }},
    { "replace": {
        "select": "/parent/array-node('child4')",
        "content": [ "REPLACED4a", "REPLACED4b" ]
    }},
    { "replace": {
        "select": "/parent/child5",
        "content": "REPLACED5"
    }},
    { "replace": {
        "select": "/parent/array-node('child6')/node()[2]",
        "content": [ "REPLACED6a", "REPLACED6b" ]
    }},
    { "replace": {
        "select": "/parent/child7[2]",
        "content": "REPLACED7"
    }}
]}

{ "parent": {
    "child1": { 
      "grandchild": "value"
    },
    "child2": "simple",
    "child3": [ "av1", "av2" ],
    "child4": [ "av1", "av2" ],
    "child5": [ "av1", "av2" ],
    "child6": [ 
      "av1", 
     ["nav1", "nav2"], 
     "av2"
    ],
    "child7": [
      "av1", 
      ["nav1", "nav2"],
      "av2" ]
} }

{ "parent": {
    "child1": {
      "REPLACE1": "REPLACED1"
    },
    "child2": "REPLACED2",
    "child3": ["REPLACED3", "av2" ],
    "child4": ["REPLACED4a", "REPLACED4b"],
    "child5": ["REPLACED5", "REPLACED5"],
    "child6": [
      "av1",
      [ "REPLACED6a", "REPLACED6b" ],
      "av2"
    ],
    "child7": [
      "av1",
      [ "REPLACED7", "nav2" ],
      "av2"
    ]
} }

{"replace": {
  "select": "/parent/child1",
  "content": { 
    "REPLACE1": "REPLACED1" }
}}

{"replace": {
  "select": "/parent/child2",
  "content": "REPLACED2"
}}

{"replace": {
  "select": "/parent/child3[1]",
  "content": "REPLACED3"
}}

{"replace": {
  "select": "/parent/array-node('child4')",
  "content": [ 
    "REPLACED4a", "REPLACED4b"
  ]
}}

{"replace": {
  "select": "/parent/child5",
  "content": "REPLACED5"
}}

{"replace": {
  "select":
    "/parent/array-node('child6')/node()[2]",
  "content": [ 
    "REPLACED6a", "REPLACED6b"
  ]
}}

{"replace": {
  "select": "/parent/child7[2]",
  "content": "REPLACED7"
}}

{ "patch": [
    { "replace-insert": {
        "select": "/parent/child1[1]",
        "context": "/parent/array-node('child1')",
        "position": "last-child",
        "content": "REPLACED1"
    }},
    { "replace-insert": {
        "select": "/parent/child1[3]",
        "context": "/parent/child1[2]",
        "position": "after",
        "content": "INSERTED1"
    }},
    { "replace-insert": {
        "select": "/parent/child2[. = 'c2_v1']",
        "context": "/parent/node('child2')",
        "position": "last-child",
        "content": "REPLACED2"
    }},
    { "replace-insert": {
        "select": "/parent/child2[. = 'INSERTED2']",
        "context": "/parent/node('child2')",
        "position": "last-child",
        "content": "INSERTED2"
    }},
    { "replace-insert": {
        "select": "/parent/child3[c3_a]",
        "context": "/parent/node('child3')",
        "position": "last-child",
        "content": { "REPLACED3" : "REPLACED_V" }
    }},
    { "replace-insert": {
        "select": "/parent/child3[INSERTED3]",
        "context": "/parent/node('child3')",
        "position": "last-child",
        "content": { "INSERTED3" : "INSERTED_V" }
    }}
]}

{"parent": {
  "child1": [ "c1_v1", "c1_v2" ],
  "child2": [ "c2_v1", "c2_v2" ],
  "child3": [
    { "c3_a": "c3_v1" },
    { "c3_b": "c3_v2" }
  ]
} }

{ "parent": {
    "child1": [
      "REPLACED1",
      "c1_v2",
      "INSERTED1"
    ],
    "child2": [
      "REPLACED2",
      "c2_v2",
      "INSERTED2"
    ],
    "child3": [
      { "REPLACED3": "REPLACED_V" },
      { "c3_b": "c3_v2" },
      { "INSERTED3": "INSERTED_V" }
    ]
} }

{ "replace-insert": {
    "select": "/parent/child1[1]",
    "context": "/parent/array-node('child1')",
    "position": "last-child",
    "content": "REPLACED1"
}}

{"parent"{"child1": ["c1_v1", "c1_v2"], ... }
    ==> {"parent"{"child1": ["REPLACED1", "c1_v2"], ... }
{"parent"{"child1": [], ... }
    ==> {"parent"{"child1": ["REPLACED1"], ... }

"child3": [ 
  { "c3_a": "c3_v1" },
  { "c3_b": "c3_v2" } 
]

{ "patch": [
  { "insert": {
          "context": "/parent[fn:empty(TARGET)]",
          "position": "last-child",
          "content": { "TARGET": "INSERTED" }
  }},
  { "replace": {
        "select": "/parent/TARGET",
        "content": "REPLACED"
  }}
]}

{"parent": {
  "child": "some_value"
}}

{"parent":{
  "child": "some_value",
  "TARGET": "INSERTED"
}}

{"parent": {
  "child": "some_value",
  "TARGET": "INSERTED"
}}

{"parent": {
  "child": "some_value",
  "TARGET": "REPLACED"
}}

{{ "patch": [
    { "delete" : {
        "select": "/props/node('anyType')"
    }},
    { "delete" : {
        "select": "/props/objOrLiteral"
    }},
    { "delete" : {
        "select": "/props/array-node('arrayVal')"
    }},
    { "delete" : {
        "select": "/arrayItems/all"
    }},
    { "delete" : {
        "select": "/arrayItems/byPos[1]"
    }},
    { "delete" : {
        "select": "/arrayItems/byVal[. = 'DELETE']"
    }},
    { "delete" : {
        "select": "/arrayItems/byName[DELETE]"
    }}
] }

{ "props": {
    "anyType": [1, 2],
    "objOrLiteral": "anything",
    "arrayVal": [3, 4]
  },
  "arrayItems": {
    "byPos": ["DELETE", "PRESERVE"],
    "byVal": ["DELETE", "PRESERVE"],
    "byName": [ 
      {"DELETE": 5}, 
      {"PRESERVE": 6} 
    ],
    "all": ["DELETE1", "DELETE2"]
  }
}

{ "props": { },
  "arrayItems": {
    "byPos": [ "PRESERVE" ],
    "byVal": [ "PRESERVE" ],
    "byPropName": [
      { "PRESERVE": 6 }
    ],
    "all": [ ]
  }
}

/props/node('anyType')
/props/objOrLiteral'
/props/array-node('arrayVal')

pb.remove('/props/arrayVal')
==> arrayVal: [ ]

Patching Metadata

You can partially update metadata for any document type, including XML, JSON, text, and binary. To update a portion of the metadata for a document, send a POST request to the /documents service with a URI of the following form.

http://host:port/version/documents?uri=document-uri&category=category

The category parameter should either be metadata or one of the metadata sub-categories of collections, properties, permissions, metadata-values, and quality. The category affects what pieces of metadata MarkLogic Server considers for updating. For example, if you use category=permissions, but your patch only contains an operation applied to properties, the patch has no effect.

Category does not affect how you construct path expressions in a patch. That is, context and select paths within the patch are not relative to the specified metadata category.

The request body must contain an XML or JSON patch descriptor. Construct your metadata patch to apply to the structure described in Working with Metadata, using the same syntax and semantics as for a content patch. For details, see in XML Patch Reference or JSON Patch Reference.

Document quality (the <quality/> XML element or quality JSON property) can only be the target of a replace operation.

You can update content and metadata in the same request by including operations on both in your patch descriptor and setting the category request parameter appropriately. For example:

http://localhost:8000/LATEST/documents?uri=/my.xml&category=content&category=metadata

When patching content and metadata in the same request, the usual content type restrictions apply: You can only apply content patches to XML and JSON documents, and you cannot use a JSON patch descriptor on XML content, or vice versa.

Since JSON does not use qualified names, you can construct a context or select path that addresses both metadata and content items. If your content contains JSON properties with the same name as the metadata properties (properties, collections, permissions, metadataValues, and quality) and you cannot construct an unambiguous path, do not combine content and metadata partial updates in the same request.

Paths for context and select can be relative or absolute with respect to the metadata structure. For example, both of the following operations insert a document into a new collection, using different context paths.

<rapi:insert
   context="/rapi:metadata/rapi:collections"
   position="last-child">
  <rapi:collection>INSERTED</rapi:collection>
</rapi:insert>

{"insert": {
   "context":"$.collections",
   "position":"last-child",
   "content":"INSERTED"
}}

<rapi:insert 
    context="rapi:collections"
    position="last-child">
  <rapi:collection>INSERTED</rapi:collection>
</rapi:insert>

{"insert": {
   "context":"collections",
   "position":"last-child",
   "content":"INSERTED"
}}

The following example is a patch that performs multiple metadata updates: Add the target document to a new collection, add or replace a property, and replace the document quality:

<rapi:patch xmlns:rapi="http://marklogic.com/rest-api"
     xmlns:prop="http://marklogic.com/xdmp/property">
  <rapi:insert context="/rapi:metadata/rapi:collections"
      position="last-child">
    <rapi:collection>INSERTED</rapi:collection>
  </rapi:insert>
  <rapi:replace-insert select="my-property"
      context="/rapi:metadata/prop:properties"
      position="last-child">
    <my-property>REPLACE_OR_INSERT</my-property>
  </rapi:replace-insert>
  <rapi:replace select="/rapi:metadata/rapi:quality">2</rapi:replace>
</rapi:patch>

{ "patch" : [
  { "insert": {
      "context": "$.collections",
      "position": "last-child",
      "content" : "INSERTED"
  }},
  { "replace-insert": {
      "select": "my-property",
      "context": "$.properties",
      "position": "last-child",
      "content" : { "my-property": "REPLACE_OR_INSERT" }
  }},
  { "replace": {
      "select": "$.quality",
      "content" : 2
  }}
  ], 
  "pathlang": "jsonpath" 
}

When patching properties using XML, you must explicitly declare the namespace http://marklogic.com/xdmp/property on the root <patch/> element because the <properties/> metadata element is in a different namespace from the other metadata elements.

Values metadata (category=metadata-values) is represented differently in XML and JSON. The equivalent of the XML metadata-values element is a JSON property named metadataValues, and where XML has metadata-value child elements to represent each key-value pair, JSON has key:value child properties. For details, see Working with Metadata.

Constructing Replacement Data on the Server

You can use builtin or user-defined replacement functions to generate the content for a partial update operation dynamically on MarkLogic Server. The builtin functions support simple arithmetic and string manipulation. For example, you can use a builtin function to increment the current value of numeric data or concatenate strings. For more complex operations, create and install a user-defined function.

The following topics are covered:

• Using a Replacement Constructor Function
• Using Builtin Replacement Constructors
• Writing an XQuery User-Defined Replacement Constructor
• Writing a JavaScript User-Defined Replacement Constructor
• Installing or Updating a User-Defined Replace Library

<rapi:patch
    xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace 
    select="/inventory/price"
    apply="ml.multiply">1.1</rapi:replace>
</rapi:patch>

{"patch": [
  {"replace": {
    "select": "/inventory/price",
    "apply": "ml.multiply",
    "content": 1.1
  } }
]}

<rapi:patch
    xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-library
    at="/my.domain/my-lib.xqy" 
    ns="http://my/ns" />
  <rapi:replace 
    select="/inventory/price"
    apply="dbl"/>
</rapi:patch>

{"patch": [
  {"replace-library": {
    "at": "/my.domain/my-lib.xqy",
    "ns": "http://my/ns"
  } },
  {"replace": {
    "select": "/inventory/price",
    "apply": "dbl"
  } }
] }

<rapi:patch
    xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace select="/my/data"
      apply="ml.concat-between">
    <rapi:value>my prefix</rapi:value>
    <rapi:value>my suffix</rapi:value>
  </rapi:replace>
</rapi:patch>

{"patch": [
  {"replace": {
    "select": "/my/data",
    "apply": "ml.concat-between",
    "content": [
      { "$value" : "my prefix" },
      { "$value" : "my suffix" }
    ]
  } }
] }

<rapi:patch
    xmlns:rapi="http://marklogic.com/rest-api"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <rapi:replace select="/elem" apply="my-func">
    <rapi:value xsi:type="xs:date">2013-03-15</rapi:value>
  </rapi:replace>
</rapi:patch>

{"patch": [
  {"replace": {
    "select": "/my/data",
    "apply": "ml.concat-between",
    "content": [
      { "$value" : "2013-03-15", $datatype: "date"}
    ]
  } }
] }

declare function module-ns:func-name(
  $current as node()?,
  $args as item()*
) as node()*

xquery version "1.0-ml";

module namespace my-lib = "http://marklogic.com/example/my-lib";

(: Double the value of a node :)
declare function my-lib:dbl(
  $current as node()?,
  $args as item()*
) as node()*
{
  if ($current/data() castable as xs:decimal)
  then
    let $new-value := xs:decimal($current) * 2
    return
      typeswitch($current)
      case number-node()       (: JSON :)
        return number-node {$new-value}
      case element()           (: XML :)
        return element {fn:node-name($current)} {$new-value}
      default return fn:error((), "RESTAPI-SRVEXERR",
         ("400", "Bad Request", 
          fn:concat("Not an element or number node: ",
                    xdmp:path($current))
          ))
  else fn:error((), "RESTAPI-SRVEXERR",
    ("400", "Bad Request", fn:concat("Non-decimal data: ", $current)))
};

(: Find the minimum value in a sequence of value composed of :)
(: the current node and a set of input values. :)
declare function my-lib:min(
  $current as node()?,
  $args as item()*
) as node()*
{
  if ($current/data() castable as xs:decimal)
  then
    let $new-value := fn:min(($current, $args))
    return
      typeswitch($current)
      case element()           (: XML :)
        return element {fn:node-name($current)} {$new-value}
      case number-node()       (: JSON :)
        return number-node {$new-value}
      default return fn:error((), "RESTAPI-SRVEXERR",
        ("400", "Bad Request", 
           fn:concat("Not an element or number node: ",
                     xdmp:path($current))))
  else fn:error((), "RESTAPI-SRVEXERR", ("400", "Bad Request",
         fn:concat("Non-decimal data: ", $current)))
};

Patch

<rapi:patch
    xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-library at="/ext/replace/my-lib.xqy" 
      ns="http://marklogic.com/example/my-lib" />
  <rapi:replace select="/parent/child" apply="dbl"/>
</rapi:patch>

{"patch": [
  {"replace-library": {
    "at": "/ext/replace/my-lib.xqy",
    "ns": "http://marklogic.com/example/my-lib"
  } },
  {"replace": {
    "select": "/parent/child",
    "apply": "dbl"
  } }
] }

function funcname(current, args)

'use strict';

function dbl(current,args) {
  switch(xdmp.nodeKind(current)) {
    case "number": {
      return new NodeBuilder()
          .addNumber(fn.data(current).valueOf() * 2)
          .toNode();
    }
    case "element": {
      const currentValue = fn.data(current).valueOf();
      if (xdmp.castableAs(
           "http://www.w3.org/2001/XMLSchema", 
           "decimal", currentValue)) {
        return new NodeBuilder()
          .addElement(fn.nodeName(current).toString(),
                      xs.string(currentValue * 2))
          .toNode();
      } else {
        fn.error(null, 'RESTAPI-SRVEXERR', Sequence.from([
          '400', 'Bad Request', 
          'Non-decimal data: ' + xdmp.path(current)]));
      }
    }
    default:
      fn.error(null, 'RESTAPI-SRVEXERR', Sequence.from([
        '400', 'Bad Request', 
        'Not an element or number node: ' + xdmp.path(current)]));
  }
};

function min(current, args) {
  switch(xdmp.nodeKind(current)) {
    case "number": {
      return new NodeBuilder()
        .addNumber(Math.min(
          ...args.toArray().concat(fn.data(current).valueOf())))
        .toNode();
    }
    case "element": {
      const currentValue = fn.data(current).valueOf();
      if (xdmp.castableAs(
          "http://www.w3.org/2001/XMLSchema", 
          "decimal", currentValue)) {
        return new NodeBuilder()
          .addElement(fn.nodeName(current).toString(),
             xs.string(Math.min(...args.toArray().concat(currentValue))))
          .toNode();
      } else {
        fn.error(null, 'RESTAPI-SRVEXERR', Sequence.from([
          '400', 'Bad Request', 
          'Non-decimal data: ' + xdmp.path(current)]));
      }
    }
    default:
      fn.error(null, 'RESTAPI-SRVEXERR', Sequence.from([
        '400', 'Bad Request', 
        'Not an element or number node: ' + xdmp.path(current)]));
  }
};


exports.dbl = dbl;
exports.min = min;

Patch

<rapi:patch
    xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-library at="/ext/replace/my-lib.sjs" />
  <rapi:replace select="/parent/child" apply="dbl"/>
</rapi:patch>

{"patch": [
  {"replace-library": {
    "at": "/ext/replace/my-lib.sjs"
  } },
  {"replace": {
    "select": "/parent/child",
    "apply": "dbl"
  } }
] }

Patch

<rapi:patch
    xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:replace-library at="/ext/replace/my-lib.sjs" />
  <rapi:replace select="/parent/child" apply="min"/>
    <rapi:value>20</rapi:value>
    <rapi:value>2</rapi:value>
  </rapi:replace>
</rapi:patch>

{"patch": [
  {"replace-library": {
    "at": "/ext/replace/my-lib.sjs"
  } },
  {"replace": {
    "select": "/parent/child",
    "apply": "min",
    "content" : [{"$value": 20}, {"$value": 2}]
  } }
] }

http://host:port/version/ext/your/path/your-lib.xqy?perm:perm

http://host:port/version/ext/your/path/your-lib.sjs?perm:perm

$ curl --anyauth --user user:password -X PUT -d @./my-lib.xqy \
    -i -H "Content-type: application/xquery" \
  http://localhost:8000/LATEST/ext/replace/my-lib.xqy?perm:my-user:execute

<rapi:replace-library at="/ext/replace/my-lib.xqy"
   ns="http://marklogic.com/example/my-lib" />

How Position Affects the Insertion Point

The insert and replace-insert patch operations include a position member that defines the point of insertion when coupled with a context expression. This section describes the details of how position affects the insertion point. The following topics are covered:

• Specifying Position in XML
• Specifying Position in JSON

<parent>
  ***
  <child>
    <grandchild/>
  </child>
  <child a1="val" a2="val"/>
</parent>

<parent>
  <child>
    <grandchild/>
  </child>
  ***
  <child a1="val" a2="val"/>
  ***
</parent>

<parent>
  <child>
    <grandchild/>
    ***
  </child>
  <child a1="val" a2="val"/>
</parent>

<parent>
  <child>
    <grandchild/>
  </child>
  <child *** a1="val" a2="val"/>
</parent>

<parent>
  <child>
    <grandchild/>
  </child>
  <child a1="val" *** a2="val"/>
</parent>

<parent>
  <child ***>
    <grandchild/>
  </child>
  <child a1="val" a2="val" *** />
</parent>

{"theTop" : {
    "child1" : [ "val1", "val2" ],
    ***
    "child2" : [ 
      {"one": "val1"}, 
      {"two": "val2"} 
    ]
} }

{"theTop" : {
    "child1" : [ *** "val1", "val2" ],
    "child2" : [ 
      {"one": "val1"}, 
      {"two": "val2"} 
    ]
} }

{"theTop" : {
    "child1" : [ "val1", "val2" ***],
    "child2" : [ 
      {"one": "val1"}, 
      {"two": "val2"} 
    ]
} }

{"theTop" : {
    "child1" : [ "val1", "val2"],
    "child2" : [ 
      {"one": "val1"}, 
      {"two": "val2"} 
    ]
    ***
} }

Path Expressions Usable in Patch Operations

A patch operation uses one or more path expressions to identify the target of the operation.

For performance and security reasons, the path expression used in the context and select specifications of a patch operation is limited to a subset of XPath. JSONPath expressions are constrained by equivalent limitations.

For details, see Patch Feature of the Client APIs in the XQuery and XSLT Reference Guide.

Limitations of JSON Path Expressions

You can replace just the value of a property, but you cannot replace an entire property with a new one in a single operation.

For example, given content of the following form, you can use a replace operation to change the value property a from 1 to 2:

{ "a": 1 }

However, you cannot use a replace operation to change the content to the following:

{ "b": "anyvalue" }

In this example, the path expression /a addresses a number node with name a and value 1. The select expression in the replace operation enables you to replace the value of the selected name, but not the name.

To achieve the desired effect, you must first delete the existing property, and then insert a new one.

This limitation also means you cannot use a replace-insert operation to perform an operation such as insert this property if it doesn't exist; otherwise, replace it. However, you can use separate insert and replace operations in a single patch to perform an operation such as insert this property if it doesn't exist; otherwise, replace its value. For an example, see JSON Examples of Partial Update.

Introduction to JSONPath

When applying partial updates to JSON documents, you can specify the target of a patch operation using JSONPath expressions in the context and select properties of a patch operation. This section gives a brief overview of JSONPath syntax. For a complete description of JSONPath, see http://goessner.net/articles/JsonPath/.

The default path expression language is XPath. Using JSONPath is deprecated. To use JSONPath, you must set the pathlang property to jsonpath. For details, see pathlang.

• The anonymous root of a JSON object is identified by a dollar sign ($).
• Delimit path steps by either periods (.) or square brackets ([]).
• Use square brackets to enclose predicate expressions and array element selectors.
• The first element in an array has position 0, which is different from XPath.
• Predicates must be enclosed in ?(), as in [?(@.value < 10)].

The set of JSONPath expressions usable for document operations like partial update is limited to JSONPath equivalent to XPath expressions that can be used to define a path range index. For details, see Path Expressions Usable in Patch Operations.

The following table contains some simple JSONPath examples that satisfy the restrictions. The bold text indicates what is selected by each expression.

$.parent

$['parent']

{ "parent": [
    {"child": [ "one", "two" ] },
    {"child": {"grandchild": "three" }}
] }

$.parent[0]

$['parent'][0]

{ "parent": [
    {"child": [ "one", "two" ] },
    {"child": {"grandchild": "three" }}
] }

$.parent['child']

{ "parent": [
    {"child": [ "one", "two" ] },
    {"child": {"grandchild": "three" }}
] }

$.parent.child[*]['one']

{ "parent": [
    {"child": [ "one", "two" ] },
    {"child": {"grandchild": "three" }}
] }

Removing a Document or Metadata

To remove a document and its metadata from the database, construct a DELETE request with a URL of the form:

http://host:port/version/documents?uri=document_uri

When you delete a document, its metadata is also deleted.

You can remove multiple documents (or reset their metadata) by specifying multiple URIs. Optimistic locking is not supported when you specify multiple URIs.

To remove or reset just metadata for a document, construct a DELETE request with a URL of the form:

http://host:port/version/documents?uri=document_uri&category=metadata_category

The category parameter can appear multiple times, with the values described in Metadata Categories. Resetting permissions resets the document permissions to the default permissions for the current user. Resetting quality resets the document quality to the default (0).

Deleting a binary document with extracted metadata stored in a separate XHTML document also deletes the XHTML metadata document. For more information, see Working with Binary Documents.

Removing Multiple Documents

In addition to removing multiple documents by URI as described in Removing a Document or Metadata, you can remove all documents in a collection or in a database directory.

To remove all documents in a collection, send a DELETE request with a URL of the following form:

http://host:port/version/search?collection=collection_name

Similarly, to remove all documents in a directory, send a DELETE request with a URL of the following form:

http://host:port/version/search?directory=directory_name

Where directory_name is the name of a directory in the database. The directory name must include a trailing /.

You can specify only one collection or one directory in a single request.

Failing to specify either a directory or a collection removes all documents in the database.

Removing a document also removes its metadata. Deleting a binary document with extracted metadata stored in a separate XHTML document also deletes the XHTML metadata document. For more information, see Working with Binary Documents.

The following example remove all documents in the /plays directory:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -i -X DELETE \
    http://localhost:8000/LATEST/search?directory=/plays/
...
HTTP/1.1 204 Updated
Server: MarkLogic
Content-Length: 0
Connection: close

Removing All Documents

To remove all documents in the database, send a DELETE request with a URL of the following form:

http://host:port/version/search

Clearing the database requires the rest-admin role or equivalent.

There is no confirmation or other safety net when you clear the database in this way. Creating a backup is advised.

The following example removes all documents and metadata from the content database:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -i -X DELETE \
    http://localhost:8000/LATEST/search

Removing a Semantic Graph

To remove the triples in a graph, send a DELETE to the /graphs service with a URL of one of the following forms:

http://host:port/version/graphs?graph=graph-urihttp://host:port/version/graphs?default

The /graphs service only affects triples stored in a triple document (a document with a <sem:triples> root element). Therefore, if the graph includes triples embedded in normal XML documents, the embedded triples are unaffected and the graph will continue to exist.

You can also delete a graph by using an empty graph in a PUT request to the /graphs service.

For details, see Semantics Developer's Guide.

Understanding Optimistic Locking

Consider an application that reads a document, makes modifications, and then updates the document in the database with the changes. The traditional approach to ensuring document integrity is to perform the read, modification, and update in a multi-statement transaction. This holds a lock on the document from the point when the document is read until the update is committed. However, this pessimistic locking blocks access to the document and incurs more overhead on the App Server.

With optimistic locking, the application does not hold a lock on a document between read and update. Instead, the application saves the document state on read, and then checks for changes at the time of update. The update fails if the document has changed between read and update. This is a conditional update.

Optimistic locking is useful in environments where integrity is important, but contention is rare enough that it is useful to minimize server load by avoiding unnecessary multi-statement transactions. The REST Client API also supports multi-statement transactions. For details, see Managing Transactions.

The REST Client API uses content versioning to implement optimistic locking. When content versioning is enabled, MarkLogic Server associates an opaque version id with a document. The version id changes each time you update the document. The version id is returned when you read a document, and you can pass it back in an update or delete operation to test for changes prior to commit.

Content versioning in the REST Client API does not implement document versioning. When content versioning is enabled, MarkLogic Server does not keep multiple versions of a document or track what changes occur. The version id can only be used to detect that a change occurred.

Using optimistic locking in your application requires the following steps:

1. Enabling Optimistic Locking by setting the update-policy REST instance configuration property.
2. Obtaining a Version Id for documents you wish to conditionally update.
3. Applying a Conditional Update by supplying the version id in your update operations.

Enabling Optimistic Locking

Enable optimistic locking using the update-policy instance configuration property, as described in Configuring Instance Properties. For example, the following command sets update-policy to version-optional:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    -d'{"update-policy":"version-optional"}' \
    -H "Content-type: application/json" \
    http://localhost:8000/LATEST/config/properties

The update-policy property can be set to merge-metadata (the default), version-required, or version-optional. Set the property to version-required if you want every document update or delete operation to use optimistic locking. Set the property to version-optional to allow selective use of optimistic locking.

The update-policy property replaces the older content-versions policy; content-versions is deprecated. Setting update-policy to version-optional is equivalent to setting content-versions to optional. Setting update-policy to version-required is equivalent to setting content-versions to required.

The table below describes how each setting for this property affects document operations.

Obtaining a Version Id

When optimistic locking is enabled, a version id is included in the response when you read documents. You can only obtain a version id if optimistic locking is enabled by setting update-policy; for details, see Enabling Optimistic Locking.

You can obtain a version id for use in conditional updates in the following ways:

• Perform a single document GET or HEAD request to /documents. The version id is returned in the ETag header of the response.
• Perform a multi-document read request, such as GET /documents with multiple URIs. The version id is returned in the Content-Disposition header of each content part in the multipart response.

You can mix and match these methods for obtaining a version id.

The following example command returns the version id for a single document:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -i -X HEAD \
    -H "Accept: application/xml" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.xml
...
HTTP/1.1 200 Document Retrieved
Content-type: application/xml
ETag: "13473172834878540"
Server: MarkLogic
Connection: close

The following example command returns the version id for multiple documents in a single request:

curl --anyauth --user user:password -X GET -i \
  -H "Accept: multipart/mixed; boundary=BOUNDARY" \
  http://localhost:8000/LATEST/documents?uri=doc1.xml\&uri=doc2.json
...
--BOUNDARY
Content-Type: application/xml
Content-Disposition: attachment; filename="doc1.xml"; category=content; format=xml; versionId=14075140367230760
Content-Length: 87

...document contents...
--BOUNDARY
Content-Type: application/json
Content-Disposition: attachment; filename="doc2.json"; category=content; format=json; versionId=14075140367230760
Content-Length: 15

...document contents...

For details on multi-document requests, see Reading and Writing Multiple Documents.

Applying a Conditional Update

To apply a conditional update, supply a version id on your update operation. The version id is ignored if optimistic locking is not enabled; for details, see Enabling Optimistic Locking.

When a document update includes a version id, MarkLogic Server checks for changes to the version id before committing the update (or delete) operation. If the version id has changed, the update fails. In a multi-document update, the entire batch of updates is rejected if any conditional update fails.

You can supply a version id on an update in the following ways:

• Pass the version id in the If-Match header of a single document PUT, POST, PATCH, or DELETE request to /documents.
• Pass the version id in the Content-Disposition header of a content part in a multi-document POST request to /documents.

The following example performs a conditional update of a single document by passing a version id in the If-Match request header:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -i -X PUT -d"<modified-data/>"  \
    -H "Content-type: application/xml" \
    -H "If-Match: 13473769780393030" \
    http://localhost:8000/LATEST/documents?uri=/docs/example.xml

The following POST body excerpt for a multi-document update unconditionally updates the first document (doc1.xml) and conditionally updates the second document (doc2.xml). Note that if update-policy is set to version-required, the request will fail if doc1.xml already exists because no version id is supplied for it.

--BOUNDARY
Content-Type: application/xml
Content-Disposition: attachment; filename="doc1.xml"
Content-Length: 87

...document contents...
--BOUNDARY
Content-Type: application/json
Content-Disposition: attachment; filename="doc2.json"; versionId=14075140367230760
Content-Length: 15

...document contents...
--BOUNDARY--

The following command applies the above update, if the complete POST body is in the file /example/optimistic-locking.

$ curl --anyauth --user user:password -X POST -i \
  --data-binary @/example/optimistic-locking \
  -H "Content-type: multipart/mixed; boundary=BOUNDARY" \
  -H "Accept: application/xml" \
  http://localhost:8000/LATEST/documents

For details on multi-document requests, see Reading and Writing Multiple Documents.

Example: Updating a Document Using Optimistic Locking

The following example demonstrates using optimistic locking to conditionally update a single document. Both a successful and a failed update are shown.

This example uses a PUT request to demonstrate a conditional update. For a POST, PATCH, or DELETE operation, follow a similar procedure, passing the version id in the If-Match header of your update or delete request. For an equivalent multi-document update, pass the version id in the Content-Disposition header of each content part.

1. If content versioning is not already enabled, enable it. The following command sets update-policy to version-optional:

   # Windows users, see Modifying the Example Commands for Windows 
   $ curl --anyauth --user user:password -X PUT \
       -d'{"update-policy":"version-optional"}' \
       -H "Content-type: application/json" \
       http://localhost:8000/LATEST/config/properties

2. Insert an example document into the database to initialize the example:

   $ curl --anyauth --user user:password -i -X PUT -d"<data/>" \
       -H "Content-type: application/xml" \
       http://localhost:8000/LATEST/documents?uri=/docs/example.xml

3. Unconditionally retrieve a local copy of the document. Note the version id is returned in the ETag header:

   $ curl --anyauth --user user:password -i -X GET \
       -H "Accept: application/xml" \
       http://localhost:8000/LATEST/documents?uri=/docs/example.xml
   ...
   HTTP/1.1 200 Document Retrieved
   vnd.marklogic.document-format: xml
   Content-type: application/xml
   ETag: "13473769780393030"
   Server: MarkLogic
   Content-Length: 47
   Connection: close
   
   <?xml version="1.0" encoding="UTF-8"?>
   <data/>

4. Conditionally update the document by sending new contents with the version id from Step 3 in the If-Match header. Since the document has not changed since Step 3, the update succeeds. The document version id is changed by this operation.

   $ curl --anyauth --user user:password -i -X PUT -d"<modified-data/>"  \
       -H "Content-type: application/xml" \
       -H "If-Match: 13473769780393030" \
       http://localhost:8000/LATEST/documents?uri=/docs/example.xml
   ...
   HTTP/1.1 204 Content Updated
   Server: MarkLogic
   Content-Length: 0
   Connection: close

5. To illustrate update failure when the version ids do not match, attempt to update the document again, using the version id from Step 3, which is no longer current. The update fails.

   $ curl --anyauth --user user:password -i -X PUT -d"<data/>"  \
       -H "Content-type: application/xml" \
       -H "If-Match: 13473769780393030" \
       http://localhost:8000/LATEST/documents?uri=/docs/example.xml
   ...
   HTTP/1.1 412 Precondition Failed
   Content-type: application/xml
   Server: MarkLogic
   Content-Length: 370
   Connection: close
   
   <?xml version="1.0"?>
   <rapi:error xmlns:rapi="http://marklogic.com/rest-api">
     <rapi:status-code>412</rapi:status-code>
     <rapi:status>Precondition Failed</rapi:status>
     <rapi:message-code>RESTAPI-CONTENTWRONGVERSION</rapi:message-code>
     <rapi:message>RESTAPI-CONTENTWRONGVERSION: (err:FOER0000) Content version mismatch:  uri: /docs/example.xml version: 13473788748796580</rapi:message>
   </rapi:error>

Enabling Content Versioning

Enable content versioning using the update-policy instance configuration property, as described in Configuring Instance Properties.

The default update policy is merge-metadata. If you set update-policy to version-required or version-optional, content versioning is enabled, and a GET or HEAD request to /documents returns a version id in the ETag response header.

Enabling content versioning can affects document insertion, update, and deletion. For details, see Enabling Optimistic Locking.

Content versioning in the REST Client API does not implement document versioning. When content versioning is enabled, MarkLogic Server does not keep multiple versions of a document or track what changes occur. The version id can only be used to detect that a change occurred.

Using Content Versioning for Cache Refreshing

When content versioning is enabled, sending a GET request to /documents with a version id in the If-None-Match HTTP header only retrieves a new copy of the document if it has changed relative to the version id in the header.

The update-policy property replaces the older content-versions property; content-versions is deprecated. Setting update-policy to version-optional is equivalent to setting content-versions to optional. Setting update-policy to version-required is equivalent to setting content-versions to required.

Follow the procedure below to use content version for cache refreshing. For a complete example, see Example: Refreshing a Cached Document.

1. If content versioning is not already enabled, set the update-policy instance configuration property to version-optional or version-required; see Enabling Content Versioning.
2. Read a document by making a GET request to /documents. The response includes the version id in the ETag header.
3. Save the version id returned in the ETag response header.
4. When you want to refresh the cache, send another GET request to /documents with the version id from Step 2 in the If-None-Match HTTP header.

   If the current version id matches the one in the If-None-Match header, no document is retrieved and MarkLogic Server returns status 304. If the current version id differs from the one in the If-None-Match header, the document is returned, along with the new version id in the ETag response header.

Example: Refreshing a Cached Document

The following example demonstrates using content versioning to refresh a client-side document cache. The example includes includes a case where the document is unchanged in the database, as well as the case where the local cache is out of date.

1. If content versioning is not already enabled, enable it by setting update-policy to version-optional:

   # Windows users, see Modifying the Example Commands for Windows 
   $ curl --anyauth --user user:password -X PUT \
       -d'{"update-policy":"version-optional"}' \
       -H "Content-type: application/json" \
       http://localhost:8000/LATEST/config/properties

2. Insert a sample document into the database to initialize the example:

   $ curl --anyauth --user user:password -i -X PUT -d"<data/>" \
       -H "Content-type: application/xml" \
       http://localhost:8000/LATEST/documents?uri=/docs/example.xml

3. Unconditionally retrieve a local copy of the document, as if to cache it. Note the version id is returned in the ETag header:

   $ curl --anyauth --user user:password -i -X GET \
       -H "Accept: application/xml" \
       http://localhost:8000/LATEST/documents?uri=/docs/example.xml
   ...
   HTTP/1.1 200 Document Retrieved
   vnd.marklogic.document-format: xml
   Content-type: application/xml
   ETag: "13473769780393030"
   Server: MarkLogic
   Content-Length: 47
   Connection: close
   
   <?xml version="1.0" encoding="UTF-8"?>
   <data/>

4. Conditionally retrieve the document, as if to refresh the cache. Supply the version id from Step 3 in the If-None-Match header. Since the document has not changed, no content is retrieved.

   $ curl --anyauth --user user:password -i -X GET \
       -H "Accept: application/xml" \
       -H "If-None-Match: 13473769780393030" \
       http://localhost:8000/LATEST/documents?uri=/docs/example.xml
   ...
   HTTP/1.1 304 Content Version Not Modified
   ETag: "13473769780393030"
   Server: MarkLogic
   Content-Length: 0
   Connection: close

5. Update the document in the database, which changes the version id.

   $ curl --anyauth --user user:password -i -X PUT -d"<modified-data/>"  \
       -H "Content-type: application/xml" \
       http://localhost:8000/LATEST/documents?uri=/docs/example.xml
   ...
   HTTP/1.1 204 Content Updated
   Server: MarkLogic
   Content-Length: 0
   Connection: close

6. Conditionally retrieve the document again, as if to refresh the cache. Supply the version id from Step 3 in the If-None-Match header. Since the document has changed, the content is retrieved. The new version id is also returned via the ETag header.

   $ curl --anyauth --user user:password -i -X GET \
       -H "Accept: application/xml" \
       -H "If-None-Match: 13473769780393030" \
       http://localhost:8000/LATEST/documents?uri=/docs/example.xml
   ...
   HTTP/1.1 200 Document Retrieved
   vnd.marklogic.document-format: xml
   Content-type: application/xml
   ETag: "13473770707201670"
   Server: MarkLogic
   Content-Length: 56
   Connection: close
   
   <?xml version="1.0" encoding="UTF-8"?>
   <modified-data/>

Types of Binary Documents

This section provides a brief summary of binary document types. For details, see Working With Binary Documents in the Application Developer's Guide.

MarkLogic Server can store binary documents in three representations:

• Small binary documents are stored entirely in the database.
• Large binary documents are stored on disk with a small reference fragment in the database. The on-disk content is managed by MarkLogic Server.
• External binary documents are stored on disk with a small reference fragment in the database. However, the on-disk content is not managed by MarkLogic Server.

Small and large binary documents are created automatically for you, depending on the document size. External binary documents cannot be created using the REST Client API.

Large binary documents can be streamed out of the database using Range requests. For details, see Streaming Binary Content.

Streaming Binary Content

Streaming binary content out of the database avoids loading the entire document into memory. You can stream binary documents by sending GET requests to /documents that include range requests under following conditions:

• The size of the binary content returned is over the large binary size threshold. For details, see Working With Binary Documents in the Application Developer's Guide.
• The request is for content only. That is, no metadata is requested.
• The MIME type of the content is determinable from the Accept header or the document URI file extension.

The following example requests the first 500K of the binary document with URI /binaries/large.jpg:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -i -o piece.jpg -X GET \
    -H "Accept: application/jpg" -r "0-511999" \
    http://localhost:8000/LATEST/documents?uri=/binaries/large.jpg
...
HTTP/1.1 206 Binary Part Retrieved
Content-type: application/jpeg
Content-Range: bytes 0-511999/533817
Server: MarkLogic
Content-Length: 511999
Connection: close

Metadata Categories

Where the REST Client API accepts specification of metadata categories, the following categories are recognized:

• collections
• permissions
• properties
• quality
• metadata-values
• metadata

The metadata category is shorthand for all the other categories. That is, metadata includes collections, permissions, properties, key-value metadata and quality.

Some requests also support a content category as a convenience for requesting or updating both metadata and document content in a single request.

The metadata-values category represents a metadata field. This type of metadata is expressed as simple key-value pairs and you must configure a database field on a key before you can search it. While similar in some ways to document properties, this type of metadata is stored separately from the associated document and can be operated on like any other field. For more details, see Metadata Fields in the Administrator's Guide.

XML Metadata Format

Metadata contains information about document collections, permissions, properties, quality, and key-value metadata. The format is fully described by the schema file:

MARKLOGIC_INSTALL_DIR/Config/restapi.xsd

The following is a summary of the structure of the metadata. All elements are in the namespace http://marklogic.com/rest-api. You can have 0 or more <collection/>, <permission/>, <metdata-values>, <metadata-value/> or property child elements. There can be only one <quality/> element. The element name and contents of each document property element depends on the property.

<metadata xmlns="http://marklogic.com/rest-api">
  <collections>
    <collection>collection-name</collection>
  </collections>
  <permissions>
    <permission>
      <role-name>name</role-name>
      <capability>capability</capability>
    </permission>
  </permissions>
  <properties>
    <property-element/>
  </properties>
  <quality>integer</quality>
  <metadata-values>
    <metadata-value key="keyName">value</metadata-value>
  </metdata-values>
</metadata>

The following example shows a document in two collections, with one permission, two properties, and one key-value metadata item.

<rapi:metadata xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:collections>
    <rapi:collection>shapes</rapi:collection>
    <rapi:collection>squares</rapi:collection>
  </rapi:collections>
  <rapi:permissions>
    <rapi:permission>
      <rapi:role-name>hadoop-user-read</rapi:role-name>
      <rapi:capability>read</rapi:capability>
    </rapi:permission>
  </rapi:permissions>
  <prop:properties xmlns:prop="http://marklogic.com/xdmp/property">
    <myprop>this is my prop</myprop>
    <myotherprop>this is my other prop</myotherprop>
  </prop:properties>
  <rapi:quality>0</rapi:quality>
  <rapi:metadata-values>
    <rapi:metadata-value key="level">high</rapi:metadata-value>
  </rapi:metadata-values>
</rapi:metadata>

JSON Metadata Format

Metadata contains information about document collections, permissions, properties, quality, and key-value metadata. A block of metadata can contain multiple collections, permissions and properties, but only 1 quality. The structure of each child of the properties JSON property depends on the property contents.

{
  "collections" : [ string ],
  "permissions" : [
    { 
      "role-name" : string,
      "capabilities" : [ string ]
    }
  ],
  "properties" : {
    property-name : property-value
  },
  "quality" : integer
  "metadataValues": {
    key: value
  }
}

The following example shows a document in two collections, with one permission, two document properties, and two key-value metadata items.

{
  "collections": [
    "shapes",
    "squares"
  ],
  "permissions": [
    {
      "role-name": "hadoop-user-read",
      "capabilities": [
        "read"
      ]
    }
  ],
  "properties": {
    "myprop": "this is my prop",
    "myotherprop": "this is my other prop"
  },
  "quality": 0,
  "metadataValues": {
    "level": "high",
    "rating": 5
  }
}

Working With Document Properties Using JSON

In most cases, your application should work with properties in a consistent format. That is, if you insert or update properties as XML, then you should retrieve them in XML. If you insert or update properties as JSON, you should retrieve them in JSON.

You should only insert document property data as JSON when you can specify the content type, such as through the request or part header of POST /v1/documents. Do not use the prop request parameter of PUT /v1/documents to insert document properties with JSON values as such input is untyped.

Internally, document properties are always stored as XML. When you insert a user-defined property using JSON, the XML representation is an XML element in the namespace http://marklogic.com/xdmp/json/basic with a local name that matches the property name in JSON. When you retrieve the property data as JSON, it is converted from XML back to JSON.

As long as you handle the data consistently, the conversion to and from XML is largely transparent to your application. However, you need to be aware of the XML representation when searching properties. Also, the implementation of transforms and extensions only sees the XML representation.

If you configure an index based on a user-defined property inserted using JSON, treat them as XML elements and use the http://marklogic.com/xdmp/json/basic in your configuration.

Protected system properties such as last-updated cannot be modified by your application. When retrieved as JSON, such protected properties are wrapped in an object with the JSON property name $ml.prop. For example:

{ "properties": {
    "$ml.prop": {
      "last-updated": "2013-11-06T10:01:11-08:00"
    }
} }

The following example code adds a document property expressed as JSON to /my/doc.json:

$ curl --anyauth --user user:password -X PUT -i \
  -H "Content-type: application/json" \
  -d '{"properties": {"pname":"pvalue"}}' \
  'http://localhost:8000/LATEST/documents?uri=/doc/my.json&category=properties'

If you retrieve the properties of /doc/my.json as JSON, you get back what was inserted. The internal representation as XML is hidden from your application. For example:

$ curl --anyauth --user user:password -X GET \
  -H "Accept: application/json" \
  'http://localhost:8000/LATEST/documents?uri=/doc/my.json&category=properties'
{"properties":{"pname":"pvalue"}}

However, if you retrieve the properties as XML or examine them using the Explore feature of Query Console, you will see the XML representation. For example:

$ curl --anyauth --user user:password -X GET \
  -H "Accept: application/xml" \
  'http://localhost:8000/LATEST/documents?uri=/doc/my.json&category=properties'
<rapi:metadata uri="/doc/my.json" ...>
  <prop:properties xmlns:prop="http://marklogic.com/xdmp/property">
    <pname type="string" xmlns="http://marklogic.com/xdmp/json/basic">
      pvalue
    </pname>
  </prop:properties>
</rapi:metadata>

Disabling Metadata Merging

If you use the REST Client API to ingest a large number of documents at one time and you find the performance unacceptable, you might see a small performance improvement by disabling metadata merging. This topic explains the tradeoff and how to disable metadata merging.

• When to Consider Disabling Metadata Merging
• Understanding Metadata Merging
• How to Disable Metadata Merging

$ cat props.xml
<properties xmlns="http://marklogic.com/rest-api">
  <update-policy>overwrite-metadata</update-policy>
</properties>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT -d@"./props.xml" \
    -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/config/properties

{
  "update-policy" : "overwrite-metadata"
}