Loading TOC...
REST Application Developer's Guide (PDF)

REST Application Developer's Guide — Chapter 1

Introduction to the MarkLogic REST API

The MarkLogic REST API provides a set of RESTful services for creating, updating, retrieving, deleting and query documents and metadata. This section provides a brief overview of the features of the API.

Getting Started with the MarkLogic REST API

This section leads you through a simple example that uses the MarkLogic REST API to insert documents into a database and perform a search. We will follow these steps:

  1. Create a REST API Instance
  2. Load Documents Into the Database
  3. Search the Database
  4. Tear Down the REST API Instance

Preparation

Before beginning this walkthrough, you should have the following software installed:

  • MarkLogic Server, version 6.0-1 or later
  • curl, a command line tool for issuing HTTP requests, or an equivalent tool.

Though the examples rely on curl, you may use any tool capable of sending HTTP requests. If you are not familiar with curl or do not have curl on your system, see Introduction to the curl Tool.

To create the input documents used by the walkthrough:

  1. Create a text file named one.xml with the following contents:
    <one>
      <child>The noble Brutus has told Caesar was ambitious</child>
    </one>
  2. Create a text file named two.json with the following contents:
    {
      "two": {
        "child": "I come to bury Caesar, not to praise him."
      }
    }

Create a REST API Instance

A MarkLogic REST API instance is an HTTP App Server configured with special code installed to service HTTP requests against the API. The /rest-apis service on port 8002 allows you to set up, query, and tear down REST API instances. The /rest-apis service can optionally create the content database when setting up an instance.

Each REST API instance can host a single application. You cannot share the modules database across multiple REST API instances.

This section covers creating an instance using HTTP. To use Information Studio instead, see Creating a REST API Instance in the Information Studio Developer's Guide.

Follow this procedure to create a 'rest-example' database with one forest and a 'rest-example' HTTP App Server attached to the database.

  1. Create the instance and database by sending a POST request to /rest-apis on port 8002:
    $ curl --anyauth --user user:password -X POST \
        -d'{"rest-api":{"name":"rest-example"}}' \
        -H "Content-type: application/json" \
        http://localhost:8002/v1/rest-apis

By specifying only the instance name, we request MarkLogic Server to create a database with the same name, including one forest, and to use the next available port above 8002.

  1. Navigate to the Admin Interface in your browser:
    http://localhost:8001

Observe the creation of a database named 'rest-example' with a forest named 'rest-example-1', and an HTTP App Server called 'rest-example' attached to this database. A modules database is also created for the instance.

  1. Test the instance by navigating to the following URL in your browser. (Your port number may differ if port 8003 was already in use). When prompted for a username and password, authenticate with a user that has the rest-reader role.
    http://localhost:8003

If the instance is properly configured, you will see a short HTML summary of capabilities provided by the instance.

If you prefer to configure your instance using XML, use the following for the POST body and change the HTTP Content-type header to application/xml in Step 1, above.

<rest-api xmlns="http://marklogic.com/rest-api">
  <name>app-server-name</name>
</rest-api>

Load Documents Into the Database

This procedure loads sample content into the 'rest-example' database using the /documents service. The /documents service allows you to create, read, update and delete documents in the database.

To load the sample documents into the rest-example database:

  1. Navigate to the directory containing the sample documents you created in Preparation.
  2. Execute the following command to load one.xml into the database with the URI /xml/one.xml:
    $ curl --anyauth --user user:password -X PUT -d@'./one.xml' \
        'http://localhost:8003/v1/documents?uri=/xml/one.xml&format=xml'

    The URL tells the /documents service to create an XML document (format=xml) with database URI /xml/one.xml (uri=...) from the contents in the request body. If the request succeeds, the service returns status 201 (Document Created).

  3. Execute the following command to load two.json into the database with the URI /json/two.json:
    $ curl --anyauth --user user:password -X PUT -d@'./two.json' \
      'http://localhost:8003/v1/documents?uri=/json/two.json&format=json'
  4. Optionally, use Query Console to explore the rest-example database. The database should contain 2 documents, /xml/one.xml and /json/two.json.

To learn more about the document manipulation features of the documents service, see Manipulating Documents.

Search the Database

The MarkLogic REST API provides several query services. This procedure uses the search service to perform a simple string query search of the 'rest-examples' database, finding documents containing 'caesar'. For details, see Using and Configuring Query Features.

To search the database:

  1. Execute the following command to send a search request to the instance, requesting matches to the search string 'caesar'. Results are returned as XML by default.
    $ curl --anyauth --user user:password \
        'http:localhost:8003/v1/search?q=caesar'
  2. Examine the XML search results returned in the response body. Notice that there are two matches, one in each document.
    <search:response snippet-format="snippet" total="2" start="1" ...>
      <search:result index="1" uri="/xml/one.xml" ...>
        <search:snippet>
          <search:match path="fn:doc(&quot;/xml/one.xml&quot;)/one/child">The noble Brutus has told <search:highlight>Caesar</search:highlight> was ambitious</search:match>
        </search:snippet>
      </search:result>
      <search:result index="2" uri="/json/two.json" path="fn:doc(&quot;/json/two.json&quot;)" score="2048" confidence="0.283107" fitness="0.235702">
        <search:snippet>
          <search:match path="fn:doc(&quot;/json/two.json&quot;)/*:json/*:two/*:child">I come to bury <search:highlight>Caesar</search:highlight>, not to praise him.</search:match>
        </search:snippet>
      </search:result>
      <search:qtext>caesar</search:qtext>
      <search:metrics>...</search:metrics>
    </search:response>
  3. Run the search command again, generating JSON output by adding format=json:
    $ curl --anyauth --user user:password \
        'http://localhost:8004/v1/search?format=json&q=caesar'
    {
      "snippet-format": "snippet",
      "total": 2,
      "start": 1,
      "page-length": 10,
      "results": [
        {
          "index": 1,
          "uri": "\/xml\/one.xml",
          "path": "fn:doc(\"\/xml\/one.xml\")",
          "score": 2048,
          "confidence": 0.283107,
          "fitness": 0.235702,
          "matches": [
            {
              "path": "fn:doc(\"\/xml\/one.xml\")\/one\/child",
              "match-text": [
                "The noble Brutus has told ",
                {
                  "highlight": "Caesar"
                },
                " was ambitious"
              ]
            }
          ]
        },
        {
          "index": 2,
          "uri": "\/json\/two.json",
          "path": "fn:doc(\"\/json\/two.json\")",
          "score": 2048,
          "confidence": 0.283107,
          "fitness": 0.235702,
          "matches": [
            {
              "path": "fn:doc(\"\/json\/two.json\")\/*:json\/*:two\/*:child",
              "match-text": [
                "I come to bury ",
                {
                  "highlight": "Caesar"
                },
                ", not to praise him."
              ]
            }
          ]
        }
      ],
      "qtext": "caesar",
      "metrics": { ... }
    }

Additional query features allow you to search using structured queries, to search by JSON key and value or element and element attribute values, and to search and analyze lexicons and range indexes. You can also define search options to tailor your search and results. For details, see Using and Configuring Query Features.

Tear Down the REST API Instance

This procedure uses the rest-apis service on port 8002 to remove the MarkLogic REST API instance created in Create a REST API Instance. By default, removing the instance leaves the content and modules databases associated with the instance intact, but in this example we remove them by using the include request parameter.

Tearing down a REST API instance causes a server restart.

Follow this procedure to remove the 'rest-examples' instance and associated content and modules databases:

  1. Execute the following shell command to remove the instance and database:
    $ curl --anyauth --user user:password -X DELETE \
      'http://localhost:8002/v1/rest-apis/rest-example?include=content&include=modules'
  2. Navigate to the Admin Interface in your browser to confirm removal of the databases and App Server:
    http://localhost:8001

Overview of MarkLogic REST API Services

This sections gives a brief overview of the services provided by the MarkLogic REST API and where to find out more information. Additional finer grained services are available in many cases. For example, in addition to /config/query (manage query options), there is a /config/query/{name} service (manage a specific option). For details, refer to the MarkLogic REST API Reference.

These services are made available through an HTTP App Server when you create an instance of the MarkLogic REST API. For details, see Creating an Instance.

In addition to these services, you can create resource service extensions using XQuery. For details, see Extending the REST API.

Service Description More Information
/rest-apis MarkLogic REST API instance administration, including creating and tearing down instances. Administering MarkLogic REST API Instances
/documents Document manipulation, including creating, updating and deleting documents and meta data. Manipulating Documents
/search Search content and metadata using string and structured queries. Using and Configuring Query Features
/keyvalue Search by XML content by element, element attribute, and field name and value. Search JSON content by JSON key and value. Using and Configuring Query Features
/values Retrieve lexicon and range index values and value co-occurrences. Apply builtin and user-defined aggregate functions to lexicon and range index values and value co-occurrences. Using and Configuring Query Features
/transactions Support for evaluating REST requests in multi-statement transactions. Create, commit, rollback, and monitor transactions. Managing Transactions
/resources Access to user-defined resource service extensions. Extending the REST API
/config/query Create, modify, delete, and read configuration options used to control queries made using /search, /keyvalue, and /values. Configuring Query Options
/config/indexes Compare query options against the database configuration to determine whether all required indexes are configured in the database. Checking Index Availability
/config/properties Configure instance-wide properties, such as enabling debug output and setting the content type of error messages. Configuring Instance Properties
/config/transforms Create, update, delete, and read user-defined content transformations. Transformations can be used to modify content when it is inserted into or retrieved from the database using the /documents service. Working With Content Transformations
/config/resources Manage resource service extensions. Extending the REST API
/config/namespaces Create and manage instance-wide namespace prefix bindings. Such bindings allow you to use namespace prefixes in queries. Using Namespace Bindings

Security Requirements

The user with which you make a MarkLogic REST API request must have appropriate privileges for the content accessed by the job, such as permission to read or update documents in the target database.

In addition, the user must use one of the pre-defined roles listed below, or the equivalent privleges. The role/privilege requirement for each MarkLogic REST API operation is listed in the MarkLogic REST API Reference.

Role Description
rest-reader Enables read operations through the MarkLogic REST API, such as retrieving documents and metadata. This role does not grant any other privileges, so the user might still require additional privileges to read content.
rest-writer Enables write operations through the MarkLogic REST API, such as creating documents, metadata, or configuration information. This role does not grant any other privileges, so the user might still require additional privileges to write content.
rest-admin Enables administrative operations through the MarkLogic REST API, such as creating an instance and managing instance configuration. This role does not grant any other privileges, so the user might still require additional privileges.

If no permissions are explicitly set, documents you create with the MarkLogic REST API have a read permission for the rest-reader role and an update permission for the rest-writer role.

For details about roles and privileges, see the Understanding and Using Security Guide.

Terms and Definitions

The following terms and definitions are used in this guide:

Term Definition
REST REpresentational State Transfer, an architecture style that, in the context of the MarkLogic REST API, describes the use of HTTP to make calls between a client application and MarkLogic Server to create, update, delete and query content and metadata in the database.
resource An abstraction of a MarkLogic REST API service, as presented by the REST architecture.
resource address A URL that identifies a MarkLogic Server resource. Resource addresses are described in Understanding REST Resources.
rewriter An XQuery module that interprets the URL of an incoming HTTP request and rewrites it to an internal URL that services the request.
REST API instance An instantiation of the MarkLogic REST API against which applications can make RESTful HTTP requests. An instance consists of an HTTP App Server, a URL rewriter, a content database, a modules database, and the modules that implement the API. For details, see Administering MarkLogic REST API Instances.
extension An user-defined XQuery module that implements additional resource services that are made available through the MarkLogic REST API. For details, see Extending the REST API.
string query A simple search string constructed using either the default MarkLogic Server search grammar, or a user-defined grammar. For example, 'cat' and 'cat OR dog' are string queries. For details, see Querying Documents and Metadata.
structured query The pre-parsed representation of a query, expressed as XML or JSON. Structured queries allow you to express complex queries very efficiently. For details, see Querying Documents and Metadata and Using Structured Search as an Alternate to cts:query in the Search Developer's Guide.
lexicon A list of unique words or values, either throughout an entire database or within named elements, attributes, or fields. You can also define lexicons that allow quick access to the document and collection URIs in the database. Lexicons are usually backed by a range index. For details, see Querying Lexicons and Range Indexes and Browsing With Lexicons in the Search Developer's Guide.
endpoint An XQuery module on MarkLogic Server that is invoked by and responds to an HTTP request for monitoring information.

Understanding REST Resources

This section covers the basic structure of a MarkLogic REST API URL. If you are already familiar with REST resource addressing, you can skip this section. The following topics are covered:

Addressing a Resource

A resource address takes the form of a URL that includes a host name and a port number:

http://host:port/version/resource/

The host and port must reference a host running MarkLogic Server with a MarkLogic REST API instance running on that port. A MarkLogic REST API instance is served by an HTTP App Server. For details, see Creating an Instance.

A resource address always includes the API version in URL. For details, see Specifying the REST API Version.

You can optionally include parameters in a resource address as follows:

http://host:port/version/resource?param=value&param=value

For details, see Specifying Parameters in a Resource Address.

Specifying the REST API Version

To guarantee stable behavior of the MarkLogic REST API as new versions are released, each resource address in the MarkLogic REST API includes a version number. The examples in this chapter show the version as v1 or simply version. The version has the format:

v#

Where # is the version number. For example, in the initial version of the API:

http://localhost:8003/v1/documents

The current version number is v1.

The version number is only updated when resource addresses and/or parameters have changed. It is not updated when resource addresses and/or parameters are added or removed.

Specifying Parameters in a Resource Address

Resource addresses accept request parameters to tailor behavior or control input and output format.

To specify multiple parameters, use the ‘?' sign before the first parameter and the ‘&' sign before any additional parameters:

http://host:port/version/resource?param1=value&param2=value....

See the MarkLogic REST API Reference for a list of parameters available with each resource.

Understanding the Example Commands

The examples in this guide use the curl command line tool to send HTTP requests that exercise the MarkLogic REST API. The examples also use Unix command line syntax. Review this section if you are not familiar with curl or Unix command line syntax. If you are not familiar with RESTful URL conventions, see Understanding REST Resources.

Introduction to the curl Tool

curl is a command line tool for sending HTTP requests. If you do not have curl, you can download a copy from http://curl.haxx.se/download.html, or use an equivalent tool. This section provides a brief overview of the curl command line options used in this guide. For details, see the curl man page or the online documentation at http://curl.haxx.se/docs/.

The curl command line is of the form:

curl options URL

The options most often used in the examples in this guide are summarized in the table below.

Option Description
--anyauth Have curl figure out the authentication method. The method depends on your REST API instance App Server configuration. You can specify an explicit method using options such as --digest or --basic.
--user username:password Username and password with which to authenticate the request. Use a MarkLogic Server user that has sufficient privileges to carry out the requested operation. For details, see Security Requirements.
-X http_method The type of HTTP request (GET, PUT, POST, DELETE) that curl should send. If -X is not given, GET is assumed.
-d data Data to include in the request body. Data may be placed directly on the command line as an argument to -d, or read from a file by using @filename. The examples in this guide usually read from file to simplify the command line. For example, curl -X POST -d @./my-body.xml ... reads the post body contents from the file ./my-body.xml.
-H headers HTTP headers to include in the request. This is most often used by the examples to specify Accept and Content-type headers.
-i Specifies that the curl output should include the HTTP response headers in the output. By default, curl doesn't display the response header.

For example, the following command sends a POST request with the contents of the file my-body.json in the request body, and specifies the Content-type as application/json:

$ curl --anyauth --user me:mypassword -X POST -d @my-body.json \
    -H "Content-type: application/json" \
    http://localhost:8003/v1/config/query/my-options

Modifying the Example Commands for Windows

The command line examples in this guide use Unix command line syntax, usable from either a Unix or Cygwin command line. If you are the Windows command interpreter, Cmd.exe, use the following guidelines to modify the example commands for your environment:

  • Omit the '$' character at the beginning of the command. This is the Unix default prompt, equivalent to '>' in Windows.
  • For aesthetic reasons, long example command lines are broken into multiple lines using the Unix line continuation character '\'. Remove the line continuation characters and place the entire command on one line, or replace the line continuation characters with the Windows equivalent, '^'.
  • Replace arguments enclosed in single quotes (') with double quotes ("). If the single-quoted string contains embedded double quotes, escape the inner quotes.
  • Escape any unescaped characters that have special meaning to the Windows command interpreter.

Controlling Input Content Type

Many services of the MarkLogic REST API accept input in either XML or JSON. Unless otherwise noted, where content type control is supported, you can specify the content type using either the format request parameter or the HTTP Content-type header. If both the format parameter and the Content-type header are present, the format parameter takes precedence.

For POST methods that have data in both the request body and the response body, the format parameter overrides both Content-type and Accept headers and applies to both request and response data. If you want to use different content types for input and output on such methods, use the HTTP headers only; do not set format. For an example, see the reference documentation for POST /v1/search.

For JSON input, set the format request parameter to 'json' or set the HTTP Content-type header to application/json. For example:

$ curl -X POST -d @'my-body.json' --anyauth --user user:password \
    http://localhost:8003/v1/config/query/my-opts?format=json
$ curl -X POST -d @'my-body.json' --anyauth --user user:password \
    -H "Content-type: application/json" \
    http://localhost:8003/v1/config/query/my-opts

For XML input, set the format request parameter to 'xml' or set the HTTP Content-type header to application/xml. For example:

$ curl -X POST -d @'my-body.xml' --anyauth --user user:password \
    http://localhost:8003/v1/config/query/my-opts?format=xml
$ curl -X POST -d @'my-body.json' --anyauth --user user:password \
    -H "Content-type: application/xml" \
    http://localhost:8003/v1/config/query/my-opts

Controlling Output Content Type

Many services of the MarkLogic REST API that return metadata or configuration information allow you to select JSON or XML output. XML output is the default unless otherwise specified in the MarkLogic REST API Reference for a specific operation.

Where output content type control is supported, use the format request parameter or the HTTP Accept header to specify the response content type. If both the parameter and the Accept header are set, the format parameter takes precendence.

For POST methods that have data in both the request body and the response body, the format parameter overrides both Content-type and Accept headers and applies to both request and response data. If you want to use different content types for input and output on such methods, use the HTTP headers only; do not set format. For an example, see the reference documentation for POST /v1/search.

To request JSON output, set the format request parameter to 'json' or set the HTTP Accept header to application/json. For example:

$ curl -X GET --anyauth --user user:password http://localhost:8002/v1/rest-apis?format=json
$ curl -X GET --anyauth --user user:password -H "Accept: application/json" http://localhost:8002/v1/rest-apis

To explicitly request XML output, set format to xml or set the HTTP Accept header to application/xml. For example:

$ curl -X GET --anyauth --user user:password http://localhost:8002/v1/rest-apis?format=xml
$ curl -X GET --anyauth --user user:password -H "Accept: application/xml" http://localhost:8002/v1/rest-apis

Error Reporting

This section covers the error reporting conventions followed by the MarkLogic REST API.

If a request to a MarkLogic REST API instance fails, an error status code is returned and additional error detail is provided in the response body. You can configure your instance to return error details in either XML or JSON using the error-format configuration property. XML is the default content type. For details, see Configuring Instance Properties.

The following example shows error output in XML for a request containing unsupported parameters. The return status code is 400 (Bad Request) and the details of the error, identifying the failure as a REST-UNSUPPORTEDPARAM exception, are contained in the response body.

HTTP/1.1 400 Bad Request
Content-type: application/xml
Server: MarkLogic
Content-Length: 333
Connection: close

<rapi:error xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:status-code>400</rapi:status-code>
  <rapi:status>Bad Request</rapi:status>
  <rapi:message-code>REST-UNSUPPORTEDPARAM</rapi:message-code>
  <rapi:message>REST-UNSUPPORTEDPARAM: (rest:UNSUPPORTEDPARAM) Endpoint does not support query parameter: unknown</rapi:message>
</rapi:error>

The following example is the same error, with the error detail returned as JSON:

HTTP/1.1 400 Bad Request
Content-type: application/json
Server: MarkLogic
Content-Length: 206
Connection: close

{
  "error": {
    "status-code": "400",
    "status": "Bad Request",
    "message-code": "REST-UNSUPPORTEDPARAM",
    "message": "REST-UNSUPPORTEDPARAM: (rest:UNSUPPORTEDPARAM) Endpoint does not support query parameter: unknown"
  }
}

Errors that can be corrected by the client application, such as an invalid parameter or an unsupported HTTP method, are usually reported as a 4XX error with a REST- or RESTAPI- message code.

Errors that cannot be addressed by the client application are usually reported as a 500 Internal Server Error. A 500 error does not necessarily mean that the problem cannot be corrected or that MarkLogic Server got an internal error. A 500 error usually indicates a problem that requires correction on the server host rather than in the client application.

An example of a 500 error that is correctable on the server side is failing to create an element range index required to support an operation. If a client application uses the /search service with a search constraint that requires a non-existent index, a 500 error is returned. To correct the error, an administrator would create the required index in MarkLogic Server.

XQuery and XSLT transforms and XQuery extensions you write should report errors using RESTAPI-EXTNERR, as described in Reporting Errors.

« Table of contents
Next chapter »