Loading TOC...
REST Application Developer's Guide (PDF)

REST Application Developer's Guide — Chapter 4

Using and Configuring Query Features

This chapter covers the following topics:

Query Feature Overview

The REST Client API includes several services for querying content in a MarkLogic Server database. The usage model for the query features is:

  1. Optionally, use the /config/query service to install a set of named query options to apply to future queries. The name is used to apply the options to subsequent queries. For details, see Configuring Query Options.
  2. Optionally, use the /config/namespaces service to install namespace bindings for namespace aliases you will use in queries. For details, see Using Namespace Bindings.
  3. Use /search, /qbe, or /values to perform a query.
    1. Depending on the service, the query may be expressed using a request parameter or in the request body. For details, see the reference documentation for each method.
    2. Apply options by specifying the name of previously installed query options in the options request parameter and/or providing dynamic query options in the request body using a combined query. For details, see Adding Query Options to a Request.
    3. Specify the content type (XML or JSON) of the query results using the format request parameter or Accept headers.
  4. Query results of the requested content type are returned in the response body.

By default, a query operation returns search response data that summarizes the matches and can contain snippets of matching content. This chapter focuses on such operations. However, you can also use /search and /qbe to retrieve the matching documents or metadata instead of a search response; for details, see Reading and Writing Multiple Documents.

The table below gives a brief summary of each query related service.

Service Description
/search Use string, structured, and combined queries to search documents and metadata. Queries may be expressed in XML or JSON. For details, see Querying Documents and Metadata.
/keyvalue

This interface is deprecated. Use /search or /qbe instead.

Use key-value pair queries to search XML content and metadata, and JSON content. For example, search by element, element attribute, or JSON property values. Dynamic query options are not supported by this service. For details, see Querying Documents and Metadata.

/qbe Use Query By Example (QBE) for rapid prototyping of XML and JSON document searches. Queries using QBE syntax may be expressed in XML or JSON. For details, see Using Query By Example to Prototype a Query.
/values Query lexicon and range index values and value co-occurrences; analyze lexicon and range index values and value-co-occurrences with builtin and user-defined aggregate functions. For details, see Querying Lexicons and Range Indexes.
/config/query Use XML or JSON to configure persistent query options for use with /search, /keyvalue, and /values. For details, see Configuring Query Options.
/config/namespaces Configure bindings between namespace prefixes and namespace URIs so you can use QNames in query contexts where it is not possible to dynamically specify a namespace. For details, see Using Namespace Bindings.

The /search and /values services use query options to control and configure queries and search results. For details, see Appendix: Query Options Reference in the Search Developer's Guide.

Querying Documents and Metadata

This section describes how to use the /search and /keyvalue services to search documents and metadata.

You can use /search to retrieve a search response, matching documents and metadata, or both. The examples in this section only return a search response. To retrieve whole documents and/or their metadata, see Reading Multiple Documents Matching a Query.

Constraining a Query by Collection or Directory

Use the collection and directory request parameters to the /search and /keyvalue services to limit search results to matches in specific collections or database directories.

You can specify multiple collections or directories. For example, the following URL finds documents containing 'julius' in the 'tragedy' and 'comedy' collections:

http://localhost:8000/LATEST/search?q=julius&collection=tragedy&collection=comedy

You can use collection and directory constraints together. For example, by adding a directory to the above search, you can further limit matches to documents in the '/shakespeare/plays' directory:

http://localhost:8000/LATEST/search?q=julius&collection=tragedy&collection=comedy&directory=/shakespeare/plays

For details about collections and directories, see Collections in the Search Developer's Guide and Directories in the Application Developer's Guide.

Searching With String Queries

The MarkLogic Server Search API default search grammar allows you to quickly construct simple searches such as 'cat', 'cat AND dog', or 'cat NEAR dog'. You can also customize the search grammar. For details, see Search Grammar in the Search Developer's Guide.

To search for matches to a simple string query, send a GET request to the /search service with a URL of the form:

http://host:port/version/search?q=query_string

Where query_string is a string conforming to the search grammar.

You can include both a string query and a structured query in the same request. In this case, the two queries are AND'd together. For more information, see Searching With Structured Queries and Specifying Dynamic Query Options with Combined Query.

You can request search results in XML or JSON. Use the Accept header or format request parameter to select the output content type. For details, see Controlling Input and Output Content Type.

For details on the structure of search results, see Appendix: Query Options Reference in the Search Developer's Guide.

For a complete list of parameters available with the /search service, see GET /v1/search in the REST Resources API.

Searching With Structured Queries

Structured queries enable you to create complex queries, represented in XML or JSON. For details, see Searching Using Structured Queries in the Search Developer's Guide.

To search using a structured query, send a GET or POST request to the /search service. To pass the structured query as a request parameter, send a GET request with a URL of the form:

http://host:port/version/search?structuredQuery=query

Where query is an XML or JSON representation of a structured query.

To pass a structured query in the request body, send a POST request of the following form and place a structured or combined query in the POST body. Set the Content-type header appropriately:

http://host:port/version/search

If the request also includes a string query, the string query and structured query are AND'd together. For details, see Searching With String Queries and Specifying Dynamic Query Options with Combined Query.

You can request search results in XML or JSON. Use the Accept header or format request parameter to select the output content type. For details, see Controlling Input and Output Content Type.

For a complete list of parameters available with the /search service, see GET /v1/search or POST /v1/search in the REST Resources API.

The following example combines a structured query equivalent to 'Yorick NEAR Horatio' and a string query for 'knew', requesting search results in JSON. The effect is equivalent to searching for '(Yorick NEAR Horatio) AND knew'.

$ cat sq.xml
<search:query xmlns:search="http://marklogic.com/appservices/search">
  <search:near-query>
    <search:term-query>
      <search:text>Yorick</search:text>
    </search:term-query>
    <search:term-query>
      <search:text>Horatio</search:text>
    </search:term-query>
  </search:near-query>
</search:query>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X POST -d@"./sq.xml" \
  -H 'Content-type: application/xml' \
  -H 'Accept: application/json' \
  'http://localhost:8000/LATEST/search?q=knew'

Searching By JSON Property Name and Value

This interface is deprecated. To search the database based on the value of a JSON property, XML element, or XML element attribute, use QBE or structured query instead.

When you store JSON documents in the database using the /documents service of the REST Client API, you can search based on the values of JSON properties using the /keyvalue service.

Efficient key-value searches require an element range index on the key. For details, see Creating Indexes on JSON Properties.

To search by key value, send a GET request to the /keyvalue service with a URL of the form:

http://host:port/version/keyvalue?key=the_key&value=the_value

Where the_key is the name of the key to match against the_value. For example, the following search matches all JSON documents containing a key called 'species' with a value of 'canis lupus':

http://localhost:8000/LATEST/keyvalue?key=species&value=canis%20lupus

Only JSON documents are considered in the search when you use the key parameter. To query XML content, see Searching By Element or Attribute Name and Value.

You can request search results in XML or JSON. Use the Accept header or format request parameter to select the output content type. For details, see Controlling Input and Output Content Type.

For a complete list of parameters available with the /keyvalue service, see GET /v1/keyvalue in the REST Resources API.

The following example searches for occurrences of the key 'SPEAKER' with the value 'BERNARDO'. First, insert a JSON document into the database:

$ cat hamlet.json
{
  "TITLE":"Hamlet",
  "ACT":{
    "TITLE":"ACT I",
    "SCENE":{
      "TITLE":"SCENE I. Elsinore. A platform before the castle.",
      "STAGEDIR":"FRANCISCO at his post. Enter to him BENARDO",
      "SPEECHES":[
        {"SPEECH":{
          "SPEAKER":"BERNARDO",
          "LINE":"Whos there?"
        }},
        {"SPEECH":{
          "SPEAKER":"FRANCISCO",
          "LINE":"Nay, answerme: stand, and unfold yourself."
        }},
        {"SPEECH":{
          "SPEAKER":"BERNARDO",
          "LINE":"Long live the king!"
        }}
      ]
    }
  }
}
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT -d @"./hamlet.json" \
  -H 'Content-type: application/json' \
  http://localhost:8000/LATEST/documents?uri=/example/hamlet.json

Then search for occurrences of SPEAKER with the value BERNARDO. The search matches the document just inserted and returns a snippet for the two speeches by Bernardo:

$ curl --anyauth --user user:password -X GET \
  'http://localhost:8000/LATEST/keyvalue?key=SPEAKER&value=BERNARDO'
...
{
  "snippet-format":"snippet",
  "total":1,
  "start":1,
  "page-length":10,
  "results":[{
    "index":1,
    "uri":"/example/hamlet.json",
    "path":"fn:doc(\"/example/hamlet.json\")",
    "score":154880,
    "confidence":0.790972,
    "fitness":0.790972,
    "matches":[       {"path":"fn:doc(\"/example/hamlet.json\")/*:json/*:ACT/*:SCENE/*:SPEECHES/*:json[1]/*:SPEECH/*:SPEAKER",
       "match-text":[{"highlight":"BERNARDO"}]},
      {"path":"fn:doc(\"/example/hamlet.json\")/*:json/*:ACT/*:SCENE/*:SPEECHES/*:json[3]/*:SPEECH/*:SPEAKER",
       "match-text":[{"highlight":"BERNARDO"}]}
    ]
  }],
  "metrics":{
    "query-resolution-time":"PT0.001398S",
    "facet-resolution-time":"PT0.000052S",
    "snippet-resolution-time":"PT0.001411S",
    "total-time":"PT0.003032S"
  }
}

Searching By Element or Attribute Name and Value

This interface is deprecated. To search the database based on the value of a JSON property, XML element, or XML element attribute, use QBE or structured query instead.

The /keyvalue service provides a simple interface to match element and element attribute values in XML content. For more sophisticated element and element attribute value searches, use structured queries; see Searching With Structured Queries.

Efficient element and element attribute searches require element and attribute range indexes. For details, see Range Indexes and Lexicons in the Administrator's Guide.

To find all XML elements with a specific element name and value, send a GET request to the /keyvalue service with a URL of the form:

http://host:port/version/keyvalue?element=elem_name&value=elem_value

If the element name is defined within a namespace, you must pre-define a namespace binding and use the defined prefix in elem_name. For details, see Using Namespace Bindings.

The following example matches documents containing a <SPEAKER> element with the value 'BERNARDO':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
   'http://localhost:8000/LATEST/keyvalue?element=SPEAKER&value=BERNARDO'

To find all XML elements with a specific element name that have an attribute with a specific value, send a GET request to the /keyvalue service with a URL of the form:

http://host:port/version/keyvalue?element=elem_name&attribute=attr_name&value=elem_value

This example matches documents containing <nominee> elements with a year attribute value of 1946:

$ curl --anyauth --user user:password -X GET \
    'http://localhost:8000/LATEST/keyvalue?element=nominee&attribute=year&value=1946'

MarkLogic Server can return search results in XML or JSON. Use the Accept header or format request parameter to select the output content type. For details, see Controlling Input and Output Content Type.

For a complete list of parameters available with the /keyvalue service, see GET /v1/keyvalue in the REST Resources API.

Debugging /search Queries With Logging

When you enable the debug REST API instance property and make queries using /search, query details are sent to the MarkLogic Server error log. Enable this logging by setting the debug property to true, as described in Configuring Instance Properties.

When you enable debug logging, requests to /search log the following information helpful in debugging queries:

  • EFFECTIVE OPTIONS. These are the query options in effect for your query. If the request does not include an options request parameter or a combined query, these are the default options.
  • CTS-QUERY. Your query, expressed as a cts:query object. A cts:query object is the low level XML representation of any query expression. For details, see Understanding cts:query in the Search Developer's Guide.
  • SEARCH-QUERY. Your query, expressed as a search:query object. This information is only included if your request includes a structured query. A search:query is the XML representation of a structured query. For details, see Searching Using Structured Queries in the Search Developer's Guide.

Examining the logging output can help you understand how MarkLogic Server sees your query. In the following example, notice that a string query for 'Welles' is really a cts:word-query for the text string 'Welles'. With the addition of a structured query in the request body to limit the query to ocurrences of 'Welles' to documents that also include 'John' when it occurs near 'Huston', the logging output includes a cts:query and a search:query that reflect the result of combining the string query and structured query.

  1. Enable debug logging for your REST API instance.
    $ cat debug-on.xml
    <properties xmlns="http://marklogic.com/rest-api">
      <debug>true</debug>
    </properties>
    $ curl --anyauth  --user user:password -X PUT \
        -d @./debug-on.xml -H "Content-type: application/xml" \
        http://localhost:8000/LATEST/config/properties
  2. Issue a query to /search.
    $ curl --anyauth --user user:password -X GET \
        -H "Accept: application/xml" \
        http://localhost:8000/LATEST/search?q="Welles"
    ...search matches returned
  3. View the debug output in the MarkLogic Server error log, located in MARKLOGIC_DIR/Logs/ErrorLog.txt. There is no search:query in the log because this is a simple string query. (The timestamp and Info: header on each log line have been elided).
    $ tail -15 /var/opt/MarkLogic/Logs/ErrorLog.txt
    ... Request environment:
    ... GET /v1/search?q=Douglas
    ... Rewritten to:
        /MarkLogic/rest-api/endpoints/search-list-query.xqy?q=Douglas
    ... ACCEPT */*
    ... PARAMS:
    ...   q: (Welles)
    ...
    ... Endpoint Details:
    ... EFFECTIVE OPTIONS:
    ... <options xmlns="http://marklogic.com/appservices/search">
    ...   <search-option>unfiltered</search-option>
    ...   <quality-weight>0</quality-weight>
    ... </options>
    ... CTS-QUERY:
    ... <cts:word-query qtextref="cts:text"
                xmlns:cts="http://marklogic.com/cts">
          <cts:text>Welles</cts:text>
        </cts:word-query>
  4. Add a structured query to the search to constrain matches to documents that also contain 'John' occurring near 'Huston'.
    $ cat ./structq.json
    {"query":
      {"near-query":
        {"queries":[
          {"term-query":{"text":"John"}},
          {"term-query":{"text":"Huston"}}
        ]}
      }
    }
    $ curl --anyauth --user user:password -X POST -d@./structq.json \
        -H "Content-type: application/json" \
        http://localhost:8000/LATEST/search?q=Welles
    ...search results returned
  5. View the logging output again and notice the inclusion of a search:query section in the log. Also, notice that the cts:query and the search:query represent the combined string and structured queries.
    ... Request environment:
    ... POST /v1/search?q=Welles
    ... Rewritten to:
        /MarkLogic/rest-api/endpoints/search-list-query.xqy?q=Welles
    ... ACCEPT */*
    ... PARAMS:
    ...   q: (Welles)
    ...
    ... Endpoint Details:
    ... EFFECTIVE OPTIONS:
    ... <options xmlns="http://marklogic.com/appservices/search">
    ...   <search-option>unfiltered</search-option>
    ...   <quality-weight>0</quality-weight>
    ... </options>
    ... CTS-QUERY:
    ... cts:and-query(
          (
            cts:word-query("Welles", ("lang=en"), 1), 
            cts:near-query((
              cts:word-query("John", ("lang=en"), 1),
              cts:word-query("Huston", ("lang=en"), 1)), 
            10, (), 1)
           ), ()
         )
    ... SEARCH-QUERY:
    ... <search:query
              xmlns:search="http://marklogic.com/appservices/search">
    ...   <search:qtext>Welles</search:qtext>
    ...   <search:near-query>
    ...     <search:term-query>
    ...       <search:text>John</search:text>
    ...     </search:term-query>
    ...     <search:term-query>
    ...       <search:text>Huston</search:text>
    ...     </search:term-query>
    ...   </search:near-query>
    ... </search:query>

Querying Lexicons and Range Indexes

The /values service supports the following operations:

For related search concepts, see Browsing With Lexicons in the Search Developer's Guide.

This section covers the following topics:

Querying the Values in a Lexicon or Range Index

Use the /values/{name} service to query the values in a lexicon or range index. Such queries must be supported by query options that include a <values/> element identifying the target lexicon or index; for details, see Defining Queryable Lexicon or Range Index Values.

To query the values in a lexicon or range index, use the /values/{name} service as follows:

  1. Install query options or define dynamic query options that include a values option naming the target lexicon or index. For details, see Adding Query Options to a Request.
  2. Send a GET or POST request to the /values/{name} service, where name is the name of a values definition in the query options from Step 1 or in the default query options. If you use persistent query options, include the options parameter and replace options_name with the name under which the query options are installed.
    http://host:port/version/values/name?options=options_name

When constructing your request:

  1. Substitute a named range specification for name in the URL. This must be a values range specification in the query options named by the options parameter, in the options portion of a combined query in the POST body, or in the default query options.
  2. To use custom query options, specify them using the options parameter and/or the options portion of a combined query in the POST body. For details, see Identifying Lexicon and Range Index Values in Query Options and Adding Query Options to a Request.
  3. To use the default query options, omit the options parameter and options portion of a combined query. The default query options should include a range specification matching name.
  4. To constrain the analysis to values in certain fragments, specify a query using the q and/or structuredQuery parameters, or a structured or combined query in the POST body. For details, see Using a Query to Constrain Results.
  5. Specify the input (POST only) and output content type (XML or JSON) using the format parameter or the HTTP Content-type and Accept headers. For details, see Controlling Input and Output Content Type.

Additional request parameters are available. For details, see the MarkLogic REST API Reference.

The following example assumes the query options shown in the table below are installed under the name index-options:

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
    <values name="speaker">
        <range type="xs:string">
            <element ns="" name="SPEAKER"/>
        </range>
    </values>
</options>
JSON
{
  "options": {
    "values": [
      {
        "name": "speaker",
        "range": {
          "type": "xs:string",
          "element": { "ns": "", "name": "SPEAKER" }
        }
      }
    ]
  }
}

Then the example command below queries /values/speaker to retrieve the values of all SPEAKER elements:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    http://localhost:8000/LATEST/values/speaker?options=index-options
...
<values-response name="speaker" type="xs:string" \
    xmlns="http://marklogic.com/appservices/search" \
    xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <distinct-value frequency="1">[GOWER]</distinct-value>
  <distinct-value frequency="1">[PROSPERO]</distinct-value>
  ...
</values-response>

If you use the format parameter to request JSON output from the same request, the results are similar to the following:

{
  "values-response": {
    "name": "speaker",
    "type": "xs:string",
    "distinct-value": [
      {
        "frequency": 1,
        "_value": "[GOWER]"
      },
      {
        "frequency": 1,
        "_value": "[PROSPERO]"
      },
      ...
    ],
    "metrics": {
      "values-resolution-time": "PT0.016665S",
      "aggregate-resolution-time": "PT0.00001S",
      "total-time": "PT0.018102S"
    }
  }
}

If you add a string query, structured query, or both to the request, you can limit the results to index values in matching fragments. For example, the following requests returns only the SPEAKER values in fragments containing 'HAMLET':

$ curl --anyauth --user user:password -X GET \
    'http://localhost:8000/LATEST/values/speaker?options=index-options&q="HAMLET"'

Finding Value Co-Occurrences in Lexicons

A co-occurrence is a set of index or lexicon values occurring in the same document fragment. The REST Client API enables you to query for n-way co-occurrences. That is, tuples of values from multiple lexicons or indexes, occurring in the same fragment.

Use this procedure to query for co-occurrences of values in lexicons or range indexes with the /values/{name} service:

  1. Specify query options that include a tuples range specification for the target lexicons or indexes. For details, see Defining Queryable Lexicon or Range Index Co-Occurrences and Adding Query Options to a Request.
  2. Send a GET or POST request of the following form to the /values/{name} service, where name is the name of a tuples definition in the query options from Step 1. If you use persistent query options, include the options parameter and replace options_name with the name under which the query options are installed.
    http://host:port/version/values/name?options=options_name

When constructing your request:

  1. Substitute a named range specification for name in the URL. This must be a tuples range specification in the query options named by the options parameter, in the options portion of a combined query in the POST body, or in the default query options if options is omitted.
  2. To use custom query options, specify them using the options parameter and/or the options portion of a combined query in the POST body. For details, see Identifying Lexicon and Range Index Values in Query Options and Adding Query Options to a Request.
  3. To use the default query options, omit the options parameter and options portion of a combined query. The default query options should include a range specification matching name.
  4. To constrain the analysis to values in certain fragments, specify a query using the q and/or structuredQuery parameters, or a structured or combined query in the POST body. For details, see Using a Query to Constrain Results.
  5. Specify the input (POST only) and output content type (XML or JSON) using the format parameter or the HTTP Content-type and Accept headers. For details, see Controlling Input and Output Content Type.

Additional request parameters are available. For details, see the MarkLogic REST API Reference.

For more information about co-occurrences, see Value Co-Occurrences Lexicons in the Search Developer's Guide.

The following example assumes the query options shown in the table below are installed under the name index-options. Note that the options include a <tuples/> definition named 'speaker-scene'.

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
  <tuples name="speaker-scene">
    <range type="xs:string">
      <element ns="" name="SPEAKER"/>
    </range>
    <range type="xs:string">
      <path-index>
        /PLAY/ACT/SCENE/TITLE
      </path-index>
    </range>
  </tuples>
</options>
JSON
{
  "options": {
    "tuples": [
      {
        "name": "speaker-scene",
        "range": [
          {
            "type": "xs:string",
            "element": {
              "ns": "",
              "name": "SPEAKER"
            }
          },
          {
            "type": "xs:string",
            "path-index": {
              "text": "\/PLAY\/ACT\/SCENE\/TITLE"
            }
          }
        ]
      }
    ]
  }
}

Given these query options, this example queries /values/speaker-scene to retrieve co-occurrences of SPEAKER and scene titles:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
   http://localhost:8000/LATEST/values/speaker-scene?options=index-options
...
<values-response name="speaker-scene"
    xmlns="http://marklogic.com/appservices/search"
    xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <tuple frequency="1">
    <distinct-value xsi:type="xs:string" ...>A Lord</distinct-value>
    <distinct-value ...>SCENE II.  The forest.</distinct-value>
  </tuple>
  ...
</values-response>

If you add a string query, structured query, or both to the request, you can limit the results to co-occurrences in matching fragments. For example, the following requests returns only the SPEAKER values in fragments containing 'HAMLET':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    'http://localhost:8000/LATEST/values/speaker-scene?options=index-options&q="HAMLET"'

Using a Query to Constrain Results

You can use a string query, structured query, or combined query with /values/{name} to limit results to fragments that match the query. The values must occur in fragments matching the query. The fragments are selected in the same manner as an 'unfiltered' cts:search; for details, see Understanding Unfiltered Searches in the Query Performance and Tuning Guide.

If you use a string query, the query is treated as a word query; for details, see cts:word-query. Supply a string query using one of the following:

  • The q request parameter of a POST or GET request
  • The qtext element or or sub-object of a combined query supplied in the body of a POST request.

Specify a structured query in either XMl or JSON using one of the following:

  • The structuredQuery request parameter on a GET request.
  • The query element or sub-object of a combined query supplied in the body of a POST request.
  • In the body of a POST request that does not use a combined query.

The following example limits the results to just those fragments containing the word 'moon':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    'http://localhost:8000/LATEST/values/speaker?options=index-options&q=moon'

For the full example without a query constraint, see Querying the Values in a Lexicon or Range Index.

For details on query syntax, see the following sources:

Identifying Lexicon and Range Index Values in Query Options

Before you can use the /values/{name} service, you must install query options that identify lexicons and range indexes available to queries. Use the values option to make the values in a single lexicon or index available. Use the tuples option to make co-occurrences of values in multiple lexicons or indexes available.

This section covers the following topics:

Defining Queryable Lexicon or Range Index Values

Use this procedure to make the values in a lexicon or range index available through the /values/{name} service:

  1. Create a lexicon or range index on the database, as described in Range Indexes and Lexicons in the Administrator's Guide and
  2. Associate a name with the index or lexicon by defining a <values/> element (or JSON sub-object) in query options.
  3. Supply the query options from Step 2 in your request, as described in Adding Query Options to a Request.

For more information on lexicons and range indexes, see Browsing With Lexicons in the Search Developer's Guide and Creating Indexes on JSON Properties.

For example, if the database configuration includes an element range index on SPEAKER, such as the one shown below:

Then the following query options enable the SPEAKER element values to be referenced using as the resource /values/speaker, once you install the options:

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
    <values name="speaker">
        <range type="xs:string">
            <element ns="" name="SPEAKER"/>
        </range>
    </values>
</options>
JSON
{
  "options": {
    "values": [
      {
        "name": "speaker",
        "range": {
          "type": "xs:string",
          "element": { "ns": "", "name": "SPEAKER" }
        }
      }
    ]
  }
}

The following example command installs the XML query options with the name 'index-options':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    -d@"./index-options.xml" -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/config/query/index-options

Now, you can query the SPEAKER index using the /values/speaker resource with the query options named 'index-options'.

Defining Queryable Lexicon or Range Index Co-Occurrences

A <tuples/> element in query options specifies the indexes to use in constructing n-way value co-occurrences. For more information about co-occurrences, see Value Co-Occurrences Lexicons in the Search Developer's Guide.

Use this procedure to make co-occurrences of values in multiple lexicons or range indexes available through the /values/{name} service:

  1. Create the lexicons or range indexes on the database, as described in Range Indexes and Lexicons in the Administrator's Guide.
  2. Associate a name with the lexicon/index tuple by defining a <tuples> element in query options (or a 'tuples' object in JSON).
  3. Supply the query options from Step 2 in your request, as described in Adding Query Options to a Request.

For example, suppose the database configuration contains a string-valued element range index on <SPEAKER> and a path range index on the XPath expression /PLAY/ACT/SCENE/TITLE. The following query options enable querying co-occurrrences of these two indexes under the name 'speaker-scene':

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
    <tuples name="speaker-scene">
        <range type="xs:string">
            <element ns="" name="SPEAKER"/>
        </range>
        <range type="xs:string">
            <path-index>/PLAY/ACT/SCENE/TITLE</path-index>
        </range>
    </tuples>
</options>
JSON
{
  "options": {
    "tuples": {
      "name": "speaker-scene",
      "range": {
        "type": "xs:string",
        "path-index": {
          "text": "/PLAY/ACT/SCENE/TITLE",
        }
      }
    }
  }
}

The following example command installs the XML query options with the name 'index-options':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    -d@"./index-options.xml" -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/config/query/index-options

Now, you can query the SPEAKER index using the /values/speaker-scene resource with the query options named 'index-options'.

Creating Indexes on JSON Properties

To efficiently search using JSON properties, you should define indexes on the properties. For example,a json-property structured query performs best when you define a range index on the property you're querying. In addition, range queries require a backing index.

To create an index on a JSON property, treat the JSON property as an XML element for purposes of index creation. That is, use the interfaces for creating element index, such as an element range index.

For details, see Creating Indexes and Lexicons Over JSON Documents in the Application Developer's Guide.

Limiting the Number of Results

You can use the limit, start, and pageLength request parameters to limit the number of values or co-occurrences returned by GET /v1/values or POST:/v1/values.

The limit parameter specifies the maxiumum number of value to retrieve from a lexicon. Use start and pageLength to return results one page at a time, similar to the way you can page through results from the /search service. If limit is present, then start and pageLength are applied to the subset of values selected by limit, so the values on a page never extend beyond the values selected by limit.

For example, in the following request, at most 2 values or tuples are returned because start + pageLength would extend beyond the 5 values selected by limit.

GET /LATEST/values?limit=5&start=4&pageLength=3

If you specify a start value, you must also specify a pageLength. For a detailed discussion of how these parameters affect the results returned by values requests, see Returning Lexicon Values With search:values in the Search Developer's Guide.

Using Query By Example to Prototype a Query

This section describes how to search XML and JSON documents using a Query By Example (QBE). You cannot use QBE to search other document types or to search metadata.

This section covers the following topics:

What is QBE

A Query By Example (QBE) enables rapid prototyping of queries for 'documents that look like this' using search criteria that resemble the structure of documents in your database.

For example, if your documents include an author element or key, you can use the following QBE to find documents with an author value of 'Mark Twain'.

Format Example
XML
<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <author>Mark Twain</author>
  </q:query>
</q:qbe>
JSON
{
  "$query": { "author": "Mark Twain" }
}

You can only use QBE to search XML and JSON documents. Metadata search is not supported. You can search by element, element attribute, and JSON property; fields are not supported. For structural details, see Searching Using Query By Example in Search Developer's Guide.

When you're satisfied with your prototype or ready to use more powerful Search API features, you can use the API to convert a QBE into a combined query for use with the /search service.

The REST Client API includes the following support for QBE through the /qbe service:

  • Search XML and JSON documents using a QBE.
  • Validate the correctness of a QBE.
  • Convert a QBE to a combined query for improved performance and full expressiveness.

Searching Documents With QBE

To search using QBE, send a GET or POST request to the /qbe service. To pass the QBE as a request parameter, send a GET request with a URL of the form:

http://host:port/version/qbe?query=your-qbe

Where your-qbe is an XML or JSON representation of a QBE.

To pass a QBE in the request body, send a POST request of the following form and place a QBE in the POST body. Set the Content-type header appropriately.

http://host:port/version/qbe

You can also create a multipart POST request that contains a QBE and query options in the request body. A request of this form enables you to specify dynamic query options with a QBE, similar to using a combined query with POST /v1/search and enables you to specify the QBE and query options in different formats. When you use a multipart request body, the QBE must be the first part and the query options must be the second part. For details, see POST /v1/qbe.

You can request search results in XML or JSON. Use the Accept header or format request parameter to select the output content type. For details, see Controlling Input and Output Content Type.

You can validate the correctness of your input QBE as part of your search or as a standalone operation. For details, see Validating a QBE.

For a complete list of parameters available with the /qbe service, see GET /v1/qbe or POST /v1/qbe in the REST Resources API.

The following example matches XML documents that have an author element value of 'Mark Twain'. Results are returned as XML.

$ cat qbe.xml
<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <author>Mark Twain</author>
  </q:query>
</q:qbe>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X POST -d@"./qbe.xml" \
  -H 'Content-type: application/xml' \
  'http://localhost:8000/LATEST/qbe'

The following example shows an equivalent search using JSON.

$ cat qbe.sjon
{"$query": {
    "author": "Mark Twain"
} }
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X POST -d@"./qbe.json" \
  -H 'Content-type: application/json' \
  -H 'Accept: application/json' \
  'http://localhost:8000/LATEST/qbe

The /qbe service supports most of the same features as the /search service, such as using pre-installed persistent query options, result pagination, and search result transformations. For details, see GET /v1/qbe or POST /v1/qbe in the REST Resources API.

You can also use /qbe to retrieve matching documents and metadata. The examples in this section only return a search response. To retrieve whole documents and/or their metadata, see Reading Multiple Documents Matching a Query.

Validating a QBE

When you perform a search, MarkLogic Server does not verify the correctnesss of your QBE. If your QBE is syntactically or semantically incorrect, you might get errors or surprising results. To avoid such issues, you can validate your QBE prior to or as part of a search.

To validate your query as a standalone operation, add the request parameter view=validate to a GET or POST request to the /qbe service. Rather than performing a search, MarkLogic Server checks your QBE for correctness and returns an indication of validity returned.

For example, the following command validates a JSON QBE:

$ curl --anyauth --user user:password -i -X POST \
    -H "Content-type: application/json" -d @./qbe.json \
    'http://localhost:8000/LATEST/qbe?view=validate'

If your query is valid, MarkLogic Server responds with status 200 (OK) and the response body contains a valid-query element, similar to the following:

<q:valid-query
  xmlns:q="http://marklogic.com/appservices/querybyexample"/>

If your query is invalid, MarkLogic Server can respond with either a status 200 (OK) or status 400 (Bad Request), depending on the nature of the error. When the status code is 400, the response body contains an error. When the status code is 200, the response body contains an invalid-query element that encapsulates the reason validation failed. For example:

<q:invalid-query
    xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:report id="QBE-QUERY">Query can only contain a filtered flag, score configuration,  composers, word queries, and criteria</q:report>
</q:invalid-query>

To validate a query as part of your search, use the request parameters view=validate in conjunction with view=results. If your query is valid, the search proceeds as usual. If you query is not valid, an error report is returned.

$ curl --anyauth --user user:password -i -X POST \
    -H "Content-type: application/json" -d @./qbe.json \
    'http://localhost:8000/LATEST/qbe?view=validate&view=results'

Generating a Combined Query from a QBE

Generating a combined query from a QBE has the following potential benefits:

  • Improve search performance.
  • Access a wider array of search features.
  • Debug your QBE by examining the lower level Search API constructs it generates.

To generate a combined query from a QBE, add the view=structured request parameter to a GET or POST request to the /qbe service. Rather than performing a search, the request returns a combined query in the response. You can use the resulting query with the /search service.

You cannot combine view=structured with other view settings, such as validate or results.

The following command generates a combined query from a QBE:

$ cat qbe.xml
<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <author>Mark Twain</author>
  </q:query>
</q:qbe>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X POST -d@"./qbe.xml" \
  -H 'Content-type: application/xml' \
  'http://localhost:8000/LATEST/qbe?view=structured'
HTTP/1.1 200 OK
...
<search:search xmlns:search="http://marklogic.com/appservices/search">
  <search:query>
    <search:value-query>
      <search:element ns="" name="author"/>
      <search:text>Mark Twain</search:text>
      <search:term-option>exact</search:term-option>
    </search:value-query>
  </search:query>
  <search:options>
    <search:search-option>unfiltered</search:search-option>
    <search:quality-weight>0</search:quality-weight>
    <search:result-decorator apply="href-decorator"
      ns="http://marklogic.com/rest-api/lib/href-decorator"
      at="/MarkLogic/rest-api/lib/rest-result-decorator.xqy"/>
  </search:options>
</search:search>

For more details, see Searching Using Structured Queries in Search Developer's Guide and Specifying Dynamic Query Options with Combined Query.

Analyzing Lexicons and Range Indexes With Aggregate Functions

This section covers the following topics:

Aggregate Function Overview

An aggregate function performs an operation over values or value co-occurrences in lexicons and range indexes. For example, you can use an aggregate function to compute the sum of values in a range index.

There are two kinds of aggregate functions, builtin and user-defined. MarkLogic Server provides builtin aggregate functions for several common analytical functions; see the list in Using Builtin Aggregate Functions in the Search Developer's Guide.

In addition, you can also implement aggregate user-defined functions (UDFs) in C++ and deploy them as native plugins. Aggregate UDFs must be installed before you can use them. For details, see Implementing an Aggregate User-Defined Function in the Application Developer's Guide.

You can use the REST Client API to apply aggregate functions using /values/{name} in two ways:

  • Include one or more <aggregate/> elements (XML) or sub-objects (JSON) in a <values/> or <tuples/> range specification in the query options.
  • Include one or more aggregate request parameters.

If aggregate functions are specified through both query options and request parameters, the request parameter(s) overrides the aggregates specified in the query options.

You can only specify multiple aggregate UDFs from more than one plugin using query options.

You cannot use the REST Client API to apply aggregate UDFs that require additional parameters.

Using Query Options to Apply Aggregate Functions

To specify an aggregate function in query options, include an <aggregate/> element in a <values/> or <tuples/> range specification. If you include multiple <aggregate/> specifications, MarkLogic Server applies all the functions.

For a builtin aggregate, specify the function name in the 'apply' attribute of an <aggregate/> element. For example, the query options below specify the builtin aggregate 'count', which is equivalent to the XQuery builtin cts:count-aggregate.

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
  <values name="speaker">
    <range type="xs:string">
      <element ns="" name="SPEAKER"/>
    </range>
    <aggregate apply="count"/>
  </values>
</options>
JSON
{
  "options": {
    "values": [
      {
        "name": "speaker",
        "range": {
          "type": "xs:string",
          "element": { "ns": "", "name": "SPEAKER" }
        },
        "aggregate": { "apply":"count" }
      }
    ]
  }
}

An aggregate UDF is identified by the function name and a relative path to the plugin that implements the aggregate, as described in Using Aggregate User-Defined Functions in the Search Developer's Guide. Specify the function name with the 'apply' attribute and the plugin path with the 'udf' attribute in an <aggregate/> element or object. For example, the following query options specify a native UDF called 'count' provided by a plugin installed under 'native/sampleplugin':

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
  <values name="speaker">
    <range type="xs:string">
      <element ns="" name="SPEAKER"/>
    </range>
    <aggregate apply="count" udf="native/sampleplugin" />
  </values>
</options>
JSON
{
  "options": {
    "values": [
      {
        "name": "speaker",
        "range": {
          "type": "xs:string",
          "element": { "ns": "", "name": "SPEAKER" }
        },
        "aggregate": { 
          "apply":"count", 
          "udf":"native/sampleplugin"
        }
      }
    ]
  }
}

To use query options to apply an aggregate function:

  1. Define query options that include one or more aggregate definitions, as shown above.
  2. Supply the query options from Step 1 in your request, as described inAdding Query Options to a Request.
  3. Apply the aggregate by sending a GET or POST request to /values/{name} and including the options from Step 2. For example: GET /LATEST/values/speaker?options=index-options.

For details on using query options, see Configuring Query Options.

Using Request Parameters to Apply Aggregate Functions

To analyze lexicon or index values or co-occurrences with builtin aggregate function, make a GET or POST request to the /values/{name} service with a URL of the form:

http://host:port/version/values/name?aggregate=aggr_name&options=options_name

To analyze lexicon or index values or co-occurrences with a previously installed aggregate UDF, make a GET request to the /values/{name} service with a URL of the form:

http://host:port/version/values/name?aggregate=aggr_name&aggregatePath=aggr_path&options=options_name

When constructing the request:

  1. Substitute a named range specification for name in the URL. This must be a range specification (<values/> or <tuples/>) in the query options supplied in the request, or in the default query options if options are omitted.
  2. To use custom query options, specify them using the options parameter and/or the options portion of a combined query in the POST body. For details, see Adding Query Options to a Request.
  3. To use the default query options, omit the options parameter and options portion of a combined query. The default query options should include a range specification matching name.
  4. Specify the aggregate function name using the aggregate parameter.

    The name must be one of the builtin aggregate functions listed in Using Builtin Aggregate Functions the Search Developer's Guide, or a function implemented by the plugin identified by aggregatePath.

  5. If you're applying an aggregate UDF, specify the relative path to the plugin implementing the aggregate function using the aggregatePath parameter.
  6. To constrain the analysis to values in certain fragments, specify a query using the q and/or structuredQuery parameters, or a structured or combined query in the POST body. For details, see Using a Query to Constrain Results.
  7. Specify the result content type (XML or JSON) using the format parameter or the HTTP Accept headers. The default content type is XML. For details, see Controlling Input and Output Content Type.
  8. If you only want the aggregate value in the results, set the view parameter to aggregate. By default, MarkLogic Server returns both the lexicon or index values and the aggregate result.

Additional request parameters are available. For details see the MarkLogic REST API Reference.

When applying an aggregate UDF, the output is dependent on the UDF. Aggregate UDFs return a sequence of items, which can be atomic values or key-value maps. For details, see User-Defined Functions in the Application Developer's Guide and the XQuery builtin function cts:aggregate.

Example: Applying a Builtin Aggregate Function

The following example counts the number of values in the index identified as 'speaker' in 'index-options' (options=index-options). The counted values include only those in fragments containing 'HAMLET' (q=HAMLET). The output should contain only the aggregate result (view=aggregate).

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    'http://localhost:8000/LATEST/values/speaker?options=index-options&aggregate=count&view=aggregate&q=HAMLET'
...
<values-response name="speaker" type="xs:string"
    xmlns="http://marklogic.com/appservices/search"
    xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <aggregate-result name="count">92</aggregate-result>
  <metrics>
    <aggregate-resolution-time>PT0.001108S</aggregate-resolution-time>
    <total-time>PT0.003719S</total-time>
  </metrics>
</values-response>

Example: Applying an Aggregate UDF

This example demonstrates using an aggregate user-defined function to count the number of values in an element range index using /values/{name}. The aggregate UDF is specified via request parameters, as described in Using Request Parameters to Apply Aggregate Functions.

This example assumes the following pre-requisites are already met:

  • A native plugin is installed with the path 'native/sampleplugin'.
  • The plugin implements a 'count' function that counts the values in a range index. That is, a function equivalent to the 'count' builtin aggregate function.
  • Query options are installed with the name 'index-options'.
  • The query options include a range specification named 'speaker' for the SPEAKER element range index.

The following command uses the 'count' aggregate UDF to count the number of values in the SPEAKER index in fragments containing 'HAMLET'. The output only contains the aggregate result (view=aggregate).

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    'http://localhost:8000/LATEST/values/speaker?options=index-options&aggregate=count&aggregatePath=native/sampleplugin&view=aggregate&q=HAMLET'

The use of view=aggregate limits the output to only the aggregate results, as shown below. XML is the default output format. Use the format parameter or the HTTP Accept headers to request JSON output.

Format Example Output
XML
<values-response name="speaker" type="xs:string"
    xmlns="http://marklogic.com/appservices/search"
    xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <aggregate-result name="count">92</aggregate-result>
  <metrics>
    <aggregate-resolution-time>PT0.001514S</aggregate-resolution-time>
    <total-time>PT0.004049S</total-time>
  </metrics>
</values-response>
JSON
{
  "values-response": {
    "name": "speaker",
    "type": "xs:string",
    "aggregate-result": [
      {
        "name": "count",
        "_value": "92"
      }
    ],
    "metrics": {
      "aggregate-resolution-time": "PT0.001336S",
      "total-time": "PT0.004104S"
    }
  }
}

Specifying Dynamic Query Options with Combined Query

A combined query is an XML or JSON wrapper around a string and/or structured query and query options. Use a combined query to specify query options on the fly, without first persisting them as named options using the /config/query service. Combined queries are useful for rapid prototying during development, and for applications that need to modify query options on a per query basis.

To use a combined query, send a POST request to /v1/search or /v1/values/{name} with the combined query in the request body. See the following topics for more details:

Syntax

A combined query can contain a string query, a structured query, query options, or any combination of these. For example, you can create a combined query that contains only query options, only a structured query, or a structured query, string query and query options.

Using certain options in the options section of a combined query requires the rest-admin role or equivalent privileges. For details, see Using Dynamically Defined Query Options.

If and only if you are using a combined query with POST /v1/graphs/sparql to perform a semantic query, you can also include a SPARQL query in a combined query. In this case, the string query, structured query, and options parts of the combined query are used used to perform a search that constrains the result of the SPARQL query. For details, see POST /v1/graphs/sparql.

The following table shows the structure of a combined query. Within the combined query wrapper, the queries and options use the same syntax as when they occur standalone.

Format Combined Query Template
XML
<search xmlns="http://marklogic.com/appservices/search">
  <query>
    <!-- structured query, same syntax as standalone -->
  </query>
  <qtext>your string query</qtext>
  <sparql>your SPARQL query</sparql>
  <options>
    <!-- same syntax as standalone query options -->
  </options>
</search>
JSON
{"search": {
  "query": { structured query, same syntax as standalone },
  "qtext": "your string query here",
  "sparql": "your SPARQL query here",
  "options": { same syntax as standalone query options }
} } 

For details on sub-query and query options syntax, see the following sections of the Search Developer's Guide:

Interaction with Queries in Request Parameters

When making a POST /v1/search or POST /v1/values/{name} request, you can use a combined query in conjunction with the q request parameter. The string query in the request parameter value is AND'd with the sub-query(s) in the combined query.

The following table summarizes the interaction between the q request parameter and the sub-queries of a combined query. Note that the 'queries' in the table are an abstraction rather than actual example queries.

Request Parameter Combined Query Final Query
q=query-rp
<search>
  <query>query-cq</query>
</search>
query-rp AND query-cq
q=query-rp
<search>
  <qtext>query-cq</qtext>
</search>
query-rp AND query-cq
q=query-rp
<search>
  <query>query-cq1</query>
  <qtext>query-cq2</qtext>
</search>
query-rp AND query-cq1 AND query-cq2
none
<search>
  <query>query-cq1</query>
  <qtext>query-cq2</qtext>
</search>
query-cq1 AND query-cq2

Interaction with Persistent Query Options

Dynamic query options supplied in a combined query are merged with persistent and default options that are in effect for the search. If the same non-constraint option is specified in both the combined query and persistent options, the setting in the combined query takes precedence.

Constraints are overridden by name. That is, if the dynamic and persistent options contain a <constraint/> element with the same @name, the definition in the dynamic query options is the one that applies to the query. Two constraints with different name are both merged into the final options.

<options xmlns="http://marklogic.com/appservices/search">
  <fragment-scope>properties</fragment-scope>
  <return-metrics>false</return-metrics>
  <constraint name="same">
    <collection prefix="http://server.com/persistent/"/>
  </constraint>
  <constraint name="not-same">
    <element-query name="title" ns="http://my/namespace" />
  </constraint>
</options>

Further, suppose you submit a POST /v1/search request that uses my-options and includes the following query options in a combined query in the request body:

$ cat body.xml
<search xmlns="http://marklogic.com/appservices/search">
  <options>
    <return-metrics>true</return-metrics>
    <debug>true</debug>
  <constraint name="same">
    <collection prefix="http://server.com/dynamic/"/>
  </constraint>
    <constraint name="different">
      <element-query name="scene" ns="http://my/namespace" />
    </constraint>
  </options>
</search>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X POST \
    -d@./body.xml -H "content-type: application/xml" \
    'http://localhost:8000/LATEST/search?q=TRAGEDY&options=my-options'

The query is evaluated with the following merged options. The persistent options contribute the fragment-scope option and the constraint named not-same. The dynamic options in the combined query contribute the return-metrics and debug options and the constraints named same and different. The return-metrics setting and the constraint named same from my-options are discarded.

<options xmlns="http://marklogic.com/appservices/search">
  <fragment-scope>properties</fragment-scope>
  <return-metrics>true</return-metrics>
  <debug>true</debug>
  <constraint name="same">
    <collection prefix="http://server.com/dynamic/"/>
  </constraint>
    <constraint name="different">
      <element-query name="scene" ns="http://my/namespace" />
    </constraint>
  </options>
  <constraint name="not-same">
    <element-query name="title" ns="http://my/namespace" />
  </constraint>
</options>

Performance Considerations

Using persistent query options usually performs better than using a combined query. In most cases, the difference between the two approaches is slight.

When MarkLogic Server processes a combined query, the per request query options must be parsed and merged with named and default options on every search. When you only use persistent named or default query options, you reduce this overhead.

If your application does not require dynamic per-request query options, you should use the /config/query/{name} service to persist your options under a name and use the options request parameter to associate the options with a simple string, structured, or values query. For details, see Configuring Query Options.

Combined Query Examples

This section includes the following examples:

Example: Overriding Persistent Constraints

The following example uses a bucketed constraint, backed by an element range index, to group items into facets by price. Assume following options that define price range buckets are stored as persistent options using the /config/query/{name} service, as described in Creating or Modifying Query Options.

<options xmlns="http://marklogic.com/appservices/search">
 <constraint name="price" facet="true">
    <range type="xs:int">
    <element ns="" name="price"/>
    <bucket name="under50" ge="0" lt="50">under $50</bucket>
    <bucket name="under100" ge="50" lt="101">$50-$100</bucket>
    <bucket name="over100" ge="101">over $100</bucket>
  </range>
 </constraint>
</options>

The application can use these persistent options to generate a faceted navigation page that allows users to browse by price. You can use dynamic query options to render a page that includes a custom facet from a price range entered by the user. The resulting combined query might look like the following if the user defined a price range of $100-150:

<search xmlns="http://marklogic.com/appservices/search">
  <options>
    <constraint name="price" facet="true">
      <range type="xs:int">
        <element ns="" name="price"/>
        <bucket name="under50" ge="0" lt="50">under $50</bucket>
        <bucket name="under100" ge="50" lt="101">$50-$100</bucket>
        <bucket name="over100" ge="101">over $100</bucket>
        <bucket name="custom" ge="100" lt="151">$100 to $150</bucket>
      </range>
    </constraint>
  </options>
  <query>
    <range-constraint-query>
      <constraint-name>price</constraint-name>
      <value>custom</value>
    </range-constraint-query>
  <query>
</search>
Example: Modifying the Search Response

The following example uses a combined query that contains only query options to enable the return-query option on the fly to explore how a string query is represented as a cts:query after parsing. The return-query setting in the dynamic options overrides any return-query setting in the default options. All other settings in the default options are unchanged and apply to the query evaluation.

$ cat combo-query.xml
<search xmlns="http://marklogic.com/appservices/search">
  <options>
    <return-query>true</return-query>
  </options>
</search>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X POST \
    -d@./combo-query.xml -H "content-type: application/xml" \
    'http://localhost:8000/LATEST/search?q=Horatio NEAR Yorick'
<search:response snippet-format="snippet" total="0" start="1" ...> 
  <search:qtext>Horation NEAR Yorick</search:qtext>
  <search:query>
    <cts:near-query qtextjoin="NEAR" strength="30" ...>
      <cts:word-query qtextref="cts:text">
        <cts:text>Horatio</cts:text>
      </cts:word-query>
      <cts:word-query qtextref="cts:text">
        <cts:text>Yorick</cts:text>
      </cts:word-query>
    </cts:near-query>
  </search:query>
  ...
</search:response>

For details, see Interaction with Persistent Query Options.

Querying Triples

You can query semantic data in the database by sending a GET request to the /graphs/sparql service with a URL of the following form:

http://host:port/version/graphs/sparql?query=sparql-query

Optionally, you can define the RDF Dataset over which to query by specifying one or more graph URIs using the named-graph-uri and/or default-graph-uri request parameters:

http://host:port/version/graphs/sparql?query=sparql-query&named-graph-uri=graph-uri&default-graph-uri=graph-uri

You can also specify the dataset within the query. If you specify a dataset in both the request parameters and the query, the dataset defined with named-graph-uri and default-graph-uri takes precedence. If no dataset is defined in the request parameters or in the query, the dataset includes all triples, regardless of graph.

The SPARQL query in the query request parameter must be URL-encoded.

You can also put the query in the body of a POST request to /graphs/sparql. As with the GET request, define the RDF Dataset using named-graph-uri and/or default-graph-uri. For example, make a POST request with a URL of the following form:

http://host:port/version/graphs/sparql?named-graph-uri=graph-uri&default-graph-uri=graph-uri

You must enable the collection lexicon on your database before you can use the semantics REST services or use the GRAPH '?g' construct in a SPARQL query.

When you use POST, the request body can contain either a SPARQL query or a combined query that includes a SPARQL query. For details, see POST /v1/graphs.

For details, see Configuring the Database to Work with Triples and Semantic Queries in the Semantics Developer's Guide.

Configuring Query Options

Use the /config/query resources to install, list, and manage sets of query options. Use the options request parameter to apply installed query options to requests to /search, /keyvalue, and /values. This section covers the following topics:

Controlling Queries With Options

You can use persistent or dynamic query options to customize your queries. Query options provide capabilities such as:

  • define word, value and element constraints
  • define lexicon and range index specifications
  • control search characteristics such as case sensitivity and ordering
  • extend the search grammar
  • customize query results including pagination, snippeting, and filtering

For details on these and other options, see Appendix: Query Options Reference in the Search Developer's Guide.

MarkLogic Server comes configured with default query options. You can extend and modify the default options using /config/query/default.

Use the options request parameter to the /search, /values, and /keyvalue services to customize your queries. Query options provide capabilities such as:

You can also create custom persistent or dynamic query options. Persistent query options are named, pre-defined query options that you install using the /config/query/{name}. Dynamic query options are per-request, transient options defined in a combined query. For details, see Creating or Modifying Query Options and Specifying Dynamic Query Options with Combined Query

Once you install query options under a name, you can apply them to a /search, /keyvalue, or /values request using the options request parameter. The following example searches for the word 'julius', using the query options named 'my-options':

http://localhost:8000/LATEST/search?q=julius&options=my-options

Adding Query Options to a Request

You can customize a query with query options in the following ways:

  • Use the options request parameter of a GET request to /search, /keyvalue, or /values/{name} to supply the name of pre-installed persistent options.
  • Use the options parameter of a POST request to /search or /values/{name} to supply the name of pre-installed persistent options.
  • Use the options element (XML) or sub-object (JSON) of a combined query passed in the body of a POST request to /search or /values/{name} to supply dynamic options.

Pre-installed, persistent query options usually provide better performance. Using dynamic query options introduces option parsing and merging overhead to every query.

Persistent and dynamic query options can be specified in either XML or JSON. Persistent options must be installed before you can use them; for details, see Creating or Modifying Query Options.

Dynamic options are only usable with services that support POSTing a combined query in the request body. Methods that support combined query allow you to specify persistent and dynamic options in the same request. Where both are present, they are merged; in case of a conflict, a dynamic option setting overrides a persistent option setting. For details, see Specifying Dynamic Query Options with Combined Query.

Creating or Modifying Query Options

To install or modify named persistent query options, send a PUT or POST request to the /config/query service with a URL of the form:

http://host:port/version/config/query/name

When constructing the request:

  1. Set the name portion of the URL to a unique name for these options, or to default. Use the name to identify the query options in subsequent request, as described in Controlling Queries With Options.
  2. Place the XML or JSON option data in the request body.

    For syntax details, see Appendix: Query Options Reference in the Search Developer's Guide.

  3. Specify the MIME type of the body content in the format parameter or the HTTP Content-type header, as described in Controlling Input and Output Content Type. You may only send XML or JSON. The default format is XML.

When choosing the HTTP verb, consider the following:

  • To install new query options, use either PUT or POST.
  • To replace the named query options, use PUT.
  • To add options to the named query options, use POST. Any options not included in the payload remain unchanged. New options are added.

If query option validation is enabled, the request will fail if it results in invalid query options. If the request fails, the options are unchanged. Option validation is enabled by default. You can disable option validation using the validate-options instance configuration property. For details, see Configuring Instance Properties.

The following example installs query options named 'title-only' that define a search constraint named 'title'. The constraint limits queries to terms appearing in a TITLE element. The query options are then used to find occurrences of 'julius' in TITLE elements:

$ cat bill-options.txt
<options xmlns="http://marklogic.com/appservices/search">
  <constraint name="title">
    <word>
      <element ns="" name="TITLE" />
    </word>
  </constraint>
</options>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -T './bill-options.txt' \
    -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/config/query/title-only
...
HTTP/1.1 204 Content Updated
$ curl --anyauth --user user:password -X GET \
  'http://localhost:8000/LATEST/search?q=title:julius&options=title-only'
...
<search:response total="1" start="1" page-length="10" 
   xmlns="" xmlns:search="http://marklogic.com/appservices/search">
    ...
</search:response>

To add case-sensitivity to the query options installed above, this example sends a POST request to /config/query. The body content type, JSON, is given via the Content-type header.

$ cat add-cs.json
{
  "options":
    { "term":
      { "term-option":"case-sensitive" }
    }
}
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X POST -d@'add-cs.json' \
    -H "Content-type: application/json" \
    http://localhost:8000/LATEST/config/query/title-only
...
HTTP/1.1 201 Content Created

To confirm the change, the modified option is fetched using a GET request:

$ curl --anyauth --user user:password -X GET \
    http://localhost:8000/LATEST/config/query/title-only
...
<options xmlns="http://marklogic.com/appservices/search">
  <constraint name="title">
    <word>
      <element ns="" name="TITLE"/>
    </word>
  </constraint>
  <term>
    <term-option>case-sensitive</term-option>
  </term>
</options>

Creating or Modifying One Option

To add or modify just one setting in a set of query options, send a PUT or POST request to the /config/query service with a URL of the form:

http://host:port/version/config/query/name/option_name

When constructing the request:

  1. Set the name portion of the URL to the name of the enclosing query options, or to default.
  2. Set the option_name portion of the URL to a query option name, such as constraint or term.
  3. Place the XML or JSON option data in the request body. The data should be an options node (XML) or map (JSON) that includes the option named in the URL.
  4. Specify the content type of the body in the format parameter or the HTTP Content-type header, as described in Controlling Input and Output Content Type. You may only send XML or JSON. The default format is XML.

The option_name portion of the URL must be the name of an option that can appear as an immediate child of a query options XML node or JSON object. Finer grained access is not supported. For details on query options names and structure, see Appendix: Query Options Reference in the Search Developer's Guide.

When choosing the HTTP verb, consider the following:

  • To add a new option to the set, use either PUT or POST.
  • To replace all existing options of the same name, use PUT.
  • To add a new occurrence of an existing option that can appear multiple times, use POST. For example, query options can contain multiple <constraint/> elements.

If query option validation is enabled, the request will fail if it results in invalid query options. If the request fails, the options are unchanged. Option validation is enabled by default. You can disable option validation using the validate-options instance configuration property. For details, see Configuring Instance Properties.

Checking Index Availability

Some query options require the database configuration to include supporting indexes. For example, if your query options contain a range constraint, then you can only use those options on a database whose configuration includes a corresponding range index.

You can use the /config/indexes service to compare query options to the database configuration and get a report on whether or not all required indexes are present. For missing indexes, the report includes information to help create the missing index. You can either check all query options configurations, or a particular one (by name).

To check all query options configurations, send a GET request to the /config/indexes service of the form:

http://host:port/version/config/indexes

To check a specific query options configuration, send a GET request to the /config/indexes/{name} service of the form:

http://host:port/version/config/indexes/name

Where name is the name under which the options were installed using the /config/query/{name} service, as described in Creating or Modifying Query Options.

You can request an index report in XML, JSON, or HTML, using either the format request parameter or the HTTP Accept headers. The HTML report is a user-friendly report that contains more details.

For example, suppose the following query options are installed under the name 'tuples'. These options require 2 range indexes: An element range index on <SPEAKER/> and a path range index on the XPath expression /PLAY/ACT/SCENE/TITLE:

<options xmlns="http://marklogic.com/appservices/search">
  <tuples name="speaker-title">
    <range type="xs:string">
      <element ns="" name="SPEAKER"/>
    </range>
    <range type="xs:string">
      <path-index>/PLAY/ACT/SCENE/TITLE</path-index>
    </range>
  </tuples>
</options>

The following command requests an index check check report in XML for the 'tuples' options. Use the format parameter or the Accept headers to request a different report format.

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/xml" \
    http://localhost:8000/LATEST/config/indexes/tuples

The table below shows the generated report. Notice that the report indicates that the index configuration is not complete, and shows which query option is not complete. You can use the path-index value to create the required path range index.

Format Example Output
XML
<rapi:index-summaries xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:index-count>1</rapi:index-count>
  <rapi:complete>false</rapi:complete>
  <rapi:index-summary>
    <rapi:name>/v1/config/query/tuples</rapi:name>
    <rapi:complete>false</rapi:complete>
    <range type="xs:string"
        xmlns="http://marklogic.com/appservices/search">
      <path-index>/PLAY/ACT/SCENE/TITLE</path-index>
    </range>
  </rapi:index-summary>
</rapi:index-summaries>
JSON
{
  "index-summaries": {
    "index-summary": [
      {
        "name": "\/v1\/config\/query\/tuples",
        "complete": "false",
        "range": {
          "type": "xs:string",
          "path-index": "\/PLAY\/ACT\/SCENE\/TITLE"
        }
      }
    ],
    "index-count": "1",
    "complete": "false"
  }
}

Retrieving Options

To retrieve previously installed persistent query options, send a GET request to the /config/query service with a URL of the form:

http://host:port/version/config/query/name

Where name is the name of the query options.

MarkLogic Server responds with the contents of the named query options, as XML or JSON. XML is the default format. Use the Accept header or format request parameter to select the output content type. For details, see Controlling Input and Output Content Type.

You can also retrieve the settings for a specific option within the query options by sending a GET request with a URL of the form:

http://host:port/version/config/query/name/option_name

Where option_name is an option name. For details, see Appendix: Query Options Reference in the Search Developer's Guide.

As when retrieving a whole set, results can be requested as XML or JSON. Use the Accept header or format request parameter to select the output content type. For details, see Controlling Input and Output Content Type. No version information is returned when examining only one option.

The following example retrieves the contents of the query options called 'title-only':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    http://localhost:8000/LATEST/config/query/title-only
...
<options xmlns="http://marklogic.com/appservices/search">
  <constraint name="title">
    <word>
      <element ns="" name="TITLE"/>
    </word>
  </constraint>
  <term>
    <term-option>case-sensitive</term-option>
  </term>
</options>

The following example retrieves only the term option of the 'title-only' query options:

$ curl --anyauth --user user:password -X GET \
    http://localhost:8000/LATEST/config/query/title-only/term
...
<options xmlns="http://marklogic.com/appservices/search">
  <term>
    <term-option>case-sensitive</term-option>
  </term>
</options>

Retrieving a List of Installed Query Options

To retrieve a list of the names of all installed query options, send a GET request to /config/query with a URL of the form:

http://host:port/version/config/query

MarkLogic Server responds with a list of names in XML or JSON. XML is the default format. Use the Accept header or format request parameter to select the output content type. For details, see Controlling Input and Output Content Type.

If there are no custom query options installed, MarkLogic Server responds with an empty XML <options> node or JSON array.

The following example retrieves the list of named query options as XML. The results show 2 sets of query options, named 'title-only' and 'play-type'.

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    http://localhost:8000/LATEST/config/query
...
<rapi:query-options xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:options>
    <rapi:name>title-only</rapi:name>
    <rapi:uri>/v1/config/query/title-only</rapi:uri>
  </rapi:options>
  <rapi:options>
    <rapi:name>play-type</rapi:name>
    <rapi:uri>/v1/config/query/play-type</rapi:uri>
  </rapi:options>
</rapi:query-options>

The following example requests the same information as JSON, using the format request parameter:

$ curl --anyauth --user user:password -X GET \
    http://localhost:8000/LATEST/config/query
...
[
  {"name":"title-only","uri":"/v1/config/query/title-only"}
  {"name":"play-type","uri":"/v1/config/query/play-type"}
]

Removing Query Options

You can remove a single setting, one set of query options, or all query options, as described in the following topics:

Removing Query Options

To remove the query options installed under a particular name, send a DELETE request to the /config/query service with a URL of the form:

http://host:port/version/config/query/name

Where name is the name of the query options to remove. MarkLogic Server responds with 204 if the option set is successfully deleted.

Removing a Single Option

To remove a specific setting in a set of query options, send a DELETE request to the /config/query service with a URL of the form:

http://host:port/version/config/query/name/option_name

Where option_name is the name of the option to remove and name is the name of the containing query options. MarkLogic Server responds with 204 if the option is successfully deleted.

The option_name portion of the URL must be the name of an option that can appear as an immediate child of a query options XML node or JSON object. Finer grained access is not supported. All options with this name are removed. For example, if the query options named 'my-options' contain multiple <constraint/> options, then the following request removes all of them:

DELETE http://localhost:8000/LATEST/config/query/my-options/constraint

For details on query options names and structure, see Appendix: Query Options Reference in the Search Developer's Guide.

If query option validation is enabled, the request will fail if it results in invalid query options. If the request fails, the options are unchanged. Option validation is enabled by default. You can disable option validation using the validate-options instance configuration property. For details, see Configuring Instance Properties.

For details on query option names, see Appendix: Query Options Reference in the Search Developer's Guide.

Removing All Named Query Options

To remove all named query options from a REST Client API instance, send a DELETE request to the /config/query service with a URL of the form:

http://host:port/version/config/query

MarkLogic Server responds with 204 if the options successfully deleted.

Using Namespace Bindings

This sections covers the following topics:

When Do You Need a Namespace Binding

Use the /config/namespaces service to define namespace prefixes that you can use in your queries. For example, to specify an element with QName theNS:theElement in a /keyvalue search, you might refer to it as follows:

http://host:port/version/keyvalue?element=theNS:theElement&value=foo

There is no way to dynamically specify the namespace in your request, so you must pre-define the namespace prefix binding using /config/namespaces or /config/namespace/{name}.

Creating or Updating a Namespace Binding

This section describes how to create a single namespace binding. You can also define multiple bindings in a single request; for details, see Creating or Updating Multiple Namespace Bindings.

To define a namespace binding, send a PUT request to the /config/namespaces/{name} service with a URL of the form:

http://host:port/version/config/namespaces/name

Where name is the namespace prefix you want to define. If a binding already exists for this prefix, it is replaced.

When constructing your request:

  1. Set the name portion of the URL to the desired namespace prefix.
  2. Place the XML or JSON binding definition in the request body. See the table below.
  3. Specify the content type of the body in the format parameter or the HTTP Content-type header, as described in Controlling Input and Output Content Type. You may only send XML or JSON. The default format is XML.

The request body must have the following form:

Format Body
XML
<namespace xmlns="http://marklogic.com/rest-api">
  <prefix>the_prefix</prefix>
  <uri>the_uri</uri>
</namespace>
JSON
{
  "prefix" : "the_prefix",
  "uri" : "the_uri"
}

The following example binds the prefix 'bill' to the namespace URI 'http://marklogic.com/examples/shakespeare':

$ cat ns-binding.xml
<namespace xmlns="http://marklogic.com/rest-api">
  <prefix>bill</prefix>
  <uri>http://marklogic.com/examples/shakespeare</uri>
</namespace>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT -d@./ns-binding.xml \
    -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/config/namespaces/bill

You can examine the binding by sending a GET request to /config/namespaces/{name} or to /config/namespaces. The latter lists all bindings. For example:

$  curl --anyauth --user user:password -X GET \
    http://localhost:8000/LATEST/config/namespaces/bill
<rapi:namespace xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:prefix>bill</rapi:prefix>
  <rapi:uri>http://marklogic.com/examples/shakespeare</rapi:uri>
</rapi:namespace>

To get the equivalent output as JSON, use the format parameter or specify application/json in the HTTP Accept header. For example:

$  curl --anyauth --user user:password -X GET \
    -H "Accept: application/json" \
    http://localhost:8000/LATEST/config/namespaces/bill
{
  "prefix":"bill",
  "uri":"http://marklogic.com/examples/shakespeare"
}

Creating or Updating Multiple Namespace Bindings

This section describes how to define multiple namespace bindings with a single request. You can also define a specific single binding; for details, see Creating or Updating a Namespace Binding.

To define multiple namespace bindings, send a PUT or POST request to the /config/namespaces service with a URL of the form:

http://host:port/version/config/namespaces

When constructing your request:

  1. Choose PUT to replace all bindings with those the request body. Choose POST to append to existing bindings.
  2. Place the XML or JSON binding definitions in the request body. See the table below.
  3. Specify the content type of the body in the format parameter or the HTTP Content-type header, as described in Controlling Input and Output Content Type. You may only send XML or JSON. The default format is XML.

If you use POST and a binding already exists for one defined in the request body, MarkLogic Server returns status 400 (Bad Request).

The request body must have the following form:

Format Body
XML
<namespace-bindings xmlns="http://marklogic.com/rest-api">
  <namespace>
    <prefix>a_prefix</prefix>
    <uri>a_namespace_uri</uri>
  </namespace>
  <namespace>
    <prefix>another_prefix</prefix>
    <uri>another_namespace_uri</uri>
  </namespace>
</namespace-bindings>
JSON
{
  "namespace-bindings": [
    {
      "prefix": "a_prefix",
      "uri": "a_namespace_uri"
    },
    {
      "prefix": "another_prefix",
      "uri": "another_namespace_uri"
    }
  ]
}

The following example binds the prefix 'one' to the namespace URI 'http://marklogic.com/examples/one' and the prefix 'two' to the namespace URI 'http://marklogic.com/examples/two':

$ cat ns-bindings.xml
<namespace-bindings xmlns="http://marklogic.com/rest-api">
  <namespace>
    <prefix>one</prefix>
    <uri>http://marklogic.com/examples/one</uri>
  </namespace>
  <namespace>
    <prefix>two</prefix>
    <uri>http://marklogic.com/examples/two</uri>
  </namespace>
</namespace-bindings>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT -d@./ns-bindings.xml \
    -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/config/namespaces

The following example creates equivalent bindings using JSON input:

$ cat ns-bindings.json
{
  "namespace-bindings": [
    {
      "prefix": "one",
      "uri": "http:\/\/marklogic.com\/examples\/one"
    },
    {
      "prefix": "two",
      "uri": "http:\/\/marklogic.com\/examples\/two"
    }
  ]
}
$ curl --anyauth --user user:password -X PUT -d@./ns-bindings.json \
    -H "Content-type: application/json" \
    http://localhost:8000/LATEST/config/namespaces

You can examine the binding by sending a GET request to /config/namespaces/{name} or to /config/namespaces.

Listing Available Namespace Bindings

To list all available namespace bindings, send a GET request to /config/namespaces with a URL of the form:

http://host:port/version/config/namespaces

To retrieve the binding for a single namespace prefix, send a GET request to /config/namespaces/{name} with a URL of the form:

http://host:port/version/config/namespaces/name

Where name is a bound namespace prefix.

You can request output as either XML or JSON. Use the Accept header or format request parameter to select the output content type. For details, see Controlling Input and Output Content Type.

The following example command requests all namespace bindings, as XML:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/xml" \
    http://localhost:8000/LATEST/config/namespaces

The output from the command is shown in the table below, assuming two namespace bindings are installed, for the prefixes 'one' and 'two'.

Format Example Output
XML
<rapi:namespace-bindings
    xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:namespace>
    <rapi:prefix>one</rapi:prefix>
    <rapi:uri>http://marklogic.com/examples/one</rapi:uri>
  </rapi:namespace>
  <rapi:namespace>
    <rapi:prefix>two</rapi:prefix>
    <rapi:uri>http://marklogic.com/examples/two</rapi:uri>
  </rapi:namespace>
</rapi:namespace-bindings>
JSON
{
  "namespace-bindings": [
    {
      "prefix": "one",
      "uri": "http://marklogic.com/examples/one"
    },
    {
      "prefix": "two",
      "uri": "http://marklogic.com/examples/two"
    }
  ]
}

When you use GET /config/namespaces/{name}, the output is similar, but without the namespace-bindings wrapper. For example to retrieve the binding for the 'one' prefix:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/xml" \
    http://localhost:8000/LATEST/config/namespaces/one
<rapi:namespace xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:prefix>one</rapi:prefix>
  <rapi:uri>http://marklogic.com/examples/one</rapi:uri>
</rapi:namespace>

The following command retrieves the same information as JSON:

$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/json" \
    http://localhost:8000/LATEST/config/namespaces/one
{"prefix":"one","uri":"http://marklogic.com/examples/one"}

Deleting Namespace Bindings

To remove all namespace bindings, send a DELETE request to /config/namespaces with a URL of the form:

http://host:port/version/config/namespaces

To remove a specific binding, send a DELETE request to /config/namespaces/{name} with a URL of the form:

http://host:port/version/config/namespaces/name

Where name is a bound namespace prefix. If no binding exists for name, MarkLogic Server returns status 404 (Not Found).

The following example deletes just the binding for the prefix 'one':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X DELETE \
    http://localhost:8000/LATEST/config/namespaces/one

Generating Search Facets

The MarkLogic Server Search API enables you to expose search facets in your application. Facets enable users to filter search results by narrowing down the search criteria. To learn more about facets, see Constrained Searches and Faceted Navigation in the Search Developer's Guide.

To generate facet information in search results, use query options that include constraints that support facets, such as collection and element constraints. You can also define custom constraints; see Creating a Custom Constraint in the Search Developer's Guide.

The following example returns facet information about play types and enables searching by the facet 'type', assuming the documents in the database have been added to the collections /play-type/Comedy, /play-type/Tragedy, and /play-type/History. For more examples, see Constraint Options in the Search Developer's Guide.

This example uses a collection constraint. To search on a collection facet, you must enable the 'collection lexicon' database configuration parameter in the Admin Interface.

To enable the play type facet, create query options that define a collection constraint:

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
  <constraint name="type">
    <collection prefix="/play-type/"/>
  </constraint>
  <return-facets>true</return-facets>
</options>
JSON
{
  "options": {
    "constraint": [
      {
        "name": "type",
        "collection": { "prefix": "\/play-type\/" }
      } ],
    "return-facets": true
  }
}

Install the options with the name 'facet-options':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    -d@"./facet-options.xml" -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/config/query/facet-options

If you query /search or /keyvalue using 'index-options' as the query options, facet results are included in the search results. For example, the following query finds all occurrences of 'castle' and the search response includes a count of the matches for each play type:

$ curl --anyauth --user user:password -X GET \
  'http://localhost:8000/LATEST/search?options=facet-options&q=castle'
...
<search:response snippet-format="snippet" total="88" ...>
  <search:result />
  <search:facet name="type" type="collection">
    <search:facet-value name="Comedy" count="5">
      Comedy
    </search:facet-value>
    <search:facet-value name="History" count="35">
      History
    </search:facet-value>
    <search:facet-value name="Tragedy" count="47">
      Tragedy
    </search:facet-value>
  </search:facet>
...
</search:response>

To get the equivalent results as JSON, use format=json or include application/json in the HTTP Accept header. For example:

$ curl --anyauth --user user:password -X GET \
  -H "Accept: application/json" \
  'http://localhost:8000/LATEST/search?options=facet-options&q=castle'
...
{
  "snippet-format": "snippet",
  "total": 88,
  "start": 1,
  "page-length": 10,
  "results": [...],
  "facets": {
    "type": {
      "type": "collection",
      "facetValues": [
        {
          "name": "Comedy",
          "count": 5
        },
        {
          "name": "History",
          "count": 35
        },
        {
          "name": "Tragedy",
          "count": 47
        }
      ]
    }
  },
  "qtext": "castle",
  ...
}

You can query by facet by including a facet-name:facet-value search term. The following command matches occurrences of 'castle' in plays of type Comedy:

$ curl --anyauth --user user:password -X GET \
    'http://localhost:8000/LATEST/search?options=facet-options&q=castle type:Comedy'

Paginating Results

When you query the database, you can paginate the query results using the start and pageLength request parameters. Use start to specify the index of the first result to return, and pageLength to control the number results to return.

By default, queries return the first 10 results. That is, the default start position is 1 and the default page length is 10. You can fetch successive non-overlapping pages of results by incrementing the start position by the page length in each call.

For more information, see the Search Developer's Guide and Fast Pagination and Unfiltered Searches in the Scalability, Availability, and Failover Guide.

The following example command fetches the first 5 results matching 'castle'. Notice that the search response includes the total number of matches and each search:result includes an index.

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
   'http://localhost:8000/LATEST/search?q=castle&start=1&pageLength=5'
...
<search:response snippet-format="snippet" total="88"    start="1" page-length="5" ...>
  <search:result index="1" uri="/shakespeare/plays/hen_vi_2.xml" .../>
  <search:result index="2" uri="/shakespeare/plays/rich_ii.xml" .../>
  <search:result index="3" uri="/shakespeare/plays/macbeth.xml" .../>
  <search:result index="4" uri="/shakespeare/plays/hen_vi_3.xml" .../>
  <search:result index="5" uri="/shakespeare/plays/othello.xml" .../>
...
</search:response>

To fetch the next 5 results, increment start by 5:

$ curl --anyauth --user user:password -X GET \
   'http://localhost:8000/LATEST/search?q=castle&start=6&pageLength=5'
...
<search:response snippet-format="snippet" total="88" 
    start="1" page-length="5" ...>
  <search:result index="6" uri="/shakespeare/plays/lear.xml" .../>
  <search:result index="7" uri="/shakespeare/plays/hamlet.xml" .../>
  <search:result index="8" uri="/shakespeare/plays/rich_iii.xml" .../>
  <search:result index="9" uri="/shakespeare/plays/hen_v.xml" .../>
  <search:result index="10" uri="/shakespeare/plays/m_wives.xml" .../>
...
</search:response>

Customizing Search Results

This section covers several features that the REST Client API provides for search result customization.

  • The transform-results query option enables you to fine tune the default snippets. For example, you can control how many matches to return.
  • Custom snippet extensions identified in query options enable you to modify the contents of snippets returned in the search:match portion of a search:response.
  • Tranform functions enable you to completely change the structure returned by a search or values query. Your transform takes a search:response or a matched document as input and produces a result document.
  • The extract-document-data query option enables you to return a portion of each matching document. For details, see Extracting a Portion of Matching Documents in the Search Developer's Guide.

The following topics cover these features:

Customizing Search Snippets

Search results usually include portions of matching documents with the search matches highlighted, perhaps with some text showing the context of the search matches. These search result pieces are known as snippets. MarkLogic Server has a default search snippet format, but you can customize the snippet format by either modifying the configuration of the default snippet function or creating a custom snippet extension.

This section covers the following topics:

Customizing the Default Search Snippets

MarkLogic Server creates snippets by applying a default transformation to search results. You can modify the results of the default transformation using the transform-results query option, as described in Modifying Your Snippet Results in the Search Developer's Guide. To use this feature with the REST API:

  1. Create query options that contain a transform-results XML element or JSON object.
  2. Install these query options under a name or as the default options using the /config/query service, or include them directly in a combined query. For details, see Configuring Query Options.
  3. If you installed the configuration as named query options, apply the options to your query by supplying the name in the options request parameter. For details, see Controlling Queries With Options.
  4. If you installed the configuration as default options, they will be automatically applied to search results returned by any query that does not use named query options.

The following example searches for the term 'hamlet' in a collection of Shakespeare plays using the default snippet options. The search results include four snippets in each match.

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/xml" \
    http://localhost:8000/LATEST/search?q=hamlet
<search:response snippet-format="snippet" total="20" ...>
  <search:result index="1" uri="/shakespeare/plays/hamlet.xml" ...>     <search:snippet>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/TITLE">The Tragedy of <search:highlight>Hamlet</search:highlight>, Prince of Denmark
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/PERSONAE/PERSONA[2]"><search:highlight>HAMLET</search:highlight>, son to the late, and nephew to the present king.
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/PERSONAE/PERSONA[4]">HORATIO, friend to <search:highlight>Hamlet</search:highlight>.
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/PERSONAE/PERSONA[16]">GERTRUDE, queen of Denmark, and mother to <search:highlight>Hamlet</search:highlight>. 
      </search:match>
    </search:snippet>
  </search:result>
...
</search:response>

If you request results as JSON, you get snippets of the following form instead of the XML shown above.

...
"matches": [
  {
    "path":
      "fn:doc(\"\/shakespeare\/plays\/hamlet.xml\")\/PLAY\/TITLE",
    "match-text": [
      "The Tragedy of ",
      { "highlight": "Hamlet" },
      ", Prince of Denmark"
    ]
  },
]
...

Notice that the resulting snippets are from the <TITLE/> and <PERSONAE/> elements. If you apply the options below to the search instead, at most three snippets are returned and only matches in <LINE/> elements are included.

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
  <transform-results apply="snippet">
    <max-matches>3</max-matches>
    <preferred-matches>
      <element ns="" name="LINE" />
    </preferred-matches>
  </transform-results>
  <search-option>filtered</search-option>
</options>
JSON
{
  "options": {
    "transform-results": {
      "apply": "snippet",
      "max-matches": 3,
      "preferred-matches": {
        "element": [{
          "ns": "",
          "name": "LINE"
        }]
      }
    },
    "search-option": ["filtered"]
  }
}

This example installs the above XML options and applies them to the same query:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    -d @./transform-results.xml \
    -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/config/query/snippet-lines
$ curl --anyauth --user user:password -X GET \
   -H "Accept: application/xml" \
   http://localhost:8000/LATEST/search?q=hamlet&options=snippet-lines<search:response snippet-format="snippet" total="20" ...>
  <search:result index="1" uri="/shakespeare/plays/hamlet.xml" ...>
    <search:snippet>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[48]/LINE[6]">Dared to the combat; in which our valiant <search:highlight>Hamlet</search:highlight>--
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[48]/LINE[17]">His fell to <search:highlight>Hamlet</search:highlight>. Now, sir, young Fortinbras,
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[59]/LINE[6]">Unto young <search:highlight>Hamlet</search:highlight>; for, upon my life,
      </search:match>
    </search:snippet>
  </search:result>
...
</search:response>

To install the equivalent JSON options, set the Content-type header to application/json. For example:

$ curl --anyauth --user user:password -X PUT \
    -d @./transform-results.json \
    -H "Content-type: application/json" \
    http://localhost:8000/LATEST/config/query/snippet-lines

To receive JSON search results, set the Accept header to application/json or use the format request parameter. For example:

$ curl --anyauth --user user:password -X GET \
   -H "Accept: application/json" \
   'http://localhost:8000/LATEST/search?q=hamlet&options=snippet-lines'
{
  "snippet-format": "snippet",
  "total": 20,
  "start": 1,
  "page-length": 10,
  "results": [
    {
      "index": 1,
      "uri": "\/shakespeare\/plays\/hamlet.xml",
      "path": "fn:doc(\"\/shakespeare\/plays\/hamlet.xml\")",
      "score": 158720,
      "confidence": 0.80079,
      "fitness": 1,
      "matches": [
        {
          "path": "fn:doc(\"\/shakespeare\/plays\/hamlet.xml\")\/PLAY\/ACT[1]\/SCENE[1]\/SPEECH[48]\/LINE[6]",
          "match-text": [
            "Dared to the combat; in which our valiant ",
            {
              "highlight": "Hamlet"
            },
            "--"
          ]
        },
        {
          "path": "fn:doc(\"\/shakespeare\/plays\/hamlet.xml\")\/PLAY\/ACT[1]\/SCENE[1]\/SPEECH[48]\/LINE[17]",
          "match-text": [
            "His fell to ",
            {
              "highlight": "Hamlet"
            },
            ". Now, sir, young Fortinbras,"
          ]
        },
        {
          "path": "fn:doc(\"\/shakespeare\/plays\/hamlet.xml\")\/PLAY\/ACT[1]\/SCENE[1]\/SPEECH[59]\/LINE[6]",
          "match-text": [
            "Unto young ",
            {
              "highlight": "Hamlet"
            },
            "; for, upon my life,"
          ]
        }
      ]
    }
  ],
  ...
}

Alternatively, you can use a combined query that contains your options, instead of installing persistent options. For example, you can pass the following query in the body of a POST /v1/search request. For details, see Generating a Combined Query from a QBE.

<search xmlns="http://marklogic.com/appservices/search">
  <qtext>hamlet</qtext>
  <options>
    <transform-results apply="snippet">
      <max-matches>3</max-matches>
      <preferred-elements>
        <element ns="" name="LINE" />
      </preferred-elements>
    </transform-results>
    <search-option>filtered</search-option>
  </options>
</search>
Creating Your Own Snippet Extension

If the tranform-results query options with the default snippet format does not meet the needs of your application, you can create a custom snippet transformation function, as described in Specifying Your Own Code in transform-results in the Search Developer's Guide.

Install custom snippet transformations using the /ext service to load them into the Modules database associated with your REST API instance. Use the apply, at, and ns attributes of the transform-results query option to specify your custom module.

To create and use a custom snippeting function:

  1. Create an XQuery library module that implements your custom snippet function, as described in Specifying Your Own Code in transform-results in the Search Developer's Guide.
  2. Install your module in the modules database of your REST API instance using the /ext service, similar to installing a dependent library for a resource service extension. For details, see Installing or Updating an Asset.
  3. Create and install query options that include a transform-results option that uses your custom snippeting function. For details, see Search Customization Via Options and Extensions in the Search Developer's Guide.
  4. Apply the options to your query by supplying the name of the options with the options request parameter. For details, see Controlling Queries With Options.

If your application requires non-trivial JSON snippet customization, your snippet function must generate the XML representation of the desired JSON. For details, see Generating Custom JSON Snippets.

The following example builds on the example from Customizing the Default Search Snippets. The example searches for the term 'hamlet' in a collection of Shakespeare plays using the default snippet format with the custom options that extract the first three matches in lines of dialog. These options are installed under the name 'snippet-lines'. With the default snippet transformation function, the search produces the following results:

# Install the options
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    -d @./transform-results.xml \
    -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/config/query/snippet-lines
# Use the options in a query
$ curl --anyauth --user user:password -X GET \
   -H "Accept: application/xml" \
   'http://localhost:8000/LATEST/search?q=hamlet&options=snippet-lines'
<search:response snippet-format="snippet" total="20" ...>
  <search:result index="1" uri="/shakespeare/plays/hamlet.xml" ...>
    <search:snippet>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[48]/LINE[6]">Dared to the combat; in which our valiant <search:highlight>Hamlet</search:highlight>--
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[48]/LINE[17]">His fell to <search:highlight>Hamlet</search:highlight>. Now, sir, young Fortinbras,
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[59]/LINE[6]">Unto young <search:highlight>Hamlet</search:highlight>; for, upon my life,
      </search:match>
    </search:snippet>
  </search:result>
...
</search:response>

The custom snippeting function below returns the act, scene, speech and line number in each match, instead of the text surrounding the search term.

xquery version "1.0-ml";

module namespace example = "http://marklogic.com/example";

import module namespace search =
  "http://marklogic.com/appservices/search"
  at "/MarkLogic/appservices/search/search.xqy";


declare function example:snippet(
   $result as node(),
   $ctsquery as schema-element(cts:query),
   $options as element(search:transform-results)?
) as element(search:snippet)
{
  let $default-snippet := search:snippet($result, $ctsquery, $options)
  return element
    { fn:QName(fn:namespace-uri($default-snippet),
               fn:name($default-snippet)) }
    { $default-snippet/@*,
      for $child in $default-snippet/node()
      return
        if ($child instance of element(search:match))
        then element {fn:QName(fn:namespace-uri($child),
                      fn:name($child)) } {
          $child/@*,
          let $parts :=
            fn:tokenize(
              fn:substring-after($child/@path, "/PLAY/ACT["), "\[")
          let $location := fn:concat(
            "Act ", fn:substring-before($parts[1], "]"),
            ", Scene ", fn:substring-before($parts[2], "]"),
            ", Speech ", fn:substring-before($parts[3], "]"),
            ", Line ", fn:substring-before($parts[4], "]"))
          return text {$location}
        }
        else $child
    }
};

If the above module is installed in the instance modules database with the URI /ext/my.domain/my-snippets.xqy, then we can use the following options to generate custom snippets by supplying the localname of the custom function (snippet) in apply, the namespace of the module (http://marklogic.com/example) in ns, and the URI of the module in the instance modules database (/my.domain/my-snippets.xqy) in at.

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
  <transform-results apply="snippet"
      ns="http://marklogic.com/example"
      at="/ext/my.domain/my-snippets.xqy">
    <max-matches>3</max-matches>
    <preferred-elements>
      <element ns="" name="LINE" />
    </preferred-elements>
  </transform-results>
  <search-option>filtered</search-option>
</options>
JSON
{
  "options": {
    "transform-results": {
      "apply": "snippet",
      "ns": "http://marklogic.com/example",
      "at": "/ext/my.domain/my-snippets.xqy",
      "max-matches": 3,
      "preferred-matches": {
        "element": [{
          "ns": "",
          "name": "LINE"
        }]
      }
    },
    "search-option": ["filtered"]
  }
}

This example installs the above XML options and applies them to the query for 'hamlet' to produce snippets that display the act, scene, speech and dialog number of each match:

# Windows users, see Modifying the Example Commands for Windows 
# Install the custom snippeting module
$ curl --anyauth --user user:password -X PUT \
    -d @./my-snippets.xqy \
    -H "Content-type: application/xquery" \
    http://localhost:8000/LATEST/ext/my.domain/my-snippets.xqy
# Install the options
$ curl --anyauth --user user:password -X PUT \
    -d @./transform-results.xml \
    -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/config/query/custom-snippet
# Use the options in a query
$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/xml" \
    "http://localhost:8000/LATEST/search?q=hamlet&options=custom-snippet"
<search:response snippet-format="custom" total="20" start="1" ...>
  <search:result index="1" uri="/shakespeare/plays/hamlet.xml"
      path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)" ...>
    <search:snippet>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[48]/LINE[6]">Act 1, Scene 1, Speech 48, Line 6
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[48]/LINE[17]">Act 1, Scene 1, Speech 48, Line 17
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[59]/LINE[6]">Act 1, Scene 1, Speech 59, Line 6
      </search:match>
    </search:snippet>
  </search:result>
  ...
</search:response>

To generate the equivalent results in JSON, change the Accept header to application/json. For example:

$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/json" \
    "http://localhost:8000/LATEST/search?q=hamlet&options=custom-snippet"
{
  "snippet-format": "custom",
  "total": 20,
  "start": 1,
  "page-length": 10,
  "results": [
    {
      "index": 1,
      "uri": "/shakespeare/plays/hamlet.xml",
      "path": "fn:doc(\"/shakespeare/plays/hamlet.xml\")",
      "score": 158720,
      "confidence": 0.80079,
      "fitness": 1,
      "matches": [
        {
          "path": "fn:doc(\"\/shakespeare/plays/hamlet.xml\")/PLAY/ACT[1]/SCENE[1]/SPEECH[48]/LINE[6]",
          "match-text": [
            "Act 1, Scene 1, Speech 48, Line 6"
          ]
        },
        {
          "path": "fn:doc(\"\/shakespeare\/plays\/hamlet.xml\")/PLAY/ACT[1]/SCENE[1]/SPEECH[48]/LINE[17]",
          "match-text": [
            "Act 1, Scene 1, Speech 48, Line 17"
          ]
        },
        {
          "path": "fn:doc(\"/shakespeare/plays/hamlet.xml\")/PLAY/ACT[1]/SCENE[1]/SPEECH[59]/LINE[6]",
          "match-text": [
            "Act 1, Scene 1, Speech 59, Line 6"
          ]
        }
      ]
    }
  ],
  ...
}
Generating Custom JSON Snippets

If you need to create custom JSON snippets that cannot be expressed as XML, set @format to "json" on the search:snippet you return, and populate the search:snippet with serialized JSON. For example:

declare function my:snippeter(
   $result as node(),
   $ctsquery as schema-element(cts:query),
   $options as element(search:transform-results)?
) as element(search:snippet) {
    element search:snippet {
        attribute format { "json" },
        text {'{"MY":"CUSTOM SNIPPET"}'}
    }
};

You cannot change just a portion of the snippet to serialized JSON. For example, you cannot just change the search:match text.

Transforming the Search Response

You can make arbitrary changes to the response from a search or values query by applying a transformation function to the response. Search response transforms use the same interface and framework as content transformations applied during document ingestion, described in Working With Content Transformations.

When you specify a transform function on a search operation, your function is invoked on the search response and any documents returned by the search. For example, if you search as part of a multi-document read, then the matching documents are returned, and your transform is called on each docment. Therefore, your transform should be prepared to receive either a matched document or a XML or JSON search response, such as a document node with a <search:response/> root element, in the content parameter.

Any customizations made by the transform-results query option or result decorators are applied before calling your transform function.

You can probe the document type to test whether the input to your transform receives JSON or XML input. For example, in server-side JavaScript, you can test the documentFormat property of a document node:

function myTransform(context, params, content) {
  if (content.documentFormat == "JSON") {
    // handle as JSON or a JavaScript object
  } else {
    // handle as XML
}
  ...
}

In XQuery and XSLT, you can test the node kind of the root of the document, which will be element for XML and object for JSON.

declare function dumper:transform(
  $context as map:map,
  $params as map:map,
  $content as document-node()
) as document-node()
{
  if (xdmp:node-kind($content/node() eq "element") 
  then(: process as XML :)
  else (: process as JSON :)

As with read and write transforms, the content object is immutable in JavaScript, so you must call toObject to create a mutable copy:

var output = content.toObject();
...modify output...
return output;

The type of document you return must be consistent with the output-type (outputType) context value. If you do not return the same type of document as was passed to you, set the new output type on the context parameter.

To use a transform function:

  1. Create a transform function according to the interface described in Writing Transformations.
  2. Install your transform function on the REST API instance following the instructions in Installing Transformations.
  3. Apply your transform to a request to the /search or /values service by specifying the name in the transform request parameter, as described in Applying Transformations.

For example, assuming you installed a transform function under the name 'example', the following command applies the function to the results of a search:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
   'http://localhost:8000/LATEST/search?q=dog&transform=example'

Generating Search Term Completion Suggestions

Use the /suggest service to generate search term completion suggestions that match a wildcard terminated string. For example, if the user enters the text 'doc' into a search box, you can query /suggest with 'doc' to retrieve a list of terms matching 'doc*', and then display them to user. This service is analogous to calling the XQuery function search:suggest.

The following topics are covered:

Basic Steps

To retrieve a list of search suggestions using the REST Client API, use the following procedure. For a detailed example, see Example: Generating Search Suggestions Using GET.

  1. Configure at least one index on the XML element, XML attribute, or JSON property you want to include in the search for suggestions. For performance reasons, a range or collection index is recommended over a word lexicon; for details, see search:suggest.
  2. Define query options that use your index as a suggestion source by including it in the definition of a default-suggestion-source or suggestion-source option. For details, see Search Term Completion Using search:suggest in the Search Developer's Guide.
  3. Send a GET or POST request to the /suggest service with a URL of the following form, where partial-q is the text for which to retrieve suggestions.
    http://host:port/version/suggest?partial-q=text_to_match&options=your_options

If you use a GET request to generate suggestions, you must pre-install the query options. For details, see Configuring Query Options.

If you use a POST request to generate suggestions, the POST body must contains a combined query. You can use persistent query options and/or include dynamic query options in the combined query. For details, see Specifying Dynamic Query Options with Combined Query.

If you define your suggestion source in the default query options, you can omit the options request parameter or dynamic query options.

You can further constrain the suggestions by including one or more additional queries using the q request parameter or in the combined query of a POST request body. The request returns only suggestions in documents that also match the additional queries. Multiple additional queries are AND'd together. The following example URL returns only those suggestions found in fragments that match the query prefix:xdmp.

http://localhost:8000/LATEST/suggest/partial-q=doc&q=prefix:xdmp

Additional request parameters allow you to limit the number of suggestions returned and specify a substring of partial-q to match against. For details, see GET /v1/suggest or POST /v1/suggest in MarkLogic REST API Reference.

Example: Generating Search Suggestions Using GET

This example walks you through configuring your database and REST instance to try retrieving search suggestions. The Documents database is assumed in this example, but you can use any database.

  1. If you do not already have a REST API instance, create one. This example assumes your instance is on port 8000.
  2. Initialize the Database.
  3. Install Query Options
  4. Retrieve Unconstrained Search Suggestions
  5. Retrieve Constrained Search Suggestions
Initialize the Database

Run the following query in Query Console to load the sample data into your database, or use PUT or POST requests to /documents to create equivalent documents. The example will retrieve suggestions for the <name/> element, with and without a constraint based on the <prefix/> element.

xdmp:document-insert("/suggest/load.xml",
  <function xmlns="http://marklogic.com/example">
    <prefix>xdmp</prefix>
    <name>document-load</name>
  </function>
  );
xdmp:document-insert("/suggest/insert.xml",
  <function xmlns="http://marklogic.com/example">
    <prefix>xdmp</prefix>
    <name>document-insert</name>
  </function>
  );
xdmp:document-insert("/suggest/query.xml",
  <function xmlns="http://marklogic.com/example">
    <prefix>cts</prefix>
    <name>document-query</name>
  </function>
  );
xdmp:document-insert("/suggest/search.xml",
  <function xmlns="http://marklogic.com/example">
    <prefix>cts</prefix>
    <name>search</name>
  </function>
  );

To create the range index used by the example, run the following query in Query Console, or use the Admin Interface to create an equivalent index on the name element. The following query assumes you are using the Documents database; modify as needed.

xquery version "1.0-ml";
import module namespace admin = "http://marklogic.com/xdmp/admin" 
  at "/MarkLogic/admin.xqy";
admin:save-configuration(
  admin:database-add-range-element-index(
    admin:get-configuration(),
    xdmp:database("Documents"),
    admin:database-range-element-index(
    "string", "http://marklogic.com/example",
    "name", "http://marklogic.com/collation/", fn:false())
  )
);
Install Query Options

This step installs persistent query options that define name as the default suggestion source, plus an additional value constraint on prefix. Copy the following query options into a file named suggest-options.xml:

<options xmlns="http://marklogic.com/appservices/search">
 <default-suggestion-source>
   <range type="xs:string" facet="true">
      <element ns="http://marklogic.com/example" name="name"/>
   </range>
 </default-suggestion-source>
 <constraint name="prefix">
   <value>
      <element ns="http://marklogic.com/example" name="prefix"/>
   </value>
 </constraint>
</options>

Install the options using the /config/query service, as described in Creating or Modifying Query Options. Use a command similar to the following:

$ curl --anyauth --user user:password -X PUT \
    -d @./suggest-options.xml -i -H "Content-type: application/xml" \
    http://localhost:8000/LATEST/config/query/opt-suggest

The database and REST API instance are now configured to retrieve the example search suggestions.

Retrieve Unconstrained Search Suggestions

The following request returns all values of <name/> that match the wildcard string 'doc*':

$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/xml" \
  'http://localhost:8000/LATEST/suggest?partial-q=doc&options=opt-suggest'
...
<search:suggestions xmlns:search=...>
  <search:suggestion>document-insert</search:suggestion>
  <search:suggestion>document-load</search:suggestion>
  <search:suggestion>document-query</search:suggestion>
</search:suggestions>

If you perform the same request with an Accept header of application/json, the response data looks like the following:

{
  "suggestions": [
    "document-insert",
    "document-load",
    "document-query"
  ]
}
Retrieve Constrained Search Suggestions

Recall that the query options include a value constraint on the prefix element:

<options xmlns="http://marklogic.com/appservices/search">
 <default-suggestion-source>
   <range type="xs:string" facet="true">
      <element ns="http://marklogic.com/example" name="name"/>
   </range>
 </default-suggestion-source>
 <constraint name="prefix">
   <value>
      <element ns="http://marklogic.com/example" name="prefix"/>
   </value>
 </constraint>
</options>

The following command uses this constraint in the additional query prefix:xdmp to limit results to suggestions in fragments with the prefix value xdmp. This eliminates the function document-query from the results because it has the prefix cts.

$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/xml" \
  'http://localhost:8000/LATEST/suggest?partial-q=doc&options=opt-suggest&q=prefix:xdmp'
...
<search:suggestions xmlns:search=...>
  <search:suggestion>document-insert</search:suggestion>
  <search:suggestion>document-load</search:suggestion>
</search:suggestions>

If you perform the same request with an Accept header of application/json, the response data looks like the following:

{
  "suggestions": [
    "document-insert",
    "document-load"
  ]
}

Example: Generating Search Suggestions Using POST

This example uses a POST request with the following combined query to limit results to suggestions in fragments with the prefix value xdmp. The example assumes your database contains the documents loaded in Initialize the Database.

<search xmlns="http://marklogic.com/appservices/search" >
  <query>
    <value-query>
      <element ns="http://marklogic.com/example" name="prefix"/>
      <text>xdmp</text>
    </value-query>
  </query>
  <options xmlns="http://marklogic.com/appservices/search">
    <default-suggestion-source>
      <range type="xs:string" facet="true">
        <element ns="" name="name"/>
      </range>
    </default-suggestion-source>
  </options>
</search>

Note that you do not need to pre-install persistent query options in this case because the options are included in the query.

The following command uses this query to retrieve suggestions for the input string "doc".

$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/xml" \
  'http://localhost:8000/LATEST/suggest?partial-q=doc
...
<search:suggestions xmlns:search=...>
  <search:suggestion>document-insert</search:suggestion>
  <search:suggestion>document-load</search:suggestion>
</search:suggestions>

The following query is the equivalent combined query, expressed as JSON:

{ "search": {
    "qtext": [ "document" ],
    "query": {
      "value-query": {
        "element": {
          "ns": "http://marklogic.com/example",
          "name": "prefix"
        },
        "text": [ "xdmp" ]
      }
    },
    "options": {
      "default-suggestion-source": {
        "range": {
          "type": "xs:string",
          "facet": true,
          "element": {
            "ns": "",
            "name": "name"
          }
        }
      }
    }
} }

If you perform the same request with an Accept header of application/json, the response data looks like the following:

{
  "suggestions": [
    "document-insert",
    "document-load"
  ]
}

Where to Find More Information

For more details on using search suggestions, including performance recommendations and additional examples, see the following:

« Previous chapter
Next chapter »