Loading TOC...
REST Application Developer's Guide (PDF)

REST Application Developer's Guide — Chapter 4

Using and Configuring Query Features

This chapter covers the following topics:

Query Feature Overview

The MarkLogic REST API includes several services for querying content in a MarkLogic Server database. The usage model for the query features is:

  1. Optionally, use the /config/query service to install a set of named query options to apply to future queries. The name is used to apply the options to subsequent queries. For details, see Configuring Query Options.
  2. Optionally, use the /config/namespaces service to install namespace bindings for namespace aliases you will use in queries. For details, see Using Namespace Bindings.
  3. Use /search, /keyvalue, or /values to perform a query.
    1. Depending on the service, the query may be expressed using request parameter or in the request body. For details, see the reference documentation for each method.
    2. Apply options by specifying the name of previously installed query options in the options request parameter and/or providing dynamic query options in the request body using a combined query. For details, see Adding Query Options to a Request.
    3. Specify the content type (XML or JSON) of the query results using the format request parameter or Accept headers.
  4. Query results of the requested content type are returned in the response body.

The table below gives a brief summary of each query related service.

Service Description
/search Use string and/or structured queries to search documents and metadata. Queries may be expressed in XML or JSON. For details, see Querying Documents and Metadata.
/keyvalue Use key-value pair queries to search XML content and metadata, and JSON content. For example, search by element, element attribute, or JSON key values. Dynamic query options are not supported by this service. For details, see Querying Documents and Metadata.
/values Query lexicon and range index values and value co-occurrences; analyze lexicon and range index values and value-co-occurrences with builtin and user-defined aggregate functions. For details, see Querying Lexicons and Range Indexes.
/config/query Use XML or JSON to configure persistent query options for use with /search, /keyvalue, and /values. For details, see Configuring Query Options.
/config/namespaces Configure bindings between namespace prefixes and namespace URIs so you can use QNames in query contexts where it is not possible to dynamically specify a namespace. For details, see Using Namespace Bindings.

The /search, /keyvalue, and /values services use query options to control and configure queries and search results. For details on options syntax, see:

For more information on search concepts, see the Search Developer's Guide.

Querying Documents and Metadata

This section describes how to use the /search and /keyvalue services to search documents and metadata.

Constraining a Query by Collection or Directory

Use the collection and directory request parameters to the /search and /keyvalue services to limit search results to matches in specific collections or database directories.

You can specify multiple collections or directories. For example, the following URL finds documents containing 'julius' in the 'tragedy' and 'comedy' collections:

http://localhost:8003/v1/search?q=julius&collection=tragedy&collection=comedy

You can use collection and directory constraints together. For example, by adding a directory to the above search, you can further limit matches to documents in the '/shakespeare/plays' directory:

http://localhost:8003/v1/search?q=julius&collection=tragedy&collection=comedy&directory=/shakespeare/plays

For details about collections and directories, see Collections in the Search Developer's Guide and Directories in the Application Developer's Guide.

Searching With String Queries

The MarkLogic Server Search API default search grammar allows you to quickly construct simple searches such as 'cat', 'cat AND dog', or 'cat NEAR dog'. You can also customize the search grammar. For details, see Search Grammar in the Search Developer's Guide.

To search for matches to a simple string query, send a GET request to the /search service with a URL of the form:

http://host:port/version/search?q=query_string

Where query_string is a string conforming to the search grammar.

You can include both a string query and a structured query in the same request. In this case, the two queries are AND'd together. For more information, see Searching With Structured Queries and Specifying Dynamic Query Options with Combined Query.

You can request search results in XML or JSON. To control the output format, use the format request parameter or Accept headers, as described in Controlling Output Content Type.

For details on the structure of search results, see Appendix A: JSON Query Options Reference or Appendix B: XML Query Options Reference.

For a complete list of parameters available with the /search service, see 'GET /v1/search' in the REST Resources API.

Searching With Structured Queries

Structured queries enable you to create complex queries, represented in XML or JSON. For details, see Using Structured Search as an Alternate to cts:query in the Search Developer's Guide.

To search using a structured query, send a GET or POST request to the /search service. To pass the structured query as a request parameter, send a GET request with a URL of the form:

http://host:port/version/search?structuredQuery=query

Where query is an XML or JSON representation of a structured query.

To pass a structured query in the request body, send a POST request of the following form and place the structured or combined query in the POST body. Set the Content-type header appropriately:

http://host:port/version/search

If the request also includes a string query, the string query and structured query are AND'd together. For details, see Searching With String Queries and Specifying Dynamic Query Options with Combined Query.

You can request search results in XML or JSON. To control the output content type on GET, use the format request parameter or Accept headers. To control the input and output content type on POST, use the format parameter or the Accept and Content-type headers. For details, see Controlling Output Content Type.

For a complete list of parameters available with the /search service, see 'GET /v1/search' or 'POST /v1/search' in the REST Resources API.

The following example combines a structured query equivalent to 'Yorick NEAR Horatio' and a string query for 'knew', requesting search results in JSON. The effect is equivalent to searching for '(Yorick NEAR Horatio) AND knew'.

$ cat sq.xml
<search:query xmlns:search="http://marklogic.com/appservices/search">
  <search:near-query>
    <search:term-query>
      <search:text>Yorick</search:text>
    </search:term-query>
    <search:term-query>
      <search:text>Horatio</search:text>
    </search:term-query>
  </search:near-query>
</search:query>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X POST -d@"./sq.xml" \
  -H 'Content-type: application/xml' \
  'http://localhost:8003/v1/search?q=knew&format=json'

Searching By JSON Key and Value

When you store JSON documents in the database using the /documents service of the MarkLogic REST API, you can search based on the values of keys in the JSON content using the /keyvalue service.

Efficient key-value searches require an element range index on the key. For details, see Creating Indexes on JSON Keys.

To search by key value, send a GET request to the /keyvalue service with a URL of the form:

http://host:port/version/keyvalue?key=the_key&value=the_value

Where the_key is the name of the key to match against the_value. For example, the following search matches all JSON documents containing a key called 'species' with a value of 'canis lupus':

http://localhost:2003/v1/keyvalue?key=species&value=canis%20lupus

Only JSON documents are considered in the search when you use the key parameter. To query XML content, see Searching By Element or Attribute Name and Value.

You can request search results in XML or JSON. To control the output format, use the format request parameter or Accept headers, as described in Controlling Output Content Type.

For a complete list of parameters available with the /keyvalue service, see 'GET /v1/keyvalue' in the REST Resources API.

The following example searches for occurrences of the key 'SPEAKER' with the value 'BERNARDO'. First, insert a JSON document into the database:

$ cat hamlet.json
{
  "TITLE":"Hamlet",
  "ACT":{
    "TITLE":"ACT I",
    "SCENE":{
      "TITLE":"SCENE I. Elsinore. A platform before the castle.",
      "STAGEDIR":"FRANCISCO at his post. Enter to him BENARDO",
      "SPEECHES":[
        {"SPEECH":{
          "SPEAKER":"BERNARDO",
          "LINE":"Whos there?"
        }},
        {"SPEECH":{
          "SPEAKER":"FRANCISCO",
          "LINE":"Nay, answerme: stand, and unfold yourself."
        }},
        {"SPEECH":{
          "SPEAKER":"BERNARDO",
          "LINE":"Long live the king!"
        }}
      ]
    }
  }
}
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT -d @"./hamlet.json" \
  -H 'Content-type: application/json' \
  http://localhost:8003/v1/documents?uri=/example/hamlet.json

Then search for occurrences of SPEAKER with the value BERNARDO. The search matches the document just inserted and returns a snippet for the two speeches by Bernardo:

$ curl --anyauth --user user:password -X GET \
  'http://localhost:8003/v1/keyvalue?key=SPEAKER&value=BERNARDO'
...
{
  "snippet-format":"snippet",
  "total":1,
  "start":1,
  "page-length":10,
  "results":[{
    "index":1,
    "uri":"/example/hamlet.json",
    "path":"fn:doc(\"/example/hamlet.json\")",
    "score":154880,
    "confidence":0.790972,
    "fitness":0.790972,
    "matches":[       {"path":"fn:doc(\"/example/hamlet.json\")/*:json/*:ACT/*:SCENE/*:SPEECHES/*:json[1]/*:SPEECH/*:SPEAKER",
       "match-text":[{"highlight":"BERNARDO"}]},
      {"path":"fn:doc(\"/example/hamlet.json\")/*:json/*:ACT/*:SCENE/*:SPEECHES/*:json[3]/*:SPEECH/*:SPEAKER",
       "match-text":[{"highlight":"BERNARDO"}]}
    ]
  }],
  "metrics":{
    "query-resolution-time":"PT0.001398S",
    "facet-resolution-time":"PT0.000052S",
    "snippet-resolution-time":"PT0.001411S",
    "total-time":"PT0.003032S"
  }
}

Searching By Element or Attribute Name and Value

The /keyvalue service provides a simple interface to match element and element attribute values in XML content. For more sophisticated element and element attribute value searches, use structured queries; see Searching With Structured Queries.

Efficient element and element attribute searches require element and attribute range indexes. For details, see Range Indexes and Lexicons in the Administrator's Guide.

To find all XML elements with a specific element name and value, send a GET request to the /keyvalue service with a URL of the form:

http://host:port/version/keyvalue?element=elem_name&value=elem_value

If the element name is defined within a namespace, you must pre-define a namespace binding and use the defined prefix in elem_name. For details, see Using Namespace Bindings.

The following example matches documents containing a <SPEAKER> element with the value 'BERNARDO':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
   'http://localhost:8003/v1/keyvalue?element=SPEAKER&value=BERNARDO'

To find all XML elements with a specific element name that have an attribute with a specific value, send a GET request to the /keyvalue service with a URL of the form:

http://host:port/version/keyvalue?element=elem_name&attribute=attr_name&value=elem_value

This example matches documents containing <nominee> elements with a year attribute value of 1946:

$ curl --anyauth --user user:password -X GET \
    'http://localhost:8003/v1/keyvalue?element=nominee&attribute=year&value=1946'

MarkLogic Server can return search results in XML or JSON. To specify the output format, use the format request parameter or Accept headers, as described in Controlling Output Content Type.

For a complete list of parameters available with the /keyvalue service, see 'GET /v1/keyvalue' in the REST Resources API.

Debugging /search Queries With Logging

When you enable the debug REST API instance property and make queries using /search, query details are sent to the MarkLogic Server error log. Enable this logging by setting the debug property to true, as described in Configuring Instance Properties.

When you enable debug logging, requests to /search log the following information helpful in debugging queries:

  • EFFECTIVE OPTIONS. These are the query options in effect for your query. If the request does not include an options request parameter or a combined query, these are the default options.
  • CTS-QUERY. Your query, expressed as a cts:query object. A cts:query object is the low level XML representation of any query expression. For details, see Understanding cts:query in the Search Developer's Guide.
  • SEARCH-QUERY. Your query, expressed as a search:query object. This information is only included if your request includes a structured query. A search:query is the XML representation of a structured query. For details, see Using Structured Search as an Alternate to cts:query in the Search Developer's Guide.

Examining the logging output can help you understand how MarkLogic Server sees your query. In the following example, notice that a string query for 'Welles' is really a cts:word-query for the text string 'Welles'. With the addition of a structured query in the request body to limit the query to ocurrences of 'Welles' to documents that also include 'John' when it occurs near 'Huston', the logging output includes a cts:query and a search:query that reflect the result of combining the string query and structured query.

  1. Enable debug logging for your REST API instance.
    $ cat debug-on.xml
    <properties xmlns="http://marklogic.com/rest-api">
      <debug>true</debug>
    </properties>
    $ curl --anyauth  --user user:password -X PUT \
        -d @./debug-on.xml -H "Content-type: application/xml" \
        http://localhost:8003/v1/config/properties
  2. Issue a query to /search.
    $ curl --anyauth --user user:password -X GET \
        -H "Accept: application/xml" \
        http://localhost:8003/v1/search?q="Welles"
    ...search matches returned
  3. View the debug output in the MarkLogic Server error log, located in MARKLOGIC_DIR/Logs/ErrorLog.txt. There is no search:query in the log because this is a simple string query. (The timestamp and Info: header on each log line have been elided).
    $ tail -15 /var/opt/MarkLogic/Logs/ErrorLog.txt
    ... Request environment:
    ... GET /v1/search?q=Douglas
    ... Rewritten to:
        /MarkLogic/rest-api/endpoints/search-list-query.xqy?q=Douglas
    ... ACCEPT */*
    ... PARAMS:
    ...   q: (Welles)
    ...
    ... Endpoint Details:
    ... EFFECTIVE OPTIONS:
    ... <options xmlns="http://marklogic.com/appservices/search">
    ...   <search-option>unfiltered</search-option>
    ...   <quality-weight>0</quality-weight>
    ... </options>
    ... CTS-QUERY:
    ... <cts:word-query qtextref="cts:text"
                xmlns:cts="http://marklogic.com/cts">
          <cts:text>Welles</cts:text>
        </cts:word-query>
  4. Add a structured query to the search to constrain matches to documents that also contain 'John' occurring near 'Huston'.
    $ cat ./structq.json
    {"query":
      {"near-query":
        {"queries":[
          {"term-query":{"text":"John"}},
          {"term-query":{"text":"Huston"}}
        ]}
      }
    }
    $ curl --anyauth --user user:password -X POST -d@./structq.json \
        -H "Content-type: application/json" \
        http://localhost:8003/v1/search?q=Welles
    ...search results returned
  5. View the logging output again and notice the inclusion of a search:query section in the log. Also, notice that the cts:query and the search:query represent the combined string and structured queries.
    ... Request environment:
    ... POST /v1/search?q=Welles
    ... Rewritten to:
        /MarkLogic/rest-api/endpoints/search-list-query.xqy?q=Welles
    ... ACCEPT */*
    ... PARAMS:
    ...   q: (Welles)
    ...
    ... Endpoint Details:
    ... EFFECTIVE OPTIONS:
    ... <options xmlns="http://marklogic.com/appservices/search">
    ...   <search-option>unfiltered</search-option>
    ...   <quality-weight>0</quality-weight>
    ... </options>
    ... CTS-QUERY:
    ... cts:and-query(
          (
            cts:word-query("Welles", ("lang=en"), 1), 
            cts:near-query((
              cts:word-query("John", ("lang=en"), 1),
              cts:word-query("Huston", ("lang=en"), 1)), 
            10, (), 1)
           ), ()
         )
    ... SEARCH-QUERY:
    ... <search:query
              xmlns:search="http://marklogic.com/appservices/search">
    ...   <search:qtext>Welles</search:qtext>
    ...   <search:near-query>
    ...     <search:term-query>
    ...       <search:text>John</search:text>
    ...     </search:term-query>
    ...     <search:term-query>
    ...       <search:text>Huston</search:text>
    ...     </search:term-query>
    ...   </search:near-query>
    ... </search:query>

Querying Lexicons and Range Indexes

The /values service supports the following operations:

For related search concepts, see Browsing With Lexicons in the Search Developer's Guide.

This section covers the following topics:

Querying the Values in a Lexicon or Range Index

Use the /values/{name} service to query the values in a lexicon or range index. Such queries must be supported by query options that include a <values/> element identifying the target lexicon or index; for details, see Defining Queryable Lexicon or Range Index Values.

To query the values in a lexicon or range index, use the /values/{name} service as follows:

  1. Install query options or define dynamic query options that include a values option naming the target lexicon or index. For details, see Adding Query Options to a Request.
  2. Send a GET or POST request to the /values/{name} service, where name is the name of a values definition in the query options from Step 1 or in the default query options. If you use persistent query options, include the options parameter and replace options_name with the name under which the query options are installed.
    http://host:port/version/values/name?options=options_name

When constructing your request:

  1. Substitute a named range specification for name in the URL. This must be a values range specification in the query options named by the options parameter, in the options portion of a combined query in the POST body, or in the default query options.
  2. To use custom query options, specify them using the options parameter and/or the options portion of a combined query in the POST body. For details, see Identifying Lexicon and Range Index Values in Query Options and Adding Query Options to a Request.
  3. To use the default query options, omit the options parameter and options portion of a combined query. The default query options should include a range specification matching name.
  4. To constrain the analysis to values in certain fragments, specify a query using the q and/or structuredQuery parameter, or a structured or combined query in the POST body. For details, see Using a Query to Constrain Results.
  5. Specify the input (POST only) and output content type (XML or JSON) using the format parameter or the HTTP Content-type and Accept headers. For details, see Controlling Output Content Type.

Additional request parameters are available. For details, see the MarkLogic REST API Reference.

The following example assumes the query options shown in the table below are installed under the name index-options:

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
    <values name="speaker">
        <range type="xs:string">
            <element ns="" name="SPEAKER"/>
        </range>
    </values>
</options>
JSON
{
  "options": {
    "values": [
      {
        "name": "speaker",
        "range": {
          "type": "xs:string",
          "element": { "ns": "", "name": "SPEAKER" }
        }
      }
    ]
  }
}

Then the example command below queries /values/speaker to retrieve the values of all SPEAKER elements:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    http://localhost:8003/v1/values/speaker?options=index-options
...
<values-response name="speaker" type="xs:string" \
    xmlns="http://marklogic.com/appservices/search" \
    xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <distinct-value frequency="1">[GOWER]</distinct-value>
  <distinct-value frequency="1">[PROSPERO]</distinct-value>
  ...
</values-response>

If you use the format parameter to request JSON output from the same request, the results are similar to the following:

{
  "values-response": {
    "name": "speaker",
    "type": "xs:string",
    "distinct-value": [
      {
        "frequency": 1,
        "_value": "[GOWER]"
      },
      {
        "frequency": 1,
        "_value": "[PROSPERO]"
      },
      ...
    ],
    "metrics": {
      "values-resolution-time": "PT0.016665S",
      "aggregate-resolution-time": "PT0.00001S",
      "total-time": "PT0.018102S"
    }
  }
}

If you add a string query, structured query, or both to the request, you can limit the results to index values in matching fragments. For example, the following requests returns only the SPEAKER values in fragments containing 'HAMLET':

$ curl --anyauth --user user:password -X GET \
    'http://localhost:8003/v1/values/speaker?options=index-options&q="HAMLET"'

Finding Value Co-Occurrences in Lexicons

A co-occurence is a set of index or lexicon values occurring in the same document fragment. The MarkLogic REST API enables you to query for 2-way co-occurrences. That is, for pairs of values from two lexicons or indexes, occurring in the same fragment.

Use this procedure to query for co-occurrences of values in lexicons or range indexes with the /values/{name} service:

  1. Specify query options that include a tuples range specification for the target lexicons or indexes. For details, see Defining Queryable Lexicon or Range Index Co-Occurrences and Adding Query Options to a Request.
  2. Send a GET or POST request of the following form to the /values/{name} service, where name is the name of a tuples definition in the query options from Step 1. If you use persistent query options, include the options parameter and replace options_name with the name under which the query options are installed.
    http://host:port/version/values/name?options=options_name

When constructing your request:

  1. Substitute a named range specification for name in the URL. This must be a tuples range specification in the query options named by the options parameter, in the options portion of a combined query in the POST body, or in the default query options if options is omitted.
  2. To use custom query options, specify them using the options parameter and/or the options portion of a combined query in the POST body. For details, see Identifying Lexicon and Range Index Values in Query Options and Adding Query Options to a Request.
  3. To use the default query options, omit the options parameter and options portion of a combined query. The default query options should include a range specification matching name.
  4. To constrain the analysis to values in certain fragments, specify a query using the q and/or structuredQuery parameters, or a structured or combined query in the POST body. For details, see Using a Query to Constrain Results.
  5. Specify the input (POST only) and output content type (XML or JSON) using the format parameter or the HTTP Accept and Content-type headers. For details, see Controlling Output Content Type.

Additional request parameters are available. For details, see the MarkLogic REST API Reference.

For more information about co-occurrences, see Value Co-Occurrences Lexicons in the Search Developer's Guide.

The following example assumes the query options shown in the table below are installed under the name index-options. Note that the options include a <tuples/> definition named 'speaker-scene'.

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
  <tuples name="speaker-scene">
    <range type="xs:string">
      <element ns="" name="SPEAKER"/>
    </range>
    <range type="xs:string">
      <path-index>
        /PLAY/ACT/SCENE/TITLE
      </path-index>
    </range>
  </tuples>
</options>
JSON
{
  "options": {
    "tuples": [
      {
        "name": "speaker-scene",
        "range": [
          {
            "type": "xs:string",
            "element": {
              "ns": "",
              "name": "SPEAKER"
            }
          },
          {
            "type": "xs:string",
            "path-index": {
              "text": "\/PLAY\/ACT\/SCENE\/TITLE"
            }
          }
        ]
      }
    ]
  }
}

Given these query options, this example queries /values/speaker-scene to retrieve co-occurrences of SPEAKER and scene titles:

$ curl --anyauth --user user:password -X GET \
   http://localhost:8003/v1/values/speaker-scene?options=index-options
...
<values-response name="speaker-scene"
    xmlns="http://marklogic.com/appservices/search"
    xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <tuple frequency="1">
    <distinct-value xsi:type="xs:string" ...>A Lord</distinct-value>
    <distinct-value ...>SCENE II.  The forest.</distinct-value>
  </tuple>
  ...
</values-response>

If you add a string query, structured query, or both to the request, you can limit the results to co-occurrences in matching fragments. For example, the following requests returns only the SPEAKER values in fragments containing 'HAMLET':

$ curl --anyauth --user user:password -X GET \
    'http://localhost:8003/v1/values/speaker-scene?options=index-options&q="HAMLET"'

Using a Query to Constrain Results

You can use a string query, structured query, or combined query with /values/{name} to limit results to fragments that match the query. The values must occur in fragments matching the query. The fragments are selected in the same manner as an 'unfiltered' cts:search; for details, see Understanding Unfiltered Searches in the Query Performance and Tuning Guide.

If you use a string query, the query is treated as a word query; for details, see cts:word-query. Supply a string query using one of the following:

  • The q request parameter of a POST or GET request
  • The qtext element or or sub-object of a combined query supplied in the body of a POST request.

Specify a structured query in either XMl or JSON using one of the following:

  • The structuredQuery request parameter on a GET request.
  • The query element or sub-object of a combined query supplied in the body of a POST request.
  • In the body of a POST request that does not use a combined query.

The following example limits the results to just those fragments containing the word 'moon':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    'http://localhost:8003/v1/values/speaker?options=index-options&q=moon'

For the full example without a query constraint, see Querying the Values in a Lexicon or Range Index.

For details on query syntax, see the following sources:

Identifying Lexicon and Range Index Values in Query Options

Before you can use the /values/{name} service, you must install query options that identify lexicons and range indexes available to queries. Use the values option to make the values in a single lexicon or index available. Use the tuples option to make co-occurrences of values in multiple lexicons or indexes available.

This section covers the following topics:

Defining Queryable Lexicon or Range Index Values

Use this procedure to make the values in a lexicon or range index available through the /values/{name} service:

  1. Create a lexicon or range index on the database, as described in Range Indexes and Lexicons in the Administrator's Guide and
  2. Associate a name with the index or lexicon by defining a <values/> element in query options.
  3. Supply the query options from Step 2 in your request, as described in Adding Query Options to a Request.

For more information on lexicons and range indexes, see Browsing With Lexicons in the Search Developer's Guide and Creating Indexes on JSON Keys.

For example, if the database configuration includes an element range index on SPEAKER, such as the one shown below:

Then the following query options enable the SPEAKER element values to be referenced using as the resource /values/speaker, once you install the options:

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
    <values name="speaker">
        <range type="xs:string">
            <element ns="" name="SPEAKER"/>
        </range>
    </values>
</options>
JSON
{
  "options": {
    "values": [
      {
        "name": "speaker",
        "range": {
          "type": "xs:string",
          "element": { "ns": "", "name": "SPEAKER" }
        }
      }
    ]
  }
}

The following example command installs the XML query options with the name 'index-options':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    -d@"./index-options.xml" -H "Content-type: application/xml" \
    http://localhost:8003/v1/config/query/index-options

Now, you can query the SPEAKER index using the /values/speaker resource with the query options named 'index-options'.

Defining Queryable Lexicon or Range Index Co-Occurrences

A <tuples/> element in query options specifies the indexes to use in constructing N-way value co-occurrences. Though MarkLogic Server supports N-way co-occurrences, the MarkLogic REST API only supports 2-way co-occurrences.

For more information about co-occurrences, see Value Co-Occurrences Lexicons in the Search Developer's Guide.

Use this procedure to make co-occurrences of values in two lexicons or range indexes available through the /values/{name} service:

  1. Create the lexicons or range indexes on the database, as described in Range Indexes and Lexicons in the Administrator's Guide.
  2. Associate a name with the lexicon/index pair by defining a <tuples> element in query options (or a 'tuples' object in JSON).
  3. Supply the query options from Step 2 in your request, as described in Adding Query Options to a Request.

For example, suppose the database configuration contains a string-valued element range index on <SPEAKER> and a path range index on the XPath expression /PLAY/ACT/SCENE/TITLE. The following query options enable querying co-occurrrences of these two indexes under the name 'speaker-scene':

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
    <tuples name="speaker-scene">
        <range type="xs:string">
            <element ns="" name="SPEAKER"/>
        </range>
        <range type="xs:string">
            <path-index>/PLAY/ACT/SCENE/TITLE</path-index>
        </range>
    </tuples>
</options>
JSON
{
  "options": {
    "tuples": {
      "name": "speaker-scene",
      "range": {
        "type": "xs:string",
        "path-index": {
          "text": "/PLAY/ACT/SCENE/TITLE",
        }
      }
    }
  }
}

The following example command installs the XML query options with the name 'index-options':

# Windows users, see Modifying the Example Commands for Windows$ curl --anyauth --user user:password -X PUT \
    -d@"./index-options.xml" -H "Content-type: application/xml" \
    http://localhost:8003/v1/config/query/index-options

Now, you can query the SPEAKER index using the /values/speaker-scene resource with the query options named 'index-options'.

Creating Indexes on JSON Keys

To efficiently search using JSON keys, you should define indexes on the keys. For example, using /keyvalue to query by JSON key values performs best when you define a value range index on the key you want to query against. To create such an index, you treat the JSON key as an XML element and define a corresponding element index, as described here.

JSON documents loaded using the MarkLogic REST API are stored internally as XML. Each JSON key has a corresponding XML element in the internal representation. The element name usually matches the JSON key name, but can be different if the key contains whitespace or other characters that cannot appear in an XML element name. Key names are encoded using the XQuery builtin xdmp:encode-for-NCName.

Element names in the XML representation of a JSON document are in the namespace 'http://marklogic.com/xdmp/json/basic'. You must use this namespace when defining indexes on keys in JSON documents.

For more information on JSON support in MarkLogic Server, see Working With JSON in the Application Developer's Guide.

To create an index on a JSON key, you must determine the XML element name that corresponds to the key in the XML representation of your JSON content, and then create an element range index on that element. For example, to efficiently search the values of the JSON key 'my-key', use the Admin Interface to create a range index with the following settings:

Analyzing Lexicons and Range Indexes With Aggregate Functions

This section covers the following topics:

Aggregate Function Overview

An aggregate function performs an operation over values or value co-occurrences in lexicons and range indexes. For example, you can use an aggregate function to compute the sum of values in a range index.

There are two kinds of aggregate functions, builtin and user-defined. MarkLogic Server provides builtin aggregate functions for several common analytical functions; see the list in Using Builtin Aggregate Functions in the Search Developer's Guide.

In addition, you can also implement aggregate user-defined functions (UDFs) in C++ and deploy them as native plugins. Aggregate UDFs must be installed before you can use them. For details, see Implementing an Aggregate User-Defined Function in the Application Developer's Guide.

You can use the MarkLogic REST API to apply aggregate functions using /values/{name} in two ways:

  • Include one or more <aggregate/> elements (XML) or 'aggregate' objects (JSON) in a <values/> or <tuples/> range specification in the query options.
  • Include one or more aggregate request parameters.

If aggregate functions are specified through both query options and request parameters, the request parameter(s) overrides the aggregates specified in the query options.

You can only specify multiple aggregate UDFs from more than one plugin using query options.

You cannot use the MarkLogic REST API to apply aggregate UDFs that require additional parameters.

Using Query Options to Apply Aggregate Functions

To specify an aggregate function in query options, include an <aggregate/> element in a <values/> or <tuples/> range specification. If you include multiple <aggregate/> specifications, MarkLogic Server applies all the functions.

For a builtin aggregate, specify the function name in the 'apply' attribute of an <aggregate/> element. For example, the query options below specify the builtin aggregate 'count', which is equivalent to the XQuery builtin cts:count-aggregate.

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
  <values name="speaker">
    <range type="xs:string">
      <element ns="" name="SPEAKER"/>
    </range>
    <aggregate apply="count"/>
  </values>
</options>
JSON
{
  "options": {
    "values": [
      {
        "name": "speaker",
        "range": {
          "type": "xs:string",
          "element": { "ns": "", "name": "SPEAKER" }
        },
        "aggregate": { "apply":"count" }
      }
    ]
  }
}

An aggregate UDF is identified by the function name and a relative path to the plugin that implements the aggregate, as described in Using Aggregate User-Defined Functions in the Search Developer's Guide. Specify the function name with the 'apply' attribute and the plugin path with the 'udf' attribute in an <aggregate/> element or object. For example, the following query options specify a native UDF called 'count' provided by a plugin installed under 'native/sampleplugin':

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
  <values name="speaker">
    <range type="xs:string">
      <element ns="" name="SPEAKER"/>
    </range>
    <aggregate apply="count" udf="native/sampleplugin" />
  </values>
</options>
JSON
{
  "options": {
    "values": [
      {
        "name": "speaker",
        "range": {
          "type": "xs:string",
          "element": { "ns": "", "name": "SPEAKER" }
        },
        "aggregate": { 
          "apply":"count", 
          "udf":"native/sampleplugin"
        }
      }
    ]
  }
}

To use query options to apply an aggregate function:

  1. Define query options that include one or more aggregate definitions, as shown above.
  2. Supply the query options from Step 1 in your request, as described in Adding Query Options to a Request.
  3. Apply the aggregate by sending a GET or POST request to /values/{name} and including the options from Step 2. For example: GET /v1/values/speaker?options=index-options.

For details on installing and using query options, see Configuring Query Options.

Using Request Parameters to Apply Aggregate Functions

To analyze lexicon or index values or co-occurrences with builtin aggregate function, make a GET or POST request to the /values/{name} service with a URL of the form:

http://host:port/version/values/name?aggregate=aggr_name&options=options_name

To analyze lexicon or index values or co-occurrences with a previously installed aggregate UDF, make a GET or POST request to the /values/{name} service with a URL of the form:

http://host:port/version/values/name?aggregate=aggr_name&aggregatePath=aggr_path&options=options_name

When constructing the request:

  1. Substitute a named range specification for name in the URL. This must be a range specification (<values/> or <tuples/>) in the query options supplied in the request, or in the default query options if options are omitted.
  2. To use custom query options, specify them using the options parameter and/or the options portion of a combined query in the POST body. For details, see Adding Query Options to a Request.
  3. To use the default query options, omit the options parameter and options portion of a combined query. The default query options should include a range specification matching name.
  4. Specify the aggregate function name using the aggregate parameter.

The name must be one of the builtin aggregate functions listed in Using Builtin Aggregate Functions the Search Developer's Guide, or a function implemented by the plugin identified by aggregatePath.

  1. If you're applying an aggregate UDF, specify the relative path to the plugin implementing the aggregate function using the aggregatePath parameter.
  2. To constrain the analysis to values in certain fragments, specify a query using the q and/or structuredQuery parameters, or a structured or combined query in the POST body. For details, see Using a Query to Constrain Results.
  3. Specify the result content type (XML or JSON) using the format parameter or the HTTP Accept headers. The default format is XML. For details, see Controlling Output Content Type.
  4. If you only want the aggregate value in the results, set the view parameter to aggregate. By default, MarkLogic Server returns both the lexicon or index values and the aggregate result.

Additional request parameters are available. For details see the MarkLogic REST API Reference.

When applying an aggregate UDF, the output is dependent on the UDF. Aggregate UDFs return a sequence of items, which can be atomic values or key-value maps. For details, see User-Defined Functions in the Application Developer's Guide and the XQuery builtin function cts:aggregate.

Example: Applying a Builtin Aggregate Function

The following example counts the number of values in the index identified as 'speaker' in 'index-options' (options=index-options). The counted values include only those in fragments containing 'HAMLET' (q=HAMLET). The output should contain only the aggregate result (view=aggregate).

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    'http://localhost:8003/v1/values/speaker?options=index-options&aggregate=count&view=aggregate&q=HAMLET'
...
<values-response name="speaker" type="xs:string"
    xmlns="http://marklogic.com/appservices/search"
    xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <aggregate-result name="count">92</aggregate-result>
  <metrics>
    <aggregate-resolution-time>PT0.001108S</aggregate-resolution-time>
    <total-time>PT0.003719S</total-time>
  </metrics>
</values-response>

Example: Applying an Aggregate UDF

This example demonstrates using an aggregate user-defined function to count the number of values in an element range index using /values/{name}. The aggregate UDF is specified via request parameters, as described in Using Request Parameters to Apply Aggregate Functions.

This example assumes the following pre-requisites are already met:

  • A native plugin is installed with the path 'native/sampleplugin'.
  • The plugin implements a 'count' function that counts the values in a range index. That is, a function equivalent to the 'count' builtin aggregate function.
  • Query options are installed with the name 'index-options'.
  • The query options include a range specification named 'speaker' for the SPEAKER element range index.

The following command uses the 'count' aggregate UDF to count the number of values in the SPEAKER index in fragments containing 'HAMLET'. The output only contains the aggregate result (view=aggregate).

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    'http://localhost:8003/v1/values/speaker?options=index-options&aggregate=count&aggregatePath=native/sampleplugin&view=aggregate&q=HAMLET'

The use of view=aggregate limits the output to only the aggregate results, as shown below. XML is the default output format. Use the format parameter or the HTTP Accept headers to request JSON output.

Format Example Output
XML
<values-response name="speaker" type="xs:string"
    xmlns="http://marklogic.com/appservices/search"
    xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <aggregate-result name="count">92</aggregate-result>
  <metrics>
    <aggregate-resolution-time>PT0.001514S</aggregate-resolution-time>
    <total-time>PT0.004049S</total-time>
  </metrics>
</values-response>
JSON
{
  "values-response": {
    "name": "speaker",
    "type": "xs:string",
    "aggregate-result": [
      {
        "name": "count",
        "_value": "92"
      }
    ],
    "metrics": {
      "aggregate-resolution-time": "PT0.001336S",
      "total-time": "PT0.004104S"
    }
  }
}

Specifying Dynamic Query Options with Combined Query

A combined query is an XML or JSON wrapper around a string and/or structured query and query options. Use a combined query to specify query options on the fly, without first persisting them as named options using the /config/query service. Combined queries are useful for rapid prototying during development, and for applications that need to modify query options on a per query basis.

To use a combined query, send a POST request to /v1/search or /v1/values/{name} with the combined query in the request body. See the following topics for more details:

Syntax

A combined query can contain a string query, a structured query, query options, or any combination of these. For example, you can create a combined query that contains only query options, only a structured query, or a structured query, string query and query options. If the combined query contains both a structured query and a string query, the two sub-queries are AND'd together.

The following table shows the structure of a combined query. Within the combined query wrapper, the queries and options use the same syntax as when they occur standalone.

Format Combined Query Template
XML
<search xmlns="http://marklogic.com/appservices/search">
  <query>
    <!-- structured query, same syntax as standalone -->
  </query>
  <qtext>your string query</qtext>
  <options>
    <!-- same syntax as standalone query options -->
  </options>
</search>
JSON
{"search":
  "query": { structured query, same syntax as standalone },
  "qtext": { "your string query here" }
  "options": { same syntax as standalone query options },
} 

For details on sub-query and query options syntax, see the following sections of the Search Developer's Guide:

Interaction with Queries in Request Parameters

When making a POST request to /v1/search or /v1/values/{name}, you can use a combined query in conjunction with the q request parameter. The string query in the request parameter value is AND'd with the sub-query(s) in the combined query.

The following table summarizes the interaction between the q request parameter and the sub-queries of a combined query. Note that the 'queries' in the table are an abstraction rather than actual example queries.

Request Parameter Combined Query Final Query
q=query-rp
<search>
  <query>query-cq</query>
</search>
query-rp AND query-cq
q=query-rp
<search>
  <qtext>query-cq</qtext>
</search>
query-rp AND query-cq
q=query-rp
<search>
  <query>query-cq1</query>
  <qtext>query-cq2</qtext>
</search>
query-rp AND query-cq1 AND query-cq2
none
<search>
  <query>query-cq1</query>
  <qtext>query-cq2</qtext>
</search>
query-cq1 AND query-cq2

Interaction with Persistent Query Options

Dynamic query options supplied in a combined query are merged with persistent and default options that are in effect for the search. If the same non-constraint option is specified in both the combined query and persistent options, the setting in the combined query takes precedence.

Constraints are overridden by name. That is, if the dynamic and persistent options contain a <constraint/> element with the same @name, the definition in the dynamic query options is the one that applies to the query. Two constraints with different name are both merged into the final options.

For example, suppose the following query options are installed under the name my-options:

<options xmlns="http://marklogic.com/appservices/search">
  <fragment-scope>properties</fragment-scope>
  <return-metrics>false</return-metrics>
  <constraint name="same">
    <collection prefix="http://server.com/persistent/"/>
  </constraint>
  <constraint name="not-same">
    <element-query name="title" ns="http://my/namespace" />
  </constraint>
</options>

Further, suppose you submit a POST /v1/search request that uses my-options and includes the following query options in a combined query in the request body:

$ cat body.xml
<search xmlns="http://marklogic.com/appservices/search">
  <options>
    <return-metrics>true</return-metrics>
    <debug>true</debug>
  <constraint name="same">
    <collection prefix="http://server.com/dynamic/"/>
  </constraint>
    <constraint name="different">
      <element-query name="scene" ns="http://my/namespace" />
    </constraint>
  </options>
</search>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X POST \
    -d@./body.xml -H "content-type: application/xml" \
    'http://localhost:8003/v1/search?q=TRAGEDY&options=my-options'

The query is evaluated with the following merged options. The persistent options contribute the fragment-scope option and the constraint named not-same. The dynamic options in the combined query contribute the return-metrics and debug options and the constraints named same and different. The return-metrics setting and the constraint named same from my-options are discarded.

<options xmlns="http://marklogic.com/appservices/search">
  <fragment-scope>properties</fragment-scope>
  <return-metrics>true</return-metrics>
  <debug>true</debug>
  <constraint name="same">
    <collection prefix="http://server.com/dynamic/"/>
  </constraint>
    <constraint name="different">
      <element-query name="scene" ns="http://my/namespace" />
    </constraint>
  </options>
  <constraint name="not-same">
    <element-query name="title" ns="http://my/namespace" />
  </constraint>
</options>

Performance Considerations

Using persistent query options usually performs better than using a combined query. In most cases, the difference between the two approaches is slight.

When MarkLogic Server processes a combined query, the per request query options must be parsed and merged with named and default options on every search. When you only use persistent named or default query options, you reduce this overhead.

If your application does not require dynamic per-request query options, you should use the /config/query/{name} service to persist your options under a name and use the options request parameter to associate the options with a simple string, structured, or values query. For details, see Configuring Query Options.

Combined Query Examples

This section includes the following examples:

Example: Overriding Persistent Constraints

The following example uses a bucketed constraint, backed by an element range index, to group items into facets by price. Assume following options that define price range buckets are stored as persistent options using the /config/query/{name} service, as described in Creating or Modifying Query Options.

<options xmlns="http://marklogic.com/appservices/search">
 <constraint name="price" facet="true">
    <range type="xs:int">
    <element ns="" name="price"/>
    <bucket name="under50" ge="0" lt="50">under $50</bucket>
    <bucket name="under100" ge="50" lt="101">$50-$100</bucket>
    <bucket name="over100" ge="101">over $100</bucket>
  </range>
 </constraint>
</options>

The application can use these persistent options to generate a faceted navigation page that allows users to browse by price. You can use dynamic query options to render a page that includes a custom facet from a price range entered by the user. The resulting combined query might look like the following if the user defined a price range of $100-150:

<search xmlns="http://marklogic.com/appservices/search">
  <options>
    <constraint name="price" facet="true">
      <range type="xs:int">
        <element ns="" name="price"/>
        <bucket name="under50" ge="0" lt="50">under $50</bucket>
        <bucket name="under100" ge="50" lt="101">$50-$100</bucket>
        <bucket name="over100" ge="101">over $100</bucket>
        <bucket name="custom" ge="100" lt="151">$100 to $150</bucket>
      </range>
    </constraint>
  </options>
  <query>
    <range-constraint-query>
      <constraint-name>price</constraint-name>
      <value>custom</value>
    </range-constraint-query>
  <query>
</search>
Example: Modifying the Search Response

The following example uses a combined query that contains only query options to enable the return-query option on the fly to explore how a string query is represented as a cts:query after parsing. The return-query setting in the dynamic options overrides any return-query setting in the default options. All other settings in the default options are unchanged and apply to the query evaluation.

$ cat combo-query.xml
<search xmlns="http://marklogic.com/appservices/search">
  <options>
    <return-query>true</return-query>
  </options>
</search>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X POST \
    -d@./combo-query.xml -H "content-type: application/xml" \
    'http://localhost:8003/v1/search?q=Horatio NEAR Yorick'
<search:response snippet-format="snippet" total="0" start="1" ...> 
  <search:qtext>Horation NEAR Yorick</search:qtext>
  <search:query>
    <cts:near-query qtextjoin="NEAR" strength="30" ...>
      <cts:word-query qtextref="cts:text">
        <cts:text>Horatio</cts:text>
      </cts:word-query>
      <cts:word-query qtextref="cts:text">
        <cts:text>Yorick</cts:text>
      </cts:word-query>
    </cts:near-query>
  </search:query>
  ...
</search:response>

For details, see Interaction with Persistent Query Options.

Configuring Query Options

Use the /config/query resources to install, list, and manage sets of query options. Use the options request parameter to apply installed query options to requests to /search, /keyvalue, and /values. This section covers the following topics:

Controlling Queries With Options

Use the options request parameter to the /search, /values, and /keyvalue services to customize your queries. Query options provide capabilities such as:

  • define word, value and element constraints
  • define lexicon and range index specifications
  • control search characteristics such as case sensitivity and ordering
  • extend the search grammar
  • customize query results including pagination, snippeting, and filtering

For details on these and other options, see Appendix A: JSON Query Options Reference or Appendix B: XML Query Options Reference.

MarkLogic Server comes configured with default query options. You can extend and modify the default options using /config/query/default.

You can also create custom query options. Before you can use a set of custom query options, you must install them as named options, using /config/query/{name}. For details, see Configuring Query Options.

Once you install query options under a name, you can apply them to a /search, /keyvalue, or /values request using the options request parameter. The following example searches for the word 'julius', using the query options named 'my-options':

http://localhost:8003/v1/search?q=julius&options=my-options

Adding Query Options to a Request

You can customize a query with query options in the following ways:

  • Use the options request parameter of a GET request to /search, /keyvalue, or /values/{name} to supply the name of pre-installed persistent options.
  • Use the options parameter of a POST request to /search or /values/{name} to supply the name of pre-installed persistent options.
  • Use the options element (XML) or sub-object (JSON) of a combined query passed in the body of a POST request to /search or /values/{name} to supply dynamic options.

Pre-installed, persistent query options usually provide better performance. Using dynamic query options introduces option parsing and merging overhead to every query.

Both persistent and dynamic query options can be specified in either XML or JSON. Persistent options must be installed before you can use them; for details, see Creating or Modifying Query Options.

Dynamic options are only usable with services that support POSTing a combined query in the request body. Methods that support combined query allow you to specify persistent and dynamic options in the same request. Where both are present, they are merged; in case of a conflict, a dynamic option setting overrides a persistent option setting. For details, see Specifying Dynamic Query Options with Combined Query.

Creating or Modifying Query Options

To install or modify a set of query options, send a PUT or POST request to the /config/query service with a URL of the form:

http://host:port/version/config/query/name

When constructing the request:

  1. Set the name portion of the URL to a unique name for these options, or to default. Use the name to identify the query options in subsequent request, as described in Controlling Queries With Options.
  2. Place the XML or JSON option data in the request body.

For syntax details, see Appendix A: JSON Query Options Reference or Appendix B: XML Query Options Reference.

  1. Specify the MIME type of the body content in the format parameter or the HTTP Content-type header, as described in Controlling Input Content Type. You may only send XML or JSON. The default format is XML.

When choosing the HTTP verb, consider the following:

  • To install new query options, use either PUT or POST.
  • To replace the named query options, use PUT.
  • To add options to the named query options, use POST. Any options not included in the payload remain unchanged. New options are added.

If query option validation is enabled, the request will fail if it results in invalid query options. If the request fails, the options are unchanged. Option validation is enabled by default. You can disable option validation using the validate-options instance configuration property. For details, see Configuring Instance Properties.

The following example installs query options named 'title-only' that define a search constraint named 'title'. The constraint limits queries to terms appearing in a TITLE element. The query options are then used to find occurrences of 'julius' in TITLE elements:

$ cat bill-options.txt
<options xmlns="http://marklogic.com/appservices/search">
  <constraint name="title">
    <word>
      <element ns="" name="TITLE" />
    </word>
  </constraint>
</options>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -T './bill-options.txt' \
    -H "Content-type: application/xml" \
    http://localhost:8003/v1/config/query/title-only
...
HTTP/1.1 204 Content Updated
$ curl --anyauth --user user:password -X GET \
  'http://localhost:8003/v1/search?q=title:julius&options=title-only'
...
<search:response total="1" start="1" page-length="10" 
   xmlns="" xmlns:search="http://marklogic.com/appservices/search">
    ...
</search:response>

To add case-sensitivity to the query options installed above, this example sends a POST request to /config/query. The body content type, JSON, is given via the format request parameter instead of the Content-type header.

$ cat add-cs.json
{
  "options":
    { "term":
      { "term-option":"case-sensitive" }
    }
}
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X POST -d@'add-cs.json' http://localhost:8003/v1/config/query/title-only?format=json
...
HTTP/1.1 201 Content Created

To confirm the change, the modified option is fetched using a GET request:

$ curl --anyauth --user user:password -X GET http://localhost:8003/v1/config/query/title-only
...
<options xmlns="http://marklogic.com/appservices/search">
  <constraint name="title">
    <word>
      <element ns="" name="TITLE"/>
    </word>
  </constraint>
  <term>
    <term-option>case-sensitive</term-option>
  </term>
</options>

Creating or Modifying One Option

To add or modify just one setting in a set of query options, send a PUT or POST request to the /config/query service with a URL of the form:

http://host:port/version/config/query/name/option_name

When constructing the request:

  1. Set the name portion of the URL to the name of the enclosing query options, or to default.
  2. Set the option_name portion of the URL to a query option name, such as constraint or term.
  3. Place the XML or JSON option data in the request body. The data should be an options node (XML) or map (JSON) that includes the option named in the URL.
  4. Specify the content type of the body in the format parameter or the HTTP Content-type header, as described in Controlling Input Content Type. You may only send XML or JSON. The default format is XML.

The option_name portion of the URL must be the name of an option that can appear as an immediate child of a query options XML node or JSON object. Finer grained access is not supported. For details on query options names and structure, see Appendix A: JSON Query Options Reference or Appendix B: XML Query Options Reference.

When choosing the HTTP verb, consider the following:

  • To add a new option to the set, use either PUT or POST.
  • To replace all existing options of the same name, use PUT.
  • To add a new occurrence of an existing option that can appear multiple times, use POST. For example, query options can contain multiple <constraint/> elements.

If query option validation is enabled, the request will fail if it results in invalid query options. If the request fails, the options are unchanged. Option validation is enabled by default. You can disable option validation using the validate-options instance configuration property. For details, see Configuring Instance Properties.

Checking Index Availability

Some query options require the database configuration to include supporting indexes. For example, if your query options contain a range constraint, then you can only use those options on a database whose configuration includes a corresponding range index.

You can use the /config/indexes service to compare query options to the database configuration and get a report on whether or not all required indexes are present. For missing indexes, the report includes information to help create the missing index. You can either check all query options configurations, or a particular one (by name).

To check all query options configurations, send a GET request to the /config/indexes service of the form:

http://host:port/version/config/indexes

To check a specific query options configuration, send a GET request to the /config/indexes/{name} service of the form:

http://host:port/version/config/indexes/name

Where name is the name under which the options were installed using the /config/query/{name} service, as described in Creating or Modifying Query Options.

You can request an index report in XML, JSON, or HTML, using either the format request parameter or the HTTP Accept headers. The HTML report is a user-friendly report that contains more details.

For example, suppose the following query options are installed under the name 'tuples'. These options require 2 range indexes: An element range index on <SPEAKER/> and a path range index on the XPath expression /PLAY/ACT/SCENE/TITLE:

<options xmlns="http://marklogic.com/appservices/search">
  <tuples name="speaker-title">
    <range type="xs:string">
      <element ns="" name="SPEAKER"/>
    </range>
    <range type="xs:string">
      <path-index>/PLAY/ACT/SCENE/TITLE</path-index>
    </range>
  </tuples>
</options>

The following command requests an index check check report in XML for the 'tuples' options. Use the format parameter or the Accept headers to request a different report format.

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    http://localhost:8003/v1/config/indexes/tuples?format=xml

The table below shows the generated report. Notice that the report indicates that the index configuration is not complete, and shows which query option is not complete. You can use the path-index value to create the required path range index.

Format Example Output
XML
<rapi:index-summaries xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:index-count>1</rapi:index-count>
  <rapi:complete>false</rapi:complete>
  <rapi:index-summary>
    <rapi:name>/v1/config/query/tuples</rapi:name>
    <rapi:complete>false</rapi:complete>
    <range type="xs:string"
        xmlns="http://marklogic.com/appservices/search">
      <path-index>/PLAY/ACT/SCENE/TITLE</path-index>
    </range>
  </rapi:index-summary>
</rapi:index-summaries>
JSON
{
  "index-summaries": {
    "index-summary": [
      {
        "name": "\/v1\/config\/query\/tuples",
        "complete": "false",
        "range": {
          "type": "xs:string",
          "path-index": "\/PLAY\/ACT\/SCENE\/TITLE"
        }
      }
    ],
    "index-count": "1",
    "complete": "false"
  }
}

Retrieving Options

To retrieve previously installed query options, send a GET request to the /config/query service with a URL of the form:

http://host:port/version/config/query/name

Where name is the name of the query options.

MarkLogic Server responds with the contents of the named query options, as XML or JSON. Use the format request parameter or Accept header to control the response format. XML is the default format. For details, see Controlling Output Content Type.

You can also retrieve the settings for a specific option within the query options by sending a GET request with a URL of the form:

http://host:port/version/config/query/name/option_name

Where option_name is an option name. For details, see Appendix A: JSON Query Options Reference or Appendix B: XML Query Options Reference.

As when retrieving a whole set, results can be requested as XML or JSON, using the format parameter or Accept header. For details, see Controlling Output Content Type. No version information is returned when examining only one option.

The following example retrieves the contents of the query options called 'title-only':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    http://localhost:8003/v1/config/query/title-only
...
<options xmlns="http://marklogic.com/appservices/search">
  <constraint name="title">
    <word>
      <element ns="" name="TITLE"/>
    </word>
  </constraint>
  <term>
    <term-option>case-sensitive</term-option>
  </term>
</options>

The following example retrieves only the term option of the 'title-only' query options:

$ curl --anyauth --user user:password -X GET \
    http://localhost:8003/v1/config/query/title-only/term
...
<options xmlns="http://marklogic.com/appservices/search">
  <term>
    <term-option>case-sensitive</term-option>
  </term>
</options>

Retrieving a List of Installed Query Options

To retrieve a list of the names of all installed query options, send a GET request to /config/query with a URL of the form:

http://host:port/version/config/query/name

MarkLogic Server responds with a list of names in XML or JSON. XML is the default format. Use the format request parameter or the HTTP Accept header to specify the response format, as described in Controlling Output Content Type.

If there are no custom query options installed, MarkLogic Server responds with an empty XML <options> node or JSON array.

The following example retrieves the list of named query options as XML. The results show 2 sets of query options, named 'title-only' and 'play-type'.

$ curl --anyauth --user user:password -X GET \
    http://localhost:8003/v1/config/query
...
<rapi:query-options xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:options>
    <rapi:name>title-only</rapi:name>
    <rapi:uri>/v1/config/query/title-only</rapi:uri>
  </rapi:options>
  <rapi:options>
    <rapi:name>play-type</rapi:name>
    <rapi:uri>/v1/config/query/play-type</rapi:uri>
  </rapi:options>
</rapi:query-options>

The following example requests the same information as JSON, using the format request parameter:

$ curl --anyauth --user user:password -X GET \
    http://localhost:8003/v1/config/query
...
[
  {"name":"title-only","uri":"/v1/config/query/title-only"}
  {"name":"play-type","uri":"/v1/config/query/play-type"}
]

Removing Query Options

You can remove a single setting, one set of query options, or all query options, as described in the following topics:

Removing Query Options

To remove the query options installed under a particular name, send a DELETE request to the /config/query service with a URL of the form:

http://host:port/version/config/query/name

Where name is the name of the query options to remove. MarkLogic Server responds with 204 if the option set is successfully deleted.

Removing a Single Option

To remove a specific setting in a set of query options, send a DELETE request to the /config/query service with a URL of the form:

http://host:port/version/config/query/name/option_name

Where option_name is the name of the option to remove and name is the name of the containing query options. MarkLogic Server responds with 204 if the option is successfully deleted.

The option_name portion of the URL must be the name of an option that can appear as an immediate child of a query options XML node or JSON object. Finer grained access is not supported. All options with this name are removed. For example, if the query options named 'my-options' contain multiple <constraint/> options, then the following request removes all of them:

DELETE http://localhost:8003/v1/config/query/my-options/constraint

For details on query options names and structure, see Appendix A: JSON Query Options Reference or Appendix B: XML Query Options Reference.

If query option validation is enabled, the request will fail if it results in invalid query options. If the request fails, the options are unchanged. Option validation is enabled by default. You can disable option validation using the validate-options instance configuration property. For details, see Configuring Instance Properties.

For details on query option names, see Appendix A: JSON Query Options Reference or Appendix B: XML Query Options Reference.

Removing All Named Query Options

To remove all named query options from a MarkLogic REST API instance, send a DELETE request to the /config/query service with a URL of the form:

http://host:port/version/config/query

MarkLogic Server responds with 204 if the options successfully deleted.

Using Namespace Bindings

This sections covers the following topics:

When Do You Need a Namespace Binding

Use the /config/namespaces service to define namespace prefixes that you can use in your queries. For example, to specify an element with QName theNS:theElement in a /keyvalue search, you might refer to it as follows:

http://host:port/version/keyvalue?element=theNS:theElement&value=foo

There is no way to dynamically specify the namespace in your request, so you must pre-define the namespace prefix binding using /config/namespaces or /config/namespace/{name}.

Creating or Updating a Namespace Binding

This section describes how to create a single namespace binding. You can also define multiple bindings in a single request; for details, see Creating or Updating Multiple Namespace Bindings.

To define a namespace binding, send a PUT request to the /config/namespaces/{name} service with a URL of the form:

http://host:port/version/config/namespaces/name

Where name is the namespace prefix you want to define. If a binding already exists for this prefix, it is replaced.

When constructing your request:

  1. Set the name portion of the URL to the desired namespace prefix.
  2. Place the XML or JSON binding definition in the request body. See the table below.
  3. Specify the content type of the body in the format parameter or the HTTP Content-type header, as described in Controlling Input Content Type. You may only send XML or JSON. The default format is XML.

The request body must have the following form:

Format Body
XML
<namespace xmlns="http://marklogic.com/rest-api">
  <prefix>the_prefix</prefix>
  <uri>the_uri</uri>
</namespace>
JSON
{
  "prefix" : "the_prefix",
  "uri" : "the_uri"
}

The following example binds the prefix 'bill' to the namespace URI 'http://marklogic.com/examples/shakespeare':

$ cat ns-binding.xml
<namespace xmlns="http://marklogic.com/rest-api">
  <prefix>bill</prefix>
  <uri>http://marklogic.com/examples/shakespeare</uri>
</namespace>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT -d@./ns-binding.xml \
    -H "Content-type: application/xml" \
    http://localhost:8004/v1/config/namespaces/bill

You can examine the binding by sending a GET request to /config/namespaces/{name} or to /config/namespaces. The latter lists all bindings. For example:

$  curl --anyauth --user user:password -X GET \
    http://localhost:8004/v1/config/namespaces/bill
<rapi:namespace xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:prefix>bill</rapi:prefix>
  <rapi:uri>http://marklogic.com/examples/shakespeare</rapi:uri>
</rapi:namespace>

To get the equivalent output as JSON, use the format parameter or specify application/json in the HTTP Accept header. For example:

$  curl --anyauth --user user:password -X GET \
    http://localhost:8004/v1/config/namespaces/bill?format=json
{
  "prefix":"bill",
  "uri":"http://marklogic.com/examples/shakespeare"
}

Creating or Updating Multiple Namespace Bindings

This section describes how to define multiple namespace bindings with a single request. You can also define a specific single binding; for details, see Creating or Updating a Namespace Binding.

To define multiple namespace bindings, send a PUT or POST request to the /config/namespaces service with a URL of the form:

http://host:port/version/config/namespaces

When constructing your request:

  1. Choose PUT to replace all bindings with those the request body. Choose POST to append to existing bindings.
  2. Place the XML or JSON binding definitions in the request body. See the table below.
  3. Specify the content type of the body in the format parameter or the HTTP Content-type header, as described in Controlling Input Content Type. You may only send XML or JSON. The default format is XML.

If you use POST and a binding already exists for one defined in the request body, MarkLogic Server returns status 400 (Bad Request).

The request body must have the following form:

Format Body
XML
<namespace-bindings xmlns="http://marklogic.com/rest-api">
  <namespace>
    <prefix>a_prefix</prefix>
    <uri>a_namespace_uri</uri>
  </namespace>
  <namespace>
    <prefix>another_prefix</prefix>
    <uri>another_namespace_uri</uri>
  </namespace>
</namespace-bindings>
JSON
{
  "namespace-bindings": [
    {
      "prefix": "a_prefix",
      "uri": "a_namespace_uri"
    },
    {
      "prefix": "another_prefix",
      "uri": "another_namespace_uri"
    }
  ]
}

The following example binds the prefix 'one' to the namespace URI 'http://marklogic.com/examples/one' and the prefix 'two' to the namespace URI 'http://marklogic.com/examples/two':

$ cat ns-bindings.xml
<namespace-bindings xmlns="http://marklogic.com/rest-api">
  <namespace>
    <prefix>one</prefix>
    <uri>http://marklogic.com/examples/one</uri>
  </namespace>
  <namespace>
    <prefix>two</prefix>
    <uri>http://marklogic.com/examples/two</uri>
  </namespace>
</namespace-bindings>
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT -d@./ns-bindings.xml \
    -H "Content-type: application/xml" \
    http://localhost:8004/v1/config/namespaces

The following example creates equivalent bindings using JSON input:

$ cat ns-bindings.json
{
  "namespace-bindings": [
    {
      "prefix": "one",
      "uri": "http:\/\/marklogic.com\/examples\/one"
    },
    {
      "prefix": "two",
      "uri": "http:\/\/marklogic.com\/examples\/two"
    }
  ]
}
# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT -d@./ns-bindings.json \
    -H "Content-type: application/json" \
    http://localhost:8004/v1/config/namespaces

You can examine the binding by sending a GET request to /config/namespaces/{name} or to /config/namespaces.

Listing Available Namespace Bindings

To list all available namespace bindings, send a GET request to /config/namespaces with a URL of the form:

http://host:port/version/config/namespaces

To retrieve the binding for a single namespace prefix, send a GET request to /config/namespaces/{name} with a URL of the form:

http://host:port/version/config/namespaces/name

Where name is a bound namespace prefix.

You can request output as either XML or JSON using the format request parameter or the Accept headers. For details, see Controlling Output Content Type.

The following example command requests all namespace bindings, as XML:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    http://localhost:8004/v1/config/namespaces?format=xml

The output from the command is shown in the table below, assuming two namespace bindings are installed, for the prefixes 'one' and 'two'.

Format Example Output
XML
<rapi:namespace-bindings
    xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:namespace>
    <rapi:prefix>one</rapi:prefix>
    <rapi:uri>http://marklogic.com/examples/one</rapi:uri>
  </rapi:namespace>
  <rapi:namespace>
    <rapi:prefix>two</rapi:prefix>
    <rapi:uri>http://marklogic.com/examples/two</rapi:uri>
  </rapi:namespace>
</rapi:namespace-bindings>
JSON
{
  "namespace-bindings": [
    {
      "prefix": "one",
      "uri": "http://marklogic.com/examples/one"
    },
    {
      "prefix": "two",
      "uri": "http://marklogic.com/examples/two"
    }
  ]
}

When you use GET /config/namespaces/{name}, the output is similar, but without the namespace-bindings wrapper. For example to retrieve the binding for the 'one' prefix:

$ curl --anyauth --user user:password -X GET \
    http://localhost:8004/v1/config/namespaces/one?format=xml
<rapi:namespace xmlns:rapi="http://marklogic.com/rest-api">
  <rapi:prefix>one</rapi:prefix>
  <rapi:uri>http://marklogic.com/examples/one</rapi:uri>
</rapi:namespace>

The following command retrieves the same information as JSON:

$ curl --anyauth --user user:password -X GET \
    http://localhost:8004/v1/config/namespaces/one?format=json
{"prefix":"one","uri":"http://marklogic.com/examples/one"}

Deleting Namespace Bindings

To remove all namespace bindings, send a DELETE request to /config/namespaces with a URL of the form:

http://host:port/version/config/namespaces

To remove a specific binding, send a DELETE request to /config/namespaces/{name} with a URL of the form:

http://host:port/version/config/namespaces/name

Where name is a bound namespace prefix. If no binding exists for name, MarkLogic Server returns status 404 (Not Found).

The following example deletes just the binding for the prefix 'one':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X DELETE \
    http://localhost:8004/v1/config/namespaces/one

Generating Search Facets

The MarkLogic Server Search API enables you to expose search facets in your application. Facets enable users to filter search results by narrowing down the search criteria. To learn more about facets, see Constrained Searches and Faceted Navigation in the Search Developer's Guide.

To generate facet information in search results, use query options that include constraints that support facets, such as collection and element constraints. You can also define custom constraints; see Creating a Custom Constraint in the Search Developer's Guide.

The following example returns facet information about play types and enables searching by the facet 'type', assuming the documents in the database have been added to the collections /play-type/Comedy, /play-type/Tragedy, and /play-type/History. For more examples, see Constraint Options in the Search Developer's Guide.

This example uses a collection constraint. To search on a collection facet, you must enable the 'collection lexicon' database configuration parameter in the Admin Interface.

To enable the play type facet, create query options that define a collection constraint:

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
  <constraint name="type">
    <collection prefix="/play-type/"/>
  </constraint>
  <return-facets>true</return-facets>
</options>
JSON
{
  "options": {
    "constraint": [
      {
        "name": "type",
        "collection": { "prefix": "\/play-type\/" }
      } ],
    "return-facets": true
  }
}

Install the options with the name 'facet-options':

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    -d@"./facet-options.xml" -H "Content-type: application/xml" \
    http://localhost:8003/v1/config/query/facet-options

If you query /search or /keyvalue using 'index-options' as the query options, facet results are included in the search results. For example, the following query finds all occurrences of 'castle' and the search response includes a count of the matches for each play type:

$ curl --anyauth --user user:password -X GET \
  'http://localhost:8003/v1/search?options=facet-options&q=castle'
...
<search:response snippet-format="snippet" total="88" ...>
  <search:result />
  <search:facet name="type" type="collection">
    <search:facet-value name="Comedy" count="5">
      Comedy
    </search:facet-value>
    <search:facet-value name="History" count="35">
      History
    </search:facet-value>
    <search:facet-value name="Tragedy" count="47">
      Tragedy
    </search:facet-value>
  </search:facet>
...
</search:response>

To get the equivalent results as JSON, use format=json or include application/json in the HTTP Accept header. For example:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
  'http://localhost:8003/v1/search?options=facet-options&q=castle&format=json'
...
{
  "snippet-format": "snippet",
  "total": 88,
  "start": 1,
  "page-length": 10,
  "results": [...],
  "facets": {
    "type": {
      "type": "collection",
      "facetValues": [
        {
          "name": "Comedy",
          "count": 5
        },
        {
          "name": "History",
          "count": 35
        },
        {
          "name": "Tragedy",
          "count": 47
        }
      ]
    }
  },
  "qtext": "castle",
  ...
}

You can query by facet by including a facet-name:facet-value search term. The following command matches occurrences of 'castle' in plays of type Comedy:

$ curl --anyauth --user user:password -X GET \
    'http://localhost:8003/v1/search?options=facet-options&q=castle type:Comedy'

Paginating Results

When you query the database using /search and /keyvalues, you can paginate the query results using the start and pageLength request parameters. Use start to specify the index of the first result to return, and pageLength to control the number results to return.

By default, queries return the first 10 results. That is, the default start position is 1 and the default page length is 10. You can fetch successive non-overlapping pages of results by incrementing the start position by the page length in each call.

For more information, see the Search Developer's Guide and Fast Pagination and Unfiltered Searches in the Scalability, Availability, and Failover Guide.

The following example command fetches the first 5 results matching 'castle'. Notice that the search response includes the total number of matches and each search:result includes an index.

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    'http://localhost:8003/v1/search?q=castle&start=1&pageLength=5'
...
<search:response snippet-format="snippet" total="88"    start="1" page-length="5" ...>
  <search:result index="1" uri="/shakespeare/plays/hen_vi_2.xml" .../>
  <search:result index="2" uri="/shakespeare/plays/rich_ii.xml" .../>
  <search:result index="3" uri="/shakespeare/plays/macbeth.xml" .../>
  <search:result index="4" uri="/shakespeare/plays/hen_vi_3.xml" .../>
  <search:result index="5" uri="/shakespeare/plays/othello.xml" .../>
...
</search:response>

To fetch the next 5 results, increment start by 5:

$ curl --anyauth --user user:password -X GET \
    'http://localhost:8003/v1/search?q=castle&start=6&pageLength=5'
...
<search:response snippet-format="snippet" total="88" 
    start="1" page-length="5" ...>
  <search:result index="6" uri="/shakespeare/plays/lear.xml" .../>
  <search:result index="7" uri="/shakespeare/plays/hamlet.xml" .../>
  <search:result index="8" uri="/shakespeare/plays/rich_iii.xml" .../>
  <search:result index="9" uri="/shakespeare/plays/hen_v.xml" .../>
  <search:result index="10" uri="/shakespeare/plays/m_wives.xml" .../>
...
</search:response>

Customizing Search Snippets

Search results usually include portions of matching documents with the search matches highlighted, perhaps with some text showing the context of the search matches. These search result pieces are known as snippets. MarkLogic Server has a default search snippet format, but you can customize the snippet format in two ways:

  • Use query options to customize the results returned by the default snippeting function.
  • Write your own snippeting function in XQuery and install it as an extension.

Customizing Search Snippets With Query Options

MarkLogic Server creates snippets by applying a default transformation to search results. You can modify the results of the default transformation using the transform-results query options, as described in Modifying Your Snippet Results in the Search Developer's Guide. To use this feature with the REST API:

  1. Create query options that contain a transform-results element (XML) or child object (JSON).
  2. Install these query options under a name or as the default options using the /config/query service. For details, see Configuring Query Options.
  3. If you installed the configuration as named query options, apply the options to your query by supplying the name in the options request parameter. For details, see Controlling Queries With Options.
  4. If you installed the configuration as default options, they will be automatically applied to search results returned by any query that does not use named query options.

The following example searches for the term 'hamlet' in a collection of Shakespeare plays using the default snippet options. The search results include four snippets in each match.

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/xml" \
    http://localhost:8003/v1/search?q=hamlet
<search:response snippet-format="snippet" total="20" ...>
  <search:result index="1" uri="/shakespeare/plays/hamlet.xml" ...>     <search:snippet>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/TITLE">The Tragedy of <search:highlight>Hamlet</search:highlight>, Prince of Denmark
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/PERSONAE/PERSONA[2]"><search:highlight>HAMLET</search:highlight>, son to the late, and nephew to the present king.
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/PERSONAE/PERSONA[4]">HORATIO, friend to <search:highlight>Hamlet</search:highlight>.
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/PERSONAE/PERSONA[16]">GERTRUDE, queen of Denmark, and mother to <search:highlight>Hamlet</search:highlight>. 
      </search:match>
    </search:snippet>
  </search:result>
...
</search:response>

Notice that the resulting snippets are from the <TITLE/> and <PERSONAE/> elements. If you apply the options below to the search instead, at most three snippets are returned and only matches in <LINE/> elements are included.

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
  <transform-results apply="snippet">
    <max-matches>3</max-matches>
    <preferred-elements>
      <element ns="" name="LINE" />
    </preferred-elements>
  </transform-results>
  <search-option>filtered</search-option>
</options>
JSON
{
  "options": {
    "transform-results": {
      "apply": "snippet",
      "max-matches": 3,
      "preferred-elements": [
        {
          "ns": "",
          "name": "LINE"
        }
      ]
    },
    "search-option": true
  }
}

This example installs the above XML options and applies them to the same query:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    -d @./transform-results.xml \
    -H "Content-type: application/xml" \
    http://localhost:8003/v1/config/query/snippet-lines
$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/xml" \
    "http://localhost:8003/v1/search?q=hamlet&options=snippet-lines"
<search:response snippet-format="snippet" total="20" ...>
  <search:result index="1" uri="/shakespeare/plays/hamlet.xml" ...>
    <search:snippet>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[48]/LINE[6]">Dared to the combat; in which our valiant <search:highlight>Hamlet</search:highlight>--
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[48]/LINE[17]">His fell to <search:highlight>Hamlet</search:highlight>. Now, sir, young Fortinbras,
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[59]/LINE[6]">Unto young <search:highlight>Hamlet</search:highlight>; for, upon my life,
      </search:match>
    </search:snippet>
  </search:result>
...
</search:response>

To install the equivalent JSON options, set the Content-type header to application/json. For example:

$ curl --anyauth --user user:password -X PUT \
    -d @./transform-results.json \
    -H "Content-type: application/json" \
    http://localhost:8003/v1/config/query/snippet-lines

To receive JSON search results, set the Accept header to application/json or use the format request parameter. For example:

$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/json" \
    "http://localhost:8003/v1/search?q=hamlet&options=snippet-lines"
{
  "snippet-format": "snippet",
  "total": 20,
  "start": 1,
  "page-length": 10,
  "results": [
    {
      "index": 1,
      "uri": "\/shakespeare\/plays\/hamlet.xml",
      "path": "fn:doc(\"\/shakespeare\/plays\/hamlet.xml\")",
      "score": 158720,
      "confidence": 0.80079,
      "fitness": 1,
      "matches": [
        {
          "path": "fn:doc(\"\/shakespeare\/plays\/hamlet.xml\")\/PLAY\/ACT[1]\/SCENE[1]\/SPEECH[48]\/LINE[6]",
          "match-text": [
            "Dared to the combat; in which our valiant ",
            {
              "highlight": "Hamlet"
            },
            "--"
          ]
        },
        {
          "path": "fn:doc(\"\/shakespeare\/plays\/hamlet.xml\")\/PLAY\/ACT[1]\/SCENE[1]\/SPEECH[48]\/LINE[17]",
          "match-text": [
            "His fell to ",
            {
              "highlight": "Hamlet"
            },
            ". Now, sir, young Fortinbras,"
          ]
        },
        {
          "path": "fn:doc(\"\/shakespeare\/plays\/hamlet.xml\")\/PLAY\/ACT[1]\/SCENE[1]\/SPEECH[59]\/LINE[6]",
          "match-text": [
            "Unto young ",
            {
              "highlight": "Hamlet"
            },
            "; for, upon my life,"
          ]
        }
      ]
    }
  ],
  ...
}

Creating Your Own Snippet Extension

If the tranform-results query options with the default snippet format does not meet the needs of your application, you can create a custom snippet transformation function, as described in Specifying Your Own Code in transform-results in the Search Developer's Guide.

Custom snippet transformations cannot be installed or applied directly using the MarkLogic REST API. Instead, you must manually install your transformation function in the modules database of your REST instance and apply by specifying it in custom query options. In particular, you cannot install your custom snippeting function using the /config/transforms service.

To create and use a custom snippeting function:

  1. Create an XQuery library module that implements your custom snippet function, as described in Specifying Your Own Code in transform-results in the Search Developer's Guide.
  2. Manually install your module in the modules database of your REST API instance, similar to installing a dependent library for a resource service extension. For details, see Installing Dependent XQuery Library Modules.
  3. Create and install query options that include a transform-results option that uses your custom snippeting function. For details, see Search Customization Via Options and Extensions in the Search Developer's Guide.
  4. Apply the options to your query by supplying the name of the options with the options request parameter. For details, see Controlling Queries With Options.

The following example builds on the example from Customizing Search Snippets With Query Options. The example searches for the term 'hamlet' in a collection of Shakespeare plays using the default snippet format with the custom options that extract the first three matches in lines of dialog. These options are installed under the name 'snippet-lines'. With the default snippet transformation function, the search produces the following results:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    -d @./transform-results.xml \
    -H "Content-type: application/xml" \
    http://localhost:8003/v1/config/query/snippet-lines
$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/xml" \
    "http://localhost:8003/v1/search?q=hamlet&options=snippet-lines"
<search:response snippet-format="snippet" total="20" ...>
  <search:result index="1" uri="/shakespeare/plays/hamlet.xml" ...>
    <search:snippet>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[48]/LINE[6]">Dared to the combat; in which our valiant <search:highlight>Hamlet</search:highlight>--
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[48]/LINE[17]">His fell to <search:highlight>Hamlet</search:highlight>. Now, sir, young Fortinbras,
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[59]/LINE[6]">Unto young <search:highlight>Hamlet</search:highlight>; for, upon my life,
      </search:match>
    </search:snippet>
  </search:result>
...
</search:response>

The custom snippeting function below returns the act, scene, speech and line number in of each match, instead of the text surrounding the search term.

xquery version "1.0-ml";

module namespace example = "http://marklogic.com/example";

import module namespace search =
  "http://marklogic.com/appservices/search"
  at "/MarkLogic/appservices/search/search.xqy";


declare function example:snippet(
   $result as node(),
   $ctsquery as schema-element(cts:query),
   $options as element(search:transform-results)?
) as element(search:snippet)
{
  let $default-snippet := search:snippet($result, $ctsquery, $options)
  return element
    { fn:QName(fn:namespace-uri($default-snippet),
               fn:name($default-snippet)) }
    { $default-snippet/@*,
      for $child in $default-snippet/node()
      return
        if ($child instance of element(search:match))
        then element {fn:QName(fn:namespace-uri($child),
                      fn:name($child)) } {
          $child/@*,
          let $parts :=
            fn:tokenize(
              fn:substring-after($child/@path, "/PLAY/ACT["), "\[")
          return
          text {fn:concat(
            "Act ", fn:substring-before($parts[1], "]"),
            ", Scene ", fn:substring-before($parts[2], "]"),
            ", Speech ", fn:substring-before($parts[3], "]"),
            ", Line ", fn:substring-before($parts[4], "]"))}
        }
        else $child
    }
};

If the above module is installed in the instance modules database with the URI /my.domain/my-snippets.xqy, then we can use the following options to generate custom snippets by supplying the localname of the custom function (snippet) in apply, the namespace of the module (http://marklogic.com/example) in ns, and the URI of the module in the instance modules database (/my.domain/my-snippets.xqy) in at.

Format Query Options
XML
<options xmlns="http://marklogic.com/appservices/search">
  <transform-results apply="snippet"
      ns="http://marklogic.com/example"
      at="/my.domain/my-snippets.xqy">
    <max-matches>3</max-matches>
    <preferred-elements>
      <element ns="" name="LINE" />
    </preferred-elements>
  </transform-results>
  <search-option>filtered</search-option>
</options>
JSON
{
  "options": {
    "transform-results": {
      "apply": "snippet",
      "ns": "http://marklogic.com/example",
      "at": "/my.domain/my-snippets.xqy",
      "max-matches": 3,
      "preferred-elements": [
        {
          "ns": "",
          "name": "LINE"
        }
      ]
    },
    "search-option": true
  }
}

This example installs the above XML options and applies them to the query for 'hamlet' to produce snippets that display the act, scene, speech and dialog number of each match:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X PUT \
    -d @./transform-results.xml \
    -H "Content-type: application/xml" \
    http://localhost:8003/v1/config/query/custom-snippet
$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/xml" \
    "http://localhost:8003/v1/search?q=hamlet&options=custom-snippet"
<search:response snippet-format="custom" total="20" start="1" ...>
  <search:result index="1" uri="/shakespeare/plays/hamlet.xml"
      path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)" ...>
    <search:snippet>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[48]/LINE[6]">Act 1, Scene 1, Speech 48, Line 6
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[48]/LINE[17]">Act 1, Scene 1, Speech 48, Line 17
      </search:match>
      <search:match
        path="fn:doc(&quot;/shakespeare/plays/hamlet.xml&quot;)/PLAY/ACT[1]/SCENE[1]/SPEECH[59]/LINE[6]">Act 1, Scene 1, Speech 59, Line 6
      </search:match>
    </search:snippet>
  </search:result>
  ...
</search:response>

To generate the equivalent results in JSON, change the Accept header to application/xml or use the format request parameter. For example:

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET \
    -H "Accept: application/json" \
    "http://localhost:8003/v1/search?q=hamlet&options=custom-snippet"
{
  "snippet-format": "custom",
  "total": 20,
  "start": 1,
  "page-length": 10,
  "results": [
    {
      "index": 1,
      "uri": "\/shakespeare\/plays\/hamlet.xml",
      "path": "fn:doc(\"\/shakespeare\/plays\/hamlet.xml\")",
      "score": 158720,
      "confidence": 0.80079,
      "fitness": 1,
      "matches": [
        {
          "path": "fn:doc(\"\/shakespeare\/plays\/hamlet.xml\")\/PLAY\/ACT[1]\/SCENE[1]\/SPEECH[48]\/LINE[6]",
          "match-text": [
            "Act 1, Scene 1, Speech 48, Line 6"
          ]
        },
        {
          "path": "fn:doc(\"\/shakespeare\/plays\/hamlet.xml\")\/PLAY\/ACT[1]\/SCENE[1]\/SPEECH[48]\/LINE[17]",
          "match-text": [
            "Act 1, Scene 1, Speech 48, Line 17"
          ]
        },
        {
          "path": "fn:doc(\"\/shakespeare\/plays\/hamlet.xml\")\/PLAY\/ACT[1]\/SCENE[1]\/SPEECH[59]\/LINE[6]",
          "match-text": [
            "Act 1, Scene 1, Speech 59, Line 6"
          ]
        }
      ]
    }
  ],
  ...
}

« Previous chapter
Next chapter »