MarkLogic Server 11.0 Product Documentation

Search Developer's Guide — Chapter 5

Search Criteria Based on Document Structure

A QBE uses search criteria expressed as XML elements, XML element attributes, or JSON properties that closely resemble portions of documents in the database.

For example, if the database contains documents of the following form:

XML

<book>
  <title>Tom Sawyer</title>
  <author>Mark Twain</author>
  <edition format="paperback"/>
</book>

JSON

"book": {
  "title": "Tom Sawyer",
  "author" : "Mark Twain",
  "edition": [
    { "format": "paperback" }
  ]
} }

Then you can construct a QBE to find all paperback books by a given author by creating criteria that model the author, edition format. The following QBE finds all paperback books by Mark Twain.

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <author>Mark Twain</author>
    <edition format="paperback"/>
  </q:query>
</q:qbe>

JSON

{"$query": {
    "author": "Mark Twain",
    "edition": {
      "format": "paperback"
    }
} }

By default, the literal values in criteria must exactly match document contents. That is, the above query matches if the author value is Mark Twain, but it will not match documents where the author is M. Twain or mark twain. You can change this behavior using word queries and options. For details, see Understanding QBE Sub-Query Types and Adding Options to a QBE.

You can construct criteria that express value, word, and range queries. For example, you can construct a QBE that satisfies all of the following criteria. The Example Criteria column shows an XML and a JSON criteria that expresses each requirement.

<author><q:word>twain</q:word></author>

"author": {"$word": "shakespeare"}

<edition format="paperback"/>

"edition": {"format": "paperback"}

<edition>
  <price><q:lt>9.00</q:lt></price>
</edition>

"edition": {"price": {"$lt": 9.00} }

When you combine the above criteria into a single query, you get the following QBE. Notice that the child elements of query are implicitly AND'd together.

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <author><q:word>twain</q:word></author>
    <edition format="paperback">
      <price><q:lt>9.00</q:lt></price>
    </edition>
    <q:filtered>true</q:filtered>
  </q:query>
</q:qbe>

JSON

{
  "$query": {
    "author": {
      "$word": "twain"
    },
    "edition": {
      "format": "paperback",
      "price": { "$lt": 9.00 }
    },
    "$filtered": true
  }
}

The above examples demonstrate searching for direct containment, such as the author is Mark Twain. You can also search for matches anywhere within a containing XML element or JSON property. For example, suppose a book contains author and editor names, broken down into first-name and last-name:

XML

<book>
  <author>
    <first-name>Mark</first-name>
    <last-name>Twain</last-name>
  </author>
  <editor>
    <first-name>Mark</first-name>
    <last-name>Matthews</last-name>
  </editor>
</book>

JSON

"book": {
  "author" : {
    "first-name": "Mark",
    "last-name": "Twain"
  },
"editor" : {
    "first-name": "Mark",
    "last-name": "Matthews"
  }
} }

You can search for any occurences of Mark as a first name contained by a book using criteria such as the following:

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <book><first-name>Mark</last-name></book>
  </q:query>
</q:qbe>

JSON

{"$query": {
    "book": { "first-name": "Mark" }
} }

Such criteria represent container queries. For details, see Container Query.

Logical Operators

You can use logical operators to create powerful composed queries. The QBE grammar supports and, or, not, and near composers. The following example matches documents that contain twain or shakespeare in the author XML element or JSON property.

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <q:or>
      <author><q:word>twain</q:word></author>
      <author><q:word>shakespeare</q:word></author>
    </q:or>
  </q:query>
</q:qbe>

JSON

{
  "$query" : {
    "$or": [
      { "author": {"$word": "twain" } },
      { "author": {"$word": "shakespeare" } }
    ]
  }
}

Sub-queries that are immediate children of query represent an implicit and query.

For details, see Composed Query.

Comparison Operators

The QBE grammar supports the following comparison operators for constructing range queries on XML element, XML attribute, and JSON property values: lt, le, eq, ne, ge, gt. For example, the following query matches all documents where the price is greater than or equal to 10.00 and less than or equal to 20.00.

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <price><q:ge>10.00</q:ge></price>
    <price><q:le>20.00</q:le></price>
    <q:filtered>true</q:filtered>
  </q:query>
</q:qbe>

JSON

{ "$query": {
    "$and": [
        {"price" : { "$ge": 10.00 } },
        {"price" : { "$le": 20.00 } } ],
    "$filtered": true
}}

The filtered flag is included in the above query because you must either use filtered search or back the range queries on price with a range index. For details, see How Indexing Affects Your Query.

For details, see Range Query.

In MarkLogic 10, a user must have both rest-reader and eval-search-string privileges to execute a QBE that uses relational comparisons (in particular, EQ, NE, LT, LE, GT, and GE) for elements that don't have a range index. The eval-search-string privilege is available in MarkLogic 9 so adopters can add the privilege to the users's role prior to rolling upgrade.

Query by Value or Word

When you construct a criteria on a literal value, it is an implicit value query that matches an exact value. For example, the following criteria matches only when author is Mark Twain. It will not match mark twainor M. Twain:

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <author>Mark Twain</author>
  </q:query>
</q:qbe>

JSON

{
  "$query": { "author": "Mark Twain" }
}

When this is not the desired behavior, you can use a word query and/or options to modify the default behavior. A word query differs from a value query in two ways: It relaxes the default exact match semantics of a value query, and it matches a subset of the value in a document.

For example, the following query matches if the author contains twain, with any capitalization, so it matches values that are not matched by the original query, such as Mark Twain, M. Twain and mark twain.

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <author><q:word>twain</q:word></author>
  </q:query>
</q:qbe>

JSON

{
  "$query": { "author": { "$word": "twain" } }
}

For details, see Value Query and Word Query.

Search Result Customization

You can include a response XML element or JSON property to customize the contents of returned search results. The default search results include a highlighted snippet of matching XML elements or JSON properties. Use the response section of a QBE to disable snippeting, extract additional elements, or return an entire document.

For details, see Customizing Search Results.

Options for Controlling Search Behavior

The QBE grammar includes several flags and options to control your search. Flags usually have a global effect on your search, such as how to score search results. Options affect a portion of your query, such as whether or not perform an exact match against a particular XML element or JSON property value.

The following example uses the exact option to disable exact matches on value queries.

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <author><q:value exact="false">mark twain</q:value></author>
  </q:query>
</q:qbe>

JSON

{
  "$query": {
    "author": { 
      "$exact": false,
      "$value" : "mark twain"
    }
  }
}

For more details, see Adding Options to a QBE.

XML Example

This example assumes the database contains documents with the following structure:

<book>
  <title>Tom Sawyer</title>
  <author>Mark Twain</author>
  <edition format="paperback">
    <publisher>Clipper</publisher>
    <pub-date>2011-08-01</pub-date>
    <price>9.99</price>
    <isbn>1613800917</isbn>
  </edition>
</book>

The following query uses most of the features of QBE and matches the above document. The sub-queries that are immediate children of query are implicitly AND'd together, so all these conditions must be met by matching documents.

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <title>
      <q:value exact="false">Tom Sawyer</q:value>
    </title>
    <q:near distance="2">
      <author><q:word>mark</q:word></author>
      <author><q:word>twain</q:word></author>
    </q:near>
    <edition format="paperback">
      <q:or>
        <publisher>Clipper</publisher>
        <publisher>Daw</publisher>
      </q:or>
    </edition>
    <q:and>
      <price><q:lt>10.00</q:lt></price>
      <price><q:ge>8.00</q:ge></price>
    </q:and>
    <q:filtered>true</q:filtered>
  </q:query>
</q:qbe>

The following table explains the requirement expressed by each component of the query. Each of the subquery types used in this example is explored in more detail in Understanding QBE Sub-Query Types.

<title>
  <q:value exact="false">Tom Sawyer</q:value>
</title>

<q:near distance="2">
  <author><q:word>mark</q:word></author>
  <author><q:word>twain</q:word></author>
</q:near>

<edition format="paperback">
  <q:or>
    <publisher>Clipper</publisher>
    <publisher>Daw</publisher>
  </q:or>
</edition>

<q:and>
  <price><q:lt>10.00</q:lt></price>
  <price><q:ge>8.00</q:ge></price>
</q:and>

<q:filtered>true</q:filtered>

JSON Example

This example assumes the database contains documents with the following structure:

{"book": {
  "title": "Tom Sawyer",
  "author" : "Mark Twain",
  "edition": [
    { "format": "paperback",
      "publisher": "Clipper",
      "pub-date": "2011-08-01",
      "price" : 9.99,
      "isbn": "1613800917",
    }
  ]
} }

The following query uses most of the features of QBE and matches the above document. The sub-queries that are immediate children of query are implicitly AND'd together, so all these conditions must be met by matching documents.

{"$query": {
    "title": {
      "$value": "Tom Sawyer",
      "$exact": false
    },
    "$near": [
      { "author": { "$word": "mark" } },
      { "author": { "$word": "twain" } }
    ], "$distance": 2,
    "edition": {
      "format": "paperback",
      "$or" : [
        { "publisher": "Clipper" },
        { "publisher": "Daw" }
      ]
    },
    "$and": [
      {"price": { "$lt": 10.00 }},
      {"price": { "$ge": 8.00 }}
    ],
    "$filtered": true
} }

The following table explains the requirement expressed by each component of the query. Each of the subquery types used in this example is explored in more detail in Understanding QBE Sub-Query Types.

"title": {
  "$value": "Tom Sawyer",
  "$exact": false
}

"$near": [
  { "author": { "$word": "mark" } },
  { "author": { "$word": "twain" } }
], 
"$distance": 2

"edition": {
  "format": "paperback",
  "$or" : [
    { "publisher": "Clipper" },
    { "publisher": "Daw" }
  ]
}

"$and": [
  {"price": { "$lt": 10.00 }},
  {"price": { "$ge": 8.00 }}
]

"$filtered": true

Value Query

A value query matches an entire literal value, such as a string, date, or number.

By default, an XML element or JSON property criteria represents a value query with exact match semantics:

• The value in the criteria is matched with case, diacritic, punctuation, and whitespace sensitivity enabled.
• Stemming and wildcarding are not enabled.
• The specified value must be an immediate child of the containing XML element or JSON property.
• The value in the query will not match if it is a subset of the value in a document.

For example, the following criteria only matches documents where the author XML element or JSON property contains exactly and only the text Twain. It will not match author values such as Mark Twain or twain.

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <author>Twain</author>
  </q:query>
</q:qbe>

JSON

{
  "$query": { "author": "Twain" }
}

You can override some of the exact match semantics with options. For example, you can disable case-sensitive matches. For details, see Adding Options to a QBE.

A value query can be explicit or implicit. The example above is an implicit value query. You can make an explicit value query using the value QBE keyword. This is useful when you want to add options to a value query. The following example is an explicit value query that uses the case-sensitive option.

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <author>
      <q:value case-sensitive="false">Twain<q:value>
    </author>
  </q:query>
</q:qbe>

JSON

{ "$query": { 
    "author": {
      "$value": "Twain",
      "$case-sensitive": false
    }
} }

Word Query

A word query matches a word or phrase appearing anywhere in a text value. A word query will match a subset of a text value. By default, word queries do not use exact match semantics.

• The value in the criteria is matched with case, diacritic, punctuation, and whitespace sensitivity disabled.
• Stemmed matches are included.
• Wildcard matching is performed if wildcarding is enabled for the database.
• The specified word or phrase can occur in the value of the immediately containing XML element or JSON property, or in the value of child components.

You can use options to override some of the match semantics. For details, see Adding Options to a QBE.

Word queries occurring within another container, such as an XML element or JSON property that describes content in your document, match occurrences within the container. Word queries that are not in a container, such as word queries that are immediate children of the top level QBE query wrapper, match occurrences anywhere in a document. For details, see Container Query and Searching Entire Documents.

The following example QBE matches if the author contains twain with any capitalization, so it matches values such as Mark Twain, M. Twain and mark twain.

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <author><q:word>twain</q:word></author>
  </q:query>
</q:qbe>

JSON

{
  "$query": { "author": { "$word": "twain" } }
}

In JSON, the value in a word query can be either a string or an array of strings. An array of values is treated as an AND-related list of word queries. For example, the following query matches documents where author contains word matches for mark and twain. The matched values need not be array item values.

{
  "$query": { "author": { "$word": [ "mark", "twain" ] } }
}

Range Query

A range query matches values that satisfy a relational expression applied to a string, number, date, time, or dateTime value, such as less than 5 or not equal to 10. This section includes the following topics:

• JSON Property Value Range Query
• XML Element Value Range Query
• XML Element Attribute Value Range Query
• Type Conversion in Range Expressions

  You must either back a range query by a range index or use the filtered flag. For details see How Indexing Affects Your Query.

{ "$operator" : boundary-value }

"format": {"$ne": "paperback" }

<container>
  <q:operator>boundary-value</q:operator>
</container>

<pub-date>
  <q:gt>2010-01-01</q:gt>
</pub-date>

<container attr="$operator value" />

<edition format="$ne paperback" />

Composed Query

A composed query is one composed of sub-queries joined by a logical operator such and, or, not, or near. The following example matches documents where the value of author is Mark Twain or Robert Frost.

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <q:or>
      <author>Mark Twain</author>
      <author>Robert Frost</author>
    </q:or>
  </q:query>
</q:qbe>

JSON

{"$query": {
    "$or": [
      "author": "Mark Twain",
      "author": "Robert Frost"
    ]
} }

The near operator models a cts:near-query and accepts an optional distance XML attribute or JSON property to specify a maximum acceptable distance in words between matches for the operands queries. For example, the following near query specifies a maximum distance of 2 words. The default distance is 10.

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <q:near distance="2">
      <author><q:word>mark</q:word></author>
      <author><q:word>twain</q:word></author>
    </q:near>
  </q:query>
</q:qbe>

JSON

{"$query": {
    "$near": [
      { "author": { "$word": "mark" } },
      { "author": { "$word": "twain" } }
    ],
    "$distance": 2
} }

Container Query

A container query matches when sub-query conditions are met within the scope of a specific XML element or JSON property. In a container query, the relationship between the named container and XML element or JSON property names used in the sub-queries is contained by not merely child of.

A container query is implicitly defined when you use search criteria that model your document and that contain a composed query or structural sub-queries (XML element, XML attribute, or JSON property).

For example, an XML criteria such as the following defines defines a container query on edition because it contains an implicit value query on another element, price.

<edition><price>8.99</price></edition> 

By contrast, the following criteria is a value query, not a container query, on author:

<author>twain</author>

Similarly, the following JSON criteria is a container query on edition because it contains an implicit value query on another property, price.

"edition":{"price": 8.99}

By contrast, a criteria such as the following is a value query, not a container query, on author.

"author":"twain"

The examples below demonstrate how a container query for price contained by book matches at multiple levels.

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <book><price>8.99</price></book>
  </q:query>
</q:qbe>

<book>
  <price>8.99</price>
</book>

<book>
  <edition>
    <price>8.99</price>
  </edition>
</book>

{"$query": {
  "book":{"price": 8.99}
} }

{ "book": {
    "price": 8.99
} }

{ "book": {
    "edition": {"price": 8.99 }
} }

{ "book": {
    "edition": [ {"price": 8.99} ]
} }

A query on an XML element attribute is a container query in that the element contains the attribute. However, only attributes on the containing element can match. The following criteria matches @format only when it appears as an attribute of edition. It does not match occurrences of @format on child elements of edition.

<edition format="paperback"/>

The following table contains XML examples of container queries.

<edition>
  <price>8.99</price>
</edition>

<edition format="paperback"/>

<edition>
  <q:or>
    <price>8.99</price>
    <publisher>Fawcett</publisher>
  </q:or>
</edition>

The following table contains JSON examples of container queries.

"edition": {"price": 8.99 }

"edition": {
  "$or": [
    "price": 8.99,
    "publisher": "Fawcett"
  ]
}

"edition": {
  "$and": [
    "price": 8.99,
    "publisher": "Fawcett"
  ]
}

"$and": [
  "edition": {"price": 8.99},
  "edition": {"publisher":"Fawcett"}
]

XML Search Criteria Quick Reference

The table below provides a quick reference for constructing QBE search criteria and composed queries in XML. Use these examples as templates for your own criteria. For more details, see QBE Structural Reference.

The examples below assume that the namespace prefix q is bound to http://marklogic.com/appservices/querybyexample.

<e>v</e>

<e><q:value>v</q:value></e>

<e a="v"/>

<e a="$value v"/>

<e><q:word>w</q:word></e>

<e a="$word w"/>

<e><q:gt>5</q:gt></e>

<e a="$gt 5"/>

<e><q:exists/></e>

not supported

<e1>
    <e2>v</e2>
</e1>

<e1>
    <e2><q:word>w</q:word></e2>
</e1>

<e1>
    <e2 a="v"/>
</e1>

<e1>
    <e2 a="$word w"/>
</e1>

not supported

<q:or>
    <e>v1</e>
    <e>v2</e>
</q:or>

<q:or>
    <e><q:word>w1</q:word></e>
    <e><q:word>w2</q:word></e>
</q:or>

<e1>
    <q:or>
        <e2>v1</e2>
        <e2>v2</e2>
    </q:or>
</e1>

<q:or>
    <e a="v1"/>
    <e a="v2"/>
</q:or>

<q:or>
    <e a="$word w1"/>
    <e a="$word w2"/>
</q:or>

<e1>
    <q:or>
        <e2 a="v1"/>
        <e2 a="v2"/>
    </q:or>
</e1>

JSON Search Criteria Quick Reference

The table below provides a quick reference for constructing QBE search criteria and composed queries in JSON. Where the example property name begins with c, the criteria represents a container query. For more details, see QBE Structural Reference.

This list of example criteria is not exhaustive. Additional forms are supported. For example, not all variants of explicit and implicit value queries are shown for a given criteria.

{ "k": "v"}

{ "k": {"$value": "v"}}

{"k":{"$word":"w"}}

{"k":{"$gt":5}}

{"k":{"$exists":{}}}

{"c": {"k":"v"} }

{"c": {
  "k": {"$value": "v"}
} }

{"c":{
  "k":{"$word":"w"}
}}

{"$or":[
  {"k":"v1"},
  {"k":"v2"}
]}

{"$or":[
  {"k": {"$value": "v1"}},
  {"k": {"$value": "v2"}}
]}

{"k": {"$or": [
  {"$value": "v1"},
  {"$value": "v2"}
]}}

{"$or":[
  {"k":{"$word":"w1"}},
  {"k":{"$word":"w2"}}
]}

{"k": {"$or": [
  {"$word": "v1"},
  {"$word": "v2"}
]}}

{"c":{"$or":[
  {"k":"v1"},
  {"k":"v2"}
]}}

{"c":{"$and":[
  {"k":"v1"},
  {"k":"v2"}
]}}

{"c":{"$and":[
  {"k": {"$value": "v1"}},
  {"k": {"$value": "v2"}}
]}}

{
  "k1":"v1",
  "k2":"v2"
}

{"$and":[
  {"k1":"v1"},
  {"k2":"v2"}
]}

{
  "k1": {"$value": "v1"},
  "k2": {"$value": "v2"}
}

{"$and":[
  {"k1": {"$value": "v1"}},
  {"k2": {"$value": "v2"}}
]}

Searching Entire Documents

This section describes how to construct a query that matches words or phrases anywhere in a document, rather than constraining the match to occurrences in a particular XML element, XML attribute, or JSON property.

A word query has document scope if it is not contained in an XML element or JSON property criteria. For example, a word query that is an immediate child of the top level query element, or one that is a child at any depth of a hierarchy of composed queries (and, or, not, near). This also applies to the implicit and query that joins the immediate children of query.

For example, the following query matches all documents containing the phrase moonlight sonata:

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <q:word>moonlight sonata</q:word>
  </q:query>
</q:qbe>

JSON

{"$query": {
  "$word": "moonlight sonata"
} }

The following example matches all documents containing either the phrase moonlight sonata or the word sunlight.

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <q:or>
      <q:word>moonlight sonata</q:word>
      <q:word>sunlight</q:word>
    </q:or>
  </q:query>
</q:qbe>

JSON

{"$query": {
  "$or": [
    {"$word": "moonlight sonata"},
    {"$word": "sunlight"}
  ]
} }

An AND relationship between words and phrases can be either explicit or implicit. The following example queries match all documents contains both the phrase moonlight sonata and the word sunlight:

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <q:word>moonlight sonata</q:word>
    <q:word>sunlight</q:word>
  </q:query>
</q:qbe> 

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <q:and>
      <q:word>moonlight sonata</q:word>
      <q:word>sunlight</q:word>
    </q:and>
  </q:query>
</q:qbe>

JSON

{"$query": [
  {"$word": "moonlight"},
  {"$word": "sunlight"}
] } 

{"$query": {
  "$and": [
    {"$word": "moonlight"},
    {"$word": "sunlight"}
  ]
} }

Top Level Structure

At the top level, a QBE must contain a query and can optionally contain a response and/or a format flag. A QBE has the following top level parts:

• query: Define matching document requirements in the query.
• response: Customize your search results in the response; if there is no response, the default search response is returned.
• format: Use the format flag to override the interpretation of bare names as JSON property names or XML element names in no namespace, based on the query format. For details, see Scoping a Search by Document Type.
• validate: Use the validate flag to enable query validation before evaluating the search. The default is no validation, which can result in surprising search results if your QBE contains errors. However, validation has a performance cost, so it is best used only for debugging during development.

The following table outlines the top level of a QBE:

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    search parameters
  </q:query>
  <q:response>
    search result customizations
  </q:response>
  <q:format>xml-or-json</q:format>
  <q:validate>true-or-false</q:validate>
</q:qbe>

JSON

{
  "$query": {
    search parameters
  },
  "$response": {
    search result customizations
  },
  "$format": xml-or-json,
  "$validate": boolean
}

A query contains one or more XML elements or JSON properties defining element or property criteria or composed queries. Use criteria to model document structure. Use a composed query to logically join sub-queries using operators such as and, or, not, and near.

In XML, a QBE has a qbe wrapper element. Element and attribute names pre-defined by the QBE grammar, such as qbe, query, and word, are in the namespace http://marklogic.com/appservices/querybyexample. All other element and attributes names represent element and attribute names in your documents. For details, see Managing Namespaces

In JSON, all property names pre-defined by the QBE grammar have a $ prefix, such as $query or $word. Any property name without a $ prefix represents a property in your documents. For details, see Property Naming Convention.

You will not usually need to set the format flag. You only need to set the format flag to use a JSON QBE to match XML documents, or vice versa. For details, see Scoping a Search by Document Type.

Query Components

The table below describes the components of the query portion of a QBE. Additional format-specific details are covered in XML-Specific Considerations and JSON-Specific Considerations.

query

$query

your element name

your property name

and

or

not

near

$and

$or

$not

$near

lt, le

gt, ge

eq, ne

$lt, $le

$gt, $ge

$eq, $ne

value

word

exists

$value

$word

$exists

filtered

score

$filtered

$score

The following table summarizes where each component type can be used. Options are covered in Adding Options to a QBE.

Response Components

You can use the response portion of a QBE to customize the format of your search results. The following table describes the components of a response. A response is optional, and can only occur at the top level of a QBE, as a sibling of query.

A response can contain the following formatter components:

For details, see Customizing Search Results.

XML-Specific Considerations

This section covers structural and semantic details you should know when constructing a QBE in XML.

• Managing Namespaces
• Querying Attributes

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
   ...
</q:qbe>

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample"
       xmlns:ns1="http://marklogic.com/example1">
  <q:query xmlns:ns2="http://marklogic.com/example2">
    <ns1:author xmlns="http://marklogic.com/example">
      Mark Twain
    </ns1:author>
    <ns2:edition format="paperback"/>
    <title xmlns="http://marklogic.com/example3">Tom Sawyer</title>
  </q:query>
</q:qbe>

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <edition format="paperback"/>
  </q:query>
</q:qbe>

$keyword value

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <edition price="$lt 9.00"/>
  </q:query>
</q:qbe>

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <edition price="$lt 9.00" format="$word paperback" />
    <q:filtered>true</q:filtered>
  </q:query>
</q:qbe>

JSON-Specific Considerations

This section covers structural and semantic details you should know when constructing a QBE in JSON. The following topics are covered:

• Property Naming Convention
• Matching Array Items
• Searching Array and Object Containers
• Constructing a QBE with the Node.js QueryBuilder

{ "$query": {"k": ["v"]} }

{ "k": "v" }

{ "k": ["v"] }

{ "$query": {"k": "v"} }

{"$query":
  {"k": ["v1", "v2"]}
}

{"$query": {
  "$and": [
    {"k": "v1"},
    {"k": "v2"}
  ]
}}

{ "k": ["v1", "v2"] }

{ "k": ["v1", "v2", "v3"] }

{ "c": [{"k": "v1"}, {"k": "v2"}] }

{"c": {"k": "v1", "c2": {"k": "v2"}}

name : value

{ "price" : 8.99 }

name : { 
  word-or-value : value}

{ "title" : {
  "$word" : "sawyer"
} }

name : { 
  relational-op : value}

{ "price" : {
  "$lt" : 9
} }

name : object

{"edition": 
  {"price" : 8.99 } 
}

name : { 
  logical-op : [
    sub-query+
  ]
}

{"edition" : { 
  "$or": [
    {"format": "paperback"},
    {"format": "hardback"}
  ] 
} }

{"$query":
  { "k": "v" }
}

{ "k": "v" }

{ "k": ["v"] }

{"$query":
  {"k": ["v1", "v2"]}
}

{ "k": ["v1", "v2"] }

{ "c": [{"k": "v1"}, {"k": "v2"}] }

{"c": {"k": "v1", "c2": {"k": "v2"}}

{"$query":
  {"c": {"k": "v"}}
}

{ "c" : {"k" : "v" } }

{ "c" :
   {"c2": {"k": "v" }}
}

{"$query": {
  "author": {"$word": "twain"},
  "$filtered": true
}}

qb.byExample({
  author: {$word: 'twain'},
  $filtered: true
})

qb.byExample(
  { $query: {
    author: {$word: 'twain'},
    $filtered: true
  }
)

Specifying Options in XML

In an XML QBE, an option is an attributes of the predefined QBE element it modifies, such <q:lt/>, <q:word/> or <q:value>. The following query demonstrates use of the exact option on a value query.

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <author><q:value exact="false">mark twain</q:value></author>
  </q:query>
</q:qbe>

You cannot apply options to queries on attributes because the range, word, or value query is embedded in the attribute value. For example, you cannot add a case-sensitive option to the following attribute word query:

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <edition @format="$word paperback"/></edition>
  </q:query>
</q:qbe>

If you need such control over an element attribute query, you should use a structured or combined query.

Specifying Options in JSON

In a JSON QBE, an option is a sibling of the QBE object it modifies, such as a value, word, or range query. Option names always have a $ prefix.

The following example query uses the exact option to modify a value query by including it as a JSON property at the same level as the $value object:

{
  "$query": {
    "author": { 
      "$exact": false, 
      "$value": "mark twain"
    }
  }
}

Option List

The following table describes the options available for use in a QBE. The MarkLogic Server Search API supports additional options through other query formats, such as string or structured query, and through the use of persistent query options. For details, see Search Customization Using Query Options.

case-sensitive

diacritic-sensitive

punctation-sensitive

whitespace-sensitive

stemmed

exact

score-function

slope-factor

min-occurs

max-occurs

lang

weight

constraint

@xsi:type (XML)
$datatype (JSON)

Using Persistent Query Options

The REST and Java APIs enable you to install persistent query options on your REST instance and apply them to subsequent searches. You can also use persistent query options with the Node.js Client API, but the API has no facility for creating and maintaining the persistent options.

Using persistent query options with a QBE allows you to use options not supported directly by the QBE grammar. Using persistent options with a QBE also allows you to define global options to apply throughout your query, such as making all word queries case-sensitive instead of specifying the case-sensitive option on each word query in your QBE.

Query options applied through through the constraint option override options specified inline on a QBE.

You can apply persistent query options to a QBE using the constraint option. To use this option:

1. Install named, persistent query options following the directions appropriate for the API you are using.
2. Specify the name of a constraint defined in the persistent options from Step 1 as the value of a constraint option on a word, value, or range query in your QBE. See the example, below.
3. When you execute a search with your QBE, associate the persistent query options from Step 1 with your search in the manner prescribed by the client API (REST, Java, or Node.js).

For details on defining, installing and using persistent query options, see Configuring Query Options in REST Application Developer's Guide or Query Options in Java Application Developer's Guide.

The pre-defined constraint named by the constraint option should match the type of query to which it is applied. That is, name a range constraint for a range query, a value constraint for a value query, and a word constraint for a word query.

The following example pre-defines a word constraint called w-t that gives weight 2.0 to matches in a title XML element or JSON property, and then applies it to a QBE that contains a word query on title. This enables word queries on title a default weight that can be overridden by omitting the constraint option.

If the folloiwng persistent query options are installed specified as a parameter to the search performed with the QBE:

<search:options
    xmlns:search="http://marklogic.com/appservices/search">
  <search:constraint name="w-t">
    <search:word>
      <search:element name="title" ns=""/>
      <search:weight>2.0</search:weight>
    </search:word>
  </search:constraint>
</search:options>

{"options": {
  "constraint": [ {
    "name": "w-t",
    "word": {
      "json-property": "title",
      "weight": 2
    }
  } ]
} }

Then the following QBE applies the w-t option to a word query on title to give weight 2.0 to matches in a title element.

<q:qbe
    xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <title>
      <q:word constraint="w-t">sawyer</q:word>
    </title>
  </q:query>
</q:qbe>

{ "$query": {
    "title": {
      "$word": "sawyer",
      "$constraint": "w-t"
    }
} }

When to Include a Response in Your Query

Add an optional response section to a QBE to do one or more of the following:

• Return matching documents instead of snippets. (snippet)
• Return only information about the document and the match, such as database URI, document format, and relevance score. (snippet)
• Specify XML elements or JSON properties to prefer when constructing snippets. (snippet)
• Specify XML elements or JSON properties to extract from matching documents, whether or not the match occurs within those elements or properties. (extract)

Advanced customization is available using result decorators, transforms, and persistent query options. For details, see Customizing Search Results in REST Application Developer's Guide or Transforming Search Results in Java Application Developer's Guide.

Using the snippet Formatter

Use snippet to control what, if anything, is included in the snippet portion of a search match and to identify preferred XML elements or JSON properties to include a snippet. The default snippet is a small text excerpt with the matching text tagged for highlighting. The following table contains an excerpt of the snippet section of a search response generated with the default policy.

XML

<search:response ...>
  <search:result ...>
    <search:snippet>
      <search:match
          path="fn:doc(&quot;/books/sawyer.xml&quot;)/book">
        <search:highlight>Mark Twain</search:highlight>
      </search:match>
    </search:snippet>
  </search:result>
  ...
</searchresponse>

JSON

{
  ...
  "results": [ {
      ...
      "matches": [ {
        "path": "fn:doc(\"/books/sawyer.json\")/*:json/*:book/*:author",
        "match-text": [ { "highlight": "Mark Twain" } ]
      } ]
  } ],
  ...
}

The snippet formatter has the following form:

<q:response>
  <q:snippet>
    <q:policy/>
    preferred-element
  </q:snippet>
</q:response>

{ "$response": {
    "$snippet": { 
      policy: {},
      preferred-property: {}
    },
} }

The policy, preferred-element, and preferred-property are optional.

The snippeting policy controls whether or not snippets are included in the output and whether to include a small text excerpt (default) or the entire document when snippets are enabled. Use one of the following element or property names for policy.

default

$default

document

$document

none

$none

The following example disables snippet generation by setting the snippet policy to none. In JSON, specify an empty object value for the policy property.

<q:response>
  <q:snippet>
    <q:none/>
  </q:snippet>
</q:response>

{ "$response": {
    "$snippet": { 
      "$none": {}
    }
} }

You can also specify one or more XML element or JSON property names to be preferred when generating snippets. For example, if you specify a preference for the title element or property, and both title and author contain a match, the snippet is generated from the match in title. In JSON, specify the preferred property with an empty object value.

<q:response>
  <q:snippet>
    <title/>
  </q:snippet>
</q:response>

{ "$response": {
    "$snippet": { 
      "title": {}
    }
} }

Using the extract Formatter

Use the extract formatter to specify additional XML elements or JSON properties to include in the search output. If snippets are included, the extracted components supplement any snippet in a match, rather than replacing it.

<q:response>
  <q:extract>
    <your-element/>
  </q:extract>
</q:response>

{ "$response": {
    "$extract": { 
      "your-property-name": {}
    }
} }

For example, the following response says to extract the title and author from a matching document. The title and author need not contain the matching terms or values.

<q:response>
  <q:extract>
    <title/>
    <author/>
  </q:extract>
</q:response>

{ "$response": {
    "$extract": { 
      "title": {},
      "author": {}
    }
} }

Extracted elements or properties go into the metadata section of the enclosing match. For an example, see Example: Search Customization.

Example: Search Customization

The following QBE modifies the search results to exclude snippets and to extract the title XML element or JSON property into the search result metadata section.

XML

<q:qbe xmlns:q="http://marklogic.com/appservices/querybyexample">
  <q:query>
    <author>Mark Twain</author>
  </q:query>
  <q:response>
    <q:extract><title/></q:extract>
    <q:snippet><q:none/></q:snippet>
  </q:response>
</q:qbe>

JSON

{
  "$query": {
    "author": "Mark Twain"
  },
  "$response": {
    "$snippet": { "$none": {} },
    "$extract": { "title": {} }
  }
}

The following table shows the default output and the modified output produced by the above query.

XML

<search:response
    snippet-format="snippet"
    total="1" start="1"
    page-length="10"
    ...>
  <search:result index="1"
      uri="/books/sawyer.xml"
      ...>
    <search:snippet>
      <search:match ...>
        <search:highlight>
          Mark Twain
        </search:highlight>
      </search:match>
    </search:snippet>
  </search:result>
  ...
</search:response>

<search:response
  snippet-format="empty-snippet"
    total="1" start="1"
    page-length="10"
    ..>
  <search:result index="1"
      uri="/books/sawyer.xml"
      ...>
    <search:snippet/>
    <search:metadata>
      <title>Tom Sawyer</title>
    </search:metadata>
  </search:result>
  ...
</search:response>

JSON

{
  "snippet-format": "snippet",
  "total": 1,
  "start": 1,
  "page-length": 10,
  "results": [{
    "index": 1,
    "uri": "/books/sawyer.json",
    "matches": [{
      "path": ...,
      "match-text": [{
        "highlight": "Mark Twain"
      }]
    }]
  }],
  ...
}

{
  "snippet-format":
    "empty-snippet",
  "total": 1,
  "start": 1,
  "page-length": 10,
  "results": [{
    "index": 1,
    "uri": "/books/sawyer.json",
    ...,
    "matches": [],
    "metadata": [{
      "title": "Tom Sawyer"
    }]
  }],
  ...
}