MarkLogic Server 11.0 Product Documentation
Java Application Developer's Guide — Chapter 5

« Previous chapter

Next chapter »

Searching

This chapter describes how to submit searches using the Java API, and includes the following sections:

Overview of Search Using the Java API
Using SearchHandle to Examine Query Results
Search Using String Query Definition
Search Documents Using Structured Query Definition
Prototype a Query Using Query By Example
Apply Dynamic Query Options to Document Searches
Search On Tuples (Tuples Query / Values Query)
Limiting A Search To Specific Collections And/Or A Directory
Searching Values Metadata Fields
Transforming Search Results
Generating Search Term Completion Suggestions
Extracting a Portion of Matching Documents

Overview of Search Using the Java API

The MarkLogic Java API provides the following fundamental ways of querying the database:

Searches on documents, which return search results, snippets, and facets.
Value or Tuple (co-occurrences) searches, which return data from range indexes and the results of aggregate functions (including user-defined aggregate functions) from range indexes.

In addition to typical document searches, you can search Java POJOs that have been stored in the database. For details, see POJO Data Binding Interface.

When you search documents you can express search criteria using one of the following kinds of query:

String query: Use a Google-style query string to search documents and metadata. For details, see Search Using String Query Definition.
Query By Example: Search documents by constructing a query that directly models the structure of the documents you want to match. For details, see Prototype a Query Using Query By Example.
Structured query: A simple and easy way to construct queries as a Java, XML, or JSON structure, allowing you to manipulate complex queries (such as geospatial polygons) in the Java client. For details, see Search Documents Using Structured Query Definition
Combined query: Combine a string or structured query with dynamic query options. For details, see Apply Dynamic Query Options to Document Searches.

When you query aggregate range indexes, you express your search criteria using a values query.

All search methods can also use persistent query options. Persistent query options are stored on the REST Server and referenced by name in future queries. Once created and persisted, you can apply query options to multiple searches, or even set to be the default options for all searches. Note that in XQuery, query option configurations are called options nodes.

Some search methods support dynamic query options that you specify at search time. A combined query allows you to bundle a string and/or structured query with dynamic query options to further customize a search on a per search basis. You can also specify persistent query options with a combined query search. The search automatically merges the persistent (or default) query options and the dynamic query options together. For details, see Apply Dynamic Query Options to Document Searches.

Query options can be very simple or very complex. If you accept the defaults, for example, there is no need to specify explicit query options. You can also make them as complex as is needed.

For details on how to create and work with query option configurations, see Query Options. For details on individual query options and their values, see Appendix: Query Options Reference in the Search Developer's Guide. For more information on search concepts, see the Search Developer's Guide.

In the examples in this chapter, assume a DatabaseClient called client has already been defined.

Using SearchHandle to Examine Query Results

Usually, you will use a SearchHandle object to contain your query results. The exact nature of results varies, depending on both the handle's configuration and what query options and values were used for the search operation.

You can specify snippets to return in various ways. By default, they return as Java objects. But for custom or raw snippets, they are returned as DOM documents by using the forceDOM flag.

There are several ways to access different parts of the search result or control search results from a SearchHandle.

The getMatchResults() method returns an array of MatchDocumentSummary objects of the matched documents, from which you can further extract for each result its match locations, path, metadata, an array of snippets, fitness, confidence measure, and URI. For details, see the MatchDocumentSummary entry in Java API JavaDoc.
getMetrics() returns a SearchMetrics object containing various timing metrics about the search.
getFacetNames(), getFacetResult(name), getFacetResults() return, respectively, a list of returned facet names, the specified named facet result, and an array of facet results for this search.
getTotalResults() returns an estimate of the number of results from the search.
setForceDOM(boolean) sets the force DOM flag, which if true causes snippets to always be returned as DOM documents.

See the Java API JavaDoc for SearchHandle for the full interface.

The following is a typical programming technique for accessing search results using a search handle:

// iterate over MatchDOcumentSummary array locations, getting
// the snippet text for each location (you would then do something
// with the snippet text)
MatchDocumentSummary[] summaries = results.getMatchResults();
for (MatchDocumentSummary summary : summaries ) {
    MatchLocation[] locations = summary.getMatchLocations();
    for (MatchLocation location : locations) {
        location.getAllSnippetText();
      // do something with the snippet text
    }
}

Search Using String Query Definition

The MarkLogic Server Search API lets you do searches on string arguments, including the usual search operators such as AND and OR. For example, you could search on Batman, Batman AND Robin, Batman OR Robin, etc. For details, see Search Grammar in the Search Developer's Guide.

Instantiate a QueryManager. The manager deals with interaction between the client and the database.
```
QueryManager queryMgr = client.newQueryManager();
```
Instantiate a StringQueryDefinition object. Use StringQueryDefinition.setCriteria() to specify your search string.
```
StringQueryDefinition qd = queryMgr.newStringDefinition();
qd.setCriteria("Batman AND Robin");
```

Run a search with the StringQueryDefinition object as an argument, returning a SearchHandle object or an XML or JSON handle to get the search results in either of those formats:

SearchHandle results = queryMgr.search(qd, new SearchHandle());
DOMHandle results = queryMgr.search(qd, new DOMHandle());
JacksonHandle results = queryMgr.search(qd, new JacksonHandle());

Process and/or display the results using the handle.

Search Documents Using Structured Query Definition

Structured queries let you construct and modify complex queries in Java, XML, or JSON. For details, see Searching Using Structured Queries in the Search Developer's Guide. This section includes the following parts:

Ways to Create a Structured Query
Basic Steps to Define a Structured Query Definition
Creating a Structured Query From Raw XML or JSON
Structured Query Examples

Ways to Create a Structured Query

You can create a structured query in XML, in JSON, or using the StructuredQueryBuilder or PojoQueryBuilder interfaces in the Java API.

To specify a structured query directly in XML or JSON, use RawStructuredQueryDefinition; for details, see Creating a Structured Query From Raw XML or JSON. If you construct a structured query directly, it is up to you to make sure the query is constructed correctly. Incorrectly constructed queries can result in syntax errors, a query that does not do what you expect, or other exceptions. For syntax details, see Searching Using Structured Queries in the Search Developer's Guide.

The StructuredQueryBuilder interface in the Java API enables you build out a structured query one piece at a time in Java. The PojoQueryBuilder interface is similar, but you use it specifically for searching persistent POJOs; for details see Searching POJOs in the Database.

Basic Steps to Define a Structured Query Definition

The following are the basic steps needed to define a structured query definition in the Java API. This procedure creates a structured query definition using StructuredQueryBuilder. You can also create one directly in XML/JSON; for details, see Creating a Structured Query From Raw XML or JSON.

Instantiate a QueryManager. The manager deals with interaction between the client and the database.
```
QueryManager queryMgr = client.newQueryManager();
```
Instantiate a StructuredQueryBuilder, optionally passing in the name of persistent query options to use with your search.
```
StructuredQueryBuilder qb = new StructuredQueryBuilder(OPTIONS_NAME);
```

Use the query builder to create a StructuredQueryDefinition object with the desired search criteria.

StructuredQueryDefinition querydef = 
    qb.and(qb.term("neighborhood"), 
           qb.valueConstraint("industry", "Real Estate"));

Run a search with the StringQueryDefinition object as an argument, returning a result handle:
```
SearchHandle results = queryMgr.search(querydef, new SearchHandle());
```

Creating a Structured Query From Raw XML or JSON

To create a structured query from a raw XML or JSON representation, use any handle class that implements com.marklogic.client.io.marker.StructureWriteHandle.

The Java API includes StructureWriteHandle implementations that support creating a structure in XML or JSON from a string (StringHandle), a file (FileHandle), a stream (InputStreamHandle), and popular abstractions (DOMHandle, DOM4JHandle, JDOMHandle). For a complete list of implementations, see the Java API JavaDoc.

Follow this procedure to create a structured query using a handle:

Instantiate a QueryManager. The manager deals with interaction between the client and the database.
```
QueryManager queryMgr = client.newQueryManager();
```

Create a JSON or XML representation of the query, using a text editor or other tool or library. Use the syntax detailed in Searching Using Structured Queries in the Search Developer's Guide. The following example uses String for the raw representation:

String rawXMLQuery =
    "<search:query "+
          "xmlns:search='http://marklogic.com/appservices/search'>"+
      "<search:term-query>"+
          "<search:text>neighborhoods</search:text>"+
      "</search:term-query>"+
      "<search:value-constraint-query>"+
          "<search:constraint-name>industry</search:constraint-name>"+
          "<search:text>Real Estate</search:text>"+
      "</search:value-constraint-query>"+
    "</search:query>";

String rawJSONQuery =
        "{\"query\": {" +
                "   \"term-query\": {" +
                "      \"text\": \"neighborhoods\"" +
                "   }," +
                "   \"value-constraint-query\": {" +
                "      \"constraint-name\": \"industry\"," +
                "      \"text\": \"Real Estate\"" +
                "   }" +
                "}" +
        "}";

Create a handle on your raw query using a class that that implements StructureWriteHandle. Set the handle content format appropriately. For example:

// For an XML query
StringHandle rawHandle = 
    new StringHandle(rawXMLQuery).withFormat(Format.XML);

// For a JSON query
StringHandle rawHandle = 
    new StringHandle(rawJSONQuery).withFormat(Format.JSON);

Create a RawStructuredQueryDefinition from the handle. Optionally, include the name of persistent query options. For example:

// Use the default persistent query options
RawStructuredQueryDefinition querydef =
    queryMgr.newRawStructuredQueryDefinition(rawHandle);
// Use the persistent options previously saved as "myoptions"
RawStructuredQueryDefinition querydef =
    queryMgr.newRawStructuredQueryDefinition(rawHandle, "myoptions");

Perform a search using the RawStructuredQueryDefinition and a results handle.

SearchHandle resultsHandle = 
    queryMgr.search(querydef, new SearchHandle());

Structured Query Examples

This section shows some structured query examples, showing the XML for a structured query and the corresponding Java code using StructuredQueryBuilder. You can put each of these examples in context by inserting the StructuredQueryDefinition line in the following code:

QueryManager queryMgr = dbClient.newQueryManager();
StructuredQueryBuilder sb = 
   queryMgr.newStructuredQueryBuilder("myopt");

// put code from examples here
StructuredQueryDefinition criteria = 
   ... example of building query definition ...
// end code from examples

StringHandle searchHandle = 
  queryMgr.search(
    criteria, new StringHandle()).get();

Additionally, these examples use query options from the following code:

String xmlOptions = 
    "<search:options " +
        "xmlns:search='http://marklogic.com/appservices/search'>" +
      "<search:constraint name='date'>" +
        "<search:range type='xs:date'>" +
          "<search:element name='date' ns='http://purl.org/dc/elements/1.1/'/>" +
        "</search:range>" +
      "</search:constraint>" +
      "<search:constraint name='popularity'>" +
        "<search:range type='xs:int'>" +
          "<search:element name='popularity' ns=''/>" +
        "</search:range>" +
      "</search:constraint>" +
      "<search:constraint name='title'>" +
        "<search:word>" +
          "<search:element name='title' ns=''/>" +
        "</search:word>" +
      "</search:constraint>" +
      "<search:return-results>true</search:return-results>" +
      "<search:transform-results apply='raw' />" +
    "</search:options>";

//JSON equivalant
String jsonOptions =
        "{\"options\":{" +
                "   \"constraint\": [" +
                "      {" +
                "         \"name\": \"date\"," +
                "         \"range\": {" +
                "            \"type\":\"xs:date\", " +
                "            \"element\": {" +
                "               \"name\": \"date\"," +
                "               \"ns\": \"http://purl.org/dc/elements/1.1/\"" +
                "            }" +
                "         }" +
                "      }," +
                "      {" +
                "         \"name\": \"popularity\"," +
                "         \"range\": {" +
                "            \"type\":\"xs:int\", " +
                "            \"element\": {" +
                "               \"name\": \"popularity\"," +
                "               \"ns\": \"\"" +
                "            }" +
                "         }" +
                "      }," +
                "      {" +
                "         \"name\": \"title\"," +
                "         \"word\": {" +
                "            \"element\": {" +
                "               \"name\": \"title\"," +
                "               \"ns\": \"\"" +
                "            }" +
                "         }" +
                "      }" +
                "   ]," +
                "   \"return-results\": \"true\"," +
                "   \"transform-results\": {" +
                "      \"apply\": \"raw\"" +
                "   }" +
                "}}";


QueryOptionsManager optionsMgr =
  dbClient.newServerConfigManager().newQueryOptionsManager();
optionsMgr.writeOptions("myopt", 
    new StringHandle(xmlOptions).withFormat(Format.XML));
// Or, with JsonOptions:
    new StringHandle(jsonOptions).withFormat(Format.JSON));

This section contains the following examples:

Example: Date Range Structured Query
Example: Element Index Structured Query
Example: Document Property Structured Query
Example: Directory Structured Query
Example: Document Structured Query
Example: JSON Property Structured Query
Example: Collection Structured Query