Loading TOC...
Entity Services Developer's Guide (PDF)

Entity Services Developer's Guide — Chapter 6

Querying a Model or Entity Instances

This chapter contains the following topics related to searching entity instances and models using MarkLogic. Unless otherwise noted, all the examples in this chapter use the entity model and data from Getting Started With Entity Services.

This chapter covers the following topics:

Additional examples are available in the Entity Services GitHub repository. For more details, see Exploring the Entity Services Open-Source Examples.

Query Support Provided by Entity Services

The Entity Services API includes the following utility functions that make it easier to create and configure an application that searches entity models and entity instances.

You can customize all of these generated artifacts to suit the requirements of your application.

You are not required to generate and use any of these artifacts, but doing so can make it easier to build a search application around your model. The examples in this chapter take advantage of these artifacts where appropriate.

Search Basics for Models

You can use Semantic search to search and make inferences about a model.

Recall that when you persist a model descriptor as part of the special Entity Services collection, MarkLogic generates a set of facts that define the core of your model, expressed as semantic triples. You can also enrich your model with additional facts (triples) that are not derivable from the model descriptor. For details, see Introduction.

The auto-generated triples include facts such as the following. For the complete ontology, see MARKLOGIC_INSTALL_DIR/Modules/MarkLogic/entity-services/entity-services.ttl.

  • Model M defines entity type T
  • Entity type T has a property P
  • Property P of entity type T has data type D
  • Entity Type T has primary key P

You can inspect all the triples associated with a model by evaluating a SPARQL query such as the following in Query Console:

XQuery Server-Side JavaScript
xquery version "1.0-ml";
cts:triples(
  (), (), (), (), (),
  cts:document-query(yourModelURI))
'use strict';
cts.triples(
  null, null, null, null, null, 
  cts.documentQuery(yourModelURI));

You can use SPARQL or the Optic API to perform a semantic search of the model. The following interfaces accept SPARQL input:

  • The sem:sparql XQuery function or the sem.sparql Server-Side JavaScript function.
  • The REST, Java, and Node.js client APIs accept SPARQL queries as input to their search interfaces. You can embed a SPARQL query in a combined query, or use an appropriate Java or Node.js query builder.
  • Evaluate SPARQL directly in Query Console during development.

For a server-side model query example, see Example: Using SPARQL for Model Queries. For the Client APIs, refer to the respective developer guides listed in Where to Find Additional Information.

The Optic API enables semantic queries directly using JavaScript and XQuery, without requiring you to use a secondary query language (SPARQL). You can use the Optic API to query your model server-side using the op:from-triples XQuery function or the op.fromTriples Server-Side JavaScript function. For more details, see Optic API for Multi-Model Data Access in the Application Developer's Guide.

Search Basics for Instance Data

You can query your instance data as documents, rows, or triples. See the following topics for more details:

Document search is always available. Row and semantic search are only available if you generate and install a TDE template, as described in Generating a TDE Template. In addition, semantic search is only available if an entity type defines a primary key.

Document Search

If you follow the Entity Services conventions, your instance data, as well as original source data is stored in envelope documents. The default structure of envelope documents is covered in What is an Envelope Document?.

You can use any of the available document search interfaces to search your envelope documents. For example:

To learn more about any of these interfaces, see the links in Where to Find Additional Information.

The Search API and the Client APIs can take advantage of the query options you can generate using the Entity Services API. These options can help streamline and customize your searches. See the examples and Generating Query Options for Searching Instances.

You can also generate a database configuration artifact based on your model. The artifact includes index configuration for selected properties identified in the model. Creating these indexes can enhance search performance. For details, see Generating a Database Configuration Artifact.

Row Search

You can search your entity instance data as rows if you generate and install a TDE template based on your model. Broadly speaking there is an implicit table that corresponds to each entity type, with a row for each instance and columns for each property. For more details, see Generating a TDE Template.

You can use SQL or the Optic API to search your entities as rows using the following interfaces:

You can also evaluate SQL directly in Query Console during development.

For more information about these interfaces, see the resources listed in Where to Find Additional Information.

Semantic Search

You can search your entity instances using semantic queries if and only if all of the following conditions are met:

When these requirements are met, MarkLogic automatically generates a few facts about each instance when you insert an envelope document into the database. The facts take the form of semantic triples, which you can query using SPARQL or the Optic API. You can also extend the TDE template to include your own triples.

For an example of semantic queries on instance data, see Example: Using SPARQL for Instance Queries and Example: Using the Optic API for Instance Queries.

You can use the following interfaces to perform a semantic search of your entity instance data:

You can also evaluate SPARQL directly in Query Console during development.

To learn more about these interfaces, see the resources listed in Where to Find Additional Information.

Pre-Installing Query Options

Recall that you can generate and customize model-specific query options for use with the Search API and the REST, Java, and Node.js Client APIs; see Generating Query Options for Searching Instances.

You must pre-install these options on MarkLogic if and only if all the following are true:

  • You search your model or entity instances using one of the Client APIs (REST, Java, or Node.js).
  • You do not want to specify options dynamically at query time, such as in a combined query.

You can install query options using the REST and Java Client APIs. For details, see the following topics:

You can use persistent query options with the Node.js Client API, but you cannot install them. Use REST or Java instead.

Example: Using SPARQL for Model Queries

When you insert a model descriptor document into MarkLogic as part of the special Entity Services collection, MarkLogic creates a model from the descriptor. The model is expressed as semantic triples; for details, see Search Basics for Models.

You can also extend the model with your own triples; for details, see Extending a Model with Additional Facts.

You can query triples in MarkLogic using the following APIs:

The following SPARQL query returns the name of all required properties of the Person entity type of the model created in Getting Started With Entity Services.

prefix es:<http://marklogic.com/entity-services#>
select ?ptitle
where {
  ?x a es:EntityType;
       es:title "Person";
       es:property ?property .
  ?property a es:RequiredProperty;
              es:title ?ptitle
}

If you run this query in Query Console against the data from Getting Started With Entity Services, it will return the property names lastName, firstName, and fullName.

The following example uses sem:sparql or sem.sparql to evaluate the same SPARQL query.

Language Example
XQuery
xquery version "1.0-ml";
sem:sparql('
  prefix es:<http://marklogic.com/entity-services#>
  select ?ptitle
  where {
    ?x a es:EntityType;
         es:title "Person";
         es:property ?property .
    ?property a es:RequiredProperty;
                es:title ?ptitle
  }'
)
JavaScript
sem.sparql(
  'prefix es:<http://marklogic.com/entity-services#> ' +
  'select \?ptitle ' +
  'where {' +
    '?x a es:EntityType;' +
         'es:title "Person";' +
         'es:property ?property .' +
    '?property a es:RequiredProperty;' +
                'es:title ?ptitle' +
  '}'
)

Example: Using cts:query or cts.query for Instance Queries

The cts query interface serves as the foundation for most higher level document search APIs in MarkLogic. Using the cts layer gives you fine-grained control over your searches while the XQuery Search API, JavaScript JSearch API, and the Client APIs provide higher level abstractions on top of this layer. For details, see APIs for Multiple Programming Languages in the Search Developer's Guide.

The following example uses the cts:search XQuery function or cts.search JavaScript function to find all Person envelope documents where the instance data includes a lastName element with the value washington. For the sake of simplicity, the example prints out just the value of the fullName property in the matched documents, rather than complete documents.

Language Example
XQuery
xquery version "1.0-ml";

cts:search(fn:collection('person-envelopes'),
  cts:element-query(
    fn:QName("http://marklogic.com/entity-services", "instance"),
    cts:element-value-query(xs:QName("lastName"), "washington")
  )
)//fullName/fn:data()
JavaScript
const results = cts.search(cts.andQuery((
  cts.collectionQuery('person-envelopes'),
  cts.elementQuery(
    fn.QName('http://marklogic.com/entity-services', 'instance'),
    cts.elementValueQuery(xs.QName('lastName'), 'washington')
  )
)));

// Accumulate the matched names in an array for easy display 
// in Query Console.
const names = [];
for (const doc of results) {
  names.push(doc.xpath('//Person/fullName/fn:data()'));
}
names

You could also use a path query instead of an element query to limit the search to es:instance elements.

If you run the example code in Query Console against the envelope documents created in Getting Started With Entity Services, the results are George Washington and Martha Washington.

Example: Using the Search API for Instance Queries

The XQuery Search API is an interface that abstracts away some of the complexity of cts:search operations such as the generation of facets and snippets. For details, see Search API: Understanding and Using in the Search Developer's Guide.

Server-Side JavaScript developers should use the JSearch API instead of the XQuery Search API. You can use the Search API from JavaScript, but the search configuration and results are expressed in XML, so it is not as convenient or natural. See Example: Using JSearch for Instance Queries, instead.

Recall that you can generate Search API compatible query options using the Entity Services API; for details, see Generating Query Options for Searching Instances. The code samples in this section assume you generated options from the model in Getting Started With Entity Services. To learn more about the generated options, see Characteristics of the Generated Options.

The following example uses generated options to find all Person envelope documents where the instance data includes the word washington. For simplicity, only the value of the fullName property is displayed. (In practice, you would probably customize the generated options for your application.)

xquery version "1.0-ml";
import module namespace search =
  "http://marklogic.com/appservices/search"
  at "/MarkLogic/appservices/search/search.xqy";
import module namespace es = "http://marklogic.com/entity-services"
  at "/MarkLogic/entity-services/entity-services.xqy";

let $options := es:search-options-generate(
  fn:doc('/es-gs/models/person-1.0.0.json'))
let $matches := 
  search:search("entity-type:Person AND washington", $options)
return $matches//Person/fullName/fn:data()

If you run this code in Query Console against the envelope documents created in Getting Started With Entity Services, then you should see output similar to the following:

Martha Washington
George Washington

The search term entity-type:Person constrains the search to Person entities. The entity-type constraint is automatically generated for all models.

The generated options also include an additional-query option that constrains results to the instance data in an envelope document. For example:

<search:constraint name="entity-type">
  <search:value>
    <search:element ns="http://marklogic.com/entity-services" name="title"/>
  </search:value>
</search:constraint>

<search:additional-query>
  <cts:element-query xmlns:cts="http://marklogic.com/cts">
    <cts:element xmlns:es="...">es:instance</cts:element>
    <cts:true-query/>
  </cts:element-query>
</search:additional-query>

Though the code above returns just the value of the fullName property in each matched instance, the search results contain the entire entity, as if you called es:entity-from-document on the envelope document. This data is contained in the search:extracted element of each search:result. For example:

<search:response snippet-format="empty-snippet" total="2" start="1" 
      page-length="10" selected="include" xmlns:search=...>
  <search:result index="1" uri="/es-gs/env/2345.xml" 
      path="fn:doc(&quot;/es-gs/env/2345.xml&quot;)" score="15872" 
      confidence="0.4703847" fitness="0.7823406">
    <search:snippet/>
    <search:extracted kind="element">
      <Person>
        <id>2345</id>
        <firstName>Martha</firstName>
        <lastName>Washington</lastName>
        <fullName>Martha Washington</fullName>
      </Person>
    </search:extracted>
  </search:result>
  <search:result .../>
  <search:qtext>entity-type:Person AND washington</search:qtext>
  <search:metrics>...</search:metrics>
</search:response>

The generated options enable this behavior by disabling snippeting and faceting, and defining an extract-document-data option that extracts just the instance from the envelope document. For example:

<search:extract-document-data selected="include">
  <search:extract-path xmlns:es=...>//es:instance/(Person)</search:extract-path>
</search:extract-document-data>

<search:additional-query>
  <cts:element-query xmlns:cts="http://marklogic.com/cts">
    <cts:element xmlns:es=...>es:instance</cts:element>
    <cts:true-query/>
  </cts:element-query>
</search:additional-query>

<search:return-facets>false</search:return-facets>
<search:transform-results apply="empty-snippet"/>

If the model included more than one entity type definition, then the extract-document-data option would use an extract path that matched any of the defined types. For example, if the model defines a second entity type named Family, then the extract path would be the following:

//es:instance/(Family|Person)

If an entity type definition includes range index or word lexicon specifications, then the options would include additional range or word constraints options. For example, if we extend the Person entity to include a rating property of type float with a pathRange-index specification, then the generated options would include a path range constraint similar to the following:

<search:constraint name="rating">
  <search:range type="xs:float" facet="true">
    <search:path-index xmlns:es="http://marklogic.com/entity-services">
        //es:instance/Person/rating
    </search:path-index>
  </search:range>
</search:constraint>

This enables a query string such as entity-type:Person AND rating GT 3.0.

For an example of a complete set of generated options, see Example: Generating Query Options.

To learn more about query options, see Search Customization Using Query Options and Appendix: Query Options Reference in the Search Developer's Guide.

Example: Using JSearch for Instance Queries

The JSearch API is a fluent Server-Side JavaScript search interface. You can use it to search documents using a variety of query styles, as well as for querying lexicons and range indexes. For details, see Creating JavaScript Search Applications in the Search Developer's Guide.

The following example use a cts.query to find all Person envelope documents where the instance data includes a lastName element with the value washington. For the sake of display simplicity, a custom mapper is used to extract just the value of the fullName property from each matched instance, instead of returning full search results.

'use strict';
const jsearch = require('/MarkLogic/jsearch.sjs');

jsearch.collections('person-envelopes').documents()
  .where(cts.elementQuery(
      fn.QName('http://marklogic.com/entity-services', 'instance'),
      cts.elementValueQuery('lastName', 'washington')))
  .map(function(match) { 
         return match.document.xpath('//fullName/fn:data()'); 
       })
  .result();

If you run the example in Query Console against the envelope documents created in Getting Started With Entity Services, the results should be similar to the following:

{ "results":[
    "Martha Washington", 
    "George Washington"], 
  "estimate":2}

Example: Using the Client APIs for Instance Queries

This section provides examples of querying instances with the REST, Java, and Node.js Client APIs. Note that these APIs support more query styles than are shown here. For details, refer to the development guide for each API. These guides are listed in Where to Find Additional Information.

Java Client API

The Java Client API is an API for creating client applications that interact with MarkLogic. The API enables you to search documents using a variety of query styles. For more details, see the Java Application Developer's Guide and the Java Client API Documentation. The Java Client API can take advantage of the Search API compatible query options you can generate with the Entity Services API, as discussed in Generating Query Options for Searching Instances.

The following example uses a string query to find all Person envelope documents where the instance data includes the word washington. The code assumes you have already generated query options using the Entity Services API and installed them on MarkLogic as persistent query options under the name OPTIONS_NAME; see the complete example below for an example of how to install the options.

QueryManager qm = client.newQueryManager();
StringQueryDefinition query = 
    qm.newStringDefinition(OPTIONS_NAME)
      .withCriteria("entity-type:Person AND washington");
SearchHandle results = qm.search(query, new SearchHandle());

For a discussion of how the generated options enable this query string, see Example: Using the Search API for Instance Queries.

You could also create a RawCombinedQueryDefinition and embed the generated options inside the combined query. This enables you to use the generated options without first persisting them on MarkLogic. For more details, see Apply Dynamic Query Options to Document Searches in the Java Application Developer's Guide.

The following code is a complete example of installing options and performing the above search. This code installs the query options (if necessary), performs the search, and prints out the value of the fullName property in the matched entities.

Modify the values in bold to fit your environment.

package examples;

import java.io.File;

import com.marklogic.client.DatabaseClient;
import com.marklogic.client.DatabaseClientFactory;
import com.marklogic.client.admin.QueryOptionsManager;
import com.marklogic.client.io.FileHandle;
import com.marklogic.client.io.Format;
import com.marklogic.client.io.QueryOptionsListHandle;
import com.marklogic.client.io.SearchHandle;
import com.marklogic.client.query.ExtractedItem;
import com.marklogic.client.query.ExtractedResult;
import com.marklogic.client.query.MatchDocumentSummary;
import com.marklogic.client.query.QueryManager;
import com.marklogic.client.query.StringQueryDefinition;

import javax.xml.xpath.XPathExpression;
import javax.xml.xpath.XPathExpressionException;
import javax.xml.xpath.XPathFactory;

import org.w3c.dom.Document;


public class EntityServices {
    private static DatabaseClient client = 
        DatabaseClientFactory.newClient(
            "localhost", 8000, "es-gs",
            new DatabaseClientFactory.DigestAuthContext(USER, PASSWORD));
    static String OPTIONS_NAME = "person-1.0.0";
    static String OPTIONS_PATHNAME = 
        "/path/to/options/person-options-1.0.0.xml";
    
    // Install the options generated by ES, if needed.
    public static void installOptions(String filename, String optionsName) {
        QueryOptionsManager optMgr = 
            client.newServerConfigManager()
                  .newQueryOptionsManager();
        QueryOptionsListHandle optList = 
            optMgr.optionsList(new QueryOptionsListHandle());

        if (optList.getValuesMap().get(OPTIONS_NAME) == null) {
            FileHandle options = 
                new FileHandle(new File(filename))
                    .withFormat(Format.XML);
            optMgr.writeOptions(optionsName, options);
        }
    }
    
    public static void main(String[] args) throws XPathExpressionException {
        // Install the options generated by ES, if necessary
        installOptions(OPTIONS_PATHNAME, OPTIONS_NAME);
        
        // Build the query
        QueryManager qm = client.newQueryManager();
        StringQueryDefinition query = 
            qm.newStringDefinition(OPTIONS_NAME)
              .withCriteria("entity-type:Person AND washington");
        
        // Perform the search
        SearchHandle results = qm.search(query, new SearchHandle());
        
        // Iterate over the results, and write out just the value of
        // the "fullName" property.
        XPathExpression xpath =
            XPathFactory.newInstance().newXPath().compile("//fullName");
        for (MatchDocumentSummary match : results.getMatchResults()) {
            ExtractedResult extracted = match.getExtracted();
            for (ExtractedItem item : extracted) {
                Document person = item.getAs(Document.class);
                System.out.println(xpath.evaluate(person));
            }
        }

        client.release();
    }
}

If you run this example, it will print the values Martha Washington and George Washington.

As discussed in Example: Using the Search API for Instance Queries, the matched entities are returned as extracted items in the search response. The following part of the example iterates over the search results, accesses the extracted entity data, and then prints out just the value of the fullName property. The person variable holds the entity, as a DOM Document.

XPathExpression xpath =
    XPathFactory.newInstance().newXPath().compile("//fullName");
for (MatchDocumentSummary match : results.getMatchResults()) {
    ExtractedResult extracted = match.getExtracted();
    for (ExtractedItem item : extracted) {
        Document person = item.getAs(Document.class);
        System.out.println(xpath.evaluate(person));
    }
}

Node.js Client API

The Node.js Client API enables you to create Node.js client applications that interact with MarkLogic. The API enables you to search documents using a variety of query styles. For more details, see the Node.js Application Developer's Guide and the Node.js API Reference.

Recall that you can generate Search API compatible query options using the Entity Services API; for details, see Generating Query Options for Searching Instances. You can only take advantage of these options if you pre-install them as described in Pre-Installing Query Options and then reference them in a combined query.

However, you can use the Node.js query builder to create equivalent behavior without using the generated options. This section explores both approaches:

Search Using Pre-Installed Options

This example uses a combined query and pre-installed query options. The example assumes you generated options from the model in Getting Started With Entity Services, and then installed the options on MarkLogic with the name person-1.0.0. You can install the options using the REST Client API or Java Client API; for details, see Pre-Installing Query Options.

The following example finds all Person envelope documents where the instance data includes the word washington. The search returns just the matched instance data, as serialized XML.

const marklogic = require('marklogic');

// MODIFY THIS VAR TO MATCH YOUR ENV
const connInfo = {
    host: 'localhost',
    port: 8000,
    user: 'username',
    password: 'password',
    database: 'es-gs'
  };
const db = marklogic.createDatabaseClient(connInfo);
const qb = marklogic.queryBuilder;

// entity-type is a constraint defined by the options.
// The options should already be installed, with name 'person-1.0.0'.
const combinedQuery = {
  search: { 
    query: 'entity-type:Person AND washington'
  },
  optionsName: 'person-1.0.0'
};

db.documents.query(
  { search: {
      qtext: 'entity-type:Person AND washington'
    },
    optionsName: 'person-1.0.0'
  }
).result( function(results) {
  for (let result of results) {
    console.log(JSON.stringify(result.content));
  }
});

The query matches entities with fullName property values of Martha Washington and George Washington. The options limit the returned data to just the matched entities through the extract-document-data option. Since the envelope documents are XML, each extracted entity is returned as a string containing serialized XML, with a root element of <search:extracted/>. For example, the result for Martha Washington looks like the following. (Line breaks have been added for readability; the value of the content property is one string.)

{ "uri":"/es-gs/env/2345.xml",
  "category":"content",
  "format":"xml",
  "contentType":"application/xml",
  "contentLength":"394",
  "content":
    "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n
     <search:extracted kind=\"element\" format=\"xml\"
       context=\"fn:doc(&quot;/es-gs/env/2345.xml&quot;)\"
       xmlns:search=\"http://marklogic.com/appservices/search\">
      <Person>
        <id>2345</id>
        <firstName>Martha</firstName>
        <lastName>Washington</lastName>
        <fullName>Martha Washington</fullName>
      </Person>
    </search:extracted>"}
Search Without Pre-Installing Options

The following example uses the Node.js queryBuilder interface to perform a search equivalent to Search Using Pre-Installed Options. This approach requires a more in-depth understanding of the relationship between the builder interface and the underlying Search API query options.

Before you can run this example, you must configure the REST Client API instance through which you connect to MarkLogic so that it defines a namespace binding for the prefix es. The binding is required because the example uses queryBuilder.extract to extract just the es:instance portion of an envelope document.

The Node.js Client API does not directly support namespace binding configuration, so this example uses the REST Client API and the curl command line tool to do so. For more details, see Using Namespace Bindings in the REST Application Developer's Guide. You can replace the use of curl with any tool that can send HTTP requests.

Run the following command to define a binding between the es prefix and the http://marklogic.com/entity-services. Change the user, password, host, and port as needed to match your environment.

# Windows users, see Modifying the Example Commands for Windows 
curl --anyauth --user user:password -X PUT \
  -d '{ "prefix": "es", "uri": "http://marklogic.com/entity-services" }' \
  -H "Content-type: application/json" -i \
  http://localhost:8000/v1/config/namespaces/es

If the command is successful, MarkLogic returns a 201 Created status.

The following Node.js script finds all Person envelope documents where the instance data includes the word washington. The search returns just the matched instance data, as serialized XML. A discussion of the relationship between the built query below and the generated query options follows.

const marklogic = require('marklogic');

// MODIFY THIS VAR TO MATCH YOUR ENV
const connInfo = {
    host: 'localhost',
    port: 8000,
    user: 'username',
    password: 'password',
    database: 'es-gs'
  };
const db = marklogic.createDatabaseClient(connInfo);
const qb = marklogic.queryBuilder;

db.documents.query(
  qb.where(
    qb.collection('person-envelopes'),
    qb.scope( 
      qb.element(
        qb.qname('http://marklogic.com/entity-services','instance')),
      qb.and()),
    qb.parsedFrom('entity-type:person AND washington',
      qb.parseBindings(
        qb.value(
          qb.element(
            qb.qname('http://marklogic.com/entity-services','title')),
          qb.bind('entity-type'))))
  ).slice(qb.extract({
       paths: ['//es:instance/(Person)'],
       selected: 'include'
     }))
).result( function(results) {
  for (let result of results) {
    console.log(JSON.stringify(result.content));
  }
});

Use qb.scope to create a container query that mimics the generated additional-query option restricting results to matches within es:instance elements.

Description Example
Generated Option
<search:additional-query>
  <cts:element-query xmlns:cts="...">
    <cts:element xmlns:es="http://marklogic.com/entity-services">
      es:instance
    </cts:element>
    <cts:true-query/>
  </cts:element-query>
</search:additional-query>
Node.js Equivalent
qb.scope( 
  qb.element(
    qb.qname('http://marklogic.com/entity-services',
             'instance')),
    qb.and())

Use a parse binding to bind the tag entity-type to the title element of an entity instance so that you can constrain string queries to specific entity types. The bind enables search terms such as entity-type:Person.

Description Example
Generated Option
<search:constraint name="entity-type">
  <search:value>
    <search:element ns="http://marklogic.com/entity-services"
                    name="title"/>
  </search:value>
</search:constraint>
Node.js Equivalent
qb.parseBindings(
  qb.value(
    qb.element( qb.qname(
      'http://marklogic.com/entity-services','title')),
    qb.bind('entity-type')))

If your entity type definition assigns properties to range indexes or word lexicons, the generated options will include additional named constraints. You can define similar parse bindings for these constraints. For more details, see Using Constraints in a String Query in the Node.js Application Developer's Guide.

Use qb.slice(qb.extract...)) to mimic the behavior of the extract-document-data option. This causes the search to return just the matched entity instance, instead of the entire envelope document. This is the section of the query that required us to define a namespace prefix binding for es.

Description Example
Generated Option
<search:extract-document-data selected="include">
  <search:extract-path xmlns:es="http://marklogic.com/entity-services">
    //es:instance/(Person)
  </search:extract-path>
</search:extract-document-data>
Node.js Equivalent
qb.where(...)
  .slice(qb.extract({
     paths: ['//es:instance/(Person)'],
     selected: 'include'
  }))

REST Client API

The REST Client API enables client applications to interact with MarkLogic using HTTP requests. The API enables you to search documents using a variety of query styles, including string query, structured query, QBE, and combined query. For more details, see Using and Configuring Query Features in the REST Application Developer's Guide.

Recall that you can generate Search API compatible query options using the Entity Services API; for details, see Generating Query Options for Searching Instances. To take advantage of these option, you must either pre-install the options as described in Pre-Installing Query Options, or embed them in a combined query.

The following command uses a string query to find all Person envelope documents where the instance data contains the word washington. The command uses a string query and assumes the options are pre-installed under the name person-1.0.0. The search is performed by a request to GET /v1/search.

# Windows users, see Modifying the Example Commands for Windows 
$ curl --anyauth --user user:password -X GET -i \
    'http://localhost:8000/LATEST/search?q=entity-type:person AND washington&options=person-1.0.0&database=es-gs'

If you run the command against the model and instance data from Getting Started With Entity Services, the request returns the entity instance data for Martha Washington and George Washington in the <search:extracted/> element of the response. For example:

<search:response snippet-format="empty-snippet" total="2" 
    start="1" page-length="10" selected="include" 
    xmlns:search="http://marklogic.com/appservices/search">
  <search:result index="1" uri="/es-gs/env/2345.xml" 
      path="fn:doc(&quot;/es-gs/env/2345.xml&quot;)" 
      score="15872" confidence="0.4703847" fitness="0.7823406" 
      href="/v1/documents?uri=%2Fes-gs%2Fenv%2F2345.xml&amp;database=es-ex" 
      mimetype="application/xml" format="xml">
    <search:snippet/>
    <search:extracted kind="element">
      <Person>
        <id>2345</id>
        <firstName>Martha</firstName>
        <lastName>Washington</lastName>
        <fullName>Martha Washington</fullName>
      </Person>
    </search:extracted>
  </search:result>
  ...
  <search:qtext>entity-type:person AND washington</search:qtext>
  <search:metrics>...</search:metrics>
</search:response>

The response includes only the matched entity instances because of the extract-document-data option. For a discussion of the generated options used in this example, see Example: Using the Search API for Instance Queries.

You can use the request Accept headers to retrieve results as JSON, but the extracted property value in the JSON response will contain serialized XML because entity data is stored as XML in the envelope documents.

To perform an equivalent search without pre-installing the options use a combined query that embeds the options in a <search:search/> element. Use the combined query as the request body for POST /v1/search. For example, create a combined query of the following form:

<search xmlns="http://marklogic.com/appservices/search">
  <qtext>entity-type:Person AND washington</qtext>
  <options> <!-- the generated options here -->
    ...
  </options>
</search>

For more details, see Specifying Dynamic Query Options with Combined Query in the REST Application Developer's Guide.

Example: Using SPARQL for Instance Queries

The default TDE template that you can generate with the Entity Services API auto-generates triples from your entity envelope documents, as long as the instance entity type defines a primary key.

You must install the template before this triple generation can occur. For details, see Generating a TDE Template.

The default generated triples express facts such as the following, where the instance is identified by primary key. For more details, see Characteristics of a Generated Template.

  • This instance has this entity type. For example, this triple expresses the fact that an entity instance has the type defined by the IRI <http://example.org/example-person/Person-1.0.0/Person> The type IRI takes the form of {baseURI}{modelTitle}-{modelVersion}/{entityTypeName}.
    <http://example.org/example-person/Person-1.0.0/Person/1234>
      <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
      <http://example.org/example-person/Person-1.0.0/Person>
  • This instance is defined by this envelope document (identified by URI). For example, the following triple expresses the fact that a particular entity instance is defined by the envelope document with URI /es-gs/env/1234.xml. The entity instance IRI takes the form of {baseURI}{modelTitle}-{modelVersion}/{entityTypeName}/{primaryKey}.
    <http://example.org/example-person/Person-1.0.0/Person/1234>
      <http://www.w3.org/2000/01/rdf-schema#isDefinedBy>
      "/es-gs/env/1234.xml"^^xs:anyURI

You can also extend the template to generate additional triples or manually add triples to the database.

The following SPARQL query returns the URIs of all Person entities.

prefix es: <http://marklogic.com/entity-services#>
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
prefix xs: <http://www.w3.org/2001/XMLSchema#>
 
select ?uri
where {
  ?person a ?personType .  
  ?person rdfs:isDefinedBy ?docUri .
  ?personType es:title 'Person'
  bind(xs:string(?docUri) as ?uri)
}

If you generate and install a TDE template using the model from Getting Started With Entity Services, then the query display the following entity envelope document URIs:

/es-gs/env/1234.xml
/es-gs/env/2345.json
/es-gs/env/3456.xml

You can query facts about your instance data using the following APIs.

Example: Using SQL for Instance Queries

If you generate and install a TDE template for your model, then MarkLogic auto-generates row data from your entity envelope documents. The row data enables you to query your entity instances as rows.

You must install the template before this row generation can occur. For details, see Generating a TDE Template. To learn more about the characteristics of the row data, see Characteristics of a Generated Template.

You can evaluate SQL using the xdmp:sql XQuery function or the xdmp.sql Server-Side JavaScript function, as shown below. You can also use the Optic API to query row data; see Example: Using the Optic API for Instance Queries.

The following example finds all Person rows where the lastName column has the value Washington and returns the value of the fullName column for the matched rows.

Language Example
XQuery
xquery version "1.0-ml";
xdmp:sql("
  SELECT Person.fullName
  FROM Person
  WHERE Person.lastName='Washington'
", "format")
JavaScript
xdmp.sql(
  'SELECT Person.fullName ' +
  'FROM Person ' +
  'WHERE Person.lastName=\'Washington\''
, "format");

If you generate and install a TDE template from the model from Getting Started With Entity Services and run the query against the instance data, then you should see output similar to the following:

| Person.Person.fullName| 
| Martha Washington| 
| George Washington|

Example: Using the Optic API for Instance Queries

If you generate and install a TDE template for your model, then MarkLogic auto-generates row data from your entity envelope documents. The row data enables you to query your entity instances as rows. If an entity defines a primary key, the template also causes MarkLogic to auto-generate semantic triples about each instance.

You must install the template before this auto-generation can occur. For details, see Generating a TDE Template.

The examples in this section are based on the model and instance data from Getting Started With Entity Services. The examples also assume you have generated and installed a template based on this model, as shown in Generating a TDE Template.

Querying Triples Using the Optic API

This example uses the Optic API to query semantic facts about instance data. You can also use the Optic API for semantic queries on an entity model. For examples using SPARQL, see Example: Using SPARQL for Instance Queries and Example: Using SPARQL for Model Queries.

The following example finds all entity instances that have Person type.

Language Example
XQuery
import module namespace op = 
    "http://marklogic.com/optic" at "/MarkLogic/optic.xqy";

let $ps :=
  op:prefixer("http://example.org/example-person/Person-1.0.0/")
let $rdf :=
  op:prefixer("http://www.w3.org/1999/02/22-rdf-syntax-ns#")
return
  op:from-triples( ( op:pattern( op:col("instanceIri"),
                                 $rdf("type"),
                                 op:col("type") ) ) )
          =>op:where( op:eq( op:col("type"), $ps("Person")))
          =>op:result()
JavaScript
const op = require('/MarkLogic/optic');
const ps =
  op.prefixer('http://example.org/example-person/Person-1.0.0/');
const rdf =
  op.prefixer('http://www.w3.org/1999/02/22-rdf-syntax-ns#');
        
op.fromTriples( op.pattern( op.col('instanceIri'),
                             rdf('type'),
                             op.col('type')) )
    .where(op.eq(op.col('type'), ps('Person')))
    .result();

If you run the query in Query Console against the expected configuration, it matches the following instance IRIs:

http://example.org/example-person/Person-1.0.0/Person/1234
http://example.org/example-person/Person-1.0.0/Person/2345
http://example.org/example-person/Person-1.0.0/Person/3456
Querying Rows Using the Optic API

This example uses the Optic API to query instance data as rows. For examples using SQL, see Example: Using SQL for Instance Queries.

The following query finds the Person entity with an id property of 2345. Each entity instance is represented by a row in the Person table, with a column for each property.

Language Example
XQuery
import module namespace op = 
  "http://marklogic.com/optic" at "/MarkLogic/optic.xqy";
import module namespace opxs =
  "http://marklogic.com/optic/expression/xs" 
  at "/MarkLogic/optic/optic-xs.xqy";

op:from-view("Person", "Person")
        =>op:where(op:eq(op:col("id"), opxs:string("2345")))
        =>op:select( (op:col("fullName")) )
        =>op:result()
JavaScript
var op = require("/MarkLogic/optic");
var opxs = op.xs;

op.fromView("Person", "Person")
.where(op.eq(op.col("id"), opxs.string("2345")))
.select( [op.col("fullName") ] )  
.result();

If you run the query in Query Console against the expected configuration, it returns Martha Washington.

« Previous chapter