MarkLogic Server 11.0 Product Documentation

Entity Services Developer's Guide — Chapter 4

Purpose of a Converter Module

An instance converter is a key part of a model-driven application. The instance converter provides functions that facilitate the following tasks:

• Creating an entity instance from raw source data.
• Creating an entity envelope document that encapsulates an instance, metadata, and raw source data.
• Extracting a canonical instance or its attachments (such as the raw source) from an envelope document.

For more details on envelope documents, see What is an Envelope Document?.

You usually use the instance converter to create entity instance envelope documents and to extract canonical instances for use by downstream entity consumers.

You are expected to customize the generated converter module to meet the needs of your application.

Generating a Converter Module Template

Generate an instance converter from the JSON object-node or json:object representation of a model descriptor by calling the XQuery function es:instance-converter-generate or the JavaScript function es.instanceConverterGenerate. The result is an XQuery library module containing both model-specific and entity type specific code.

The input to the generator is a JSON descriptor. If you have an XML descriptor, you must first convert it to the expected format; for details, see Working With an XML Model Descriptor. The output of the generator function is an XQuery library module.

You can use the generated code as-is, but most applications will require customization of the converter implementation. For details, see Customizing a Converter Module.

The following example code generates a converter module from a previously persisted descriptor, and then saves the generated code as a file on the filesystem.

xquery version "1.0-ml";
import module namespace es =
  "http://marklogic.com/entity-services"
  at "/MarkLogic/entity-services/entity-services.xqy";

let $desc := fn:doc('/es-gs/models/person-1.0.0.json')
return xdmp:save(
  '/space/es/gs/person-1.0.0-conv.xqy',
  es:instance-converter-generate($desc)
)

'use strict';
const es = require('/MarkLogic/entity-services/entity-services.xqy');

const desc = cts.doc('/es-gs/models/person-1.0.0.json');
xdmp.save(
  '/space/es/gs/person-1.0.0-conv.xqy',
  es.instanceConverterGenerate(desc)
);

You could also insert the converter directly into the modules database, but the converter is an important project artifact that should be placed under source control. You will want to track changes to it as your application evolves.

Understanding the Default Converter Implementation

This section explores the default code generated for an instance converter module. The following topics are covered:

• Module Namespace Declaration
• Generated Functions

module namespace normalizedTitle = "baseUri/title-version";

"info": {
  "title": "Example",
  "version": "1.0.0",
  "baseUri": "http://marklogic.com/examples/"
}

module namespace example =
  "http://marklogic.com/examples/Example-1.0.0";

module namespace example =
  "http://marklogic.com/examples#Example-1.0.0";

ns:extract-instance-T

ns:instance-to-envelope

ns:instance-to-canonical

declare function ns:extract-instance-T(
  $source-node as node()
) as map:map
{
  normalize the input source reference
  initialize variables for the values of each entity property
  initialize an empty instance of type T
  attach the source data to the instance
  assign values to the instance properties
};

declare function example:extract-instance-Name(
    $source-node as node()
) as map:map
{
    let $source-node := es:init-source($source, 'Person')
    (: begin customizations here :)
    let $id  := $source-node/id ! xs:string(.)
    let $firstName  := $source-node/firstName ! xs:string(.)
    let $lastName  := $source-node/lastName ! xs:string(.)
    let $fullName  := $source-node/fullName ! xs:string(.)
    (: end customizations :)
    let $instance := es:init-instance($source-node, 'Person')
    (: Comment or remove the following line to suppress attachments :)
        =>es:add-attachments($source)

    return
    if (empty($source-node/*))
    then $instance
    else $instance
        =>   map:with('id', $id)
        =>   map:with('firstName', $firstName)
        =>   map:with('lastName', $lastName)
        =>   map:with('fullName', $fullName)
};

Customizing a Converter Module

Most customization involves changing the portion of each ns:extract-instance-T function that sets the values of the instance properties.

The default implementation of this portion of an extract function assumes that some property P in the entity instance gets its value from a node of the same name in the source data, and that a simple typecast is sufficient to convert the source value to the instance property type defined by the model.

For example, if an entity type named Person defines a string-valued property named firstName, then the generated code in firstName in example:extract-instance-Person related to initializing this property looks like the following:

let $firstName  := $source-node/firstName ! xs:string(.)
...
let $instance := es:init-instance($source-node, 'Person')
....
if (empty($source-node/*))
then $instance
else $instance
        ...
        =>   map:with('firstName', $firstName)
        ...

You might need to modify the code to perform a more complex transformation of the value, or extract the value from a different location in the source node. For example, if your source data uses the property name given to hold this information, then you would modify the generated code as follows:

let $firstName  := $source-node/given ! xs:string(.)

The following list describes other common customization use cases:

• Synthesize a property value from other data. For example, aggregate an instance property from other values in your source data, or extract a value from other sources, based on information in the source node.
• Normalize data formats. For example, data such as dates, telephone numbers, and social security numbers often occur in multiple formats in raw data. You can normalize such data to a single format in your instances for easy search and comparison.
• Assign a default value for missing data. If you know that a required property in your entity instance is not always present in your source data, you can modify the code to ensure the entity instance contains a reasonable default value.

Once you finish customizing the code, you must deploy the module to your App Server before you can use the code. For details, see Deploying Generated Code and Artifacts.

For a more complete example, see Getting Started With Entity Services or the Entity Services examples on GitHub. For details on locating the GitHub examples, see Exploring the Entity Services Open-Source Examples.

Purpose of a Version Translator

A version translator is an XQuery library module that helps you convert instance data conforming to one model version into another.

The version translator only addresses instance conversion. Model changes can also require changes to other artifacts, such as the TDE template, schema, query options, instance converter, and database configuration. For more details, see Managing Model Changes.

Though you can run the generated translator code as-is, it is meant to serve as a starting point for your customizations. Depending on the ways in which your source and target models differ, you might be required to modify the code.

Generating a Version Translator Module Template

Generate a version translator using the XQuery function es:version-translator-generate or the JavaScript function es.versionTranslatorGenerate. The output is an XQuery library module that you can customize and install in your modules database.

The inputs to the generator are source and target model descriptors, as JSON. If you have an XML descriptor, you must first convert it to the expected format; for details, see Working With an XML Model Descriptor.

You can use the generated code as-is, but most applications will require customization of the converter implementation. For details, see Customizing a Version Translator Module.

You must install the translator module in your modules database before you can use it. For details, see Deploying Generated Code and Artifacts.

The following example code generates a translator module from previously persisted descriptors, and then saves the generated code as a file on the filesystem. The resulting module is designed to convert instances of version 1.0.0 to instances of version 2.0.0.

xquery version "1.0-ml";
import module namespace es =
  "http://marklogic.com/entity-services"
  at "/MarkLogic/entity-services/entity-services.xqy";

let $v1 := fn:doc('/es-gs/models/person-1.0.0.json')
let $v2 := fn:doc('/es-gs/models/person-2.0.0.json')
return xdmp:save(
  '/space/es/gs/models/person-1.0.0-to-2.0.0.xqy',
  es:version-translator-generate($v1,$v2)
)

'use strict';
const es = require('/MarkLogic/entity-services/entity-services.xqy');

const v1 = cts.doc('/es-gs/models/person-1.0.0.json');
const v2 = cts.doc('/es-gs/models/person-2.0.0.json');
xdmp.save(
  '/space/es/gs/models/person-1.0.0-to-2.0.0.xqy',
  es.versionTranslatorGenerate($v1,$v2)
);

You could also insert the translator directly into the modules database, but the translator is an important project artifact that should be placed under source control. You will want to track changes to it as your application evolves.

Understanding the Default Version Translator Implementation

This section explores the default code generated for a version translator module. This information can help guide your customizations. This section covers the following topics:

• Module Namespace Declaration
• Generated Functions
• Customizing a Version Translator Module

module namespace title2-from-title1 = 
  "baseUri2/title2-version2-from-title1-version1";

"info": {
  "title": "Person",
  "version": "1.0.0",
  "baseUri": "http://example.org/example-person/"
}

"info": {
  "title": "Person",
  "version": "2.0.0",
  "baseUri": "http://example.org/example-person/"
}

module namespace person-from-person
  = "http://example.org/example-person/Person-2.0.0-from-Person-1.0.0";

module namespace person-from-person
  = "http://example.org/example-person#Person-2.0.0-from-Person-1.0.0";

Customizing a Version Translator Module

This section describes some common model changes, how they are handled by the default translation code, and when customizations are likely to be required.

Most of your translator customizations go in the block of variable declarations near the beginning of the conversion function. For example, the block of code shown in bold, below. These declarations set up the values to be assigned to the properties of the new instance, later in the conversion function. The variable names and default initial values are model-dependent.

declare function person-from-person:convert-instance-Person(
    $source as node()
) as map:map
{
    let $source-node := es:init-translation-source($source, 'Person')

    let $id := $source-node/id ! xs:string(.)
    let $firstName := $source-node/firstName ! xs:string(.)
    let $lastName := $source-node/lastName ! xs:string(.)
    let $fullName := $source-node/fullName ! xs:string(.)
    return...

The table below provides a brief overview of some common entity type definition changes and what customizations they might require. The context for the code snippets is the property value initialization block shown in the previous example. All the code snippets assume a required property; if the property under consideration is optional, then the call to map:with would be replaced by a call to es:optional.

(: atomic type (string, in this case) :)
let $thing := $source-node/thing ! xs:string(.)

(: array type (item type string, in this case) :)
let $thing := ex:extract-array($source-node/thing, xs:string#))

(: reference to a locally resolvable "Name" entity of type :)
let $extract-reference-Name := es:init-instance(?, 'Name')
let $thing := $soure-node/thing/* ! $extract-reference-Name(.)

let $rating := $source-node/rating ! xs:float(.)

let $rating := 
    es:extract-array($source-node/rating, xs:float#1))

(: Warning: potential data loss, truncated array.  :)
let $rating := xs:string( fn:head($source-node/rating) )

let $name := $source-node/name ! es:init-instance(?, 'Name')(.)

(: The following property was missing from the source type.
   The XPath will not up-convert without intervention.  :)
let $rating := $source-node/rating ! xs:float(.)

let $rating := 0 ! xs:float(.)

let $rating := $source-node/rating ! xs:float(.)=> map:with('rating', xs:float(
       $source//es:attachments/Person/rating/fn:data()))

(: The following properties are in the source, but not the target 
=> map:with('NO TARGET',
      xs:string($source-node/Person/address))
:)

let $first := $source-node/first ! xs:string(.)

let $first := $source-node/firstName ! xs:string(.)

Generating a TDE Template

Use the es:extraction-template-generate XQuery function or the es.extractionTemplateGenerate JavaScript function to create a TDE template. The input to the template generation function is a JSON or json:object representation of a model descriptor. You can use the template as-is, or customize it for your application. You must install the template before your application can benefit from it. For details, see Deploying a TDE Template.

Any hyphens (-) in the model title, entity type names, or entity property names are converted to underscores (_) when used in the generated template, in order to avoid invalid SQL names.

For example, the following code snippet generates a template from a model previously persisted in the database. For a more complete example, see Example: TDE Template Generation and Deployment.

es:extraction-template-generate(
  fn:doc('/es-gs/models/person-1.0.0.json'))

es.extractionTemplateGenerate(
  cts.doc('/es-gs/models/person-1.0.0.json'));

The template is an important project artifact that you should put under source control.

If you customize the template, you should validate it. You can use the tde:validate XQuery function or the tde.validate JavaScript function for standalone validation, or combine validation with insertion, as described in Deploying a TDE Template.

Characteristics of a Generated Template

A TDE template generated by the Entity Services API is intended to apply to entity envelope documents with the structure produced by an instance converter module. If you use a different structure, you will have to customize the template. For more details, see What is an Envelope Document?.

The generated template has the following characteristics:

• The default root context for the template matches instance data in both XML and JSON envelopes, assuming the envelopes conform to the Entity Services envelope convention. The generated template includes comments on how to change the context path for better performance if you only use a single envelope format (only XML or only JSON).
• A triples sub-template is defined for each entity type in the model that defines a primary key. This enables Semantic queries and inferencing on entity instances. For details, see Triples Sub-Template Characteristics.
• A rows sub-template is defined for each entity type in the model that defines at least one required property. This enables querying instances as rows using SQL or the Optic API. For details, see Rows Sub-Template Characteristics and Rows Template Array Property View Characteristics.
• If you define a namespace prefix for an entity type as described in Defining a Namespace URI for an Entity Type, the prefix is used in XPath expressions in the template. Namespace prefixes are not used for references to entity types external to the model because such prefixes are unknown to the template generator.

Customizing a TDE Template

The following entity type characteristics result in a TDE template that requires customization:

• If no primary key is defined for an entity type that contains an array-typed property, you will like need to customize the template to define an appropriate type and value for the left column in the array view. This view is discussed in more detail in Rows Template Array Property View Characteristics.
• The template generator cannot determine the data type of an external entity type reference, so it defaults to string. You must manually set the type in the template.
• If you choose to embed entity instances inside one another, then the context element of the embedded type must be changed to reflect its position in instance documents.

You can make other customizations required by your application. For example, you might want to generate additional facts about your instances, or remove some columns from a row sub-template.

The generated template should work for both XML and JSON envelope documents in most cases, but some entity type structures might require customization of XPath expressions in the template in order to accommodate both formats.

For more details on the structure and content of TDE templates, see Template Driven Extraction (TDE) in the Application Developer's Guide.

Deploying a TDE Template

You must install your TDE template in the schemas database associated with your content database. The template must be in the special collection http://marklogic.com/xdmp/tde for MarkLogic to recognize it as template document.

Choose one of the following template installation methods:

• Use the tde:template-insert XQuery function or the tde.templateInsert JavaScript function. This method combines validation and installation in one step, and automatically inserts the template into the required collection.
• Use any general-purpose document insertion interface, such as xdmp:document-insert (XQuery) or xdmp.documentInsert (JavaScript). You must explicitly insert the template document into the special collection http://marklogic.com/xdmp/tde. No validation is performed.

For more details, see Validating and Inserting a Template in the Application Developer's Guide.

Once your template is installed, MarkLogic will update the row index and generate triples related to your instances whenever you ingest instances or reindexing occurs.

Example: TDE Template Generation and Deployment

The following example generates a TDE template from the model used in Getting Started With Entity Services, and then installs the template in the schemas database.

The following code generates a template from a previously persisted model, and then saves the template to a file on the filesystem as $ARTIFACT_DIR/person-templ.xml.

xquery version "1.0-ml";
import module namespace es =
    "http://marklogic.com/entity-services"
    at "/MarkLogic/entity-services/entity-services.xqy";

let $ARTIFACT_DIR := '/space/es/gs/'
return xdmp:save(
  fn:concat($ARTIFACT_DIR, 'person-templ.xml'),
  es:extraction-template-generate(
    fn:doc('/es-gs/models/person-1.0.0.json')))

'use strict';
const es = require('/MarkLogic/entity-services/entity-services');

const ARTIFACT_DIR = '/space/es/gs/';
xdmp.save(
  ARTIFACT_DIR + 'person-templ.xml',
  es.extractionTemplateGenerate(
    cts.doc('/es-gs/models/person-1.0.0.json'))
);

You are not required to save the template to the filesystem. However, the template is an important project artifact that you should place under source control. Saving the template to the filesystem makes it easier to do so.

If you apply the code above to the model from Getting Started With Entity Services, the resulting template defines two sub-templates. The first sub-template defines how to extract semantic triples from Person entity instances. The second sub-template defines how to extract a row-oriented projection of Person entity instances.

<template xmlns="http://marklogic.com/xdmp/tde">
...
  <templates>
    <template xmlns:tde="http://marklogic.com/xdmp/tde">
      <context>./Person</context>
      <vars>
        <var>
          <name>subject-iri</name>
          <val>sem:iri(...)</val>
        </var>
        ...
      </vars>
      <triples>...</triples>
    </template>
    <template xmlns:tde="http://marklogic.com/xdmp/tde">
      <context>./Person</context>
      <rows>...</rows>
      ...
    </template>
  </templates>
</template>

If the model includes additional entity types, then the template contains additional, similar sub-templates for these types.

The following code validates and installs a template using the convenience function provided by the TDE library module. Evaluate this code in the context of your content database.

xquery version "1.0-ml";
import module namespace tde = "http://marklogic.com/xdmp/tde" 
  at "/MarkLogic/tde.xqy";

let $ARTIFACT_DIR := '/space/es/gs/'
return tde:template-insert(
  '/es-gs/templates/person-1.0.0.xml',
  xdmp:document-get(
    fn:concat($ARTIFACT_DIR, 'person-templ.xml'))
)

'use strict';
const tde = require('/MarkLogic/tde');

const ARTIFACT_DIR = '/space/es/gs/';
tde.templateInsert(
  '/es-gs/templates/person-1.0.0.xml',
  fn.head(xdmp.documentGet(ARTIFACT_DIR + 'person-templ.xml'))
);

If the query runs successfully, the document /es-gs/templates/person-1.0.0.xml is created in the schemas database. If you explore the schemas database in Query Console, you should see that the template is in the special collection http://marklogic.com/xdmp/tde.

Schema Generation Overview

To generate a schema, apply the es:schema-generate XQuery function or the es.schemaGenerate JavaScript function to the object-node or json:object representation of a model descriptor, as shown in the following table. For a more complete example, see Example: Generating and Installing an Instance Schema.

es:schema-generate(fn:doc('/es-gs/models/person-1.0.0.json'))

es.schemaGenerate(cts.doc('/es-gs/models/People-1.0.0.json'));

The schema is an important project artifact, so you should place it under source control.

Before you can use the generated schema(s) for instance validation, you must deploy the schema to the schemas database associated with your content database. You can use any of the usual document insertion APIs for this operation.

If your model defines multiple entity types and the entity type definitions do not all use the same namespace, a schema is generated for each unique namespace. Install all of the generated schemas in the schemas database.

Use the xdmp:validate XQuery function or the xdmp.validate JavaScript function to validate instances against your schema. For an example, see Example: Validating an Instance Against a Schema.

Note that you can only validate entity instances expressed as XML. You can extract the XML representation of an instance from an envelope document using the es:instance-xml-from-document XQuery function or the es.instanceXmlFromDocument JavaScript function.

Schema Characteristics

The Entity Services API applies the following rules when generating a schema from a model:

• A scalar property type is translated into a simple, type-enforced xs:element.
• The schema includes an xs:complexType for each entity type defined by the model. This type contains a sequence of elements representing the entity type properties.
• For each external entity type reference, a type is generated that can hold a value for a reference of that type by using the string after the last slash (...Äò/') in the external reference URI.
• For each local entity type reference, an es:complexType is generated.
• Array typed entity properties are handled using minOccurs and maxOccurs on the property's xs:element.
• Any entity property that is not a primary key or required is set to minOccurs=0.
• A required property has cardinality 1.
• The automated schema generation cannot resolve multiple properties with same name, but different data type. If this occurs, an xs:element is generated for one property, and then the xs:element definitions for the other properties will be commented out. You must customize the schema (or modify your model) to resolve this conflict.
• A separate schema is generated for each namespace declared in the model. For more details on using namespaces in entity type definitions, see Defining a Namespace URI for an Entity Type.

Schema Customization

The following list describes some situations in which schema customization might be needed.

• If your model contains multiple entity type properties with the same name, only one of them will be reflected in the schema. The other(s) will be commented out. Change the schema (or your model) to resolve this conflict.
• Depending on how entity references are used in the model, parts of the schema might be superfluous and can be removed.
• You might have to choose between validating entity references or validating embedded entity instances, depending on the choices you make with respect to normalization and entity document structure.

Example: Generating and Installing an Instance Schema

The following example generates a schema from a previously persisted model, and then inserts it into the schemas database.

Since the model is in the content database and the schema must be inserted into the schemas database, xdmp:eval is used to switch database contexts for the schema insertion. If you generated the schema and saved it to the filesystem first, then you would only have to work with the schemas database, so the eval would be unnecessary.

The following code inserts a schema with the URI /es-gs/person-1.0.0.xsd into the schemas database associated with the content database that holds the source model. Assume the model was previously persisted as a document with URI /es-gs/models/person-1.0.0.json.

xquery version "1.0-ml";
import module namespace es = "http://marklogic.com/entity-services"
  at "/MarkLogic/entity-services/entity-services.xqy";

(: The query that inserts the schema into the schemas db :)
let $query :=
 'xquery version "1.0-ml";
  declare variable $schema as element(xs:schema) external;
  declare variable $uri as xs:string external;
  xdmp:document-insert($uri, $schema)'

(: Generate the schema :)
let $schema := 
  es:schema-generate(fn:doc('/es-gs/models/person-1.0.0.json'))

(: Insert the schema into the Schemas db :)
return xdmp:eval($query, 
  (xs:QName("schema"), $schema, 
   xs:QName("uri"), '/es-gs/person-1.0.0.xsd'),
  <options xmlns="xdmp:eval">
    <database>{xdmp:schema-database()}</database>
  </options>
)

'use strict';
const es = require('/MarkLogic/entity-services/entity-services')

// The query that inserts the schema into the schemas db
const query = 'declareUpdate(); xdmp.documentInsert(uri, schema);'
// Generate the schema
const schema = fn.head(
  es.schemaGenerate(cts.doc('/es-gs/models/person-1.0.0.json')));
xdmp.eval(
  query, 
  {schema: schema, uri: '/es-gs/person-1.0.0.xsd'},   // vars
  {database: xdmp.schemaDatabase()}                   // options
);

Example: Validating an Instance Against a Schema

The following example validates an instance against a schema generated using the es:schema-generate XQuery function or the es.schemaGenerate Server-Side JavaScript function. It is assumed that the schema is already installed in the schema database associated with the content database, as shown in Example: Generating and Installing an Instance Schema.

The following code validates an entity instance within a previously persisted envelope document. Assume this instance was created using the instance converter module for its entity type, and therefore is valid. Thus, the validation succeeds. The query returns an empty xdmp:validation-errors element in XQuery and an empty object in JavaScript.

xquery version "1.0-ml";
import module namespace es =
  "http://marklogic.com/entity-services"
  at "/MarkLogic/entity-services/entity-services.xqy";

xdmp:validate(
  es:instance-xml-from-document(
    fn:doc('/es-gs/envelopes/1234.xml')),
  'type', xs:QName('PersonType'))

'use strict';
const es = require('/MarkLogic/entity-services/entity-services')

xdmp.validate(
  es.instanceXmlFromDocument(
    cts.doc('/es-gs/envelopes/1234.xml')),
  'type', 'PersonType')

The following example validates an in-memory instance against the schema. The schema is based on the model from Getting Started With Entity Services. The instance was intentionally created without a required property (id) so that it will fail validation.

xquery version "1.0-ml";
import module namespace es = "http://marklogic.com/entity-services"
  at "/MarkLogic/entity-services/entity-services.xqy";

let $invalid-entity :=
  <Person>
    <firstName>George</firstName>
    <lastName>Washington</lastName>
    <fullName>George Washington</fullName>
  </Person>
return 
xdmp:validate($invalid-entity, 'type', xs:QName('PersonType'))

'use strict';
const invalidEntity = fn.head(xdmp.unquote(
  '<Person>'+
    '<firstName>George</firstName>' +
    '<lastName>Washington</lastName>' +
    '<fullName>George Washington</fullName>' +
'</Person>'));
xdmp.validate(invalidEntity, 'type', 'PersonType');

Options Generation Overview

Generate model-based query options using the es:search-options-generate XQuery function or the es.searchOptionsGenerate JavaScript function. Pass in the JSON object-node or json:object representation of a model descriptor.

For example, if the document /es-gs/models/person-1.0.0.json is a previously persisted descriptor, then you can generate query options from the model with one of the following calls.

es:search-options-generate(
  fn:doc('/es-gs/models/person-1.0.0.json'))

es.searchOptionsGenerate(
  cts.doc('/es-gs/models/person-1.0.0.json'));

For a more complete example, see Example: Generating Query Options.

You can use the generated options in the following ways:

• Pass them as the second parameter of the search:search or search:resolve XQuery functions, or the search.search or search.resolve JavaScript functions.
• Embed them in a combined query used with the REST, Java, or Node.js APIs.
• Install them in the database and use them as persistent query options with the REST, Java, or Node.js APIs.
• Use them as a jumping off point for creating constraint bindings for use with the cts:parse XQuery function or the cts.parse JavaScript function. Then use the resulting cts:query object with cts:search or the JSearch API.

For an example and discussion of the options, see Example: Using the Search API for Instance Queries.

Characteristics of the Generated Options

The generated options include the following:

• A value constraint named entity-type for constraining searches to entities of a particular type. For example:

  <search:constraint name="entity-type">
    <search:value>
      <search:element ns="http://marklogic.com/entity-services" name="title"/>
    </search:value>
  </search:constraint>

• A URI value constraint named uris. For example:

  <search:values name="uris">
    <search:uri/>
  </search:values>

• An extract-document-data option for returning just the canonical entity instance(s) from matched documents. For example, the following option extracts just the Person entity instance from matched documents:

  <search:extract-document-data selected="include">
    <search:extract-path xmlns:es="...">
      //es:instance/(Person)
    </search:extract-path>
  </search:extract-document-data>

• An additional-query option that constrains results to documents containing es:instance elements. For example:

  <search:additional-query>
    <cts:element-query xmlns:cts="http://marklogic.com/cts">
      <cts:element xmlns:es="...">es:instance</cts:element>
      <cts:true-query/>
    </cts:element-query>
  </search:additional-query>

• Options that disable faceting and snippeting (in favor of just extracting the instances). For example:

  <search:return-facets>false</search:return-facets>
  <search:transform-results apply="empty-snippet"/>

• An option that enables unfiltered search. For example:

  <search:search-option>unfiltered</search:search-option>

• If the model defines a primary key, a value constraint on the primary key property. For example:

  <search:constraint name="id">
    <search:value>
      <search:element ns="" name="id"/>
    </search:value>
  </search:constraint>

• For each property named in the pathRangeIndex or rangeIndex property of an entity type definition, a path range index constraint with the same name as the entity property. For example:

  <search:constraint name="rating">
    <search:range type="xs:float" facet="true">
      <search:element ns="" name="rating" />
    </search:range>
  </search:constraint>

• For each property named in the elementRangeIndex property of an entity type definition, an element range index constraint with the same name as the entity property. For example:

  <search:constraint name="rating">
    <search:range type="xs:float" facet="true">
      <search:path-index xmlns:es="...">
        //es:instance/Person/rating
      </search:path-index>
    </search:range>
  </search:constraint>

• For each property named in the wordLexicon property of an entity type definition, a word constraint with the same name as the entity property. For example:

  <search:constraint name="bio">
    <search:word>
      <search:element ns="" name="bio"/>
    </search:word>
  </search:constraint>

• If an entity type includes more than one property in the range index specification, a tuples option with the same name as the entity type for finding co-occurrences of the indexed properties. For example:

  <search:tuples name="Item">
    <search:range type="xs:int" facet="true">
      <search:path-index xmlns:es="...">
        //es:instance/Item/price
      </search:path-index>
    </search:range>
    <search:range type="xs:float" facet="true">
      <search:path-index xmlns:es="...">
        //es:instance/Item/rating
      </search:path-index>
    </search:range>
  </search:tuples>

The generated options include extensive comments to assist you with customization. The options are usable as-is, but optimal search configuration is highly application dependent, so it is likely that you will extend or modify the generated options.

If the primary key property is also listed in the range index specification, then both a value constraint and a range constraint would be generated with the same name. Since this is not allowed, one of these constraints will be commented out. You can change the name and uncomment it. For an example of this conflict, see Example: Generating Query Options.

Example: Generating Query Options

The following example generates a set of query options from a model and saves the results to a file on the filesystem so you can place it under source control or make modifications.

This example assumes the following descriptor has been inserted into the database with the URI /es-ex/models/people-1.0.0.json.

{ "info": {
    "title": "People",
    "description": "People Example",
    "version": "1.0.0"
  },
  "definitions": {
    "Person": {
      "properties": {
        "id": { "datatype": "int" },
        "name": { "datatype": "string" },
        "bio": { "datatype": "string" },
        "rating": { "datatype": "float" }
      },
      "required": [ "name" ],
      "primaryKey": "id",
      "pathRangeIndex": [ "id", "rating" ],
      "wordLexicon": [ "bio" ]
}}}

The following code generates a set of query options from the above model. The options are saved to the file ARTIFACT_DIR/people-options.xml.

xquery version "1.0-ml";
import module namespace es =
    "http://marklogic.com/entity-services"
    at "/MarkLogic/entity-services/entity-services.xqy";

let $ARTIFACT_DIR := '/space/es/ex/'    (: CHANGE THIS VALUE :)
return xdmp:save(
  fn:concat($ARTIFACT_DIR, 'people-options.xml'),
  es:search-options-generate(
    fn:doc('/es-ex/models/people-1.0.0.json')))

'use strict';
const es = require('/MarkLogic/entity-services/entity-services');

const ARTIFACT_DIR = '/space/es/ex/';    // CHANGE THIS VALUE
xdmp.save(
  ARTIFACT_DIR + 'people-options.xml',
  es.searchOptionsGenerate(
    cts.doc('/es-ex/models/people-1.0.0.json'))
);

The resulting options should be similar to the following.

<search:options 
    xmlns:search="http://marklogic.com/appservices/search">
  <search:constraint name="entity-type">
    <search:value>
      <search:element ns="http://marklogic.com/entity-services" name="title"/>
    </search:value>
  </search:constraint>
  <search:constraint name="id">
    <search:value>
      <search:element ns="" name="id"/>
    </search:value>
  </search:constraint>
  <!--This item is a duplicate and is commented out so as to create 
      a valid artifact.
 <search:constraint name="id" 
    xmlns:search="http://marklogic.com/appservices/search">
  <search:range type="xs:int" facet="true">
    <search:path-index 
        xmlns:es="http://marklogic.com/entity-services">
      //es:instance/Person/id
    </search:path-index>
  </search:range>
</search:constraint>
-->
  <search:constraint name="rating">
    <search:range type="xs:float" facet="true">
      <search:path-index 
          xmlns:es="http://marklogic.com/entity-services">
        //es:instance/Person/rating
      </search:path-index>
    </search:range>
  </search:constraint>
  <search:constraint name="bio">
    <search:word>
      <search:element ns="" name="bio"/>
    </search:word>
  </search:constraint>
  <search:tuples name="Person">
    <search:range type="xs:int" facet="true">
      <search:path-index 
          xmlns:es="http://marklogic.com/entity-services">
        //es:instance/Person/id
      </search:path-index>
    </search:range>
    <search:range type="xs:float" facet="true">
      <search:path-index 
          xmlns:es="http://marklogic.com/entity-services">
        //es:instance/Person/rating
      </search:path-index>
    </search:range>
  </search:tuples>
  <!--Uncomment to return no results for a blank search, rather 
      than the default of all results
  <search:term xmlns:search="http://marklogic.com/appservices/search">
    <search:empty apply="no-results"/>
  </search:term>
-->
  <search:values name="uris">
    <search:uri/>
  </search:values>
  <!--Change to 'filtered' to exclude false-positives in certain
      searches-->
  <search:search-option>unfiltered</search:search-option>
  <!--Modify document extraction to change results returned-->
  <search:extract-document-data selected="include">
    <search:extract-path 
        xmlns:es="http://marklogic.com/entity-services">
      //es:instance/(Person)
    </search:extract-path>
  </search:extract-document-data>
  <!--Change or remove this additional-query to broaden search 
      beyond entity instance documents-->
  <search:additional-query>
    <cts:element-query xmlns:cts="http://marklogic.com/cts">
      <cts:element xmlns:es="http://marklogic.com/entity-services">
        es:instance
      </cts:element>
      <cts:true-query/>
    </cts:element-query>
  </search:additional-query>
  <!--To return facets, change this option to 'true' and edit constraints-->
  <search:return-facets>false</search:return-facets>
  <!--To return snippets, comment out or remove this option-->
  <search:transform-results apply="empty-snippet"/>
</search:options>

Notice that two constraints are generated for the id property. A value constraint is generated because id is the primary key for a Person entity. A path range constraint is generated because id is listed in the pathRangeIndex property of the Person entity type definition. Since it is not possible for two constraints to have the same name in a set of options, the second constraint is commented out:

<search:constraint name="id">
    <search:value>
      <search:element ns="" name="id"/>
    </search:value>
  </search:constraint>
  <!--This item is a duplicate and is commented out so as to create 
      a valid artifact.
 <search:constraint name="id"
    xmlns:search="http://marklogic.com/appservices/search">
  <search:range type="xs:int" facet="true">
    <search:path-index
        xmlns:es="http://marklogic.com/entity-services">
      //es:instance/Person/id
    </search:path-index>
  </search:range>
</search:constraint>

If you do not need both constraint types on id, you can remove one of them. Alternatively, you can change the name of at least one of these constraints and uncomment the path range constraint.

For an example of using the generated options, see Example: Using the Search API for Instance Queries.