Loading TOC...
Entity Services Developer's Guide (PDF)

Entity Services Developer's Guide — Chapter 5

Managing Entity Instances

This chapter describes how to create, retrieve, update, and delete entity instances derived from a model created with MarkLogic Entity Services. The chapter covers the following topics:

Entity Instance Concepts

This section introduces entity instance concepts helpful in creating, persisting, querying, and extracting entity instance data. The following topics are included:

What is an Instance?

An entity instance is a concrete instantiation of an entity type defined in a model.

For example, suppose you have a JSON model descriptor that defines a Person entity type with the following properties. This is based on the model in Getting Started With Entity Services.

"Person": {
  "properties": {
    "id": {"datatype": "string"}, 
    "firstName": {"datatype": "string"}, 
    "lastName": {"datatype": "string"}, 
    "fullName": {"datatype": "string"}, 
    "friends": {
      "datatype": "array", 
      "items": {"$ref": "#/definitions/Person"
    }
  }}, 
  ...
}

Then the canonical XML representation of a Person instance would have the following form:

<Person>
  <id>1234</id>
  <firstName>George</firstName>
  <lastName>Washington</lastName>
  <fullName>George Washington</fullName>
</Person>

By convention, instances are stored as child elements of an XML envelope document. You can extract them from an envelope in several formats. For details, see What is an Envelope Document? and Extracting an Entity Instance from an Envelope Document.

An instance can have multiple repesentations, depending on the context:

  • While you are synthesizing an instance from raw source or converting one between model versions, you work with an in-memory representation of the instance as a map:map containing not only the entity type property values, but additional information such as type and source. This representation is designed to be easy to modify during instance construction.
  • By Entity Services convention, instances are persisted in envelope documents. An envelope document includes an es:instance XML element with a child element that is the canonical XML representation of the instance. This is the representation on which queries are based. For details, see What is an Envelope Document?.
  • You can extract an instance from an envelope document as XML, JSON, or a map:map. You might use one or more of these representations to pass instances to downstream applications. For details, see Extracting an Entity Instance from an Envelope Document.

For more details, see Example: Entity Instance Representations.

What is an Envelope Document?

If you follow the Entity Services conventions, your entity instances are persisted in MarkLogic as part of an envelope document. An envelope document encapsulates instance data with related metadata that might be useful to your application. Entity Services only supports XML envelope documents at this time.

An envelope document for some entity type T is created using the instance-to-envelope function in T's instance converter module. For more details, see Creating an Entity Instance from a Data Source and Creating an Instance Converter Module.

An envelope document has the following form by default:

<es:envelope xmlns:es="http://marklogic.com/entity-services">
  <es:instance>
    <es:info>
      <es:title>model title</es:title>
      <es:version>model version</es:version>
    </es:info>
    <T>
      ...T's entity properties as elements...
    </T>
  </es:instance>
  <es:attachments>...source data...</es:attachments>
</es:envelope>

The es:instance element contains the canonical XML representation of the instance, plus metadata such as the model title and version from which entity type is derived. The es:attachments element contains the source data, by convention; you can add additional attachments.

You can customize an envelope document to include other information, but you should generally not modify the <es:instance/> portion. The instance data should accurately reflect the entity type definition in your model. If you need to normalize or derive property values, do so in the extract-instance-T function of your instance converter.

If you customize the envelope by adding data to the es:attachments element, then you can use the es:instance-get-attachments XQuery function or the es.instanceGetAttachments JavaScript function to retrieve the data. If you put it elsewhere in the envelope, then you are solely responsible for retrieving it from the envelope.

The Entity Services API includes functions for retrieving the instance data and attachments from an envelope. For details, see Extracting an Entity Instance from an Envelope Document and Extracting the Original Source from an Envelope Document.

Example: Entity Instance Representations

This example illustrates the various instance representations discussed in What is an Instance?. The example uses the Person entity type from the model defined in Getting Started With Entity Services.

RepresentationExample
1Raw Source
<person>
  <pid>1234</pid>
  <given>George</given>
  <family>Washington</family>
</person>
2

In-memory instance, as returned by extract-instance-Person

Shown here as JSON for readability, but really a json:object (map:map) with keys $attachments, $type, id, etc.

{"$attachments": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<person>\n  <pid>1234</pid>\n  <given>George</given>\n  <family>Washington</last>\n</family>", 
  "$type": "Person", 
  "id": "1234", 
  "firstName": "George", 
  "lastName": "Washington", 
  "fullName": "George Washington"
}
3

Canonical XML instance generated by instance-to-canonical-xml

Used to construct the instance within an envelope document.

<Person>
  <id>1234</id>
  <firstName>George</firstName>
  <lastName>Washington</lastName>
  <fullName>George Washington</fullName>
</Person>
4Envelope document, as generated by instance-to-envelope
<es:envelope
   xmlns:es="http://marklogic.com/entity-services">
  <es:instance>
    <es:info>
      <es:title>Person</es:title>
      <es:version>0.0.1</es:version>
    </es:info>
    <Person>
      <id>1234</id>
      <firstName>George</firstName>
      <lastName>Washington</lastName>
      <fullName>George Washington</fullName>
    </Person>
  </es:instance>
  <es:attachments>
    <person>
      <pid>1234</pid>
        <first>George</first>
        <last>Washington</last>
    </person>
  </es:attachments>
</es:envelope>
5

json:object (map:map) representation extracted from envelope document by es:instance-from-document

Shown here as JSON for readability, this is really a map:map in XQuery. In JavaScript, this function returns a JavaScript object. The value is mutable.

{ "id": "1234", 
  "firstName": "George", 
  "lastName": "Washington", 
  "fullName": "George Washington",
  "$type": "Person"
}
6

XML representation extracted from envelope document by es:instance-xml-from-document

The value is immutable.

<Person>
  <id>1234</id>
  <firstName>George</firstName>
  <lastName>Washington</lastName>
  <fullName>George Washington</fullName>
</Person>
7

JSON representation extracted from envelope document by es:instance-json-from-document

This function returns a JSON object node. The value is immutable.

{ "Person": {
    "id": "1234", 
    "firstName": "George", 
    "lastName": "Washington", 
    "fullName": "George Washington"
} }

The representations you see on lines 2, 3, and 4 were created by an instance converter module. For details, see Creating an Instance Converter Module. The representation on line 2 is a transient, mutable in-memory representation designed for ease of use in instance converter code. If you pass an envelope document to the convert-instance-T function of a version translator module, it returns a similar representation; for details, see Creating a Model Version Translator Module.

The envelope document representation on line 4 is the recommended way to store entity instances in MarkLogic. You can customize the contents of your envelope, but should usually leave the es:instance portion as-is. This is the layout produced by the instance-to-envelope function of an instance converter.

The representations on lines 5, 6, and 7 are instances extracted from an envelope document using the Entity Services API. The map:map representation on line 5 differs from the other extracted entities in that it is mutable and carries explicit type information in the $type property. This representation differs from the one on line 2 in that it contains only the instance entity type properties. There is no $attachments. For more details, see Extracting an Entity Instance from an Envelope Document.

Creating an Entity Instance from a Data Source

The Entity Services API does not dictate how you create an entity instance from source data, but the recommended process is as follows:

  • Generate, customize, and install an instance converter module, as described in Creating an Instance Converter Module.
  • Use the extract-instance-T and instance-to-envelope functions of the instance converter module to create instance envelope documents for some entity type T from source data.
  • Insert your envelope documents in the database.

By convention, instances are stored as child elements of an XML envelope document. You can extract an instance from an envelope document in several formats. For details, see Extracting an Entity Instance from an Envelope Document.

The following code illustrates one way to create envelope documents from raw source. In this example, the source data comes from documents in MarkLogic that are in a collection named 'raw', and instances are generated for an entity type named Person. This example uses the converter and data from Getting Started With Entity Services.

LanguageExample
XQuery
(: Create envelope documents from raw source documents :)
xquery version "1.0-ml";
import module namespace es = "http://marklogic.com/entity-services"
    at "/MarkLogic/entity-services/entity-services.xqy";
import module namespace person =
    "http://example.org/example-person/Person-0.0.1"
    at "/es-gs/person-0.0.1-conv.xqy";

for $source in fn:collection('raw') return
  let $instance := person:extract-instance-Person($source)
  let $uri := 
    fn:concat('/es-gs/env/', map:get($instance, 'id'), '.xml')
  return xdmp:document-insert(
    $uri,
    person:instance-to-envelope($instance),
    <options xmlns="xdmp:document-insert">
      <collections>
        <collection>person-envelopes</collection>
      </collections>
    </options>
  )
JavaScript
'use strict';
declareUpdate();
const es = require('/MarkLogic/entity-services/entity-services.xqy');
const person = require('/es-gs/person-0.0.1-conv.xqy');

for (const source of fn.collection('raw')) {
  let instance = person.extractInstancePerson(source);
  let uri = '/es-gs/env/' + instance.id + '.xml';
  xdmp.documentInsert(
    uri, person.instanceToEnvelope(instance),
    {collections: ['person-envelopes']}
  );
}

The resulting envelope documents have the following form by default. The instance data is available as //es:instance. The original source from which the instance was derived is in //es:attachments.

<es:envelope xmlns:es="http://marklogic.com/entity-services">
  <es:instance>
    <es:info>
      <es:title>Person</es:title>
      <es:version>0.0.1</es:version>
    </es:info>
    <Person>
      <id>1234</id>
      <firstName>George</firstName>
      <lastName>Washington</lastName>
      <fullName>George Washington</fullName>
    </Person>
  </es:instance>
  <es:attachments>
    <person>
      <pid>1234</pid>
      <given>George</given>
      <family>Washington</family>
    </person>
  </es:attachments>
</es:envelope>

For an end-to-end example of creating envelope documents using this model, see Getting Started With Entity Services.

Generating Test Entity Instances

You can generate test instances from a model using the es:model-get-test-instances XQuery function or es.modelGetTestInstances Server-Side JavaScript function. You can use test instances for tasks such as experimenting with model refinement and testing code that manipulates instances.

The test instances are based purely on the model and do not reflect data normalization or customization you add to your instance converter. The test instances can help you identify properties for which converter customization is required.

The es:model-get-test-instances and es.modelGetTestInstances functions return a sequence of instances, one for each entity type defined in the input model.

If an entity type property definition contains a local reference, the referenced entity type is assumed to be embedded in the referencing entity. If an entity type property definition contains an external reference, no meaningful test value can be generated.

For example, assume the following model defining two entity types, Name and Person. A Person contains a local reference to a Name.

{ "info": {
    "title": "Example",
    "version": "1.0.0",
    "description": "ES Examples"
  },
  "definitions": {
    "Name": {
      "properties": {
        "first": { "datatype": "string" },
        "last": { "datatype": "string" }
      }
    },
    "Person": {
      "properties": {
        "id": { "datatype": "int" },
        "name": { "$ref": "#/definitions/Name" },
      }
} } }

If you generate test instances from this model, the name property of the Person test instance contains a Name instance value:

<Person>
  <id>123</id>
  <name>
    <Name>
      <first>some string</first>
      <last>some string</last>
    </Name>
  </name>
</Person>

If the name property of a Person entity was an external reference to such as 'http://example.com/SomeType' instead, then no meaningful test value can be generated. The Person test instance would look like the following:

<Person>
  <id>123</id>
  <name><SomeType>externally-referenced-instance</SomeType></name>
</Person>

To generate instances from real source data, use an instance converter. For more details, see Creating an Instance Converter Module and Creating an Entity Instance from a Data Source.

Extracting an Entity Instance from an Envelope Document

Though Entity Services encourages storing your instances in MarkLogic in the form of envelope documents, downstream consumers of your data, such as client applications, will probably expect to receive the canonical instance data, not the entire envelope.

The Entity Services API includes the following XQuery functions for extracting an instance from an envelope document. The corresponding JavaScript functions follow.

XQuery FunctionExtracted Instance Format
es:instance-from-document
map:map (json:object, mutable)
es:instance-json-from-document
object-node() (immutable)
es:instance-xml-from-document
element() (immutable)

The Entity Services API includes the following Server-Side JavaScript functions for extracting an instance from an envelope document.

JavaScript FunctionExtracted Instance Format
es.instanceFromDocument
JavaScript object (mutable)
es.instanceJsonFromDocument
object-node() (immuntable)
es.instanceXmlFromDocument
element() (immuntable)

For example, suppose you have the following envelope document in the database with the URI /es-gs/env/1234.xml:

<es:envelope xmlns:es="http://marklogic.com/entity-services">
  <es:instance>
    <es:info>
      <es:title>Person</es:title>
      <es:version>0.0.1</es:version>
    </es:info>
    <Person>
      <id>1234</id>
      <firstName>George</firstName>
      <lastName>Washington</lastName>
      <fullName>George Washington</fullName>
    </Person>
  </es:instance>
  <es:attachments>
    <person>
      <pid>1234</pid>
      <given>George</given>
      <family>Washington</family>
    </person>
  </es:attachments>
</es:envelope>

Then, the following code snippet extracts an instance from the envelope document as a json:object in XQuery or a JavaScript object in JavaScript.

LanguageExample
XQuery
xquery version "1.0-ml";
import module namespace es = "http://marklogic.com/entity-services"
  at "/MarkLogic/entity-services/entity-services.xqy";

es:instance-from-document(fn:doc('/es-gs/env/1234.xml'))[1]
JavaScript
'use strict';
const es = require('/MarkLogic/entity-services/entity-services.xqy');

fn.head(
  es.instanceFromDocument(cts.doc('/es-gs/env/1234.xml'))
);

The result is a sequence containing one item, equivalent to the following JSON:

{ "id":"1234", 
  "firstName":"George", 
  "lastName":"Washington", 
  "fullName":"George Washington",
  "$type": "Person"
}

The following table illustrates the result of calling each of the instance envelope extraction functions.

FunctionResult
es:instance-from-document

es.instanceFromDocument
A json:object (XQuery) or JavaScript object (JavaScript) equivalent to the following:
{ "id":"1234", 
  "firstName":"George", 
  "lastName":"Washington", 
  "fullName":"George Washington",
  "$type":"Person"
}
es:instance-json-from-document

es.instanceJsonFromDocument
A JSON object-node() equivalent to the following:
{ "Person": {
    "id":"1234", 
    "firstName":"George", 
    "lastName":"Washington", 
    "fullName":"George Washington"
}
es:instance-xml-from-document

es.instanceXmlFromDocument
The following XML element:
<Person xmlns:es=...>
  <id>1234</id>
  <firstName>George</firstName>
  <lastName>Washington</lastName>
  <fullName>George Washington</fullName>
</Person>

For more detailed coverage of instance representations, see What is an Instance? and Example: Entity Instance Representations.

Extracting the Original Source from an Envelope Document

If you follow the Entity Services conventions, an envelope document encapsulates both the canonical instance data and the raw source from which it was derived. This encapsulation happens when you call the instance-to-envelope XQuery function in a model's generated instance converter module.

You can extract the attachments from an envelope document using the es:instance-get-attachments XQuery function or the es.instanceGetAttachments JavaScript function. You can use these function on a customized envelope, as long as the attacments are locatable via the XPath expression //es:attachments.

The raw source data is saved in the envelope as an attachment. For example, the highlighted <person/> element below is the raw XML source from which the enveloped instance was derived.

<es:envelope xmlns:es="http://marklogic.com/entity-services">
  <es:instance>...</es:instance>
  <es:attachments>
    <person>
      <pid>1234</pid>
      <given>George</given>
      <family>Washington</family>
    </person>
  </es:attachments>
</es:envelope>

If the source data is JSON, then the raw source is stored as text corresponding to the serialized JSON. For example, the following envelope document is derived from raw JSON source.

<es:envelope xmlns:es="http://marklogic.com/entity-services">
  <es:instance>...</es:instance>
  <es:attachments>{"pid":2345, "given":"Martha", "family":"Washington"}</es:attachments>
</es:envelope>

The following code extracts the raw source attachment from an envelope document, assuming it is the only attachment.

LanguageExample
XQuery
xquery version "1.0-ml";
import module namespace es = "http://marklogic.com/entity-services"
  at "/MarkLogic/entity-services/entity-services.xqy";

es:instance-get-attachments(fn:doc('/es-gs/env/1234.xml'))[1]
JavaScript
'use strict';
const es = require('/MarkLogic/entity-services/entity-services.xqy');

fn.head(
  es.instanceGetAttachments(cts.doc('/es-gs/env/2345.xml'))
);

If there are multiple children in the //es:attachments element, you are responsible for picking out the raw source from the other attachments. There will only be multiple attachments if you explicitly add extra attachments.

If the original source attachment was JSON data, you must convert the serialized JSON into a json:object or object-node() if you want to work with it as JSON. For example, the following XQuery code converts the serialized JSON into a json:object and then uses the map:map interfaces to retrieve the value of the pid property:

xquery version "1.0-ml";
import module namespace es = "http://marklogic.com/entity-services"
  at "/MarkLogic/entity-services/entity-services.xqy";

map:get(
  xdmp:from-json-string(
    es:instance-get-attachments(fn:doc('/es-gs/env/2345.xml'))[1]
  )[1], "pid"
)

The same deserialization is necessary in JavaScript. You can treat the output of xdmp.fromJsonString as a JavaScript object. For example, the following code extracts the serialized JSON, converts it to a JSON object, and then retrieves the value of the 'pid' property

'use strict';
const es = require('/MarkLogic/entity-services/entity-services.xqy');

fn.head(xdmp.fromJsonString(
  fn.head(es.instanceGetAttachments(cts.doc('/es-gs/env/2345.xml')))
)).pid;

Updating Entity Instance Data When Your Model Changes

As your model changes, you might need to update your instance data to match. Model changes can also impact generated and configuration artifacts. For details, see Managing Model Changes.

« Previous chapter
Next chapter »
Powered by MarkLogic Server 7.0-4.1 and rundmc | Terms of Use | Privacy Policy