Envelope Pattern

Overview

The MarkLogic Data Hub uses the envelope pattern to encapsulate data. In the envelope pattern, the original content and the associated metadata are stored in the same envelope (an entity) but remain separate. This preserves the original content, while allowing additional metadata to be added.

   {
    "envelope": {
      "headers": [],
      "triples": [],
      "instance": {
        "your original data": "goes here"
      }
    }
  }

   <envelope>
    <headers/>
    <triples/>
    <instance>
      your original data goes here
    </instance>
  </envelope>

Example: Envelope Pattern

In MarkLogic Data Hub, metadata can be added to the envelope to harmonize data with different field names and/or formats, while the original data is preserved.

Suppose you have two data sources that both contain gender information but in fields with different names and different formats.

source-1.json source-2.json 
   {
    "gender": "f",
    "age": "32",
    ...,
    "firstName": "Rebecca"
  }

   {
    "gndr": "female",
    "age": "39",
    ...,
    "firstName": "Leona"
  }

With the envelope, you can normalize the fields into a single field, which is stored as metadata in the same envelope.

harmonized-1.json harmonized-2.json 
   {
    "headers": [
      { "normalizedGender": "female" }
    ],
    "triples": [],
    "instance": {
      "gender": "f",
      "age": "32",
      ...,
      "firstName": "Rebecca"
    }
  }

   {
    "headers": [
      { "normalizedGender": "female" }
    ],
    "triples": [],
    "instance": {
      "gndr": "female",
      "age": "39",
      ...,
      "firstName": "Leona"
    }
  }

Applications can now query your data hub using the same field name (normalizedGender) and get the result in the same format (female), regardless of the data source.