Loading TOC...
Node.js Application Developer's Guide (PDF)

Node.js Application Developer's Guide — Chapter 6

Working With Semantic Data

This chapter discusses the following topics related to using the Node.js Client API to load semantic triples, manage semantics graphs, and query semantic data:

This chapter only covers details specific to using the Node.js Client API for semantic operations. For more details, see the Semantics Developer's Guide.

Overview of Common Semantics Tasks

The following table lists some common tasks related to Semantics, along with the method best suited for the completing the task. For a complete list of interfaces, see the Node.js API Reference.

If you want to Then use
Load semantic triples into a named graph or the default graph without using SPARQL Update.
DatabaseClient.graphs.write
For details, see Loading Triples.
Manage graphs or graph data with SPARQL Update.
DatabaseClient.graphs.sparlUpdate
For details, see Using SPARQL Update to Manage Graphs and Graph Data.
Read a semantic graph from the database.
DatabaseClient.graphs.read
For details, see Retrieving the Contents, Metadata, or Permissions of a Graph.
Query semantic data with SPARQL
DatabaseClient.graphs.sparql
For details, see Querying Semantic Triples With SPARQL

To read a graph or perform a semantic query at fixed point in time, pass a Timestamp object as a parameter to your call. For more details, see Performing Point-in-Time Operations.

Loading Triples

This topic covers using DatabaseClient.graphs.write or DatabaseClient.graphs.writeStreamto load semantic triples into the database in several formats. For a list of supported formats, see Supported RDF Triple Formats in Semantics Developer's Guide. You can also insert triples with a SPARQL Update request; for details, see Using SPARQL Update to Manage Graphs and Graph Data.

The collection lexicon must be enabled on your database when using the semantics REST services or use the GRAPH '?g' construct in a SPARQL query.

Use DatabaseClient.graphs.write to upload a block of triples to MarkLogic Server in a single request. Use DatabaseClient.graphs.writeStream to incrementally stream a large number of triples to MarkLogic Server; for details, see Streaming Into the Database. The input semantic data can be expressed as a string, object, or Readable stream.

A call to DatabaseClient.graphs.write or DatabaseClient.graphs.writeStream always includes at least a MIME type parameter indicating the format of the input triples. If you also include a graph URI, the triples are loaded into that graph. If you do not include a graph URI, the triples are loaded into the default graph. That is, you can use one of these two forms, depending on the destination graph:

// load into the default graph
db.graphs.write(mimeType, triples)
// load into a named graph
db.graphs.write(graphURI, mimeType, triples)

Optionally, you can pass a boolean repair flag. If present and set to true, MarkLogic Server attempts to repair invalid input triples during ingestion. For example:

db.graphs.write(graphURI, true, mimeType, triples)

You can also call the write functions with a call object instead of with positional parameters. The call object has the following properties:

db.graphs.write({
  uri: graphURI,             // optional, omit for default graph
  contentType: mimeType,    // required
  data: triples,            // required
  repair: boolean           // optional
})

The output from a call to DatabaseClient.graphs.write or DatabaseClient.graphs.writeStream is an object that indicates whether the triples were loaded into the default graph or a named graph. If the destination is a named graph, the graph name is returned in the graph property. For example:

// result of loading into default graph
{ defaultGraph: true, graph: null }
// result of loading into a named graph
{ defaultGraph: false, graph: 'example-graph-uri' }

The following example uses DatabaseClient.graphs.write to load a set of triples in RDF/JSON format into the default graph.

const fs = require('fs');
const marklogic = require('marklogic');
const my = require('./my-connection.js');

const db = marklogic.createDatabaseClient(my.connInfo);

const triples = {
  'http://dbpedia.org/resource/Joyce_Carol_Oates' : {
    'http://dbpedia.org/property/influences' : [ {
      'type' : 'uri', 
      'value' : 'http://dbpedia.org/resource/Ernest_Hemingway' 
    } ] ,
    'http://dbpedia.org/ontology/influencedBy' : [ {
      'type' : 'uri', 
      'value' : 'http://dbpedia.org/resource/Ernest_Hemingway' 
    } ]
  },
  'http://dbpedia.org/resource/Death_in_the_Afternoon' : {
    'http://dbpedia.org/ontology/author' : [ {
      'type' : 'uri', 
      'value' : 'http://dbpedia.org/resource/Ernest_Hemingway' 
    } ] ,
    'http://dbpedia.org/property/author' : [ {
      'type' : 'uri', 
      'value' : 'http://dbpedia.org/resource/Ernest_Hemingway' 
    } ]
  }
};

db.graphs.write('application/rdf+json', triples).result(
    function(response) { 
      if (response.defaultGraph) {
        console.log('Loaded into default graph'); 
      } else {
        console.log('Loaded into graph ' + response.graph); 
      };
    },
    function(error) { console.log(JSON.stringify(error)); }
);

The following example uses DatabaseClient.graphs.writeStream to load a set of triples in N-Quads format into a named graph (example-graph). The triples are streamed from the file input.nq using a Readable stream and piped into the Writable stream returned by DatabaseClient.graphs.createWriteStream. The transaction is not committed until the all the data from the input stream is transmitted.

const fs = require('fs');
const marklogic = require('marklogic');
const my = require('./my-connection.js');

const db = marklogic.createDatabaseClient(my.connInfo);
// Load into a named graph using a write stream
const writer = 
  db.graphs.createWriteStream('example-graph', 'application/n-quads');
writer.result(  
  function(response) { 
    if (response.defaultGraph) {
      console.log('Loaded triples into default graph'); 
    } else {
      console.log('Loaded triples into graph ' + response.graph); 
    };
  },
  function(error) { console.log(JSON.stringify(error)); }
);
fs.createReadStream('input.nq').pipe(writer);

To read a semantic graph, use DatabaseClient.graphs.read; for details, see Retrieving the Contents, Metadata, or Permissions of a Graph. To query semantic data, use DatabaseClient.graphs.sparql; for details, see Querying Semantic Triples With SPARQL. For additional operations, see the Node.js API Reference.

Querying Semantic Triples With SPARQL

Use the DatabaseClient.graphs.sparql method to evaluate SPARQL queries against triples in the database. For more details on using SPARQL queries with MarkLogic, see the Semantics Developer's Guide.

The collection lexicon must be enabled on your database when using the semantics REST services or use the GRAPH '?g' construct in a SPARQL query.

You can evaluate a SPARQL query by calling DatabaseClient.graphs.sparql. The query is not sent to MarkLogic for evaluation until you call result(). You can invoke the sparql method in the following ways:

// (1) db.graphs.sparql(responseContentType, query)
db.graphs.sparql(
  'application/sparql-results+json',
  'SELECT ?s ?p WHERE {?s ?p Paris }')

// (2) db.graphs.sparql(responseContentType, defaultGraphUris, query)
db.graphs.sparql(
  'application/sparql-results+json',
  'http://def/graph1', 'http://def/graph2',
  'SELECT ?s ?p WHERE {?s ?p Paris }')

// (3) db.graphs.sparql(callObject)
db.graphs.sparql({
  contentType: 'application/sparql-results+json', 
  query: 'SELECT ?s ?p WHERE {?s ?p Paris }',
  begin: 0,
  end: 15
})

You must include at least a SPARQL query string and a response content type (MIME type) in your call. For a complete list of the accepted response MIME types, see SPARQL Query Types and Output Formats in the Semantics Developer's Guide.

Passing in a call object enables you to configure attributes of your query such as named graph URIs, an additional document query, result subset controls, and inference rulesets. For a complete list of the configuration properties, see graphs.sparql in the Node.js Client API Reference.

Advanced users can supply a CombinedQueryDefinition as the docQuery parameter of db.graphs.sparql. A combined query enables advanced users finer control over the optional document constraint query you can include in your query. For details, see Searching with Combined Query.

Example: SPARQL Query

The following example evaluates a SPARQL query expressed as a string literal against the default graph.

const marklogic = require('marklogic');
const my = require('./my-connection.js');

const db = marklogic.createDatabaseClient(my.connInfo);

const query = [
  'PREFIX foaf: <http://xmlns.com/foaf/0.1/>' ,
  'PREFIX ppl:  <http://people.org/>' ,
  'SELECT ?personName1' ,
  'WHERE {' ,
  '    ?personUri1 foaf:name  ?personName1 ;' ,
  '                foaf:knows ppl:person3 .' ,
  '    ?personUri1 foaf:name  ?personName1 .' ,
  '    }'
];
db.graphs.sparql('application/sparql-results+json', query.join('\n')
).result(function (result) {
  console.log(JSON.stringify(result, null, 2));
}, function(error) {
  console.log(JSON.stringify(error, null, 2));
});

Running the script produces output similar to the following, given triples that indicate Person 1 and Person 2 know Person 3.

{ "head": {
    "vars": [ "personName1" ]
  },
  "results": {
    "bindings": [
      {
        "personName1": {
          "type": "literal",
          "value": "Person 1",
          "datatype": "http://www.w3.org/2001/XMLSchema#string"
        }
      },
      {
        "personName1": {
          "type": "literal",
          "value": "Person 2",
          "datatype": "http://www.w3.org/2001/XMLSchema#string"
        }
      }
    ]
}}

For more examples, see test-basic/graphs.js in the node-client-api project source directory.

Managing Graphs

This section covers the following graph management operations:

You can also use SPARQL Update to perform similar operations; for details, see Using SPARQL Update to Manage Graphs and Graph Data.

Many graph management operations only apply to managed triples.

Creating or Replacing a Graph

You can use DatabaseClient.graphs.write to create or replace a graph without using SPARQL Update. For details and examples, see Loading Triples. You can also use DatabaseClient.graphs.sparqlUpdate to create or modify a graph; for details, see Using SPARQL Update to Manage Graphs and Graph Data.

If a graph does not exist when you write to it, the graph is created. If the graph already exists when you write to it, the unmanaged triples in the graph are replaced with the new triples; this is equivalent to removing and recreating the graph.

Any unmanaged triples in the graph are unaffected by this operation.

For more details, see the Node.js Client API Reference.

Adding Triples to an Existing Graph

Use DatabaseClient.graphs.merge to add triples to an existing graph without replacing the current contents. For other update operations or finer control, use the DatabaseClient.graphs.sparqlUpdate to interact with a graph using SPARQL Update; for more details, see Using SPARQL Update to Manage Graphs and Graph Data. To replace the contents of a graph, use DatabaseClient.graphs.write; for details, see Creating or Replacing a Graph.

You can invoke graphs.merge using the following forms:

// Add triples to a named graph
db.graphs.merge(graphUri, contentType, tripleData)

// Add triples to the default graph
db.graphs.merge(null, contentType, tripleData)

// Add triples to a named graph or the default graph using a call object
db.graphs.merge({uri: ..., contentType: ..., data: ..., ...})

Use the call object pattern to specify operation parameters such as permissions and a transaction id; for details, see the Node.js Client API Reference. When you use the call object pattern, you can omit the uri property or set it to null to merge triples into the default graph.

The following example uses the call object pattern to insert two new triples into the graph named MyGraph, repairing the data as needed. In this example, the triples are passed as a string in Turtle format. You can use other triple formats. You can also supply triple data as a stream, buffer, or object instead of a string.

const marklogic = require('marklogic');
const my = require('./my-connection.js');

const db = marklogic.createDatabaseClient(my.connInfo);

const triples = [
  '@prefix p1: <http://example.org/marklogic/predicate/> .',
  '@prefix p0: <http://example.org/marklogic/people/> .',
  'p0:Julie_Smith  p1:livesIn \"Sterling\" .',
  'p0:Jim_Smith    p1:livesIn \"Bath\" .'
];
db.graphs.merge({
    uri: 'MyGraph',
    contentType: 'text/turtle',
    data: triples.join('\n'),
    repair: true
}).result(
    function(response) {
      console.log(JSON.stringify(response));
    },
    function(error) { console.log(JSON.stringify(error)); }
);

If the operation is successful, the script produces output similar to the following:

{"defaultGraph":false,"graph":"MyGraph","graphType":"named"}

For more details, see the Node.js Client API Reference.

Removing a Graph

Use DatabaseClient.graphs.remove to remove the triples in a named graph or the default graph. This operation only affects managed triples. If the graph includes unmanaged triples, the embedded triples are unaffected and the graph will continue to exist after this operation.

You can also use a SPARQL Update request to remove a graph; for details, see Using SPARQL Update to Manage Graphs and Graph Data.

If you call remove with no parameters, it removes the default graph. If you call remove with a graph URI, it removes that named graph.

The following example removes all triples in the graph with URI example-graph:

const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.graphs.remove('example-graph').result(
  function(response) { console.log(JSON.stringify(response)); }
);
// expected output:
// {"defaultGraph":false,"graph":"example-graph"}

To remove the default graph, omit the graph URI from your call. The following example removes all triples in the default graph. The value of the graph property in the response is null because this is the default graph. For a named graph, the graph property contains the graph URI.

const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.graphs.remove().result(
  function(response) { console.log(JSON.stringify(response)); }
);
// expected output: 
// {"defaultGraph":true,"graph":null}

For more details, see the Node.js Client API Reference.

Retrieving the Contents, Metadata, or Permissions of a Graph

Use DatabaseClient.graphs.read to retrieve all the triples in a graph, the graph metadata, or graph permissions. You can use this method on the default graph or a named graph.

You can invoke graphs.read in the following forms:

// (1) Retrieve all triples in the default graph
db.graphs.read(responseContentType)

// (2) Retrieve all triples in a named graph
db.graphs.read(uri, responseContentType)

// (3) Retrieve triples, metadata, or permissions using a call object
db.graphs.read({contentType: ..., ...})

When you use the call object pattern, the call object must include at least a contentType property. To operate on a named graph, you must also include a uri property. If you omit the uri property or set it to null, the operation applies to the default graph.

You can retrieve triples in several RDF formats; for a list of supported formats, see the API reference for graphs.read.

The following example reads all triples from the graph named MyGraph. The triples are returned in Turtle format.

const marklogic = require('marklogic');
const my = require('./my-connection.js');

const db = marklogic.createDatabaseClient(my.connInfo);

db.graphs.read('MyGraph','text/turtle')
  .result(
    function(response) {
      for (const line of response.split('\n')) {
        console.log(line);
      }
    },
    function(error) { console.log(JSON.stringify(error)); }
  );

The call object pattern is required when retrieving graph metadata or permissions; a call object can also be used to retrieve triples. Use the category property of the call object to specify what to retrieve (content, metadata, or permissions).

The following example retrieves metadata about the graph named MyGraph:

db.graphs.read({
  uri: 'MyGraph',
  contentType: 'application/json',
  category: 'metadata'
});

For more details, see the Node.js Client API Reference.

Testing for Graph Existence

Use DatabaseClient.graphs.probe to test for the existence of a graph. The following example tests for the existence of a graph with the URI http://marklogic.com/example/graph.

db.graphs.probe('http://marklogic.com/example/graph')
  .result(
    function(response) {
      console.log(JSON.stringify(response));
    },
    function(error) { console.log(JSON.stringify(error)); }
  );

If the graph exists, the call produces output similar to the following:

{ "contentType": null,
  "contentLength": null,
  "versionId": null,
  "location": null,
  "systemTime": null,
  "exists": true,
  "defaultGraph": false,
  "graph": "http://marklogic.com/example/graph",
  "graphType": "named"
}

If the graph does not exist, the graph produces output similar to the following:

{ "exists": false,
  "defaultGraph": false,
  "graph": "NoMyGraph",
  "graphType": "named"
}

Probe for the existence of the default graph by omitting the graph URI.

Retrieving a List of Graphs

Use DatabaseClient.graphs.list to retrieve a list of graphs stored in MarkLogic. You must specify an expected content MIME type for the response. You can retrieve the list as text/plain or text/html.

The following example retrieves a list of available graph URI, one URI per line.

const marklogic = require('marklogic');
const my = require('./my-connection.js');

const db = marklogic.createDatabaseClient(my.connInfo);

db.graphs.list('text/uri-list')
  .result(
    function(response) {
      for (const uri of response.split('\n')) {
        console.log(uri);
      }
    },
    function(error) { console.log(JSON.stringify(error)); }
);

If the database includes the default graph and a graph with the URI MyGraph, then the raw response from the above operation is a string of the following form. Each URI is separated by a newline ('\n').

"MyGraph\nhttp://marklogic.com/semantics#graphs\n"

For more details, see the Node.js Client API Reference.

Using SPARQL Update to Manage Graphs and Graph Data

You can use a SPARQL Update request to manage graphs and graph data from Node.js by calling DatabaseClient.graphs.sparqlUpdate. You can only use this interface to work with managed triples. You can also manage graphs and graph data without using SPARQL Update; for details, see Managing Graphs and Loading Triples.

You can call graphs.sparqlUpdate in the following ways:

// (1) pass only the SPARQL Update as a string or ReadableStream
db.graphs.sparqlUpdate(updateOperation)

// (2) pass a call config object that includes the SPARQL Update
db.graphs.sparqlUpdate({data: updateOperation, ...})

The call object pattern enables you to control operations details, such as permissions, inferencing rulesets, transaction control, and variable bindings. When you use a call object, the data property holds your SPARQL Update request string or stream.

The following example creates a graph, passing in only a SPARQL Update request.

const marklogic = require('marklogic');
const my = require('./my-connection.js');

const db = marklogic.createDatabaseClient(my.connInfo);

db.graphs.sparqlUpdate(
    'CREATE GRAPH <http://marklogic.com/semantics/tutorial/update> ;'
).result(
    function(response) { 
      console.log('Graph created.');
      console.log(JSON.stringify(response));
    },
    function(error) { console.log(JSON.stringify(error)); }
);

Running this script produces the following output:

Graph created.
{"defaultGraph":false,"graph":null,"graphType":"inline"}

The following example uses a call object to create a graph and insert triples. The call object defines permissions on the graph, and defines variable bindings used in the SPARQL Update request.

const graphURI = 'http://marklogic.com/sparqlupd/example';
db.graphs.sparqlUpdate({
    data: 
      'CREATE GRAPH <' + graphURI + '> ;' +
      'INSERT {GRAPH <' + graphURI + '> {?s ?p ?b1 }} '+
      'WHERE  {GRAPH <' + graphURI + '> ' +
          '{?s ?p ?o. filter (?p = ?b2) }}',
    bindings: { 
      b1: 'bindval1', 
      b2: {value: 'bindval2', type: 'string'}
    },
    permissions: [
      { 'role-name': 'app-user', capabilities: ['read']},
      { 'role-name': 'admin', capabilities: ['read','update']},
    ]
})

For a complete list of configuration properties usable in the call object, see graphs.sparqlUpdate in the Node.js Client API Reference.

For more details on using SPARQL Update with MarkLogic, see SPARQL Update in the Semantics Developer's Guide.

Applying Inferencing Rules to a SPARQL Query or Update

You can specify one or more inferencing rulesets to apply to a SPARQL query or SPARQL Update request. Inference rules enable you to discover new facts about your data at query or update time.

SPARQL inference is discussed in detail in Inference in the Semantics Developer's Guide. This section only covers usage details specific to the Node.js Client API.

The following topics are covered:

Basic Inference Ruleset Usage

To include one or more inference rulesets when using graphs.sparql or graphs.sparqlUpdate, pass a call configuration object that includes a rulesets property.

For example, to specify one or more rulesets with graphs.sparql, construct an input call object that includes at least the following properties.

db.graphs.sparql({'content-type': ..., query: ..., rulesets: ...})

To specify one or more rulesets with graphs.sparqlUpdate, construct an input call object that includes at least the following properties:

db.graphs.sparqlUpdate({data: ..., rulesets: ...})

The value of the rulesets property can be a single string or an array of strings. Each string in rulesets must be either the name of a built-in ruleset file or the URI of a custom ruleset installed in the Schemas database.

For more details on built-in rulesets, see Rulesets in the Semantics Developer's Guide.

For details on creating custom rulesets, see Creating a New Ruleset in the Semantics Developer's Guide. You can install custom rulesets in the Schemas database using standard document operations, such as DatabaseClient.documents.write.

To learn more about semantic inferencing with MarkLogic, see Inference in the Semantics Developer's Guide.

Example: SPARQL Query With Inference Ruleset

The following example applies the built-in subPropertyOf.rules ruleset to a query. This ruleset is automatically installed in the MARKLOGIC_INSTALL_DIR/Config directory when you install MarkLogic.

db.graphs.sparql({
  contentType: 'application/sparql-results+json', 
  query: 'SELECT ?s ?p WHERE {?s ?p Paris }',
  rulesets: 'subPropertyOf.rules'
})

Example: SPARQL Update With Inference Rulesets

The following example applies the built-in sameAs.rules ruleset and a custom ruleset installed in the Schemas database with the URI /my/rules/custom.rules to a SPARQL Update request:

const update = [
  PREFIX exp: <http://example.org/marklogic/people> 
  PREFIX pre: <http://example.org/marklogic/predicate>
  INSERT DATA {
    GRAPH <MyGraph>{
      exp:John_Smith pre:livesIn "London" .
      exp:Jane_Smith pre:livesIn "London" .
      exp:Jack_Smith pre:livesIn "Glasgow" .
  }}
];
db.graphs.sparqlUpdate({
  data: update.join('\n'),
  rulesets: ['sameAs.rules', '/my/rules/custom.rules']
})

Controlling the Default Database Ruleset

Every database has an implicit, default inferencing ruleset. You can customize the default ruleset for a database, as described in Using the Admin UI to Specify a Default Ruleset for a Database in the Semantics Developer's Guide.

The database default ruleset is normally applied to all SPARQL query and update operations. However, you can control whether or not to include the default ruleset for the database in a query or update operation by using the defaultRulesets property of the input call object.

Set defaultRulesets to exclude to exclude the database default ruleset from inferencing. Set defaultRulesets to include to include the database default ruleset. If you do not explicitly set defaultRulesets, the database default ruleset is included in the operation.

The following example excludes the default ruleset from a query operation:

db.graphs.sparql({
  contentType: 'application/sparql-results+json', 
  query: 'SELECT ?s ?p WHERE {?s ?p Paris }',
  rulesets: 'subPropertyOf.rules',
  defaultRuleSets: 'exclude'
})

« Previous chapter
Next chapter »