Application Developer's Guide — Chapter 20

Overview of Machine Learning

The MarkLogic approach to machine learning is to accelerate and improve the data curation life cycle by developing models using high quality data. Bad inputs result in bad outputs (garbage in = garbage out). In the case of machine learning, the model used to convert input to output is written by the machine itself during training, and that is based on the training input. Bad training data can damage the model in ways you cannot understand, rendering it useless. Because the models are opaque, you may not even know they are damaged. You don't use machine learning to solve easy problems and hard questions answered wrong are hard to identify. MarkLogic has many features, such as the Data Hub Framework and Entity Services, you can leverage to ensure the quality of the data used to create your models.

Machine learning can be conveniently perceived as a function approximator. There is an indescribable law that determines if a picture is a picture of a cat, or if the price of a stock will go up tomorrow, and machine learning can approximate that law (with various degrees of accuracy). The law itself is a black box that takes input and produces output. For image classification, the input is pixel values and the output is cat or not; for a stock price, the input is stock trades and the output is price. A machine learning model takes input in a form understandable by the machine (high dimensional matrix of numbers, called tensors), performs a series of computation on the input, and then produces an output. The machine learns from comparing its output to the ground truth (the output of that law), and adjust its computations of the input, to produce better output that is closer to the ground truth.

Consider again the example of image classification. A simple machine learning model can be like this: convert the image into a matrix of pixel values x; multiply it with another matrix W. If the result Wx is larger than a Threshold, it's a cat, otherwise it's not. For the model to succeed, it needs labeled training data of images. The model starts with a totally random matrix W, and produces output on all training images. It will make lots of mistakes, and for every mistake it makes, it adjusts W so that the output Wx is closer to the ground truth label. The precise amount of adjustment of W is determined through a process called error back propagation. In the example described here, the computation is a simple one matrix multiplication; however, in real world applications, you can have hundreds of layers of computations, with millions of different W parameters.

Terms

The material in this guide assumes you are familiar with the basic concepts of machine learning. Some terms have ambiguous popular definitions, so they are described below.

               True Positive + True Negatives
__________________________________________________________________

True Positive + True Negatives + False Positives + False Negatives

X(t+1) = b0 + b1*X(t-1) + b2*X(t-2)

Types of Machine Learning

This section describes the types of machine learning:

• Supervised Learning
• Unsupervised Learning
• Reinforcement Learning

Why Using ONNX Runtime in MarkLogic Makes Sense

As a MarkLogic developer, there are many advantages to using ONNX for creating Machine Learning applications. For instance:

1. Different development teams throughout your enterprise may each use any Machine Learning stack of their choice to create their models. They may then export these models the ONNX format and use them all within a MarkLogic application.
2. In some cases, they can use their models as they are, because ONNX currently has native support for PyTorch, CNTK, MXNet, and Caffe2.
3. There are also converters available for TensorFlow and CoreML.
4. By using ONNX on MarkLogic, your Machine Learning applications are safe from vendor lock-in.

Capabilities of the ONNX Runtime

ONNX stands for Open Neural Network eXchange. As per its official website:

ONNX is an open format to represent deep learning models. With ONNX, AI developers can more easily move models between state-of-the-art tools and choose the combination that is best for them. ONNX is developed and supported by a community of partners.

It is an open source project with an MIT license, with its development led by Microsoft.

A machine learning model can be represented as a computation network, where nodes in the network represent mathematical operations (operators). There are many different machine learning frameworks out there (tensorflow, PyTorch, MXNet, CNTK, etc), all of which have their own representation of a computation network. You cannot simply load a model trained by PyTorch into tensorflow to perform inference. This creates barriers in collaboration. ONNX is designed to solve this problem. Although different frameworks have different representation of a model, they use a very similar set of operators. After all, they are all based on the same mathematical concepts. ONNX supports a wide set of operators, and has both official and unofficial converters for other frameworks. For example, a tensorflow-onnx converter has the ability of taking a tensorflow model, do a traversal of the computation network (it's a just a graph), reconstruct the graph replacing all operators with their ONNX equivalent. Ideally, if all operators supported by tensorflow are also supported by ONNX, we can have a perfect converter, being able to convert any tensorflow model to ONNX format. However this is not the case for most machine learning frameworks. All these frameworks are constantly adding new operators (with some being highly specialized), and it's very hard to keep up with all frameworks. ONNX is under active development, with new operator support added in each release, trying to catch up with the super set of all operators supported by all framework.

ONNX runtime is a high efficiency inference engine for ONNX models. Per its github page :

ONNX runtime's capability can be summarized as:

1. Load an ONNX model.
2. Define input values.
3. Perform inference of the model on the input values.

For (1), ONNX runtime supports loading a model from the filesystem or from a byte array in memory, which is convenient for us; For (2), we need to construct values in CPU memory; For (3), ONNX runtime automatically uses available accelerators and runtimes available on the host machine. An abstraction of a runtime is called an execution provider. Current execution providers include CUDA, TensorRT, Intel MKL, etc. ONNX runtime partitions the computation network (the model) into subgraphs, and run each sub-graph on the most efficient execution provider. A default fallback execution provider (the CPU) is guaranteed able to run all operators, so that even if no special accelerator or runtime (GPU, etc.) exists, ONNX runtime can still perform inference on an ONNX model, albeit at a much slower speed.

Beginning with version 10.0-3, MarkLogic server includes version 1.0.0 of the ONNX runtime.

ONNX XQuery and JavaScript API

The ONNX runtime is under active development, and its C API changes frequently. For this reason, we only provide core functionalities to allow you to achieve all of your objectives with only a minimal knowledge of the C API underneath.

We chose to expose a very small subset of the C API of onnxruntime representing the core functionality. The rest of the C APIs are implemented as options passed to those core APIs.

• New Types for the ONNX Runtime
• Exposed ONNX Runtime API
• Security
• Limitations

Example ONNX Applications

This section describes two example ONNX applications:

• Example ONNX Application using JavaScript
• Example ONNX Application using XQuery

declareUpdate();
xdmp.documentLoad('c:\\temp\\squeezenet1.1.onnx',
    {
      uri : '/squeezenet.onnx',
      permissions : xdmp.defaultPermissions(),
      format : 'binary'
});

'use strict';

const session = ort.session(cts.doc("/squeezenet.onnx"))
const inputCount = ort.sessionInputCount(session)
const outputCount = ort.sessionOutputCount(session)
var inputNames = []
var i,j
for (i=0;i<inputCount;i++){
  inputNames.push(ort.sessionInputName(session, i))
}
var outputNames = []
for (i=0;i<outputCount;i++){
  outputNames.push(ort.sessionOutputName(session, i))
}
var inputTypes = []
for (i=0;i<inputCount;i++){
  inputTypes.push(ort.sessionInputType(session, i))
}
var outputTypes = []
for (i=0;i<outputCount;i++){
  outputTypes.push(ort.sessionOutputType(session, i))
}

var inputValues = []
for (i=0;i<inputCount;i++){
  var p = 1
  for(j=0;j<inputTypes[i]["shape"].length;j++){
    p *= inputTypes[i]["shape"][j]
  }
  var data = []
  for(j=0;j<p;j++){
    data.push(j);
  }
  inputValues.push(ort.value(data, inputTypes[i]["shape"], "float"))
}
var inputMap = {}
for (i=0;i<inputCount;i++){
  inputMap[inputNames[i]] = inputValues[i]
}
ort.run(session, inputMap)

{
"softmaxout_1": "OrtValue(Shape:[1, 1000, 1, 1], Type: FLOAT)"
}

xquery version "1.0-ml";

let $session := ort:session(fn:doc("/squeezenet.onnx"))
let $input-count := ort:session-input-count($session)
let $output-count := ort:session-output-count($session)
let $input-names :=
  for $i in (0 to $input-count - 1) return ort:session-input-name($session, $i)
let $output-names :=
  for $i in (0 to $output-count - 1) return ort:session-output-name($session, $i)
let $input-types :=
  for $i in (0 to $input-count - 1) return ort:session-input-type($session, $i)
let $output-types :=
  for $i in (0 to $output-count - 1) return ort:session-output-type($session, $i)

let $input-values :=
  for $i in (1 to $input-count)
  (: generate some arbitrary input data. :)
  let $data := (1 to fn:fold-left(function($a, $b) { $a * $b }, 1, map:get($input-types, "shape")))
  return ort:value($data, map:get($input-types, "shape"), map:get($input-types, "tensor-type"))

let $input-map := map:map()
let $input-maps :=
  for $i in (1 to $input-count)
  return map:with($input-map, $input-names[$i], $input-values[$i])
let $input-map := $input-maps[$input-count]
return ort:run($session, $input-map)

<map:map xmlns:map="http://marklogic.com/xdmp/map" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xs="http://www.w3.org/2001/XMLSchema">
    <map:entry key="softmaxout_1">
          <map:value xsi:type="ort:value" xmlns:ort="http://marklogic.com/onnxruntime">OrtValue(Shape:[1, 1000, 1, 1], Type: FLOAT)</map:value>
    </map:entry>
</map:map>