MarkLogic Server 11.0 Product Documentation

Application Developer's Guide — Chapter 3

Key Transaction Attributes

MarkLogic supports the following transaction models:

• Single-statement transactions, which are automatically committed at the end of a statement. This is the default transaction model in MarkLogic.
• Multi-statement transactions, which can span multiple requests or statements and must be explicitly committed.

Updates made by a statement in a multi-statement transaction are visible to subsequent statements in the same transaction, but not to code running outside the transaction.

An application can use either or both transaction models. Single statement transactions are suitable for most applications. Multi-statement transactions are powerful, but introduce more complexity to your application. Focus on the concepts that match your chosen transactional programming model.

In addition to being single or multi-statement, transactions are typed as either update or query. The transaction type determines what operations are permitted and if, when, and how locks are acquired. By default, MarkLogic automatically detects the transaction type, but you can also explicitly specify the type.

The transactional model (single or multi-statement), commit mode (auto or explicit), and the transaction type (auto, query, or update) are fixed at the time a transaction is created. For example, if a block of code is evaluated by an xdmp:eval (XQuery) or xdmp.eval (JavaScript) call using same-statement isolation, then it runs in the caller's transaction context, so the transaction configuration is fixed by the caller, even if the called code attempts to change the settings.

The default transaction semantics vary slightly between XQuery and Server-Side JavaScript. The default behavior for each language is shown in the following table, along with information about changing the behavior. For details, see Transaction Type.

A statement can be either a query statement (read only) or an update statement. In XQuery, the first (or only) statement type determines the transaction type unless you explicitly set the transaction type. The statement type is determined through static analysis. In JavaScript, query statement type is assumed unless you explicitly set the transaction to update.

In the context of transactions, a statement has different meanings for XQuery and JavaScript. For details, see Understanding Statement Boundaries.

Understanding Statement Boundaries

Since transactions are often described in terms of statements, you must understand what constitutes a statement in your server-side programming language:

• Transactional Statements in XQuery
• Transactional Statements in Server-Side JavaScript

xquery version "1.0-ml";
xdmp:document-insert('/some/uri/doc.xml', <data/>);
(: end of statement 1 :)

xquery version "1.0-ml";
fn:doc('/some/uri/doc.xml'); 
(: end of statement 2 :)

'use strict';
declareUpdate();
xdmp.documentInsert('/some/uri/doc.json', {property: 'value'});
console.log('I did something!');
// end of module

Single-Statement Transaction Concept Summary

If you use the default model (single-statement, auto-commit), it is important to understand the following concepts:

Multi-Statement Transaction Concept Summary

If you use multi-statement transactions, it is important to understand the following concepts:

Transaction Type Overview

Transaction type determines the type of operations permitted by a transaction and whether or not the transaction uses locks. Transactions have either update or query type. Statements also have query or update type, depending on the type of operations they perform.

Update transactions and statements can perform both query and update operations. Query transactions and statements are read-only and may not attempt update operations. A query transaction can contain an update statement, but an error is raised if that statement attempts an update operation at runtime; for an example, see Query Transaction Mode.

MarkLogic Server determines transaction type in the following ways:

• Auto: (XQuery only) MarkLogic determines the transaction type through static analysis of the first (or only) statement in the transaction. Auto is the default behavior in XQuery.
• Explicit: Your code explicitly specifies the transaction type as update or query through an option, a call to xdmp:set-transaction-mode (XQuery) or xdmp.setTransactionMode (JavaScript), or by calling declareUpdate (JavaScript only).

For more details, see Controlling Transaction Type in XQuery or Controlling Transaction Type in JavaScript.

Query transactions use a system timestamp to access a consistent snapshot of the database at a particular point in time, rather than using locks. Update transactions use readers/writers locks. See Query Transactions: Point-in-Time Evaluation and Update Transactions: Readers/Writers Locks.

The following table summarizes the interactions between transaction types, statements, and locking behavior. These interactions apply to both single-statement and multi-statement transactions.

Controlling Transaction Type in XQuery

You do not need to explicitly set transaction type unless the default auto-detection is not suitable for your application. When the transaction type is auto (the default), MarkLogic determines the transaction type through static analysis of your code. In a multi-statement transaction, MarkLogic examines only the first statement when auto-detecting transaction type.

To explicitly set the transaction type:

• Declare the xdmp:update option in the XQuery prolog, or
• Call xdmp:set-transaction-mode prior to creating transactions that run in that mode, or
• Set the update option in the options node passed to functions such as xdmp:eval, xdmp:invoke, or xdmp:spawn.

Use the xdmp:update prolog option when you need to set the transaction type before the first transaction is created, such as at the beginning of a main module. For example, the following code runs as a multi-statement update transaction because of the prolog options:

declare option xdmp:commit "explicit";
declare option xdmp:update "true";

let $txn-name := "ExampleTransaction-1"
return (
  xdmp:set-transaction-name($txn-name),
  fn:concat($txn-name, ": ",
    xdmp:host-status(xdmp:host())
      //hs:transaction[hs:transaction-name eq $txn-name]
        /hs:transaction-mode)
  );
  xdmp:commit();

For more details, see xdmp:update and xdmp:commit in the XQuery and XSLT Reference Guide.

Setting transaction mode with xdmp:set-transaction-mode affects both the commit semantics (auto or explicit) and the transaction type (auto, query, or update). Setting the transaction mode in the middle of a transaction does not affect the current transaction. Setting the transaction mode affects the transaction creation semantics for the entire session.

The following example uses xdmp:set-transaction-mode to demonstrate that the currently running transaction is unaffected by setting the transaction mode to a different value. The example uses xdmp:host-status to examine the mode of the current transaction. (The example only uses xdmp:set-transaction-name to easily pick out the relevant transaction in the xdmp:host-statusresults.)

xquery version "1.0-ml";

declare namespace hs="http://marklogic.com/xdmp/status/host";

(: The first transaction created will run in update mode :)
declare option xdmp:commit "explicit";
declare option xdmp:update "true";

let $txn-name := "ExampleTransaction-1"
return (
  xdmp:set-transaction-name($txn-name),
  xdmp:set-transaction-mode("query"),  (: no effect on current txn :)
  fn:concat($txn-name, ": ",
    xdmp:host-status(xdmp:host())
      //hs:transaction[hs:transaction-name eq $txn-name]
        /hs:transaction-mode)
  );

(: complete the current transaction :)
xdmp:commit();

(: a new transaction is created, inheriting query mode from above :)
declare namespace hs="http://marklogic.com/xdmp/status/host";
let $txn-name := "ExampleTransaction-2"
return (
  xdmp:set-transaction-name($txn-name),
    fn:concat($txn-name, ": ",
      xdmp:host-status(xdmp:host())
        //hs:transaction[hs:transaction-name eq $txn-name]
          /hs:transaction-mode)
  );

If you paste the above example into Query Console, and run it with results displayed as text, you see the first transaction runs in update mode, as specified by xdmp:transaction-mode, and the second transaction runs in query mode, as specified by xdmp:set-transaction-mode:

ExampleTransaction-1: update
ExampleTransaction-2: query

You can include multiple option declarations and calls to xdmp:set-transaction-mode in your program, but the settings are only considered at transaction creation. A transaction is implicitly created just before evaluating the first statement. For example:

xquery version "1.0-ml";
declare option xdmp:commit "explicit";
declare option xdmp:update "true";

(: begin transaction :)
"this is an update transaction";
xdmp:commit();
(: end transaction :)

xquery version "1.0-ml";
declare option xdmp:commit "explicit";
declare option xdmp:update "false";

(: begin transaction :)
"this is a query transaction";
xdmp:commit();
(: end transaction :)

The following functions support commit and update options that enable you to control the commit mode (explicit or auto) and transaction type (update, query, or auto). For details, see the function reference for xdmp:eval or xdmp.eval.

Controlling Transaction Type in JavaScript

By default, Server-Side JavaScripts runs in a single-statement, auto-commit, query transaction. You can control transaction type in the following ways:

• Use the declareUpdate function to set the transaction type to update and/or specify the commit semantics, or
• Set the update option in the options node passed to functions such as xdmp.eval, xdmp.invoke, or xdmp.spawn; or
• Call xdmp.setTransactionModeprior to creating transactions that will run in that mode.

declareUpdate({explicitCommit: boolean});

Query Transactions: Point-in-Time Evaluation

Query transactions are read-only and never obtain locks on documents. This section explores the following concepts related to query transactions:

• System Timestamps and Fragment Versioning
• Query Transactions Run at a Timestamp (No Locks)
• Query Transactions See Latest Version of Documents Up To Timestamp of Transaction

Update Transactions: Readers/Writers Locks

Update transactions have the potential to change the database, so they obtain locks on documents to ensure transactional integrity. Update transactions run with readers/writers locks, not at a timestamp like query transactions. This section covers the following topics:

• Identifying Update Transactions
• Locks Are Acquired on Demand and Held Throughout a Transaction
• Visibility of Updates

if ( 1 = 2 ) 
then ( xdmp:document-insert("fake.xml", <a/>) ) 
else ()

  (: query statement, no locks needed :)
fn:doc("/docs/test.xml");
  (: update statement, readers/writers lock acquired :)
xdmp:node-replace(fn:doc("/docs/test.xml")/node(), <a>hello</a>);
  (: readers/writers lock released :)
  (: query statement, no locks needed :)
fn:doc("/docs/test.xml");

declare option xdmp:transaction-mode update;

  (: read lock acquired :)
fn:doc("/docs/test.xml");
  (: the following converts the lock to a readers/writers lock :)
xdmp:node-replace(fn:doc("/docs/test.xml")/node(), <a>hello</a>);
  (: readers/writers lock still held :)
fn:doc("/docs/test.xml");

  (: after the following statement, txn ends and locks released :)
xdmp:commit()

Example: Query and Update Transaction Interaction

The following figure shows three different transactions, T1, T2, and T3, and how the transactional semantics work for each one:

Assume T1 is a long-running update transaction which starts when the system is at timestamp 10 and ends up committing at timestamp 40 (meaning there were 30 updates or other changes to the system while this update statement runs).

When T2 reads the document being updated by T1 (doc.xml), it sees the latest version that has a system timestamp of 20 or less, which turns out to be the same version T1 uses before its update.

When T3 tries to update the document, it finds that T1 has readers/writers locks on it, so it waits for them to be released. After T1 commits and releases the locks, then T3 sees the newly updated version of the document, and performs its update which is committed at a new timestamp of 41.

Single-Statement, Automatically Committed Transactions

By default, all transactions in MarkLogic Server are single-statement, auto-commit transactions. In this default model, MarkLogic creates a transaction to evaluate each statement. When the statement completes, MarkLogic automatically commits (or rolls back, in case of error) the transaction, and then the transaction ends.

In Server-Side JavaScript, a JavaScript program (or script) is considered a single statement in the transactional sense. For details, see Understanding Statement Boundaries.

In a single statement transaction, updates made by a statement are not visible outside the statement until the statement completes and the transaction is committed.

The single-statement model is suitable for most applications. This model requires less familiarity with transaction details and introduces less complexity into your application:

• Statement and transaction are nearly synonymous.
• The server determines the transaction type through static analysis.
• If the statement completes successfully, the server automatically commits the transaction.
• If an error occurs, the server automatically rolls back any updates made by the statement.

Updates made by a single-statement transaction are not visible outside the statement until the statement completes. For details, see Visibility of Updates.

Use the semi-colon separator extension in XQuery to include multiple single-statement transactions in your program. For details, see Semi-Colon as a Statement Separator.

In Server-Side JavaScript, you need to use the declareUpdate() function to run an update. For details, see Controlling Transaction Type in JavaScript.

Multi-Statement, Explicitly Committed Transactions

When a transaction is created in a context in which the commit mode is set to explicit, the transaction will be a multi-statement transaction. This section covers the following related topics:

• Characteristics of Multi-Statement Transactions
• Committing Multi-Statement Transactions
• Rolling Back Multi-Statement Transactions
• Sessions

For details on setting the transaction type and commit mode, see Transaction Type.

For additional information about using multi-statement transactions in Java, see Multi-Statement Transactions in the XCC Developer's Guide.

xquery version "1.0-ml";

declare option xdmp:commit "explicit";
(: Begin transaction 1 :)
xdmp:document-insert('/docs/mst1.xml', <data/>);
(: This statement runs in the same txn, so sees /docs/mst1.xml :)
xdmp:document-insert('/docs/mst2.xml', fn:doc('/docs/mst1.xml'));
xdmp:commit();
(: Transaction ends, updates visible in database :)

declare option xdmp:commit "explicit";
(: Begin transaction 2 :)
xdmp:document-delete('/docs/mst1.xml');
xdmp:rollback();
(: Transaction ends, updates discarded :)

declare option xdmp:commit "explicit";
(: Begin transaction 3 :)
xdmp:document-delete('/docs/mst1.xml');
(: Transaction implicitly ends and rolls back due to
 :   reaching end of program without a commit :)

xquery version "1.0-ml";

declare option xdmp:commit "explicit";

(: Begin transaction 1 :)
xdmp:document-insert('/docs/mst1.xml', <data/>);
(: This statement runs in the same txn, so sees /docs/mst1.xml :)
xdmp:document-insert('/docs/mst2.xml', fn:doc('/docs/mst1.xml'));
xdmp:commit();
(: Transaction ends, updates visible in database :)

xquery version "1.0-ml";

declare option xdmp:commit "explicit";

(: transaction created :)
xdmp:document-insert("not-lost.xml", <data/>)
, xdmp:commit();
fn:error(xs:QName("EXAMPLE-ERROR"), "An error occurs here");
(: end of session or program :)

(: ==> Insert is retained because the statement
       calling commit completes successfully. :)

xquery version "1.0-ml";

declare option xdmp:commit "explicit";

(: transaction created :)
xdmp:document-insert("lost.xml", <data/>)
, xdmp:commit()
, fn:error(xs:QName("EXAMPLE-ERROR"), "An error occurs here");
(: end of session or program :)

(: ==> Insert is lost because the statement
       terminates with an error before commit can occur. :)

xquery version "1.0-ml";

declare option xdmp:commit "explicit";

xdmp:document-insert("/docs/test.xml", <a>hello</a>);
try {
  xdmp:document-delete("/docs/nonexistent.xml")
} catch ($ex) {
  (: handle error or rethrow :)
  if ($ex/error:code eq 'XDMP-DOCNOTFOUND') then ()
  else xdmp:rethrow()
}, xdmp:commit();
(: start of a new txn :)
fn:doc("/docs/test.xml")//a/text()

xquery version "1.0-ml";

declare option xdmp:commit "explicit";
                                            (: begin transaction :)
xdmp:document-insert("/docs/mst.xml", <data/>);
xdmp:commit()
, "this expr is evaluated and committed";
                                            (: end transaction :)
                                            (:begin transaction :)
declare option xdmp:commit "explicit";
xdmp:document-insert("/docs/mst.xml", <data/>);
xdmp:rollback()                             (: end transaction :)
, "this expr is never evaluated";
                                            (:begin transaction :)
"execution continues here, in a new transaction"
                                            (: end transaction :)

Semi-Colon as a Statement Separator

MarkLogic Server extends the XQuery language to include the semi-colon ( ; ) in the XQuery body as a separator between statements. Statements are evaluated in the order in which they appear. Each semi-colon separated statement in a transaction is fully evaluated before the next statement begins.

In a single-statement transaction, the statement separator is also a transaction separator. Each statement separated by a semi-colon is evaluated as its own transaction. It is possible to have a program where some semi-colon separated parts are evaluated as query statements and some are evaluated as update statements. The statements are evaluated in the order in which they appear, and in the case of update statements, one statement commits before the next one begins.

Semi-colon separated statements in auto commit mode (the default) are not multi-statement transactions. Each statement is a single-statement transaction. If one update statement commits and the next one throws a runtime error, the first transaction is not rolled back. If you have logic that requires a rollback if subsequent transactions fail, you must add that logic to your XQuery code, use multi-statement transactions, or use a pre-commit trigger. For information about triggers, see Using Triggers to Spawn Actions.

In a multi-statement transaction, the semi-colon separator does not act as a transaction separator. The semi-colon separated statements in a multi-statement transaction see updates made by previous statements in the same transaction, but the updates are not committed until the transaction is explicitly committed. If the transaction is rolled back, updates made by previously evaluated statements in the transaction are discarded.

The following diagram contrasts the relationship between statements and transactions in single and multi-statement transactions:

Transaction Mode Overview

Transaction mode combines the concepts of commit mode (auto or explicit) and transaction type (auto, update, or query). The transaction mode setting is session wide. You can control transaction mode in the following ways:

• Call the xdmp:set-transaction-mode XQuery function or the xdmp.setTransactionMode JavaScript function.
• Deprecated: Use the transaction-mode option of xdmp:eval (XQuery) or xdmp.eval (JavaScript) or related functions eval/invoke/spawn functions. Use the commit and update options instead.
• Deprecated: Use the XQuery prolog option xdmp:transaction-mode. Use the xdmp:commit and xdmp:update XQuery prolog options instead.

Be aware that the xdmp:commit and xdmp:update XQuery prolog options affect only the next transaction created after their declaration; they do not affect an entire session. Use xdmp:set-transaction-mode or xdmp.setTransactionMode if you need to change the settings at the session level.

Use the more specific commit mode and transaction type controls instead of setting transaction mode. These controls provide finer grained control over transaction configuration.

For example, use the following table to map the xdmp:transaction-mode XQuery prolog options to the xdmp:commit and xdmp:update prolog options. For more details, see Controlling Transaction Type in XQuery.

declare option xdmp:commit "auto";
declare option xdmp:update "auto";

declare option xdmp:commit "auto";
declare option xdmp:update "true";

declare option xdmp:commit "auto";
declare option xdmp:update "false";

declare option xdmp:commit "explicit";
declare option xdmp:update "auto";

declare option xdmp:commit "explicit";
declare option xdmp:update "true";

declare option xdmp:commit "explicit";
declare option xdmp:update "false";

Use the following table to map between the transaction-mode option and the commit and update options for xdmp:eval and related eval/invoke/spawn functions.

commit: "auto"
update: "auto"

commit: "auto"
update: "true"

commit: "auto"
update: "false"

commit: "explicit"
update: "auto"

commit: "explicit"
update: "true"

commit "explicit"
update "false"

Server-Side JavaScript modules use the declareUpdate function to indicate when the transaction mode is update-auto-commit or update. For more details, see Controlling Transaction Type in JavaScript.

To use multi-statement transactions in XQuery, you must explicitly set the transaction mode to multi-auto, query, or update. This sets the commit mode to explicit and specifies the transaction type. For details, see Transaction Type.

Selecting the appropriate transaction mode enables the server to properly optimize your queries. For more information, see Multi-Statement, Explicitly Committed Transactions.

The transaction mode is only considered during transaction creation. Changing the mode has no effect on the current transaction.

Explicitly setting the transaction mode affects only the current session. Queries run under xdmp:eval or xdmp.eval or a similar function with different-transaction isolation, or under xdmp:spawn do not inherit the transaction mode from the calling context. See Interactions with xdmp:eval/invoke.

Auto Transaction Mode

The default transaction mode is auto. This is equivalent to auto commit mode and auto transaction type. In this mode, all transactions are single-statement transactions. See Single-Statement, Automatically Committed Transactions.

Most XQuery applications use auto transaction mode. Using auto transaction mode allows the server to optimize each statement independently and minimizes locking on your files. This leads to better performance and decreases the chances of deadlock, in most cases.

Most Server-Side JavaScript applications use auto mode for code that does not perform updates, and update-auto-commit mode for code that performs updates. Calling declareUpdate with no arguments activates update-auto-commit mode; for more details, see Controlling Transaction Type in JavaScript..

In auto transaction mode:

• All transactions are single-statement transactions, so a new transaction is created for each statement.
• Static analysis of the statement prior to evaluation determines whether the created transaction runs in update or query mode.
• The transaction associated with a statement is automatically committed when statement execution completes, or automatically rolled back if an error occurs.

The update-auto-commit differs only in that the transaction is always an update transaction.

In XQuery, you can set the mode to auto explicitly with xdmp:set-transaction-mode or the xdmp:transaction-mode prolog option, but this is not required unless you've previously explicitly set the mode to update or query.

Query Transaction Mode

Query transaction mode is equivalent to explicit commit mode plus query transaction type.

In XQuery, query transaction mode is only in effect when you explicitly set the mode using xdmp:set-transaction-mode or the xdmp:transaction-mode prolog option. Transactions created in this mode are always multi-statement transactions, as described in Multi-Statement, Explicitly Committed Transactions.

You cannot create a multi-statement query transaction from Server-Side JavaScript.

In query transaction mode:

• Transactions can span multiple statements.
• The transaction is assumed to be read-only, so no locks are acquired. MarkLogic executes all statements in the transaction as point-in-time queries, using the system timestamp at the start of the transaction.
• All statements in the transaction must functionally be query (read-only) statements. MarkLogic raises an error at runtime if an update operation is attempted.
• Transactions which are not explicitly committed using xdmp:commit roll back when the session times out. However, since there are no updates to commit, rollback is only distinguishable if an explicit rollback occurs before statement completion.

An update statement can appear in a multi-statement query transaction, but it must not actually make any update calls at runtime. If a transaction running in query mode attempts an update operation, XDMP-UPDATEFUNCTIONFROMQUERY is raised. For example, no exception is raised by the following code because the program logic causes the update operation not to run:

xquery version "1.0-ml";
declare option xdmp:transaction-mode "query";

if (fn:false())then
  (: XDMP-UPDATEFUNCTIONFROMQUERY only if this executes :)
  xdmp:document-insert("/docs/test.xml", <a/>)
else ();
xdmp:commit();

Update Transaction Mode

Update transaction mode is equivalent to explicit commit mode plus update transaction type.

In XQuery, update transaction mode is only in effect when you explicitly set the mode using xdmp:set-transaction-mode or the xdmp:transaction-mode prolog option. Transactions created in update mode are always multi-statement transactions, as described in Multi-Statement, Explicitly Committed Transactions.

In Server-Side JavaScript, setting explicitCommit to true when calling declareUpdate puts the transaction into update mode.

In update transaction mode:

• Transactions can span multiple statements.
• The transaction is assumed to change the database, so readers/writers locks are acquired as needed.
• Statements in an update transaction can be either update or query statements.
• Transactions which are not explicitly committed using xdmp:commit roll back when the session times out.

Update transactions can contain both query and update statements, but query statements in update transactions still acquire read locks rather than using a system timestamp. For more information, see Update Transactions: Readers/Writers Locks.

Query-Single-Statement Transaction Mode

The query-single-statement transaction mode is equivalent to auto commit mode plus query transaction type.

In XQuery, this transaction mode is only in effect when you explicitly set the mode using xdmp:set-transaction-mode or the xdmp:transaction-mode prolog option. Transactions created in this mode are always single-statement transactions, as described in Single-Statement Transaction Concept Summary.

You cannot explicitly create a single-statement query-only transaction from Server-Side JavaScript, but this is the default transaction mode when declareUpdate is not present.

In this transaction mode:

• All transactions are single-statement transactions, so a new transaction is created for each statement.
• The transaction is assumed to be read-only, so no locks are acquired. The statement is evaluated as a point-in-time query, using the system timestamp at the start of the transaction.
• An error is raised at runtime if an update operation is attempted by the transaction.
• The transaction is automatically committed when statement execution completes, or automatically rolled back if an error occurs.

An update operation can appear in a this type of transaction, but it must not actually make any updates at runtime. If a transaction running in query mode attempts an update operation, XDMP-UPDATEFUNCTIONFROMQUERY is raised.

Multi-Auto Transaction Mode

Setting transaction mode to multi-auto is equivalent to explicit commit mode plus auto transaction type. In this mode, all transactions are multi-statement transactions, and MarkLogic determines for you whether the transaction type is query or update.

In multi-auto transaction mode:

• Transactions can span multiple statements.
• Static analysis of the first statement in the transaction determines whether the transaction type is query or update.
• Transactions which are not explicitly committed using xdmp:commit or xdmp.commit roll back when the session times out.

In XQuery, multi-auto transaction mode is only in effect when you explicitly set the mode using xdmp:set-transaction-modeor the xdmp:transaction-mode prolog option.

There is no equivalent to multi-auto transaction mode for Server-Side JavaScript.

Isolation Option to xdmp:eval/invoke

The xdmp:eval and xdmp:invoke XQuery functions and their JavaScript counterparts accept a set of options as an optional third parameter. The isolation option determines the behavior of the transaction that results from the eval/invoke operation, and it must be one of the following values:

• same-statement
• different-transaction

In same-statement isolation, the code executed by eval or invoke runs as part of the same statement and in the same transaction as the calling statement. Any updates done in the eval/invoke operation with same-statement isolation are not visible to subsequent parts of the calling statement. However, when using multi-statement transactions, those updates are visible to subsequent statements in the same transaction.

You may not perform update operations in code run under eval/invoke in same-statement isolation called from a query transaction. Since query transactions run at a timestamp, performing an update would require a switch between timestamp mode and readers/writers locks in the middle of a transaction, and that is not allowed. Statements or transactions that do so will throw XDMP-UPDATEFUNCTIONFROMQUERY.

You may not use same-statement isolation when using the database option of eval or invoke to specify a different database than the database in the calling statement's context. If your eval/invoke code needs to use a different database, use different-transaction isolation.

When you set the isolation to different-transaction, the code that is run by eval/invoke runs in a separate session and a separate transaction from the calling statement. The eval/invoke session and transaction will complete before continuing the rest of the caller's transaction. If the calling transaction is an update transaction, any committed updates done in the eval/invoke operation with different-transaction isolation are visible to subsequent parts of the calling statement and to subsequent statements in the calling transaction. However, if you use different-transaction isolation (which is the default isolation level), you need to ensure that you do not get into a deadlock situation (see Preventing Deadlocks).

The following table shows which isolation options are allowed from query statements and update statements.

This table is slightly simplified. For example, if an update statement calls a query statement with same-statement isolation, the query statement is actually run as part of the update statement (because it is run as part of the same transaction as the calling update statement), and it therefore runs with readers/writers locks, not in a timestamp.

Preventing Deadlocks

A deadlock is where two processes or threads are each waiting for the other to release a lock, and neither process can continue until the lock is released. Deadlocks are a normal part of database operations, and when the server detects them, it can deal with them (for example, by retrying one or the other transaction, by killing one or the other or both requests, and so on).

There are, however, some deadlock situations that MarkLogic Server cannot do anything about except wait for the transaction to time out. When you run an update statement that calls an xdmp:eval or xdmp:invoke statement, and the eval/invoke in turn is an update statement, you run the risk of creating a deadlock condition. These deadlocks can only occur in update statements; query statements will never cause a deadlock.

A deadlock condition occurs when a transaction acquires a lock of any kind on a document and then an eval/invoke statement called from that transaction attempts to get a write lock on the same document. These deadlock conditions can only be resolved by cancelling the query or letting the query time out.

To be completely safe, you can prevent these deadlocks from occurring by setting the prevent-deadlocks option to true, as in the following example:

xquery version "1.0-ml";
(: the next line ensures this runs as an update statement :)
declare option xdmp:update "true";
xdmp:eval("xdmp:node-replace(doc('/docs/test.xml')/a, <b>goodbye</b>)",
          (),
          <options xmlns="xdmp:eval">
            <isolation>different-transaction</isolation>
            <prevent-deadlocks>true</prevent-deadlocks>
          </options>) ,
doc("/docs/test.xml")

This statement will then throw the following exception:

XDMP-PREVENTDEADLOCKS: Processing an update from an update with
  different-transaction isolation could deadlock

In this case, it will indeed prevent a deadlock from occurring because this statement runs as an update statement, due to the xdmp:document-insert call, and therefore uses readers/writers locks. In line 2, a read lock is placed on the document with URI /docs/test.xml. Then, the xdmp:eval statement attempts to get a write lock on the same document, but it cannot get the write lock until the read lock is released. This creates a deadlock condition. Therefore the prevent-deadlocks option stopped the deadlock from occurring.

If you remove the prevent-deadlocks option, then it defaults to false (that is, it will allow deadlocks). Therefore, the following statement results in a deadlock:

This code is for demonstration purposes; if you run this code, it will cause a deadlock and you will have to cancel the query or wait for it to time out to clear the deadlock.

(: the next line ensures this runs as an update statement :)
if ( 1 = 2) then ( xdmp:document-insert("foobar", <a/>) ) else (),
doc("/docs/test.xml"),
xdmp:eval("xdmp:node-replace(doc('/docs/test.xml')/a, <b>goodbye</b>)",
          (),
          <options xmlns="xdmp:eval">
            <isolation>different-transaction</isolation>
          </options>) ,
doc("/docs/test.xml")

This is a deadlock condition, and the deadlock will remain until the transaction either times out, is manually cancelled, or MarkLogic is restarted. Note that if you take out the first call to doc("/docs/test.xml") in line 2 of the above example, the statement will not deadlock because the read lock on /docs/test.xml is not called until after the xdmp:eval statement completes.

Seeing Updates From eval/invoke Later in the Transaction

If you are sure that your update statement in an eval/invoke operation does not try to update any documents that are referenced earlier in the calling statement (and therefore does not result in a deadlock condition, as described in Preventing Deadlocks), then you can set up your statement so updates from an eval/invoke are visible from the calling transaction. This is most useful in transactions that have the eval/invoke statement before the code that accesses the newly updated documents.

If you want to see the updates from an eval/invoke operation later in your statement, the transaction must be an update transaction. If the transaction is a query transaction, it runs in timestamp mode and will always see the version of the document that existing before the eval/invoke operation committed.

For example, consider the following example, where doc("/docs/test.xml") returns <a>hello</a> before the transaction begins:

(: doc("/docs/test.xml") returns <a>hello</a> before running this :)
(: the next line ensures this runs as an update statement :)
if ( 1 = 2 ) then ( xdmp:document-insert("fake.xml", <a/>) ) else (),
xdmp:eval("xdmp:node-replace(doc('/docs/test.xml')/node(), <b>goodbye</b>)", (),
          <options xmlns="xdmp:eval">
            <isolation>different-transaction</isolation>
            <prevent-deadlocks>false</prevent-deadlocks>
          </options>) ,
doc("/docs/test.xml")

The call to doc("/docs/test.xml") in the last line of the example returns <a>goodbye</a>, which is the new version that was updated by the xdmp:eval operation.

You can often solve the same problem by using multi-statement transactions. In a multi-statement transaction, updates made by one statement are visible to subsequent statements in the same transaction. Consider the above example, rewritten as a multi-statement transaction. Setting the transaction mode to update removes the need for fake code to force classification of statements as updates, but adds a requirement to call xdmp:commit to make the updates visible in the database.

declare option xdmp:transaction-mode "update";

(: doc("/docs/test.xml") returns <a>hello</a> before running this :)
xdmp:eval("xdmp:node-replace(doc('/docs/test.xml')/node(), <b>goodbye</b>)", (),
          <options xmlns="xdmp:eval">
            <isolation>different-transaction</isolation>
            <prevent-deadlocks>false</prevent-deadlocks>
          </options>);
(: returns <a>goodbye</b> within this transaction :)
doc("/docs/test.xml"),
(: make updates visible in the database :)
xdmp:commit()

Running Multi-Statement Transactions under xdmp:eval/invoke

When you run a query using xdmp:eval or xdmp:invoke or their JavaScript counterparts with different-transaction isolation, or via xdmp:spawn or xdmp.spawn, a new transaction is created to execute the query, and that transaction runs in a newly created session. This has two important implications for multi-statement transactions evaluated with xdmp:eval or xdmp:invoke:

• Transaction mode is not inherited from the caller.
• Uncommitted updates are automatically rolled back when an eval/invoke'd or spawned query completes.

Therefore, when using multi-statement transactions in code evaluated under eval/invoke with different-transaction isolation or under xdmp:spawn or xdmp.spawn:

• Set the commit option to explicit in the options node if the transaction must run as a multi-statement transaction or use the XQuery xdmp:commit prolog option or JavaScript declareUpdate function to specify explicit commit mode.
• Always call xdmp:commit or xdmp.commit inside an eval/invoke'd multi-statement query if updates must be preserved.

Setting the commit mode in the XQuery prolog of the eval/invoke'd query is equivalent to setting it by passing an options node to xdmp:eval/invoke with commit set to explicit. Setting the mode through the options node enables you to set the commit mode without modifying the eval/invoke'd query.

For an example of using multi-statement transactions with different-transaction isolation, see Example: Multi-Statement Transactions and Different-transaction Isolation.

The same considerations apply to multi-statement queries evaluated using xdmp:spawn or xdmp.spawn.

Transactions run under same-statement isolation run in the caller's context, and so use the same transaction mode and benefit from committing the caller's transaction. For a detailed example, see Example: Multi-statement Transactions and Same-statement Isolation.

Example: Multi-statement Transactions and Same-statement Isolation

The following example demonstrates the interactions between multi-statement transactions and same-statement isolation, discussed in Interactions with xdmp:eval/invoke.

The goal of the sample is to insert a document in the database using xdmp:eval, and then examine and modify the results in the calling module. The inserted document will be visible to the calling module immediately, but not visible outside the module until transaction completion.

xquery version "1.0-ml";
declare option xdmp:transaction-mode "update";

(: insert a document in the database :)
let $query :=
  'xquery version "1.0-ml";
   xdmp:document-insert("/examples/mst.xml", <myData/>)
  '
return xdmp:eval(
  $query, (),
  <options xmlns="xdmp:eval">
    <isolation>same-statement</isolation>
  </options>);

(: demonstrate that it is visible to this transaction :)
if (fn:empty(fn:doc("/examples/mst.xml")//myData))
then ("NOT VISIBLE")
else ("VISIBLE");

(: modify the contents before making it visible in the database :)
xdmp:node-insert-child(doc('/examples/mst.xml')/myData, <child/>),
xdmp:commit()

(: result: VISIBLE :)

The same operation (inserting and then modifying a document before making it visible in the database) cannot be performed as readily using the default transaction model. If the module attempts the document insert and child insert in the same single-statement transaction, an XDMP-CONFLICTINGUPDATES error occurs. Performing these two operations in different single-statement transactions makes the inserted document immediately visible in the database, prior to inserting the child node. Attempting to perform the child insert using a pre-commit trigger creates a trigger storm, as described in Avoiding Infinite Trigger Loops (Trigger Storms).

The eval'd query runs as part of the calling module's multi-statement update transaction since the eval uses same-statement isolation. Since transaction mode is not inherited by transactions created in a different context, using different-transaction isolation would evaluate the eval'd query as a single-statement transaction, causing the document to be immediately visible to other transactions.

The call to xdmp:commit is required to preserve the updates performed by the module. If xdmp:commit is omitted, all updates are lost when evaluation reaches the end of the module. In this example, the commit must happen in the calling module, not in the eval'd query. If the xdmp:commit occurs in the eval'd query, the transaction completes when the statement containing the xdmp:eval call completes, making the document visible in the database prior to inserting the child node.

Example: Multi-Statement Transactions and Different-transaction Isolation

The following example demonstrates how different-transaction isolation interacts with transaction mode for multi-statement transactions. The same interactions apply to queries executed with xdmp:spawn. For more background, see Transaction Mode and Interactions with xdmp:eval/invoke.

In this example, xdmp:eval is used to create a new transaction that inserts a document whose content includes the current transaction id using xdmp:transaction. The calling query prints its own transaction id and the transaction id from the eval'd query.

xquery version "1.0-ml";

(: init to clean state; runs as single-statement txn :)
xdmp:document-delete("/docs/mst.xml");

(: switch to multi-statement transactions :)
declare option xdmp:transaction-mode "query";

let $sub-query :=
  'xquery version "1.0-ml";
   declare option xdmp:transaction-mode "update";       (: 1 :)
   xdmp:document-insert("/docs/mst.xml", <myData/>);
   
   xdmp:node-insert-child(
     fn:doc("/docs/mst.xml")/myData,
     <child>{xdmp:transaction()}</child>
   );
   xdmp:commit()                                        (: 2 :)
  '
return xdmp:eval($sub-query, (),
  <options xmlns="xdmp:eval">
    <isolation>different-transaction</isolation>
  </options>);
  
(: commit to end this transaction and get a new system
 : timestamp so the updates by the eval'd query are visible. :)
xdmp:commit();                                                (: 3 :)
(: print out my transaction id and the eval'd query transaction id :)
fn:concat("My txn id: ", xdmp:transaction()),           (: 4 :)
fn:concat("Subquery txn id: ", fn:doc("/docs/mst.xml")//child)

Setting the transaction mode in statement (: 1 :) is required because different-transaction isolation makes the eval'd query a new transaction, running in its own session, so it does not inherit the transaction mode of the calling context. Omitting xdmp:transaction-mode in the eval'd query causes the eval'd query to run in the default, auto, transaction mode.

The call to xdmp:commit at statement (: 2 :)is similarly required due to different-transaction isolation. The new transaction and the containing session end when the end of the eval'd query is reached. Changes are implicitly rolled back if a transaction or the containing session end without committing.

The xdmp:commit call at statement (: 3 :) ends the multi-statement query transaction that called xdmp:eval and starts a new transaction for printing out the results. This causes the final transaction at statement (: 4 :) to run at a new timestamp, so it sees the document inserted by xdmp:eval. Since the system timestamp is fixed at the beginning of the transaction, omitting this commit means the inserted document is not visible. For more details, see Query Transactions: Point-in-Time Evaluation.

If the query calling xdmp:eval is an update transaction instead of a query transaction, the xdmp:commit at statement (: 3 :) can be omitted. An update transaction sees the latest version of a document at the time the document is first accessed by the transaction. Since the example document is not accessed until after the xdmp:eval call, running the example as an update transaction sees the updates from the eval'd query. For more details, see Update Transactions: Readers/Writers Locks.

Example: Generating a Transaction Report With xdmp:host-status

Use the built-in xdmp:host-status to generate a list of the transactions running on a host, similar to the information available through the Host Status page of the Admin Interface.

This example generates a simple HTML report of the duration of all transactions on the local host:

xquery version "1.0-ml";

declare namespace html = "http://www.w3.org/1999/xhtml";
declare namespace hs="http://marklogic.com/xdmp/status/host";

<html>
  <body>
    <h2>Running Transaction Report for {xdmp:host-name()}</h2>
    <table border="1" cellpadding="5">
    <tr>
      <th>Transaction Id</th>
      <th>Database</th><th>State</th>
      <th>Duration</th>
    </tr>
    {
      let $txns:= xdmp:host-status(xdmp:host())//hs:transaction
      let $now := fn:current-dateTime()
      for $t in $txns
      return
        <tr>
          <td>{$t/hs:transaction-id}</td>
          <td>{xdmp:database-name($t/hs:database-id)}</td>
          <td>{$t/hs:transaction-state}</td>
          <td>{$now - $t/hs:start-time}</td>
        </tr>
    }
    </table>
  </body>
</html>

If you paste the above query into Query Console and run it with HTML output, the query generates a report similar to the following:

Many details about each transaction are available in the xdmp:host-status report. For more information, see xdmp:host-status in the XQuery & XSLT API Reference.

If we assume the first transaction in the report represents a deadlock, we can manually cancel it by calling xdmp:transaction-rollback and supplying the transaction id. For example:

xquery version "1.0-ml";
xdmp:transaction-rollback(xdmp:host(), 6335215186646946533)

You can also rollback transactions from the Host Status page of the Admin Interface.