Loading TOC...
Matches for transaction have been highlighted. remove
Application Developer's Guide (PDF)

Application Developer's Guide — Chapter 3

Understanding Transactions in MarkLogic Server

MarkLogic Server is a transactional system that ensures data integrity. This chapter describes the transaction model of MarkLogic Server, and includes the following sections:

For additional information about using multi-statement and XA/JTA transactions from XCC Java applications, see the XCC Developer's Guide.

Terms and Definitions

Although transactions are a core feature of most database systems, various systems support subtly different transactional semantics. Clearly defined terminology is key to a proper and comprehensive understanding of these semantics. To avoid confusion over the meaning of any of these terms, this section provides definitions for several terms used throughout this chapter and throughout the MarkLogic Server documentation. The definitions of the terms also provide a useful starting point for describing transactions in MarkLogic Server.

TermDefinition
statement

An XQuery main module, as defined by the W3C XQuery standard, to be evaluated by MarkLogic Server. A main module consists of an optional prolog and a complete XQuery expression.

Statements are either query statements or update statements, determined statically through static analysis prior to beginning the statement evaluation.

query statement

A statement that contains no update calls.

Query statements have a read-consistent view of the database. Since a query statement does not change the state of the database, the server optimizes it to hold no locks or lightweight locks, depending on the type of the containing transaction.

update statement

A statement with the potential to perform updates (that is, it contains one or more update calls).

A statement can be categorized as an update statement whether or not the statement performs an update at runtime. Update statements run with readers/writers locks, obtaining locks as needed for documents accessed in the statement.

transaction

A set of one or more statements which either all fail or all succeed.

A transaction is either an update transaction or a query (read-only) transaction, depending on the transaction mode and the kind of statements in the transaction. A transaction is either a single-statement transaction or a multi-statement transaction, depending on the transaction mode at the time it is created.

transaction mode

Controls the type and the commit semantics of newly created transactions. Mode is one of auto, query, or update. In the default mode, auto, all transactions are single-statement transactions. In query or update mode, all transactions are multi-statement transactions.

To learn more about transaction mode, see Transaction Mode.

single-statement transactionAny transaction created in auto transaction mode. Single-statement transactions always contain only one statment and are automatically committed on successful completion or rolled back on error.
multi-statement transactionA transaction created in query or update transaction mode, consisting of one or more statements which commit or rollback together. Changes made by one statement in the transaction are visible to subsequent statements in the same transaction prior to commit. Multi-statement transactions must be explicitly committed by calling xdmp:commit.
query transaction

A transaction which cannot perform any updates; a read-only transaction. A transaction consisting of a single query statement in auto transaction mode, or any transaction created in query transaction mode. Attempting to perform an update in a query transaction raises XDMP-UPDATEFUNCTIONFROMQUERY.

Instead of acquiring locks, query transactions run at a particular system timestamp and have a read-consistent view of the database.

update transaction

A transaction that may perform updates (make changes to the database). A transaction consisting of a single update statement in auto transaction mode, or any transaction created under update transaction mode.

Update transactions run with readers/writers locks, obtaining locks as needed for documents accessed in the transaction.

commitEnd a transaction and make the changes made by the transaction visible in the database. Single-statement transactions are automatically committed upon successful completion of the statement. Multi-statement transactions are explicitly committed using xdmp:commit, but the commit only occurs if and when the calling statement successfully completes.
rollbackImmediately terminate a transaction and discard all updates made by the transaction. All transactions are automatically rolled back on error. Multi-statement transactions can also be explicitly rolled back using xdmp:rollback, or implicitly rolled back due to timeout or reaching the end of the session without calling xdmp:commit.
system timestampA number maintained by MarkLogic Server that increases every time a change or a set of changes occurs in any of the databases in a system (including configuration changes from any host in a cluster). Each fragment stored in a database has system timestamps associated with it to determine the range of timestamps during which the fragment is valid.
readers/writers locks

A set of read and write locks that lock documents for reading and update at the time the documents are accessed.

MarkLogic Server uses readers/writers locks during update statements. Because update transactions only obtain locks as needed, update statements always see the latest version of a document. The view is still consistent for any given document from the time the document is locked. Once a document is locked, any update statements in other transactions wait for the lock to be released before updating the document. For more details, see Update Transactions: Readers/Writers Locks.

programThe expanded version of some XQuery code that is submitted to MarkLogic Server for evaluation, such as a query expression in a .xqy file or XQuery code submitted in an xdmp:eval statement. The program consists not only of the code in the calling module, but also any imported modules that are called from the calling module, and any modules they might call, and so on.
sessionA 'conversation' with a database on a MarkLogic Server instance. The session encapsulates state information such as connection information, credentials, and transaction mode. The precise nature of a session depends on the context of the conversation. For details, see Sessions.
requestAny invocation of a program, whether through an App Server, through a task server, through xdmp:eval, or through any other means. In addition, certain client calls to App Servers (for example, loading an XML document through XCC, downloading an image through HTTP, or locking a document through WebDAV) are also requests.

Overview of MarkLogic Server Transactions

This section summarizes the following key transaction concepts in MarkLogic Server for quick reference.

The remainder of the chapter covers these concepts in detail.

Transaction Attributes: Mode, Type, and Number of Statements

The MarkLogic Server API and XQuery extensions support two transaction programming models. The model affects the number of statements in the transaction, the visibility of updates, and the commit semantics. The two models are:

  • Single-statement transactions created in auto transaction mode, which are automatically committed (default).
  • Multi-statement transactions which must be explicitly committed. Updates made by a statement in a multi-statement transaction are visible to subsequent statements in the same transaction.

A program can use either or both models. The default model is suitable for most applications. Multi-statement transactions are powerful, but introduce more complexity to your application. Focus on the concepts which match your chosen transactional programming model.

In addition to being single or multi-statement, transactions are typed as either update or query. The type determines what operations are permitted and if, when, and how locks are acquired.

The transactional model (single or multi-statement) and the transaction type (query or update) are derived from the transaction mode in effect at the time the transaction is created. The transaction mode can be auto, update, or query. The default mode, auto, creates transactions that use the single-statement model. The query and update modes create multi-statement transactions. Changing the transaction mode in the middle of a transaction does not affect existing transactions.

The following table summarizes how these transaction attributes interlock:

Mode Model TypeCommit Uses Locks
auto (default)single-statementquery or update, depending upon static analysisautomaticonly for an update transaction
querymulti-statementqueryexplicitno
updatemulti-statementupdateexplicityes

A statement is either a query statement or an update statement, based on static analysis of the statement prior to evaluation. The kind of statement determines the transaction type in auto transaction mode and affects lock handling in multi-statement update transactions.

Single-Statement Transaction Concept Summary

If you use the default model (single-statement, auto-commit), it is important to understand the following concepts:

Single-Statement Transaction ConceptWhere to Learn More
Statements run in a transaction.Single vs. Multi-statement Transactions
A transaction contains exactly one statement.Single-Statement, Automatically Committed Transactions
Transactions are automatically committed at the end of every statement.Single-Statement, Automatically Committed Transactions
Transactions have either update or query type.
  • Query transactions use a system timestamp instead of locks.
  • Update transactions acquire locks.
Transaction Type
Transaction type is determined by static analysis of the statement.Transaction Type
Updates made by a statement are not visible until the statement (transaction) completes.Update Transactions: Readers/Writers Locks
Semi-colon can be used as a statement/transaction separator to include multiple statements in a main module.

Single-Statement, Automatically Committed Transactions

Semi-Colon as a Statement Separator

Multi-Statement Transaction Concept Summary

If you use multi-statement transactions, it is important to understand the following concepts:

Multi-Statement Transaction ConceptWhere to Learn More
Statements run in a transaction.Single vs. Multi-statement Transactions
A transaction contains one or more statements that either all succeed or all fail.Multi-Statement, Explicitly Committed Transactions
Multi-statement transactions must be explicitly committed using xdmp:commit.

Multi-Statement, Explicitly Committed Transactions

Committing Multi-Statement Transactions

Rollback can be implicit or explicit (xdmp:rollback).

Multi-Statement, Explicitly Committed Transactions

Rolling Back Multi-Statement Transactions

Transactions have either update or query type.
  • Query transactions use a system timestamp instead of acquiring locks
  • Update transactions acquire locks.
Transaction Type
Transactions run in a session.Sessions
Sessions have a transaction mode property which can be auto, query or update. The mode affects:
  • transaction type
  • commit semantics
  • how many statements a transaction can contain
Transaction Mode
The default transaction mode, auto, always creates single-statement, auto-commit transactions, with the type determined by static analysis of the statement.Single vs. Multi-statement Transactions
update and query transaction mode always create multi-statement, explicit commit transactions, with the type determined by the transaction mode.

Transaction Mode

Multi-Statement, Explicitly Committed Transactions

Updates made by a statement are not visible until the statement completes.Update Transactions: Readers/Writers Locks
Semi-colon can be used as a statement separator to include multiple statements in a transaction.Semi-Colon as a Statement Separator
Updates made by a statement are visible to subsequent statements in the same transaction.Multi-Statement, Explicitly Committed Transactions

Transaction Type

This section covers the following information related to transaction type. This information applies to both single-statement and multi-statement transactions.

Transaction Type Overview

Transaction type determines the type of operations permitted by a transaction and whether or not the transaction uses locks. Transactions have either update or query type. Statements also have query or update type, depending on the type of operations they perform.

Update transactions and statements can perform both query and update operations. Query transactions and statements are read-only and may not attempt update operations. A query transaction can contain an update statement, but an error is raised if that statement attempts an update operation at runtime; for an example, see Query Transaction Mode.

MarkLogic Server determines transaction type in two ways, depending upon the transaction programming model in effect when a transaction is created:

  • In the default model (single-statement, auto-commit), the transaction type is determined through static analysis of the statement in the transaction.
  • In the multi-statement model, transaction type is derived from the transaction mode explicitly set by the prolog option xdmp:transaction-mode or the XQuery built-in function xdmp:set-transaction-mode.

The model, in turn, is determined by the transaction mode, as discussed in Transaction Mode.

Query transactions use a system timestamp to access a consistent snapshot of the database at a particular point in time, rather than using locks. Update transactions use readers/writers locks. See Query Transactions: Point-in-Time Evaluation and Update Transactions: Readers/Writers Locks.

The following table summarizes the interactions between transaction types, statements, and locking behavior. These interactions apply to both single-statement and multi-staement transactions.

Transaction TypeStatement Behavior
queryqueryPoint-in-time view of documents. No locking required.
updateRuntime error.
updatequeryRead locks acquired, as needed.
updateReaders/writers locks acquired, as needed.

Query Transactions: Point-in-Time Evaluation

Query transactions are read-only and never obtain locks on documents. This section explores the following concepts related to query transactions:

System Timestamps and Fragment Versioning

To understand how transactions work in MarkLogic Server, it is important to understand how documents are stored. Documents are made up of one or more fragments. After a document is created, each of its fragments are stored in one or more stands. The stands are part of a forest, and the forest is part of a database. A database contains one or more forests.

Each fragment in a stand has system timestamps associated with it, which correspond to the range of system timestamps in which that version of the fragment is valid. When a document is updated, the update process creates new versions of any fragments that are changed. The new versions of the fragments are stored in a new stand and have a new set of valid system timestamps associated with them. Eventually, the system merges the old and new stands together and creates a new stand with only the latest versions of the fragments. Point-in-time queries also affect which versions of fragments are stored and preserved during a merge. After the merge, the old stands are deleted.

The range of valid system timestamps associated with fragments are used when a statement determines which version of a document to use during a transaction. For more details about merges, see the 'Understanding and Controlling Database Merges' chapter of the Administrator's Guide. For more details on how point-in-time queries affect which versions of documents are stored, see Point-In-Time Queries. For an example of how system timestamps change as updates occur to the system, see Example: Incrementing the System Timestamp.

Query Transactions Run at a Timestamp (No Locks)

Query transactions run at the system timestamp corresponding to transaction creation time. Calls to xdmp:request-timestamp return the same system timestamp at any point during a query transaction; they never return the empty sequence. Query transactions do not obtain locks on any documents, so other transactions can read or update the document while the transaction is executing.

Query Transactions See Latest Version of Documents Up To Timestamp of Transaction

When a query transaction is created, MarkLogic Server gets the current system timestamp (the number returned when calling the xdmp:request-timestamp function) and uses only the latest versions of documents whose timestamp is less than or equal to that number. Even if any of the documents that the transaction accesses are updated or deleted outside the transaction while the transaction is open, the use of timestamps ensures that all statements in the transaction always see a consistent view of the documents the transaction accesses.

Update Transactions: Readers/Writers Locks

Update transactions have the potential to change the database, so they obtain locks on documents to ensure transactional integrity. Update transactions run with readers/writers locks, not at a timestamp like query transactions. This section covers the following topics:

Identifying Update Transactions

In a single-statement transaction, if the potential for updates is found in the statement during static analysis, then the transaction is considered an update transaction. Depending on the specific logic of the transaction, it might not actually update anything, but a single-statement transaction that is determined (during static analysis) to be an update transaction runs as an update transaction, not a query transaction. For example, the following transaction runs as an update transaction even though the xdmp:document-insert can never occur:

if ( 1 = 2 ) 
then ( xdmp:document-insert("fake.xml", <a/>) ) 
else ()

In a multi-statement transaction, the transaction type always corresponds to the transaction mode when the transaction is created. If the transaction mode is update, then the transaction is an update transaction, even if none of the contained statements perform updates. Locks are acquired for all statements in an update transaction, whether or not they perform updates.

Calls to xdmp:request-timestamp always return the empty sequence during an update transaction; that is, if xdmp:request-timestamp returns a value, the transaction is a query transaction, not an update transaction.

Locks Are Acquired on Demand and Held Throughout a Transaction

Because update transactions do not run at a set timestamp, they see the latest view of any given document at the time it is first accessed by any statement in the transaction. Because an update transaction must successfully obtain locks on all documents it reads or writes in order to complete evaluation, there is no chance that a given update transaction will see 'half' or 'some' of the updates made by some other transactions; the statement is indeed transactional.

Once a lock is acquired, it is held until the transaction ends. This prevents other transactions from updating the read locked document and ensures a read-consistent view of the document. Query (read) operations require read locks. Update operations require readers/writers locks.

When a statement in an update transaction wants to perform an update operation, a readers/writers lock is acquired (or an existing read lock is converted into a readers/writers lock) on the document. A readers/writers lock is an exclusive lock. The readers/writers lock cannot be acquired until any locks held by other transactions are released.

Lock lifetime is an especially important consideration in multi-statement transactions. Consider the following single-statement example, in which a readers/writers lock is acquired only around the call to xdmp:node-replace:

  (: query statement, no locks needed :)
fn:doc("/docs/test.xml");
  (: update statement, readers/writers lock acquired :)
fn:node-replace(fn:doc("/docs/test.xml")/node(), <a>hello</al>);
  (: readers/writers lock released :)
  (: query statement, no locks needed :)
fn:doc("/docs/test.xml");

If the same example is rewritten as a multi-statement transactions, locks are held across all three statements:

declare option xdmp:transaction-mode 'update';

  (: read lock acquired :)
fn:doc("/docs/test.xml");
  (: the following converts the lock to a readers/writers lock :)
fn:node-replace(fn:doc("/docs/test.xml")/node(), <a>hello</al>);
  (: readers/writers lock still held :)
fn:doc("/docs/test.xml");

  (: after the following statement, txn ends and locks released :)
xdmp:commit()
Visibility of Updates

Updates are only visible within a transaction after the updating statement completes; updates are not visible within the updating statement. Updates are only visible to other transactions after the updating transaction commits. Pre-commit triggers run as part of the updating transaction, so they see updates prior to commit. Transaction mode affects the visibility of updates, indirectly, because it affects when commit occurs.

In the default single-statement transaction model, the commit occurs automatically when the statement completes. To use a newly updated document, you must separate the update and the access into two single-statement transactions or use multi-statement transactions.

In a multi-statement transaction, changes made by one statement in the transaction are visible to subsequent statements in the same transaction as soon as the updating statement completes. Changes are not visible outside the transaction until you call xdmp:commit.

An update statement cannot perform an update to a document that will conflict with other updates occurring in the same statement. For example, you cannot update a node and add a child element to that node in the same statement. An attempt to perform such conflicting updates to the same document in a single statement will fail with an XDMP-CONFLICTINGUPDATES exception.

Example: Query and Update Transaction Interaction

The following figure shows three different transactions, T1, T2, and T3, and how the transactional semantics work for each one:

Assume T1 is a long-running update transaction which starts when the system is at timestamp 10 and ends up committing at timestamp 40 (meaning there were 30 updates or other changes to the system while this update statement runs).

When T2 reads the document being updated by T1 (doc.xml), it sees the latest version that has a system timestamp of 20 or less, which turns out to be the same version T1 uses before its update.

When T3 tries to update the document, it finds that T1 has readers/writers locks on it, so it waits for them to be released. After T1 commits and releases the locks, then T3 sees the newly updated version of the document, and performs its update which is committed at a new timestamp of 41.

Single vs. Multi-statement Transactions

This section discusses the details of and differences between the two transaction programming models supported by MarkLogic Server, single-statement and multi-statement transactions. The following topics are covered:

Single-Statement, Automatically Committed Transactions

By default, all transactions in MarkLogic Server are single-statement, auto-commit transactions. In this default model, a transaction is created to evaluate each statement. When the statement completes, the transaction is automatically committed (or rolled back, in case of error), and the transaction ends.

Updates made by the statement are not visible outside the statement until the statement completes and the transaction is committed.

The single-statement model is suitable for most applications. This model requires less familiarity with transaction details and introduces less complexity into your application:

  • Statement and transaction are nearly synonymous.
  • The server determines the transaction type through static analysis.
  • If the statement completes successfully, the server automatically commits the transaction.
  • If an error occurs, the server automatically rolls back any updates made by the statement.

Updates made by a single-statement transaction are not visible outside the statement until the statement completes. For details, see Visibility of Updates.

Use the semi-colon separator extension to include multiple single-statement transactions in your program. For details, see Semi-Colon as a Statement Separator.

In Server-Side JavaScript, you need to use the declareUpdate() function to run an update. For details, see declareUpdate Function in the JavaScript Reference Guide.

Multi-Statement, Explicitly Committed Transactions

When a transaction is created in a context in which the transaction mode is explicitly set to query or update, the transaction will be a multi-statement transaction. This section covers:

For details on setting the transaction mode, see Controlling the Transaction Mode.

For additional information about using multi-statement transactions in Java, see 'Multi-Statement Transactions' in the XCC Developer's Guide.

In Server-Side JavaScript, to run a multi-statement transaction you need to use the declareUpdate() function with the explicitCommit option set to true. For details, see declareUpdate Function in the JavaScript Reference Guide.

Characteristics of Multi-Statement Transactions

Using multi-statement transactions introduces more complexity into your application and requires a deeper understanding of transaction handling in MarkLogic Server. In a multi-statement transaction:

  • Semi-colon acts as a separator between statements in the same transaction.
  • Each statement in the transaction sees changes made by previously evaluated statements in the same transaction.
  • The statements in the transaction either all commit or all fail.
  • You must use xdmp:commit to commit the transaction.
  • You can use xdmp:rollback to abort the transaction.

A multi-statement transaction is bound to the database in which it is created. You cannot use a transaction id created in one database context to perform an operation in the same transaction on another database.

The statements in a multi-statement transaction are serialized, even if they run in different requests. That is, one statement in the transaction completes before another one starts, even if the statements execute in different requests.

A multi-statement transaction ends only when it is explicitly committed using xdmp:commit, when it is explicitly rolled back using xdmp:rollback, or when it is implictly rolled back through timeout, error, or session completion. Failure to explicitly commit or roll back a multi-statement transaction might retain locks and keep resources tied up until the transaction times out or the containing session ends. At that time, the transaction rolls back. Best practice is to always explicitly commit or rollback a multi-statement transaction.

The following example contains 3 multi-statement transactions. The first transaction is explicitly committed, the second is explicitly rolled back, and the third is implicitly rolled back when the session ends without a commit or rollback call. Running the example in Query Console is equivalent to evaluating it using xdmp:eval with different transaction isolation, so the final transaction rolls back when the end of the query is reached because the session ends. For details about multi-statement transaction interaction with sessions, see Sessions.

xquery version "1.0-ml";

declare option xdmp:transaction-mode "update";

(: Begin transaction 1 :)
xdmp:document-insert('/docs/mst1.xml', <data/>);
(: This statement runs in the same txn, so sees /docs/mst1.xml :)
xdmp:document-insert('/docs/mst2.xml', fn:doc('/docs/mst1.xml'));
xdmp:commit();
(: Transaction ends, updates visible in database :)

(: Begin transaction 2 :)
xdmp:document-delete('/docs/mst1.xml');
xdmp:rollback();
(: Transaction ends, updates discarded :)

(: Begin transaction 3 :)
xdmp:document-delete('/docs/mst1.xml');
(: Transaction implicitly ends and rolls back due to
 :   reaching end of program without a commit :)

As discussed in Update Transactions: Readers/Writers Locks, multi-statement update transactions use locks. A multi-statement update transaction can contain both query and update operations. Query operations in a multi-statement update transaction acquire read locks as needed. Update operations in the transaction will upgrade such locks to read/write locks or acquire new read/write locks if needed.

Instead of acquiring locks, a multi-statement query transaction uses a system timestamp to give all statements in the transaction a read consistent view of the database, as discussed in Query Transactions: Point-in-Time Evaluation. The system timestamp is determined when the query transaction is created, so all statements in the transaction see the same version of accessed documents.

Committing Multi-Statement Transactions

Multi-statement transactions are explicitly committed by calling xdmp:commit. If a multi-statement update transaction does not call xdmp:commit, all its updates are lost when the transaction ends. Leaving a transaction open by not committing updates ties up locks and other resources.

Once updates are committed, the transaction ends and evaluation of the next statement continues in a new transaction. For example:

xquery version "1.0-ml";

declare option xdmp:transaction-mode "update";

(: Begin transaction 1 :)
xdmp:document-insert('/docs/mst1.xml', <data/>);
(: This statement runs in the same txn, so sees /docs/mst1.xml :)
xdmp:document-insert('/docs/mst2.xml', fn:doc('/docs/mst1.xml'));
xdmp:commit();
(: Transaction ends, updates visible in database :)

Calling xdmp:commit commits updates and ends the transaction only after the calling statement successfully completes. This means updates can be lost even after calling xdmp:commit, if an error occurs before the committing statement completes. For this reason, it is best practice to call xdmp:commit at the end of a statement.

The following example preserves updates even in the face of error because the statement calling xdmp:commit always completes.:

xquery version "1.0-ml";

declare option xdmp:transaction-mode "update";

(: transaction created :)
xdmp:document-insert('not-lost.xml', <data/>)
, xdmp:commit();
fn:error('An error occurs here');
(: end of session or program :)

(: ==> Insert is retained because the statement
       calling commit completes sucessfully. :)

By contrast, the update in this example is lost because the error occurring in the same statement as the xdmp:commit call prevents successful completion of the committing statement:

xquery version "1.0-ml";

declare option xdmp:transaction-mode "update";

(: transaction created :)
xdmp:document-insert('lost.xml', <data/>)
, xdmp:commit()
, fn:error('An error occurs here');
(: end of session or program :)

(: ==> Insert is lost because the statement
       terminates with an error before commit can occur. :)

Uncaught exceptions cause a transaction rollback. If code in a multi-statement transaction might raise an exception that should not abort the transaction, wrap the code in a try-catch block and take appropriate action in the catch handler. For example:

xquery version "1.0-ml";

declare option xdmp:transaction-mode "update";

xdmp:document-insert("/docs/test.xml", <a>hello</a>);
try {
  xdmp:document-delete("/docs/nonexistent.xml")
} catch ($ex) {
  (: handle error or rethrow :)
  if ($ex/error:code eq 'XDMP-DOCNOTFOUND') then ()
  else xdmp:rethrow()
}, xdmp:commit();
(: start of a new txn :)
fn:doc("/docs/test.xml")//a/text()
Rolling Back Multi-Statement Transactions

Multi-statement transactions are rolled back either implicitly (on error or when the containing session terminates), or explicitly (using xdmp:rollback). Calling xdmp:rollback immediately terminates the current transaction. Evaluation of the next statement continues in a new transaction. For example:

xquery version "1.0-ml";

declare option xdmp:transaction-mode "update";
                                            (: begin transaction :)
xdmp:document-insert("/docs/mst.xml", <data/>);
xdmp:commit()
, "this expr is evaluated and committed";
                                            (: end transaction :)
                                            (:begin transaction :)
xdmp:document-insert("/docs/mst.xml", <data/>);
xdmp:rollback()                             (: end transaction :)
, "this expr is never evaluated";
                                            (:begin transaction :)
"execution continues here, in a new transaction"
                                            (: end transaction :)

The result of a statement terminated with xdmp:rollback is always the empty sequence.

Best practice is to explicitly rollback using xdmp:rollback. Waiting on implicit rollback at session end leaves the transaction open and ties up locks and other resources until the session times out. This can be a relatively long time. For example, an HTTP session can span multiple HTTP requests. For details, see Sessions.

Sessions

A session is a 'conversation' with a database in a MarkLogic Server instance. A session encapsulates state about the conversation, such as connection information, credentials, and transaction mode. When using multi-statement transactions, you should understand when evaluation might occur in a different session because:

  • transaction mode is an attribute of a session.
  • uncommitted transactions automatically roll back when the containing session ends.

For example, since a query evaluated by xdmp:eval with different-transaction isolation runs in its own session, it does not inherit the transaction mode setting from the caller. Also, if the transaction is still open (uncommitted) when evaluation reaches the end of the eval'd query, the transaction automatically rolls back.

By contrast, in an HTTP session, the transaction mode might apply to queries run in response to multiple HTTP requests. Uncommitted transactions remain open until the HTTP session times out, which can be a relatively long time.

The exact nature of a session depends on the 'conversation' context. The following table summarizes the most common types of sessions encountered by a MarkLogic Server application and their lifetimes:

Session Type Session Lifetime

HTTP

An HTTP client talking to an HTTP App Server.

A session is created when the first HTTP request is received from a client for which no session already exists. The session persists across requests until the session times out.

XCC

An XCC Java application talking to an XDBC App Server

A session is created when a Session object is instantiated and persists until the Session object is finalized, you call Session.close(), or the session times out.
Standalone query evaluated:
  • by xdmp:eval or xdmp:invoke with different-transaction isolation
  • by xdmp:spawn
  • as task on the Task Server)
A session is created to evaluate the eval/invoke/spawn'd query or task and ends when the query or task completes.

Session timeout is an App Server configuration setting. For details, see admin:appserver-set-session-timeout in XQuery and XSLT Reference Guide or the Session Timeout configuration setting in the Admin Interface for the App Server.

Semi-Colon as a Statement Separator

MarkLogic Server extends the XQuery language to include the semi-colon ( ; ) in the XQuery body as a separator between statements. Statements are evaluated in the order in which they appear. Each semi-colon separated statement in a transaction is fully evaluated before the next statement begins.

In a single-statement transaction, the statement separator is also a transaction separator. Each statement separated by a semi-colon is evaluated as its own transaction. It is possible to have a program where some semi-colon separated parts are evaluated as query statements and some are evaluated as update statements. The statements are evaluated in the order in which they appear, and in the case of update statements, one statement commits before the next one begins.

Semi-colon separated statements in auto transaction mode (the default) are not multi-statement transactions. Each statement is a single-statement transaction. If one update statement commits and the next one throws a runtime error, the first transaction is not rolled back. If you have logic that requires a rollback if subsequent transactions fail, you must add that logic to your XQuery code, use multi-statement transactions, or use a pre-commit trigger. For information about triggers, see Using Triggers to Spawn Actions.

In a multi-statement transaction, the semi-colon separator does not act as a transaction separator. The semi-colon separated statements in a multi-statement transaction see updates made by previous statements in the same transaction, but the updates are not committed until the transaction is explicitly committed. If the transaction is rolled back, updates made by previously evaluated statements in the transaction are discarded.

The following diagram contrasts the relationship between statements and transactions in single and multi-statement transactions:

Transaction Mode

This section covers the following topics related to transaction mode:

Transaction Mode Overview

The transaction mode in effect when a transaction is created determines the transaction type. Thus, the mode determines what kinds of statement can run, when and how transactions are committed, and whether or not updates are visible across statement boundaries.

Most applications will use the default transaction mode, auto. In this mode, all transactions are single-statement transactions, as discussed in Single-Statement, Automatically Committed Transactions.

To use multi-statement transactions, you must explicitly set the transaction mode to query or update. Use query mode for read-only transactions. Use update mode for transactions that might perform updates. Selecting the appropriate mode allows the server to properly optimize your queries. For more information, see Multi-Statement, Explicitly Committed Transactions.

The transaction mode is only considered during transaction creation. Changing the mode has no effect on the current transaction. For details, see Controlling the Transaction Mode.

Explictly setting the transaction mode affects only the current session. Queries run under xdmp:eval with different-transaction isolation or under xdmp:spawn do not inherit the transaction mode from the calling context. See Interactions with xdmp:eval/invoke.

Controlling the Transaction Mode

You need not set the transaction mode if your application uses single-statement transactions since the default mode is auto. If you need to use multi-statement transactions, then you must explicitly set the mode.

To explicitly set the transaction mode:

Use the xdmp:transaction-mode prolog option when you need to set the transaction mode before the first transaction is created.

Changing the transaction mode in the middle of a transaction does not affect the current transaction.

The following example demonstrates that xdmp:set-transaction-mode has no effect on the current transaction by using xdmp:host-status to examine the mode of the current transaction. (The example only uses xdmp:set-transaction-name to easily pick out the relevant transaction in the xdmp:host-status results.)

xquery version "1.0-ml";

declare namespace hs="http://marklogic.com/xdmp/status/host";

(: The first transaction created will run in update mode :)
declare option xdmp:transaction-mode "update";
let $txn-name := "ExampleTransaction-1"
return (
  xdmp:set-transaction-name($txn-name),
  xdmp:set-transaction-mode("query"),  (: no effect on current txn :)
  fn:concat($txn-name, ": ",
    xdmp:host-status(xdmp:host())
      //hs:transaction[hs:transaction-name eq $txn-name]
        /hs:transaction-mode)
  );

(: complete the current transaction :)
xdmp:commit();

(: a new transaction is created, inheriting query mode from above :)
declare namespace hs="http://marklogic.com/xdmp/status/host";
let $txn-name := "ExampleTransaction-2"
return (
  xdmp:set-transaction-name($txn-name),
    fn:concat($txn-name, ": ",
      xdmp:host-status(xdmp:host())
        //hs:transaction[hs:transaction-name eq $txn-name]
          /hs:transaction-mode)
  );

If you paste the above example into Query Console, and run it with results displayed as text, you see the first transaction runs in update mode, as specified by xdmp:transaction-mode, and the second transaction runs in query mode, as specified by xdmp:set-transaction-mode:

ExampleTransaction-1: update
ExampleTransaction-2: query

You can include multiple option declarations and calls to xdmp:set-transaction-mode in your program, but transaction mode is only considered at transaction creation. A transaction is implicitly created just before evaluating the first statement in the transaction. For example:

xquery version '1.0-ml';

declare option xdmp:transaction-mode 'update';

(: begin transaction :)
'this is an update transaction';
xdmp:commit();
(: end transaction :)

declare option xdmp:transaction-mode 'query';

(:begin transaction :)
'this is a query transaction';
xdmp:commit();
(: end transaction :)

Auto Transaction Mode

The default transaction mode is auto. In this mode, all transactions are single-statement transactions. See Single-Statement, Automatically Committed Transactions.

Most applications should use auto transaction mode. Using auto transaction mode allows the server to optimize each statement independently and minimizes locking on your files. This leads to better performance and decreases the chances of deadlock, in most cases.

In auto transaction mode:

  • All transactions are single-statement transactions, so a new transaction is created for each statement.
  • Static analysis of the statement prior to evaluation determines whether the created transaction runs in update or query mode.
  • The transaction associated with a statement is automatically committed when statement execution completes, or automatically rolled back if an error occurs.

You can set the mode to auto explicitly with xdmp:set-transaction-mode or the xdmp:transaction-mode prolog option, but this is not required unless you've previously explicitly set the mode to update or query.

Query Transaction Mode

Query transaction mode is only in effect when you explicitly set the mode using xdmp:set-transaction-mode or the xdmp:transaction-mode prolog option. Transactions created in this mode are always multi-statement transactions, as described in Multi-Statement, Explicitly Committed Transactions.

In query transaction mode:

  • Transactions can span multiple statements.
  • The transaction is assumed to be read-only, so no locks are acquired. All statements in the transaction are executed as point-in-time queries, using the system timestamp at the start of the transaction.
  • All statements in the transaction should functionally be query (read-only) statements. An error is raised at runtime if an update operation is attempted.
  • Transactions which are not explicitly committed using xdmp:commit roll back when the session times out. However, since there are no updates to commit, rollback is only distinguishable if an explicit rollback occurs before statement completion.

An update statement can appear in a multi-statement query transaction, but it must not actually make any update calls at runtime. If a transaction running in query mode attempts an update operation, XDMP-UPDATEFUNCTIONFROMQUERY is raised. For example, no exception is raised by the following code because the program logic causes the update operation not to run:

xquery version "1.0-ml";
declare option xdmp:transaction-mode "query";

if (fn:false())then
  (: XDMP-UPDATEFUNCTIONFROMQUERY only if this executes :)
  xdmp:document-insert("/docs/test.xml", <a/>)
else ();
xdmp:commit();

Update Transaction Mode

Update transaction mode is only in effect when you explicitly set the mode using xdmp:set-transaction-mode or the xdmp:transaction-mode prolog option. Transactions created in update mode are always multi-statement transactions, as described in Multi-Statement, Explicitly Committed Transactions.

In update transaction mode:

  • Transactions can span multiple statements.
  • The transaction is assumed to change the database, so readers/writers locks are acquired as needed.
  • Statements in an update transaction can be either update or query statements.
  • Transactions which are not explicitly committed using xdmp:commit roll back when the session times out.

Update transactions can contain both query and update statements, but query statements in update transactions still acquire read locks rather than using a system timestamp. For more information, see Update Transactions: Readers/Writers Locks.

Interactions with xdmp:eval/invoke

The xdmp:eval and xdmp:invoke functions allow you to start one transaction from the context of another. The xdmp:eval function submits a string to be evaluated and the xdmp:invoke function evaluates a stored module. You can control the semantics of xdmp:eval and xdmp:invoke with options to the functions, and this can subtly change the transactional semantics of your program. This section describes some of those subtleties and includes the following parts:

Isolation Option to xdmp:eval/invoke

The xdmp:eval and xdmp:invoke functions take an options node as the optional third parameter. The isolation option determines the behavior of the transaction that results from the eval/invoke operation, and it must be one of the following values:

  • same-statement
  • different-transaction

In same-statement isolation, the code executed by xdmp:eval or xdmp:invoke runs as part of the same statement and in the same transaction as the calling statement. Any updates done in the eval/invoke operation with same-statement isolation are not visible to subsequent parts of the calling statement. However, when using multi-statement transactions, those updates are visible to subsequent statements in the same transaction.

You may not perform update operations in code run under eval/invoke in same-statement isolation called from a query transaction. Since query transactions run at a timestamp, performing an update would require a switch between timestamp mode and readers/writers locks in the middle of a transaction, and that is not allowed. Statements or transactions that do so will throw XDMP-UPDATEFUNCTIONFROMQUERY.

You may not use same-statement isolation when using the database option of xdmp:eval or xdmp:invoke to specify a different database than the database in the calling statement's context. If your eval/invoke code needs to use a different database, use different-transaction isolation.

When you set the isolation to different-transaction, the code that is run when you call xdmp:eval or xdmp:invoke runs in a separate session and a separate transaction from the calling statement. The eval/invoke session and transaction will complete before continuing the rest of the caller's transaction. If the calling transaction is an update transaction, any committed updates done in the eval/invoke operation with different-transaction isolation are visible to subsequent parts of the calling statement and to subsequent statements in the calling transaction. However, if you use different-transaction isolation (which is the default isolation level), you need to ensure that you do not get into a deadlock situation (see Preventing Deadlocks).

The following table shows which isolation options are allowed from query statements and update statements.

Calling StatementCalled Statement (xdmp:eval, xdmp:invoke)
same-statement isolationdifferent-transaction isolation
query statementupdate statementquery statementupdate statement
query statement (timestamp mode)YesYes, if no update takes place. If an update takes place, throws exception.YesYes
update statement (readers/writers locks mode)Yes (see Note)YesYesYes (possible deadlock if updating a document with any lock)

This table is slightly simplified. For example, if an update statement calls a query statement with same-statement isolation, the 'query statement' is actually run as part of the update statement (because it is run as part of the same transaction as the calling update statement), and it therefore runs with readers/writers locks, not in a timestamp.

Preventing Deadlocks

A deadlock is where two processes or threads are each waiting for the other to release a lock, and neither process can continue until the lock is released. Deadlocks are a normal part of database operations, and when the server detects them, it can deal with them (for example, by retrying one or the other transaction, by killing one or the other or both requests, and so on).

There are, however, some deadlock situations that MarkLogic Server cannot do anything about except wait for the transaction to time out. When you run an update statement that calls an xdmp:eval or xdmp:invoke statement, and the eval/invoke in turn is an update statement, you run the risk of creating a deadlock condition. These deadlocks can only occur in update statements; query statements will never cause a deadlock.

A deadlock condition occurs when a transaction acquires a lock of any kind on a document and then an eval/invoke statement called from that transaction attempts to get a write lock on the same document. These deadlock conditions can only be resolved by cancelling the query or letting the query time out.

To be completely safe, you can prevent these deadlocks from occurring by setting the prevent-deadlocks option to true, as in the following example:

xquery version "1,0-ml";
(: the next line ensures this runs as an update statement :)
declare option xdmp:update "true";
xdmp:eval("xdmp:node-replace(doc('/docs/test.xml')/a, <b>goodbye</b>)",
          (),
          <options xmlns="xdmp:eval">
            <isolation>different-transaction</isolation>
            <prevent-deadlocks>true</prevent-deadlocks>
          </options>) ,
doc("/docs/test.xml")

This statement will then throw the following exception:

XDMP-PREVENTDEADLOCKS: Processing an update from an update with
  different-transaction isolation could deadlock

In this case, it will indeed prevent a deadlock from occurring because this statement runs as an update statement, due to the xdmp:document-insert call, and therefore uses readers/writers locks. In line 2, a read lock is placed on the document with URI /docs/test.xml. Then, the xdmp:eval statement attempts to get a write lock on the same document, but it cannot get the write lock until the read lock is released. This creates a deadlock condition. Therefore the prevent-deadlocks option stopped the deadlock from occurring.

If you remove the prevent-deadlocks option, then it defaults to false (that is, it will allow deadlocks). Therefore, the following statement results in a deadlock:

This code is for demonstration purposes; if you run this code, it will cause a deadlock and you will have to cancel the query or wait for it to time out to clear the deadlock.

(: the next line ensures this runs as an update statement :)
if ( 1 = 2) then ( xdmp:document-insert("foobar", <a/>) ) else (),
doc("/docs/test.xml"),
xdmp:eval("xdmp:node-replace(doc('/docs/test.xml')/a, <b>goodbye</b>)",
          (),
          <options xmlns="xdmp:eval">
            <isolation>different-transaction</isolation>
          </options>) ,
doc("/docs/test.xml")

This is a deadlock condition, and the deadlock will remain until the transaction either times out, is manually cancelled, or MarkLogic is restarted. Note that if you take out the first call to doc("/docs/test.xml") in line 2 of the above example, the statement will not deadlock because the read lock on /docs/test.xml is not called until after the xdmp:eval statement completes.

Seeing Updates From eval/invoke Later in the Transaction

If you are sure that your update statement in an eval/invoke operation does not try to update any documents that are referenced earlier in the calling statement (and therefore does not result in a deadlock condition, as described in Preventing Deadlocks), then you can set up your statement so updates from an eval/invoke are visible from the calling transaction. This is most useful in transactions that have the eval/invoke statement before the code that accesses the newly updated documents.

If you want to see the updates from an eval/invoke operation later in your statement, the transaction must be an update transaction. If the transaction is a query transaction, it runs in timestamp mode and will always see the version of the document that existing before the eval/invoke operation committed.

For example, consider the following example, where doc("/docs/test.xml") returns <a>hello</a> before the transaction begins:

(: doc("/docs/test.xml") returns <a>hello</a> before running this :)
(: the next line ensures this runs as an update statement :)
if ( 1 = 2 ) then ( xdmp:document-insert("fake.xml", <a/>) ) else (),
xdmp:eval("xdmp:node-replace(doc('/docs/test.xml')/node(), <b>goodbye</b>)", (),
          <options xmlns="xdmp:eval">
            <isolation>different-transaction</isolation>
            <prevent-deadlocks>false</prevent-deadlocks>
          </options>) ,
doc("/docs/test.xml")

The call to doc("/docs/test.xml") in the last line of the example returns <a>goodbye</a>, which is the new version that was updated by the xdmp:eval operation.

You can often solve the same problem by using multi-statement transactions. In a multi-statement transaction, updates made by one statement are visible to subsequent statements in the same transaction. Consider the above example, rewritten as a multi-statement transaction. Setting the transaction mode to update removes the need for 'fake' code to force classification of statements as updates, but adds a requirement to call xdmp:commit to make the updates visible in the database.

declare option xdmp:transaction-mode "update";

(: doc("/docs/test.xml") returns <a>hello</a> before running this :)
xdmp:eval("xdmp:node-replace(doc('/docs/test.xml')/node(), <b>goodbye</b>)", (),
          <options xmlns="xdmp:eval">
            <isolation>different-transaction</isolation>
            <prevent-deadlocks>false</prevent-deadlocks>
          </options>);
(: returns <a>goodbye</b> within this transaction :)
doc("/docs/test.xml"),
(: make updates visible in the database :)
xdmp:commit()

Running Multi-Statement Transactions under xdmp:eval/invoke

When you run a query using xdmp:eval or xdmp:invoke with different-transaction isolation, or via xdmp:spawn, a new transaction is created to execute the query, and that transaction runs in a newly created session. This has two important implications for multi-statement transactions evaluated with xdmp:eval or xdmp:invoke:

  • Transaction mode is not inherited from the caller.
  • Uncommitted updates are automatically rolled back when an eval/invoke'd or spawned query completes.

Therefore, when using multi-statement transactions in code evaluated under xdmp:eval/invoke with different-transaction isolation or under xdmp:spawn,

  • Set the transaction mode explicitly inside the eval/invoke'd query or set transaction-mode in the options node if the transaction should run as a multi-statement transaction.
  • Always call xdmp:commit inside an eval/invoke'd multi-statement query if updates should be preserved.

Setting the transaction mode explicitly in the prolog of the eval/invoke'd query is equivalent to setting it by passing an options node to xdmp:eval/invoke with transaction-mode set. Setting the mode through the options mode enables you to set the transaction mode without modifying the eval/invoke'd query.

For an example of using multi-statement transactions with different-transaction isolation, see Example: Multi-Statement Transactions and Different-transaction Isolation.

The same considerations apply to multi-statement queries evaluated using xdmp:spawn.

Transactions run under same-statement isolation run in the caller's context, and so use the same transaction mode and benefit from committing the caller's transaction. For a detailed example, see Example: Multi-statement Transactions and Same-statement Isolation.

Functions With Non-Transactional Side Effects

Update transactions use various update built-in functions which, at the time the transaction commits, update documents in a database. These updates are technically known as side effects, because they cause a change to happen outside of what the statements in the transaction return. The side effects from the update built-in functions (xdmp:node-replace, xdmp:document-insert, and so on) are transactional in nature; that is, they either complete fully or are rolled back to the state at the beginning of the update statement.

There are several functions that evaluate asynchronously as soon as they are called, whether called from an update transaction or a query transaction. Examples of these functions are xdmp:spawn, xdmp:http-get, and xdmp:log. These functions have side effects outside the scope of the calling statement or the containing transaction (non-transactional side effects).

When evaluating an XQuery module that performs an update transaction, it is possible for the update to either fail or retry. That is the normal, transactional behavior, and the database will always be left in a consistent state if a transaction fails or retries. However, if your update transaction calls a function with non-transactional side effects, that function evaluates even if the calling update transaction fails and rolls back.

Use care or avoid calling any of these functions from an update transaction, as they are not guaranteed to only evaluate once (or to not evaluate if the transaction rolls back). If you are logging some information with xdmp:log in your transaction, it might or might not be appropriate for that logging to occur on retries (for example, if the transaction is retried because a deadlock is detected). Even if it is not what you intended, it might not do any harm.

Other side effects, however, can cause problems in updates. For example, if you use xdmp:spawn in this context, the action might be spawned multiple times if the calling transaction retries, or the action might be spawned even if the transaction fails; the xdmp:spawn call evaluates asyncronously as soon as it is called. Similarly, if you are calling a web service with xdmp:http-get from an update transaction, it might evaluate when you did not mean for it to evaluate. If you do use these functions in updates, your application logic must handle the side effects appropriately. These types of use cases are usually better suited to triggers and the Content Processing Framework. For details, see Using Triggers to Spawn Actions and the Content Processing Framework Guide manual.

Reducing Blocking with Multi-Version Concurrency Control

You can set the 'multi-version concurrency control' App Server configuration parameter to nonblocking to minimize transaction blocking, at the cost of queries potentially seeing a less timely view of the database. This option controls how the timestamp is chosen for lock-free queries. For details on how timestamps affect queries, see Query Transactions: Point-in-Time Evaluation.

Nonblocking mode can be useful for your application if:

  • Low query latency is more important than update latency.
  • Your application participates in XA transactions. XA transactions can involve multiple participants and non-MarkLogic Server resources, so they can take longer than usual.
  • Your application accesses a replica database which is expected to significantly lag the master. For example, if the master becomes unreachable for some time.

The default multi-version concurrency control is contemporaneous. In this mode, MarkLogic Server chooses the most recent timestamp for which any transaction is known to have committed, even if other transactions have not yet fully committed for that timestamp. Queries can block waiting for the contemporaneous transactions to fully commit, but the queries will see the most timely results. The block time is determined by the slowest contemporaneous transaction.

In nonblocking mode, the server chooses the latest timestamp for which all transactions are known to have comitted, even if there is a slightly later timestamp for which another transaction has committed. In this mode, queries do not block waiting for contemporaneous transactions, but they might not see the most up to date results.

You can run App Servers with different multi-version concurrency control settings against the same database.

Administering Transactions

The MarkLogic Server XQuery API include built-in functions helpful for debugging, monitoring, and administering transactions.

Use xdmp:host-status to get information about running transactions. The status information includes a <transactions> element containing detailed information about every running transaction on the host. For example:

<transactions xmlns="http://marklogic.com/xdmp/status/host">
  <transaction>
    <transaction-id>10030469206159559155</transaction-id>
    <host-id>8714831694278508064</host-id>
    <server-id>4212772872039365946</server-id>
    <name/>
    <mode>query</mode>
    <timestamp>11104</timestamp>
    <state>active</state>
    <database>10828608339032734479</database>
    <canceled>false</canceled>
    <start-time>2011-05-03T09:14:11-07:00</start-time>
    <time-limit>600</time-limit>
    <max-time-limit>3600</max-time-limit>
    <user>15301418647844759556</user>
    <admin>true</admin>
  </transaction>
  ...
</transactions>

In a clustered installation, transactions might run on remote hosts. If a remote transaction does not terminate normally, it can be committed or rolled back remotely using xdmp:transaction-commit or xdmp:transaction-rollback. These functions are equivalent to calling xdmp:commit and xdmp:rollback when xdmp:host is passed as the host id parameter. You can also rollback a transaction through the Host Status page of the Admin Interface. For details, see Rolling Back a Transaction in the Administrator's Guide.

Though a call to xdmp:transaction-commit returns immediately, the commit only occurs after the currently executing statement in the target transaction succesfully completes. Calling xdmp:transaction-rollback immediately interrupts the currently executing statement in the target transaction and terminates the transaction.

For an example of using these features, see Example: Generating a Transaction Report With xdmp:host-status. For details on the built-ins, see the XQuery & XSLT API Reference.

Transaction Examples

This section includes the following examples:

Example: Incrementing the System Timestamp

The following example shows how updates to the system increment the system timestamp. For background on system timestamps, see System Timestamps and Fragment Versioning.

<results>{
  <start-of-query>{xdmp:request-timestamp()}</start-of-query>,
for $x in (1 to 10)
return
(xdmp:eval("xdmp:document-insert('test.xml', <test/>)"),
 xdmp:eval("xdmp:document-delete('test.xml')" ) ),
<end-of-query>{xdmp:eval('xdmp:request-timestamp()')}</end-of-query>,
  <number-of-transactions>{xdmp:eval('xdmp:request-timestamp()') -
  xdmp:request-timestamp()}</number-of-transactions>,
<elapsed-time>{xdmp:query-meters()/*:elapsed-time/text()}
  </elapsed-time>
}
</results>

This XQuery program executes as a query statement, which runs at a timestamp. Each of the xdmp:eval statements is evaluated as a separate transaction, and they are each update statements. The query starts off by printing out the system timestamp, then it starts a FLWOR expression which evaluates 10 times. Each iteration of the FLWOR creates a document called test.xml in a separate transaction, then deletes the document in another transaction. Finally, it calculates the number of transactions that have occurred since the query started by starting a new transaction to get the current system timestamp and subtracting that from the timestamp in which the query runs. Therefore, this query produces the following results, which show that 20 transactions have occurred.

<results>
  <start-of-query>382638</start-of-query>
  <end-of-query>382658</end-of-query>
  <number-of-transactions>20</number-of-transactions>
  <elapsed-time>PT0.102211S</elapsed-time>
</results>

This example executes 20 update statements with xdmp:eval, each of which starts its own transaction, and each of which increments the system timestamp by 1. If there are other transactions happening concurrently on the system while these transactions are executing, those transactions might also increment the system timestamp, and it is possible that the last xdmp:eval statement which executes the xdmp:request-timestamp function might return a number more that 20 greater than the first call to xdmp:request-timestamp.

Example: Multi-statement Transactions and Same-statement Isolation

The following example demonstrates the interactions between multi-statement transactions and same-statement isolation, discussed in Interactions with xdmp:eval/invoke.

The goal of the sample is to insert a document in the database using xdmp:eval, and then examine and modify the results in the calling module. The inserted document should be visible to the calling module immediately, but not visible outside the module until transaction completion.

xquery version "1.0-ml";
declare option xdmp:transaction-mode "update";

(: insert a document in the database :)
let $query :=
  'xquery version "1.0-ml";
   xdmp:document-insert("/examples/mst.xml", <myData/>)
  '
return xdmp:eval(
  $query, (),
  <options xmlns="xdmp:eval">
    <isolation>same-statement</isolation>
  </options>);

(: demonstrate that it is visible to this transaction :)
if (fn:empty(fn:doc("/examples/mst.xml")//myData))
then ("NOT VISIBLE")
else ("VISIBLE");

(: modify the contents before making it visible in the database :)
xdmp:node-insert-child(doc('/examples/mst.xml')/myData, <child/>),
xdmp:commit()

(: result: VISIBLE :)

The same operation (inserting and then modifying a document before making it visible in the database) cannot be performed as readily using the default transaction model. If the module attempts the document insert and child insert in the same single-statement transaction, an XDMP-CONFLICTINGUPDATES error occurs. Performing these two operations in different single-statement transactions makes the inserted document immediately visible in the database, prior to inserting the child node. Attempting to perform the child insert using a pre-commit trigger creates a trigger storm, as described in Avoiding Infinite Trigger Loops (Trigger Storms).

The eval'd query runs as part of the calling module's multi-statement update transaction since the eval uses same-statement isolation. Since transaction mode is not inherited by transactions created in a different context, using different-transaction isolation would evaluate the eval'd query as a single-statement transaction, causing the document to be immediately visible to other transactions.

The call to xdmp:commit is required to preserve the updates performed by the module. If xdmp:commit is omitted, all updates are lost when evaluation reaches the end of the module. In this example, the commit must happen in the calling module, not in the eval'd query. If the xdmp:commit occurs in the eval'd query, the transaction completes when the statement containing the xdmp:eval call completes, making the document visible in the database prior to inserting the child node.

Example: Multi-Statement Transactions and Different-transaction Isolation

The following example demonstrates how different-transaction isolation interacts with transaction mode for multi-statement transactons. The same interactions apply to queries executed with xdmp:spawn. For more background, see Transaction Mode and Interactions with xdmp:eval/invoke.

In this example, xdmp:eval is used to create a new transaction that inserts a document whose content includes the current transaction id using xdmp:transaction. The calling query prints its own transaction id and the transaction id from the eval'd query.

xquery version "1.0-ml";

(: init to clean state; runs as single-statement txn :)
xdmp:document-delete("/docs/mst.xml");

(: switch to multi-statement transactions :)
declare option xdmp:transaction-mode "query";

let $sub-query :=
  'xquery version "1.0-ml";
   declare option xdmp:transaction-mode "update";       (: 1 :)
   xdmp:document-insert("/docs/mst.xml", <myData/>);
   
   xdmp:node-insert-child(
     fn:doc("/docs/mst.xml")/myData,
     <child>{xdmp:transaction()}</child>
   );
   xdmp:commit()                                        (: 2 :)
  '
return xdmp:eval($sub-query, (),
  <options xmlns="xdmp:eval">
    <isolation>different-transaction</isolation>
  </options>);
  
(: commit to end this transaction and get a new system
 : timestamp so the updates by the eval'd query are visible. :)
xdmp:commit();                                                (: 3 :)
(: print out my transaction id and the eval'd query transaction id :)
fn:concat("My txn id: ", xdmp:transaction()),           (: 4 :)
fn:concat("Subquery txn id: ", fn:doc("/docs/mst.xml")//child)

Setting the transaction mode in statement (: 1 :) is required because different-transaction isolation makes the eval'd query a new transaction, running in its own session, so it does not inherit the transaction mode of the calling context. Omitting xdmp:transaction-mode in the eval'd query causes the eval'd query to run in the default, auto, transaction mode.

The call to xdmp:commit at statement (: 2 :)is similarly required due to different-transaction isolation. The new transaction and the containing session end when the end of the eval'd query is reached. Changes are implicitly rolled back if a transaction or the containing session end without committing.

The xdmp:commit call at statement (: 3 :) ends the multi-statement query transaction that called xdmp:eval and starts a new transaction for printing out the results. This causes the final transaction at statement (: 4 :) to run at a new timestamp, so it sees the document inserted by xdmp:eval. Since the system timestamp is fixed at the beginning of the transaction, omitting this commit means the inserted document is not visible. For more details, see Query Transactions: Point-in-Time Evaluation.

If the query calling xdmp:eval is an update transaction instead of a query transaction, the xdmp:commit at statement (: 3 :) can be omitted. An update transaction sees the latest version of a document at the time the document is first accessed by the transaction. Since the example document is not accessed until after the xdmp:eval call, running the example as an update transaction sees the updates from the eval'd query. For more details, see Update Transactions: Readers/Writers Locks.

Example: Generating a Transaction Report With xdmp:host-status

Use the built-in xdmp:host-status to generate a list of the transactions running on a host, similar to the information available through the Host Status page of the Admin Interface.

This example generates a simple HTML report of the duration of all transactions on the local host:

xquery version "1.0-ml";

declare namespace html = "http://www.w3.org/1999/xhtml";
declare namespace hs="http://marklogic.com/xdmp/status/host";

<html>
  <body>
    <h2>Running Transaction Report for {xdmp:host-name()}</h2>
    <table border="1" cellpadding="5">
    <tr>
      <th>Transaction Id</th>
      <th>Database</th><th>State</th>
      <th>Duration</th>
    </tr>
    {
      let $txns:= xdmp:host-status(xdmp:host())//hs:transaction
      let $now := fn:current-dateTime()
      for $t in $txns
      return
        <tr>
          <td>{$t/hs:transaction-id}</td>
          <td>{xdmp:database-name($t/hs:database-id)}</td>
          <td>{$t/hs:transaction-state}</td>
          <td>{$now - $t/hs:start-time}</td>
        </tr>
    }
    </table>
  </body>
</html>

If you paste the above query into Query Console and run it with HTML output, the query generates a report similar to the following:

Many details about each transaction are available in the xdmp:host-status report. For more information, see xdmp:host-status in the XQuery & XSLT API Reference.

If we assume the first transaction in the report represents a deadlock, we can manually cancel it by calling xdmp:transaction-rollback and supplying the transaction id. For example:

xquery version "1.0-ml";
xdmp:transaction-rollback(xdmp:host(), 6335215186646946533)

You can also rollback transactions from the Host Status page of the Admin Interface.

« Previous chapter
Next chapter »