This page was generated
January  28,  2016
4:06  AM
XQuery & XSLT Built-In & Modules Function Reference

Module: TieredStorage

This module provides functions for managing forests and partitions in storage tiers and for managing super-databases and sub-databases. For details, see 'Tiered Storage' in the Administrator's Guide.

The Tiered Storage function module is installed as the following file:

  • install_dir/Modules/MarkLogic/tieredstorage.xqy

where install_dir is the directory in which MarkLogic Server is installed.

To use the tieredstorage.xqy module in your own XQuery modules, include the following line in your XQuery prolog:

import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
          at "/MarkLogic/tieredstorage.xqy";
Function Summary
tieredstorage:database-create-sub-database This function creates a database with the name $database-name and make the created database the sub database of the database designated by $database-id.
tieredstorage:database-create-super-database This function creates a database with the name $database-name and make the created database the super database of the database designated by $database-id.
tieredstorage:database-delete-sub-database This function disassociates the sub database designated by $delete-dbid with the super-database designated by $database-id and deletes the sub database.
tieredstorage:database-delete-super-database This function disassociates the super database designated by $delete-dbid with the sub-database designated by $database-id and deletes the super database.
tieredstorage:database-partitions This function returns the names of the partitions in the named database.
tieredstorage:forest-combine This function combines data in multiple forests into one new forest.
tieredstorage:forest-migrate This function moves data in a forest to new data directories and optionally sets new host for the forest.
tieredstorage:partition-create This function creates forests on the specified hosts and forms a partition with the specified partition range.
tieredstorage:partition-delete This function deletes all forests in the specified partition.
tieredstorage:partition-forests This function returns a sequence of ids for the forests in the named partition.
tieredstorage:partition-migrate This function migrates all forests in a partition to the specified data directory and hosts.
tieredstorage:partition-resize This function either creates or combines forests in a partition.
tieredstorage:partition-set-availability This function sets the availability of the partition to either online or offline.
tieredstorage:partition-set-updates-allowed This function sets update-allowed state for the forests in the specified partition.
tieredstorage:partition-transfer This function detaches forests in a partition from one database and attaches them to another.
Function Detail
tieredstorage:database-create-sub-database(
$database-id as xs:unsignedLong*,
$database-name as xs:string,
[$kind as xs:string]
)  as   empty-sequence()
Summary:

This function creates a database with the name $database-name and make the created database the sub database of the database designated by $database-id. The newly created database inherits the configuration of the original database. For details, see 'Creating Sub-databases' in the Administrator's Guide.

Parameters:
$database-id : ID of the database as super-database of the one to be created.
$database-name : Name of the new database.
$kind (optional): active or archive, where active is the default. If $kind is archive, the newly created database will have the following configuration set automatically: index detection none, expunge locks none, reindexer enable false, rebalancer enable false, format compatibility 5.0.

Example:
xquery version "1.0-ml";
import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
          at "/MarkLogic/tieredstorage.xqy";

ts:database-create-sub-database(xdmp:database("Documents"),"subdb")
=>
()
  

tieredstorage:database-create-super-database(
$database-id as xs:unsignedLong,
$database-name as xs:string
)  as   empty-sequence()
Summary:

This function creates a database with the name $database-name and make the created database the super database of the database designated by $database-id. The newly created database inherits the configuration of the original database. For details, see 'Creating Super-databases' in the Administrator's Guide.

Parameters:
$database-id : ID of the database as sub-database of the one to be created.
$database-name : Name of the new database.

Example:
xquery version "1.0-ml";
import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
          at "/MarkLogic/tieredstorage.xqy";

ts:database-create-super-database(xdmp:database("Documents"),"superdb")
=>
()
  

tieredstorage:database-delete-sub-database(
$database-id as xs:unsignedLong,
$delete-dbid as xs:unsignedLong
)  as   empty-sequence()
Summary:

This function disassociates the sub database designated by $delete-dbid with the super-database designated by $database-id and deletes the sub database. An error is thrown if the database designated by $delete-dbid is not a sub database of the one designated by $database-id or contains other super-databases than the one designated by $database-id. For details, see 'Tiered Storage' in the Administrator's Guide.

Parameters:
$database-id : ID of the database as super-database of the one to be deleted
$delete-dbid : Name of the database to be deleted.

Example:
xquery version "1.0-ml";
import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
          at "/MarkLogic/tieredstorage.xqy";

ts:database-delete-sub-database(xdmp:database("Documents"),xdmp:database())
=>
()
  

tieredstorage:database-delete-super-database(
$database-id as xs:unsignedLong,
$delete-dbid as xs:unsignedLong
)  as   empty-sequence()
Summary:

This function disassociates the super database designated by $delete-dbid with the sub-database designated by $database-id and deletes the super database. An error is thrown if the database designated by $delete-dbid is not a super database of the one designated by $database-id or contains other sub-databases than the one designated by $database-id. For details, see 'Tiered Storage' in the Administrator's Guide.

Parameters:
$database-id : ID of the database as sub-database of the one to be deleted
$delete-dbid : The ID of the database to be deleted.

Example:
xquery version "1.0-ml";
import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
          at "/MarkLogic/tieredstorage.xqy";

ts:database-delete-super-database(xdmp:database("Documents"),xdmp:database())
=>
()
  

tieredstorage:database-partitions(
$database-id as xs:unsignedLong
)  as   xs:string*
Summary:

This function returns the names of the partitions in the named database. For details, see 'Tiered Storage' in the Administrator's Guide.

Parameters:
$database-id : ID of the database in containing the partitions.

Example:
xquery version "1.0-ml";
import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
          at "/MarkLogic/tieredstorage.xqy";

ts:database-partitions(xdmp:database("Documents"))

(: Returns the names of the partitions in the Documents database. :)
  

tieredstorage:forest-combine(
$forest-ids as xs:unsignedLong*,
$forest-name as xs:string,
$host-id as xs:unsignedLong,
[$data-directory as xs:string?],
[$large-data-directory as xs:string?],
[$fast-data-directory as xs:string?],
[$options as xs:string*]
)  as   empty-sequence()
Summary:

This function combines data in multiple forests into one new forest. If no data directory is specified, the default data directory is used. The source forests are required to be either all in "open" or "open-replica" state or all in "sync replicating" state. If a forest's state changes during the combine operation, an error may be thrown. If all source forests are in "open" or "open replica" state and are all attached to a database, the combined forest will be attached to the database and the source forests (or their masters) detached to the database. For details, see 'Combining Forests' in the Administrator's Guide.

Parameters:
$forest-ids : IDs of the forests to be combined.
$forest-name : Name of the combined forest.
$host-id : ID of the destination host.
$data-directory (optional): The optional data directory of the forest. If no directory is specified, then it will be a private forest.
$large-data-directory (optional): The optional directory where large objects are stored. If no directory is specified, then large objects will be stored in DATA_DIR/Forests/FOREST_NAME/Large.
$fast-data-directory (optional): The optional directory for the fast data directory for this forest. The fast data directory is typically located on a fast storage device, such as an SSD.
$options (optional): Valid option values include:
timeout=N
Specifies the number of seconds to wait for a forest to re-open after a change requiring the forest to close and re-mount is made during the migration operation.
local-to-local
Indicates that the migration is between local storage to local storage. This is the default when no other direction parameter is specified and the type of storage cannot be derived from either the data directory path.
local-to-shared
Indicates that the migration is between local storage and shared storage. This type of migration supports changing the host.
shared-to-local
Indicates that the migration is to move data from shared storage to local storage. This type of migration supports changing the host.
shared-to-shared
Indicates that the migration is to move data from shared storage to shared storage. This type of migration supports changing the host.
keep-source-forests
Specifies to keep source forests after the combined forest is created. By default, the source forests are removed before the combine function returns. The source forests will not be removed when this option is turned on. This option can be used to combine forests associated with other forests as masters or replicas.
snapshot
Specifies to create a combined forest based on a consistent snapshot of the source forests. The source forests will not be removed when this option is turned on. This option can be used to combine replica forests.

Usage Notes:

If the forests reside in a database configured for database replication, you must specify the snapshot option when combining forests with overrides. Otherwise an TS-FORESTHASOVERRIDE exception will be thrown.

Example:
xquery version "1.0-ml";
import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
          at "/MarkLogic/tieredstorage.xqy";

ts:forest-combine(
   (xdmp:forest("Forest1"), xdmp:forest("Forest2")),
   "ForestCombined",
   xdmp:host(), 
   "/tmp/test")
 
(: Combines Forest1 and Forest2 into a single forest, named ForestCombined, in
   the /tmp/test data directory. :)
  

tieredstorage:forest-migrate(
$forest-ids as xs:unsignedLong*,
[$new-host-id as xs:unsignedLong?],
[$new-data-directory as xs:string?],
[$new-large-data-directory as xs:string?],
[$new-fast-data-directory as xs:string?],
[$options as xs:string*]
)  as   empty-sequence()
Summary:

This function moves data in a forest to new data directories and optionally sets new host for the forest. If no data directory is specified, the default data directory will be used. For details, see 'Migrating Forests and Partitions' in the Administrator's Guide.

Parameters:
$forest-ids : IDs of the forests to migrate.
$new-host-id (optional): ID of the destination host.
$new-data-directory (optional): The optional data directory of the forest. If no directory is specified, then it will be a private forest.
$new-large-data-directory (optional): The optional directory where large objects are stored. If no directory is specified, then large objects will be stored in DATA_DIR/Forests/FOREST_NAME/Large.
$new-fast-data-directory (optional): The optional directory for the fast data directory for this forest. The fast data directory is typically located on a fast storage device, such as an SSD.
$options (optional): Valid option values include:
timeout=N
Specifies the number of seconds to wait for a forest to re-open after a configuration is made during the migration operation.
local-to-local
Indicates that the migration is between local storage to local storage. This is the default when no other direction parameter is specified and the type of storage cannot be derived from either the data directory path.
local-to-shared
Indicates that the migration is between local storage and shared storage. This type of migration supports changing the host.
shared-to-local
Indicates that the migration is to move data from shared storage to local storage. This type of migration supports changing the host.
shared-to-shared
Indicates that the migration is to move data from shared storage to shared storage. This type of migration supports changing the host.

Example:
xquery version "1.0-ml";
import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
          at "/MarkLogic/tieredstorage.xqy";

ts:forest-migrate(
    xdmp:forest("Forest1"), 
    xdmp:host("host1"), 
    "/tmp/test", 
    (), 
    (), 
    ("timeout=20"))
 
(: Moves the Forest1 forest from their current location to 
   /tmp/test and sets their timeouts to 20 seconds. )
  

tieredstorage:partition-create(
$database-id as xs:unsignedLong,
$partition-name as xs:string,
$lower as xs:anyAtomicType?,
$upper as xs:anyAtomicType?,
$forests-per-host as xs:unsignedInt,
$host-ids as xs:unsignedLong*,
[$data-directory as xs:string?],
[$large-data-directory as xs:string?],
[$fast-data-directory as xs:string?],
[$options as xs:string*]
)  as   empty-sequence()
Summary:

This function creates forests on the specified hosts and forms a partition with the specified partition range. All of the forests will share the partition name as their prefix, as well as the range policy settings. The names of the created forests will either be in the form of partition- name-[fixed-digit-number] or partition-name-[fixed-digit-number]-[M,Rn], if local disk failover is configured. For details, see 'Creating Partitions' in the Administrator's Guide.

Parameters:
$database-id : ID of the database in which the partition will reside.
$partition-name : Name of the partition.
$lower : Lower bound of partition range.
$upper : Upper bound of partition range.
$forests-per-host : Number of forests per host in the partition to be created.
$host-ids : Ids of hosts on which to create the partition forests.
$data-directory (optional): The optional data directory of the forests. If no directory is specified, then the forests will be private.
$large-data-directory (optional): The optional directory where large objects are stored. If no directory is specified, then large objects will be stored in DATA_DIR/Forests/FOREST_NAME/Large.
$fast-data-directory (optional): The optional directory for the fast data directory for the forests. The fast data directory is typically located on a fast storage device, such as an SSD.
$options (optional): Valid option values include:
failover =[none, local, shared]
If failover is local and multiple hosts are specified, new replica(s) will be created.

Shared disk failover will be set up if failover is shared and multiple hosts are specified. The failover hosts will be chosen to cover as many different zones as possible.

If failover is none, neither will be set up.

If failover is left unspecified, it will be local unless data directories are on shared platforms such as HDFS and S3.

replicas=N
Specifies the number of replicas/failover forests to configure for local-disk or shared-disk failover.

Example:
xquery version "1.0-ml";
import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
          at "/MarkLogic/tieredstorage.xqy";

ts:partition-create(
xdmp:database("Documents"),
"2012",
xs:date("2012-01-01"),xs:date("2012-12-31"), 
2, 
(xdmp:host("host-1.marklogic.com"), xdmp:host("host-2.marklogic.com")), 
"/forest", 
(), 
(), 
("failover=local","replicas=1"))

(: Creates a partition, named 2012, on the Documents database with a date range of 1/1/2012
   through 1/31/2012. The partition consists of four master forests and four replica forests
   distributed across two hosts and configured for local disk failover. :)
  

tieredstorage:partition-delete(
$database-id as xs:unsignedLong,
$partition-name as xs:string,
[$delete-data as xs:boolean]
)  as   empty-sequence()
Summary:

This function deletes all forests in the specified partition. For details, see 'Deleting Partitions' in the Administrator's Guide.

Parameters:
$database-id : ID of the database in which the partition will reside.
$partition-name : Name of the partition.
$delete-data (optional): Whether or not data will be deleted.

Example:
xquery version "1.0-ml";
import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
          at "/MarkLogic/tieredstorage.xqy";

ts:partition-delete(xdmp:database("Documents"),"2012-April",xs:boolean("true"))
=>
()
  

tieredstorage:partition-forests(
$database-id as xs:unsignedLong,
$partition-name as xs:string,
[$include-replicas as xs:boolean]
)  as   xs:unsignedLong*
Summary:

This function returns a sequence of ids for the forests in the named partition. If the $include-replicas parameter is not specified or is set to false, replica forests are not included. For details, see 'Tiered Storage' in the Administrator's Guide.

Parameters:
$database-id : ID of the database in containing the partition.
$partition-name : Name of the partition.
$include-replicas (optional): Whether or not replicas will be included.

Example:
xquery version "1.0-ml";
import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
          at "/MarkLogic/tieredstorage.xqy";

ts:partition-forests(xdmp:database("Documents"),"2012-April")
=>
8708998620766669503 57548986207876743
  

tieredstorage:partition-migrate(
$database-id as xs:unsignedLong,
$partition-name as xs:string,
$host-ids as xs:unsignedLong*,
[$data-directory as xs:string?],
[$large-data-directory as xs:string?],
[$fast-data-directory as xs:string?],
[$options as xs:string*]
)  as   empty-sequence()
Summary:

This function migrates all forests in a partition to the specified data directory and hosts. For details, see 'Migrating Forests and Partitions' in the Administrator's Guide.

Parameters:
$database-id : ID of the database in which the partition will reside.
$partition-name : Name of the partition.
$host-ids : Ids of hosts on which the forests in the partition will be migrated to.
$data-directory (optional): The optional data directory of the forest. If no directory is specified, then it will be a private forest.
$large-data-directory (optional): The optional directory where large objects are stored. If no directory is specified, then large objects will be stored in DATA_DIR/Forests/FOREST_NAME/Large.
$fast-data-directory (optional): The optional directory for the fast data directory for this forest. The fast data directory is typically located on a fast storage device, such as an SSD.
$options (optional): Valid option values include:
timeout=N
Specifies the number of seconds to wait for a forest to re-open after a configuration is made during the migration operation.
local-to-local
Indicates that the migration is between local storage to local storage. This is the default when no other direction parameter is specified and the type of storage cannot be derived from either the data directory path.
local-to-shared
Indicates that the migration is between local storage and shared storage. This type of migration supports changing the host.
shared-to-local
Indicates that the migration is to move data from shared storage to local storage. This type of migration supports changing the host.
shared-to-shared
Indicates that the migration is to move data from shared storage to shared storage. This type of migration supports changing the host.
failover=[local, shared, none]
If failover is set to local and multiple hosts are specified, any existing replica forests will be migrated and new replica forests will be created, if no replica forests exist.

Shared disk failover will be set up if failover is specified to be shared and multiple hosts are specified. The failover hosts will be chosen to cover as many different zones as possible.

If failover is specified to be none, any existing replica forests will be deleted.

If failover is left unspecified, and if local-disk failover is configured, and if the data is being migrated to shared storage, the local disk replicas will be removed, and shared-disk failover will be configured. If shared-disk failover is configured and if the data is being migrated from shared storage to local storage, local- disk failover will replace shared-disk failover. In all other scenarios, the failover policy will be left unchanged when not specified.

replicas=N
Specifies the number of replicas/failover hosts to maintain if failover.

Example:
xquery version "1.0-ml";
import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
          at "/MarkLogic/tieredstorage.xqy";

ts:partition-migrate(xdmp:database("Documents"),"2012-April",xdmp:host(),"/space/ts/")
=>
()
  

tieredstorage:partition-resize(
$database-id as xs:unsignedLong,
$partition-name as xs:string,
$forests-per-host as xs:unsignedInt,
$host-ids as xs:unsignedLong*,
[$data-directory as xs:string?],
[$large-data-directory as xs:string?],
[$fast-data-directory as xs:string?],
[$options as xs:string*]
)  as   empty-sequence()
Summary:

This function either creates or combines forests in a partition. For details, see 'Resizing Partitions' in the Administrator's Guide.

Parameters:
$database-id : ID of the database in which the partition will reside.
$partition-name : Name of the partition.
$forests-per-host : Number of forests per host in the partition to be created.
$host-ids : Ids of hosts on which the forests in the partition are to be created on.
$data-directory (optional): The optional data directory of the forest. If no directory is specified, then it will be a private forest.
$large-data-directory (optional): The optional directory where large objects are stored. If no directory is specified, then large objects will be stored in DATA_DIR/Forests/FOREST_NAME/Large.
$fast-data-directory (optional): The optional directory for the fast data directory for this forest. The fast data directory is typically located on a fast storage device, such as an SSD.
$options (optional): Valid option values include:
timeout=N
Specifies the number of seconds to wait for a forest to re-open after a configuration is made during the migration operation.
local-to-local
Indicates that the partition is to be moved between local storage to local storage. This is the default when no other direction parameter is specified and the type of storage cannot be derived from either the data directory path.
local-to-shared
Indicates that the partition is to be moved between local storage and shared storage. This option supports changing the host.
shared-to-local
Indicates that the partition is to be moved from shared storage to local storage. This option supports changing the host.
shared-to-shared
Indicates that the partition is to be moved from shared storage to shared storage. This type option supports changing the host.
failover=[none, local, shared]
If failover is local and multiple hosts are specified, any existing replica forests will be migrated/combined and new replica forests will be created if no replica forests exist.

Shared disk failover will be set up if failover is shared and multiple hosts are specified. The failover hosts will be chosen to cover as many different zones as possible.

If failover is none, any existing replica forests will be deleted.

If failover is left unspecified, and if local-disk failover is configured, and the data is being migrated/combined to shared storage, the local disk replicas will be removed, and shared-disk failover will be configured. If shared-disk failover is configured, and the data is being migrated/combined from shared storage to local storage, local-disk failover is going to replace to shared-disk failover. In all other scenarios, the failover policy will be left unchanged when not specified.

replicas=N
Specifies the number of replicas/failover hosts to maintain if failover.

Usage Notes:

If the partition resides in a database configured for database replication, you cannot combine partitions in which forest overrides exist. An attempt to do so will result in a TS-FORESTHASOVERRIDE exception.

Example:
xquery version "1.0-ml";
import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
          at "/MarkLogic/tieredstorage.xqy";

ts:partition-resize(xdmp:database("Documents"),"2012-July",15,xdmp:host(),"/home/2012-July")
=>
()
  

tieredstorage:partition-set-availability(
$database-id as xs:unsignedLong,
$partition-name as xs:string,
$value as xs:string
)  as   empty-sequence()
Summary:

This function sets the availability of the partition to either online or offline. An online partition is fully accessable through queries when its forests are attached to a database. The forests in an offline partition, as well as any replica forests, cannot be accessed.

Parameters:
$database-id : ID of the database containing the partition.
$partition-name : Name of the partition.
$value : The availability of the partition. Values can be online or offline.

Example:

xquery version "1.0-ml"; 
 
import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
      at "/MarkLogic/tieredstorage.xqy";

ts:partition-set-availability(xdmp:database("Documents"),"2012-April", "offline")
 
(: Takes the 2012-April partition in the Documents database offline. :)
 
    

tieredstorage:partition-set-updates-allowed(
$database-id as xs:unsignedLong,
$partition-name as xs:string,
$value as xs:string
)  as   empty-sequence()
Summary:

This function sets update-allowed state for the forests in the specified partition. For details, see 'Tiered Storage' in the Administrator's Guide.

Parameters:
$database-id : ID of the database in which the partition will reside.
$partition-name : Name of the partition.
$value : The new updates-allowed state of the partition. Must be one of all for all updates allowed, delete-only for only deletes allowed, read-only for no updates allowed and to cause updating transactions to abort immediately, flash-backup for no updates allowed and to make updating transactions retry for a time period specified in the group's retry timeout. These are the same setting as used on the forests, as described in 'Forests' in the Administrator's Guide.

Example:
xquery version "1.0-ml";
import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
          at "/MarkLogic/tieredstorage.xqy";

ts:partition-set-updates-allowed(xdmp:database("Documents"), "2012-April", "all")
=>
()
  

tieredstorage:partition-transfer(
$partition-db-id as xs:unsignedLong,
$partition-name as xs:string,
$dst-db-id as xs:unsignedLong
)  as   empty-sequence()
Summary:

This function detaches forests in a partition from one database and attaches them to another. For details, see 'Transferring Partitions between Databases' in the Administrator's Guide.

Parameters:
$partition-db-id : ID of the database from which the partition will be moved.
$partition-name : Name of the partition.
$dst-db-id : ID of the database to which the partition will be moved.

Example:
xquery version "1.0-ml";
import module namespace ts="http://marklogic.com/xdmp/tieredstorage" 
          at "/MarkLogic/tieredstorage.xqy";


ts:partition-transfer(xdmp:database("srcDB"), "2012-April", xdmp:database("dstDB"))
=>
()