MarkLogic Server 11.0 Product Documentation

Search Developer's Guide — Chapter 22

The collection() Function

The collection() function can be used anywhere in your XQuery that the doc() or input() functions are used. The collection() function has the following signature:

fn:collection($URI as xs:string*) as node*

The MarkLogic Server implementation of the collection() function takes a sequence of URIs, so you can call the collection() function on one or more collections. The signature of the function in the W3C XQuery documentation only takes a single string. Also, the fn namespace is built-in to MarkLogic Server, so it is not necessary to prefix the function with its namespace.

To illustrate what the collection() function is used for, consider the following two XPath expressions:

fn:doc()//sonnet/line[cts:contains(., "flower")]
collection("english-lit/shakespeare")//sonnet/
                                    line[cts:contains(., "flower")]

The first expression returns a sequence of line nodes, each of which must be the child of a sonnet node, and each of which must contain the term flower, matched on a case-insensitive basis.

The second expression returns the same sequence, except that only line nodes contained within documents that are members of the english-lit/shakespeare collection. MarkLogic Server optimizes this expression. The operation that uses the collection() function, along with the rest of the XPath expression, is executed very efficiently through a series of index lookups.

As mentioned previously, the collection() function accepts either a single collection, as illustrated above, or a sequence of collections, as illustrated below:

collection(("english-lit/shakespeare", 
            "american-lit/poetry"))//sonnet/
                             line[cts:contains(., "flower")]

The query above returns a sequence of line nodes that match the stated predicates that are members of either the english-lit/shakespeare collection or the american-lit/poetry collection or both. With this modification to the collection() function, its format now closely matches the format of the doc() function, which also takes a sequence of URIs. While there is currently no XPath-level support for more complex boolean membership conditions (for example, requiring membership in multiple collections (and), excluding documents that belong to certain collections (not) or requiring pure either-or membership (exclusive or)), you can achieve these conditions through the where clause in a surrounding FLWOR expression (see Collection Membership for an example).

Collections Versus Directories

Collections are used to organize documents in a database. You can also use directories to organize documents in a database. The key differences in using collections to organize documents versus using directories are:

• Collections do not require member documents to conform to any URI patterns. They are not hierarchical; directories are. Any document can belong to any collection, and any document can also belong to multiple collections.
• You can delete all documents in a collection with the xdmp:collection-delete function. Similarly, you can delete all documents in a directory (as well as all recursive subdirectories and any documents in those directories) with the xdmp:directory-delete function.
• You cannot set properties on a collection; you can on a directory.

Except for the fact that you can use both collections and directories to organize documents, collections are unrelated to directories. For details on directories, see Properties Documents and Directories in the Application Developer's Guide.

Defining Collections

Collection membership for a document is defined implicitly. Rather than describing collections top-down (that is, specifying the list of documents that belong to a given collection), MarkLogic Server determines membership in a bottoms-up fashion, by aggregating the set of documents that describe themselves as being a member of the collection. You can use MarkLogic Server's security scheme to manage policies around collection membership.

Collections are named using URIs. Any URI is a legal name for a collection. The URI must be unique within the set of collections (both protected and unprotected) in your database.

The URIs that are used to name collections serve only as identifiers to the server. In particular, collections are not modeled on filesystem directories. Rather, collections are interpreted as sets, not as hierarchies. A document that belongs to collection english-lit/poetry/sonnets need not belong to collection english-lit/poetry. In fact, the existence of a collection with URI english-lit/poetry/sonnets does not imply the existence of collections with URI english-lit/poetry or URI english-lit.

There are two types of collections supported by MarkLogic Server: unprotected collections and protected collections. The two types are identical in terms of the syntactic application of the collection() function. However, differences emerge in the way they are defined, in who can access the collections, and in who can modify, add or remove documents from them. The following subsections descripe these two ways of defining collection:

• Implicitly Defining Unprotected Collections
• Explicitly Defining Protected Collections

Collection Membership

As described above, the collections (unprotected and protected) to which a specific document belongs can be specified at load-time and can be modified once the document has been loaded into the database. Documents can belong to many collections simultaneously.

If specific collections are not defined at load-time, the server will automatically assign collection membership for the document based on both the user's and the user's aggregate roles' default collection membership settings. To load a document that does not belong to any collections, explicitly specify the empty sequence as the collections parameter.

Collection membership can be leveraged in any XPath expression that the collection(), doc(), or input() functions are used. In addition, collection membership for a particular document or node can be queried using the xdmp:document-get-collections built-in.

For example, the following expression returns a sequence of line nodes, each of which must be the child of a sonnet node, and each of which must contain the term flower, matched on a case-insensitive basis, that belong to either the english-lit/shakespeare collection or the american-lit/poetry collection or both:

collection(("english-lit/shakespeare", 
            "american-lit/poetry"))//sonnet/
                                line[cts:contains(., "flower")]

By contrast, the following expression returns a similar sequence of line nodes, except that the resulting nodes must belong to either the english-lit/poetry collection or the american-lit/poetry collection or both, but not to the english-lit/shakespeare collection:

for $line in collection(("english-lit/poetry", "american-lit/
     poetry"))//sonnet/line[cts:contains(., "flower")]
where xdmp:document-get-collections($line) != 
     "english-lit/shakespeare"
return $line

Collections and Security

Collections interact with the MarkLogic Server security model in three basic ways:

• All users and roles can optionally specify default collections. These are the collections to which newly inserted documents are added if collections are not explicitly specified at load-time.
• Adding a document to a collection--both at load-time and after the document has been loaded into the database--is contingent on the user possessing permissions to insert or update the document in question.
• Removing a document from a collection and using xdmp:collection-delete are similarly contingent on the user's having appropriate permissions to update the document(s) in question.

Protected collections interact with the MarkLogic Server security model in three additional ways:

• Protected collections can be configured using the security module of the Admin Interface or by means of the POST /manage/v2/protected-collections REST endpoint.
• Protected collections specify the roles that have read, insert and/or update permissions for the protected collection.
• Collection permissions control who can add documents to a protected collection, but they do not provide document access control. You must use document permissions to control document access. For example, a user with read permissions on a document in a protected collection can read the document whether or not they have any permissions on the collection.
• You can only add a document to a protected collection if you have insert or update permissions on the collection, as well as appropriate document permissions.

Performance Characteristics

MarkLogic's implementation of collections is designed to optimize query performance against large volumes of documents. As with all designs, the implementation involves some trade-offs. This section provides a brief overview of the performance characteristics of collections and includes the following subsections:

• Number of Collections to Which a Document Belongs
• Adding/Removing Existing Documents To/From Collections