MarkLogic Server 11.0 Product Documentation

mlcp User Guide — Chapter 6

Basic Steps

To copy one database to another with mclp:

1. Set -input_host, -input_port, -input_username, and -input_password to identify the source MarkLogic Server instance and user.
2. Set -output_host, -output_port, -output_username, and -output_password to identify the destination MarkLogic Server instance and user.
3. Select what documents to copy. For details, see Filtering Archive and Copy Contents.
   • To select documents in one or more collections, set -collection_filter to a comma separated list of collection URIs.
   • To select documents in one or more database directories, set -directory_filter to a comma separated list of directory URIs.
   • To select documents matching an XPath expression, use -document_selector. To use namespace prefixes in the XPath expression, define the prefix binding using -path_namespace.
   • To select document matching a query, use -query_filter. You can use this option alone or in combination with a directory, collection or document selector filter. False positives are possible; for details, see Understanding When Filters Are Accurate.
   • To select all documents in the database, leave -collection_filter, -directory_filter, -document_selector, and -query_filter unset.
4. If you want to exclude some or all source document metadata:
   • Set -copy_collections to false to exclude document collections metadata.
   • Set -copy_permissions to false to exclude document permissions metadata.
   • Set -copy_properties to false to exclude document properties.
   • Set -copy_quality to false to exclude document quality metadata.
   • Set -copy_metadata to false to exclude document key-value metadata.
5. If you want to add or override document metadata in the destination database:
   • Set -output_collections to add destination documents to a collection.
   • Set -output_permissions to add permissions to destination documents.
   • Set -output_quality to set the quality of destination documents.
6. If you want the destination documents to have database URIs different from the source URIs, set -output_uri_replace, -output_uri_prefix, and/or -output_uri_suffix. For details, see Controlling Database URIs During Ingestion.

For a complete list of mlcp copy command options, see Copy Command Line Options.

Examples

The following example copies all documents and their metadata from the source database to the destination database:

# Windows users, see Modifying the Example Commands for Windows 
$ mlcp.sh copy -mode local -input_host srchost -input_port 8000 \
    -input_username user1 -input_password password1 \
    -output_host desthost -output_port 8010 -output_username user2 \
    -output_password password2

The following example copies selected documents, excluding the source permissions and adding the documents to 2 new collections in the destination database:

# Windows users, see Modifying the Example Commands for Windows 
$ mlcp.sh copy -mode local -input_host srchost -input_port 8000 \
    -input_username user1 -input_password password1 \
    -output_host desthost -output_port 8000 -output_username user2 \
    -output_password password2 -copy_permissions false \
    -output_collections shakespeare,plays

For an example of using -query_filter, see Example: Exporting Documents Matching a Query.

Redacting Content During a Copy

Redaction is the process of eliminating or obscuring portions of a document when retrieving the document from MarkLogic. For example, you can eliminate or mask sensitive personal information such as credit card numbers, phone numbers, or email addresses from documents. You can only redact document content, not document properties.

Redaction is performed as documents are read from the source database. For example, if you copy documents between databases in two different MarkLogic installations, the unredacted content never leaves the source installation.

Redaction support in MarkLogic is covered in detail in Redacting Content During Export or Copy Operations and Redacting Document Content in the Application Developer's Guide.

Use the -redaction option to apply redaction rules during a copy. For example, the following command copies documents in the my_docs collection from one database to another, and applies the redaction rules in the rule collections hipaa-rules and biz-rules to the source documents before copying them to the destination database.

# Windows users, see Modifying the Example Commands for Windows 
$ mlcp.sh copy -mode local -input_host srchost -input_port 8000 \
    -input_username user1 -input_password password1 \
    -output_host desthost -output_port 8000 -output_username user2 \
    -output_password password2 -collection_filter my_docs \
    -redaction "hipaa-rules,biz-rules"

For more details, see Redacting Content During Export or Copy Operations.

Copy Command Line Options

This section summarizes the command line options available with the mlcp copy command. The following command line options define your connection to MarkLogic:

-input_host comma-list

-input_port number

-input_username string

-input_password string

-output_host comma-list

-output_port number

-output_username string

-output_password string

The following table lists command line options that define the characteristics of the copy operation:

-batch_size number

-collection_filter comma-list

-copy_collections boolean

-copy_metadata boolean

-copy_permissions boolean

-copy_properties boolean

-copy_quality boolean

-directory_filter comma-list

-document_selector string

-fastload boolean

-input_database string

-input_ssl boolean

-input_ssl_protocol string

-max_split_size number

-mode string

-path_namespace comma-list

-options_file string

-output_collections comma-list

-output_database string

-output_permissions comma-list

-output_quality string

-output_partition string

-output_ssl boolean

-output_ssl_protocol string

-output_uri_prefix string

-output_uri_replace comma-list

-output_uri_suffix string

-query_filter string

-redaction comma-list

-restrict_input_hosts boolean

-restrict_output_hosts boolean

-snapshot boolean

-temporal_collection string

-thread_count number

-transform_function string

-transform_module string

-transform_namespace string

-transform_param string

-transaction_size number