com.marklogic.client.datamovement

Class ExportListener

java.lang.Object

com.marklogic.client.datamovement.ExportListener

All Implemented Interfaces:

BatchListener<QueryBatch>, QueryBatchListener

Direct Known Subclasses:

ExportToWriterListener

public class ExportListener
extends java.lang.Object
implements QueryBatchListener

Reads document contents (and optionally metadata) for each batch, then sends each document to any listeners registered with onDocumentReady for further processing or writing to any target supported by Java. Supports reading partial documents via transforms. Supports exporting all documents at a consistent point-in-time using withConsistentSnapshot.

For example:

     QueryBatcher exportBatcher = moveMgr.newQueryBatcher(query)
         .withConsistentSnapshot()
         .onUrisReady(
           new ExportListener()
             .withConsistentSnapshot()
             .onDocumentReady(doc -> {
               logger.debug("Contents=[{}]", doc.getContentAs(String.class));
             })
         )
         .onQueryFailure(exception -> exception.printStackTrace());

     JobTicket ticket = moveMgr.startJob(exportBatcher);
     exportBatcher.awaitCompletion();
     moveMgr.stopJob(ticket);

By default only document contents are retrieved. If you would also like metadata, make sure to call withMetadataCategory to configure which categories of metadata you desire.

As with all the provided listeners, this listener will not meet the needs of all applications but the source code for it should serve as helpful sample code so you can write your own custom listeners.

Constructor Summary

Constructors

Constructor and Description
ExportListener()

Method Summary

Modifier and Type	Method and Description
protected java.util.List<BatchFailureListener<QueryBatch>>	getBatchFailureListeners()
protected DocumentPage	getDocs(QueryBatch batch)
void	initializeListener(QueryBatcher queryBatcher) This implementation of initializeListener adds this instance of ExportListener to the two RetryListener's in this QueryBatcher so they will retry any batches that fail during the read request.
ExportListener	onDocumentPageReady(java.util.function.Consumer<DocumentPage> listener) Sets a listener to process a page of retrieved documents.
ExportListener	onDocumentReady(java.util.function.Consumer<DocumentRecord> listener) Adds a listener to process each retrieved document, which is the way users of ExportListener can provide custom code to export documents.
ExportListener	onFailure(BatchFailureListener<QueryBatch> listener) When a batch fails or a callback throws an Exception, run this listener code.
void	processEvent(QueryBatch batch) This is the method QueryBatcher calls for ExportListener to do its thing.
ExportListener	withConsistentSnapshot() Specifies that documents should be retrieved as they were when this QueryBatcher job started.
ExportListener	withMetadataCategory(DocumentManager.Metadata category) Adds a metadata category to retrieve and make available from DocumentRecord.getMetadata to listeners registered with onDocumentReady.
ExportListener	withNonDocumentFormat(Format nonDocumentFormat) Sets the format for metadata available from DocumentRecord.getMetadata to listeners registered with onDocumentReady (assuming withMetadataCategory has been called to request specific metdata).
ExportListener	withTransform(ServerTransform transform) Sets the server tranform to modify the document contents.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ExportListener

public ExportListener()

Method Detail

getDocs

protected DocumentPage getDocs(QueryBatch batch)

initializeListener

public void initializeListener(QueryBatcher queryBatcher)

This implementation of initializeListener adds this instance of ExportListener to the two RetryListener's in this QueryBatcher so they will retry any batches that fail during the read request.

Specified by:

initializeListener in interface QueryBatchListener

Parameters:

queryBatcher - the QueryBatcher which will call this Listener

processEvent

public void processEvent(QueryBatch batch)

This is the method QueryBatcher calls for ExportListener to do its thing. You should not need to call it.

Specified by:

processEvent in interface BatchListener<QueryBatch>

Specified by:

processEvent in interface QueryBatchListener

Parameters:

batch - the batch of uris and some metadata about the current status of the job

withConsistentSnapshot

public ExportListener withConsistentSnapshot()

Specifies that documents should be retrieved as they were when this QueryBatcher job started. This enables a point-in-time export so that all documents are as they were at that point in time. This requires that the server be configured to allow such queries by setting the [merge timestamp](https://docs.marklogic.com/guide/app-dev/point_in_time#id_32468) to a timestamp before the job starts or a sufficiently large negative value. This should only be used when the QueryBatcher is constructed with a query, not with an Iterator.

Returns:

this instance for method chaining

See Also:

QueryBatcher.withConsistentSnapshot()

withMetadataCategory

public ExportListener withMetadataCategory(DocumentManager.Metadata category)

Adds a metadata category to retrieve and make available from DocumentRecord.getMetadata to listeners registered with onDocumentReady. To specify the format for the metdata, call withNonDocumentFormat.

Parameters:

category - the metadata category to retrieve

Returns:

this instance for method chaining

See Also:

DocumentManager.setMetadataCategories

withNonDocumentFormat

public ExportListener withNonDocumentFormat(Format nonDocumentFormat)

Sets the format for metadata available from DocumentRecord.getMetadata to listeners registered with onDocumentReady (assuming withMetadataCategory has been called to request specific metdata). If setNonDocumentFormat is not called, the server default format will be used.

Parameters:

nonDocumentFormat - the format in which retrieve the metadata

Returns:

this instance for method chaining

See Also:

DocumentManager.setNonDocumentFormat

withTransform

public ExportListener withTransform(ServerTransform transform)

Sets the server tranform to modify the document contents.

Parameters:

transform - the name of the transform already installed in the REST server

Returns:

this instance for method chaining

onDocumentReady

public ExportListener onDocumentReady(java.util.function.Consumer<DocumentRecord> listener)

Adds a listener to process each retrieved document, which is the way users of ExportListener can provide custom code to export documents. This custom code could write the document or a portion of the document to the file system, a REST service, or any target supported by Java. If further information is required about the document beyond what DocumentRecord can provide, register a listener with QueryBatcher.onUrisReady instead.

Parameters:

listener - the code which will process each document

Returns:

this instance for method chaining

See Also:

Consumer, DocumentRecord

onDocumentPageReady

public ExportListener onDocumentPageReady(java.util.function.Consumer<DocumentPage> listener)

Sets a listener to process a page of retrieved documents. Useful for when documents should be written to an external system where it's more efficient to make batched writes to that system. Note that close() does need to be invoked on the DocumentPage; this class will handle that.

Parameters:

listener - the code which will process each page of documents

Returns:

this instance for method chaining

Since:

6.2.0

See Also:

Consumer, DocumentPage

onFailure

public ExportListener onFailure(BatchFailureListener<QueryBatch> listener)

When a batch fails or a callback throws an Exception, run this listener code. Multiple listeners can be registered with this method.

Parameters:

listener - the code to run when a failure occurs

Returns:

this instance for method chaining

getBatchFailureListeners

protected java.util.List<BatchFailureListener<QueryBatch>> getBatchFailureListeners()