com.marklogic.hub.step.impl

Class WriteStepRunner

java.lang.Object

com.marklogic.hub.step.impl.WriteStepRunner

All Implemented Interfaces:

StepRunner

public class WriteStepRunner
extends java.lang.Object
implements StepRunner

Field Summary

Fields

Modifier and Type	Field and Description
protected long	csvFilesProcessed
protected java.lang.String	inputFilePath
protected java.lang.String	inputFileType
protected java.util.concurrent.atomic.AtomicBoolean	isStopped
protected org.slf4j.Logger	logger
protected java.lang.String	outputCollections
protected java.lang.String	outputFormat
protected java.lang.String	outputPermissions
protected java.lang.String	outputURIPrefix
protected java.lang.String	outputURIReplacement
protected java.lang.String	separator

Constructor Summary

Constructors

Constructor and Description
WriteStepRunner(com.marklogic.hub.HubClient hubClient, HubProject hubProject)

Method Summary

Modifier and Type	Method and Description
void	awaitCompletion() Blocks until the step execution is complete.
void	awaitCompletion(long timeout, java.util.concurrent.TimeUnit unit) Blocks until the step execution is complete.
protected java.nio.file.Path	determineInputFilePath(java.lang.String inputFilePath)
protected java.lang.String	generateUriForCsv(java.lang.String parentPath, java.lang.String os)
int	getBatchSize() Returns the batch size
protected java.lang.String	getPrefixedEncodedURI(java.lang.String filename)
protected void	loadStepRunnerParameters()
StepRunner	onFinished(StepFinishedListener listener) Sets the finished listener for when the item has processed (similar to a finally)
StepRunner	onItemComplete(StepItemCompleteListener listener) Sets a listener on each item completing
StepRunner	onItemFailed(StepItemFailureListener listener) Sets the failure listener for each item in the flow
StepRunner	onStatusChanged(StepStatusListener listener) Sets the status change listener on the flowrunner object
RunStepResponse	run() Runs the flow and creates the job
RunStepResponse	run(java.util.Collection<java.lang.String> uris) Runs the flow and creates the job.
protected void	runStatusListener(double uriSize, StepMetrics stepMetrics)
void	stop() Stops the step run
StepRunner	withBatchSize(int batchSize) Sets the batch size for the step runner
StepRunner	withDestinationDatabase(java.lang.String destinationDatabase)
StepRunner	withFlow(Flow flow) Sets the flow to be used with the step runner
StepRunner	withJobId(java.lang.String jobId) Sets the batch size for the step runner
StepRunner	withOptions(java.util.Map<java.lang.String,java.lang.Object> options) Sets the options to be passed into the xqy or sjs flow in the $options or options variables of main.
StepRunner	withStep(java.lang.String step) Sets the batch size for the step runner
StepRunner	withStepConfig(java.util.Map<java.lang.String,java.lang.Object> stepConfig) Sets the config to be used in orchestrating the step.
StepRunner	withStepDefinition(StepDefinition stepDefinition)
StepRunner	withStopOnFailure(boolean stopOnFailure) Sets if this will stop the job on a failure, or if it will continue on
StepRunner	withThreadCount(int threadCount) Sets the thread count for the flowrunner

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

csvFilesProcessed

protected long csvFilesProcessed

logger

protected final org.slf4j.Logger logger

inputFilePath

protected java.lang.String inputFilePath

outputCollections

protected java.lang.String outputCollections

outputPermissions

protected java.lang.String outputPermissions

outputFormat

protected java.lang.String outputFormat

inputFileType

protected java.lang.String inputFileType

outputURIReplacement

protected java.lang.String outputURIReplacement

outputURIPrefix

protected java.lang.String outputURIPrefix

separator

protected java.lang.String separator

isStopped

protected java.util.concurrent.atomic.AtomicBoolean isStopped

Constructor Detail

WriteStepRunner

public WriteStepRunner(com.marklogic.hub.HubClient hubClient,
                       HubProject hubProject)

Method Detail

withFlow

public StepRunner withFlow(Flow flow)

Description copied from interface: StepRunner

Sets the flow to be used with the step runner

Specified by:

withFlow in interface StepRunner

Parameters:

flow - the flow object to be used

Returns:

the step runner object

withStep

public StepRunner withStep(java.lang.String step)

Description copied from interface: StepRunner

Sets the batch size for the step runner

Specified by:

withStep in interface StepRunner

Parameters:

step - - the step to be run in the flow

Returns:

the step runner object

withJobId

public StepRunner withJobId(java.lang.String jobId)

Description copied from interface: StepRunner

Sets the batch size for the step runner

Specified by:

withJobId in interface StepRunner

Parameters:

jobId - - the id of the job

Returns:

the step runner object

withStepDefinition

public StepRunner withStepDefinition(StepDefinition stepDefinition)

withBatchSize

public StepRunner withBatchSize(int batchSize)

Description copied from interface: StepRunner

Sets the batch size for the step runner

Specified by:

withBatchSize in interface StepRunner

Parameters:

batchSize - - the size of the batch in integer form

Returns:

the step runner object

withThreadCount

public StepRunner withThreadCount(int threadCount)

Description copied from interface: StepRunner

Sets the thread count for the flowrunner

Specified by:

withThreadCount in interface StepRunner

Parameters:

threadCount - - the number of threads for the step runner to use

Returns:

the step runner object

withDestinationDatabase

public StepRunner withDestinationDatabase(java.lang.String destinationDatabase)

withStopOnFailure

public StepRunner withStopOnFailure(boolean stopOnFailure)

Description copied from interface: StepRunner

Sets if this will stop the job on a failure, or if it will continue on

Specified by:

withStopOnFailure in interface StepRunner

Parameters:

stopOnFailure - - true to stop the job if a failure happens

Returns:

the step runner object

withOptions

public StepRunner withOptions(java.util.Map<java.lang.String,java.lang.Object> options)

Description copied from interface: StepRunner

Sets the options to be passed into the xqy or sjs flow in the $options or options variables of main.

Specified by:

withOptions in interface StepRunner

Parameters:

options - - the object map of options as string/object pair

Returns:

the step runner object

withStepConfig

public StepRunner withStepConfig(java.util.Map<java.lang.String,java.lang.Object> stepConfig)

Description copied from interface: StepRunner

Sets the config to be used in orchestrating the step.

Specified by:

withStepConfig in interface StepRunner

Parameters:

stepConfig - - the object map of configurations for the step to execute as string/object pair

Returns:

the step runner object

onItemComplete

public StepRunner onItemComplete(StepItemCompleteListener listener)

Description copied from interface: StepRunner

Sets a listener on each item completing

Specified by:

onItemComplete in interface StepRunner

Parameters:

listener - the listen object to set

Returns:

the step runner object

onItemFailed

public StepRunner onItemFailed(StepItemFailureListener listener)

Description copied from interface: StepRunner

Sets the failure listener for each item in the flow

Specified by:

onItemFailed in interface StepRunner

Parameters:

listener - the listener for the failures in the flow

Returns:

the step runner object

onStatusChanged

public StepRunner onStatusChanged(StepStatusListener listener)

Description copied from interface: StepRunner

Sets the status change listener on the flowrunner object

Specified by:

onStatusChanged in interface StepRunner

Parameters:

listener - - the listener for when the status changes

Returns:

the step runner object

onFinished

public StepRunner onFinished(StepFinishedListener listener)

Description copied from interface: StepRunner

Sets the finished listener for when the item has processed (similar to a finally)

Specified by:

onFinished in interface StepRunner

Parameters:

listener - - the listener for the flow item when it finishes

Returns:

the step runner object

awaitCompletion

public void awaitCompletion()

Description copied from interface: StepRunner

Blocks until the step execution is complete.

Specified by:

awaitCompletion in interface StepRunner

awaitCompletion

public void awaitCompletion(long timeout,
                            java.util.concurrent.TimeUnit unit)
                     throws java.lang.InterruptedException,
                            java.util.concurrent.TimeoutException

Description copied from interface: StepRunner

Blocks until the step execution is complete.

Specified by:

awaitCompletion in interface StepRunner

Parameters:

timeout - the maximum time to wait

unit - the time unit of the timeout argument

Throws:

java.lang.InterruptedException - if interrupted while waiting

java.util.concurrent.TimeoutException - if times out

getBatchSize

public int getBatchSize()

Description copied from interface: StepRunner

Returns the batch size

Specified by:

getBatchSize in interface StepRunner

Returns:

batch size for step runner

run

public RunStepResponse run()

Description copied from interface: StepRunner

Runs the flow and creates the job

Specified by:

run in interface StepRunner

Returns:

RunStepResponse object for the flow that is run

run

public RunStepResponse run(java.util.Collection<java.lang.String> uris)

Description copied from interface: StepRunner

Runs the flow and creates the job. This bypasses the collector

Specified by:

run in interface StepRunner

Parameters:

uris - the ids to pass to the harmonization flow

Returns:

RunStepResponse object for the flow that is run

stop

public void stop()

Description copied from interface: StepRunner

Stops the step run

Specified by:

stop in interface StepRunner

loadStepRunnerParameters

protected void loadStepRunnerParameters()

determineInputFilePath

protected java.nio.file.Path determineInputFilePath(java.lang.String inputFilePath)

generateUriForCsv

protected java.lang.String generateUriForCsv(java.lang.String parentPath,
                                             java.lang.String os)

getPrefixedEncodedURI

protected java.lang.String getPrefixedEncodedURI(java.lang.String filename)
                                          throws java.net.URISyntaxException

Throws:

java.net.URISyntaxException

runStatusListener

protected void runStatusListener(double uriSize,
                                 StepMetrics stepMetrics)