Run a Flow Using Gradle

You can use Gradle to run flows outside a GUI.

Tip: To avoid depending on Gradle and project files, especially in production environments, run your flows using the Data Hub Client JAR.
Important: When you load/ingest delimited text files into MarkLogic Server, URIs are randomly generated each time you load/ingest. To generate the same URIs each time you load/ingest delimited text files, use MLCP.

Before you begin

You need:

Procedure

  1. Open a command-line window, and go to your project root directory.
  2. At your project root, run the Gradle task hubRunFlow.
    ./gradlew hubRunFlow -PflowName=YourFlowName -PentityName=YourEntityName -PbatchSize=100 -PthreadCount=4 -PshowOptions=[true|false] -PfailHard=[true|false] -Psteps="1,2" -PjobId="abc123" [ -Poptions="{ customkey: customvalue, ... }" | -PoptionsFile=/path/to.json ] -igradlew.bat hubRunFlow -PflowName=YourFlowName -PentityName=YourEntityName -PbatchSize=100 -PthreadCount=4 -PshowOptions=[true|false] -PfailHard=[true|false] -Psteps="1,2" -PjobId="abc123" [ -Poptions="{ customkey: customvalue, ... }" | -PoptionsFile=/path/to.json ] -i
    flowName
    (Required) The name of the harmonize flow to run.
    entityName
    (Required if the flow includes a mapping step) The name of the entity used with the mapping step.
    batchSize
    The maximum number of items to process in a batch. The default is 100.
    threadCount
    The number of threads to run. The default is 4.
    showOptions
    If true, options that were passed to the command are printed out. The default is false.
    failHard
    If true, the flow's execution is ended immediately if a step fails. The default is false.
    steps
    The comma-separated numbers of the steps to run. If not provided, the entire flow is run.
    jobId
    A unique job ID to associate with the flow run. This option can be used if the flow run is part of a larger process (e.g., a process orchestrated by NiFi with its own job/process ID). Must not be the same as an existing Data Hub job ID. If not provided, a unique Data Hub job ID will be assigned.
    options
    A JSON structure containing key-value pairs to be passed as custom parameters to your step modules.
    optionsFile
    The path to a JSON file containing key-value pairs to be passed as custom parameters to your step modules.