Run a Flow Using the Data Hub Java API

You can programmatically run a flow by using the MarkLogic Data Hub Java API in a Java application, such as an external orchestration tool or a custom web application.

The FlowRunner class provides a runFlow method which does not require project files in the local filesystem. This method accepts a single parameter of type FlowInputs and is the preferred method for running flows programmatically.

Procedure

  1. In your build configuration file, declare the dependency on MarkLogic Data Hub Java API. 
       dependencies {
    compile('com.marklogic:marklogic-data-hub:VERSION_NUMBER')
  }

       <dependency>
    <groupId>com.marklogic</groupId>
    <artifactId>marklogic-data-hub</artifactId>
    <version>VERSION_NUMBER</version>
    <type>pom</type>
  </dependency>

       <dependency org='com.marklogic' name='marklogic-data-hub' rev='VERSION_NUMBER'>
    <artifact name='$AID' ext='pom'></artifact>
  </dependency>
  2. Copy the appropriate code and customize it according to your needs.
    • If you are running the flow against a DHS instance, copy and customize the following code. 
         import com.marklogic.hub.flow.FlowInputs;
  import com.marklogic.hub.flow.FlowRunner;
  import com.marklogic.hub.flow.RunFlowResponse;
  import com.marklogic.hub.flow.impl.FlowRunnerImpl;
  import com.marklogic.hub.impl.HubConfigImpl;
  import com.marklogic.mgmt.util.SimplePropertySource;
  import java.util.Properties;

  public class Main {

      public static void main(String[] args) {

          // Instantiate a HubConfig with DHF's default set of properties, and then start customizing it
          HubConfigImpl hubConfig = new HubConfigImpl();

          hubConfig.setHost("myHost");
          hubConfig.setMlUsername("myUser");
          hubConfig.setMlPassword("myPassword");

          // Customization for dhs
          Properties props = new Properties();
          props.setProperty("hubDhs", "true");
          props.setProperty("hubSsl", "true");
          hubConfig.applyProperties(new SimplePropertySource(props));


          FlowRunner flowRunner = new FlowRunnerImpl(hubConfig.newHubClient());
          FlowInputs inputs = new FlowInputs("my-flow-name");

          // To run only a subset of the steps in the flow, uncomment the following line and specify the sequence numbers of the steps to run.
          // inputs.setSteps(Arrays.asList("2","3","4"));

          // Set file path for ingestion steps.
          // inputs.setInputFilePath("<path>");

          // Run the flow.
          RunFlowResponse response = flowRunner.runFlow(inputs);

          // Wait for the flow to end.
          flowRunner.awaitCompletion();

          // Display the response.
          System.out.println("Response: " + response);

      }
  }
    • If you are running the flow on-premises or locally, copy and customize the following code. 
         package org.example;

  import com.marklogic.hub.flow.FlowInputs;
  import com.marklogic.hub.flow.FlowRunner;
  import com.marklogic.hub.flow.RunFlowResponse;
  import com.marklogic.hub.flow.impl.FlowRunnerImpl;

  public class Main {

      public static void main(String[] args) {
          // Create a FlowRunner instance.
          FlowRunner flowRunner = new FlowRunnerImpl("myHost", "myUser", "myPassword");

          // Specify the flow to run.
          FlowInputs inputs = new FlowInputs("my-flow-name");

          // To run only a subset of the steps in the flow, uncomment the following line and specify the sequence numbers of the steps to run.
          // inputs.setSteps(Arrays.asList("2","3","4"));

          // Run the flow.
          RunFlowResponse response = flowRunner.runFlow(inputs);

          // Wait for the flow to end.
          flowRunner.awaitCompletion();

          // Display the response.
          System.out.println("Response: " + response);
      }
  }
  3. Run your code.

Example

The dh-5-example project in GitHub includes the following example code files: