You can programmatically run a flow by using the MarkLogic Data Hub Java API in a Java application, such as an external orchestration tool or a custom web application.
The FlowRunner
class provides a runFlow
method which does not require project files in the local filesystem. This method accepts a single parameter of type FlowInputs
and is the preferred method for running flows programmatically.
Procedure
- In your build configuration file, declare the dependency on MarkLogic Data Hub Java API.
dependencies {
compile('com.marklogic:marklogic-data-hub:VERSION_NUMBER')
}
<dependency>
<groupId>com.marklogic</groupId>
<artifactId>marklogic-data-hub</artifactId>
<version>VERSION_NUMBER</version>
<type>pom</type>
</dependency>
<dependency org='com.marklogic' name='marklogic-data-hub' rev='VERSION_NUMBER'>
<artifact name='$AID' ext='pom'></artifact>
</dependency>
- Copy the appropriate code and customize it according to your needs.
- If you are running the flow against a DHS instance, copy and customize the following code.
import com.marklogic.hub.flow.FlowInputs;
import com.marklogic.hub.flow.FlowRunner;
import com.marklogic.hub.flow.RunFlowResponse;
import com.marklogic.hub.flow.impl.FlowRunnerImpl;
import com.marklogic.hub.impl.HubConfigImpl;
import com.marklogic.mgmt.util.SimplePropertySource;
import java.util.Properties;
public class Main {
public static void main(String[] args) {
// Instantiate a HubConfig with DHF's default set of properties, and then start customizing it
HubConfigImpl hubConfig = new HubConfigImpl();
hubConfig.setHost("myHost");
hubConfig.setMlUsername("myUser");
hubConfig.setMlPassword("myPassword");
// Customization for dhs
Properties props = new Properties();
props.setProperty("hubDhs", "true");
props.setProperty("hubSsl", "true");
hubConfig.applyProperties(new SimplePropertySource(props));
FlowRunner flowRunner = new FlowRunnerImpl(hubConfig.newHubClient());
FlowInputs inputs = new FlowInputs("my-flow-name");
// To run only a subset of the steps in the flow, uncomment the following line and specify the sequence numbers of the steps to run.
// inputs.setSteps(Arrays.asList("2","3","4"));
// Set file path for ingestion steps.
// inputs.setInputFilePath("<path>");
// Run the flow.
RunFlowResponse response = flowRunner.runFlow(inputs);
// Wait for the flow to end.
flowRunner.awaitCompletion();
// Display the response.
System.out.println("Response: " + response);
}
}
- If you are running the flow on-premises or locally, copy and customize the following code.
package org.example;
import com.marklogic.hub.flow.FlowInputs;
import com.marklogic.hub.flow.FlowRunner;
import com.marklogic.hub.flow.RunFlowResponse;
import com.marklogic.hub.flow.impl.FlowRunnerImpl;
public class Main {
public static void main(String[] args) {
// Create a FlowRunner instance.
FlowRunner flowRunner = new FlowRunnerImpl("myHost", "myUser", "myPassword");
// Specify the flow to run.
FlowInputs inputs = new FlowInputs("my-flow-name");
// To run only a subset of the steps in the flow, uncomment the following line and specify the sequence numbers of the steps to run.
// inputs.setSteps(Arrays.asList("2","3","4"));
// Run the flow.
RunFlowResponse response = flowRunner.runFlow(inputs);
// Wait for the flow to end.
flowRunner.awaitCompletion();
// Display the response.
System.out.println("Response: " + response);
}
}
- Run your code.
Example
The dh-5-example project in GitHub includes the following example code files: