Run a Flow Using the REST Client API
When running a flow, Data Hub performs the following:
- Calls the collector to get the list of IDs of the records to process.
- Splits the list into batches.
- Calls a custom REST extension that invokes the main.[xqy|sjs] for your flow.
MarkLogic Data Hub provides a REST Client API extension runFlow which allows you to specify:
- The individual records to process.
- The single step to run.
The REST Client API extension that runs a single step within the flow to process the specified records.
- A unique job ID to associate with the flow run. This option can be used if the flow run is part of a larger process (e.g., a process orchestrated by NiFi with its own job/process ID). Must not be the same as an existing Data Hub job ID. If not provided, a unique Data Hub job ID will be assigned.
- The name of the flow.
- The sequence number of the step to execute. To run multiple specific steps, use your orchestration tool to send one runFlow request for each step.
- The database containing the source data; e.g., data-hub-STAGING.
- The database where you want to store the processed data; e.g., data-hub-FINAL.
- A JSON object containing additional options to pass to the flow.
- To specify the list of records to process, add the key uris whose value is an array of the URIs of the records to process.