Getting Started
Tools by Tracks
You can choose from multiple tools to perform each task in the Data Hub process. QuickStart, the Gradle tasks, and Data Hub Java API provide the widest range of functionality, while others are more focused.
The tools can be categorized into three tracks:
- The GUI track (recommended) provides intuitive graphical interfaces to work with your data.
- QuickStart provides the same functionality available in the non-GUI tools, and it automatically performs many tasks that you might have to explicitly handle when using Gradle.
- The command-line track helps you to quickly automate your most common processes, bypassing the GUI.
- An extensive set of Gradle tasks is provided to automate the creation of artifacts and the execution of flows. Gradle is ideal in a continuous integration environment, where creation, testing, and execution are performed multiple times.
Note: To run a flow using Gradle, you must be in the local directory that contains the Data Hub project files.
- The executable Data Hub Client JAR is the ideal alternative to running a flow in a production environment where Gradle and the Data Hub project files are not available. It only requires a JVM.
- An extensive set of Gradle tasks is provided to automate the creation of artifacts and the execution of flows. Gradle is ideal in a continuous integration environment, where creation, testing, and execution are performed multiple times.
- The programming track provides the APIs you can use to create apps that run flows to manage and use your data.
- The Data Hub Java API is provided for running flows in your own Java-based apps or in an external orchestration system that supports Java-based extensions.
- MLCP provides an alternative to ingesting data into the STAGING database.
- The REST Client API provides some record-management and job-information-retrieval capabilities.
You can switch between tracks or between tools for different tasks; however, switching from the command-line track to the GUI track might be less convenient because the GUI handles some processes automatically.
Tasks by Tracks
The following table organizes the tasks you can run with the tools in each track:
Task | GUI Track | Command-Line Track | Programming Track |
---|---|---|---|
Create Project | Using QuickStart | Using Gradle | |
Set Security Credentials | Using Gradle | ||
Create Entity (required for mapping step) |
Using QuickStart | Using Gradle | |
Create Flow | Using QuickStart | Using Gradle | |
Edit the Flow Settings | Using QuickStart | Manually | |
Create Step | Using QuickStart | Using Gradle | |
Configure Step | |||
Create Mapping | Manually | ||
Add Custom Hooks | Manually | ||
Manage Steps in a Flow | Using QuickStart | Manually | |
Run Flow | Using QuickStart | ||
Merge Records Outside a Flow | Using Gradle | Using REST Client API | |
Unmerge a Record Outside a Flow | Using Gradle | Using REST Client API | |
Deploy | to Data Hub Service | ||
Redeploy | Using QuickStart | Using Gradle |
Sample Data
If you need sample data to test with, you can download and unzip store-data.zip, which contains data files (in CSV format) of a fictional board game retailer.