Create a Step Using QuickStart

Before you begin

You need:

About this task

A flow must have at least one step. In QuickStart, steps can be created when editing the flow.

Procedure


QuickStart Steps - New Step
  1. Navigate to the flow definition of the flow you want.

    QuickStart Flows - Manage Flows table - Click flow name

    1. In QuickStart's navigation bar, click Flows.
    2. In the Manage Flows table, search for the row containing the flow.
      Tip: To make your search easier, you can sort the table by one of the columns.
    3. Click the flow's name.
  2. In the flow definition page, click New Step.
  3. In the New Step dialog, choose the Step Type.
    • Ingestion
    • Mapping
    • Mastering
    • Custom
  4. Configure the step settings.

    Expand the Advanced Settings section for additional fields.


    QuickStart Step - New Ingestion Step dialog
    Field Description
    NameThe name of the step instance.
    Description(Optional) A description of the step.
    Target Database The database where to store the processed data. For ingestion, choose the STAGING database where you want to store the ingested data. Default is data-hub-STAGING.
    Additional Target Collections Collection tags to add to the processed records. By default, the processed records are added to the collection with the same name as the step. You can add the records to additional collections by specifying one or more collection tags.
    • Click to add more collection tags.
    • Click next to a collection tag to delete it.

    QuickStart Step - New Mapping Step dialog
    Field Description
    NameThe name of the step instance.
    Description(Optional) A description of the step.
    Source Type The filter to use to select the source data to process in this flow.
    • Collection
    • Query
    Source Collection (Displayed if Source Type is Collection.) The collection tag to use to search for the records to process in this step.
    Source Query (Displayed if Source Type is Query.) The CTS query to use to select the source data to process. See CTS Query.
    Target Entity The entity to map against the source data. Required only if the flow includes a mapping step.
    Source Database The database from which to take the input data. For mapping, choose the STAGING database where you obtained the data. Default is data-hub-STAGING.
    Target Database The database where to store the processed data. For mapping, choose the FINAL database where you want to store mapped data. Default is data-hub-FINAL.
    Target Format The format of the processed record: Text, JSON, XML, or Binary. Default is JSON.
    Additional Target Collections Collection tags to add to the processed records. By default, the processed records are added to the collection with the same name as the step. You can add the records to additional collections by specifying one or more collection tags.
    • Click to add more collection tags.
    • Click next to a collection tag to delete it.

    QuickStart Step - New Mastering Step dialog
    Field Description
    NameThe name of the step instance.
    Description(Optional) A description of the step.
    Source Type The filter to use to select the source data to process in this flow.
    • Collection
    • Query
    Source Collection (Displayed if Source Type is Collection.) The collection tag to use to search for the records to process in this step.
    Source Query (Displayed if Source Type is Query.) The CTS query to use to select the source data to process. See CTS Query.
    Target Entity The entity to map against the source data. Required only if the flow includes a mapping step.
    Source Database The database from which to take the input data. For mastering, choose the FINAL database where you stored processed data. Default is data-hub-FINAL.
    Target Database The database where to store the processed data. For mastering, choose the FINAL database where you want to store mastered data. Default is data-hub-FINAL.
    Important: For a mastering step, the source database and the target database must be the same. If duplicates are found, the original records are archived and the merged version is added to the same database. If you want the target database to be different, you can create a custom step with a custom module to override the default behavior of the mastering step.
    Target Format The format of the processed record: Text, JSON, XML, or Binary. Default is JSON.
    Additional Target Collections Collection tags to add to the processed records. By default, the processed records are added to the collection with the same name as the step. You can add the records to additional collections by specifying one or more collection tags.
    • Click to add more collection tags.
    • Click next to a collection tag to delete it.

    QuickStart Step - New Custom Step dialog
    Field Description
    NameThe name of the step instance.
    Description(Optional) A description of the step.
    Source Type The filter to use to select the source data to process in this flow.
    • Collection
    • Query
    Source Collection (Displayed if Source Type is Collection.) The collection tag to use to search for the records to process in this step.
    Source Query (Displayed if Source Type is Query.) The CTS query to use to select the source data to process. See CTS Query.
    Target Entity The entity to map against the source data. Required only if the flow includes a mapping step.
    Source Database The database from which to take the input data.
    • For mapping, choose the STAGING database where you obtained the data. Default is data-hub-STAGING.
    • For mastering, choose the FINAL database where you stored processed data. Default is data-hub-FINAL.
    Target Database The database where to store the processed data.
    • For ingestion, choose the STAGING database where you want to store the ingested data. Default is data-hub-STAGING.
    • For mapping, choose the FINAL database where you want to store mapped data. Default is data-hub-FINAL.
    • For mastering, choose the FINAL database where you want to store mastered data. Default is data-hub-FINAL.
    Important: For a mastering step, the source database and the target database must be the same. If duplicates are found, the original records are archived and the merged version is added to the same database. If you want the target database to be different, you can create a custom step with a custom module to override the default behavior of the mastering step.
    Target Format The format of the processed record: Text, JSON, XML, or Binary. Default is JSON.
    Additional Target Collections Collection tags to add to the processed records. By default, the processed records are added to the collection with the same name as the step. You can add the records to additional collections by specifying one or more collection tags.
    • Click to add more collection tags.
    • Click next to a collection tag to delete it.
  5. Click Save.

Results

The new step's summary box is added to the flow sequence in the flow panel at the top.

The step panels show the step details.

What to do next

Configure the step details: