Import Your Smart Mastering Core Projects

If you used the Smart Mastering Core to define matching and merging rules for your data, you can import those rules as match and merge options into a mastering step in a Data Hub flow.

Procedure

  1. Upgrade Smart Mastering Core to the latest 1.3.x version.
    1. In your Smart Mastering Core build.gradle file, change the Smart Mastering Core version to 1.3.0.
    2. Run the Gradle task mlDeploy.
      ./gradlew mlDeploy -igradlew.bat mlDeploy -i

    See Smart Mastering Core's upgrade.md for potential compatibility issues.

  2. Get your Smart Mastering Core options in JSON format.

    Use the following REST endpoints to retrieve your Smart Mastering Core options:

    • Match options: /v1/resources/sm-match-options?rs:name=your-options-name
    • Merge options: /v1/resources/sm-merge-options?rs:name=your-options-name
  3. In Data Hub, create a project.
  4. Create a flow.
  5. Create a mastering step.
  6. Manually edit your flow configuration file to add your Smart Mastering options.
    1. In your flow configuration file (your-project-root/flows/your-flow-name.flow.json), go to the steps node, then the mastering step, then the options property of that step.
    2. Under the options property, set the values of matchOptions and mergeOptions to the appropriate JSON that you retrieved from your Smart Mastering Core project.

    Example:

       {
    "name" : "MyFlow",
    ...
    "steps" : {
      ...
      "3" : {
        "name" : "MyMasteringStep",
        "description" : "",
        "options" : {
          ...
          "mergeOptions" : {
            *** Replace this value with the appropriate JSON from your Smart Mastering Core project. ***
          },
          "matchOptions" : {
            *** Replace this value with the appropriate JSON from your Smart Mastering Core project. ***
          },
          "sourceCollection" : "MyMappingStep",
          "targetDatabase" : "data-hub-FINAL"
        },
        ...
        "stepDefinitionName" : "default-mastering",
        "stepDefinitionType" : "MASTERING"
      },
      ...
    }
  }
  7. Deploy your changes.
    ./gradlew mlDeploy -igradlew.bat mlDeploy -i