Upgrade to MarkLogic Data Hub 5.2

Prerequisites

You need:

Important:

Upgrading to this release from earlier than 5.2.0 would trigger a reindexing of the STAGING and FINAL databases. See the release notes for details.

Remember to archive your old project files before performing an upgrade.

Important: If you are deploying to a Data Hub Service environment, you must contact Support to upgrade your DHS environment to use Data Hub v5.x.

The notes and steps in this tab are for the following upgrade paths:

  • Data Hub 5.1 » 5.2
  • Data Hub 5.0 » 5.2

Procedure

  1. In your project root directory, create a build.gradle file with the Grade Properties plugin setting and the Data Hub plugin setting.

    Copy the following to your build.gradle file and replace the com.marklogic.ml-data-hub version number with the Data Hub version that you are using.

       plugins {
      // Gradle Properties plugin
      id 'net.saliman.properties' version '1.4.6'

      // Data Hub plugin
      id 'com.marklogic.ml-data-hub' version '5.2.1'
  }
    net.saliman.properties Gradle Properties plugin Allows you to create different environments for your Gradle deployment and set up a gradle-{env}.properties file, where {env} is the environment name. When running a Gradle task, you can specify the target environment with the environmentName option. For more information, see https://github.com/stevesaliman/gradle-properties-plugin.
    com.marklogic.ml-data-hub Data Hub plugin Extends the ml-gradle plugin with Data Hub-specific commands.
  2. At your project root, run the Gradle task hubUpdate.

    Running the hubUpdate task with the -i option (info mode) displays specifically what the task does, including configuration settings that changed.

    ./gradlew hubUpdate -igradlew.bat hubUpdate -i
  3. Update your gradle.properties file.
    • Change the value of mlDHFVersion to 5.2.1. 
      mlDHFVersion=5.2.1
    • (Recommended) To use the new Data Hub 5.2 default permissions for modules (data-hub-module-reader,read,data-hub-module-reader,execute,data-hub-module-writer,update,rest-extension-user,execute), delete mlModulePermissions.
  4. Run the Gradle task mlRedeploy.
    ./gradlew mlRedeploy -igradlew.bat mlRedeploy -i

The notes and steps in this tab are for the following upgrade paths:

  • DHF 4.3 » Data Hub 5.2

Procedure

  1. In your project root directory, create a build.gradle file with the Grade Properties plugin setting and the Data Hub plugin setting.

    Copy the following to your build.gradle file and replace the com.marklogic.ml-data-hub version number with the Data Hub version that you are using.

       plugins {
      // Gradle Properties plugin
      id 'net.saliman.properties' version '1.4.6'

      // Data Hub plugin
      id 'com.marklogic.ml-data-hub' version '5.2.1'
  }
    net.saliman.properties Gradle Properties plugin Allows you to create different environments for your Gradle deployment and set up a gradle-{env}.properties file, where {env} is the environment name. When running a Gradle task, you can specify the target environment with the environmentName option. For more information, see https://github.com/stevesaliman/gradle-properties-plugin.
    com.marklogic.ml-data-hub Data Hub plugin Extends the ml-gradle plugin with Data Hub-specific commands.
  2. At your project root, run the Gradle task hubUpdate.

    Running the hubUpdate task with the -i option (info mode) displays specifically what the task does, including configuration settings that changed.

    ./gradlew hubUpdate -igradlew.bat hubUpdate -i

    hubUpdate automatically moves the following artifacts to the new project.

    From old project To Data Hub 5.2.1 project 
       your-project-root/plugins/entities/entity1/entity1.entity.json
  ...
  your-project-root/plugins/entities/entityN/entityN.entity.json

    The input and harmonize folders remain in the same plugins/entities/entity* folders.

    		 
       your-project-root/entities/entity1.entity.json
  ...
  your-project-root/entities/entityN.entity.json
    your-project-root/plugins/mappings (the entire directory) your-project-root/mappings
  3. Update your gradle.properties file.
    • Change the value of mlDHFVersion to 5.2.1. 
      mlDHFVersion=5.2.1
    • (Recommended) To use the new Data Hub 5.2 default permissions for modules (data-hub-module-reader,read,data-hub-module-reader,execute,data-hub-module-writer,update,rest-extension-user,execute), delete mlModulePermissions.
  4. Run the Gradle task mlRedeploy.
    ./gradlew mlRedeploy -igradlew.bat mlRedeploy -i

The notes and steps in this tab are for the following upgrade paths:

  • DHF 4.2 » Data Hub 5.2
  • DHF 4.1 » Data Hub 5.2
  • DHF 4.0 » Data Hub 5.2

Procedure

  1. Upgrade to DHF 4.3 to adopt the new roles.
  2. Upgrade from DHF 4.3 to Data Hub 5.2.1.

The notes and steps in this tab are for the following upgrade paths:

  • DHF 3.x and earlier » Data Hub 5.2

Procedure

  1. Upgrade to DHF 4.3.

    Significant changes were made in Data Hub 4.x releases, including changes in the project directory structure, security roles, and databases. These changes require that you manually update configuration files, run the Data Hub 4.3.2 versions of the Gradle tasks to correctly reconfigure your project and environment, and perform tests before proceeding to the Data Hub upgrade. For details, see the Upgrade Notes and Additional Upgrade Notes sections in Upgrade to DHF 4.3.

    The data that you ingested and processed in Data Hub Framework 2.x, 3.x, or 4.x is compatible with Data Hub 5.x. Therefore, you can install 5.x directly, instead of upgrading; however, you must recreate your 2.x/3.x/4.x flows as steps in 5.x, and you might need to update your custom code.

  2. Upgrade from DHF 4.3 to Data Hub 5.2.1.