Data Hub Service Using SSL/TLS

This document explains common connection problems when using Secure Socket Layer (SSL)/Transport Layer Security (TLS) with MarkLogic Data Hub Service (DHS). SSL/TLS is a transaction security standard that provides encrypted protection between clients and App Servers. When SSL/TLS is enabled for an App Server, clients communicate with the App Server over HTTPS connection. SSL/TLS ensures client to DHS network connection is secure.

Note: To learn more about using SSL/TLS with MarkLogic, see Using Basic Authentication over HTTPS in place of HTTP Digest Authentication.

The following scenarios explain how to set up client-side tools to connect with DHS:

I cannot use CURL to connect with DHS endpoints using SSL/TLS.

If you experience a connection error, you need to force switch the authentication type to Basic. Some curl versions use Digest authentication type if you specify the --anyauth option.

Note: For more general information on types of authentication, see Authenticating Users.

Error

If you experience a connection error, it is likely that your curl version is using Digest authentication type. For some curl versions, the --anyauth option will automatically switch the authentication type to Digest, producing an error.

 
  $ curl --anyauth --user [username]:[password] \
  https://[replace_with_dhs_endpoint]:8010/

If your curl version uses Digest authentication type, you will receive an output similar to the following:

 
  {"errorResponse": {"statusCode":401,
    "status":"Unauthorized",
    "message":"401 Unauthorized"
    }
  }

Fix

To fix the connection error, use the --basic option to force switch the authentication type to Basic.

The following curl command is an example of the correct options for making a call to DHS endpoints:

 
  $ curl --basic --user [username]:[password] \
  https://[replace_with_dhs_endpoint]:8010/

I cannot connect clients with DHS endpoints using SSL/TLS.

If you experience a connection error, you need to configure your client to use SSL/TLS.

Java Client API

Error 1

If you experience a connection error with a stack trace similar to the following, it is likely that SSL/TLS is not enabled, regardless of the authentication type.

 
  Exception in thread "main" com.marklogic.client.ForbiddenUserException: Local message: User is not allowed to write documents. Server Message: Server (not a REST instance?) did not respond with an expected REST Error message.
    at com.marklogic.client.impl.OkHttpServices.putPostDocumentImpl(OkHttpServices.java:1440)
    at com.marklogic.client.impl.OkHttpServices.putDocument(OkHttpServices.java:1256)
    at com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:920)
    at com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:758)
    at com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:688)
    at com.marklogic.eng.docs.tls.example.TlsExample.main(TlsExample.java:39)

Error 2

If you experience a connection error with a stack trace similar to the following, it is likely that SSL/TLS is enabled and authentication type is Digest.

 
  Exception in thread "main" java.lang.IllegalArgumentException: unsupported auth scheme: [Basic realm=public]
    at com.burgstaller.okhttp.digest.DigestAuthenticator.findDigestHeader(DigestAuthenticator.java:198)
    at com.burgstaller.okhttp.digest.DigestAuthenticator.authenticate(DigestAuthenticator.java:163)
    at com.burgstaller.okhttp.CachingAuthenticatorDecorator.authenticate(CachingAuthenticatorDecorator.java:30)
    at okhttp3.internal.http.RetryAndFollowUpInterceptor.followUpRequest(RetryAndFollowUpInterceptor.java:289)
    at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:157)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
    at com.burgstaller.okhttp.AuthenticationCacheInterceptor.intercept(AuthenticationCacheInterceptor.java:45)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
    at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
    at okhttp3.RealCall.execute(RealCall.java:77)
    at com.marklogic.client.impl.OkHttpServices.sendRequestOnce(OkHttpServices.java:695)
    at com.marklogic.client.impl.OkHttpServices.sendRequestOnce(OkHttpServices.java:690)
    at com.marklogic.client.impl.OkHttpServices.putPostDocumentImpl(OkHttpServices.java:1394)
    at com.marklogic.client.impl.OkHttpServices.putDocument(OkHttpServices.java:1256)
    at com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:920)
    at com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:758)
    at com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:688)
    at com.marklogic.eng.docs.tls.example.TlsExample.main(TlsExample.java:37)

Fix

To fix the connection error, enable SSL/TLS and set authentication type to Basic. For details, see Connecting to MarkLogic with SSL in the Java Application Developer's Guide.

The following code is an example of the correct configuration for connecting with DHS. Set the values for user, password, and DHS endpoint as indicated in the example code.

 
  package com.marklogic.eng.docs.mlaas.connectiontestssl;
  
  import com.marklogic.client.DatabaseClient;
  import com.marklogic.client.DatabaseClientFactory;
  import java.security.cert.X509Certificate;
  
  import javax.net.ssl.SSLContext;
  import javax.net.ssl.TrustManager;
  import javax.net.ssl.X509TrustManager;
  
  import com.marklogic.client.DatabaseClientFactory.BasicAuthContext;
  import com.marklogic.client.DatabaseClientFactory.SecurityContext;
  import com.marklogic.client.document.TextDocumentManager;
  import com.marklogic.client.io.StringHandle;
  import java.security.KeyManagementException;
  import java.security.NoSuchAlgorithmException;
  
  public class TestConnection {
  
    public static void main(String[] args) {
      SecurityContext securityContext = new BasicAuthContext("[user]", "[password]");
      securityContext
          .withSSLContext(SimpleX509TrustManager.newSSLContext(), new SimpleX509TrustManager())
          .withSSLHostnameVerifier(DatabaseClientFactory.SSLHostnameVerifier.ANY);
      DatabaseClient client = DatabaseClientFactory
          .newClient("[replace_with_dhs_endpoint]", 8010, securityContext);
      TextDocumentManager docMgr = client.newTextDocumentManager();
      // Define a URI value for a document.
      String docId = "/example/HellWorld.txt";
      // Create a handle to hold string content.
      StringHandle handle = new StringHandle();
      // Give the handle some content
      handle.set("Connected!");
      // Write the document to the database with URI from docId and content from handle
      docMgr.write(docId, handle);
      System.out.println(docMgr.read(docId, handle));
      // release the client
      client.release();
    }
  }
  
  class SimpleX509TrustManager implements X509TrustManager {
    // Factory method for creating a simple SSLContext that uses this class as its TrustManager.
    public static SSLContext newSSLContext( ) {
      return newSSLContext("TLSv1.2");
    }
    public static SSLContext newSSLContext(String protocol) {
      try {
        SSLContext sslContext = SSLContext.getInstance(protocol);
        sslContext.init(null, new TrustManager[]{new SimpleX509TrustManager()}, null);
        return sslContext;
      } catch (KeyManagementException | NoSuchAlgorithmException e) {
        throw new RuntimeException(e);
      }
    }
  }

DMSDK

See Java Client API.

MLCP

Error

If you experience a connection error, it is likely that you did not connect to DHS using SSL/TLS.

 
  $ mlcp.sh import -host [replace_with_dhs_ingestion_curation_endpoint] -port 8010 \
  -username [username] -password [password] \
  -input_file_path [replace_with_path]

If you did not connect to DHS using SSL/TLS, you will receive an output similar to the following:

 
  19/09/11 13:07:24 INFO contentpump.LocalJobRunner: Content type is set to MIXED.  The format of the  inserted documents will be determined by the MIME  type specification configured on MarkLogic Server.
  19/09/11 13:07:25 INFO contentpump.ContentPump: Job name: local_1376435123_1
  19/09/11 13:07:25 INFO contentpump.FileAndDirectoryInputFormat: Total input paths to process : 3
  2019-09-11 13:07:26.278 WARNING [1] (AbstractRequestController.runRequest): Error parsing HTTP headers: Premature EOF, partial header line read: ''
  2019-09-11 13:07:26.457 WARNING [1] (AbstractRequestController.runRequest): Error parsing HTTP headers: Premature EOF, partial header line read: ''
  2019-09-11 13:07:26.563 WARNING [1] (AbstractRequestController.runRequest): Error parsing HTTP headers: Premature EOF, partial header line read: ''
  19/09/11 13:07:26 WARN mapreduce.MarkLogicOutputFormat: Unable to connect to [foya9tvmi].[9nhjcyawcyq].a.marklogicsvc.com to query destination information
  19/09/11 13:07:26 ERROR contentpump.LocalJobRunner: Error checking output specification: 
  19/09/11 13:07:26 ERROR contentpump.LocalJobRunner: Unable to query destination information, no usable hostname found

Fix

To fix the connection error, use the -ssl and -restrict_hosts options. The -ssl option connects to DHS using SSL/TLS. When -restrict_hosts is set to true, MLCP will only connect to the hosts listed in the -host option.

The following command is an example of the correct options for connecting MLCP with DHS endpoints. Set the values for DHS ingestion or curation endpoint, username, password, and file path as indicated in the example command.

 
  $ mlcp.sh import -host [replace_with_dhs_ingestion_curation_endpoint] -port 8010 \
  -username [username] -password [password] \
  -input_file_path [replace_with_file_path] \
  -ssl -restrict_hosts true

If you connected to DHS using SSL/TLS, you will receive an output similar to the following:

 
  19/09/11 13:08:23 INFO contentpump.LocalJobRunner: Content type is set to MIXED.  The format of the  inserted documents will be determined by the MIME  type specification configured on MarkLogic Server.
  19/09/11 13:08:25 INFO contentpump.ContentPump: Job name: local_267027890_1
  19/09/11 13:08:25 INFO contentpump.FileAndDirectoryInputFormat: Total input paths to process : 3
  19/09/11 13:08:27 INFO contentpump.LocalJobRunner:  completed 100%
  19/09/11 13:08:27 INFO contentpump.LocalJobRunner: com.marklogic.mapreduce.MarkLogicCounter: 
  19/09/11 13:08:27 INFO contentpump.LocalJobRunner: INPUT_RECORDS: 3
  19/09/11 13:08:27 INFO contentpump.LocalJobRunner: OUTPUT_RECORDS: 3
  19/09/11 13:08:27 INFO contentpump.LocalJobRunner: OUTPUT_RECORDS_COMMITTED: 3
  19/09/11 13:08:27 INFO contentpump.LocalJobRunner: OUTPUT_RECORDS_FAILED: 0
  19/09/11 13:08:27 INFO contentpump.LocalJobRunner: Total execution time: 2 sec