XQuery xdmp:http-get

MarkLogic 10 Product Documentation
xdmp.httpGet

xdmp.httpGet(
   uri as String,
   [options as Object?]
) as Sequence

Summary

Sends the http GET method to the specified URI. Returns the http response as well as whatever information is identified by the specified URI (for example, an html document).

Parameters
uri The URI of the requested document.
options Options with which to customize this operation. This function supports the following options, plus certain options from the xdmp.documentLoad and xdmp.documentGet functions. For example, you can use the repair and encoding options from these functions.

headers

An object with each of its properties representing one HTTP header. The name of the property is the header's name and the value of the property is the header's value. The names can be anything, but many HTTP servers understand HTTP names such as content-type.

credentialId

The credential id to use for authentication. This is the preferred way of providing authentication credentials because they are stored securely in the security database. When a credential id is specified, the other authentication information fields should be left empty and will be ignored. For details on obtaining a credential id, see the Usage Notes, below.

authentication

The credentials and the authentication method to use for this request. This property can contain the following child properties:
  • username: The username of the user to be authenticated on the http server
  • password: The password for username.
  • method: Optional. The authentication method. Allowed values: basic, digest, aws, aws4. If a method is specified and the HTTP server requests a different type of authentication, then an error is raised.

timeout

The amount of time, in seconds, to wait until the HTTP connection times out. The default value is the http timeout for the group.

ciphers

A standard cipher string. For details on legal ciper strings, see http://www.openssl.org/docs/apps/ciphers.html#CIPHER_STRINGS.

client-cert

A PEM encoded client certificate for identifying the client to the remote server.

clientKey

The private key that corresponds to clientCert.

passPhrase

A pass phrase, if one is needed to decrypt client-key.

allowSslv3

A boolean value to specify whether to communicate using the SSL v3 protocol. The default is true, which indicates communication using the SSL v3 protocol.

allowTls

A boolean value to specify whether to communicate using the TLS protocol. The default is true, which indicates communication using the TLS protocol.

verifyCert

A boolean value to specify whether the server's certificate should be verified. The default value is true. A value of false should only be specfied after careful consideration of the security risks since it permits communication with servers whose certificates are expired, revoked, or signed by unknown or untrusted authorities. A value of false also removes protection against a man-in-the-middle attack.

sslSessionCache

A boolean value to specify whether ssl session should be cached and reused. The default value is true. A value of false should only be specfied if ssl session cache causes problem with a url.

kerberosTicketForwarding

A string value to specify how the user ticket is handled. Allowed values: disabled, required, optional. The default value is disabled.
If the value is disabled, the user ticket is not forwarded. If the value is requried, the user ticket is forwarded if it is forwardable. If the user ticket is not forwardable, it is an error. If the value is optional, the user ticket is forwarded if it is forwardable.

proxy

The URL of the proxy server

Required Privileges

http://marklogic.com/xdmp/privileges/xdmp-http-get

Usage Notes

The http functions only operate on URIs that use the http or https schemes; specifying a URI that does not begin with http:// or https:// throws an exception.

If an http function times out, it throws a socket received exception (SVC-SOCRECV).

An automatic encoding detector will be used if the value auto is specified for the encoding option . If no option is specified, the encoding defaults to the encoding specified in the http header. If there is no encoding in the http header, the encoding defaults to UTF-8.

The first node in the output of this function is the response header from the http server.

The second node in the output of this function is the response from the http server. The response is treated as text, XML, JSON or binary, depending on the content-type header sent from the http server. If the node is html, the header should indicate text/html, which is returned as a text document by default. The type of document is determined by the mimetypes mappings, and you can change the mappings in the Admin Interface as needed.

To use this function with a proxy, you need to translate the URI to the proxy uri. For example: 

function httpGetProxy(proxy, uri) {
var host = fn.subsequence(fn.tokenize(uri,'/'), 3, 3);
// you might need to modify the next line based on your proxy server config 
var proxyuri = fn.concat(proxy, uri);
return xdmp.httpGet(proxyuri,
               {headers:{host: host}});
};
  
httpGetProxy('http://some.proxy.com:8080','http://www.google.com');

If you use the credentialId option, you can use xdmp.credentialId to obtain the id of a previously stored credential. For example: 

     xdmp.httpGet(
       someuri,
       { credentialId: xdmp.credentialId('myCredentialName') })

Example

xdmp.httpGet("http://www.my.com/document.xhtml",
     {
       "authentication": {
         "method" : "basic",
         "username" : "myname",
         "password" : "mypassword"
       }
     })
=> the response from the server as well as the specified document

Example

xdmp.httpGet("https://s3.amazonaws.com/marklogic-lambda-us-east-1/",
     {
       "authentication": {
         "method" : "aws4",
         "username" : "myname",
         "password" : "mypassword"
       }
     })
=> the response from the server as well as the specified document

Example

fn.subsequence(
  xdmp.httpGet("http://www.marklogic.com",
     {
       "encoding" : "iso-8859-1"
     }), 2, 1)
=> The specified document, transcoded from ISO-8859-1
   to UTF-8 encoding.  This assumes the document is
   encoded in ISO-8859-1.
Powered by MarkLogic Server | Terms of Use | Privacy Policy