com.marklogic.client.datamovement

Class JSONSplitter<T extends JSONWriteHandle>

java.lang.Object

com.marklogic.client.datamovement.JSONSplitter<T>

Type Parameters:

T - The type of the handle used for each split payload

All Implemented Interfaces:

Splitter<T>

public class JSONSplitter<T extends JSONWriteHandle>
extends java.lang.Object
implements Splitter<T>

The JSONSplitter is used to split large JSON file into separate payloads for writing to the database. The JSON file is typically an array containing an object for each record. JSONSplitter could split each targeted object or array into separate files.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	JSONSplitter.ArrayVisitor The basic visitor only splits objects or arrays under top array.
static interface	JSONSplitter.UriMaker UriMaker which generates URI for each split file
static class	JSONSplitter.Visitor<T extends AbstractWriteHandle> The Visitor class is used to accumulate and inspect state during the depth-first traversal of the JSON tree and make the decision of how to split the JSON file.

Constructor Summary

Constructors

Constructor and Description
JSONSplitter(JSONSplitter.Visitor<T> visitor) Construct a JSONSplitter which splits the JSON file according to the visitor.

Method Summary

Modifier and Type	Method and Description
long	getCount() Returns the number of splits.
JSONSplitter.UriMaker	getUriMaker() Get the UriMaker of the splitter
JSONSplitter.Visitor<T>	getVisitor() Get the visitor used in JSONSplitter class.
static JSONSplitter<StringHandle>	makeArraySplitter() Construct a simple JSONSplitter which split objects or arrays under an array.
void	setUriMaker(JSONSplitter.UriMaker uriMaker) Set the UriMaker to the splitter
void	setVisitor(JSONSplitter.Visitor<T> visitor) Set the visitor to select objects or arrays to split in JSONSplitter.
java.util.stream.Stream<T>	split(java.io.InputStream input) Takes an InputStream of a JSON file and split it into a steam of handles.
java.util.stream.Stream<T>	split(com.fasterxml.jackson.core.JsonParser input) Take an input of JsonParser created from the JSON file and split it into a stream of handles to write to database.
java.util.stream.Stream<DocumentWriteOperation>	splitWriteOperations(java.io.InputStream input) Takes an InputStream of a JSON file and split it into a steam of DocumentWriteOperation to write to database.
java.util.stream.Stream<DocumentWriteOperation>	splitWriteOperations(java.io.InputStream input, java.lang.String splitFilename) Takes an InputStream of a JSON file and file name and split it into a steam of DocumentWriteOperation to write to database.
java.util.stream.Stream<DocumentWriteOperation>	splitWriteOperations(com.fasterxml.jackson.core.JsonParser input, java.lang.String splitFilename) Take an input of JsonParser created from the JSON file and split it into a stream of DocumentWriteOperations to write to database.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JSONSplitter

public JSONSplitter(JSONSplitter.Visitor<T> visitor)

Construct a JSONSplitter which splits the JSON file according to the visitor.

Parameters:

visitor - describes how to spit the file

Method Detail

makeArraySplitter

public static JSONSplitter<StringHandle> makeArraySplitter()

Construct a simple JSONSplitter which split objects or arrays under an array.

Returns:

a JSONSplitter class which splits each object or array into a separate payload

getVisitor

public JSONSplitter.Visitor<T> getVisitor()

Get the visitor used in JSONSplitter class.

Returns:

the visitor used in JSONSplitter class

setVisitor

public void setVisitor(JSONSplitter.Visitor<T> visitor)

Set the visitor to select objects or arrays to split in JSONSplitter.

Parameters:

visitor - the visitor describes the rule to split the JSON file

getCount

public long getCount()

Returns the number of splits.

Specified by:

getCount in interface Splitter<T extends JSONWriteHandle>

Returns:

the number of splits

split

public java.util.stream.Stream<T> split(java.io.InputStream input)
                                 throws java.io.IOException

Takes an InputStream of a JSON file and split it into a steam of handles.

Specified by:

split in interface Splitter<T extends JSONWriteHandle>

Parameters:

input - is the incoming InputStream of a JSON file.

Returns:

a stream of handles to write to database

Throws:

java.io.IOException - if the input cannot be split

splitWriteOperations

public java.util.stream.Stream<DocumentWriteOperation> splitWriteOperations(java.io.InputStream input)
                                                                     throws java.lang.Exception

Takes an InputStream of a JSON file and split it into a steam of DocumentWriteOperation to write to database.

Specified by:

splitWriteOperations in interface Splitter<T extends JSONWriteHandle>

Parameters:

input - is the incoming input stream of a JSON file

Returns:

a stream of DocumentWriteOperation to write to database

Throws:

java.lang.Exception - if the input cannot be split

splitWriteOperations

public java.util.stream.Stream<DocumentWriteOperation> splitWriteOperations(java.io.InputStream input,
                                                                            java.lang.String splitFilename)
                                                                     throws java.lang.Exception

Takes an InputStream of a JSON file and file name and split it into a steam of DocumentWriteOperation to write to database.

Specified by:

splitWriteOperations in interface Splitter<T extends JSONWriteHandle>

Parameters:

input - is the incoming input stream of a JSON file

splitFilename - is the name of input file, including name and extension. It is used to generate URLs for split files.The splitFilename could either be provided here or in user-defined UriMaker.

Returns:

a stream of DocumentWriteOperation to write to database

Throws:

java.lang.Exception - if the input cannot be split

split

public java.util.stream.Stream<T> split(com.fasterxml.jackson.core.JsonParser input)
                                 throws java.io.IOException

Take an input of JsonParser created from the JSON file and split it into a stream of handles to write to database.

Parameters:

input - JsonParser created from the JSON file

Returns:

a stream of handles to write to database

Throws:

java.io.IOException - if the input cannot be split

splitWriteOperations

public java.util.stream.Stream<DocumentWriteOperation> splitWriteOperations(com.fasterxml.jackson.core.JsonParser input,
                                                                            java.lang.String splitFilename)

Take an input of JsonParser created from the JSON file and split it into a stream of DocumentWriteOperations to write to database.

Parameters:

input - JsonParser created from the JSON file

splitFilename - is the name of input file, including name and extension. It is used to generate URLs for split files.The splitFilename could either be provided here or in user-defined UriMaker.

Returns:

a stream of DocumentWriteOperation to write to database

getUriMaker

public JSONSplitter.UriMaker getUriMaker()

Get the UriMaker of the splitter

Returns:

the UriMaker of the splitter

setUriMaker

public void setUriMaker(JSONSplitter.UriMaker uriMaker)

Set the UriMaker to the splitter

Parameters:

uriMaker - the uriMaker to generate URI of each split file.