com.marklogic.client.datamovement

Class LineSplitter

java.lang.Object

com.marklogic.client.datamovement.LineSplitter

All Implemented Interfaces:

Splitter<StringHandle>

public class LineSplitter
extends java.lang.Object
implements Splitter<StringHandle>

The LineSplitter class is used to separate lines in line-delimited JSON, XML or TEXT files. It should also work with gzip-compressed line-delimited files.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	LineSplitter.UriMaker UriMaker which generates URI for each split file

Constructor Summary

Constructors

Constructor and Description
LineSplitter()

Method Summary

Modifier and Type	Method and Description
long	getCount() Used to return the number of objects in the stream.
Format	getFormat() Returns the document format set to splitter.
LineSplitter.UriMaker	getUriMaker() Get the UriMaker of the splitter
void	setFormat(Format format) Used to set document format to splitter.
void	setUriMaker(LineSplitter.UriMaker uriMaker) Set the UriMaker to the splitter
java.util.stream.Stream<StringHandle>	split(java.io.InputStream input) Takes the input stream and converts it into a stream of StringHandle.
java.util.stream.Stream<StringHandle>	split(java.io.InputStream input, java.nio.charset.Charset charset) Takes the input stream and converts it into a stream of StringHandle.
java.util.stream.Stream<StringHandle>	split(java.io.Reader input) Takes the Reader input and converts it into a stream of StringHandle.
java.util.stream.Stream<DocumentWriteOperation>	splitWriteOperations(java.io.InputStream input) Takes the input stream and converts it into a stream of DocumentWriteOperation.
java.util.stream.Stream<DocumentWriteOperation>	splitWriteOperations(java.io.InputStream input, java.lang.String splitFilename) Takes the input stream and input file name and converts it into a stream of DocumentWriteOperation.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LineSplitter

public LineSplitter()

Method Detail

getFormat

public Format getFormat()

Returns the document format set to splitter.

Returns:

the document format set to splitter. The default is Format.JSON.

setFormat

public void setFormat(Format format)

Used to set document format to splitter. The extension of URLs generated for splits is determined by this format. The format should be set before splitting. If not set, the default is JSON.

Parameters:

format - the document content format.

getCount

public long getCount()

Used to return the number of objects in the stream.

Specified by:

getCount in interface Splitter<StringHandle>

Returns:

the number of objects in the stream.

split

public java.util.stream.Stream<StringHandle> split(java.io.InputStream input)
                                            throws java.io.IOException

Takes the input stream and converts it into a stream of StringHandle. The content could be line-delimited JSON, XML or TEXT file. It could also be gzip-compressed line-delimited files. Provide GZIPInputStream to the splitter when splitting gzip files.

Specified by:

split in interface Splitter<StringHandle>

Parameters:

input - is the incoming input stream.

Returns:

a stream of StringHandle.

Throws:

java.io.IOException - if the input cannot be split

splitWriteOperations

public java.util.stream.Stream<DocumentWriteOperation> splitWriteOperations(java.io.InputStream input)
                                                                     throws java.lang.Exception

Takes the input stream and converts it into a stream of DocumentWriteOperation. The content could be line-delimited JSON, XML or TEXT file. It could also be gzip-compressed line-delimited files. Provide GZIPInputStream to the splitter when splitting gzip files.

Specified by:

splitWriteOperations in interface Splitter<StringHandle>

Parameters:

input - is the incoming input stream.

Returns:

a stream of DocumentWriteOperation.

Throws:

java.lang.Exception - if the input cannot be split

splitWriteOperations

public java.util.stream.Stream<DocumentWriteOperation> splitWriteOperations(java.io.InputStream input,
                                                                            java.lang.String splitFilename)
                                                                     throws java.lang.Exception

Takes the input stream and input file name and converts it into a stream of DocumentWriteOperation. The content could be line-delimited JSON, XML or TEXT file. It could also be gzip-compressed line-delimited files. Provide GZIPInputStream to the splitter when splitting gzip files.

Specified by:

splitWriteOperations in interface Splitter<StringHandle>

Parameters:

input - is the incoming input stream.

splitFilename - is the name of the input file, including name and extension. It is used to generate URLs for split files. The splitFilename could either be provided here or in user-defined UriMaker.

Returns:

a stream of DocumentWriteOperation.

Throws:

java.lang.Exception - if the input cannot be split

split

public java.util.stream.Stream<StringHandle> split(java.io.InputStream input,
                                                   java.nio.charset.Charset charset)
                                            throws java.io.IOException

Takes the input stream and converts it into a stream of StringHandle. The content could be line-delimited JSON file, line-delimited XML file or gzip-compressed line-delimited JSON file.

Parameters:

input - is the incoming input stream.

charset - is the encoding scheme the document uses.

Returns:

a stream of StringHandle.

Throws:

java.io.IOException - if the input cannot be split

split

public java.util.stream.Stream<StringHandle> split(java.io.Reader input)
                                            throws java.io.IOException

Takes the Reader input and converts it into a stream of StringHandle. The content could be line-delimited JSON file, line-delimited XML file or gzip-compressed line-delimited JSON file.

Parameters:

input - is the incoming Reader.

Returns:

a stream of StringHandle.

Throws:

java.io.IOException - if the input cannot be split

getUriMaker

public LineSplitter.UriMaker getUriMaker()

Get the UriMaker of the splitter

Returns:

the UriMaker of the splitter

setUriMaker

public void setUriMaker(LineSplitter.UriMaker uriMaker)

Set the UriMaker to the splitter

Parameters:

uriMaker - the uriMaker to generate URI of each split file.