com.marklogic.client.datamovement

Class ZipSplitter

java.lang.Object

com.marklogic.client.datamovement.ZipSplitter

All Implemented Interfaces:

Splitter<BytesHandle>

public class ZipSplitter
extends java.lang.Object
implements Splitter<BytesHandle>

The ZipSplitter class is used to split compressed files. The zip file could include XML, JSON, TXT or BINARY files.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	ZipSplitter.UriMaker UriMaker which generates URI for each split file

Constructor Summary

Constructors

Constructor and Description
ZipSplitter() Create a new ZIP splitter.

Method Summary

Modifier and Type	Method and Description
long	getCount() Returns the number of splits.
java.util.function.Predicate<java.util.zip.ZipEntry>	getEntryFilter() Returns the entryFilter set to splitter.
java.util.Map<java.lang.String,Format>	getExtensionFormats() Returns the extensionFormats set to splitter.
ZipSplitter.UriMaker	getUriMaker() Get the UriMaker of the splitter
java.util.function.Function<java.lang.String,java.lang.String>	getUriTransformer() Returns the uriTransformer set to splitter.
void	setEntryFilter(java.util.function.Predicate<java.util.zip.ZipEntry> entryFilter) Used to set entryFilter to splitter.
void	setUriMaker(ZipSplitter.UriMaker uriMaker) Set the UriMaker to the splitter
void	setUriTransformer(java.util.function.Function<java.lang.String,java.lang.String> uriTransformer) Used to set uriTransformer to splitter.
java.util.stream.Stream<BytesHandle>	split(java.io.InputStream input) Takes a input stream of a ZIP file and convert it to a stream of BytesHandle.
java.util.stream.Stream<BytesHandle>	split(java.util.zip.ZipInputStream input) Takes a ZipInputStream of a ZIP file and convert it to a stream of BytesHandle.
java.util.stream.Stream<DocumentWriteOperation>	splitWriteOperations(java.io.InputStream input) Takes a input stream of a ZIP file and convert it to a stream of DocumentWriteOperation.
java.util.stream.Stream<DocumentWriteOperation>	splitWriteOperations(java.io.InputStream input, java.lang.String splitFilename) Takes a input stream and name of a ZIP file and convert it to a stream of DocumentWriteOperation.
java.util.stream.Stream<DocumentWriteOperation>	splitWriteOperations(java.util.zip.ZipInputStream input) Takes a ZipInputStream of a ZIP file and convert it to a stream of DocumentWriteOperation.
java.util.stream.Stream<DocumentWriteOperation>	splitWriteOperations(java.util.zip.ZipInputStream input, java.lang.String splitFilename) Takes a ZipInputStream and name of a ZIP file and convert it to a stream of DocumentWriteOperation.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ZipSplitter

public ZipSplitter()

Create a new ZIP splitter.

Method Detail

getExtensionFormats

public java.util.Map<java.lang.String,Format> getExtensionFormats()

Returns the extensionFormats set to splitter. The extensionFormat is pre-defined in splitter. It includes "json", "txt" and "xml" extensions. If the file has no extension, it is treated as binary file. You can also add mappings from other extensions in the zipfile to one of the four MarkLogic formats.

Returns:

a map of extensionFormats

getEntryFilter

public java.util.function.Predicate<java.util.zip.ZipEntry> getEntryFilter()

Returns the entryFilter set to splitter.

Returns:

the entryFilter set to splitter

setEntryFilter

public void setEntryFilter(java.util.function.Predicate<java.util.zip.ZipEntry> entryFilter)

Used to set entryFilter to splitter. The entryFilter is a lambda function, which can be used to inspect the zip entry and return false for any document in the zipfile that should be ignored.

Parameters:

entryFilter - the filter that applied to each zipEntry

getUriTransformer

public java.util.function.Function<java.lang.String,java.lang.String> getUriTransformer()

Returns the uriTransformer set to splitter.

Returns:

the uriTransformer set to splitter

setUriTransformer

public void setUriTransformer(java.util.function.Function<java.lang.String,java.lang.String> uriTransformer)

Used to set uriTransformer to splitter. The uriTransformer is a lambda function, which can be used to transform the name of the document in the zipfile into the document URI for the database.

Parameters:

uriTransformer - the uriTransformer which applied on each document URI

getCount

public long getCount()

Returns the number of splits.

Specified by:

getCount in interface Splitter<BytesHandle>

Returns:

the number of splits

split

public java.util.stream.Stream<BytesHandle> split(java.io.InputStream input)
                                           throws java.io.IOException

Takes a input stream of a ZIP file and convert it to a stream of BytesHandle. The input stream must be a ZipInputStream, otherwise it will throw an exception. The ZIP file could contain XML, JSON, TXT and BINARY files.

Specified by:

split in interface Splitter<BytesHandle>

Parameters:

input - is the incoming input stream

Returns:

a stream of BytesHandle

Throws:

java.io.IOException - if input cannot be split

split

public java.util.stream.Stream<BytesHandle> split(java.util.zip.ZipInputStream input)
                                           throws java.io.IOException

Takes a ZipInputStream of a ZIP file and convert it to a stream of BytesHandle. The ZIP file could contain XML, JSON, TXT and BINARY files.

Parameters:

input - is a ZipInputStream of a zip file

Returns:

a stream of BytesHandle

Throws:

java.io.IOException - if input cannot be split

splitWriteOperations

public java.util.stream.Stream<DocumentWriteOperation> splitWriteOperations(java.io.InputStream input)
                                                                     throws java.lang.Exception

Takes a input stream of a ZIP file and convert it to a stream of DocumentWriteOperation. The input stream must be a ZipInputStream, otherwise it will throw an exception. The ZIP file could contain XML, JSON, TXT and BINARY files.

Specified by:

splitWriteOperations in interface Splitter<BytesHandle>

Parameters:

input - is the incoming input stream.

Returns:

a stream of DocumentWriteOperation

Throws:

java.lang.Exception - if the input cannot be split

splitWriteOperations

public java.util.stream.Stream<DocumentWriteOperation> splitWriteOperations(java.io.InputStream input,
                                                                            java.lang.String splitFilename)
                                                                     throws java.lang.Exception

Takes a input stream and name of a ZIP file and convert it to a stream of DocumentWriteOperation. The input stream must be a ZipInputStream, otherwise it will throw an exception. The ZIP file could contain XML, JSON, TXT and BINARY files.

Specified by:

splitWriteOperations in interface Splitter<BytesHandle>

Parameters:

input - is the incoming input stream.

splitFilename - is the input file name, including name and extension. It is used to generate URLs for split files.The splitFilename could either be provided here or in user-defined UriMaker.

Returns:

a stream of DocumentWriteOperation

Throws:

java.lang.Exception - if the input cannot be split

splitWriteOperations

public java.util.stream.Stream<DocumentWriteOperation> splitWriteOperations(java.util.zip.ZipInputStream input)
                                                                     throws java.io.IOException

Takes a ZipInputStream of a ZIP file and convert it to a stream of DocumentWriteOperation. The ZIP file could contain XML, JSON, TXT and BINARY files.

Parameters:

input - is a ZipInputStream of a zip file

Returns:

a stream of DocumentWriteOperation

Throws:

java.io.IOException - if input cannot be split

splitWriteOperations

public java.util.stream.Stream<DocumentWriteOperation> splitWriteOperations(java.util.zip.ZipInputStream input,
                                                                            java.lang.String splitFilename)
                                                                     throws java.io.IOException

Takes a ZipInputStream and name of a ZIP file and convert it to a stream of DocumentWriteOperation. The ZIP file could contain XML, JSON, TXT and BINARY files.

Parameters:

input - is a ZipInputStream of a zip file

splitFilename - is the file name of input file, including name and extension. It is used to generate URLs for split files.The splitFilename could either be provided here or in user-defined UriMaker.

Returns:

a stream of DocumentWriteOperation

Throws:

java.io.IOException - if input cannot be split

getUriMaker

public ZipSplitter.UriMaker getUriMaker()

Get the UriMaker of the splitter

Returns:

the UriMaker of the splitter

setUriMaker

public void setUriMaker(ZipSplitter.UriMaker uriMaker)

Set the UriMaker to the splitter

Parameters:

uriMaker - the uriMaker to generate URI of each split file.