VALUEOUT - public class ContentOutputFormat<VALUEOUT> extends MarkLogicOutputFormat<DocumentURI,VALUEOUT>
Use this class to store results as content in a MarkLogic Server database.
The text, XML, or binary content is inserted into the database at the
given DocumentURI.
When using this MarkLogicOutputFormat, your key should be the URI of
the document to insert into the database. The value should be the content to
insert, in the form of Text or
MarkLogicNode.
Several configuration properties exist for controlling the content insertion, including permissions, collections, quality, directory, and content type.
MarkLogicConstants,
ContentLoader,
ZipContentLoader| Modifier and Type | Field and Description |
|---|---|
protected boolean |
allowFastLoad
whether stats-based policy allows fastload
|
protected AssignmentManager |
am |
protected boolean |
fastLoad |
static String |
FOREST_HOST_QUERY |
static String |
ID_PREFIX |
static String |
INIT_QUERY |
protected String |
initHostName |
protected boolean |
legacy |
static org.apache.commons.logging.Log |
LOG |
protected AssignmentPolicy.Kind |
policy |
conf, HOSTS_QUERYADVANCED_MODE, ASSIGNMENT_POLICY, BASIC_MODE, BATCH_SIZE, BIND_SPLIT_RANGE, COLLECTION_FILTER, CONTENT_TYPE, DEFAULT_BATCH_SIZE, DEFAULT_CONTENT_TYPE, DEFAULT_MAX_SPLIT_SIZE, DEFAULT_OUTPUT_CONTENT_ENCODING, DEFAULT_OUTPUT_XML_REPAIR_LEVEL, DEFAULT_PROPERTY_OPERATION_TYPE, DIRECTORY_FILTER, DOCUMENT_SELECTOR, EXECUTION_MODE, INDENTED, INPUT_DATABASE_NAME, INPUT_HOST, INPUT_KEY_CLASS, INPUT_LEXICON_FUNCTION_CLASS, INPUT_MODE, INPUT_PASSWORD, INPUT_PORT, INPUT_QUERY, INPUT_QUERY_TIMESTAMP, INPUT_SSL_OPTIONS_CLASS, INPUT_USE_SSL, INPUT_USERNAME, INPUT_VALUE_CLASS, MAX_SPLIT_SIZE, MODE_DISTRIBUTED, MODE_LOCAL, MR_NAMESPACE, NODE_OPERATION_TYPE, OUTPUT_CLEAN_DIR, OUTPUT_COLLECTION, OUTPUT_CONTENT_ENCODING, OUTPUT_CONTENT_LANGUAGE, OUTPUT_CONTENT_NAMESPACE, OUTPUT_DIRECTORY, OUTPUT_FAST_LOAD, OUTPUT_FOREST_HOST, OUTPUT_HOST, OUTPUT_KEY_TYPE, OUTPUT_KEY_VARNAME, OUTPUT_NAMESPACE, OUTPUT_PARTITION, OUTPUT_PASSWORD, OUTPUT_PERMISSION, OUTPUT_PORT, OUTPUT_PROPERTY_ALWAYS_CREATE, OUTPUT_QUALITY, OUTPUT_QUERY, OUTPUT_SSL_OPTIONS_CLASS, OUTPUT_STREAMING, OUTPUT_TOLERATE_ERRORS, OUTPUT_USE_SSL, OUTPUT_USERNAME, OUTPUT_VALUE_TYPE, OUTPUT_VALUE_VARNAME, OUTPUT_XML_REPAIR_LEVEL, PATH_NAMESPACE, PROPERTY_OPERATION_TYPE, RECORD_TO_FRAGMENT_RATIO, SPLIT_END_VARNAME, SPLIT_QUERY, SPLIT_START_VARNAME, SUBDOCUMENT_EXPRESSION, TXN_SIZE, TYPE_FILTER| Constructor and Description |
|---|
ContentOutputFormat() |
| Modifier and Type | Method and Description |
|---|---|
void |
checkOutputSpecs(org.apache.hadoop.conf.Configuration conf,
com.marklogic.xcc.ContentSource cs) |
protected LinkedMapWritable |
getForestStatusMap(org.apache.hadoop.conf.Configuration conf) |
org.apache.hadoop.mapreduce.RecordWriter<DocumentURI,VALUEOUT> |
getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) |
protected Map<String,com.marklogic.xcc.ContentSource> |
getSourceMap(boolean fastLoad,
org.apache.hadoop.mapreduce.TaskAttemptContext context) |
protected void |
initialize(com.marklogic.xcc.Session session)
Initialize initial server host name, assignment policy and fastload.
|
protected LinkedMapWritable |
queryForestInfo(com.marklogic.xcc.ContentSource cs)
must be attempting or doing fastload when this method is called.
|
checkOutputSpecs, getConf, getHosts, getOutputCommitter, queryHosts, queryHosts, setConfpublic static final org.apache.commons.logging.Log LOG
public static final String ID_PREFIX
public static final String FOREST_HOST_QUERY
public static final String INIT_QUERY
protected AssignmentManager am
protected boolean fastLoad
protected boolean allowFastLoad
protected AssignmentPolicy.Kind policy
protected boolean legacy
protected String initHostName
public void checkOutputSpecs(org.apache.hadoop.conf.Configuration conf,
com.marklogic.xcc.ContentSource cs)
throws IOException
checkOutputSpecs in class MarkLogicOutputFormat<DocumentURI,VALUEOUT>IOExceptionprotected Map<String,com.marklogic.xcc.ContentSource> getSourceMap(boolean fastLoad, org.apache.hadoop.mapreduce.TaskAttemptContext context) throws IOException
IOExceptionpublic org.apache.hadoop.mapreduce.RecordWriter<DocumentURI,VALUEOUT> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) throws IOException, InterruptedException
getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<DocumentURI,VALUEOUT>IOExceptionInterruptedExceptionprotected LinkedMapWritable getForestStatusMap(org.apache.hadoop.conf.Configuration conf) throws IOException
IOExceptionprotected void initialize(com.marklogic.xcc.Session session)
throws IOException,
com.marklogic.xcc.exceptions.RequestException
session - IOExceptioncom.marklogic.xcc.exceptions.RequestExceptionprotected LinkedMapWritable queryForestInfo(com.marklogic.xcc.ContentSource cs) throws IOException
cs - IOExceptionCopyright © 2016 MarkLogic Corporation. All Rights Reserved.
Complete online documentation for MarkLogic Server, XQuery and related components may be found at developer.marklogic.com