Examples with SequenceFileAsBinaryInputFormat org.apache.hadoop.mapred.SequenceFileAsBinaryInputFormat used on opensource projects

Example 1 with SequenceFileAsBinaryInputFormat

use of org.apache.hadoop.mapred.SequenceFileAsBinaryInputFormat in project drill by apache.

the class SequenceFileRecordReader method setup.

@Override
public void setup(OperatorContext context, OutputMutator output) throws ExecutionSetupException {
    final SequenceFileAsBinaryInputFormat inputFormat = new SequenceFileAsBinaryInputFormat();
    final JobConf jobConf = new JobConf(dfs.getConf());
    jobConf.setInputFormat(inputFormat.getClass());
    reader = getRecordReader(inputFormat, jobConf);
    final MaterializedField keyField = MaterializedField.create(keySchema, KEY_TYPE);
    final MaterializedField valueField = MaterializedField.create(valueSchema, VALUE_TYPE);
    try {
        keyVector = output.addField(keyField, NullableVarBinaryVector.class);
        valueVector = output.addField(valueField, NullableVarBinaryVector.class);
    } catch (SchemaChangeException sce) {
        throw new ExecutionSetupException("Error in setting up sequencefile reader.", sce);
    }
}

Example 2 with SequenceFileAsBinaryInputFormat

use of org.apache.hadoop.mapred.SequenceFileAsBinaryInputFormat in project drill by axbaretto.

the class SequenceFileRecordReader method setup.

@Override
public void setup(OperatorContext context, OutputMutator output) throws ExecutionSetupException {
    final SequenceFileAsBinaryInputFormat inputFormat = new SequenceFileAsBinaryInputFormat();
    final JobConf jobConf = new JobConf(dfs.getConf());
    jobConf.setInputFormat(inputFormat.getClass());
    reader = getRecordReader(inputFormat, jobConf);
    final MaterializedField keyField = MaterializedField.create(keySchema, KEY_TYPE);
    final MaterializedField valueField = MaterializedField.create(valueSchema, VALUE_TYPE);
    try {
        keyVector = output.addField(keyField, NullableVarBinaryVector.class);
        valueVector = output.addField(valueField, NullableVarBinaryVector.class);
    } catch (SchemaChangeException sce) {
        throw new ExecutionSetupException("Error in setting up sequencefile reader.", sce);
    }
}

Example 3 with SequenceFileAsBinaryInputFormat

use of org.apache.hadoop.mapred.SequenceFileAsBinaryInputFormat in project drill by apache.

the class SequenceFileBatchReader method processReader.

private void processReader(FileSchemaNegotiator negotiator) throws ExecutionSetupException {
    final SequenceFileAsBinaryInputFormat inputFormat = new SequenceFileAsBinaryInputFormat();
    split = negotiator.split();
    // After defined the split, We should also define the errorContext.
    errorContext = negotiator.parentErrorContext();
    opUserName = scan.getUserName();
    queryUserName = negotiator.context().getFragmentContext().getQueryUserName();
    final JobConf jobConf = new JobConf(negotiator.fileSystem().getConf());
    jobConf.setInputFormat(inputFormat.getClass());
    reader = getRecordReader(inputFormat, jobConf);
}