Examples with CarbonVectorizedRecordReader org.apache.carbondata.hadoop.util.CarbonVectorizedRecordReader used on opensource projects

Example 1 with CarbonVectorizedRecordReader

use of org.apache.carbondata.hadoop.util.CarbonVectorizedRecordReader in project carbondata by apache.

the class CarbonReaderBuilder method getRecordReader.

private <T> RecordReader getRecordReader(Job job, CarbonFileInputFormat format, List<RecordReader<Void, T>> readers, InputSplit split) throws IOException, InterruptedException {
    TaskAttemptContextImpl attempt = new TaskAttemptContextImpl(job.getConfiguration(), new TaskAttemptID());
    RecordReader reader;
    QueryModel queryModel = format.createQueryModel(split, attempt);
    boolean hasComplex = false;
    for (ProjectionDimension projectionDimension : queryModel.getProjectionDimensions()) {
        if (projectionDimension.getDimension().isComplex()) {
            hasComplex = true;
            break;
        }
    }
    if (useVectorReader && !hasComplex) {
        queryModel.setDirectVectorFill(filterExpression == null);
        reader = new CarbonVectorizedRecordReader(queryModel);
    } else {
        reader = format.createRecordReader(split, attempt);
    }
    try {
        reader.initialize(split, attempt);
    } catch (Exception e) {
        CarbonUtil.closeStreams(readers.toArray(new RecordReader[0]));
        throw e;
    }
    return reader;
}