Search in sources :

Example 16 with VectorContainerWriter

use of org.apache.drill.exec.vector.complex.impl.VectorContainerWriter in project drill by apache.

the class JSONRecordReader method setup.

@Override
public void setup(OperatorContext context, OutputMutator output) throws ExecutionSetupException {
    try {
        if (hadoopPath != null) {
            stream = fileSystem.openPossiblyCompressedStream(hadoopPath);
        }
        writer = new VectorContainerWriter(output, unionEnabled);
        if (isSkipQuery()) {
            jsonReader = new CountingJsonReader(fragmentContext.getManagedBuffer(), enableNanInf, enableEscapeAnyChar);
        } else {
            this.jsonReader = new JsonReader.Builder(fragmentContext.getManagedBuffer()).schemaPathColumns(ImmutableList.copyOf(getColumns())).allTextMode(enableAllTextMode).skipOuterList(true).readNumbersAsDouble(readNumbersAsDouble).enableNanInf(enableNanInf).enableEscapeAnyChar(enableEscapeAnyChar).build();
        }
        setupParser();
    } catch (Exception e) {
        handleAndRaise("Failure reading JSON file", e);
    }
}
Also used : VectorContainerWriter(org.apache.drill.exec.vector.complex.impl.VectorContainerWriter) CountingJsonReader(org.apache.drill.exec.store.easy.json.reader.CountingJsonReader) UserException(org.apache.drill.common.exceptions.UserException) OutOfMemoryException(org.apache.drill.exec.exception.OutOfMemoryException) ExecutionSetupException(org.apache.drill.common.exceptions.ExecutionSetupException) JsonParseException(com.fasterxml.jackson.core.JsonParseException) IOException(java.io.IOException)

Aggregations

VectorContainerWriter (org.apache.drill.exec.vector.complex.impl.VectorContainerWriter)16 ExecutionSetupException (org.apache.drill.common.exceptions.ExecutionSetupException)6 IOException (java.io.IOException)4 TestOutputMutator (org.apache.drill.exec.store.TestOutputMutator)4 UserException (org.apache.drill.common.exceptions.UserException)3 BufferAllocator (org.apache.drill.exec.memory.BufferAllocator)3 Test (org.junit.Test)3 JsonParseException (com.fasterxml.jackson.core.JsonParseException)2 DBException (com.mapr.db.exceptions.DBException)2 OutOfMemoryException (org.apache.drill.exec.exception.OutOfMemoryException)2 BsonRecordReader (org.apache.drill.exec.store.bson.BsonRecordReader)2 CountingJsonReader (org.apache.drill.exec.store.easy.json.reader.CountingJsonReader)2 ComplexWriter (org.apache.drill.exec.vector.complex.writer.BaseWriter.ComplexWriter)2 MapWriter (org.apache.drill.exec.vector.complex.writer.BaseWriter.MapWriter)2 DBDocumentReaderBase (com.mapr.db.ojai.DBDocumentReaderBase)1 DrillBuf (io.netty.buffer.DrillBuf)1 SchemaChangeException (org.apache.drill.exec.exception.SchemaChangeException)1 RootAllocator (org.apache.drill.exec.memory.RootAllocator)1 BufferManagerImpl (org.apache.drill.exec.ops.BufferManagerImpl)1 OperatorContext (org.apache.drill.exec.ops.OperatorContext)1