Search in sources :

Example 1 with ScanLifecycleBuilder

use of org.apache.drill.exec.physical.impl.scan.v3.ScanLifecycleBuilder in project drill by apache.

the class ReaderLifecycle method buildLoader.

public ResultSetLoader buildLoader() {
    Preconditions.checkState(state == State.START);
    ScanLifecycleBuilder scanOptions = scanOptions();
    ResultSetOptionBuilder options = new ResultSetOptionBuilder().rowCountLimit(Math.min(schemaNegotiator.batchSize, scanOptions.scanBatchRecordLimit())).vectorCache(scanLifecycle.vectorCache()).batchSizeLimit(scanOptions.scanBatchByteLimit()).errorContext(errorContext()).projectionFilter(schemaTracker().projectionFilter(errorContext())).readerSchema(schemaNegotiator.readerSchema).limit(limit);
    // Resolve the scan schema if possible.
    applyEarlySchema();
    // Create the table loader
    tableLoader = new ResultSetLoaderImpl(scanLifecycle.allocator(), options.build());
    state = State.DATA;
    return tableLoader;
}
Also used : ResultSetOptionBuilder(org.apache.drill.exec.physical.resultSet.impl.ResultSetOptionBuilder) ResultSetLoaderImpl(org.apache.drill.exec.physical.resultSet.impl.ResultSetLoaderImpl) ScanLifecycleBuilder(org.apache.drill.exec.physical.impl.scan.v3.ScanLifecycleBuilder)

Example 2 with ScanLifecycleBuilder

use of org.apache.drill.exec.physical.impl.scan.v3.ScanLifecycleBuilder in project drill by apache.

the class TestReaderErrors method testCtorUserError.

@Test
public void testCtorUserError() {
    ScanLifecycleBuilder builder = new ScanLifecycleBuilder();
    builder.errorContext(b -> b.addContext("Scan context"));
    builder.readerFactory(new SingleReaderFactory() {

        @Override
        public ManagedReader next(SchemaNegotiator negotiator) {
            return new FailingReader(negotiator, "ctor-u");
        }
    });
    ScanLifecycle scan = buildScan(builder);
    RowBatchReader reader = scan.nextReader();
    try {
        reader.open();
        fail();
    } catch (UserException e) {
        String msg = e.getMessage();
        assertTrue(msg.contains("Oops ctor"));
        assertTrue(msg.contains("My custom context"));
        assertTrue(msg.contains("Scan context"));
        assertNull(e.getCause());
    }
    scan.close();
}
Also used : RowBatchReader(org.apache.drill.exec.physical.impl.scan.RowBatchReader) ManagedReader(org.apache.drill.exec.physical.impl.scan.v3.ManagedReader) SchemaNegotiator(org.apache.drill.exec.physical.impl.scan.v3.SchemaNegotiator) UserException(org.apache.drill.common.exceptions.UserException) ScanLifecycleBuilder(org.apache.drill.exec.physical.impl.scan.v3.ScanLifecycleBuilder) EvfTest(org.apache.drill.categories.EvfTest) Test(org.junit.Test)

Example 3 with ScanLifecycleBuilder

use of org.apache.drill.exec.physical.impl.scan.v3.ScanLifecycleBuilder in project drill by apache.

the class TestReaderErrors method testNextUserError.

@Test
public void testNextUserError() {
    ScanLifecycleBuilder builder = new ScanLifecycleBuilder();
    builder.errorContext(b -> b.addContext("Scan context"));
    builder.readerFactory(new SingleReaderFactory() {

        @Override
        public ManagedReader next(SchemaNegotiator negotiator) {
            return new FailingReader(negotiator, "next-u");
        }
    });
    ScanLifecycle scan = buildScan(builder);
    RowBatchReader reader = scan.nextReader();
    assertTrue(reader.open());
    try {
        reader.next();
        fail();
    } catch (UserException e) {
        String msg = e.getMessage();
        assertTrue(msg.contains("Oops next"));
        assertTrue(msg.contains("My custom context"));
        assertTrue(msg.contains("Scan context"));
        assertNull(e.getCause());
    }
    scan.close();
}
Also used : RowBatchReader(org.apache.drill.exec.physical.impl.scan.RowBatchReader) ManagedReader(org.apache.drill.exec.physical.impl.scan.v3.ManagedReader) SchemaNegotiator(org.apache.drill.exec.physical.impl.scan.v3.SchemaNegotiator) UserException(org.apache.drill.common.exceptions.UserException) ScanLifecycleBuilder(org.apache.drill.exec.physical.impl.scan.v3.ScanLifecycleBuilder) EvfTest(org.apache.drill.categories.EvfTest) Test(org.junit.Test)

Example 4 with ScanLifecycleBuilder

use of org.apache.drill.exec.physical.impl.scan.v3.ScanLifecycleBuilder in project drill by apache.

the class TestReaderErrors method testNextError.

@Test
public void testNextError() {
    ScanLifecycleBuilder builder = new ScanLifecycleBuilder();
    builder.errorContext(b -> b.addContext("Scan context"));
    builder.readerFactory(new SingleReaderFactory() {

        @Override
        public ManagedReader next(SchemaNegotiator negotiator) {
            return new FailingReader(negotiator, "next");
        }
    });
    ScanLifecycle scan = buildScan(builder);
    RowBatchReader reader = scan.nextReader();
    assertTrue(reader.open());
    try {
        reader.next();
        fail();
    } catch (UserException e) {
        String msg = e.getMessage();
        assertTrue(msg.contains("Oops next"));
        assertTrue(msg.contains("My custom context"));
        assertTrue(msg.contains("Scan context"));
        assertTrue(e.getCause() instanceof IllegalStateException);
    }
    scan.close();
}
Also used : RowBatchReader(org.apache.drill.exec.physical.impl.scan.RowBatchReader) ManagedReader(org.apache.drill.exec.physical.impl.scan.v3.ManagedReader) SchemaNegotiator(org.apache.drill.exec.physical.impl.scan.v3.SchemaNegotiator) UserException(org.apache.drill.common.exceptions.UserException) ScanLifecycleBuilder(org.apache.drill.exec.physical.impl.scan.v3.ScanLifecycleBuilder) EvfTest(org.apache.drill.categories.EvfTest) Test(org.junit.Test)

Example 5 with ScanLifecycleBuilder

use of org.apache.drill.exec.physical.impl.scan.v3.ScanLifecycleBuilder in project drill by apache.

the class TestReaderErrors method testCloseUserError.

@Test
public void testCloseUserError() {
    ScanLifecycleBuilder builder = new ScanLifecycleBuilder();
    builder.errorContext(b -> b.addContext("Scan context"));
    builder.readerFactory(new SingleReaderFactory() {

        @Override
        public ManagedReader next(SchemaNegotiator negotiator) {
            return new FailingReader(negotiator, "close-u");
        }
    });
    ScanLifecycle scan = buildScan(builder);
    RowBatchReader reader = scan.nextReader();
    assertTrue(reader.open());
    assertFalse(reader.next());
    // to update an external system.
    try {
        reader.close();
        fail();
    } catch (UserException e) {
        String msg = e.getMessage();
        assertTrue(msg.contains("Oops close"));
        assertTrue(msg.contains("My custom context"));
        assertTrue(msg.contains("Scan context"));
        assertNull(e.getCause());
    }
    scan.close();
}
Also used : RowBatchReader(org.apache.drill.exec.physical.impl.scan.RowBatchReader) ManagedReader(org.apache.drill.exec.physical.impl.scan.v3.ManagedReader) SchemaNegotiator(org.apache.drill.exec.physical.impl.scan.v3.SchemaNegotiator) UserException(org.apache.drill.common.exceptions.UserException) ScanLifecycleBuilder(org.apache.drill.exec.physical.impl.scan.v3.ScanLifecycleBuilder) EvfTest(org.apache.drill.categories.EvfTest) Test(org.junit.Test)

Aggregations

ScanLifecycleBuilder (org.apache.drill.exec.physical.impl.scan.v3.ScanLifecycleBuilder)45 SchemaNegotiator (org.apache.drill.exec.physical.impl.scan.v3.SchemaNegotiator)43 EvfTest (org.apache.drill.categories.EvfTest)42 ManagedReader (org.apache.drill.exec.physical.impl.scan.v3.ManagedReader)42 Test (org.junit.Test)42 RowBatchReader (org.apache.drill.exec.physical.impl.scan.RowBatchReader)37 RowSet (org.apache.drill.exec.physical.rowSet.RowSet)18 UserException (org.apache.drill.common.exceptions.UserException)11 SchemaBuilder (org.apache.drill.exec.record.metadata.SchemaBuilder)9 TupleMetadata (org.apache.drill.exec.record.metadata.TupleMetadata)9 EarlyEofException (org.apache.drill.exec.physical.impl.scan.v3.ManagedReader.EarlyEofException)1 ResultSetLoaderImpl (org.apache.drill.exec.physical.resultSet.impl.ResultSetLoaderImpl)1 ResultSetOptionBuilder (org.apache.drill.exec.physical.resultSet.impl.ResultSetOptionBuilder)1 VectorContainer (org.apache.drill.exec.record.VectorContainer)1