Search in sources :

Example 16 with UserException

use of org.apache.drill.common.exceptions.UserException in project drill by apache.

the class TestResultSetLoaderMaps method testBasics.

@Test
public void testBasics() {
    final TupleMetadata schema = new SchemaBuilder().add("a", MinorType.INT).addMap("m").add("c", MinorType.INT).add("d", MinorType.VARCHAR).resumeSchema().add("e", MinorType.VARCHAR).buildSchema();
    final ResultSetLoaderImpl.ResultSetOptions options = new ResultSetOptionBuilder().readerSchema(schema).build();
    final ResultSetLoader rsLoader = new ResultSetLoaderImpl(fixture.allocator(), options);
    assertFalse(rsLoader.isProjectionEmpty());
    final RowSetLoader rootWriter = rsLoader.writer();
    // Verify structure and schema
    assertEquals(5, rsLoader.schemaVersion());
    final TupleMetadata actualSchema = rootWriter.tupleSchema();
    assertEquals(3, actualSchema.size());
    assertTrue(actualSchema.metadata(1).isMap());
    assertEquals(2, actualSchema.metadata("m").tupleSchema().size());
    assertEquals(2, actualSchema.column("m").getChildren().size());
    rsLoader.startBatch();
    // Write a row the way that clients will do.
    final ScalarWriter aWriter = rootWriter.scalar("a");
    final TupleWriter mWriter = rootWriter.tuple("m");
    final ScalarWriter cWriter = mWriter.scalar("c");
    final ScalarWriter dWriter = mWriter.scalar("d");
    final ScalarWriter eWriter = rootWriter.scalar("e");
    rootWriter.start();
    aWriter.setInt(10);
    cWriter.setInt(110);
    dWriter.setString("fred");
    eWriter.setString("pebbles");
    rootWriter.save();
    // Try adding a duplicate column.
    try {
        mWriter.addColumn(SchemaBuilder.columnSchema("c", MinorType.INT, DataMode.OPTIONAL));
        fail();
    } catch (final UserException e) {
    // Expected
    }
    // Write another using the test-time conveniences
    rootWriter.addRow(20, mapValue(210, "barney"), "bam-bam");
    // Harvest the batch
    final RowSet actual = fixture.wrap(rsLoader.harvest());
    assertEquals(5, rsLoader.schemaVersion());
    assertEquals(2, actual.rowCount());
    final MapVector mapVector = (MapVector) actual.container().getValueVector(1).getValueVector();
    assertEquals(2, mapVector.getAccessor().getValueCount());
    // Validate data
    final SingleRowSet expected = fixture.rowSetBuilder(schema).addRow(10, mapValue(110, "fred"), "pebbles").addRow(20, mapValue(210, "barney"), "bam-bam").build();
    RowSetUtilities.verify(expected, actual);
    rsLoader.close();
}
Also used : SingleRowSet(org.apache.drill.exec.physical.rowSet.RowSet.SingleRowSet) SingleRowSet(org.apache.drill.exec.physical.rowSet.RowSet.SingleRowSet) RowSet(org.apache.drill.exec.physical.rowSet.RowSet) ResultSetLoader(org.apache.drill.exec.physical.resultSet.ResultSetLoader) TupleWriter(org.apache.drill.exec.vector.accessor.TupleWriter) TupleMetadata(org.apache.drill.exec.record.metadata.TupleMetadata) SchemaBuilder(org.apache.drill.exec.record.metadata.SchemaBuilder) UserException(org.apache.drill.common.exceptions.UserException) RowSetLoader(org.apache.drill.exec.physical.resultSet.RowSetLoader) ScalarWriter(org.apache.drill.exec.vector.accessor.ScalarWriter) MapVector(org.apache.drill.exec.vector.complex.MapVector) SubOperatorTest(org.apache.drill.test.SubOperatorTest) Test(org.junit.Test)

Example 17 with UserException

use of org.apache.drill.common.exceptions.UserException in project drill by apache.

the class TestResultSetLoaderProjection method testScalarMapArrayConflict.

@Test
public void testScalarMapArrayConflict() {
    List<SchemaPath> selection = RowSetTestUtils.projectList("col[0].child");
    TupleMetadata schema = new SchemaBuilder().add("col", MinorType.VARCHAR).buildSchema();
    ResultSetOptions options = new ResultSetOptionBuilder().projection(Projections.parse(selection)).readerSchema(schema).build();
    try {
        new ResultSetLoaderImpl(fixture.allocator(), options);
        fail();
    } catch (UserException e) {
        assertTrue(e.getErrorType() == ErrorType.VALIDATION);
    }
}
Also used : SchemaPath(org.apache.drill.common.expression.SchemaPath) TupleMetadata(org.apache.drill.exec.record.metadata.TupleMetadata) SchemaBuilder(org.apache.drill.exec.record.metadata.SchemaBuilder) UserException(org.apache.drill.common.exceptions.UserException) ResultSetOptions(org.apache.drill.exec.physical.resultSet.impl.ResultSetLoaderImpl.ResultSetOptions) SubOperatorTest(org.apache.drill.test.SubOperatorTest) EvfTest(org.apache.drill.categories.EvfTest) Test(org.junit.Test)

Example 18 with UserException

use of org.apache.drill.common.exceptions.UserException in project drill by apache.

the class TestResultSetLoaderProjection method testMapMapArrayConflict.

@Test
public void testMapMapArrayConflict() {
    List<SchemaPath> selection = RowSetTestUtils.projectList("col[0].child");
    TupleMetadata schema = new SchemaBuilder().addMap("col").add("child", MinorType.VARCHAR).resumeSchema().buildSchema();
    ResultSetOptions options = new ResultSetOptionBuilder().projection(Projections.parse(selection)).readerSchema(schema).build();
    try {
        new ResultSetLoaderImpl(fixture.allocator(), options);
        fail();
    } catch (UserException e) {
        assertTrue(e.getErrorType() == ErrorType.VALIDATION);
    }
}
Also used : SchemaPath(org.apache.drill.common.expression.SchemaPath) TupleMetadata(org.apache.drill.exec.record.metadata.TupleMetadata) SchemaBuilder(org.apache.drill.exec.record.metadata.SchemaBuilder) UserException(org.apache.drill.common.exceptions.UserException) ResultSetOptions(org.apache.drill.exec.physical.resultSet.impl.ResultSetLoaderImpl.ResultSetOptions) SubOperatorTest(org.apache.drill.test.SubOperatorTest) EvfTest(org.apache.drill.categories.EvfTest) Test(org.junit.Test)

Example 19 with UserException

use of org.apache.drill.common.exceptions.UserException in project drill by apache.

the class TestGracefulShutdown method testDrillbitWithSamePortContainsShutdownThread.

// DRILL-6912
@Test
public void testDrillbitWithSamePortContainsShutdownThread() throws Exception {
    ClusterFixtureBuilder fixtureBuilder = ClusterFixture.builder(dirTestWatcher).withLocalZk().configProperty(ExecConstants.ALLOW_LOOPBACK_ADDRESS_BINDING, true).configProperty(ExecConstants.INITIAL_USER_PORT, QueryTestUtil.getFreePortNumber(31170, 300)).configProperty(ExecConstants.INITIAL_BIT_PORT, QueryTestUtil.getFreePortNumber(31180, 300));
    try (ClusterFixture fixture = fixtureBuilder.build();
        Drillbit drillbitWithSamePort = new Drillbit(fixture.config(), fixtureBuilder.configBuilder().getDefinitions(), fixture.serviceSet())) {
        // Assert preconditions :
        // 1. First drillbit instance should be started normally
        // 2. Second instance startup should fail, because ports are occupied by the first one
        assertNotNull("First drillbit instance should be initialized", fixture.drillbit());
        try {
            drillbitWithSamePort.run();
            fail("Invocation of 'drillbitWithSamePort.run()' should throw UserException");
        } catch (UserException e) {
            assertThat(e.getMessage(), containsString("RESOURCE ERROR: Drillbit could not bind to port"));
            // Ensure that drillbit with failed startup may be safely closed
            assertNotNull("Drillbit.gracefulShutdownThread shouldn't be null, otherwise close() may throw NPE (if so, check suppressed exception).", drillbitWithSamePort.getGracefulShutdownThread());
        }
    }
}
Also used : Drillbit(org.apache.drill.exec.server.Drillbit) UserException(org.apache.drill.common.exceptions.UserException) SlowTest(org.apache.drill.categories.SlowTest) Test(org.junit.Test)

Example 20 with UserException

use of org.apache.drill.common.exceptions.UserException in project drill by apache.

the class PdfUtils method getSpecificTable.

/**
 * Returns a specific table from a PDF document. Returns null in the event that
 * the user requests a table that does not exist.  If there is an error with the document
 * the function will throw a UserException.
 * @param document The source PDF document
 * @param tableIndex The index of the desired table
 * @return The desired Table, null if the table is not valid, or if the document has no tables.
 */
public static Table getSpecificTable(PDDocument document, int tableIndex, ExtractionAlgorithm algorithm) {
    NurminenDetectionAlgorithm detectionAlgorithm = new NurminenDetectionAlgorithm();
    ExtractionAlgorithm algExtractor;
    if (algorithm == null) {
        algExtractor = DEFAULT_ALGORITHM;
    } else {
        algExtractor = algorithm;
    }
    ObjectExtractor objectExtractor = new ObjectExtractor(document);
    PageIterator pages = objectExtractor.extract();
    Table specificTable;
    int tableCounter = 0;
    while (pages.hasNext()) {
        Page page = pages.next();
        List<Rectangle> rectanglesOnPage = detectionAlgorithm.detect(page);
        List<Table> tablesOnPage = new ArrayList<>();
        for (Rectangle guessRect : rectanglesOnPage) {
            Page guess = page.getArea(guessRect);
            tablesOnPage.addAll(algExtractor.extract(guess));
            if (tablesOnPage.size() == 0) {
                return null;
            }
            for (Table table : tablesOnPage) {
                if (tableCounter == tableIndex) {
                    specificTable = table;
                    return specificTable;
                }
                tableCounter++;
            }
        }
    }
    try {
        objectExtractor.close();
    } catch (Exception e) {
        throw UserException.parseError(e).message("Error extracting table: " + e.getMessage()).build(logger);
    }
    return null;
}
Also used : ExtractionAlgorithm(technology.tabula.extractors.ExtractionAlgorithm) SpreadsheetExtractionAlgorithm(technology.tabula.extractors.SpreadsheetExtractionAlgorithm) BasicExtractionAlgorithm(technology.tabula.extractors.BasicExtractionAlgorithm) PageIterator(technology.tabula.PageIterator) Table(technology.tabula.Table) Rectangle(technology.tabula.Rectangle) ArrayList(java.util.ArrayList) ObjectExtractor(technology.tabula.ObjectExtractor) Page(technology.tabula.Page) NurminenDetectionAlgorithm(technology.tabula.detectors.NurminenDetectionAlgorithm) UserException(org.apache.drill.common.exceptions.UserException)

Aggregations

UserException (org.apache.drill.common.exceptions.UserException)102 Test (org.junit.Test)76 EvfTest (org.apache.drill.categories.EvfTest)39 TupleMetadata (org.apache.drill.exec.record.metadata.TupleMetadata)30 SubOperatorTest (org.apache.drill.test.SubOperatorTest)30 SchemaBuilder (org.apache.drill.exec.record.metadata.SchemaBuilder)28 RowBatchReader (org.apache.drill.exec.physical.impl.scan.RowBatchReader)12 ManagedReader (org.apache.drill.exec.physical.impl.scan.v3.ManagedReader)12 ScanLifecycleBuilder (org.apache.drill.exec.physical.impl.scan.v3.ScanLifecycleBuilder)11 SchemaNegotiator (org.apache.drill.exec.physical.impl.scan.v3.SchemaNegotiator)11 ScanOperatorExec (org.apache.drill.exec.physical.impl.scan.ScanOperatorExec)9 ScanFixture (org.apache.drill.exec.physical.impl.scan.ScanTestUtils.ScanFixture)9 SchemaPath (org.apache.drill.common.expression.SchemaPath)8 ResultSetOptions (org.apache.drill.exec.physical.resultSet.impl.ResultSetLoaderImpl.ResultSetOptions)8 RowSet (org.apache.drill.exec.physical.rowSet.RowSet)8 MockRecordBatch (org.apache.drill.exec.physical.impl.MockRecordBatch)6 ArrayList (java.util.ArrayList)5 OperatorTest (org.apache.drill.categories.OperatorTest)5 DrillException (org.apache.drill.common.exceptions.DrillException)5 BaseTest (org.apache.drill.test.BaseTest)5