Search in sources :

Example 1 with IncompatibleSchemaException

use of org.apache.drill.exec.physical.impl.scan.project.SchemaSmoother.IncompatibleSchemaException in project drill by apache.

the class SmoothingProjection method resolveColumn.

/**
 * Resolve a prior column against the current table schema. Resolves to
 * a table column, a null column, or throws an exception if the
 * schemas are incompatible
 *
 * @param priorCol a column from the prior schema
 * @throws IncompatibleSchemaException if the prior column exists in
 * the current table schema, but with an incompatible type
 */
private void resolveColumn(ResolvedTuple outputTuple, ResolvedColumn priorCol, TupleMetadata tableSchema) throws IncompatibleSchemaException {
    int tableColIndex = tableSchema.index(priorCol.name());
    if (tableColIndex == -1) {
        resolveNullColumn(outputTuple, priorCol);
        return;
    }
    MaterializedField tableCol = tableSchema.column(tableColIndex);
    MaterializedField priorField = priorCol.schema();
    if (!tableCol.isPromotableTo(priorField, false)) {
        throw new IncompatibleSchemaException();
    }
    outputTuple.add(new ResolvedTableColumn(priorCol.name(), priorField, outputTuple, tableColIndex));
    rewrittenFields.add(priorField);
}
Also used : IncompatibleSchemaException(org.apache.drill.exec.physical.impl.scan.project.SchemaSmoother.IncompatibleSchemaException) MaterializedField(org.apache.drill.exec.record.MaterializedField)

Example 2 with IncompatibleSchemaException

use of org.apache.drill.exec.physical.impl.scan.project.SchemaSmoother.IncompatibleSchemaException in project drill by apache.

the class TestSchemaSmoothing method testSmoothingProjection.

/**
 * Low-level test of the smoothing projection, including the exceptions
 * it throws when things are not going its way.
 */
@Test
public void testSmoothingProjection() {
    final ScanLevelProjection scanProj = ScanLevelProjection.build(RowSetTestUtils.projectAll(), ScanTestUtils.parsers());
    // Table 1: (a: nullable bigint, b)
    final TupleMetadata schema1 = new SchemaBuilder().addNullable("a", MinorType.BIGINT).addNullable("b", MinorType.VARCHAR).add("c", MinorType.FLOAT8).buildSchema();
    ResolvedRow priorSchema;
    {
        final NullColumnBuilder builder = new NullBuilderBuilder().build();
        final ResolvedRow rootTuple = new ResolvedRow(builder);
        new WildcardProjection(scanProj, schema1, rootTuple, ScanTestUtils.resolvers());
        priorSchema = rootTuple;
    }
    // Table 2: (a: nullable bigint, c), column omitted, original schema preserved
    final TupleMetadata schema2 = new SchemaBuilder().addNullable("a", MinorType.BIGINT).add("c", MinorType.FLOAT8).buildSchema();
    try {
        final NullColumnBuilder builder = new NullBuilderBuilder().build();
        final ResolvedRow rootTuple = new ResolvedRow(builder);
        new SmoothingProjection(scanProj, schema2, priorSchema, rootTuple, ScanTestUtils.resolvers());
        assertTrue(schema1.isEquivalent(ScanTestUtils.schema(rootTuple)));
        priorSchema = rootTuple;
    } catch (final IncompatibleSchemaException e) {
        fail();
    }
    // Table 3: (a, c, d), column added, must replan schema
    final TupleMetadata schema3 = new SchemaBuilder().addNullable("a", MinorType.BIGINT).addNullable("b", MinorType.VARCHAR).add("c", MinorType.FLOAT8).add("d", MinorType.INT).buildSchema();
    try {
        final NullColumnBuilder builder = new NullBuilderBuilder().build();
        final ResolvedRow rootTuple = new ResolvedRow(builder);
        new SmoothingProjection(scanProj, schema3, priorSchema, rootTuple, ScanTestUtils.resolvers());
        fail();
    } catch (final IncompatibleSchemaException e) {
    // Expected
    }
    // Table 4: (a: double), change type must replan schema
    final TupleMetadata schema4 = new SchemaBuilder().addNullable("a", MinorType.FLOAT8).addNullable("b", MinorType.VARCHAR).add("c", MinorType.FLOAT8).buildSchema();
    try {
        final NullColumnBuilder builder = new NullBuilderBuilder().build();
        final ResolvedRow rootTuple = new ResolvedRow(builder);
        new SmoothingProjection(scanProj, schema4, priorSchema, rootTuple, ScanTestUtils.resolvers());
        fail();
    } catch (final IncompatibleSchemaException e) {
    // Expected
    }
    // Table 5: Drop a non-nullable column, must replan
    final TupleMetadata schema6 = new SchemaBuilder().addNullable("a", MinorType.BIGINT).addNullable("b", MinorType.VARCHAR).buildSchema();
    try {
        final NullColumnBuilder builder = new NullBuilderBuilder().build();
        final ResolvedRow rootTuple = new ResolvedRow(builder);
        new SmoothingProjection(scanProj, schema6, priorSchema, rootTuple, ScanTestUtils.resolvers());
        fail();
    } catch (final IncompatibleSchemaException e) {
    // Expected
    }
}
Also used : IncompatibleSchemaException(org.apache.drill.exec.physical.impl.scan.project.SchemaSmoother.IncompatibleSchemaException) ResolvedRow(org.apache.drill.exec.physical.impl.scan.project.ResolvedTuple.ResolvedRow) TupleMetadata(org.apache.drill.exec.record.metadata.TupleMetadata) SchemaBuilder(org.apache.drill.exec.record.metadata.SchemaBuilder) NullBuilderBuilder(org.apache.drill.exec.physical.impl.scan.project.NullColumnBuilder.NullBuilderBuilder) SubOperatorTest(org.apache.drill.test.SubOperatorTest) Test(org.junit.Test)

Aggregations

IncompatibleSchemaException (org.apache.drill.exec.physical.impl.scan.project.SchemaSmoother.IncompatibleSchemaException)2 NullBuilderBuilder (org.apache.drill.exec.physical.impl.scan.project.NullColumnBuilder.NullBuilderBuilder)1 ResolvedRow (org.apache.drill.exec.physical.impl.scan.project.ResolvedTuple.ResolvedRow)1 MaterializedField (org.apache.drill.exec.record.MaterializedField)1 SchemaBuilder (org.apache.drill.exec.record.metadata.SchemaBuilder)1 TupleMetadata (org.apache.drill.exec.record.metadata.TupleMetadata)1 SubOperatorTest (org.apache.drill.test.SubOperatorTest)1 Test (org.junit.Test)1