Search in sources :

Example 6 with ColumnHandle

use of org.apache.drill.exec.physical.impl.scan.v3.schema.MutableTupleSchema.ColumnHandle in project drill by apache.

the class TestImplicitColumnResolver method testShadowed.

/**
 * Verify that names that look like metadata columns, but appear
 * to be maps or arrays, are not interpreted as metadata. That is,
 * the projected table map or array "shadows" the metadata column.
 */
@Test
public void testShadowed() {
    ParserFixture parseFixture = new ParserFixture(RowSetTestUtils.projectList(ScanTestUtils.FILE_NAME_COL + ".a", ScanTestUtils.FILE_PATH_COL + "[0]", ScanTestUtils.partitionColName(0) + ".b", ScanTestUtils.partitionColName(1) + "[0]", ScanTestUtils.SUFFIX_COL));
    ParseResult result = parseFixture.parseImplicit();
    TupleMetadata expected = new SchemaBuilder().add(ScanTestUtils.SUFFIX_COL, MinorType.VARCHAR).build();
    assertEquals(expected, result.schema());
    List<ColumnHandle> cols = parseFixture.tracker.internalSchema().columns();
    assertFalse(isImplicit(cols, 0));
    assertFalse(isImplicit(cols, 1));
    assertFalse(isImplicit(cols, 2));
    assertFalse(isImplicit(cols, 3));
    assertTrue(isImplicit(cols, 4));
}
Also used : ColumnHandle(org.apache.drill.exec.physical.impl.scan.v3.schema.MutableTupleSchema.ColumnHandle) ParseResult(org.apache.drill.exec.physical.impl.scan.v3.file.ImplicitColumnResolver.ParseResult) ProjectionParseResult(org.apache.drill.exec.physical.impl.scan.v3.schema.ScanProjectionParser.ProjectionParseResult) TupleMetadata(org.apache.drill.exec.record.metadata.TupleMetadata) SchemaBuilder(org.apache.drill.exec.record.metadata.SchemaBuilder) SubOperatorTest(org.apache.drill.test.SubOperatorTest) EvfTest(org.apache.drill.categories.EvfTest) Test(org.junit.Test)

Example 7 with ColumnHandle

use of org.apache.drill.exec.physical.impl.scan.v3.schema.MutableTupleSchema.ColumnHandle in project drill by apache.

the class TestImplicitColumnResolver method testProvidedImplicitMatchesProject.

@Test
public void testProvidedImplicitMatchesProject() {
    TupleMetadata providedSchema = new SchemaBuilder().add("a", MinorType.INT).add("fqn", MinorType.VARCHAR).add("filePath", MinorType.VARCHAR).addNullable("dir0", MinorType.VARCHAR).build();
    SchemaUtils.markImplicit(providedSchema.metadata("fqn"), ColumnMetadata.IMPLICIT_FQN);
    SchemaUtils.markImplicit(providedSchema.metadata("filePath"), ColumnMetadata.IMPLICIT_FILEPATH.toUpperCase());
    SchemaUtils.markAsPartition(providedSchema.metadata("dir0"), 0);
    ParserFixture parseFixture = new ParserFixture(RowSetTestUtils.projectAll());
    parseFixture.tracker.applyProvidedSchema(providedSchema);
    parseFixture.options.maxPartitionDepth(1);
    ParseResult result = parseFixture.parseImplicit();
    assertEquals(3, result.columns().size());
    TupleMetadata expected = new SchemaBuilder().add("fqn", MinorType.VARCHAR).add("filePath", MinorType.VARCHAR).addNullable("dir0", MinorType.VARCHAR).build();
    assertEquals(expected, result.schema());
    List<ColumnHandle> cols = parseFixture.tracker.internalSchema().columns();
    assertFalse(isImplicit(cols, 0));
    assertTrue(isImplicit(cols, 1));
    assertTrue(isImplicit(cols, 2));
    assertTrue(isImplicit(cols, 3));
}
Also used : ColumnHandle(org.apache.drill.exec.physical.impl.scan.v3.schema.MutableTupleSchema.ColumnHandle) ParseResult(org.apache.drill.exec.physical.impl.scan.v3.file.ImplicitColumnResolver.ParseResult) ProjectionParseResult(org.apache.drill.exec.physical.impl.scan.v3.schema.ScanProjectionParser.ProjectionParseResult) TupleMetadata(org.apache.drill.exec.record.metadata.TupleMetadata) SchemaBuilder(org.apache.drill.exec.record.metadata.SchemaBuilder) SubOperatorTest(org.apache.drill.test.SubOperatorTest) EvfTest(org.apache.drill.categories.EvfTest) Test(org.junit.Test)

Example 8 with ColumnHandle

use of org.apache.drill.exec.physical.impl.scan.v3.schema.MutableTupleSchema.ColumnHandle in project drill by apache.

the class AbstractSchemaTracker method implicitColumns.

private TupleMetadata implicitColumns() {
    TupleMetadata implicitCols = new TupleSchema();
    for (ColumnHandle handle : schema.columns()) {
        if (handle.isImplicit()) {
            handle.setIndex(implicitCols.size());
            implicitCols.addColumn(handle.column());
        }
    }
    return implicitCols;
}
Also used : ColumnHandle(org.apache.drill.exec.physical.impl.scan.v3.schema.MutableTupleSchema.ColumnHandle) TupleMetadata(org.apache.drill.exec.record.metadata.TupleMetadata) TupleSchema(org.apache.drill.exec.record.metadata.TupleSchema)

Aggregations

ColumnHandle (org.apache.drill.exec.physical.impl.scan.v3.schema.MutableTupleSchema.ColumnHandle)8 TupleMetadata (org.apache.drill.exec.record.metadata.TupleMetadata)8 EvfTest (org.apache.drill.categories.EvfTest)7 ParseResult (org.apache.drill.exec.physical.impl.scan.v3.file.ImplicitColumnResolver.ParseResult)7 ProjectionParseResult (org.apache.drill.exec.physical.impl.scan.v3.schema.ScanProjectionParser.ProjectionParseResult)7 SchemaBuilder (org.apache.drill.exec.record.metadata.SchemaBuilder)7 SubOperatorTest (org.apache.drill.test.SubOperatorTest)7 Test (org.junit.Test)7 TupleSchema (org.apache.drill.exec.record.metadata.TupleSchema)1 DrillFileSystem (org.apache.drill.exec.store.dfs.DrillFileSystem)1 Configuration (org.apache.hadoop.conf.Configuration)1