Search in sources :

Example 1 with ConstantColumnSpec

use of org.apache.drill.exec.physical.impl.scan.project.ConstantColumnLoader.ConstantColumnSpec in project drill by apache.

the class TestConstantColumnLoader method testConstantColumnLoader.

/**
 * Test the static column loader using one column of each type.
 * The null column is of type int, but the associated value is of
 * type string. This is a bit odd, but works out because we detect that
 * the string value is null and call setNull on the writer, and avoid
 * using the actual data.
 */
@Test
public void testConstantColumnLoader() {
    final MajorType aType = MajorType.newBuilder().setMinorType(MinorType.VARCHAR).setMode(DataMode.REQUIRED).build();
    final MajorType bType = MajorType.newBuilder().setMinorType(MinorType.VARCHAR).setMode(DataMode.OPTIONAL).build();
    final List<ConstantColumnSpec> defns = new ArrayList<>();
    defns.add(new DummyColumn("a", aType, "a-value"));
    defns.add(new DummyColumn("b", bType, "b-value"));
    final ResultVectorCacheImpl cache = new ResultVectorCacheImpl(fixture.allocator());
    final ConstantColumnLoader staticLoader = new ConstantColumnLoader(cache, defns);
    // Create a batch
    staticLoader.load(2);
    // Verify
    final TupleMetadata expectedSchema = new SchemaBuilder().add("a", aType).add("b", bType).buildSchema();
    final SingleRowSet expected = fixture.rowSetBuilder(expectedSchema).addRow("a-value", "b-value").addRow("a-value", "b-value").build();
    new RowSetComparison(expected).verifyAndClearAll(fixture.wrap(staticLoader.load(2)));
    staticLoader.close();
}
Also used : SingleRowSet(org.apache.drill.exec.physical.rowSet.RowSet.SingleRowSet) RowSetComparison(org.apache.drill.test.rowSet.RowSetComparison) ConstantColumnSpec(org.apache.drill.exec.physical.impl.scan.project.ConstantColumnLoader.ConstantColumnSpec) MajorType(org.apache.drill.common.types.TypeProtos.MajorType) TupleMetadata(org.apache.drill.exec.record.metadata.TupleMetadata) ArrayList(java.util.ArrayList) SchemaBuilder(org.apache.drill.exec.record.metadata.SchemaBuilder) ResultVectorCacheImpl(org.apache.drill.exec.physical.resultSet.impl.ResultVectorCacheImpl) SubOperatorTest(org.apache.drill.test.SubOperatorTest) Test(org.junit.Test)

Example 2 with ConstantColumnSpec

use of org.apache.drill.exec.physical.impl.scan.project.ConstantColumnLoader.ConstantColumnSpec in project drill by apache.

the class TestConstantColumnLoader method testFileMetadata.

@Test
public void testFileMetadata() {
    FileMetadata fileInfo = new FileMetadata(new Path("hdfs:///w/x/y/z.csv"), new Path("hdfs:///w"));
    List<ConstantColumnSpec> defns = new ArrayList<>();
    FileMetadataColumnDefn iDefn = new FileMetadataColumnDefn(ScanTestUtils.SUFFIX_COL, ImplicitFileColumns.SUFFIX);
    FileMetadataColumn iCol = new FileMetadataColumn(ScanTestUtils.SUFFIX_COL, iDefn, fileInfo, null, 0);
    defns.add(iCol);
    String partColName = ScanTestUtils.partitionColName(1);
    PartitionColumn pCol = new PartitionColumn(partColName, 1, fileInfo, null, 0);
    defns.add(pCol);
    ResultVectorCacheImpl cache = new ResultVectorCacheImpl(fixture.allocator());
    ConstantColumnLoader staticLoader = new ConstantColumnLoader(cache, defns);
    // Create a batch
    staticLoader.load(2);
    // Verify
    TupleMetadata expectedSchema = new SchemaBuilder().add(ScanTestUtils.SUFFIX_COL, MinorType.VARCHAR).addNullable(partColName, MinorType.VARCHAR).buildSchema();
    SingleRowSet expected = fixture.rowSetBuilder(expectedSchema).addRow("csv", "y").addRow("csv", "y").build();
    new RowSetComparison(expected).verifyAndClearAll(fixture.wrap(staticLoader.load(2)));
    staticLoader.close();
}
Also used : Path(org.apache.hadoop.fs.Path) FileMetadataColumnDefn(org.apache.drill.exec.physical.impl.scan.file.FileMetadataColumnDefn) SingleRowSet(org.apache.drill.exec.physical.rowSet.RowSet.SingleRowSet) FileMetadata(org.apache.drill.exec.physical.impl.scan.file.FileMetadata) ArrayList(java.util.ArrayList) PartitionColumn(org.apache.drill.exec.physical.impl.scan.file.PartitionColumn) RowSetComparison(org.apache.drill.test.rowSet.RowSetComparison) ConstantColumnSpec(org.apache.drill.exec.physical.impl.scan.project.ConstantColumnLoader.ConstantColumnSpec) TupleMetadata(org.apache.drill.exec.record.metadata.TupleMetadata) SchemaBuilder(org.apache.drill.exec.record.metadata.SchemaBuilder) FileMetadataColumn(org.apache.drill.exec.physical.impl.scan.file.FileMetadataColumn) ResultVectorCacheImpl(org.apache.drill.exec.physical.resultSet.impl.ResultVectorCacheImpl) SubOperatorTest(org.apache.drill.test.SubOperatorTest) Test(org.junit.Test)

Aggregations

ArrayList (java.util.ArrayList)2 ConstantColumnSpec (org.apache.drill.exec.physical.impl.scan.project.ConstantColumnLoader.ConstantColumnSpec)2 ResultVectorCacheImpl (org.apache.drill.exec.physical.resultSet.impl.ResultVectorCacheImpl)2 SingleRowSet (org.apache.drill.exec.physical.rowSet.RowSet.SingleRowSet)2 SchemaBuilder (org.apache.drill.exec.record.metadata.SchemaBuilder)2 TupleMetadata (org.apache.drill.exec.record.metadata.TupleMetadata)2 SubOperatorTest (org.apache.drill.test.SubOperatorTest)2 RowSetComparison (org.apache.drill.test.rowSet.RowSetComparison)2 Test (org.junit.Test)2 MajorType (org.apache.drill.common.types.TypeProtos.MajorType)1 FileMetadata (org.apache.drill.exec.physical.impl.scan.file.FileMetadata)1 FileMetadataColumn (org.apache.drill.exec.physical.impl.scan.file.FileMetadataColumn)1 FileMetadataColumnDefn (org.apache.drill.exec.physical.impl.scan.file.FileMetadataColumnDefn)1 PartitionColumn (org.apache.drill.exec.physical.impl.scan.file.PartitionColumn)1 Path (org.apache.hadoop.fs.Path)1