Search in sources :

Example 1 with InternalRecordWrapper

use of org.apache.iceberg.data.InternalRecordWrapper in project hive by apache.

the class TestInputFormatReaderDeletes method rowSet.

@Override
public StructLikeSet rowSet(String name, Table table, String... columns) {
    InputFormatConfig.ConfigBuilder builder = new InputFormatConfig.ConfigBuilder(conf).readFrom(table.location());
    Schema projected = table.schema().select(columns);
    StructLikeSet set = StructLikeSet.create(projected.asStruct());
    set.addAll(TestIcebergInputFormats.TESTED_INPUT_FORMATS.stream().filter(recordFactory -> recordFactory.name().equals(inputFormat)).map(recordFactory -> recordFactory.create(builder.project(projected).conf()).getRecords()).flatMap(List::stream).map(record -> new InternalRecordWrapper(projected.asStruct()).wrap(record)).collect(Collectors.toList()));
    return set;
}
Also used : BaseTable(org.apache.iceberg.BaseTable) InternalRecordWrapper(org.apache.iceberg.data.InternalRecordWrapper) Table(org.apache.iceberg.Table) StructLikeSet(org.apache.iceberg.util.StructLikeSet) HadoopTables(org.apache.iceberg.hadoop.HadoopTables) RunWith(org.junit.runner.RunWith) IOException(java.io.IOException) Schema(org.apache.iceberg.Schema) TestIcebergInputFormats(org.apache.iceberg.mr.hive.TestIcebergInputFormats) Collectors(java.util.stream.Collectors) FileFormat(org.apache.iceberg.FileFormat) File(java.io.File) TableMetadata(org.apache.iceberg.TableMetadata) List(java.util.List) TableOperations(org.apache.iceberg.TableOperations) Configuration(org.apache.hadoop.conf.Configuration) PartitionSpec(org.apache.iceberg.PartitionSpec) DeleteReadTests(org.apache.iceberg.data.DeleteReadTests) Assert(org.junit.Assert) Parameterized(org.junit.runners.Parameterized) Before(org.junit.Before) Schema(org.apache.iceberg.Schema) InternalRecordWrapper(org.apache.iceberg.data.InternalRecordWrapper) StructLikeSet(org.apache.iceberg.util.StructLikeSet)

Aggregations

File (java.io.File)1 IOException (java.io.IOException)1 List (java.util.List)1 Collectors (java.util.stream.Collectors)1 Configuration (org.apache.hadoop.conf.Configuration)1 BaseTable (org.apache.iceberg.BaseTable)1 FileFormat (org.apache.iceberg.FileFormat)1 PartitionSpec (org.apache.iceberg.PartitionSpec)1 Schema (org.apache.iceberg.Schema)1 Table (org.apache.iceberg.Table)1 TableMetadata (org.apache.iceberg.TableMetadata)1 TableOperations (org.apache.iceberg.TableOperations)1 DeleteReadTests (org.apache.iceberg.data.DeleteReadTests)1 InternalRecordWrapper (org.apache.iceberg.data.InternalRecordWrapper)1 HadoopTables (org.apache.iceberg.hadoop.HadoopTables)1 TestIcebergInputFormats (org.apache.iceberg.mr.hive.TestIcebergInputFormats)1 StructLikeSet (org.apache.iceberg.util.StructLikeSet)1 Assert (org.junit.Assert)1 Before (org.junit.Before)1 RunWith (org.junit.runner.RunWith)1