Search in sources :

Example 11 with StructLike

use of org.apache.iceberg.StructLike in project presto by prestodb.

the class IcebergSplitSource method getPartitionKeys.

private static Map<Integer, String> getPartitionKeys(FileScanTask scanTask) {
    StructLike partition = scanTask.file().partition();
    PartitionSpec spec = scanTask.spec();
    Map<PartitionField, Integer> fieldToIndex = getIdentityPartitions(spec);
    Map<Integer, String> partitionKeys = new HashMap<>();
    fieldToIndex.forEach((field, index) -> {
        int id = field.sourceId();
        Type type = spec.schema().findType(id);
        Class<?> javaClass = type.typeId().javaClass();
        Object value = partition.get(index, javaClass);
        if (value == null) {
            partitionKeys.put(id, null);
        } else {
            String partitionValue;
            if (type.typeId() == FIXED || type.typeId() == BINARY) {
                // this is safe because Iceberg PartitionData directly wraps the byte array
                partitionValue = new String(((ByteBuffer) value).array(), UTF_8);
            } else {
                partitionValue = value.toString();
            }
            partitionKeys.put(id, partitionValue);
        }
    });
    return Collections.unmodifiableMap(partitionKeys);
}
Also used : HashMap(java.util.HashMap) StructLike(org.apache.iceberg.StructLike) PartitionSpec(org.apache.iceberg.PartitionSpec) ByteBuffer(java.nio.ByteBuffer) PartitionField(org.apache.iceberg.PartitionField) Type(org.apache.iceberg.types.Type)

Aggregations

StructLike (org.apache.iceberg.StructLike)11 List (java.util.List)5 ArrayList (java.util.ArrayList)4 HashMap (java.util.HashMap)4 PartitionSpec (org.apache.iceberg.PartitionSpec)4 TableIdentifier (org.apache.iceberg.catalog.TableIdentifier)4 ImmutableList (org.apache.iceberg.relocated.com.google.common.collect.ImmutableList)4 Test (org.junit.Test)4 IOException (java.io.IOException)3 DataFile (org.apache.iceberg.DataFile)3 HashSet (java.util.HashSet)2 Path (org.apache.hadoop.fs.Path)2 Record (org.apache.iceberg.data.Record)2 Type (org.apache.iceberg.types.Type)2 Types (org.apache.iceberg.types.Types)2 UncheckedIOException (java.io.UncheckedIOException)1 Array (java.lang.reflect.Array)1 ByteBuffer (java.nio.ByteBuffer)1 Arrays (java.util.Arrays)1 Map (java.util.Map)1