Search in sources :

Example 1 with ByteString

use of io.trino.orc.protobuf.ByteString in project trino by trinodb.

the class OrcMetadataReader method readBloomFilterIndexes.

@Override
public List<BloomFilter> readBloomFilterIndexes(InputStream inputStream) throws IOException {
    CodedInputStream input = CodedInputStream.newInstance(inputStream);
    OrcProto.BloomFilterIndex bloomFilter = OrcProto.BloomFilterIndex.parseFrom(input);
    List<OrcProto.BloomFilter> bloomFilterList = bloomFilter.getBloomFilterList();
    ImmutableList.Builder<BloomFilter> builder = ImmutableList.builder();
    for (OrcProto.BloomFilter orcBloomFilter : bloomFilterList) {
        if (orcBloomFilter.hasUtf8Bitset()) {
            ByteString utf8Bitset = orcBloomFilter.getUtf8Bitset();
            long[] bits = new long[utf8Bitset.size() / 8];
            utf8Bitset.asReadOnlyByteBuffer().order(ByteOrder.LITTLE_ENDIAN).asLongBuffer().get(bits);
            builder.add(new BloomFilter(bits, orcBloomFilter.getNumHashFunctions()));
        } else {
            builder.add(new BloomFilter(Longs.toArray(orcBloomFilter.getBitsetList()), orcBloomFilter.getNumHashFunctions()));
        }
    }
    return builder.build();
}
Also used : CodedInputStream(io.trino.orc.protobuf.CodedInputStream) ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) ImmutableList(com.google.common.collect.ImmutableList) ByteString(io.trino.orc.protobuf.ByteString) OrcProto(io.trino.orc.proto.OrcProto) BloomFilter(io.trino.orc.metadata.statistics.BloomFilter)

Aggregations

ImmutableList (com.google.common.collect.ImmutableList)1 ImmutableList.toImmutableList (com.google.common.collect.ImmutableList.toImmutableList)1 BloomFilter (io.trino.orc.metadata.statistics.BloomFilter)1 OrcProto (io.trino.orc.proto.OrcProto)1 ByteString (io.trino.orc.protobuf.ByteString)1 CodedInputStream (io.trino.orc.protobuf.CodedInputStream)1