Search in sources :

Example 6 with BloomFilter

use of org.rocksdb.BloomFilter in project flink by apache.

the class RocksDBResourceContainer method setColumnFamilyOptionsFromConfigurableOptions.

@SuppressWarnings("ConstantConditions")
private ColumnFamilyOptions setColumnFamilyOptionsFromConfigurableOptions(ColumnFamilyOptions currentOptions, Collection<AutoCloseable> handlesToClose) {
    currentOptions.setCompactionStyle(internalGetOption(RocksDBConfigurableOptions.COMPACTION_STYLE));
    currentOptions.setLevelCompactionDynamicLevelBytes(internalGetOption(RocksDBConfigurableOptions.USE_DYNAMIC_LEVEL_SIZE));
    currentOptions.setTargetFileSizeBase(internalGetOption(RocksDBConfigurableOptions.TARGET_FILE_SIZE_BASE).getBytes());
    currentOptions.setMaxBytesForLevelBase(internalGetOption(RocksDBConfigurableOptions.MAX_SIZE_LEVEL_BASE).getBytes());
    currentOptions.setWriteBufferSize(internalGetOption(RocksDBConfigurableOptions.WRITE_BUFFER_SIZE).getBytes());
    currentOptions.setMaxWriteBufferNumber(internalGetOption(RocksDBConfigurableOptions.MAX_WRITE_BUFFER_NUMBER));
    currentOptions.setMinWriteBufferNumberToMerge(internalGetOption(RocksDBConfigurableOptions.MIN_WRITE_BUFFER_NUMBER_TO_MERGE));
    TableFormatConfig tableFormatConfig = currentOptions.tableFormatConfig();
    BlockBasedTableConfig blockBasedTableConfig;
    if (tableFormatConfig == null) {
        blockBasedTableConfig = new BlockBasedTableConfig();
    } else {
        if (tableFormatConfig instanceof PlainTableConfig) {
            // column-family options
            return currentOptions;
        } else {
            blockBasedTableConfig = (BlockBasedTableConfig) tableFormatConfig;
        }
    }
    blockBasedTableConfig.setBlockSize(internalGetOption(RocksDBConfigurableOptions.BLOCK_SIZE).getBytes());
    blockBasedTableConfig.setMetadataBlockSize(internalGetOption(RocksDBConfigurableOptions.METADATA_BLOCK_SIZE).getBytes());
    blockBasedTableConfig.setBlockCacheSize(internalGetOption(RocksDBConfigurableOptions.BLOCK_CACHE_SIZE).getBytes());
    if (internalGetOption(RocksDBConfigurableOptions.USE_BLOOM_FILTER)) {
        final double bitsPerKey = internalGetOption(RocksDBConfigurableOptions.BLOOM_FILTER_BITS_PER_KEY);
        final boolean blockBasedMode = internalGetOption(RocksDBConfigurableOptions.BLOOM_FILTER_BLOCK_BASED_MODE);
        BloomFilter bloomFilter = new BloomFilter(bitsPerKey, blockBasedMode);
        handlesToClose.add(bloomFilter);
        blockBasedTableConfig.setFilterPolicy(bloomFilter);
    }
    return currentOptions.setTableFormatConfig(blockBasedTableConfig);
}
Also used : TableFormatConfig(org.rocksdb.TableFormatConfig) BlockBasedTableConfig(org.rocksdb.BlockBasedTableConfig) PlainTableConfig(org.rocksdb.PlainTableConfig) BloomFilter(org.rocksdb.BloomFilter)

Aggregations

BloomFilter (org.rocksdb.BloomFilter)6 BlockBasedTableConfig (org.rocksdb.BlockBasedTableConfig)4 TableFormatConfig (org.rocksdb.TableFormatConfig)3 IOException (java.io.IOException)2 ColumnFamilyOptions (org.rocksdb.ColumnFamilyOptions)2 DBOptions (org.rocksdb.DBOptions)2 LRUCache (org.rocksdb.LRUCache)2 PlainTableConfig (org.rocksdb.PlainTableConfig)2 File (java.io.File)1 Collection (java.util.Collection)1 OpaqueMemoryResource (org.apache.flink.runtime.memory.OpaqueMemoryResource)1 ProcessorStateException (org.apache.kafka.streams.errors.ProcessorStateException)1 RocksDBConfigSetter (org.apache.kafka.streams.state.RocksDBConfigSetter)1 Test (org.junit.Test)1 FlushOptions (org.rocksdb.FlushOptions)1 WriteBufferManager (org.rocksdb.WriteBufferManager)1 WriteOptions (org.rocksdb.WriteOptions)1