Search in sources :

Example 6 with WriteBatch

use of org.rocksdb.WriteBatch in project jstorm by alibaba.

the class RocksDbHdfsState method putBatch.

@Override
public void putBatch(Map<K, V> batch) {
    try {
        WriteBatch writeBatch = new WriteBatch();
        for (Map.Entry<K, V> entry : batch.entrySet()) {
            writeBatch.put(serializer.serialize(entry.getKey()), serializer.serialize(entry.getValue()));
        }
        rocksDb.write(new WriteOptions(), writeBatch);
    } catch (RocksDBException e) {
        LOG.error("Failed to put batch={}", batch);
        throw new RuntimeException(e.getMessage());
    }
}
Also used : WriteOptions(org.rocksdb.WriteOptions) RocksDBException(org.rocksdb.RocksDBException) WriteBatch(org.rocksdb.WriteBatch) HashMap(java.util.HashMap) Map(java.util.Map)

Example 7 with WriteBatch

use of org.rocksdb.WriteBatch in project jstorm by alibaba.

the class WindowedRocksDbHdfsState method putBatch.

@Override
public void putBatch(TimeWindow window, Map<K, V> batch) {
    try {
        ColumnFamilyHandle handler = getColumnFamilyHandle(window);
        WriteBatch writeBatch = new WriteBatch();
        for (Map.Entry<K, V> entry : batch.entrySet()) {
            writeBatch.put(handler, serializer.serialize(entry.getKey()), serializer.serialize(entry.getValue()));
        }
        rocksDb.write(new WriteOptions(), writeBatch);
    } catch (RocksDBException e) {
        LOG.error("Failed to put batch={} for window={}", batch, window);
        throw new RuntimeException(e.getMessage());
    }
}
Also used : WriteOptions(org.rocksdb.WriteOptions) RocksDBException(org.rocksdb.RocksDBException) WriteBatch(org.rocksdb.WriteBatch) HashMap(java.util.HashMap) Map(java.util.Map) ColumnFamilyHandle(org.rocksdb.ColumnFamilyHandle)

Example 8 with WriteBatch

use of org.rocksdb.WriteBatch in project kafka by apache.

the class RocksDBStore method putAll.

@Override
public void putAll(List<KeyValue<K, V>> entries) {
    try (WriteBatch batch = new WriteBatch()) {
        for (KeyValue<K, V> entry : entries) {
            final byte[] rawKey = serdes.rawKey(entry.key);
            if (entry.value == null) {
                db.delete(rawKey);
            } else {
                final byte[] value = serdes.rawValue(entry.value);
                batch.put(rawKey, value);
            }
        }
        db.write(wOptions, batch);
    } catch (RocksDBException e) {
        throw new ProcessorStateException("Error while batch writing to store " + this.name, e);
    }
}
Also used : RocksDBException(org.rocksdb.RocksDBException) WriteBatch(org.rocksdb.WriteBatch) ProcessorStateException(org.apache.kafka.streams.errors.ProcessorStateException)

Example 9 with WriteBatch

use of org.rocksdb.WriteBatch in project bookkeeper by apache.

the class MVCCStoreImpl method processPut.

synchronized PutResult<K, V> processPut(long revision, PutOp<K, V> op) {
    checkStoreOpen();
    WriteBatch batch = new WriteBatch();
    PutResult<K, V> result = null;
    try {
        result = put(revision, batch, op);
        executeBatch(batch);
        return result;
    } catch (StateStoreRuntimeException e) {
        if (null != result) {
            result.close();
        }
        throw e;
    } finally {
        RocksUtils.close(batch);
    }
}
Also used : KV(org.apache.bookkeeper.common.kv.KV) WriteBatch(org.rocksdb.WriteBatch) StateStoreRuntimeException(org.apache.bookkeeper.statelib.api.exceptions.StateStoreRuntimeException)

Example 10 with WriteBatch

use of org.rocksdb.WriteBatch in project bookkeeper by apache.

the class MVCCStoreImpl method processDelete.

synchronized DeleteResult<K, V> processDelete(long revision, DeleteOp<K, V> op) {
    checkStoreOpen();
    WriteBatch batch = new WriteBatch();
    DeleteResult<K, V> result = null;
    try {
        result = delete(revision, batch, op, true);
        executeBatch(batch);
        return result;
    } catch (StateStoreRuntimeException e) {
        if (null != result) {
            result.close();
        }
        throw e;
    } finally {
        RocksUtils.close(batch);
    }
}
Also used : KV(org.apache.bookkeeper.common.kv.KV) WriteBatch(org.rocksdb.WriteBatch) StateStoreRuntimeException(org.apache.bookkeeper.statelib.api.exceptions.StateStoreRuntimeException)

Aggregations

WriteBatch (org.rocksdb.WriteBatch)17 RocksDBException (org.rocksdb.RocksDBException)11 StateStoreRuntimeException (org.apache.bookkeeper.statelib.api.exceptions.StateStoreRuntimeException)5 ProcessorStateException (org.apache.kafka.streams.errors.ProcessorStateException)5 HashMap (java.util.HashMap)4 Map (java.util.Map)4 KV (org.apache.bookkeeper.common.kv.KV)4 WriteOptions (org.rocksdb.WriteOptions)4 ArrayList (java.util.ArrayList)2 MetricException (org.apache.storm.metricstore.MetricException)2 ColumnFamilyHandle (org.rocksdb.ColumnFamilyHandle)2 IOException (java.io.IOException)1 HashSet (java.util.HashSet)1 CompareOp (org.apache.bookkeeper.api.kv.op.CompareOp)1 CompareResult (org.apache.bookkeeper.api.kv.op.CompareResult)1 DeleteOp (org.apache.bookkeeper.api.kv.op.DeleteOp)1 IncrementOp (org.apache.bookkeeper.api.kv.op.IncrementOp)1 Op (org.apache.bookkeeper.api.kv.op.Op)1 PutOp (org.apache.bookkeeper.api.kv.op.PutOp)1 RangeOp (org.apache.bookkeeper.api.kv.op.RangeOp)1