Search in sources :

Example 1 with WriteBatch

use of org.rocksdb.WriteBatch in project bookkeeper by apache.

the class RocksdbKVStore method putRaw.

private void putRaw(K key, byte[] keyBytes, V value, long revision) {
    WriteBatch batch = new WriteBatch();
    if (revision > 0) {
        // last revision has been set to revision bytes
        batch.put(metaCfHandle, LAST_REVISION, lastRevisionBytes);
    }
    if (null == value) {
        // delete a key if value is null
        batch.remove(dataCfHandle, keyBytes);
    } else {
        byte[] valBytes = valCoder.encode(value);
        batch.put(dataCfHandle, keyBytes, valBytes);
    }
    try {
        db.write(writeOpts, batch);
    } catch (RocksDBException e) {
        throw new StateStoreRuntimeException("Error while updating key " + key + " to value " + value + " from store " + name, e);
    }
}
Also used : RocksDBException(org.rocksdb.RocksDBException) WriteBatch(org.rocksdb.WriteBatch) StateStoreRuntimeException(org.apache.bookkeeper.statelib.api.exceptions.StateStoreRuntimeException)

Example 2 with WriteBatch

use of org.rocksdb.WriteBatch in project bookkeeper by apache.

the class MVCCStoreImpl method processTxn.

synchronized TxnResult<K, V> processTxn(long revision, TxnOp<K, V> op) {
    checkStoreOpen();
    // 1. process the compares
    boolean success = processCompares(op);
    // 2. prepare the response list
    List<Op<K, V>> operations;
    List<Result<K, V>> results;
    if (success) {
        operations = op.successOps();
    } else {
        operations = op.failureOps();
    }
    if (operations == null) {
        operations = Collections.emptyList();
    }
    results = Lists.newArrayListWithExpectedSize(operations.size());
    // 3. process the operations
    try (WriteBatch batch = new WriteBatch()) {
        for (Op<K, V> o : operations) {
            results.add(executeOp(revision, batch, o));
        }
        executeBatch(batch);
        // 4. repare the result
        TxnResultImpl<K, V> txnResult = resultFactory.newTxnResult(revision);
        txnResult.isSuccess(success);
        txnResult.results(results);
        txnResult.code(Code.OK);
        return txnResult;
    } catch (StateStoreRuntimeException e) {
        results.forEach(Result::close);
        throw e;
    }
}
Also used : TxnOp(org.apache.bookkeeper.api.kv.op.TxnOp) CompareOp(org.apache.bookkeeper.api.kv.op.CompareOp) PutOp(org.apache.bookkeeper.api.kv.op.PutOp) Op(org.apache.bookkeeper.api.kv.op.Op) IncrementOp(org.apache.bookkeeper.api.kv.op.IncrementOp) RangeOp(org.apache.bookkeeper.api.kv.op.RangeOp) DeleteOp(org.apache.bookkeeper.api.kv.op.DeleteOp) KV(org.apache.bookkeeper.common.kv.KV) WriteBatch(org.rocksdb.WriteBatch) StateStoreRuntimeException(org.apache.bookkeeper.statelib.api.exceptions.StateStoreRuntimeException) PutResult(org.apache.bookkeeper.api.kv.result.PutResult) Result(org.apache.bookkeeper.api.kv.result.Result) IncrementResult(org.apache.bookkeeper.api.kv.result.IncrementResult) RangeResult(org.apache.bookkeeper.api.kv.result.RangeResult) TxnResult(org.apache.bookkeeper.api.kv.result.TxnResult) DeleteResult(org.apache.bookkeeper.api.kv.result.DeleteResult) CompareResult(org.apache.bookkeeper.api.kv.op.CompareResult)

Example 3 with WriteBatch

use of org.rocksdb.WriteBatch in project bookkeeper by apache.

the class MVCCStoreImpl method processIncrement.

synchronized IncrementResult<K, V> processIncrement(long revision, IncrementOp<K, V> op) {
    checkStoreOpen();
    WriteBatch batch = new WriteBatch();
    IncrementResult<K, V> result = null;
    try {
        result = increment(revision, batch, op);
        executeBatch(batch);
        return result;
    } catch (StateStoreRuntimeException e) {
        if (null != result) {
            result.close();
        }
        throw e;
    } finally {
        RocksUtils.close(batch);
    }
}
Also used : KV(org.apache.bookkeeper.common.kv.KV) WriteBatch(org.rocksdb.WriteBatch) StateStoreRuntimeException(org.apache.bookkeeper.statelib.api.exceptions.StateStoreRuntimeException)

Example 4 with WriteBatch

use of org.rocksdb.WriteBatch in project kafka by apache.

the class AbstractRocksDBSegmentedBytesStore method restoreAllInternal.

// Visible for testing
void restoreAllInternal(final Collection<ConsumerRecord<byte[], byte[]>> records) {
    try {
        final Map<S, WriteBatch> writeBatchMap = getWriteBatches(records);
        for (final Map.Entry<S, WriteBatch> entry : writeBatchMap.entrySet()) {
            final S segment = entry.getKey();
            final WriteBatch batch = entry.getValue();
            segment.write(batch);
            batch.close();
        }
    } catch (final RocksDBException e) {
        throw new ProcessorStateException("Error restoring batch to store " + this.name, e);
    }
}
Also used : RocksDBException(org.rocksdb.RocksDBException) WriteBatch(org.rocksdb.WriteBatch) HashMap(java.util.HashMap) Map(java.util.Map) ProcessorStateException(org.apache.kafka.streams.errors.ProcessorStateException)

Example 5 with WriteBatch

use of org.rocksdb.WriteBatch in project kafka by apache.

the class RocksDBStore method putAll.

@Override
public void putAll(final List<KeyValue<Bytes, byte[]>> entries) {
    try (final WriteBatch batch = new WriteBatch()) {
        dbAccessor.prepareBatch(entries, batch);
        StoreQueryUtils.updatePosition(position, context);
        write(batch);
    } catch (final RocksDBException e) {
        throw new ProcessorStateException("Error while batch writing to store " + name, e);
    }
}
Also used : RocksDBException(org.rocksdb.RocksDBException) WriteBatch(org.rocksdb.WriteBatch) ProcessorStateException(org.apache.kafka.streams.errors.ProcessorStateException)

Aggregations

WriteBatch (org.rocksdb.WriteBatch)17 RocksDBException (org.rocksdb.RocksDBException)11 StateStoreRuntimeException (org.apache.bookkeeper.statelib.api.exceptions.StateStoreRuntimeException)5 ProcessorStateException (org.apache.kafka.streams.errors.ProcessorStateException)5 HashMap (java.util.HashMap)4 Map (java.util.Map)4 KV (org.apache.bookkeeper.common.kv.KV)4 WriteOptions (org.rocksdb.WriteOptions)4 ArrayList (java.util.ArrayList)2 MetricException (org.apache.storm.metricstore.MetricException)2 ColumnFamilyHandle (org.rocksdb.ColumnFamilyHandle)2 IOException (java.io.IOException)1 HashSet (java.util.HashSet)1 CompareOp (org.apache.bookkeeper.api.kv.op.CompareOp)1 CompareResult (org.apache.bookkeeper.api.kv.op.CompareResult)1 DeleteOp (org.apache.bookkeeper.api.kv.op.DeleteOp)1 IncrementOp (org.apache.bookkeeper.api.kv.op.IncrementOp)1 Op (org.apache.bookkeeper.api.kv.op.Op)1 PutOp (org.apache.bookkeeper.api.kv.op.PutOp)1 RangeOp (org.apache.bookkeeper.api.kv.op.RangeOp)1