Examples with RocksDBException org.rocksdb.RocksDBException used on opensource projects

Example 61 with RocksDBException

use of org.rocksdb.RocksDBException in project kafka by apache.

the class AbstractRocksDBSegmentedBytesStore method getWriteBatches.

// Visible for testing
Map<S, WriteBatch> getWriteBatches(final Collection<ConsumerRecord<byte[], byte[]>> records) {
    // advance stream time to the max timestamp in the batch
    for (final ConsumerRecord<byte[], byte[]> record : records) {
        final long timestamp = keySchema.segmentTimestamp(Bytes.wrap(record.key()));
        observedStreamTime = Math.max(observedStreamTime, timestamp);
    }
    final Map<S, WriteBatch> writeBatchMap = new HashMap<>();
    for (final ConsumerRecord<byte[], byte[]> record : records) {
        final long timestamp = keySchema.segmentTimestamp(Bytes.wrap(record.key()));
        final long segmentId = segments.segmentId(timestamp);
        final S segment = segments.getOrCreateSegmentIfLive(segmentId, context, observedStreamTime);
        if (segment != null) {
            ChangelogRecordDeserializationHelper.applyChecksAndUpdatePosition(record, consistencyEnabled, position);
            try {
                final WriteBatch batch = writeBatchMap.computeIfAbsent(segment, s -> new WriteBatch());
                segment.addToBatch(new KeyValue<>(record.key(), record.value()), batch);
            } catch (final RocksDBException e) {
                throw new ProcessorStateException("Error restoring batch to store " + this.name, e);
            }
        }
    }
    return writeBatchMap;
}

Example 62 with RocksDBException

use of org.rocksdb.RocksDBException in project alluxio by Alluxio.

the class RocksPageStore method open.

/**
 * @param pageStoreOptions options for the rocks page store
 * @return a new instance of {@link PageStore} backed by RocksDB
 * @throws IOException if I/O error happens
 */
public static RocksPageStore open(RocksPageStoreOptions pageStoreOptions) throws IOException {
    Preconditions.checkArgument(pageStoreOptions.getMaxPageSize() > 0);
    RocksDB.loadLibrary();
    // The RocksObject will be closed together with the RocksPageStore
    Options rocksOptions = new Options().setCreateIfMissing(true).setWriteBufferSize(pageStoreOptions.getWriteBufferSize()).setCompressionType(pageStoreOptions.getCompressionType());
    RocksDB db = null;
    try {
        // TODO(maobaolong): rocksdb support only one root for now.
        db = RocksDB.open(rocksOptions, pageStoreOptions.getRootDirs().get(0).toString());
        byte[] confData = db.get(CONF_KEY);
        Cache.PRocksPageStoreOptions pOptions = pageStoreOptions.toProto();
        if (confData != null) {
            Cache.PRocksPageStoreOptions persistedOptions = Cache.PRocksPageStoreOptions.parseFrom(confData);
            if (!persistedOptions.equals(pOptions)) {
                db.close();
                throw new IOException("Inconsistent configuration for RocksPageStore");
            }
        }
        db.put(CONF_KEY, pOptions.toByteArray());
    } catch (RocksDBException e) {
        if (db != null) {
            db.close();
        }
        rocksOptions.close();
        throw new IOException("Couldn't open rocksDB database", e);
    }
    return new RocksPageStore(pageStoreOptions, db, rocksOptions);
}

Example 63 with RocksDBException

use of org.rocksdb.RocksDBException in project alluxio by Alluxio.

the class RocksPageStore method get.

@Override
public int get(PageId pageId, int pageOffset, int bytesToRead, byte[] buffer, int bufferOffset) throws IOException, PageNotFoundException {
    Preconditions.checkArgument(pageOffset >= 0, "page offset should be non-negative");
    try {
        byte[] page = mDb.get(getKeyFromPageId(pageId));
        if (page == null) {
            throw new PageNotFoundException(new String(getKeyFromPageId(pageId)));
        }
        Preconditions.checkArgument(pageOffset <= page.length, "page offset %s exceeded page size %s", pageOffset, page.length);
        try (ByteArrayInputStream bais = new ByteArrayInputStream(page)) {
            int bytesSkipped = (int) bais.skip(pageOffset);
            if (pageOffset != bytesSkipped) {
                throw new IOException(String.format("Failed to read page %s from offset %s: %s bytes skipped", pageId, pageOffset, bytesSkipped));
            }
            int bytesRead = 0;
            int bytesLeft = Math.min(page.length - pageOffset, buffer.length - bufferOffset);
            bytesLeft = Math.min(bytesLeft, bytesToRead);
            while (bytesLeft >= 0) {
                int bytes = bais.read(buffer, bufferOffset + bytesRead, bytesLeft);
                if (bytes <= 0) {
                    break;
                }
                bytesRead += bytes;
                bytesLeft -= bytes;
            }
            return bytesRead;
        }
    } catch (RocksDBException e) {
        throw new IOException("Failed to retrieve page", e);
    }
}

Example 64 with RocksDBException

use of org.rocksdb.RocksDBException in project flink by apache.

the class RocksDBListState method getInternal.

@Override
public List<V> getInternal() {
    try {
        byte[] key = serializeCurrentKeyWithGroupAndNamespace();
        byte[] valueBytes = backend.db.get(columnFamily, key);
        return listSerializer.deserializeList(valueBytes, elementSerializer);
    } catch (RocksDBException e) {
        throw new FlinkRuntimeException("Error while retrieving data from RocksDB", e);
    }
}

Example 65 with RocksDBException

use of org.rocksdb.RocksDBException in project flink by apache.

the class RocksDBValueState method value.

@Override
public V value() {
    try {
        byte[] valueBytes = backend.db.get(columnFamily, serializeCurrentKeyWithGroupAndNamespace());
        if (valueBytes == null) {
            return getDefaultValue();
        }
        dataInputView.setBuffer(valueBytes);
        return valueSerializer.deserialize(dataInputView);
    } catch (IOException | RocksDBException e) {
        throw new FlinkRuntimeException("Error while retrieving data from RocksDB.", e);
    }
}