Search in sources :

Example 1 with StorageException

use of com.alipay.sofa.jraft.rhea.errors.StorageException in project sofa-jraft by sofastack.

the class RocksRawKVStore method writeSnapshot.

void writeSnapshot(final String snapshotPath) {
    final Timer.Context timeCtx = getTimeContext("WRITE_SNAPSHOT");
    final Lock writeLock = this.readWriteLock.writeLock();
    writeLock.lock();
    try (final Checkpoint checkpoint = Checkpoint.create(this.db)) {
        final String tempPath = snapshotPath + "_temp";
        final File tempFile = new File(tempPath);
        FileUtils.deleteDirectory(tempFile);
        checkpoint.createCheckpoint(tempPath);
        final File snapshotFile = new File(snapshotPath);
        FileUtils.deleteDirectory(snapshotFile);
        if (!Utils.atomicMoveFile(tempFile, snapshotFile, true)) {
            throw new StorageException("Fail to rename [" + tempPath + "] to [" + snapshotPath + "].");
        }
    } catch (final StorageException e) {
        throw e;
    } catch (final Exception e) {
        throw new StorageException("Fail to write snapshot at path: " + snapshotPath, e);
    } finally {
        writeLock.unlock();
        timeCtx.stop();
    }
}
Also used : Checkpoint(org.rocksdb.Checkpoint) Timer(com.codahale.metrics.Timer) File(java.io.File) StorageException(com.alipay.sofa.jraft.rhea.errors.StorageException) RocksDBException(org.rocksdb.RocksDBException) StorageException(com.alipay.sofa.jraft.rhea.errors.StorageException) IOException(java.io.IOException) ReadWriteLock(java.util.concurrent.locks.ReadWriteLock) DistributedLock(com.alipay.sofa.jraft.rhea.util.concurrent.DistributedLock) ReentrantReadWriteLock(java.util.concurrent.locks.ReentrantReadWriteLock) Lock(java.util.concurrent.locks.Lock)

Example 2 with StorageException

use of com.alipay.sofa.jraft.rhea.errors.StorageException in project sofa-jraft by sofastack.

the class RocksRawKVStore method backupDB.

RocksDBBackupInfo backupDB(final String backupDBPath) throws IOException {
    final Timer.Context timeCtx = getTimeContext("BACKUP_DB");
    FileUtils.forceMkdir(new File(backupDBPath));
    final Lock writeLock = this.readWriteLock.writeLock();
    writeLock.lock();
    try (final BackupableDBOptions backupOpts = createBackupDBOptions(backupDBPath);
        final BackupEngine backupEngine = BackupEngine.open(this.options.getEnv(), backupOpts)) {
        backupEngine.createNewBackup(this.db, true);
        final List<BackupInfo> backupInfoList = backupEngine.getBackupInfo();
        if (backupInfoList.isEmpty()) {
            LOG.warn("Fail to backup at {}, empty backup info.", backupDBPath);
            return null;
        }
        // chose the backupInfo who has max backupId
        final BackupInfo backupInfo = Collections.max(backupInfoList, Comparator.comparingInt(BackupInfo::backupId));
        final RocksDBBackupInfo rocksBackupInfo = new RocksDBBackupInfo(backupInfo);
        LOG.info("Backup rocksDB into {} with backupInfo {}.", backupDBPath, rocksBackupInfo);
        return rocksBackupInfo;
    } catch (final RocksDBException e) {
        throw new StorageException("Fail to backup at path: " + backupDBPath, e);
    } finally {
        writeLock.unlock();
        timeCtx.stop();
    }
}
Also used : BackupInfo(org.rocksdb.BackupInfo) BackupEngine(org.rocksdb.BackupEngine) RocksDBException(org.rocksdb.RocksDBException) Timer(com.codahale.metrics.Timer) BackupableDBOptions(org.rocksdb.BackupableDBOptions) File(java.io.File) StorageException(com.alipay.sofa.jraft.rhea.errors.StorageException) ReadWriteLock(java.util.concurrent.locks.ReadWriteLock) DistributedLock(com.alipay.sofa.jraft.rhea.util.concurrent.DistributedLock) ReentrantReadWriteLock(java.util.concurrent.locks.ReentrantReadWriteLock) Lock(java.util.concurrent.locks.Lock)

Example 3 with StorageException

use of com.alipay.sofa.jraft.rhea.errors.StorageException in project sofa-jraft by sofastack.

the class RocksKVStoreSnapshotFile method doSnapshotLoad.

@Override
void doSnapshotLoad(final String snapshotPath, final LocalFileMeta meta, final Region region) throws Exception {
    if (RegionHelper.isMultiGroup(region)) {
        final Region snapshotRegion = readMetadata(meta, Region.class);
        if (!RegionHelper.isSameRange(region, snapshotRegion)) {
            throw new StorageException("Invalid snapshot region: " + snapshotRegion + " current region is: " + region);
        }
        this.kvStore.readSstSnapshot(snapshotPath);
        return;
    }
    if (this.kvStore.isFastSnapshot()) {
        this.kvStore.readSnapshot(snapshotPath);
        return;
    }
    final RocksDBBackupInfo rocksBackupInfo = readMetadata(meta, RocksDBBackupInfo.class);
    this.kvStore.restoreBackup(snapshotPath, rocksBackupInfo);
}
Also used : Region(com.alipay.sofa.jraft.rhea.metadata.Region) StorageException(com.alipay.sofa.jraft.rhea.errors.StorageException)

Example 4 with StorageException

use of com.alipay.sofa.jraft.rhea.errors.StorageException in project sofa-jraft by sofastack.

the class RocksRawKVStore method writeSstSnapshot.

CompletableFuture<Void> writeSstSnapshot(final String snapshotPath, final Region region, final ExecutorService executor) {
    final Timer.Context timeCtx = getTimeContext("WRITE_SST_SNAPSHOT");
    final Lock readLock = this.readWriteLock.readLock();
    readLock.lock();
    try {
        final String tempPath = snapshotPath + "_temp";
        final File tempFile = new File(tempPath);
        FileUtils.deleteDirectory(tempFile);
        FileUtils.forceMkdir(tempFile);
        final EnumMap<SstColumnFamily, File> sstFileTable = getSstFileTable(tempPath);
        final CompletableFuture<Void> snapshotFuture = new CompletableFuture<>();
        final CompletableFuture<Void> sstFuture = createSstFiles(sstFileTable, region.getStartKey(), region.getEndKey(), executor);
        sstFuture.whenComplete((aVoid, throwable) -> {
            if (throwable == null) {
                try {
                    final File snapshotFile = new File(snapshotPath);
                    FileUtils.deleteDirectory(snapshotFile);
                    if (!Utils.atomicMoveFile(tempFile, snapshotFile, true)) {
                        throw new StorageException("Fail to rename [" + tempPath + "] to [" + snapshotPath + "].");
                    }
                    snapshotFuture.complete(null);
                } catch (final Throwable t) {
                    snapshotFuture.completeExceptionally(t);
                }
            } else {
                snapshotFuture.completeExceptionally(throwable);
            }
        });
        return snapshotFuture;
    } catch (final Exception e) {
        throw new StorageException("Fail to do read sst snapshot at path: " + snapshotPath, e);
    } finally {
        readLock.unlock();
        timeCtx.stop();
    }
}
Also used : RocksDBException(org.rocksdb.RocksDBException) StorageException(com.alipay.sofa.jraft.rhea.errors.StorageException) IOException(java.io.IOException) ReadWriteLock(java.util.concurrent.locks.ReadWriteLock) DistributedLock(com.alipay.sofa.jraft.rhea.util.concurrent.DistributedLock) ReentrantReadWriteLock(java.util.concurrent.locks.ReentrantReadWriteLock) Lock(java.util.concurrent.locks.Lock) CompletableFuture(java.util.concurrent.CompletableFuture) Timer(com.codahale.metrics.Timer) File(java.io.File) StorageException(com.alipay.sofa.jraft.rhea.errors.StorageException)

Example 5 with StorageException

use of com.alipay.sofa.jraft.rhea.errors.StorageException in project sofa-jraft by sofastack.

the class RocksRawKVStore method readSstSnapshot.

void readSstSnapshot(final String snapshotPath) {
    final Timer.Context timeCtx = getTimeContext("READ_SST_SNAPSHOT");
    final Lock readLock = this.readWriteLock.readLock();
    readLock.lock();
    try {
        final EnumMap<SstColumnFamily, File> sstFileTable = getSstFileTable(snapshotPath);
        ingestSstFiles(sstFileTable);
    } catch (final Exception e) {
        throw new StorageException("Fail to write sst snapshot at path: " + snapshotPath, e);
    } finally {
        readLock.unlock();
        timeCtx.stop();
    }
}
Also used : Timer(com.codahale.metrics.Timer) File(java.io.File) StorageException(com.alipay.sofa.jraft.rhea.errors.StorageException) RocksDBException(org.rocksdb.RocksDBException) StorageException(com.alipay.sofa.jraft.rhea.errors.StorageException) IOException(java.io.IOException) ReadWriteLock(java.util.concurrent.locks.ReadWriteLock) DistributedLock(com.alipay.sofa.jraft.rhea.util.concurrent.DistributedLock) ReentrantReadWriteLock(java.util.concurrent.locks.ReentrantReadWriteLock) Lock(java.util.concurrent.locks.Lock)

Aggregations

StorageException (com.alipay.sofa.jraft.rhea.errors.StorageException)11 DistributedLock (com.alipay.sofa.jraft.rhea.util.concurrent.DistributedLock)9 Timer (com.codahale.metrics.Timer)9 Lock (java.util.concurrent.locks.Lock)9 ReadWriteLock (java.util.concurrent.locks.ReadWriteLock)9 ReentrantReadWriteLock (java.util.concurrent.locks.ReentrantReadWriteLock)9 RocksDBException (org.rocksdb.RocksDBException)9 File (java.io.File)8 IOException (java.io.IOException)4 BackupableDBOptions (org.rocksdb.BackupableDBOptions)3 Region (com.alipay.sofa.jraft.rhea.metadata.Region)2 EnumMap (java.util.EnumMap)2 Map (java.util.Map)2 BackupEngine (org.rocksdb.BackupEngine)2 ColumnFamilyHandle (org.rocksdb.ColumnFamilyHandle)2 IngestExternalFileOptions (org.rocksdb.IngestExternalFileOptions)2 RestoreOptions (org.rocksdb.RestoreOptions)2 RocksDBOptions (com.alipay.sofa.jraft.rhea.options.RocksDBOptions)1 CompletableFuture (java.util.concurrent.CompletableFuture)1 BackupInfo (org.rocksdb.BackupInfo)1