Examples with DurableDataLogException io.pravega.segmentstore.storage.DurableDataLogException used on opensource projects

Example 6 with DurableDataLogException

use of io.pravega.segmentstore.storage.DurableDataLogException in project pravega by pravega.

the class OperationProcessorTests method testWithDataLogFailures.

/**
 * Tests the ability of the OperationProcessor to process Operations when there are DataLog write failures. The expected
 * outcome is that the OperationProcessor will auto-shutdown when such errors are encountered.
 */
@Test
public void testWithDataLogFailures() throws Exception {
    int streamSegmentCount = 10;
    int appendsPerStreamSegment = 80;
    // Fail (asynchronously) after X DataFrame commits (to DataLog).
    int failAfterCommits = 5;
    @Cleanup TestContext context = new TestContext();
    // Generate some test data (no need to complicate ourselves with Transactions here; that is tested in the no-failure test).
    HashSet<Long> streamSegmentIds = createStreamSegmentsInMetadata(streamSegmentCount, context.metadata);
    List<Operation> operations = generateOperations(streamSegmentIds, new HashMap<>(), appendsPerStreamSegment, METADATA_CHECKPOINT_EVERY, false, false);
    // Setup an OperationProcessor and start it.
    @Cleanup TestDurableDataLog dataLog = TestDurableDataLog.create(CONTAINER_ID, MAX_DATA_LOG_APPEND_SIZE, executorService());
    dataLog.initialize(TIMEOUT);
    @Cleanup OperationProcessor operationProcessor = new OperationProcessor(context.metadata, context.stateUpdater, dataLog, getNoOpCheckpointPolicy(), executorService());
    operationProcessor.startAsync().awaitRunning();
    ErrorInjector<Exception> aSyncErrorInjector = new ErrorInjector<>(count -> count >= failAfterCommits, () -> new DurableDataLogException("intentional"));
    dataLog.setAppendErrorInjectors(null, aSyncErrorInjector);
    // Process all generated operations.
    List<OperationWithCompletion> completionFutures = processOperations(operations, operationProcessor);
    // Wait for all such operations to complete. We are expecting exceptions, so verify that we do.
    AssertExtensions.assertThrows("No operations failed.", OperationWithCompletion.allOf(completionFutures)::join, super::isExpectedExceptionForNonDataCorruption);
    // Wait for the OperationProcessor to shutdown with failure.
    ServiceListeners.awaitShutdown(operationProcessor, TIMEOUT, false);
    Assert.assertEquals("Expected the OperationProcessor to fail after DurableDataLogException encountered.", Service.State.FAILED, operationProcessor.state());
    performLogOperationChecks(completionFutures, context.memoryLog, dataLog, context.metadata);
    performMetadataChecks(streamSegmentIds, new HashSet<>(), new HashMap<>(), completionFutures, context.metadata, false, false);
    performReadIndexChecks(completionFutures, context.readIndex);
}

Example 7 with DurableDataLogException

use of io.pravega.segmentstore.storage.DurableDataLogException in project pravega by pravega.

the class BookKeeperLog method initialize.

// endregion
// region DurableDataLog Implementation
/**
 * Open-Fences this BookKeeper log using the following protocol:
 * 1. Read Log Metadata from ZooKeeper.
 * 2. Fence at least the last 2 ledgers in the Ledger List.
 * 3. Create a new Ledger.
 * 3.1 If any of the steps so far fails, the process is interrupted at the point of failure, and no cleanup is attempted.
 * 4. Update Log Metadata using compare-and-set (this update contains the new ledger and new epoch).
 * 4.1 If CAS fails on metadata update, the newly created Ledger is deleted (this means we were fenced out by some
 * other instance) and no other update is performed.
 *
 * @param timeout Timeout for the operation.
 * @throws DataLogWriterNotPrimaryException If we were fenced-out during this process.
 * @throws DataLogNotAvailableException     If BookKeeper or ZooKeeper are not available.
 * @throws DataLogDisabledException         If the BookKeeperLog is disabled. No fencing is attempted in this case.
 * @throws DataLogInitializationException   If a general initialization error occurred.
 * @throws DurableDataLogException          If another type of exception occurred.
 */
@Override
public void initialize(Duration timeout) throws DurableDataLogException {
    List<Long> ledgersToDelete;
    LogMetadata newMetadata;
    synchronized (this.lock) {
        Preconditions.checkState(this.writeLedger == null, "BookKeeperLog is already initialized.");
        assert this.logMetadata == null : "writeLedger == null but logMetadata != null";
        // Get metadata about the current state of the log, if any.
        LogMetadata oldMetadata = loadMetadata();
        if (oldMetadata != null) {
            if (!oldMetadata.isEnabled()) {
                throw new DataLogDisabledException("BookKeeperLog is disabled. Cannot initialize.");
            }
            // Fence out ledgers.
            val emptyLedgerIds = Ledgers.fenceOut(oldMetadata.getLedgers(), this.bookKeeper, this.config, this.traceObjectId);
            // Update Metadata to reflect those newly found empty ledgers.
            oldMetadata = oldMetadata.updateLedgerStatus(emptyLedgerIds);
        }
        // Create new ledger.
        LedgerHandle newLedger = Ledgers.create(this.bookKeeper, this.config);
        log.info("{}: Created Ledger {}.", this.traceObjectId, newLedger.getId());
        // Update Metadata with new Ledger and persist to ZooKeeper.
        newMetadata = updateMetadata(oldMetadata, newLedger, true);
        LedgerMetadata ledgerMetadata = newMetadata.getLedger(newLedger.getId());
        assert ledgerMetadata != null : "cannot find newly added ledger metadata";
        this.writeLedger = new WriteLedger(newLedger, ledgerMetadata);
        this.logMetadata = newMetadata;
        ledgersToDelete = getLedgerIdsToDelete(oldMetadata, newMetadata);
    }
    // Delete the orphaned ledgers from BookKeeper.
    ledgersToDelete.forEach(id -> {
        try {
            Ledgers.delete(id, this.bookKeeper);
            log.info("{}: Deleted orphan empty ledger {}.", this.traceObjectId, id);
        } catch (DurableDataLogException ex) {
            // A failure here has no effect on the initialization of BookKeeperLog. In this case, the (empty) Ledger
            // will remain in BookKeeper until manually deleted by a cleanup tool.
            log.warn("{}: Unable to delete orphan empty ledger {}.", this.traceObjectId, id, ex);
        }
    });
    log.info("{}: Initialized (Epoch = {}, UpdateVersion = {}).", this.traceObjectId, newMetadata.getEpoch(), newMetadata.getUpdateVersion());
}

Example 8 with DurableDataLogException

use of io.pravega.segmentstore.storage.DurableDataLogException in project pravega by pravega.

the class BookKeeperLog method tryTruncate.

/**
 * Attempts to truncate the Log. The general steps are:
 * 1. Create an in-memory copy of the metadata reflecting the truncation.
 * 2. Attempt to persist the metadata to ZooKeeper.
 * 2.1. This is the only operation that can fail the process. If this fails, the operation stops here.
 * 3. Swap in-memory metadata pointers.
 * 4. Delete truncated-out ledgers.
 * 4.1. If any of the ledgers cannot be deleted, no further attempt to clean them up is done.
 *
 * @param upToAddress The address up to which to truncate.
 */
@SneakyThrows(DurableDataLogException.class)
private void tryTruncate(LedgerAddress upToAddress) {
    long traceId = LoggerHelpers.traceEnterWithContext(log, this.traceObjectId, "tryTruncate", upToAddress);
    // Truncate the metadata and get a new copy of it.
    val oldMetadata = getLogMetadata();
    val newMetadata = oldMetadata.truncate(upToAddress);
    // Attempt to persist the new Log Metadata. We need to do this first because if we delete the ledgers but were
    // unable to update the metadata, then the log will be corrupted (metadata points to inexistent ledgers).
    persistMetadata(newMetadata, false);
    // Repoint our metadata to the new one.
    synchronized (this.lock) {
        this.logMetadata = newMetadata;
    }
    // Determine ledgers to delete and delete them.
    val ledgerIdsToKeep = newMetadata.getLedgers().stream().map(LedgerMetadata::getLedgerId).collect(Collectors.toSet());
    val ledgersToDelete = oldMetadata.getLedgers().stream().filter(lm -> !ledgerIdsToKeep.contains(lm.getLedgerId())).iterator();
    while (ledgersToDelete.hasNext()) {
        val lm = ledgersToDelete.next();
        try {
            Ledgers.delete(lm.getLedgerId(), this.bookKeeper);
        } catch (DurableDataLogException ex) {
            // Nothing we can do if we can't delete a ledger; we've already updated the metadata. Log the error and
            // move on.
            log.error("{}: Unable to delete truncated ledger {}.", this.traceObjectId, lm.getLedgerId(), ex);
        }
    }
    log.info("{}: Truncated up to {}.", this.traceObjectId, upToAddress);
    LoggerHelpers.traceLeave(log, this.traceObjectId, "tryTruncate", traceId, upToAddress);
}

Example 9 with DurableDataLogException

use of io.pravega.segmentstore.storage.DurableDataLogException in project pravega by pravega.

the class LogReader method openNextLedger.

private void openNextLedger(LedgerAddress address) throws DurableDataLogException {
    if (address == null) {
        // We have reached the end.
        close();
        return;
    }
    LedgerMetadata metadata = this.metadata.getLedger(address.getLedgerId());
    assert metadata != null : "no LedgerMetadata could be found with valid LedgerAddress " + address;
    val allMetadatas = this.metadata.getLedgers();
    // Open the ledger.
    LedgerHandle ledger;
    if (allMetadatas.size() == 0 || metadata == allMetadatas.get(allMetadatas.size() - 1)) {
        // This is our last ledger (the active one); we need to make sure open it without recovery since otherwise we
        // we would fence ourselves out.
        ledger = Ledgers.openRead(metadata.getLedgerId(), this.bookKeeper, this.config);
    } else {
        // Older ledger. Open with recovery to make sure any uncommitted fragments will be recovered. Since we do our
        // Log fencing based on the last Ledger, open-fencing this Ledger will not have any adverse effects.
        ledger = Ledgers.openFence(metadata.getLedgerId(), this.bookKeeper, this.config);
    }
    long lastEntryId = ledger.getLastAddConfirmed();
    if (lastEntryId < address.getEntryId()) {
        // This ledger is empty.
        Ledgers.close(ledger);
        this.currentLedger = new ReadLedger(metadata, ledger, null);
        return;
    }
    ReadLedger previousLedger;
    try {
        val reader = Exceptions.handleInterrupted(() -> ledger.readEntries(address.getEntryId(), lastEntryId));
        previousLedger = this.currentLedger;
        this.currentLedger = new ReadLedger(metadata, ledger, reader);
        if (previousLedger != null) {
            // Close previous ledger handle.
            Ledgers.close(previousLedger.handle);
        }
    } catch (Exception ex) {
        Ledgers.close(ledger);
        close();
        throw new DurableDataLogException("Error while reading from BookKeeper.", ex);
    }
}

Example 10 with DurableDataLogException

use of io.pravega.segmentstore.storage.DurableDataLogException in project pravega by pravega.

the class BookKeeperAdapter method createStream.

@Override
public CompletableFuture<Void> createStream(String logName, Duration timeout) {
    ensureRunning();
    int id;
    synchronized (this.internalIds) {
        if (this.internalIds.containsKey(logName)) {
            return Futures.failedFuture(new StreamSegmentExistsException(logName));
        }
        id = this.internalIds.size();
        this.internalIds.put(logName, id);
    }
    return CompletableFuture.runAsync(() -> {
        DurableDataLog log = null;
        boolean success = false;
        try {
            log = this.logFactory.createDurableDataLog(id);
            this.logs.put(logName, log);
            log.initialize(timeout);
            success = true;
        } catch (DurableDataLogException ex) {
            throw new CompletionException(ex);
        } finally {
            if (!success) {
                this.logs.remove(logName);
                synchronized (this.internalIds) {
                    this.internalIds.remove(logName);
                }
                if (log != null) {
                    log.close();
                }
            }
        }
    }, this.executor);
}