Search in sources :

Example 16 with CompletionException

use of java.util.concurrent.CompletionException in project pravega by pravega.

the class TaskTest method testLocking.

@Test
public void testLocking() {
    TestTasks testTasks = new TestTasks(taskMetadataStore, executor, HOSTNAME);
    CompletableFuture<Void> first = testTasks.testStreamLock(SCOPE, stream1);
    CompletableFuture<Void> second = testTasks.testStreamLock(SCOPE, stream1);
    try {
        first.getNow(null);
        second.getNow(null);
    } catch (CompletionException ce) {
        assertTrue(ce.getCause() instanceof LockFailedException);
    }
}
Also used : LockFailedException(io.pravega.controller.store.task.LockFailedException) CompletionException(java.util.concurrent.CompletionException) TestTasks(io.pravega.controller.task.Stream.TestTasks) Test(org.junit.Test)

Example 17 with CompletionException

use of java.util.concurrent.CompletionException in project pravega by pravega.

the class OperationProcessor method doRun.

@Override
protected CompletableFuture<Void> doRun() {
    // The QueueProcessor is responsible with the processing of externally added Operations. It starts when the
    // OperationProcessor starts and is shut down as soon as doStop() is invoked.
    val queueProcessor = Futures.loop(this::isRunning, () -> throttle().thenComposeAsync(v -> this.operationQueue.take(MAX_READ_AT_ONCE), this.executor).thenAcceptAsync(this::processOperations, this.executor), this.executor);
    // The CommitProcessor is responsible with the processing of those Operations that have already been committed to
    // DurableDataLong and now need to be added to the in-memory State.
    // As opposed from the QueueProcessor, this needs to process all pending commits and not discard them, even when
    // we receive a stop signal (from doStop()), otherwise we could be left with an inconsistent in-memory state.
    val commitProcessor = Futures.loop(() -> isRunning() || this.commitQueue.size() > 0, () -> this.commitQueue.take(MAX_COMMIT_QUEUE_SIZE).thenAcceptAsync(this::processCommits, this.executor), this.executor).whenComplete((r, ex) -> {
        // The CommitProcessor is done. Safe to close its queue now, regardless of whether it failed or
        // shut down normally.
        this.commitQueue.close();
        if (ex != null) {
            throw new CompletionException(ex);
        }
    });
    return CompletableFuture.allOf(queueProcessor, commitProcessor).exceptionally(this::iterationErrorHandler);
}
Also used : lombok.val(lombok.val) CompletionException(java.util.concurrent.CompletionException)

Example 18 with CompletionException

use of java.util.concurrent.CompletionException in project pravega by pravega.

the class SegmentAggregator method beginReconciliation.

// endregion
// region Reconciliation
/**
 * Initiates the Storage reconciliation procedure. Gets the current state of the Segment from Storage, and based on that,
 * does one of the following:
 * * Nothing, if the Storage agrees with the Metadata.
 * * Throws a show-stopping DataCorruptionException (wrapped in a CompletionException) if the situation is unrecoverable.
 * * Initiates the Reconciliation Procedure, which allows the reconcile() method to execute.
 *
 * @param timer    Timer for the operation.
 * @return A CompletableFuture that indicates when the operation completed.
 */
private CompletableFuture<Void> beginReconciliation(TimeoutTimer timer) {
    assert this.state.get() == AggregatorState.ReconciliationNeeded : "beginReconciliation cannot be called if state == " + this.state;
    return this.storage.getStreamSegmentInfo(this.metadata.getName(), timer.getRemaining()).thenAcceptAsync(sp -> {
        if (sp.getLength() > this.metadata.getLength()) {
            // actor altering the Segment. We cannot recover automatically from this situation.
            throw new CompletionException(new ReconciliationFailureException("Actual Segment length in Storage is larger than the Metadata Length.", this.metadata, sp));
        } else if (sp.getLength() < this.metadata.getStorageLength()) {
            // We cannot recover automatically from this situation.
            throw new CompletionException(new ReconciliationFailureException("Actual Segment length in Storage is smaller than the Metadata StorageLength.", this.metadata, sp));
        } else if (sp.getLength() == this.metadata.getStorageLength()) {
            // Nothing to do.
            return;
        }
        // If we get here, it means we have work to do. Set the state accordingly and move on.
        this.reconciliationState.set(new ReconciliationState(this.metadata, sp));
        setState(AggregatorState.Reconciling);
    }, this.executor);
}
Also used : CompletionException(java.util.concurrent.CompletionException)

Example 19 with CompletionException

use of java.util.concurrent.CompletionException in project pravega by pravega.

the class StreamSegmentContainer method deleteStreamSegment.

@Override
public CompletableFuture<Void> deleteStreamSegment(String streamSegmentName, Duration timeout) {
    ensureRunning();
    logRequest("deleteStreamSegment", streamSegmentName);
    this.metrics.deleteSegment();
    TimeoutTimer timer = new TimeoutTimer(timeout);
    // metadata.deleteStreamSegment will delete the given StreamSegment and all Transactions associated with it.
    // It returns a mapping of segment ids to names of StreamSegments that were deleted.
    // As soon as this happens, all operations that deal with those segments will start throwing appropriate exceptions
    // or ignore the segments altogether (such as StorageWriter).
    Collection<SegmentMetadata> deletedSegments = this.metadata.deleteStreamSegment(streamSegmentName);
    val deletionFutures = new ArrayList<CompletableFuture<Void>>();
    for (SegmentMetadata toDelete : deletedSegments) {
        deletionFutures.add(this.storage.openWrite(toDelete.getName()).thenComposeAsync(handle -> this.storage.delete(handle, timer.getRemaining()), this.executor).thenComposeAsync(v -> this.stateStore.remove(toDelete.getName(), timer.getRemaining()), this.executor).exceptionally(ex -> {
            ex = Exceptions.unwrap(ex);
            if (ex instanceof StreamSegmentNotExistsException && toDelete.isTransaction()) {
                // did not get a chance to get updated.
                return null;
            }
            throw new CompletionException(ex);
        }));
    }
    notifyMetadataRemoved(deletedSegments);
    return Futures.allOf(deletionFutures);
}
Also used : SegmentMetadata(io.pravega.segmentstore.server.SegmentMetadata) lombok.val(lombok.val) ObjectClosedException(io.pravega.common.ObjectClosedException) Storage(io.pravega.segmentstore.storage.Storage) StreamSegmentNotExistsException(io.pravega.segmentstore.contracts.StreamSegmentNotExistsException) IllegalContainerStateException(io.pravega.segmentstore.server.IllegalContainerStateException) Exceptions(io.pravega.common.Exceptions) StorageFactory(io.pravega.segmentstore.storage.StorageFactory) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) CompletableFuture(java.util.concurrent.CompletableFuture) AtomicReference(java.util.concurrent.atomic.AtomicReference) ArrayList(java.util.ArrayList) SegmentProperties(io.pravega.segmentstore.contracts.SegmentProperties) ReadIndexFactory(io.pravega.segmentstore.server.ReadIndexFactory) AttributeUpdate(io.pravega.segmentstore.contracts.AttributeUpdate) SegmentMetadata(io.pravega.segmentstore.server.SegmentMetadata) Duration(java.time.Duration) AbstractService(com.google.common.util.concurrent.AbstractService) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) Operation(io.pravega.segmentstore.server.logs.operations.Operation) StreamSegmentTruncateOperation(io.pravega.segmentstore.server.logs.operations.StreamSegmentTruncateOperation) LoggerHelpers(io.pravega.common.LoggerHelpers) WriterFactory(io.pravega.segmentstore.server.WriterFactory) Services(io.pravega.common.concurrent.Services) ContainerOfflineException(io.pravega.segmentstore.server.ContainerOfflineException) TimeoutTimer(io.pravega.common.TimeoutTimer) SegmentStoreMetrics(io.pravega.segmentstore.server.SegmentStoreMetrics) UpdateAttributesOperation(io.pravega.segmentstore.server.logs.operations.UpdateAttributesOperation) Collection(java.util.Collection) lombok.val(lombok.val) OperationLog(io.pravega.segmentstore.server.OperationLog) CompletionException(java.util.concurrent.CompletionException) Writer(io.pravega.segmentstore.server.Writer) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) Service(com.google.common.util.concurrent.Service) Objects(java.util.Objects) Consumer(java.util.function.Consumer) Slf4j(lombok.extern.slf4j.Slf4j) AsyncMap(io.pravega.common.util.AsyncMap) StreamSegmentAppendOperation(io.pravega.segmentstore.server.logs.operations.StreamSegmentAppendOperation) OperationLogFactory(io.pravega.segmentstore.server.OperationLogFactory) SegmentContainer(io.pravega.segmentstore.server.SegmentContainer) Preconditions(com.google.common.base.Preconditions) ReadIndex(io.pravega.segmentstore.server.ReadIndex) MergeTransactionOperation(io.pravega.segmentstore.server.logs.operations.MergeTransactionOperation) Futures(io.pravega.common.concurrent.Futures) ReadResult(io.pravega.segmentstore.contracts.ReadResult) StreamSegmentSealOperation(io.pravega.segmentstore.server.logs.operations.StreamSegmentSealOperation) CompletionException(java.util.concurrent.CompletionException) ArrayList(java.util.ArrayList) StreamSegmentNotExistsException(io.pravega.segmentstore.contracts.StreamSegmentNotExistsException) TimeoutTimer(io.pravega.common.TimeoutTimer)

Example 20 with CompletionException

use of java.util.concurrent.CompletionException in project pravega by pravega.

the class TxnSweeper method failOverTxn.

private CompletableFuture<Result> failOverTxn(String failedHost, TxnResource txn) {
    String scope = txn.getScope();
    String stream = txn.getStream();
    UUID txnId = txn.getTxnId();
    log.debug("Host = {}, processing transaction {}/{}/{}", failedHost, scope, stream, txnId);
    return streamMetadataStore.getTransactionData(scope, stream, txnId, null, executor).handle((r, e) -> {
        if (e != null) {
            if (Exceptions.unwrap(e) instanceof StoreException.DataNotFoundException) {
                // transaction not found, which means it should already have completed. We will ignore such txns
                return VersionedTransactionData.EMPTY;
            } else {
                throw new CompletionException(e);
            }
        }
        return r;
    }).thenComposeAsync(txData -> {
        int epoch = txData.getEpoch();
        switch(txData.getStatus()) {
            case OPEN:
                return failOverOpenTxn(failedHost, txn).handleAsync((v, e) -> new Result(txn, v, e), executor);
            case ABORTING:
                return failOverAbortingTxn(failedHost, epoch, txn).handleAsync((v, e) -> new Result(txn, v, e), executor);
            case COMMITTING:
                return failOverCommittingTxn(failedHost, epoch, txn).handleAsync((v, e) -> new Result(txn, v, e), executor);
            case UNKNOWN:
            default:
                return streamMetadataStore.removeTxnFromIndex(failedHost, txn, true).thenApply(x -> new Result(txn, null, null));
        }
    }, executor).whenComplete((v, e) -> log.debug("Host = {}, processing transaction {}/{}/{} complete", failedHost, scope, stream, txnId));
}
Also used : RetryHelper.withRetriesAsync(io.pravega.controller.util.RetryHelper.withRetriesAsync) Exceptions(io.pravega.common.Exceptions) Set(java.util.Set) CompletableFuture(java.util.concurrent.CompletableFuture) CompletionException(java.util.concurrent.CompletionException) UUID(java.util.UUID) Supplier(java.util.function.Supplier) Collectors(java.util.stream.Collectors) FailoverSweeper(io.pravega.controller.fault.FailoverSweeper) TxnResource(io.pravega.controller.store.task.TxnResource) Slf4j(lombok.extern.slf4j.Slf4j) StoreException(io.pravega.controller.store.stream.StoreException) TxnStatus(io.pravega.controller.store.stream.TxnStatus) Duration(java.time.Duration) VersionedTransactionData(io.pravega.controller.store.stream.VersionedTransactionData) Data(lombok.Data) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) RETRYABLE_PREDICATE(io.pravega.controller.util.RetryHelper.RETRYABLE_PREDICATE) Preconditions(com.google.common.base.Preconditions) StreamMetadataStore(io.pravega.controller.store.stream.StreamMetadataStore) Futures(io.pravega.common.concurrent.Futures) CompletionException(java.util.concurrent.CompletionException) UUID(java.util.UUID)

Aggregations

CompletionException (java.util.concurrent.CompletionException)199 Test (org.junit.Test)80 CompletableFuture (java.util.concurrent.CompletableFuture)62 List (java.util.List)52 ArrayList (java.util.ArrayList)51 IOException (java.io.IOException)45 Map (java.util.Map)39 Collection (java.util.Collection)31 ExecutionException (java.util.concurrent.ExecutionException)31 HashMap (java.util.HashMap)30 Collections (java.util.Collections)24 TimeUnit (java.util.concurrent.TimeUnit)22 Collectors (java.util.stream.Collectors)22 FlinkException (org.apache.flink.util.FlinkException)22 Before (org.junit.Before)21 Duration (java.time.Duration)19 Arrays (java.util.Arrays)19 BeforeClass (org.junit.BeforeClass)19 ExecutorService (java.util.concurrent.ExecutorService)18 Nonnull (javax.annotation.Nonnull)17