Search in sources :

Example 6 with TimeoutTimer

use of io.pravega.common.TimeoutTimer in project pravega by pravega.

the class StreamSegmentContainer method getStreamSegmentInfo.

@Override
public CompletableFuture<SegmentProperties> getStreamSegmentInfo(String streamSegmentName, boolean waitForPendingOps, Duration timeout) {
    ensureRunning();
    logRequest("getStreamSegmentInfo", streamSegmentName);
    this.metrics.getInfo();
    if (waitForPendingOps) {
        // We have been instructed to wait for all pending operations to complete. Use an op barrier and wait for it
        // before proceeding.
        TimeoutTimer timer = new TimeoutTimer(timeout);
        return this.durableLog.operationProcessingBarrier(timer.getRemaining()).thenComposeAsync(v -> this.segmentMapper.getStreamSegmentInfo(streamSegmentName, timer.getRemaining()), this.executor);
    } else {
        return this.segmentMapper.getStreamSegmentInfo(streamSegmentName, timeout);
    }
}
Also used : TimeoutTimer(io.pravega.common.TimeoutTimer)

Example 7 with TimeoutTimer

use of io.pravega.common.TimeoutTimer in project pravega by pravega.

the class StreamSegmentMapper method assignStreamSegmentId.

/**
 * Attempts to map a StreamSegment to an Id, by first trying to retrieve an existing id, and, should that not exist,
 * assign a new one.
 *
 * @param streamSegmentName The name of the StreamSegment to map.
 * @param timeout           Timeout for the operation.
 */
private void assignStreamSegmentId(String streamSegmentName, Duration timeout) {
    TimeoutTimer timer = new TimeoutTimer(timeout);
    withFailureHandler(this.storage.getStreamSegmentInfo(streamSegmentName, timer.getRemaining()).thenComposeAsync(si -> attachState(si, timer.getRemaining()), this.executor).thenComposeAsync(si -> submitToOperationLogWithRetry(si, ContainerMetadata.NO_STREAM_SEGMENT_ID, timer.getRemaining()), this.executor), streamSegmentName);
}
Also used : Storage(io.pravega.segmentstore.storage.Storage) StreamSegmentNotExistsException(io.pravega.segmentstore.contracts.StreamSegmentNotExistsException) SegmentRollingPolicy(io.pravega.segmentstore.storage.SegmentRollingPolicy) TooManyActiveSegmentsException(io.pravega.segmentstore.contracts.TooManyActiveSegmentsException) Exceptions(io.pravega.common.Exceptions) RequiredArgsConstructor(lombok.RequiredArgsConstructor) ContainerMetadata(io.pravega.segmentstore.server.ContainerMetadata) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) AtomicReference(java.util.concurrent.atomic.AtomicReference) Function(java.util.function.Function) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) SegmentProperties(io.pravega.segmentstore.contracts.SegmentProperties) AttributeUpdate(io.pravega.segmentstore.contracts.AttributeUpdate) SegmentMetadata(io.pravega.segmentstore.server.SegmentMetadata) Duration(java.time.Duration) BiConsumer(java.util.function.BiConsumer) StreamSegmentMapOperation(io.pravega.segmentstore.server.logs.operations.StreamSegmentMapOperation) LoggerHelpers(io.pravega.common.LoggerHelpers) Attributes(io.pravega.segmentstore.contracts.Attributes) TimeoutTimer(io.pravega.common.TimeoutTimer) Executor(java.util.concurrent.Executor) Collection(java.util.Collection) OperationLog(io.pravega.segmentstore.server.OperationLog) CompletionException(java.util.concurrent.CompletionException) ThreadSafe(javax.annotation.concurrent.ThreadSafe) UUID(java.util.UUID) GuardedBy(javax.annotation.concurrent.GuardedBy) StreamSegmentNameUtils(io.pravega.shared.segment.StreamSegmentNameUtils) Slf4j(lombok.extern.slf4j.Slf4j) AsyncMap(io.pravega.common.util.AsyncMap) StreamSegmentExistsException(io.pravega.segmentstore.contracts.StreamSegmentExistsException) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) DataCorruptionException(io.pravega.segmentstore.server.DataCorruptionException) Futures(io.pravega.common.concurrent.Futures) NotThreadSafe(javax.annotation.concurrent.NotThreadSafe) TimeoutTimer(io.pravega.common.TimeoutTimer)

Example 8 with TimeoutTimer

use of io.pravega.common.TimeoutTimer in project pravega by pravega.

the class StreamSegmentMapper method createNewStreamSegment.

// endregion
// region Create Segments
/**
 * Creates a new StreamSegment with given name (in Storage) and persists the given attributes (in Storage).
 *
 * @param streamSegmentName The case-sensitive StreamSegment Name.
 * @param attributes        The initial attributes for the StreamSegment, if any.
 * @param timeout           Timeout for the operation.
 * @return A CompletableFuture that, when completed normally, will indicate the operation completed normally.
 * If the operation failed, this will contain the exception that caused the failure.
 */
public CompletableFuture<Void> createNewStreamSegment(String streamSegmentName, Collection<AttributeUpdate> attributes, Duration timeout) {
    long traceId = LoggerHelpers.traceEnterWithContext(log, traceObjectId, "createNewStreamSegment", streamSegmentName);
    long segmentId = this.containerMetadata.getStreamSegmentId(streamSegmentName, true);
    if (isValidStreamSegmentId(segmentId)) {
        // Quick fail: see if this is an active Segment, and if so, don't bother with anything else.
        return Futures.failedFuture(new StreamSegmentExistsException(streamSegmentName));
    }
    CompletableFuture<Void> result = createSegmentInStorageWithRecovery(streamSegmentName, attributes, new TimeoutTimer(timeout));
    if (log.isTraceEnabled()) {
        result.thenAccept(v -> LoggerHelpers.traceLeave(log, traceObjectId, "createNewStreamSegment", traceId, streamSegmentName));
    }
    return result;
}
Also used : StreamSegmentExistsException(io.pravega.segmentstore.contracts.StreamSegmentExistsException) TimeoutTimer(io.pravega.common.TimeoutTimer)

Example 9 with TimeoutTimer

use of io.pravega.common.TimeoutTimer in project pravega by pravega.

the class StreamSegmentMapper method assignTransactionStreamSegmentId.

/**
 * Attempts to map a Transaction StreamSegment to its parent StreamSegment (and assign an id in the process, if needed).
 *
 * @param transactionSegmentName The Name for the Transaction to assign Id for.
 * @param parentSegmentName      The Name of the Parent StreamSegment.
 * @param timeout                The timeout for the operation.
 * @return A CompletableFuture that, when completed normally, will contain the StreamSegment Id requested. If the operation
 * failed, this will contain the exception that caused the failure.
 */
private CompletableFuture<Long> assignTransactionStreamSegmentId(String transactionSegmentName, String parentSegmentName, Duration timeout) {
    TimeoutTimer timer = new TimeoutTimer(timeout);
    AtomicReference<Long> parentSegmentId = new AtomicReference<>();
    // Get info about parent. This also verifies the parent exists.
    return withFailureHandler(getOrAssignStreamSegmentId(parentSegmentName, timer.getRemaining(), id -> {
        // Get info about Transaction itself.
        parentSegmentId.set(id);
        return this.storage.getStreamSegmentInfo(transactionSegmentName, timer.getRemaining());
    }).thenCompose(transInfo -> attachState(transInfo, timer.getRemaining())).thenCompose(transInfo -> assignTransactionStreamSegmentId(transInfo, parentSegmentId.get(), timer.getRemaining())), transactionSegmentName);
}
Also used : Storage(io.pravega.segmentstore.storage.Storage) StreamSegmentNotExistsException(io.pravega.segmentstore.contracts.StreamSegmentNotExistsException) SegmentRollingPolicy(io.pravega.segmentstore.storage.SegmentRollingPolicy) TooManyActiveSegmentsException(io.pravega.segmentstore.contracts.TooManyActiveSegmentsException) Exceptions(io.pravega.common.Exceptions) RequiredArgsConstructor(lombok.RequiredArgsConstructor) ContainerMetadata(io.pravega.segmentstore.server.ContainerMetadata) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) AtomicReference(java.util.concurrent.atomic.AtomicReference) Function(java.util.function.Function) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) SegmentProperties(io.pravega.segmentstore.contracts.SegmentProperties) AttributeUpdate(io.pravega.segmentstore.contracts.AttributeUpdate) SegmentMetadata(io.pravega.segmentstore.server.SegmentMetadata) Duration(java.time.Duration) BiConsumer(java.util.function.BiConsumer) StreamSegmentMapOperation(io.pravega.segmentstore.server.logs.operations.StreamSegmentMapOperation) LoggerHelpers(io.pravega.common.LoggerHelpers) Attributes(io.pravega.segmentstore.contracts.Attributes) TimeoutTimer(io.pravega.common.TimeoutTimer) Executor(java.util.concurrent.Executor) Collection(java.util.Collection) OperationLog(io.pravega.segmentstore.server.OperationLog) CompletionException(java.util.concurrent.CompletionException) ThreadSafe(javax.annotation.concurrent.ThreadSafe) UUID(java.util.UUID) GuardedBy(javax.annotation.concurrent.GuardedBy) StreamSegmentNameUtils(io.pravega.shared.segment.StreamSegmentNameUtils) Slf4j(lombok.extern.slf4j.Slf4j) AsyncMap(io.pravega.common.util.AsyncMap) StreamSegmentExistsException(io.pravega.segmentstore.contracts.StreamSegmentExistsException) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) DataCorruptionException(io.pravega.segmentstore.server.DataCorruptionException) Futures(io.pravega.common.concurrent.Futures) NotThreadSafe(javax.annotation.concurrent.NotThreadSafe) AtomicReference(java.util.concurrent.atomic.AtomicReference) TimeoutTimer(io.pravega.common.TimeoutTimer)

Example 10 with TimeoutTimer

use of io.pravega.common.TimeoutTimer in project pravega by pravega.

the class StreamSegmentMapper method createNewTransactionStreamSegment.

/**
 * Creates a new Transaction StreamSegment for an existing Parent StreamSegment and persists the given attributes (in Storage).
 *
 * @param parentStreamSegmentName The case-sensitive StreamSegment Name of the Parent StreamSegment.
 * @param transactionId           A unique identifier for the transaction to be created.
 * @param attributes              The initial attributes for the Transaction, if any.
 * @param timeout                 Timeout for the operation.
 * @return A CompletableFuture that, when completed normally, will contain the name of the newly created Transaction StreamSegment.
 * If the operation failed, this will contain the exception that caused the failure.
 * @throws IllegalArgumentException If the given parent StreamSegment cannot have a Transaction (because it is deleted, sealed, inexistent).
 */
public CompletableFuture<String> createNewTransactionStreamSegment(String parentStreamSegmentName, UUID transactionId, Collection<AttributeUpdate> attributes, Duration timeout) {
    long traceId = LoggerHelpers.traceEnterWithContext(log, traceObjectId, "createNewTransactionStreamSegment", parentStreamSegmentName);
    // We cannot create a Transaction StreamSegment for a what looks like another Transaction.
    Exceptions.checkArgument(StreamSegmentNameUtils.getParentStreamSegmentName(parentStreamSegmentName) == null, "parentStreamSegmentName", "Cannot create a Transaction for a Transaction.");
    // Validate that Parent StreamSegment exists.
    TimeoutTimer timer = new TimeoutTimer(timeout);
    CompletableFuture<Void> parentCheck = null;
    long mappedParentId = this.containerMetadata.getStreamSegmentId(parentStreamSegmentName, true);
    if (isValidStreamSegmentId(mappedParentId)) {
        SegmentProperties parentInfo = this.containerMetadata.getStreamSegmentMetadata(mappedParentId);
        if (parentInfo != null) {
            parentCheck = validateParentSegmentEligibility(parentInfo);
        }
    }
    if (parentCheck == null) {
        // The parent is not registered in the metadata. Get required info from Storage and don't map it unnecessarily.
        parentCheck = this.storage.getStreamSegmentInfo(parentStreamSegmentName, timer.getRemaining()).thenCompose(this::validateParentSegmentEligibility);
    }
    String transactionName = StreamSegmentNameUtils.getTransactionNameFromId(parentStreamSegmentName, transactionId);
    return parentCheck.thenComposeAsync(parentId -> createSegmentInStorageWithRecovery(transactionName, attributes, timer), this.executor).thenApply(v -> {
        LoggerHelpers.traceLeave(log, traceObjectId, "createNewTransactionStreamSegment", traceId, parentStreamSegmentName, transactionName);
        return transactionName;
    });
}
Also used : Storage(io.pravega.segmentstore.storage.Storage) StreamSegmentNotExistsException(io.pravega.segmentstore.contracts.StreamSegmentNotExistsException) SegmentRollingPolicy(io.pravega.segmentstore.storage.SegmentRollingPolicy) TooManyActiveSegmentsException(io.pravega.segmentstore.contracts.TooManyActiveSegmentsException) Exceptions(io.pravega.common.Exceptions) RequiredArgsConstructor(lombok.RequiredArgsConstructor) ContainerMetadata(io.pravega.segmentstore.server.ContainerMetadata) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) AtomicReference(java.util.concurrent.atomic.AtomicReference) Function(java.util.function.Function) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) SegmentProperties(io.pravega.segmentstore.contracts.SegmentProperties) AttributeUpdate(io.pravega.segmentstore.contracts.AttributeUpdate) SegmentMetadata(io.pravega.segmentstore.server.SegmentMetadata) Duration(java.time.Duration) BiConsumer(java.util.function.BiConsumer) StreamSegmentMapOperation(io.pravega.segmentstore.server.logs.operations.StreamSegmentMapOperation) LoggerHelpers(io.pravega.common.LoggerHelpers) Attributes(io.pravega.segmentstore.contracts.Attributes) TimeoutTimer(io.pravega.common.TimeoutTimer) Executor(java.util.concurrent.Executor) Collection(java.util.Collection) OperationLog(io.pravega.segmentstore.server.OperationLog) CompletionException(java.util.concurrent.CompletionException) ThreadSafe(javax.annotation.concurrent.ThreadSafe) UUID(java.util.UUID) GuardedBy(javax.annotation.concurrent.GuardedBy) StreamSegmentNameUtils(io.pravega.shared.segment.StreamSegmentNameUtils) Slf4j(lombok.extern.slf4j.Slf4j) AsyncMap(io.pravega.common.util.AsyncMap) StreamSegmentExistsException(io.pravega.segmentstore.contracts.StreamSegmentExistsException) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) DataCorruptionException(io.pravega.segmentstore.server.DataCorruptionException) Futures(io.pravega.common.concurrent.Futures) NotThreadSafe(javax.annotation.concurrent.NotThreadSafe) SegmentProperties(io.pravega.segmentstore.contracts.SegmentProperties) TimeoutTimer(io.pravega.common.TimeoutTimer)

Aggregations

TimeoutTimer (io.pravega.common.TimeoutTimer)97 lombok.val (lombok.val)55 CompletableFuture (java.util.concurrent.CompletableFuture)52 Duration (java.time.Duration)51 Futures (io.pravega.common.concurrent.Futures)47 Preconditions (com.google.common.base.Preconditions)41 CompletionException (java.util.concurrent.CompletionException)41 Slf4j (lombok.extern.slf4j.Slf4j)41 Collectors (java.util.stream.Collectors)40 SneakyThrows (lombok.SneakyThrows)40 Exceptions (io.pravega.common.Exceptions)39 BufferView (io.pravega.common.util.BufferView)37 Getter (lombok.Getter)37 SegmentProperties (io.pravega.segmentstore.contracts.SegmentProperties)36 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)36 StreamSegmentNotExistsException (io.pravega.segmentstore.contracts.StreamSegmentNotExistsException)34 SegmentMetadata (io.pravega.segmentstore.server.SegmentMetadata)34 RequiredArgsConstructor (lombok.RequiredArgsConstructor)34 ArrayList (java.util.ArrayList)33 List (java.util.List)31