Search in sources :

Example 11 with MetadataCheckpointOperation

use of io.pravega.segmentstore.server.logs.operations.MetadataCheckpointOperation in project pravega by pravega.

the class DurableLog method truncate.

@Override
public CompletableFuture<Void> truncate(long upToSequenceNumber, Duration timeout) {
    ensureRunning();
    Preconditions.checkArgument(this.metadata.isValidTruncationPoint(upToSequenceNumber), "Invalid Truncation Point. Must refer to a MetadataCheckpointOperation.");
    // The SequenceNumber we were given points directly to a MetadataCheckpointOperation. We must not remove it!
    // Instead, it must be the first operation that does survive, so we need to adjust our SeqNo to the one just
    // before it.
    long actualTruncationSequenceNumber = upToSequenceNumber - 1;
    // Find the closest Truncation Marker (that does not exceed it).
    LogAddress truncationFrameAddress = this.metadata.getClosestTruncationMarker(actualTruncationSequenceNumber);
    if (truncationFrameAddress == null) {
        // Nothing to truncate.
        return CompletableFuture.completedFuture(null);
    }
    TimeoutTimer timer = new TimeoutTimer(timeout);
    log.info("{}: Truncate (OperationSequenceNumber = {}, DataFrameAddress = {}).", this.traceObjectId, upToSequenceNumber, truncationFrameAddress);
    // info will be readily available upon recovery without delay.
    return add(new StorageMetadataCheckpointOperation(), timer.getRemaining()).thenComposeAsync(v -> this.durableDataLog.truncate(truncationFrameAddress, timer.getRemaining()), this.executor).thenRunAsync(() -> {
        // Truncate InMemory Transaction Log.
        int count = this.inMemoryOperationLog.truncate(actualTruncationSequenceNumber);
        // Remove old truncation markers.
        this.metadata.removeTruncationMarkers(actualTruncationSequenceNumber);
        this.operationProcessor.getMetrics().operationLogTruncate(count);
    }, this.executor);
}
Also used : ObjectClosedException(io.pravega.common.ObjectClosedException) IllegalContainerStateException(io.pravega.segmentstore.server.IllegalContainerStateException) SneakyThrows(lombok.SneakyThrows) Retry(io.pravega.common.util.Retry) StorageMetadataCheckpointOperation(io.pravega.segmentstore.server.logs.operations.StorageMetadataCheckpointOperation) ProbeOperation(io.pravega.segmentstore.server.logs.operations.ProbeOperation) Exceptions(io.pravega.common.Exceptions) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) CompletableFuture(java.util.concurrent.CompletableFuture) LogAddress(io.pravega.segmentstore.storage.LogAddress) StreamingException(io.pravega.segmentstore.contracts.StreamingException) AtomicReference(java.util.concurrent.atomic.AtomicReference) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) UpdateableContainerMetadata(io.pravega.segmentstore.server.UpdateableContainerMetadata) SequencedItemList(io.pravega.common.util.SequencedItemList) Duration(java.time.Duration) AbstractService(com.google.common.util.concurrent.AbstractService) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) Operation(io.pravega.segmentstore.server.logs.operations.Operation) DataLogDisabledException(io.pravega.segmentstore.storage.DataLogDisabledException) LoggerHelpers(io.pravega.common.LoggerHelpers) Services(io.pravega.common.concurrent.Services) ContainerOfflineException(io.pravega.segmentstore.server.ContainerOfflineException) DurableDataLog(io.pravega.segmentstore.storage.DurableDataLog) Iterator(java.util.Iterator) TimeoutTimer(io.pravega.common.TimeoutTimer) MetadataCheckpointOperation(io.pravega.segmentstore.server.logs.operations.MetadataCheckpointOperation) Set(java.util.Set) OperationLog(io.pravega.segmentstore.server.OperationLog) CompletionException(java.util.concurrent.CompletionException) ThreadSafe(javax.annotation.concurrent.ThreadSafe) GuardedBy(javax.annotation.concurrent.GuardedBy) Timer(io.pravega.common.Timer) Collectors(java.util.stream.Collectors) List(java.util.List) Slf4j(lombok.extern.slf4j.Slf4j) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) ReadIndex(io.pravega.segmentstore.server.ReadIndex) DataCorruptionException(io.pravega.segmentstore.server.DataCorruptionException) DurableDataLogFactory(io.pravega.segmentstore.storage.DurableDataLogFactory) Collections(java.util.Collections) Futures(io.pravega.common.concurrent.Futures) LogAddress(io.pravega.segmentstore.storage.LogAddress) StorageMetadataCheckpointOperation(io.pravega.segmentstore.server.logs.operations.StorageMetadataCheckpointOperation) TimeoutTimer(io.pravega.common.TimeoutTimer)

Aggregations

MetadataCheckpointOperation (io.pravega.segmentstore.server.logs.operations.MetadataCheckpointOperation)11 StorageMetadataCheckpointOperation (io.pravega.segmentstore.server.logs.operations.StorageMetadataCheckpointOperation)10 StreamSegmentMapOperation (io.pravega.segmentstore.server.logs.operations.StreamSegmentMapOperation)9 StreamSegmentAppendOperation (io.pravega.segmentstore.server.logs.operations.StreamSegmentAppendOperation)8 Operation (io.pravega.segmentstore.server.logs.operations.Operation)6 ProbeOperation (io.pravega.segmentstore.server.logs.operations.ProbeOperation)6 UpdateableContainerMetadata (io.pravega.segmentstore.server.UpdateableContainerMetadata)5 StorageOperation (io.pravega.segmentstore.server.logs.operations.StorageOperation)5 Test (org.junit.Test)5 Cleanup (lombok.Cleanup)4 MetadataBuilder (io.pravega.segmentstore.server.MetadataBuilder)3 ReadIndex (io.pravega.segmentstore.server.ReadIndex)3 TestDurableDataLogFactory (io.pravega.segmentstore.server.TestDurableDataLogFactory)3 CacheManager (io.pravega.segmentstore.server.reading.CacheManager)3 ContainerReadIndex (io.pravega.segmentstore.server.reading.ContainerReadIndex)3 AtomicReference (java.util.concurrent.atomic.AtomicReference)3 TestDurableDataLog (io.pravega.segmentstore.server.TestDurableDataLog)2 MergeTransactionOperation (io.pravega.segmentstore.server.logs.operations.MergeTransactionOperation)2 SegmentOperation (io.pravega.segmentstore.server.logs.operations.SegmentOperation)2 StreamSegmentSealOperation (io.pravega.segmentstore.server.logs.operations.StreamSegmentSealOperation)2