Search in sources :

Example 26 with StreamSegmentAppendOperation

use of io.pravega.segmentstore.server.logs.operations.StreamSegmentAppendOperation in project pravega by pravega.

the class OperationProcessorTests method testConcurrentStopAndCommit.

/**
 * Tests a scenario where the OperationProcessor is shut down while a DataFrame is being processed and will eventually
 * complete successfully - however its operation should be cancelled.
 */
@Test
public void testConcurrentStopAndCommit() throws Exception {
    @Cleanup TestContext context = new TestContext();
    // Generate some test data.
    val segmentId = createStreamSegmentsInMetadata(1, context.metadata).stream().findFirst().orElse(-1L);
    List<Operation> operations = Collections.singletonList(new StreamSegmentAppendOperation(segmentId, new byte[1], null));
    CompletableFuture<LogAddress> appendCallback = new CompletableFuture<>();
    // Setup an OperationProcessor with a custom DurableDataLog and start it.
    @Cleanup DurableDataLog dataLog = new ManualAppendOnlyDurableDataLog(() -> appendCallback);
    dataLog.initialize(TIMEOUT);
    @Cleanup OperationProcessor operationProcessor = new OperationProcessor(context.metadata, context.stateUpdater, dataLog, getNoOpCheckpointPolicy(), executorService());
    operationProcessor.startAsync().awaitRunning();
    // Process all generated operations.
    OperationWithCompletion completionFuture = processOperations(operations, operationProcessor).stream().findFirst().orElse(null);
    operationProcessor.stopAsync();
    appendCallback.complete(new TestLogAddress(1));
    // Stop the processor.
    operationProcessor.awaitTerminated();
    // Wait for the operation to complete. The operation should have been cancelled (due to the OperationProcessor
    // shutting down) - no other exception (or successful completion is accepted).
    AssertExtensions.assertThrows("Operation did not fail with the right exception.", () -> completionFuture.completion, ex -> ex instanceof CancellationException || ex instanceof ObjectClosedException);
}
Also used : lombok.val(lombok.val) DurableDataLog(io.pravega.segmentstore.storage.DurableDataLog) TestDurableDataLog(io.pravega.segmentstore.server.TestDurableDataLog) ProbeOperation(io.pravega.segmentstore.server.logs.operations.ProbeOperation) Operation(io.pravega.segmentstore.server.logs.operations.Operation) StorageOperation(io.pravega.segmentstore.server.logs.operations.StorageOperation) StreamSegmentAppendOperation(io.pravega.segmentstore.server.logs.operations.StreamSegmentAppendOperation) StreamSegmentAppendOperation(io.pravega.segmentstore.server.logs.operations.StreamSegmentAppendOperation) Cleanup(lombok.Cleanup) LogAddress(io.pravega.segmentstore.storage.LogAddress) CompletableFuture(java.util.concurrent.CompletableFuture) CancellationException(java.util.concurrent.CancellationException) ObjectClosedException(io.pravega.common.ObjectClosedException) Test(org.junit.Test)

Aggregations

StreamSegmentAppendOperation (io.pravega.segmentstore.server.logs.operations.StreamSegmentAppendOperation)26 MergeTransactionOperation (io.pravega.segmentstore.server.logs.operations.MergeTransactionOperation)13 lombok.val (lombok.val)12 Test (org.junit.Test)12 CachedStreamSegmentAppendOperation (io.pravega.segmentstore.server.logs.operations.CachedStreamSegmentAppendOperation)9 Operation (io.pravega.segmentstore.server.logs.operations.Operation)9 StorageOperation (io.pravega.segmentstore.server.logs.operations.StorageOperation)9 StreamSegmentMapOperation (io.pravega.segmentstore.server.logs.operations.StreamSegmentMapOperation)8 StreamSegmentSealOperation (io.pravega.segmentstore.server.logs.operations.StreamSegmentSealOperation)8 ArrayList (java.util.ArrayList)8 UpdateableContainerMetadata (io.pravega.segmentstore.server.UpdateableContainerMetadata)7 UpdateableSegmentMetadata (io.pravega.segmentstore.server.UpdateableSegmentMetadata)7 MetadataCheckpointOperation (io.pravega.segmentstore.server.logs.operations.MetadataCheckpointOperation)7 StreamSegmentTruncateOperation (io.pravega.segmentstore.server.logs.operations.StreamSegmentTruncateOperation)6 HashMap (java.util.HashMap)6 StorageMetadataCheckpointOperation (io.pravega.segmentstore.server.logs.operations.StorageMetadataCheckpointOperation)5 DataCorruptionException (io.pravega.segmentstore.server.DataCorruptionException)4 SegmentMetadata (io.pravega.segmentstore.server.SegmentMetadata)4 ProbeOperation (io.pravega.segmentstore.server.logs.operations.ProbeOperation)4 SegmentOperation (io.pravega.segmentstore.server.logs.operations.SegmentOperation)4