Search in sources :

Example 91 with TimeoutTimer

use of io.pravega.common.TimeoutTimer in project pravega by pravega.

the class TableBucketReaderTests method testFindEntryNotExists.

/**
 * Tests the ability to (not) locate Table Entries in a Table Bucket for deleted and inexistent keys.
 */
@Test
public void testFindEntryNotExists() throws Exception {
    val segment = new SegmentMock(executorService());
    // Generate our test data.
    val es = new EntrySerializer();
    val entries = generateEntries(es);
    // Deleted key (that was previously indexed).
    val deletedKey = entries.get(0).getKey();
    val data = es.serializeRemoval(Collections.singleton(deletedKey));
    segment.append(data, null, TIMEOUT).join();
    val reader = TableBucketReader.entry(segment, // No backpointers.
    (s, offset, timeout) -> CompletableFuture.completedFuture(-1L), executorService());
    val deletedResult = reader.find(deletedKey.getKey(), 0L, new TimeoutTimer(TIMEOUT)).get(TIMEOUT.toMillis(), TimeUnit.MILLISECONDS);
    Assert.assertNull("Expecting a TableEntry with null value for deleted key.", deletedResult.getValue());
    // Inexistent key (that did not exist previously).
    val inexistentKey = entries.get(1).getKey();
    val inexistentResult = reader.find(inexistentKey.getKey(), 0L, new TimeoutTimer(TIMEOUT)).get(TIMEOUT.toMillis(), TimeUnit.MILLISECONDS);
    Assert.assertNull("Not expecting any result for key that was never present.", inexistentResult);
}
Also used : lombok.val(lombok.val) SegmentMock(io.pravega.segmentstore.server.SegmentMock) TimeoutTimer(io.pravega.common.TimeoutTimer) Test(org.junit.Test)

Example 92 with TimeoutTimer

use of io.pravega.common.TimeoutTimer in project pravega by pravega.

the class MetadataStore method assignSegmentId.

/**
 * Attempts to map a Segment to an Id, by first trying to retrieve an existing id, and, should that not exist,
 * assign a new one. If the operation failed, either synchronously, or asynchronously, the segment assignment will be
 * failed with the causing exception.
 *
 * @param segmentName The name of the Segment to assign id for.
 * @param timeout     Timeout for the operation.
 */
private void assignSegmentId(String segmentName, Duration timeout) {
    try {
        TimeoutTimer timer = new TimeoutTimer(timeout);
        Futures.exceptionListener(getSegmentInfoInternal(segmentName, timer.getRemaining()).thenComposeAsync(si -> submitAssignmentWithRetry(SegmentInfo.deserialize(si), timer.getRemaining()), this.executor), ex -> failAssignment(segmentName, ex));
    } catch (Throwable ex) {
        log.warn("{}: Unable to assign Id for segment '{}'.", this.traceObjectId, segmentName, ex);
        failAssignment(segmentName, ex);
    }
}
Also used : StreamSegmentInformation(io.pravega.segmentstore.contracts.StreamSegmentInformation) StreamSegmentNotExistsException(io.pravega.segmentstore.contracts.StreamSegmentNotExistsException) SneakyThrows(lombok.SneakyThrows) RequiredArgsConstructor(lombok.RequiredArgsConstructor) ContainerMetadata(io.pravega.segmentstore.server.ContainerMetadata) RevisionDataInput(io.pravega.common.io.serialization.RevisionDataInput) SegmentProperties(io.pravega.segmentstore.contracts.SegmentProperties) SegmentInfo.newSegment(io.pravega.segmentstore.server.containers.MetadataStore.SegmentInfo.newSegment) ArrayView(io.pravega.common.util.ArrayView) AttributeUpdate(io.pravega.segmentstore.contracts.AttributeUpdate) BufferView(io.pravega.common.util.BufferView) Duration(java.time.Duration) SerializationException(io.pravega.common.io.SerializationException) RevisionDataOutput(io.pravega.common.io.serialization.RevisionDataOutput) VersionedSerializer(io.pravega.common.io.serialization.VersionedSerializer) Attributes(io.pravega.segmentstore.contracts.Attributes) NonNull(lombok.NonNull) Collection(java.util.Collection) CompletionException(java.util.concurrent.CompletionException) ThreadSafe(javax.annotation.concurrent.ThreadSafe) GuardedBy(javax.annotation.concurrent.GuardedBy) EOFException(java.io.EOFException) Collectors(java.util.stream.Collectors) Slf4j(lombok.extern.slf4j.Slf4j) StreamSegmentExistsException(io.pravega.segmentstore.contracts.StreamSegmentExistsException) Builder(lombok.Builder) Futures(io.pravega.common.concurrent.Futures) ObjectClosedException(io.pravega.common.ObjectClosedException) Getter(lombok.Getter) TooManyActiveSegmentsException(io.pravega.segmentstore.contracts.TooManyActiveSegmentsException) Exceptions(io.pravega.common.Exceptions) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) Function(java.util.function.Function) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) SegmentMetadata(io.pravega.segmentstore.server.SegmentMetadata) SegmentType(io.pravega.segmentstore.contracts.SegmentType) BiConsumer(java.util.function.BiConsumer) LoggerHelpers(io.pravega.common.LoggerHelpers) NameUtils(io.pravega.shared.NameUtils) TimeoutTimer(io.pravega.common.TimeoutTimer) Executor(java.util.concurrent.Executor) AttributeId(io.pravega.segmentstore.contracts.AttributeId) lombok.val(lombok.val) IOException(java.io.IOException) StreamSegmentMergedException(io.pravega.segmentstore.contracts.StreamSegmentMergedException) ObjectBuilder(io.pravega.common.ObjectBuilder) AttributeUpdateCollection(io.pravega.segmentstore.contracts.AttributeUpdateCollection) Data(lombok.Data) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) AttributeUpdateType(io.pravega.segmentstore.contracts.AttributeUpdateType) DataCorruptionException(io.pravega.segmentstore.server.DataCorruptionException) NotThreadSafe(javax.annotation.concurrent.NotThreadSafe) TimeoutTimer(io.pravega.common.TimeoutTimer)

Example 93 with TimeoutTimer

use of io.pravega.common.TimeoutTimer in project pravega by pravega.

the class SegmentAttributeBTreeIndex method initialize.

/**
 * Initializes the SegmentAttributeIndex.
 *
 * @param timeout Timeout for the operation.
 * @return A CompletableFuture that, when completed, will indicate the operation has succeeded.
 */
CompletableFuture<Void> initialize(Duration timeout) {
    TimeoutTimer timer = new TimeoutTimer(timeout);
    Preconditions.checkState(!this.index.isInitialized(), "SegmentAttributeIndex is already initialized.");
    String attributeSegmentName = NameUtils.getAttributeSegmentName(this.segmentMetadata.getName());
    // make the first write.
    return Futures.exceptionallyExpecting(this.storage.openWrite(attributeSegmentName).thenAccept(this.handle::set), ex -> ex instanceof StreamSegmentNotExistsException, null).thenComposeAsync(v -> this.index.initialize(timer.getRemaining()), this.executor).thenRun(() -> log.debug("{}: Initialized.", this.traceObjectId)).exceptionally(this::handleIndexOperationException);
}
Also used : Storage(io.pravega.segmentstore.storage.Storage) StreamSegmentNotExistsException(io.pravega.segmentstore.contracts.StreamSegmentNotExistsException) SneakyThrows(lombok.SneakyThrows) Retry(io.pravega.common.util.Retry) RequiredArgsConstructor(lombok.RequiredArgsConstructor) SegmentProperties(io.pravega.segmentstore.contracts.SegmentProperties) StreamSegmentSealedException(io.pravega.segmentstore.contracts.StreamSegmentSealedException) SegmentHandle(io.pravega.segmentstore.storage.SegmentHandle) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) BufferView(io.pravega.common.util.BufferView) Duration(java.time.Duration) Map(java.util.Map) Attributes(io.pravega.segmentstore.contracts.Attributes) NonNull(lombok.NonNull) Collection(java.util.Collection) CompletionException(java.util.concurrent.CompletionException) GuardedBy(javax.annotation.concurrent.GuardedBy) Collectors(java.util.stream.Collectors) Objects(java.util.Objects) List(java.util.List) Slf4j(lombok.extern.slf4j.Slf4j) ByteArraySegment(io.pravega.common.util.ByteArraySegment) PageEntry(io.pravega.common.util.btree.PageEntry) StreamSegmentExistsException(io.pravega.segmentstore.contracts.StreamSegmentExistsException) BadOffsetException(io.pravega.segmentstore.contracts.BadOffsetException) Futures(io.pravega.common.concurrent.Futures) CacheManager(io.pravega.segmentstore.server.CacheManager) Getter(lombok.Getter) Exceptions(io.pravega.common.Exceptions) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) AtomicReference(java.util.concurrent.atomic.AtomicReference) Function(java.util.function.Function) Supplier(java.util.function.Supplier) CacheStorage(io.pravega.segmentstore.storage.cache.CacheStorage) ArrayList(java.util.ArrayList) SegmentMetadata(io.pravega.segmentstore.server.SegmentMetadata) BTreeIndex(io.pravega.common.util.btree.BTreeIndex) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) CacheFullException(io.pravega.segmentstore.storage.cache.CacheFullException) AttributeIterator(io.pravega.segmentstore.server.AttributeIterator) StreamSegmentTruncatedException(io.pravega.segmentstore.contracts.StreamSegmentTruncatedException) NameUtils(io.pravega.shared.NameUtils) AttributeIndex(io.pravega.segmentstore.server.AttributeIndex) TimeoutTimer(io.pravega.common.TimeoutTimer) SequenceInputStream(java.io.SequenceInputStream) AttributeId(io.pravega.segmentstore.contracts.AttributeId) lombok.val(lombok.val) AsyncIterator(io.pravega.common.util.AsyncIterator) Maps(com.google.common.collect.Maps) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) IllegalDataFormatException(io.pravega.common.util.IllegalDataFormatException) DataCorruptionException(io.pravega.segmentstore.server.DataCorruptionException) AllArgsConstructor(lombok.AllArgsConstructor) Collections(java.util.Collections) InputStream(java.io.InputStream) StreamSegmentNotExistsException(io.pravega.segmentstore.contracts.StreamSegmentNotExistsException) TimeoutTimer(io.pravega.common.TimeoutTimer)

Example 94 with TimeoutTimer

use of io.pravega.common.TimeoutTimer in project pravega by pravega.

the class SegmentAttributeBTreeIndex method writePages.

private CompletableFuture<Long> writePages(List<BTreeIndex.WritePage> pages, Collection<Long> obsoleteOffsets, long truncateOffset, Duration timeout) {
    // The write offset is the offset of the first page to be written in the list.
    long writeOffset = pages.get(0).getOffset();
    // Collect the data to be written.
    val streams = new ArrayList<InputStream>();
    AtomicInteger length = new AtomicInteger();
    for (val p : pages) {
        // Validate that the given pages are indeed in the correct order.
        Preconditions.checkArgument(p.getOffset() == writeOffset + length.get(), "Unexpected page offset.");
        // Collect the pages (as InputStreams) and record their lengths.
        streams.add(p.getContents().getReader());
        length.addAndGet(p.getContents().getLength());
    }
    // Create the Attribute Segment in Storage (if needed), then write the new data to it and truncate if necessary.
    TimeoutTimer timer = new TimeoutTimer(timeout);
    return createAttributeSegmentIfNecessary(() -> writeToSegment(streams, writeOffset, length.get(), timer), timer.getRemaining()).thenApplyAsync(v -> {
        Exceptions.checkNotClosed(this.closed.get(), this);
        // Trigger an async truncation. There is no need to wait for it.
        truncateAsync(truncateOffset, timer.getRemaining());
        // Store data in cache and remove obsolete pages.
        storeInCache(pages, obsoleteOffsets);
        // Return the current length of the Segment Attribute Index.
        return writeOffset + length.get();
    }, this.executor);
}
Also used : lombok.val(lombok.val) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) ArrayList(java.util.ArrayList) TimeoutTimer(io.pravega.common.TimeoutTimer)

Example 95 with TimeoutTimer

use of io.pravega.common.TimeoutTimer in project pravega by pravega.

the class ReadOnlySegmentContainer method read.

@Override
public CompletableFuture<ReadResult> read(String streamSegmentName, long offset, int maxLength, Duration timeout) {
    Exceptions.checkNotClosed(this.closed.get(), this);
    TimeoutTimer timer = new TimeoutTimer(timeout);
    return READ_RETRY.run(() -> getStreamSegmentInfo(streamSegmentName, timer.getRemaining()).thenApply(si -> StreamSegmentStorageReader.read(si, offset, maxLength, MAX_READ_AT_ONCE_BYTES, this.storage)));
}
Also used : StreamSegmentNotExistsException(io.pravega.segmentstore.contracts.StreamSegmentNotExistsException) OperationPriority(io.pravega.segmentstore.server.logs.operations.OperationPriority) Retry(io.pravega.common.util.Retry) ExtendedChunkInfo(io.pravega.segmentstore.contracts.ExtendedChunkInfo) Exceptions(io.pravega.common.Exceptions) ReadOnlyStorage(io.pravega.segmentstore.storage.ReadOnlyStorage) StorageFactory(io.pravega.segmentstore.storage.StorageFactory) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) CompletableFuture(java.util.concurrent.CompletableFuture) SegmentProperties(io.pravega.segmentstore.contracts.SegmentProperties) AttributeUpdate(io.pravega.segmentstore.contracts.AttributeUpdate) SegmentType(io.pravega.segmentstore.contracts.SegmentType) AbstractIdleService(com.google.common.util.concurrent.AbstractIdleService) BufferView(io.pravega.common.util.BufferView) Duration(java.time.Duration) Map(java.util.Map) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) Nullable(javax.annotation.Nullable) SegmentContainerExtension(io.pravega.segmentstore.server.SegmentContainerExtension) Services(io.pravega.common.concurrent.Services) TimeoutTimer(io.pravega.common.TimeoutTimer) StreamSegmentStorageReader(io.pravega.segmentstore.server.reading.StreamSegmentStorageReader) Executor(java.util.concurrent.Executor) Collection(java.util.Collection) AttributeId(io.pravega.segmentstore.contracts.AttributeId) MergeStreamSegmentResult(io.pravega.segmentstore.contracts.MergeStreamSegmentResult) DirectSegmentAccess(io.pravega.segmentstore.server.DirectSegmentAccess) List(java.util.List) Slf4j(lombok.extern.slf4j.Slf4j) AttributeUpdateCollection(io.pravega.segmentstore.contracts.AttributeUpdateCollection) SegmentContainer(io.pravega.segmentstore.server.SegmentContainer) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) Futures(io.pravega.common.concurrent.Futures) ReadResult(io.pravega.segmentstore.contracts.ReadResult) TimeoutTimer(io.pravega.common.TimeoutTimer)

Aggregations

TimeoutTimer (io.pravega.common.TimeoutTimer)97 lombok.val (lombok.val)55 CompletableFuture (java.util.concurrent.CompletableFuture)52 Duration (java.time.Duration)51 Futures (io.pravega.common.concurrent.Futures)47 Preconditions (com.google.common.base.Preconditions)41 CompletionException (java.util.concurrent.CompletionException)41 Slf4j (lombok.extern.slf4j.Slf4j)41 Collectors (java.util.stream.Collectors)40 SneakyThrows (lombok.SneakyThrows)40 Exceptions (io.pravega.common.Exceptions)39 BufferView (io.pravega.common.util.BufferView)37 Getter (lombok.Getter)37 SegmentProperties (io.pravega.segmentstore.contracts.SegmentProperties)36 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)36 StreamSegmentNotExistsException (io.pravega.segmentstore.contracts.StreamSegmentNotExistsException)34 SegmentMetadata (io.pravega.segmentstore.server.SegmentMetadata)34 RequiredArgsConstructor (lombok.RequiredArgsConstructor)34 ArrayList (java.util.ArrayList)33 List (java.util.List)31