Search in sources :

Example 6 with VersionedMetadata

use of io.pravega.controller.store.VersionedMetadata in project pravega by pravega.

the class StreamMetadataTasks method checkScale.

/**
 * Helper method to check if scale operation against an epoch completed or not.
 *
 * @param scope          scope.
 * @param stream         stream name.
 * @param epoch          stream epoch.
 * @param requestId      request id.
 * @return returns the newly created segments.
 */
public CompletableFuture<ScaleStatusResponse> checkScale(String scope, String stream, int epoch, long requestId) {
    OperationContext context = streamMetadataStore.createStreamContext(scope, stream, requestId);
    CompletableFuture<EpochRecord> activeEpochFuture = streamMetadataStore.getActiveEpoch(scope, stream, context, true, executor);
    CompletableFuture<State> stateFuture = streamMetadataStore.getState(scope, stream, true, context, executor);
    CompletableFuture<EpochTransitionRecord> etrFuture = streamMetadataStore.getEpochTransition(scope, stream, context, executor).thenApply(VersionedMetadata::getObject);
    return CompletableFuture.allOf(stateFuture, activeEpochFuture, etrFuture).handle((r, ex) -> {
        ScaleStatusResponse.Builder response = ScaleStatusResponse.newBuilder();
        if (ex != null) {
            Throwable e = Exceptions.unwrap(ex);
            if (e instanceof StoreException.DataNotFoundException) {
                response.setStatus(ScaleStatusResponse.ScaleStatus.INVALID_INPUT);
            } else {
                response.setStatus(ScaleStatusResponse.ScaleStatus.INTERNAL_ERROR);
            }
        } else {
            EpochRecord activeEpoch = activeEpochFuture.join();
            State state = stateFuture.join();
            EpochTransitionRecord etr = etrFuture.join();
            if (epoch > activeEpoch.getEpoch()) {
                response.setStatus(ScaleStatusResponse.ScaleStatus.INVALID_INPUT);
            } else if (activeEpoch.getEpoch() == epoch || activeEpoch.getReferenceEpoch() == epoch) {
                response.setStatus(ScaleStatusResponse.ScaleStatus.IN_PROGRESS);
            } else {
                // has not completed.
                if (epoch + 1 == activeEpoch.getReferenceEpoch() && state.equals(State.SCALING) && (etr.equals(EpochTransitionRecord.EMPTY) || etr.getNewEpoch() == activeEpoch.getEpoch())) {
                    response.setStatus(ScaleStatusResponse.ScaleStatus.IN_PROGRESS);
                } else {
                    response.setStatus(ScaleStatusResponse.ScaleStatus.SUCCESS);
                }
            }
        }
        return response.build();
    });
}
Also used : OperationContext(io.pravega.controller.store.stream.OperationContext) EpochRecord(io.pravega.controller.store.stream.records.EpochRecord) EpochTransitionRecord(io.pravega.controller.store.stream.records.EpochTransitionRecord) ReaderGroupState(io.pravega.controller.store.stream.ReaderGroupState) State(io.pravega.controller.store.stream.State) ScaleStatusResponse(io.pravega.controller.stream.api.grpc.v1.Controller.ScaleStatusResponse) VersionedMetadata(io.pravega.controller.store.VersionedMetadata)

Example 7 with VersionedMetadata

use of io.pravega.controller.store.VersionedMetadata in project pravega by pravega.

the class StreamMetadataTasks method isUpdated.

@VisibleForTesting
CompletableFuture<Boolean> isUpdated(String scope, String stream, StreamConfiguration newConfig, OperationContext context) {
    CompletableFuture<State> stateFuture = streamMetadataStore.getState(scope, stream, true, context, executor);
    CompletableFuture<StreamConfigurationRecord> configPropertyFuture = streamMetadataStore.getConfigurationRecord(scope, stream, context, executor).thenApply(VersionedMetadata::getObject);
    return CompletableFuture.allOf(stateFuture, configPropertyFuture).thenApply(v -> {
        State state = stateFuture.join();
        StreamConfigurationRecord configProperty = configPropertyFuture.join();
        // if property is updating and doesn't match our request, it's a subsequent update
        if (configProperty.isUpdating()) {
            return !configProperty.getStreamConfiguration().equals(newConfig);
        } else {
            // if stream is sealed then update should not be allowed
            if (state.equals(State.SEALED)) {
                log.error("Cannot update a sealed stream {}/{}", scope, stream);
                throw new UnsupportedOperationException("Cannot update a sealed stream: " + NameUtils.getScopedStreamName(scope, stream));
            }
            // and state is not updating
            return !(configProperty.getStreamConfiguration().equals(newConfig) && state.equals(State.UPDATING));
        }
    });
}
Also used : ReaderGroupState(io.pravega.controller.store.stream.ReaderGroupState) State(io.pravega.controller.store.stream.State) StreamConfigurationRecord(io.pravega.controller.store.stream.records.StreamConfigurationRecord) VersionedMetadata(io.pravega.controller.store.VersionedMetadata) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 8 with VersionedMetadata

use of io.pravega.controller.store.VersionedMetadata in project pravega by pravega.

the class PersistentStreamBase method startRollingTxn.

@Override
public CompletableFuture<VersionedMetadata<CommittingTransactionsRecord>> startRollingTxn(int activeEpoch, VersionedMetadata<CommittingTransactionsRecord> existing, OperationContext context) {
    Preconditions.checkNotNull(context, "Operation context cannot be null");
    CommittingTransactionsRecord record = existing.getObject();
    if (record.isRollingTxnRecord()) {
        return CompletableFuture.completedFuture(existing);
    } else {
        CommittingTransactionsRecord update = record.createRollingTxnRecord(activeEpoch);
        return updateCommittingTxnRecord(new VersionedMetadata<>(update, existing.getVersion()), context).thenApply(version -> new VersionedMetadata<>(update, version));
    }
}
Also used : CommittingTransactionsRecord(io.pravega.controller.store.stream.records.CommittingTransactionsRecord) VersionedMetadata(io.pravega.controller.store.VersionedMetadata)

Example 9 with VersionedMetadata

use of io.pravega.controller.store.VersionedMetadata in project pravega by pravega.

the class PersistentStreamBase method deleteStreamCutBefore.

@Override
public CompletableFuture<Void> deleteStreamCutBefore(StreamCutReferenceRecord record, OperationContext context) {
    Preconditions.checkNotNull(context, "Operation context cannot be null");
    return getRetentionSetData(context).thenCompose(data -> {
        RetentionSet retention = data.getObject();
        RetentionSet update = RetentionSet.removeStreamCutBefore(retention, record);
        List<StreamCutReferenceRecord> toRemove = retention.retentionRecordsBefore(record);
        return Futures.allOf(toRemove.stream().map(x -> deleteStreamCutRecordData(x.getRecordingTime(), context)).collect(Collectors.toList())).thenCompose(x -> Futures.toVoid(updateRetentionSetData(new VersionedMetadata<>(update, data.getVersion()), context)));
    });
}
Also used : StreamSegmentRecord(io.pravega.controller.store.stream.records.StreamSegmentRecord) SneakyThrows(lombok.SneakyThrows) BiFunction(java.util.function.BiFunction) LoggerFactory(org.slf4j.LoggerFactory) StreamConfiguration(io.pravega.client.stream.StreamConfiguration) TagLogger(io.pravega.common.tracing.TagLogger) VersionedMetadata(io.pravega.controller.store.VersionedMetadata) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Map(java.util.Map) DataNotFoundException(io.pravega.controller.store.stream.StoreException.DataNotFoundException) EpochTransitionRecord(io.pravega.controller.store.stream.records.EpochTransitionRecord) ImmutableSet(com.google.common.collect.ImmutableSet) StreamCutReferenceRecord(io.pravega.controller.store.stream.records.StreamCutReferenceRecord) StreamTruncationRecord(io.pravega.controller.store.stream.records.StreamTruncationRecord) ImmutableMap(com.google.common.collect.ImmutableMap) Predicate(java.util.function.Predicate) Collection(java.util.Collection) Set(java.util.Set) CompletionException(java.util.concurrent.CompletionException) ActiveTxnRecord(io.pravega.controller.store.stream.records.ActiveTxnRecord) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) List(java.util.List) CompletionStage(java.util.concurrent.CompletionStage) Optional(java.util.Optional) HistoryTimeSeries(io.pravega.controller.store.stream.records.HistoryTimeSeries) Futures(io.pravega.common.concurrent.Futures) IntStream(java.util.stream.IntStream) CompletedTxnRecord(io.pravega.controller.store.stream.records.CompletedTxnRecord) CommittingTransactionsRecord(io.pravega.controller.store.stream.records.CommittingTransactionsRecord) NameUtils.computeSegmentId(io.pravega.shared.NameUtils.computeSegmentId) Exceptions(io.pravega.common.Exceptions) HistoryTimeSeriesRecord(io.pravega.controller.store.stream.records.HistoryTimeSeriesRecord) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) RetentionSet(io.pravega.controller.store.stream.records.RetentionSet) DATA_NOT_FOUND_PREDICATE(io.pravega.controller.store.stream.AbstractStreamMetadataStore.DATA_NOT_FOUND_PREDICATE) AtomicReference(java.util.concurrent.atomic.AtomicReference) Function(java.util.function.Function) NameUtils.getSegmentNumber(io.pravega.shared.NameUtils.getSegmentNumber) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) Lists(com.google.common.collect.Lists) ImmutableList(com.google.common.collect.ImmutableList) StateRecord(io.pravega.controller.store.stream.records.StateRecord) StreamSubscriber(io.pravega.controller.store.stream.records.StreamSubscriber) LinkedList(java.util.LinkedList) RecordHelper(io.pravega.controller.store.stream.records.RecordHelper) SimpleEntry(java.util.AbstractMap.SimpleEntry) LongSummaryStatistics(java.util.LongSummaryStatistics) CollectionHelpers(io.pravega.common.util.CollectionHelpers) SealedSegmentsMapShard(io.pravega.controller.store.stream.records.SealedSegmentsMapShard) NameUtils(io.pravega.shared.NameUtils) Executor(java.util.concurrent.Executor) WriterMark(io.pravega.controller.store.stream.records.WriterMark) lombok.val(lombok.val) StreamCutRecord(io.pravega.controller.store.stream.records.StreamCutRecord) AtomicLong(java.util.concurrent.atomic.AtomicLong) StreamConfigurationRecord(io.pravega.controller.store.stream.records.StreamConfigurationRecord) EpochRecord(io.pravega.controller.store.stream.records.EpochRecord) ConcurrentSkipListSet(java.util.concurrent.ConcurrentSkipListSet) Version(io.pravega.controller.store.Version) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) Comparator(java.util.Comparator) Collections(java.util.Collections) StreamCutReferenceRecord(io.pravega.controller.store.stream.records.StreamCutReferenceRecord) RetentionSet(io.pravega.controller.store.stream.records.RetentionSet)

Example 10 with VersionedMetadata

use of io.pravega.controller.store.VersionedMetadata in project pravega by pravega.

the class PersistentStreamBase method completeUpdateConfiguration.

/**
 * Update configuration at configurationPath.
 *
 * @return future of operation
 */
@Override
public CompletableFuture<Void> completeUpdateConfiguration(VersionedMetadata<StreamConfigurationRecord> existing, OperationContext context) {
    StreamConfigurationRecord current = existing.getObject();
    Preconditions.checkNotNull(current);
    if (current.isUpdating()) {
        StreamConfigurationRecord newProperty = StreamConfigurationRecord.complete(scope, name, current.getStreamConfiguration());
        log.debug(context.getRequestId(), "Completing update configuration for stream {}/{}", scope, name);
        return Futures.toVoid(setConfigurationData(new VersionedMetadata<>(newProperty, existing.getVersion()), context));
    } else {
        // idempotent
        return CompletableFuture.completedFuture(null);
    }
}
Also used : StreamConfigurationRecord(io.pravega.controller.store.stream.records.StreamConfigurationRecord) VersionedMetadata(io.pravega.controller.store.VersionedMetadata)

Aggregations

VersionedMetadata (io.pravega.controller.store.VersionedMetadata)45 Map (java.util.Map)32 Futures (io.pravega.common.concurrent.Futures)31 List (java.util.List)31 Collectors (java.util.stream.Collectors)31 ArrayList (java.util.ArrayList)30 StreamConfiguration (io.pravega.client.stream.StreamConfiguration)29 EpochTransitionRecord (io.pravega.controller.store.stream.records.EpochTransitionRecord)29 StreamConfigurationRecord (io.pravega.controller.store.stream.records.StreamConfigurationRecord)29 UUID (java.util.UUID)28 CompletableFuture (java.util.concurrent.CompletableFuture)28 EpochRecord (io.pravega.controller.store.stream.records.EpochRecord)27 Exceptions (io.pravega.common.Exceptions)26 NameUtils (io.pravega.shared.NameUtils)26 StreamTruncationRecord (io.pravega.controller.store.stream.records.StreamTruncationRecord)25 HashMap (java.util.HashMap)25 ImmutableMap (com.google.common.collect.ImmutableMap)24 NameUtils.computeSegmentId (io.pravega.shared.NameUtils.computeSegmentId)24 Collections (java.util.Collections)24 Optional (java.util.Optional)24