Search in sources :

Example 36 with SneakyThrows

use of lombok.SneakyThrows in project pravega by pravega.

the class ReaderGroupImpl method completeCheckpoint.

@SneakyThrows(CheckpointFailedException.class)
private Checkpoint completeCheckpoint(String checkpointName, StateSynchronizer<ReaderGroupState> synchronizer) {
    ReaderGroupState state = synchronizer.getState();
    Map<Segment, Long> map = state.getPositionsForCompletedCheckpoint(checkpointName);
    synchronizer.updateStateUnconditionally(new ClearCheckpoints(checkpointName));
    if (map == null) {
        throw new CheckpointFailedException("Checkpoint was cleared before results could be read.");
    }
    return new CheckpointImpl(checkpointName, map);
}
Also used : ClearCheckpoints(io.pravega.client.stream.impl.ReaderGroupState.ClearCheckpoints) Segment(io.pravega.client.segment.impl.Segment) SneakyThrows(lombok.SneakyThrows)

Example 37 with SneakyThrows

use of lombok.SneakyThrows in project pravega by pravega.

the class DurableLogTests method processOperations.

@SneakyThrows
private List<OperationWithCompletion> processOperations(Collection<Operation> operations, DurableLog durableLog, int waitEvery) {
    List<OperationWithCompletion> completionFutures = new ArrayList<>();
    int index = 0;
    for (Operation o : operations) {
        index++;
        CompletableFuture<Void> completionFuture;
        try {
            completionFuture = durableLog.add(o, TIMEOUT);
        } catch (Exception ex) {
            completionFuture = Futures.failedFuture(ex);
        }
        completionFutures.add(new OperationWithCompletion(o, completionFuture));
        if (index % waitEvery == 0) {
            completionFuture.get(TIMEOUT.toMillis(), TimeUnit.MILLISECONDS);
        }
    }
    return completionFutures;
}
Also used : ArrayList(java.util.ArrayList) StorageMetadataCheckpointOperation(io.pravega.segmentstore.server.logs.operations.StorageMetadataCheckpointOperation) ProbeOperation(io.pravega.segmentstore.server.logs.operations.ProbeOperation) Operation(io.pravega.segmentstore.server.logs.operations.Operation) StreamSegmentMapOperation(io.pravega.segmentstore.server.logs.operations.StreamSegmentMapOperation) MetadataCheckpointOperation(io.pravega.segmentstore.server.logs.operations.MetadataCheckpointOperation) StorageOperation(io.pravega.segmentstore.server.logs.operations.StorageOperation) StreamSegmentAppendOperation(io.pravega.segmentstore.server.logs.operations.StreamSegmentAppendOperation) StreamSegmentNotExistsException(io.pravega.segmentstore.contracts.StreamSegmentNotExistsException) TimeoutException(java.util.concurrent.TimeoutException) DataLogNotAvailableException(io.pravega.segmentstore.storage.DataLogNotAvailableException) StreamSegmentSealedException(io.pravega.segmentstore.contracts.StreamSegmentSealedException) ContainerOfflineException(io.pravega.segmentstore.server.ContainerOfflineException) CompletionException(java.util.concurrent.CompletionException) DataLogWriterNotPrimaryException(io.pravega.segmentstore.storage.DataLogWriterNotPrimaryException) StreamSegmentException(io.pravega.segmentstore.contracts.StreamSegmentException) DurableDataLogException(io.pravega.segmentstore.storage.DurableDataLogException) DataLogDisabledException(io.pravega.segmentstore.storage.DataLogDisabledException) IntentionalException(io.pravega.test.common.IntentionalException) IOException(java.io.IOException) DataCorruptionException(io.pravega.segmentstore.server.DataCorruptionException) SneakyThrows(lombok.SneakyThrows)

Example 38 with SneakyThrows

use of lombok.SneakyThrows in project pravega by pravega.

the class PravegaRequestProcessor method copyData.

/**
 * Copy all of the contents provided into a byteBuffer and return it.
 */
@SneakyThrows(IOException.class)
private ByteBuffer copyData(List<ReadResultEntryContents> contents) {
    int totalSize = contents.stream().mapToInt(ReadResultEntryContents::getLength).sum();
    ByteBuffer data = ByteBuffer.allocate(totalSize);
    int bytesCopied = 0;
    for (ReadResultEntryContents content : contents) {
        int copied = StreamHelpers.readAll(content.getData(), data.array(), bytesCopied, totalSize - bytesCopied);
        Preconditions.checkState(copied == content.getLength(), "Read fewer bytes than available.");
        bytesCopied += copied;
    }
    return data;
}
Also used : ReadResultEntryContents(io.pravega.segmentstore.contracts.ReadResultEntryContents) ByteBuffer(java.nio.ByteBuffer) SneakyThrows(lombok.SneakyThrows)

Example 39 with SneakyThrows

use of lombok.SneakyThrows in project pravega by pravega.

the class DurableLog method performRecovery.

@SneakyThrows(Exception.class)
private boolean performRecovery() {
    // Make sure we are in the correct state. We do not want to do recovery while we are in full swing.
    Preconditions.checkState(state() == State.STARTING || (state() == State.RUNNING && isOffline()), "Invalid State for recovery.");
    this.operationProcessor.getMetrics().operationLogInit();
    Timer timer = new Timer();
    try {
        // Initialize the DurableDataLog, which will acquire its lock and ensure we are the only active users of it.
        this.durableDataLog.initialize(RECOVERY_TIMEOUT);
        // Initiate the recovery.
        RecoveryProcessor p = new RecoveryProcessor(this.metadata, this.durableDataLog, this.memoryStateUpdater);
        int recoveredItemCount = p.performRecovery();
        this.operationProcessor.getMetrics().operationsCompleted(recoveredItemCount, timer.getElapsed());
        // Verify that the Recovery Processor has left the metadata in a non-recovery mode.
        Preconditions.checkState(!this.metadata.isRecoveryMode(), "Recovery completed but Metadata is still in Recovery Mode.");
        return recoveredItemCount > 0;
    } catch (Exception ex) {
        log.error("{} Recovery FAILED.", this.traceObjectId, ex);
        if (Exceptions.unwrap(ex) instanceof DataCorruptionException) {
            // someone can manually fix the problem).
            try {
                this.durableDataLog.disable();
                log.info("{} Log disabled due to DataCorruptionException during recovery.", this.traceObjectId);
            } catch (Exception disableEx) {
                log.warn("{}: Unable to disable log after DataCorruptionException during recovery.", this.traceObjectId, disableEx);
                ex.addSuppressed(disableEx);
            }
        }
        throw ex;
    }
}
Also used : TimeoutTimer(io.pravega.common.TimeoutTimer) Timer(io.pravega.common.Timer) DataCorruptionException(io.pravega.segmentstore.server.DataCorruptionException) ObjectClosedException(io.pravega.common.ObjectClosedException) IllegalContainerStateException(io.pravega.segmentstore.server.IllegalContainerStateException) StreamingException(io.pravega.segmentstore.contracts.StreamingException) DataLogDisabledException(io.pravega.segmentstore.storage.DataLogDisabledException) ContainerOfflineException(io.pravega.segmentstore.server.ContainerOfflineException) CompletionException(java.util.concurrent.CompletionException) DataCorruptionException(io.pravega.segmentstore.server.DataCorruptionException) SneakyThrows(lombok.SneakyThrows)

Example 40 with SneakyThrows

use of lombok.SneakyThrows in project pravega by pravega.

the class ZKStreamMetadataStore method registerBucketOwnershipListener.

@Override
@SneakyThrows
public void registerBucketOwnershipListener(BucketOwnershipListener listener) {
    Preconditions.checkNotNull(listener);
    PathChildrenCacheListener bucketListener = (client, event) -> {
        switch(event.getType()) {
            case CHILD_ADDED:
                // no action required
                break;
            case CHILD_REMOVED:
                int bucketId = Integer.parseInt(ZKPaths.getNodeFromPath(event.getData().getPath()));
                listener.notify(new BucketNotification(bucketId, BucketNotification.NotificationType.BucketAvailable));
                break;
            case CONNECTION_LOST:
                listener.notify(new BucketNotification(Integer.MIN_VALUE, BucketNotification.NotificationType.ConnectivityError));
                break;
            default:
                log.warn("Received unknown event {}", event.getType());
        }
    };
    bucketOwnershipCacheRef.compareAndSet(null, new PathChildrenCache(storeHelper.getClient(), ZKStoreHelper.BUCKET_OWNERSHIP_PATH, true));
    bucketOwnershipCacheRef.get().getListenable().addListener(bucketListener);
    bucketOwnershipCacheRef.get().start(PathChildrenCache.StartMode.BUILD_INITIAL_CACHE);
    log.info("bucket ownership listener registered");
}
Also used : SneakyThrows(lombok.SneakyThrows) StreamImpl(io.pravega.client.stream.impl.StreamImpl) RetentionPolicy(io.pravega.client.stream.RetentionPolicy) SerializationUtils(org.apache.commons.lang3.SerializationUtils) CompletableFuture(java.util.concurrent.CompletableFuture) StreamNotification(io.pravega.controller.server.retention.BucketChangeListener.StreamNotification) PathChildrenCacheListener(org.apache.curator.framework.recipes.cache.PathChildrenCacheListener) AtomicReference(java.util.concurrent.atomic.AtomicReference) ConcurrentMap(java.util.concurrent.ConcurrentMap) BucketOwnershipListener(io.pravega.controller.server.retention.BucketOwnershipListener) ZKPaths(org.apache.curator.utils.ZKPaths) BucketNotification(io.pravega.controller.server.retention.BucketOwnershipListener.BucketNotification) Data(io.pravega.controller.store.stream.tables.Data) NotificationType(io.pravega.controller.server.retention.BucketChangeListener.StreamNotification.NotificationType) BucketChangeListener(io.pravega.controller.server.retention.BucketChangeListener) Executor(java.util.concurrent.Executor) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) IOException(java.io.IOException) CompletionException(java.util.concurrent.CompletionException) Collectors(java.util.stream.Collectors) Slf4j(lombok.extern.slf4j.Slf4j) Base64(java.util.Base64) List(java.util.List) CuratorFramework(org.apache.curator.framework.CuratorFramework) Config(io.pravega.controller.util.Config) PathChildrenCache(org.apache.curator.framework.recipes.cache.PathChildrenCache) Preconditions(com.google.common.base.Preconditions) ZKHostIndex(io.pravega.controller.store.index.ZKHostIndex) PathChildrenCacheListener(org.apache.curator.framework.recipes.cache.PathChildrenCacheListener) PathChildrenCache(org.apache.curator.framework.recipes.cache.PathChildrenCache) BucketNotification(io.pravega.controller.server.retention.BucketOwnershipListener.BucketNotification) SneakyThrows(lombok.SneakyThrows)

Aggregations

SneakyThrows (lombok.SneakyThrows)592 lombok.val (lombok.val)292 Test (org.junit.Test)66 ArrayList (java.util.ArrayList)59 HashMap (java.util.HashMap)51 List (java.util.List)42 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)33 LinkedHashMap (java.util.LinkedHashMap)29 File (java.io.File)27 Collectors (java.util.stream.Collectors)25 Path (java.nio.file.Path)24 IOException (java.io.IOException)23 SpringBootTest (org.springframework.boot.test.context.SpringBootTest)22 URL (java.net.URL)20 Slf4j (lombok.extern.slf4j.Slf4j)20 InputStream (java.io.InputStream)19 Map (java.util.Map)19 Cleanup (lombok.Cleanup)17 FishingActivityQuery (eu.europa.ec.fisheries.ers.service.search.FishingActivityQuery)16 SearchFilter (eu.europa.ec.fisheries.uvms.activity.model.schemas.SearchFilter)16