Search in sources :

Example 1 with ClearCheckpointsBefore

use of io.pravega.client.stream.impl.ReaderGroupState.ClearCheckpointsBefore in project pravega by pravega.

the class SerializationTest method testReaderGroupUpdates.

@Test
public void testReaderGroupUpdates() throws Exception {
    ReaderGroupUpdateSerializer serializer = new ReaderGroupUpdateSerializer();
    verify(serializer, new AddReader(createString()));
    verify(serializer, new RemoveReader(createString(), createSegmentToLongMap()));
    verify(serializer, new ReleaseSegment(createString(), createSegment(), r.nextLong()));
    verify(serializer, new AcquireSegment(createString(), createSegment()));
    verify(serializer, new UpdateDistanceToTail(createString(), r.nextLong(), createSegmentRangeMap()));
    verify(serializer, new SegmentCompleted(createString(), createSegmentWithRange(), createMap(this::createSegmentWithRange, this::createLongList)));
    verify(serializer, new CheckpointReader(createString(), createString(), createSegmentToLongMap()));
    verify(serializer, new CreateCheckpoint(createString()));
    verify(serializer, new ClearCheckpointsBefore(createString()));
    verify(serializer, new UpdatingConfig(r.nextBoolean()));
}
Also used : ReleaseSegment(io.pravega.client.stream.impl.ReaderGroupState.ReleaseSegment) ReaderGroupUpdateSerializer(io.pravega.client.stream.impl.ReaderGroupState.ReaderGroupUpdateSerializer) AcquireSegment(io.pravega.client.stream.impl.ReaderGroupState.AcquireSegment) UpdateDistanceToTail(io.pravega.client.stream.impl.ReaderGroupState.UpdateDistanceToTail) CreateCheckpoint(io.pravega.client.stream.impl.ReaderGroupState.CreateCheckpoint) CheckpointReader(io.pravega.client.stream.impl.ReaderGroupState.CheckpointReader) ClearCheckpointsBefore(io.pravega.client.stream.impl.ReaderGroupState.ClearCheckpointsBefore) RemoveReader(io.pravega.client.stream.impl.ReaderGroupState.RemoveReader) AddReader(io.pravega.client.stream.impl.ReaderGroupState.AddReader) UpdatingConfig(io.pravega.client.stream.impl.ReaderGroupState.UpdatingConfig) SegmentCompleted(io.pravega.client.stream.impl.ReaderGroupState.SegmentCompleted) Test(org.junit.Test)

Example 2 with ClearCheckpointsBefore

use of io.pravega.client.stream.impl.ReaderGroupState.ClearCheckpointsBefore in project pravega by pravega.

the class ReaderGroupImpl method completeCheckpoint.

@SneakyThrows(CheckpointFailedException.class)
private Checkpoint completeCheckpoint(String checkpointName) {
    ReaderGroupState state = synchronizer.getState();
    Map<Segment, Long> map = state.getPositionsForCompletedCheckpoint(checkpointName);
    synchronizer.updateStateUnconditionally(new ClearCheckpointsBefore(checkpointName));
    if (map == null) {
        throw new CheckpointFailedException("Checkpoint was cleared before results could be read.");
    }
    return new CheckpointImpl(checkpointName, map);
}
Also used : ClearCheckpointsBefore(io.pravega.client.stream.impl.ReaderGroupState.ClearCheckpointsBefore) Segment(io.pravega.client.segment.impl.Segment) SneakyThrows(lombok.SneakyThrows)

Example 3 with ClearCheckpointsBefore

use of io.pravega.client.stream.impl.ReaderGroupState.ClearCheckpointsBefore in project pravega by pravega.

the class ReaderGroupImpl method completeCheckpointAndFetchStreamCut.

@SneakyThrows(CheckpointFailedException.class)
private Map<Stream, StreamCut> completeCheckpointAndFetchStreamCut(String checkPointId) {
    ReaderGroupState state = synchronizer.getState();
    Optional<Map<Stream, StreamCut>> cuts = state.getStreamCutsForCompletedCheckpoint(checkPointId);
    synchronizer.updateStateUnconditionally(new ClearCheckpointsBefore(checkPointId));
    return cuts.orElseThrow(() -> new CheckpointFailedException("Internal CheckPoint was cleared before results could be read."));
}
Also used : ClearCheckpointsBefore(io.pravega.client.stream.impl.ReaderGroupState.ClearCheckpointsBefore) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) HashMap(java.util.HashMap) SneakyThrows(lombok.SneakyThrows)

Example 4 with ClearCheckpointsBefore

use of io.pravega.client.stream.impl.ReaderGroupState.ClearCheckpointsBefore in project pravega by pravega.

the class ReaderGroupStateManager method getCheckpoint.

/**
 * Returns name of the checkpoint to be processed by this reader
 * @return String - name of the first checkpoint pending processing for this Reader.
 */
String getCheckpoint() throws ReaderNotInReaderGroupException {
    fetchUpdatesIfNeeded();
    ReaderGroupState state = sync.getState();
    long automaticCpInterval = state.getConfig().getAutomaticCheckpointIntervalMillis();
    if (!state.isReaderOnline(readerId)) {
        throw new ReaderNotInReaderGroupException(readerId);
    }
    String checkpoint = state.getCheckpointForReader(readerId);
    if (checkpoint != null) {
        checkpointTimer.reset(Duration.ofMillis(automaticCpInterval));
        return checkpoint;
    }
    if (automaticCpInterval <= 0 || checkpointTimer.hasRemaining() || state.hasOngoingCheckpoint()) {
        return null;
    }
    sync.updateState((s, u) -> {
        if (!s.hasOngoingCheckpoint()) {
            CreateCheckpoint newCp = new CreateCheckpoint();
            u.add(newCp);
            u.add(new ClearCheckpointsBefore(newCp.getCheckpointId()));
            log.debug("Created new checkpoint: {} currentState is: {}", newCp, s);
        }
    });
    checkpointTimer.reset(Duration.ofMillis(automaticCpInterval));
    return state.getCheckpointForReader(readerId);
}
Also used : CreateCheckpoint(io.pravega.client.stream.impl.ReaderGroupState.CreateCheckpoint) ClearCheckpointsBefore(io.pravega.client.stream.impl.ReaderGroupState.ClearCheckpointsBefore) ReaderNotInReaderGroupException(io.pravega.client.stream.ReaderNotInReaderGroupException)

Example 5 with ClearCheckpointsBefore

use of io.pravega.client.stream.impl.ReaderGroupState.ClearCheckpointsBefore in project pravega by pravega.

the class ReaderGroupImplTest method testFutureCancelation.

@SuppressWarnings("unchecked")
@Test
public void testFutureCancelation() throws Exception {
    AtomicBoolean completed = new AtomicBoolean(false);
    when(synchronizer.updateState(any(StateSynchronizer.UpdateGeneratorFunction.class))).thenReturn(true);
    when(state.isCheckpointComplete("test")).thenReturn(false).thenReturn(true);
    Mockito.doAnswer(invocation -> {
        completed.set(true);
        return null;
    }).when(synchronizer).updateStateUnconditionally(eq(new ClearCheckpointsBefore("test")));
    @Cleanup("shutdown") InlineExecutor executor = new InlineExecutor();
    CompletableFuture<Checkpoint> result = readerGroup.initiateCheckpoint("test", executor);
    assertFalse(result.isDone());
    result.cancel(false);
    AssertExtensions.assertEventuallyEquals(true, completed::get, 5000);
}
Also used : AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) Checkpoint(io.pravega.client.stream.Checkpoint) InlineExecutor(io.pravega.test.common.InlineExecutor) ClearCheckpointsBefore(io.pravega.client.stream.impl.ReaderGroupState.ClearCheckpointsBefore) Cleanup(lombok.Cleanup) Test(org.junit.Test)

Aggregations

ClearCheckpointsBefore (io.pravega.client.stream.impl.ReaderGroupState.ClearCheckpointsBefore)5 CreateCheckpoint (io.pravega.client.stream.impl.ReaderGroupState.CreateCheckpoint)2 SneakyThrows (lombok.SneakyThrows)2 Test (org.junit.Test)2 ImmutableMap (com.google.common.collect.ImmutableMap)1 Segment (io.pravega.client.segment.impl.Segment)1 Checkpoint (io.pravega.client.stream.Checkpoint)1 ReaderNotInReaderGroupException (io.pravega.client.stream.ReaderNotInReaderGroupException)1 AcquireSegment (io.pravega.client.stream.impl.ReaderGroupState.AcquireSegment)1 AddReader (io.pravega.client.stream.impl.ReaderGroupState.AddReader)1 CheckpointReader (io.pravega.client.stream.impl.ReaderGroupState.CheckpointReader)1 ReaderGroupUpdateSerializer (io.pravega.client.stream.impl.ReaderGroupState.ReaderGroupUpdateSerializer)1 ReleaseSegment (io.pravega.client.stream.impl.ReaderGroupState.ReleaseSegment)1 RemoveReader (io.pravega.client.stream.impl.ReaderGroupState.RemoveReader)1 SegmentCompleted (io.pravega.client.stream.impl.ReaderGroupState.SegmentCompleted)1 UpdateDistanceToTail (io.pravega.client.stream.impl.ReaderGroupState.UpdateDistanceToTail)1 UpdatingConfig (io.pravega.client.stream.impl.ReaderGroupState.UpdatingConfig)1 InlineExecutor (io.pravega.test.common.InlineExecutor)1 HashMap (java.util.HashMap)1 Map (java.util.Map)1