Search in sources :

Example 46 with MockController

use of io.pravega.client.stream.mock.MockController in project pravega by pravega.

the class ReaderGroupStateManagerTest method testCheckpointContainsAllShards.

@Test(timeout = 10000)
public void testCheckpointContainsAllShards() throws ReinitializationRequiredException {
    String scope = "scope";
    String stream = "stream";
    PravegaNodeUri endpoint = new PravegaNodeUri("localhost", SERVICE_PORT);
    MockConnectionFactoryImpl connectionFactory = new MockConnectionFactoryImpl();
    Segment segment0 = new Segment(scope, stream, 0);
    Segment segment1 = new Segment(scope, stream, 1);
    Segment segment2 = new Segment(scope, stream, 2);
    MockController controller = new MockControllerWithSuccessors(endpoint.getEndpoint(), endpoint.getPort(), connectionFactory, new StreamSegmentsWithPredecessors(ImmutableMap.of(), ""));
    MockSegmentStreamFactory streamFactory = new MockSegmentStreamFactory();
    @Cleanup ClientFactory clientFactory = new ClientFactoryImpl(scope, controller, connectionFactory, streamFactory, streamFactory, streamFactory);
    SynchronizerConfig config = SynchronizerConfig.builder().build();
    @Cleanup StateSynchronizer<ReaderGroupState> stateSynchronizer = createState(stream, clientFactory, config);
    Map<Segment, Long> segments = ImmutableMap.of(segment0, 0L, segment1, 1L, segment2, 2L);
    ReaderGroupStateManager.initializeReaderGroup(stateSynchronizer, ReaderGroupConfig.builder().stream(Stream.of(scope, stream)).build(), segments);
    val readerState1 = new ReaderGroupStateManager("reader1", stateSynchronizer, controller, null);
    readerState1.initializeReader(0);
    val readerState2 = new ReaderGroupStateManager("reader2", stateSynchronizer, controller, null);
    readerState2.initializeReader(0);
    assertEquals(segments, stateSynchronizer.getState().getUnassignedSegments());
    stateSynchronizer.updateStateUnconditionally(new CreateCheckpoint("CP1"));
    stateSynchronizer.fetchUpdates();
    assertEquals("CP1", readerState1.getCheckpoint());
    assertEquals(Collections.emptyMap(), readerState1.acquireNewSegmentsIfNeeded(1));
    assertEquals(Collections.emptyMap(), readerState2.acquireNewSegmentsIfNeeded(2));
    assertEquals("CP1", readerState2.getCheckpoint());
    readerState1.checkpoint("CP1", new PositionImpl(Collections.emptyMap()));
    readerState2.checkpoint("CP1", new PositionImpl(Collections.emptyMap()));
    assertTrue(stateSynchronizer.getState().isCheckpointComplete("CP1"));
    assertEquals(segments, stateSynchronizer.getState().getPositionsForCompletedCheckpoint("CP1"));
}
Also used : lombok.val(lombok.val) MockSegmentStreamFactory(io.pravega.client.stream.mock.MockSegmentStreamFactory) ClientFactory(io.pravega.client.ClientFactory) Cleanup(lombok.Cleanup) Segment(io.pravega.client.segment.impl.Segment) PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) MockConnectionFactoryImpl(io.pravega.client.stream.mock.MockConnectionFactoryImpl) AtomicLong(java.util.concurrent.atomic.AtomicLong) CreateCheckpoint(io.pravega.client.stream.impl.ReaderGroupState.CreateCheckpoint) MockController(io.pravega.client.stream.mock.MockController) SynchronizerConfig(io.pravega.client.state.SynchronizerConfig) Test(org.junit.Test)

Example 47 with MockController

use of io.pravega.client.stream.mock.MockController in project pravega by pravega.

the class ReaderGroupStateManagerTest method testAddReader.

@Test(timeout = 10000)
public void testAddReader() throws ReinitializationRequiredException {
    String scope = "scope";
    String stream = "stream";
    PravegaNodeUri endpoint = new PravegaNodeUri("localhost", SERVICE_PORT);
    MockConnectionFactoryImpl connectionFactory = new MockConnectionFactoryImpl();
    MockController controller = new MockController(endpoint.getEndpoint(), endpoint.getPort(), connectionFactory);
    MockSegmentStreamFactory streamFactory = new MockSegmentStreamFactory();
    @Cleanup ClientFactory clientFactory = new ClientFactoryImpl(scope, controller, connectionFactory, streamFactory, streamFactory, streamFactory);
    SynchronizerConfig config = SynchronizerConfig.builder().build();
    @Cleanup StateSynchronizer<ReaderGroupState> stateSynchronizer = createState(stream, clientFactory, config);
    Map<Segment, Long> segments = new HashMap<>();
    segments.put(new Segment(scope, stream, 0), 1L);
    ReaderGroupStateManager.initializeReaderGroup(stateSynchronizer, ReaderGroupConfig.builder().stream(Stream.of(scope, stream)).build(), segments);
    ReaderGroupStateManager readerState = new ReaderGroupStateManager("testReader", stateSynchronizer, controller, null);
    readerState.initializeReader(0);
    Segment toRelease = readerState.findSegmentToReleaseIfRequired();
    assertNull(toRelease);
    Map<Segment, Long> newSegments = readerState.acquireNewSegmentsIfNeeded(0);
    assertFalse(newSegments.isEmpty());
    assertEquals(1, newSegments.size());
    assertTrue(newSegments.containsKey(new Segment(scope, stream, 0)));
    assertEquals(1, newSegments.get(new Segment(scope, stream, 0)).longValue());
}
Also used : HashMap(java.util.HashMap) MockSegmentStreamFactory(io.pravega.client.stream.mock.MockSegmentStreamFactory) ClientFactory(io.pravega.client.ClientFactory) Cleanup(lombok.Cleanup) Segment(io.pravega.client.segment.impl.Segment) PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) MockConnectionFactoryImpl(io.pravega.client.stream.mock.MockConnectionFactoryImpl) AtomicLong(java.util.concurrent.atomic.AtomicLong) MockController(io.pravega.client.stream.mock.MockController) SynchronizerConfig(io.pravega.client.state.SynchronizerConfig) Test(org.junit.Test)

Example 48 with MockController

use of io.pravega.client.stream.mock.MockController in project pravega by pravega.

the class ReaderGroupStateManagerTest method testCheckpoint.

@Test(timeout = 10000)
public void testCheckpoint() throws ReinitializationRequiredException {
    String scope = "scope";
    String stream = "stream";
    PravegaNodeUri endpoint = new PravegaNodeUri("localhost", SERVICE_PORT);
    MockConnectionFactoryImpl connectionFactory = new MockConnectionFactoryImpl();
    Segment initialSegment = new Segment(scope, stream, 0);
    Segment successorA = new Segment(scope, stream, 1);
    Segment successorB = new Segment(scope, stream, 2);
    MockController controller = new MockControllerWithSuccessors(endpoint.getEndpoint(), endpoint.getPort(), connectionFactory, new StreamSegmentsWithPredecessors(ImmutableMap.of(new SegmentWithRange(successorA, 0.0, 0.5), singletonList(0), new SegmentWithRange(successorB, 0.5, 1.0), singletonList(0)), ""));
    MockSegmentStreamFactory streamFactory = new MockSegmentStreamFactory();
    @Cleanup ClientFactory clientFactory = new ClientFactoryImpl(scope, controller, connectionFactory, streamFactory, streamFactory, streamFactory);
    SynchronizerConfig config = SynchronizerConfig.builder().build();
    @Cleanup StateSynchronizer<ReaderGroupState> stateSynchronizer = createState(stream, clientFactory, config);
    Map<Segment, Long> segments = new HashMap<>();
    segments.put(initialSegment, 1L);
    ReaderGroupStateManager.initializeReaderGroup(stateSynchronizer, ReaderGroupConfig.builder().stream(Stream.of(scope, stream)).build(), segments);
    val readerState = new ReaderGroupStateManager("testReader", stateSynchronizer, controller, null);
    readerState.initializeReader(0);
    assertNull(readerState.getCheckpoint());
    stateSynchronizer.updateStateUnconditionally(new CreateCheckpoint("CP1"));
    stateSynchronizer.fetchUpdates();
    assertEquals("CP1", readerState.getCheckpoint());
    assertEquals("CP1", readerState.getCheckpoint());
    readerState.checkpoint("CP1", new PositionImpl(Collections.emptyMap()));
    assertNull(readerState.getCheckpoint());
    stateSynchronizer.updateStateUnconditionally(new CreateCheckpoint("CP2"));
    stateSynchronizer.updateStateUnconditionally(new CreateCheckpoint("CP3"));
    stateSynchronizer.fetchUpdates();
    assertEquals("CP2", readerState.getCheckpoint());
    readerState.checkpoint("CP2", new PositionImpl(Collections.emptyMap()));
    assertEquals("CP3", readerState.getCheckpoint());
    readerState.checkpoint("CP3", new PositionImpl(Collections.emptyMap()));
    assertNull(readerState.getCheckpoint());
}
Also used : lombok.val(lombok.val) HashMap(java.util.HashMap) MockSegmentStreamFactory(io.pravega.client.stream.mock.MockSegmentStreamFactory) ClientFactory(io.pravega.client.ClientFactory) Cleanup(lombok.Cleanup) Segment(io.pravega.client.segment.impl.Segment) PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) MockConnectionFactoryImpl(io.pravega.client.stream.mock.MockConnectionFactoryImpl) AtomicLong(java.util.concurrent.atomic.AtomicLong) CreateCheckpoint(io.pravega.client.stream.impl.ReaderGroupState.CreateCheckpoint) MockController(io.pravega.client.stream.mock.MockController) SynchronizerConfig(io.pravega.client.state.SynchronizerConfig) Test(org.junit.Test)

Example 49 with MockController

use of io.pravega.client.stream.mock.MockController in project pravega by pravega.

the class ReaderGroupStateManagerTest method testRemoveReader.

@Test(timeout = 10000)
public void testRemoveReader() throws ReinitializationRequiredException {
    String scope = "scope";
    String stream = "stream";
    PravegaNodeUri endpoint = new PravegaNodeUri("localhost", SERVICE_PORT);
    MockConnectionFactoryImpl connectionFactory = new MockConnectionFactoryImpl();
    MockController controller = new MockController(endpoint.getEndpoint(), endpoint.getPort(), connectionFactory);
    MockSegmentStreamFactory streamFactory = new MockSegmentStreamFactory();
    @Cleanup ClientFactory clientFactory = new ClientFactoryImpl(scope, controller, connectionFactory, streamFactory, streamFactory, streamFactory);
    SynchronizerConfig config = SynchronizerConfig.builder().build();
    @Cleanup StateSynchronizer<ReaderGroupState> stateSynchronizer = createState(stream, clientFactory, config);
    AtomicLong clock = new AtomicLong();
    Map<Segment, Long> segments = new HashMap<>();
    segments.put(new Segment(scope, stream, 0), 123L);
    segments.put(new Segment(scope, stream, 1), 456L);
    ReaderGroupStateManager.initializeReaderGroup(stateSynchronizer, ReaderGroupConfig.builder().stream(Stream.of(scope, stream)).build(), segments);
    ReaderGroupStateManager readerState1 = new ReaderGroupStateManager("testReader", stateSynchronizer, controller, clock::get);
    readerState1.initializeReader(0);
    Segment toRelease = readerState1.findSegmentToReleaseIfRequired();
    assertNull(toRelease);
    Map<Segment, Long> newSegments = readerState1.acquireNewSegmentsIfNeeded(0);
    assertFalse(newSegments.isEmpty());
    assertEquals(2, newSegments.size());
    ReaderGroupStateManager readerState2 = new ReaderGroupStateManager("testReader2", stateSynchronizer, controller, clock::get);
    readerState2.initializeReader(0);
    clock.addAndGet(ReaderGroupStateManager.UPDATE_WINDOW.toNanos());
    assertNotNull(readerState1.findSegmentToReleaseIfRequired());
    boolean released = readerState1.releaseSegment(new Segment(scope, stream, 0), 789L, 0L);
    assertTrue(released);
    newSegments = readerState2.acquireNewSegmentsIfNeeded(0);
    assertEquals(1, newSegments.size());
    assertEquals(Long.valueOf(789L), newSegments.get(new Segment(scope, stream, 0)));
    ReaderGroupStateManager.readerShutdown("testReader2", null, stateSynchronizer);
    AssertExtensions.assertThrows(ReinitializationRequiredException.class, () -> readerState2.releaseSegment(new Segment(scope, stream, 0), 711L, 0L));
    clock.addAndGet(ReaderGroupStateManager.UPDATE_WINDOW.toNanos());
    newSegments = readerState1.acquireNewSegmentsIfNeeded(0);
    assertEquals(1, newSegments.size());
    assertEquals(Long.valueOf(789L), newSegments.get(new Segment(scope, stream, 0)));
    AssertExtensions.assertThrows(ReinitializationRequiredException.class, () -> readerState2.acquireNewSegmentsIfNeeded(0L));
}
Also used : HashMap(java.util.HashMap) MockSegmentStreamFactory(io.pravega.client.stream.mock.MockSegmentStreamFactory) ClientFactory(io.pravega.client.ClientFactory) Cleanup(lombok.Cleanup) Segment(io.pravega.client.segment.impl.Segment) AtomicLong(java.util.concurrent.atomic.AtomicLong) PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) MockConnectionFactoryImpl(io.pravega.client.stream.mock.MockConnectionFactoryImpl) AtomicLong(java.util.concurrent.atomic.AtomicLong) MockController(io.pravega.client.stream.mock.MockController) SynchronizerConfig(io.pravega.client.state.SynchronizerConfig) Test(org.junit.Test)

Example 50 with MockController

use of io.pravega.client.stream.mock.MockController in project pravega by pravega.

the class ReaderGroupStateManagerTest method testSegmentSplit.

@Test(timeout = 20000)
public void testSegmentSplit() throws ReinitializationRequiredException {
    String scope = "scope";
    String stream = "stream";
    PravegaNodeUri endpoint = new PravegaNodeUri("localhost", SERVICE_PORT);
    MockConnectionFactoryImpl connectionFactory = new MockConnectionFactoryImpl();
    Segment initialSegment = new Segment(scope, stream, 0);
    Segment successorA = new Segment(scope, stream, 1);
    Segment successorB = new Segment(scope, stream, 2);
    MockController controller = new MockControllerWithSuccessors(endpoint.getEndpoint(), endpoint.getPort(), connectionFactory, new StreamSegmentsWithPredecessors(ImmutableMap.of(new SegmentWithRange(successorA, 0.0, 0.5), singletonList(0), new SegmentWithRange(successorB, 0.5, 1.0), singletonList(0)), ""));
    MockSegmentStreamFactory streamFactory = new MockSegmentStreamFactory();
    @Cleanup ClientFactory clientFactory = new ClientFactoryImpl(scope, controller, connectionFactory, streamFactory, streamFactory, streamFactory);
    SynchronizerConfig config = SynchronizerConfig.builder().build();
    @Cleanup StateSynchronizer<ReaderGroupState> stateSynchronizer = createState(stream, clientFactory, config);
    Map<Segment, Long> segments = new HashMap<>();
    segments.put(initialSegment, 1L);
    ReaderGroupStateManager.initializeReaderGroup(stateSynchronizer, ReaderGroupConfig.builder().stream(Stream.of(scope, stream)).build(), segments);
    val readerState = new ReaderGroupStateManager("testReader", stateSynchronizer, controller, null);
    readerState.initializeReader(0);
    Map<Segment, Long> newSegments = readerState.acquireNewSegmentsIfNeeded(0);
    assertEquals(1, newSegments.size());
    assertEquals(Long.valueOf(1), newSegments.get(initialSegment));
    readerState.handleEndOfSegment(initialSegment);
    newSegments = readerState.acquireNewSegmentsIfNeeded(0);
    assertEquals(2, newSegments.size());
    assertEquals(Long.valueOf(0), newSegments.get(successorA));
    assertEquals(Long.valueOf(0), newSegments.get(successorB));
    newSegments = readerState.acquireNewSegmentsIfNeeded(0);
    assertTrue(newSegments.isEmpty());
}
Also used : lombok.val(lombok.val) HashMap(java.util.HashMap) MockSegmentStreamFactory(io.pravega.client.stream.mock.MockSegmentStreamFactory) ClientFactory(io.pravega.client.ClientFactory) Cleanup(lombok.Cleanup) Segment(io.pravega.client.segment.impl.Segment) PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) MockConnectionFactoryImpl(io.pravega.client.stream.mock.MockConnectionFactoryImpl) AtomicLong(java.util.concurrent.atomic.AtomicLong) MockController(io.pravega.client.stream.mock.MockController) SynchronizerConfig(io.pravega.client.state.SynchronizerConfig) Test(org.junit.Test)

Aggregations

MockController (io.pravega.client.stream.mock.MockController)50 Test (org.junit.Test)49 PravegaNodeUri (io.pravega.shared.protocol.netty.PravegaNodeUri)48 MockConnectionFactoryImpl (io.pravega.client.stream.mock.MockConnectionFactoryImpl)47 Cleanup (lombok.Cleanup)38 ClientConnection (io.pravega.client.netty.impl.ClientConnection)30 UUID (java.util.UUID)22 WireCommands (io.pravega.shared.protocol.netty.WireCommands)20 SetupAppend (io.pravega.shared.protocol.netty.WireCommands.SetupAppend)19 CompletableFuture (java.util.concurrent.CompletableFuture)17 PendingEvent (io.pravega.client.stream.impl.PendingEvent)16 AppendSetup (io.pravega.shared.protocol.netty.WireCommands.AppendSetup)16 ClientFactory (io.pravega.client.ClientFactory)15 Segment (io.pravega.client.segment.impl.Segment)15 SynchronizerConfig (io.pravega.client.state.SynchronizerConfig)15 MockSegmentStreamFactory (io.pravega.client.stream.mock.MockSegmentStreamFactory)15 InOrder (org.mockito.InOrder)14 InvocationOnMock (org.mockito.invocation.InvocationOnMock)13 Append (io.pravega.shared.protocol.netty.Append)12 ByteBuffer (java.nio.ByteBuffer)12