Search in sources :

Example 31 with AppendSetup

use of io.pravega.shared.protocol.netty.WireCommands.AppendSetup in project pravega by pravega.

the class AppendProcessorTest method testConditionalAppendFailure.

@Test
public void testConditionalAppendFailure() {
    String streamSegmentName = "scope/stream/testConditionalAppendFailure";
    UUID clientId = UUID.randomUUID();
    byte[] data = new byte[] { 1, 2, 3, 4, 6, 7, 8, 9 };
    StreamSegmentStore store = mock(StreamSegmentStore.class);
    ServerConnection connection = mock(ServerConnection.class);
    val mockedRecorder = Mockito.mock(SegmentStatsRecorder.class);
    ConnectionTracker tracker = mock(ConnectionTracker.class);
    @Cleanup AppendProcessor processor = AppendProcessor.defaultBuilder().store(store).connection(new TrackedConnection(connection, tracker)).statsRecorder(mockedRecorder).build();
    setupGetAttributes(streamSegmentName, clientId, store);
    val ac1 = interceptAppend(store, streamSegmentName, 0, updateEventNumber(clientId, 1), CompletableFuture.completedFuture((long) data.length));
    processor.setupAppend(new SetupAppend(1, clientId, streamSegmentName, ""));
    processor.append(new Append(streamSegmentName, clientId, 1, 1, Unpooled.wrappedBuffer(data), 0L, requestId));
    val ac2 = interceptAppend(store, streamSegmentName, 0, updateEventNumber(clientId, 2, 1, 1), Futures.failedFuture(new BadOffsetException(streamSegmentName, data.length, 0)));
    processor.append(new Append(streamSegmentName, clientId, 2, 1, Unpooled.wrappedBuffer(data), 0L, requestId));
    val ac3 = interceptAppend(store, streamSegmentName, 0, updateEventNumber(clientId, 3, 1, 1), Futures.failedFuture(new BadAttributeUpdateException(streamSegmentName, new AttributeUpdate(AttributeId.fromUUID(clientId), null, 0), true, "test")));
    processor.append(new Append(streamSegmentName, clientId, 3, 1, Unpooled.wrappedBuffer(data), 0L, requestId));
    verify(store).getAttributes(anyString(), eq(Collections.singleton(AttributeId.fromUUID(clientId))), eq(true), eq(AppendProcessor.TIMEOUT));
    verifyStoreAppend(ac1, data);
    verifyStoreAppend(ac2, data);
    verifyStoreAppend(ac3, data);
    verify(connection).send(new AppendSetup(1, streamSegmentName, clientId, 0));
    verify(tracker, times(3)).updateOutstandingBytes(connection, data.length, data.length);
    verify(connection).send(new DataAppended(requestId, clientId, 1, 0, data.length));
    verify(connection).send(new ConditionalCheckFailed(clientId, 2, requestId));
    verify(connection).send(new InvalidEventNumber(clientId, requestId, "test"));
    verify(connection).close();
    verify(tracker, times(3)).updateOutstandingBytes(connection, -data.length, 0);
    verifyNoMoreInteractions(connection);
    verifyNoMoreInteractions(store);
    verify(mockedRecorder).recordAppend(eq(streamSegmentName), eq(8L), eq(1), any());
}
Also used : lombok.val(lombok.val) AttributeUpdate(io.pravega.segmentstore.contracts.AttributeUpdate) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) Cleanup(lombok.Cleanup) AppendSetup(io.pravega.shared.protocol.netty.WireCommands.AppendSetup) ConditionalCheckFailed(io.pravega.shared.protocol.netty.WireCommands.ConditionalCheckFailed) StreamSegmentStore(io.pravega.segmentstore.contracts.StreamSegmentStore) Append(io.pravega.shared.protocol.netty.Append) SetupAppend(io.pravega.shared.protocol.netty.WireCommands.SetupAppend) BadAttributeUpdateException(io.pravega.segmentstore.contracts.BadAttributeUpdateException) InvalidEventNumber(io.pravega.shared.protocol.netty.WireCommands.InvalidEventNumber) DataAppended(io.pravega.shared.protocol.netty.WireCommands.DataAppended) SetupAppend(io.pravega.shared.protocol.netty.WireCommands.SetupAppend) BadOffsetException(io.pravega.segmentstore.contracts.BadOffsetException) UUID(java.util.UUID) Test(org.junit.Test)

Example 32 with AppendSetup

use of io.pravega.shared.protocol.netty.WireCommands.AppendSetup in project pravega by pravega.

the class AppendProcessorTest method testAppendFailChannelClose.

@Test(timeout = 5000)
public void testAppendFailChannelClose() throws Exception {
    String streamSegmentName = "scope/stream/testAppendSegment";
    UUID clientId = UUID.randomUUID();
    byte[] data = new byte[] { 1, 2, 3, 4, 6, 7, 8, 9 };
    // Setup mocks.
    StreamSegmentStore store = mock(StreamSegmentStore.class);
    setupGetAttributes(streamSegmentName, clientId, store);
    interceptAppend(store, streamSegmentName, updateEventNumber(clientId, data.length), Futures.failedFuture(new IntentionalException()));
    @Cleanup EmbeddedChannel channel = createChannel(store);
    // Send a setup append WireCommand.
    Reply reply = sendRequest(channel, new SetupAppend(requestId, clientId, streamSegmentName, ""));
    assertEquals(new AppendSetup(requestId, streamSegmentName, clientId, 0), reply);
    // Send an append which will cause a RuntimeException to be thrown by the store.
    reply = sendRequest(channel, new Append(streamSegmentName, clientId, data.length, 1, Unpooled.wrappedBuffer(data), null, requestId));
    assertNull("No WireCommand reply is expected", reply);
    // Verify that the channel is closed by the AppendProcessor.
    assertEventuallyEquals(false, channel::isOpen, 3000);
}
Also used : StreamSegmentStore(io.pravega.segmentstore.contracts.StreamSegmentStore) Append(io.pravega.shared.protocol.netty.Append) SetupAppend(io.pravega.shared.protocol.netty.WireCommands.SetupAppend) SetupAppend(io.pravega.shared.protocol.netty.WireCommands.SetupAppend) EmbeddedChannel(io.netty.channel.embedded.EmbeddedChannel) Reply(io.pravega.shared.protocol.netty.Reply) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) UUID(java.util.UUID) Cleanup(lombok.Cleanup) IntentionalException(io.pravega.test.common.IntentionalException) AppendSetup(io.pravega.shared.protocol.netty.WireCommands.AppendSetup) Test(org.junit.Test)

Example 33 with AppendSetup

use of io.pravega.shared.protocol.netty.WireCommands.AppendSetup in project pravega by pravega.

the class AppendProcessorTest method testBadAttributeException.

@Test(timeout = 5000)
public void testBadAttributeException() throws Exception {
    String streamSegmentName = "scope/stream/testAppendSegment";
    UUID clientId = UUID.randomUUID();
    byte[] data = new byte[] { 1, 2, 3, 4, 6, 7, 8, 9 };
    // Setup mocks.
    StreamSegmentStore store = mock(StreamSegmentStore.class);
    setupGetAttributes(streamSegmentName, clientId, store);
    val ex = new BadAttributeUpdateException(streamSegmentName, new AttributeUpdate(AttributeId.randomUUID(), AttributeUpdateType.ReplaceIfEquals, 100, 101), false, "error");
    interceptAppend(store, streamSegmentName, updateEventNumber(clientId, data.length), Futures.failedFuture(ex));
    @Cleanup EmbeddedChannel channel = createChannel(store);
    // Send a setup append WireCommand.
    Reply reply = sendRequest(channel, new SetupAppend(requestId, clientId, streamSegmentName, ""));
    assertEquals(new AppendSetup(requestId, streamSegmentName, clientId, 0), reply);
    // Send an append which will cause a RuntimeException to be thrown by the store.
    reply = sendRequest(channel, new Append(streamSegmentName, clientId, data.length, 1, Unpooled.wrappedBuffer(data), null, requestId));
    // validate InvalidEventNumber Wirecommand is sent before closing the Channel.
    assertNotNull("Invalid Event WireCommand is expected", reply);
    assertEquals(WireCommandType.INVALID_EVENT_NUMBER.getCode(), ((WireCommand) reply).getType().getCode());
    assertEquals(requestId, ((InvalidEventNumber) reply).getRequestId());
    // Verify that the channel is closed by the AppendProcessor.
    assertEventuallyEquals(false, channel::isOpen, 3000);
}
Also used : lombok.val(lombok.val) WireCommand(io.pravega.shared.protocol.netty.WireCommand) AttributeUpdate(io.pravega.segmentstore.contracts.AttributeUpdate) EmbeddedChannel(io.netty.channel.embedded.EmbeddedChannel) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) Cleanup(lombok.Cleanup) AppendSetup(io.pravega.shared.protocol.netty.WireCommands.AppendSetup) StreamSegmentStore(io.pravega.segmentstore.contracts.StreamSegmentStore) Append(io.pravega.shared.protocol.netty.Append) SetupAppend(io.pravega.shared.protocol.netty.WireCommands.SetupAppend) BadAttributeUpdateException(io.pravega.segmentstore.contracts.BadAttributeUpdateException) SetupAppend(io.pravega.shared.protocol.netty.WireCommands.SetupAppend) Reply(io.pravega.shared.protocol.netty.Reply) UUID(java.util.UUID) Test(org.junit.Test)

Example 34 with AppendSetup

use of io.pravega.shared.protocol.netty.WireCommands.AppendSetup in project pravega by pravega.

the class AppendProcessorTest method testConditionalAppendSuccess.

@Test
public void testConditionalAppendSuccess() {
    String streamSegmentName = "scope/stream/testConditionalAppendSuccess";
    UUID clientId = UUID.randomUUID();
    byte[] data = new byte[] { 1, 2, 3, 4, 6, 7, 8, 9 };
    StreamSegmentStore store = mock(StreamSegmentStore.class);
    ServerConnection connection = mock(ServerConnection.class);
    val mockedRecorder = Mockito.mock(SegmentStatsRecorder.class);
    ConnectionTracker tracker = mock(ConnectionTracker.class);
    @Cleanup AppendProcessor processor = AppendProcessor.defaultBuilder().store(store).connection(new TrackedConnection(connection, tracker)).statsRecorder(mockedRecorder).build();
    setupGetAttributes(streamSegmentName, clientId, store);
    val ac1 = interceptAppend(store, streamSegmentName, updateEventNumber(clientId, 1), CompletableFuture.completedFuture((long) data.length));
    processor.setupAppend(new SetupAppend(1, clientId, streamSegmentName, ""));
    processor.append(new Append(streamSegmentName, clientId, 1, 1, Unpooled.wrappedBuffer(data), null, requestId));
    val ac2 = interceptAppend(store, streamSegmentName, data.length, updateEventNumber(clientId, 2, 1, 1), CompletableFuture.completedFuture(Long.valueOf(2 * data.length)));
    processor.append(new Append(streamSegmentName, clientId, 2, 1, Unpooled.wrappedBuffer(data), (long) data.length, requestId));
    verify(store).getAttributes(anyString(), eq(Collections.singleton(AttributeId.fromUUID(clientId))), eq(true), eq(AppendProcessor.TIMEOUT));
    verifyStoreAppend(ac1, data);
    verifyStoreAppend(ac2, data);
    verify(connection).send(new AppendSetup(1, streamSegmentName, clientId, 0));
    verify(tracker, times(2)).updateOutstandingBytes(connection, data.length, data.length);
    verify(connection).send(new DataAppended(requestId, clientId, 1, 0, data.length));
    verify(connection).send(new DataAppended(requestId, clientId, 2, 1, 2 * data.length));
    verify(tracker, times(2)).updateOutstandingBytes(connection, -data.length, 0);
    verifyNoMoreInteractions(connection);
    verifyNoMoreInteractions(store);
    verify(mockedRecorder, times(2)).recordAppend(eq(streamSegmentName), eq(8L), eq(1), any());
}
Also used : lombok.val(lombok.val) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) Cleanup(lombok.Cleanup) AppendSetup(io.pravega.shared.protocol.netty.WireCommands.AppendSetup) StreamSegmentStore(io.pravega.segmentstore.contracts.StreamSegmentStore) Append(io.pravega.shared.protocol.netty.Append) SetupAppend(io.pravega.shared.protocol.netty.WireCommands.SetupAppend) DataAppended(io.pravega.shared.protocol.netty.WireCommands.DataAppended) SetupAppend(io.pravega.shared.protocol.netty.WireCommands.SetupAppend) UUID(java.util.UUID) Test(org.junit.Test)

Example 35 with AppendSetup

use of io.pravega.shared.protocol.netty.WireCommands.AppendSetup in project pravega by pravega.

the class AppendProcessorTest method testAppendPipelining.

/**
 * Verifies that appends are "pipelined" into the underlying store and that acks are sent as appropriate via the
 * connection when they complete.
 */
@Test(timeout = 15 * 1000)
public void testAppendPipelining() {
    String streamSegmentName = "scope/stream/testAppendSegment";
    UUID clientId = UUID.randomUUID();
    byte[] data1 = new byte[] { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };
    byte[] data2 = new byte[] { 1, 2, 3, 4, 5 };
    StreamSegmentStore store = mock(StreamSegmentStore.class);
    ServerConnection connection = mock(ServerConnection.class);
    ConnectionTracker tracker = mock(ConnectionTracker.class);
    @Cleanup AppendProcessor processor = AppendProcessor.defaultBuilder().store(store).connection(new TrackedConnection(connection, tracker)).build();
    setupGetAttributes(streamSegmentName, clientId, store);
    processor.setupAppend(new SetupAppend(1, clientId, streamSegmentName, ""));
    verify(store).getAttributes(anyString(), eq(Collections.singleton(AttributeId.fromUUID(clientId))), eq(true), eq(AppendProcessor.TIMEOUT));
    verify(connection).send(new AppendSetup(1, streamSegmentName, clientId, 0));
    // Initiate two appends in short sequence, and simulate the Store blocking on both of them.
    val store1 = new CompletableFuture<Long>();
    val ac1 = interceptAppend(store, streamSegmentName, updateEventNumber(clientId, 1, 0, 1), store1);
    processor.append(new Append(streamSegmentName, clientId, 1, 1, Unpooled.wrappedBuffer(data1), null, requestId));
    val store2 = new CompletableFuture<Long>();
    val ac2 = interceptAppend(store, streamSegmentName, updateEventNumber(clientId, 2, 1, 1), store2);
    processor.append(new Append(streamSegmentName, clientId, 2, 1, Unpooled.wrappedBuffer(data2), null, requestId));
    verifyStoreAppend(ac1, data1);
    verifyStoreAppend(ac2, data2);
    verify(tracker).updateOutstandingBytes(connection, data1.length, data1.length);
    verify(tracker).updateOutstandingBytes(connection, data2.length, data1.length + data2.length);
    // Complete the second one (this simulates acks arriving out of order from the store).
    store2.complete(100L);
    // Verify an ack is sent for both appends (because of pipelining guarantees), but only the second append's length
    // is subtracted from the outstanding bytes.
    verify(connection).send(new DataAppended(requestId, clientId, 2, 0L, 100L));
    verify(tracker).updateOutstandingBytes(connection, -data2.length, data1.length);
    verifyNoMoreInteractions(connection);
    // Complete the first one, and verify that no additional acks are sent via the connection, but the first append's
    // length is subtracted from the outstanding bytes.
    store1.complete(50L);
    verify(tracker).updateOutstandingBytes(connection, -data1.length, 0);
    verifyNoMoreInteractions(connection);
    verifyNoMoreInteractions(store);
}
Also used : lombok.val(lombok.val) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) Cleanup(lombok.Cleanup) AppendSetup(io.pravega.shared.protocol.netty.WireCommands.AppendSetup) StreamSegmentStore(io.pravega.segmentstore.contracts.StreamSegmentStore) CompletableFuture(java.util.concurrent.CompletableFuture) Append(io.pravega.shared.protocol.netty.Append) SetupAppend(io.pravega.shared.protocol.netty.WireCommands.SetupAppend) DataAppended(io.pravega.shared.protocol.netty.WireCommands.DataAppended) SetupAppend(io.pravega.shared.protocol.netty.WireCommands.SetupAppend) UUID(java.util.UUID) Test(org.junit.Test)

Aggregations

AppendSetup (io.pravega.shared.protocol.netty.WireCommands.AppendSetup)54 Test (org.junit.Test)51 SetupAppend (io.pravega.shared.protocol.netty.WireCommands.SetupAppend)49 UUID (java.util.UUID)46 Append (io.pravega.shared.protocol.netty.Append)40 MockConnectionFactoryImpl (io.pravega.client.stream.mock.MockConnectionFactoryImpl)33 MockController (io.pravega.client.stream.mock.MockController)33 PravegaNodeUri (io.pravega.shared.protocol.netty.PravegaNodeUri)33 ClientConnection (io.pravega.client.connection.impl.ClientConnection)32 CompletableFuture (java.util.concurrent.CompletableFuture)26 Cleanup (lombok.Cleanup)26 ByteBuffer (java.nio.ByteBuffer)24 InOrder (org.mockito.InOrder)22 WireCommands (io.pravega.shared.protocol.netty.WireCommands)20 DataAppended (io.pravega.shared.protocol.netty.WireCommands.DataAppended)20 StreamSegmentStore (io.pravega.segmentstore.contracts.StreamSegmentStore)19 lombok.val (lombok.val)18 ArgumentMatchers.anyString (org.mockito.ArgumentMatchers.anyString)15 InvocationOnMock (org.mockito.invocation.InvocationOnMock)13 SegmentCreated (io.pravega.shared.protocol.netty.WireCommands.SegmentCreated)12