Search in sources :

Example 66 with InternalEvent

use of com.nextdoor.bender.InternalEvent in project bender by Nextdoor.

the class S3TransporterTest method testMultipleUploads.

@Test
public void testMultipleUploads() throws TransportException, IllegalStateException, IOException {
    /*
     * Create mock client, requets, and replies
     */
    AmazonS3Client mockClient = getMockClient();
    /*
     * Fill buffer with mock data
     */
    S3TransportBuffer buffer1 = new S3TransportBuffer(1000, false, new S3TransportSerializer());
    S3TransportBuffer buffer2 = new S3TransportBuffer(1000, false, new S3TransportSerializer());
    InternalEvent mockIevent = mock(InternalEvent.class);
    doReturn("foo").doReturn("bar1").when(mockIevent).getSerialized();
    buffer1.add(mockIevent);
    buffer2.add(mockIevent);
    /*
     * Create transport
     */
    Map<String, MultiPartUpload> multiPartUploads = new HashMap<String, MultiPartUpload>(0);
    S3Transport transport = new S3Transport(mockClient, "bucket", "basepath", false, multiPartUploads);
    /*
     * Do actual test
     */
    LinkedHashMap<String, String> partitions = new LinkedHashMap<String, String>();
    partitions.put(S3Transport.FILENAME_KEY, "a_filename");
    ArgumentCaptor<UploadPartRequest> argument = ArgumentCaptor.forClass(UploadPartRequest.class);
    transport.sendBatch(buffer1, partitions, new TestContext());
    transport.sendBatch(buffer2, partitions, new TestContext());
    verify(mockClient, times(2)).uploadPart(argument.capture());
    List<UploadPartRequest> arguments = argument.getAllValues();
    assertEquals(1, arguments.get(0).getPartNumber());
    // foo\n
    assertEquals(4, arguments.get(0).getPartSize());
    assertEquals("123", arguments.get(0).getUploadId());
    assertEquals(2, arguments.get(1).getPartNumber());
    // bar1\n
    assertEquals(5, arguments.get(1).getPartSize());
    assertEquals("123", arguments.get(1).getUploadId());
}
Also used : HashMap(java.util.HashMap) LinkedHashMap(java.util.LinkedHashMap) TestContext(com.nextdoor.bender.aws.TestContext) UploadPartRequest(com.amazonaws.services.s3.model.UploadPartRequest) InternalEvent(com.nextdoor.bender.InternalEvent) LinkedHashMap(java.util.LinkedHashMap) AmazonS3Client(com.amazonaws.services.s3.AmazonS3Client) Test(org.junit.Test)

Example 67 with InternalEvent

use of com.nextdoor.bender.InternalEvent in project bender by Nextdoor.

the class GenericTransportBufferTest method testGzip.

@Test
public void testGzip() throws IOException {
    GenericTransportSerializer serializer = mock(GenericTransportSerializer.class);
    doReturn("foo".getBytes()).when(serializer).serialize(any(InternalEvent.class));
    GenericTransportBuffer buffer = new GenericTransportBuffer(1, true, serializer);
    InternalEvent mockEvent = mock(InternalEvent.class);
    buffer.add(mockEvent);
    buffer.close();
    byte[] actual = buffer.getInternalBuffer().toByteArray();
    byte[] expected = { 31, -117, 8, 0, 0, 0, 0, 0, 0, 0, 75, -53, -49, 7, 0, 33, 101, 115, -116, 3, 0, 0, 0 };
    assertArrayEquals(expected, actual);
}
Also used : GenericTransportSerializer(com.nextdoor.bender.ipc.generic.GenericTransportSerializer) GenericTransportBuffer(com.nextdoor.bender.ipc.generic.GenericTransportBuffer) InternalEvent(com.nextdoor.bender.InternalEvent) Test(org.junit.Test)

Example 68 with InternalEvent

use of com.nextdoor.bender.InternalEvent in project bender by Nextdoor.

the class GenericTransportBufferTest method testAdd.

@Test
public void testAdd() throws IOException {
    GenericTransportSerializer serializer = mock(GenericTransportSerializer.class);
    doReturn("foo".getBytes()).when(serializer).serialize(any(InternalEvent.class));
    GenericTransportBuffer buffer = new GenericTransportBuffer(1, false, serializer);
    InternalEvent mockEvent = mock(InternalEvent.class);
    buffer.add(mockEvent);
    ByteArrayOutputStream baos = buffer.getInternalBuffer();
    baos.close();
    String actual = new String(baos.toByteArray());
    assertEquals("foo", actual);
    assertEquals(false, buffer.isEmpty());
}
Also used : GenericTransportSerializer(com.nextdoor.bender.ipc.generic.GenericTransportSerializer) ByteArrayOutputStream(java.io.ByteArrayOutputStream) GenericTransportBuffer(com.nextdoor.bender.ipc.generic.GenericTransportBuffer) InternalEvent(com.nextdoor.bender.InternalEvent) Test(org.junit.Test)

Aggregations

InternalEvent (com.nextdoor.bender.InternalEvent)68 Test (org.junit.Test)67 LinkedHashMap (java.util.LinkedHashMap)17 HashMap (java.util.HashMap)13 ArrayList (java.util.ArrayList)12 TestContext (com.nextdoor.bender.aws.TestContext)10 AmazonS3Client (com.amazonaws.services.s3.AmazonS3Client)9 UploadPartRequest (com.amazonaws.services.s3.model.UploadPartRequest)9 DummyDeserializedEvent (com.nextdoor.bender.testutils.DummyDeserializerHelper.DummyDeserializedEvent)9 JsonElement (com.google.gson.JsonElement)7 JsonParser (com.google.gson.JsonParser)7 OperationTest (com.nextdoor.bender.operations.json.OperationTest)7 DummyOperationFactory (com.nextdoor.bender.testutils.DummyOperationHelper.DummyOperationFactory)6 GenericTransportBuffer (com.nextdoor.bender.ipc.generic.GenericTransportBuffer)5 GenericTransportSerializer (com.nextdoor.bender.ipc.generic.GenericTransportSerializer)5 DummyOperation (com.nextdoor.bender.testutils.DummyOperationHelper.DummyOperation)4 UploadPartResult (com.amazonaws.services.s3.model.UploadPartResult)3 Stat (com.nextdoor.bender.monitoring.Stat)3 KeyNameOperation (com.nextdoor.bender.operation.json.key.KeyNameOperation)3 ByteArrayOutputStream (org.apache.commons.io.output.ByteArrayOutputStream)3