Search in sources :

Example 21 with KafkaMessageId

use of nl.minvenj.nfi.storm.kafka.util.KafkaMessageId in project kafka-spout by HolmesNL.

the class KafkaSpoutBufferBehaviourTest method testEmitOneAtATime.

@Test
public void testEmitOneAtATime() {
    final KafkaMessageId id1 = new KafkaMessageId(1, 1234);
    final KafkaMessageId id2 = new KafkaMessageId(1, 1235);
    final byte[] message1 = { 5, 6, 7, 8 };
    final byte[] message2 = { 9, 0, 1, 2 };
    _subject._queue.add(id1);
    _subject._queue.add(id2);
    _subject._inProgress.put(id1, message1);
    _subject._inProgress.put(id2, message2);
    _subject.nextTuple();
    final ArgumentCaptor<List> captor = forClass(List.class);
    // subject should have emitted only the first message
    verify(_subject._collector).emit(captor.capture(), eq(id1));
    assertArrayEquals(message1, (byte[]) captor.getValue().get(0));
    verifyNoMoreInteractions(_subject._collector);
}
Also used : KafkaMessageId(nl.minvenj.nfi.storm.kafka.util.KafkaMessageId) List(java.util.List) Test(org.junit.Test)

Example 22 with KafkaMessageId

use of nl.minvenj.nfi.storm.kafka.util.KafkaMessageId in project kafka-spout by HolmesNL.

the class KafkaSpoutBufferBehaviourTest method testEmitOnAvailable.

@Test
public void testEmitOnAvailable() {
    final KafkaMessageId id = new KafkaMessageId(1, 1234);
    final byte[] message = { 5, 6, 7, 8 };
    _subject._queue.add(id);
    _subject._inProgress.put(id, message);
    // request to emit message and id
    _subject.nextTuple();
    final ArgumentCaptor<ByteBuffer> bufferCaptor = forClass(ByteBuffer.class);
    final ArgumentCaptor<List> valuesCaptor = forClass(List.class);
    // subject should have emitted a Values object identified by id
    verify(_subject._serializationScheme).deserialize(bufferCaptor.capture());
    assertArrayEquals(message, bufferCaptor.getValue().array());
    verify(_subject._collector).emit(valuesCaptor.capture(), eq(id));
    assertArrayEquals(message, (byte[]) valuesCaptor.getValue().get(0));
}
Also used : KafkaMessageId(nl.minvenj.nfi.storm.kafka.util.KafkaMessageId) List(java.util.List) ByteBuffer(java.nio.ByteBuffer) Test(org.junit.Test)

Aggregations

KafkaMessageId (nl.minvenj.nfi.storm.kafka.util.KafkaMessageId)22 Test (org.junit.Test)18 Map (java.util.Map)4 List (java.util.List)3 FailHandler (nl.minvenj.nfi.storm.kafka.fail.FailHandler)3 ReliableFailHandler (nl.minvenj.nfi.storm.kafka.fail.ReliableFailHandler)3 UnreliableFailHandler (nl.minvenj.nfi.storm.kafka.fail.UnreliableFailHandler)3 SpoutOutputCollector (org.apache.storm.spout.SpoutOutputCollector)3 TopologyContext (org.apache.storm.task.TopologyContext)3 ByteBuffer (java.nio.ByteBuffer)1 HashMap (java.util.HashMap)1 LinkedList (java.util.LinkedList)1 SortedMap (java.util.SortedMap)1 ConsumerTimeoutException (kafka.consumer.ConsumerTimeoutException)1 ConsumerConnector (kafka.javaapi.consumer.ConsumerConnector)1 InvalidMessageException (kafka.message.InvalidMessageException)1 ConfigUtils.createFailHandlerFromString (nl.minvenj.nfi.storm.kafka.util.ConfigUtils.createFailHandlerFromString)1 Values (org.apache.storm.tuple.Values)1