Search in sources :

Example 1 with KafkaMessageId

use of nl.minvenj.nfi.storm.kafka.util.KafkaMessageId in project kafka-spout by HolmesNL.

the class UnreliableFailHandlerTest method testSideEffects.

@Test
public void testSideEffects() {
    final FailHandler subject = new UnreliableFailHandler();
    final KafkaMessageId id = new KafkaMessageId(1, 1234);
    // convenience methods should have no effect
    subject.open(mock(Map.class), mock(TopologyContext.class), mock(SpoutOutputCollector.class));
    subject.activate();
    subject.deactivate();
    subject.close();
    // ack should be ignored
    subject.ack(id);
}
Also used : KafkaMessageId(nl.minvenj.nfi.storm.kafka.util.KafkaMessageId) SpoutOutputCollector(org.apache.storm.spout.SpoutOutputCollector) TopologyContext(org.apache.storm.task.TopologyContext) Map(java.util.Map) Test(org.junit.Test)

Example 2 with KafkaMessageId

use of nl.minvenj.nfi.storm.kafka.util.KafkaMessageId in project kafka-spout by HolmesNL.

the class KafkaSpoutBufferBehaviourTest method testRefuseRefillOnNonEmptyBuffer.

@Test(expected = IllegalStateException.class)
public void testRefuseRefillOnNonEmptyBuffer() {
    _subject._queue.add(new KafkaMessageId(1, 1234));
    _subject.fillBuffer();
}
Also used : KafkaMessageId(nl.minvenj.nfi.storm.kafka.util.KafkaMessageId) Test(org.junit.Test)

Example 3 with KafkaMessageId

use of nl.minvenj.nfi.storm.kafka.util.KafkaMessageId in project kafka-spout by HolmesNL.

the class KafkaSpoutBufferBehaviourTest method testIllegalQueueState.

@Test
public void testIllegalQueueState() {
    // queue a single id with no corresponding message
    final KafkaMessageId id = new KafkaMessageId(1, 1234);
    _subject._queue.add(id);
    try {
        _subject.nextTuple();
        fail("illegal queue state didn't trigger error");
    } catch (final IllegalStateException e) {
        assertThat(e.getMessage(), containsString(id.toString()));
    }
}
Also used : KafkaMessageId(nl.minvenj.nfi.storm.kafka.util.KafkaMessageId) Test(org.junit.Test)

Example 4 with KafkaMessageId

use of nl.minvenj.nfi.storm.kafka.util.KafkaMessageId in project kafka-spout by HolmesNL.

the class KafkaSpoutBufferBehaviourTest method testRefuseRefillOnNonEmptyPending.

@Test(expected = IllegalStateException.class)
public void testRefuseRefillOnNonEmptyPending() {
    _subject._inProgress.put(new KafkaMessageId(1, 1234), new byte[0]);
    _subject.fillBuffer();
}
Also used : KafkaMessageId(nl.minvenj.nfi.storm.kafka.util.KafkaMessageId) Test(org.junit.Test)

Example 5 with KafkaMessageId

use of nl.minvenj.nfi.storm.kafka.util.KafkaMessageId in project kafka-spout by HolmesNL.

the class KafkaSpoutFailurePolicyTest method testDelegatedCalls.

@Test
public void testDelegatedCalls() {
    final FailHandler unreliable = spy(new UnreliableFailHandler());
    _subject = new KafkaSpout() {

        @Override
        protected void createConsumer(final Map<String, Object> config) {
        // do nothing (would connect to zookeeper otherwise)
        }

        @Override
        protected void createFailHandler(final String failHandler) {
            _failHandler = unreliable;
        }
    };
    _subject._failHandler = unreliable;
    _subject._consumer = mock(ConsumerConnector.class);
    // call all the methods that should trigger policy notification
    _subject.open(mock(Map.class), mock(TopologyContext.class), mock(SpoutOutputCollector.class));
    verify(unreliable).open(any(Map.class), any(TopologyContext.class), any(SpoutOutputCollector.class));
    _subject.activate();
    verify(unreliable).activate();
    _subject.deactivate();
    verify(unreliable).deactivate();
    _subject.close();
    verify(unreliable).close();
    // NB: _subject will have set consumer to null, mock a new one
    _subject._consumer = mock(ConsumerConnector.class);
    // simulate an acknowledged message
    final KafkaMessageId id = new KafkaMessageId(1, 1234);
    final byte[] message = { 1, 2, 3, 4 };
    _subject._inProgress.put(id, message);
    _subject.ack(id);
    verify(unreliable).ack(id);
// failure is tested above
}
Also used : ConsumerConnector(kafka.javaapi.consumer.ConsumerConnector) KafkaMessageId(nl.minvenj.nfi.storm.kafka.util.KafkaMessageId) SpoutOutputCollector(org.apache.storm.spout.SpoutOutputCollector) TopologyContext(org.apache.storm.task.TopologyContext) FailHandler(nl.minvenj.nfi.storm.kafka.fail.FailHandler) ReliableFailHandler(nl.minvenj.nfi.storm.kafka.fail.ReliableFailHandler) UnreliableFailHandler(nl.minvenj.nfi.storm.kafka.fail.UnreliableFailHandler) Map(java.util.Map) UnreliableFailHandler(nl.minvenj.nfi.storm.kafka.fail.UnreliableFailHandler) Test(org.junit.Test)

Aggregations

KafkaMessageId (nl.minvenj.nfi.storm.kafka.util.KafkaMessageId)22 Test (org.junit.Test)18 Map (java.util.Map)4 List (java.util.List)3 FailHandler (nl.minvenj.nfi.storm.kafka.fail.FailHandler)3 ReliableFailHandler (nl.minvenj.nfi.storm.kafka.fail.ReliableFailHandler)3 UnreliableFailHandler (nl.minvenj.nfi.storm.kafka.fail.UnreliableFailHandler)3 SpoutOutputCollector (org.apache.storm.spout.SpoutOutputCollector)3 TopologyContext (org.apache.storm.task.TopologyContext)3 ByteBuffer (java.nio.ByteBuffer)1 HashMap (java.util.HashMap)1 LinkedList (java.util.LinkedList)1 SortedMap (java.util.SortedMap)1 ConsumerTimeoutException (kafka.consumer.ConsumerTimeoutException)1 ConsumerConnector (kafka.javaapi.consumer.ConsumerConnector)1 InvalidMessageException (kafka.message.InvalidMessageException)1 ConfigUtils.createFailHandlerFromString (nl.minvenj.nfi.storm.kafka.util.ConfigUtils.createFailHandlerFromString)1 Values (org.apache.storm.tuple.Values)1