Search in sources :

Example 1 with FailHandler

use of nl.minvenj.nfi.storm.kafka.fail.FailHandler in project kafka-spout by HolmesNL.

the class KafkaSpoutFailurePolicyTest method testDelegatedCalls.

@Test
public void testDelegatedCalls() {
    final FailHandler unreliable = spy(new UnreliableFailHandler());
    _subject = new KafkaSpout() {

        @Override
        protected void createConsumer(final Map<String, Object> config) {
        // do nothing (would connect to zookeeper otherwise)
        }

        @Override
        protected void createFailHandler(final String failHandler) {
            _failHandler = unreliable;
        }
    };
    _subject._failHandler = unreliable;
    _subject._consumer = mock(ConsumerConnector.class);
    // call all the methods that should trigger policy notification
    _subject.open(mock(Map.class), mock(TopologyContext.class), mock(SpoutOutputCollector.class));
    verify(unreliable).open(any(Map.class), any(TopologyContext.class), any(SpoutOutputCollector.class));
    _subject.activate();
    verify(unreliable).activate();
    _subject.deactivate();
    verify(unreliable).deactivate();
    _subject.close();
    verify(unreliable).close();
    // NB: _subject will have set consumer to null, mock a new one
    _subject._consumer = mock(ConsumerConnector.class);
    // simulate an acknowledged message
    final KafkaMessageId id = new KafkaMessageId(1, 1234);
    final byte[] message = { 1, 2, 3, 4 };
    _subject._inProgress.put(id, message);
    _subject.ack(id);
    verify(unreliable).ack(id);
// failure is tested above
}
Also used : ConsumerConnector(kafka.javaapi.consumer.ConsumerConnector) KafkaMessageId(nl.minvenj.nfi.storm.kafka.util.KafkaMessageId) SpoutOutputCollector(backtype.storm.spout.SpoutOutputCollector) TopologyContext(backtype.storm.task.TopologyContext) FailHandler(nl.minvenj.nfi.storm.kafka.fail.FailHandler) ReliableFailHandler(nl.minvenj.nfi.storm.kafka.fail.ReliableFailHandler) UnreliableFailHandler(nl.minvenj.nfi.storm.kafka.fail.UnreliableFailHandler) Map(java.util.Map) UnreliableFailHandler(nl.minvenj.nfi.storm.kafka.fail.UnreliableFailHandler) Test(org.junit.Test)

Example 2 with FailHandler

use of nl.minvenj.nfi.storm.kafka.fail.FailHandler in project kafka-spout by HolmesNL.

the class KafkaSpoutFailurePolicyTest method testReliableFailure.

@Test
public void testReliableFailure() {
    final FailHandler reliable = spy(new ReliableFailHandler());
    _subject._failHandler = reliable;
    final KafkaMessageId id = new KafkaMessageId(1, 1234);
    final byte[] message = { 1, 2, 3, 4 };
    _subject._inProgress.put(id, message);
    // fail the message
    _subject.fail(id);
    // verify decision delegate to fail handler and replay
    verify(reliable).shouldReplay(eq(id));
    assertEquals(id, _subject._queue.peek());
    assertTrue(_subject._inProgress.containsKey(id));
}
Also used : KafkaMessageId(nl.minvenj.nfi.storm.kafka.util.KafkaMessageId) ReliableFailHandler(nl.minvenj.nfi.storm.kafka.fail.ReliableFailHandler) FailHandler(nl.minvenj.nfi.storm.kafka.fail.FailHandler) ReliableFailHandler(nl.minvenj.nfi.storm.kafka.fail.ReliableFailHandler) UnreliableFailHandler(nl.minvenj.nfi.storm.kafka.fail.UnreliableFailHandler) Test(org.junit.Test)

Example 3 with FailHandler

use of nl.minvenj.nfi.storm.kafka.fail.FailHandler in project kafka-spout by HolmesNL.

the class KafkaSpoutFailurePolicyTest method testUnreliableFailure.

@Test
public void testUnreliableFailure() {
    final FailHandler unreliable = spy(new UnreliableFailHandler());
    _subject._failHandler = unreliable;
    final KafkaMessageId id = new KafkaMessageId(1, 1234);
    final byte[] message = { 1, 2, 3, 4 };
    _subject._inProgress.put(id, message);
    // fail the message
    _subject.fail(id);
    // verify decision and message delegate to fail handler and no replay
    verify(unreliable).shouldReplay(eq(id));
    verify(unreliable).fail(eq(id), eq(message));
    assertTrue(_subject._queue.isEmpty());
    assertFalse(_subject._inProgress.containsKey(id));
}
Also used : KafkaMessageId(nl.minvenj.nfi.storm.kafka.util.KafkaMessageId) FailHandler(nl.minvenj.nfi.storm.kafka.fail.FailHandler) ReliableFailHandler(nl.minvenj.nfi.storm.kafka.fail.ReliableFailHandler) UnreliableFailHandler(nl.minvenj.nfi.storm.kafka.fail.UnreliableFailHandler) UnreliableFailHandler(nl.minvenj.nfi.storm.kafka.fail.UnreliableFailHandler) Test(org.junit.Test)

Aggregations

FailHandler (nl.minvenj.nfi.storm.kafka.fail.FailHandler)3 ReliableFailHandler (nl.minvenj.nfi.storm.kafka.fail.ReliableFailHandler)3 UnreliableFailHandler (nl.minvenj.nfi.storm.kafka.fail.UnreliableFailHandler)3 KafkaMessageId (nl.minvenj.nfi.storm.kafka.util.KafkaMessageId)3 Test (org.junit.Test)3 SpoutOutputCollector (backtype.storm.spout.SpoutOutputCollector)1 TopologyContext (backtype.storm.task.TopologyContext)1 Map (java.util.Map)1 ConsumerConnector (kafka.javaapi.consumer.ConsumerConnector)1