Search in sources :

Example 11 with StreamsException

use of org.apache.kafka.streams.errors.StreamsException in project kafka by apache.

the class SinkNodeTest method shouldThrowStreamsExceptionOnInputRecordWithInvalidTimestamp.

@Test
@SuppressWarnings("unchecked")
public void shouldThrowStreamsExceptionOnInputRecordWithInvalidTimestamp() {
    // Given
    final Serializer anySerializer = Serdes.Bytes().serializer();
    final StateSerdes anyStateSerde = StateSerdes.withBuiltinTypes("anyName", Bytes.class, Bytes.class);
    final MockProcessorContext context = new MockProcessorContext(anyStateSerde, new RecordCollectorImpl(new MockProducer<byte[], byte[]>(true, anySerializer, anySerializer), null));
    final SinkNode sink = new SinkNode<>("anyNodeName", "any-output-topic", anySerializer, anySerializer, null);
    sink.init(context);
    final Bytes anyKey = new Bytes("any key".getBytes());
    final Bytes anyValue = new Bytes("any value".getBytes());
    // When/Then
    // ensures a negative timestamp is set for the record we send next
    context.setTime(-1);
    try {
        sink.process(anyKey, anyValue);
        fail("Should have thrown StreamsException");
    } catch (final StreamsException ignored) {
    }
}
Also used : Bytes(org.apache.kafka.common.utils.Bytes) MockProducer(org.apache.kafka.clients.producer.MockProducer) StreamsException(org.apache.kafka.streams.errors.StreamsException) StateSerdes(org.apache.kafka.streams.state.StateSerdes) MockProcessorContext(org.apache.kafka.test.MockProcessorContext) Serializer(org.apache.kafka.common.serialization.Serializer) Test(org.junit.Test)

Example 12 with StreamsException

use of org.apache.kafka.streams.errors.StreamsException in project kafka by apache.

the class SinkNodeTest method shouldHandleNullValuesWhenThrowingStreamsExceptionOnKeyValueTypeSerializerMismatch.

@Test
@SuppressWarnings("unchecked")
public void shouldHandleNullValuesWhenThrowingStreamsExceptionOnKeyValueTypeSerializerMismatch() {
    // Given
    final Serializer anySerializer = Serdes.Bytes().serializer();
    final StateSerdes anyStateSerde = StateSerdes.withBuiltinTypes("anyName", Bytes.class, Bytes.class);
    final MockProcessorContext context = new MockProcessorContext(anyStateSerde, new RecordCollectorImpl(new MockProducer<byte[], byte[]>(true, anySerializer, anySerializer), null));
    context.setTime(1);
    final SinkNode sink = new SinkNode<>("anyNodeName", "any-output-topic", anySerializer, anySerializer, null);
    sink.init(context);
    final String invalidKeyToTriggerSerializerMismatch = "";
    // When/Then
    try {
        sink.process(invalidKeyToTriggerSerializerMismatch, null);
        fail("Should have thrown StreamsException");
    } catch (final StreamsException e) {
        assertThat(e.getCause(), instanceOf(ClassCastException.class));
        assertThat(e.getMessage(), containsString("unknown because value is null"));
    }
}
Also used : MockProducer(org.apache.kafka.clients.producer.MockProducer) StreamsException(org.apache.kafka.streams.errors.StreamsException) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) StateSerdes(org.apache.kafka.streams.state.StateSerdes) MockProcessorContext(org.apache.kafka.test.MockProcessorContext) Serializer(org.apache.kafka.common.serialization.Serializer) Test(org.junit.Test)

Example 13 with StreamsException

use of org.apache.kafka.streams.errors.StreamsException in project kafka by apache.

the class SinkNodeTest method shouldHandleNullKeysWhenThrowingStreamsExceptionOnKeyValueTypeSerializerMismatch.

@Test
@SuppressWarnings("unchecked")
public void shouldHandleNullKeysWhenThrowingStreamsExceptionOnKeyValueTypeSerializerMismatch() {
    // Given
    final Serializer anySerializer = Serdes.Bytes().serializer();
    final StateSerdes anyStateSerde = StateSerdes.withBuiltinTypes("anyName", Bytes.class, Bytes.class);
    final MockProcessorContext context = new MockProcessorContext(anyStateSerde, new RecordCollectorImpl(new MockProducer<byte[], byte[]>(true, anySerializer, anySerializer), null));
    context.setTime(1);
    final SinkNode sink = new SinkNode<>("anyNodeName", "any-output-topic", anySerializer, anySerializer, null);
    sink.init(context);
    final String invalidValueToTriggerSerializerMismatch = "";
    // When/Then
    try {
        sink.process(null, invalidValueToTriggerSerializerMismatch);
        fail("Should have thrown StreamsException");
    } catch (final StreamsException e) {
        assertThat(e.getCause(), instanceOf(ClassCastException.class));
        assertThat(e.getMessage(), containsString("unknown because key is null"));
    }
}
Also used : MockProducer(org.apache.kafka.clients.producer.MockProducer) StreamsException(org.apache.kafka.streams.errors.StreamsException) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) StateSerdes(org.apache.kafka.streams.state.StateSerdes) MockProcessorContext(org.apache.kafka.test.MockProcessorContext) Serializer(org.apache.kafka.common.serialization.Serializer) Test(org.junit.Test)

Example 14 with StreamsException

use of org.apache.kafka.streams.errors.StreamsException in project kafka by apache.

the class StoreChangelogReaderTest method shouldThrowStreamsExceptionIfTimeoutOccursDuringPartitionsFor.

@SuppressWarnings("unchecked")
@Test
public void shouldThrowStreamsExceptionIfTimeoutOccursDuringPartitionsFor() throws Exception {
    final MockConsumer<byte[], byte[]> consumer = new MockConsumer(OffsetResetStrategy.EARLIEST) {

        @Override
        public List<PartitionInfo> partitionsFor(final String topic) {
            throw new TimeoutException("KABOOM!");
        }
    };
    final StoreChangelogReader changelogReader = new StoreChangelogReader(consumer, new MockTime(), 5);
    try {
        changelogReader.validatePartitionExists(topicPartition, "store");
        fail("Should have thrown streams exception");
    } catch (final StreamsException e) {
    // pass
    }
}
Also used : StreamsException(org.apache.kafka.streams.errors.StreamsException) PartitionInfo(org.apache.kafka.common.PartitionInfo) MockConsumer(org.apache.kafka.clients.consumer.MockConsumer) MockTime(org.apache.kafka.common.utils.MockTime) TimeoutException(org.apache.kafka.common.errors.TimeoutException) Test(org.junit.Test)

Example 15 with StreamsException

use of org.apache.kafka.streams.errors.StreamsException in project kafka by apache.

the class StreamsKafkaClient method ensureOneNodeIsReady.

/**
     *
     * @param nodes List of nodes to pick from.
     * @return The first node that is ready to accept requests.
     */
private String ensureOneNodeIsReady(final List<Node> nodes) {
    String brokerId = null;
    final long readyTimeout = Time.SYSTEM.milliseconds() + streamsConfig.getInt(StreamsConfig.REQUEST_TIMEOUT_MS_CONFIG);
    boolean foundNode = false;
    while (!foundNode && (Time.SYSTEM.milliseconds() < readyTimeout)) {
        for (Node node : nodes) {
            if (kafkaClient.ready(node, Time.SYSTEM.milliseconds())) {
                brokerId = Integer.toString(node.id());
                foundNode = true;
                break;
            }
        }
        kafkaClient.poll(streamsConfig.getLong(StreamsConfig.POLL_MS_CONFIG), Time.SYSTEM.milliseconds());
    }
    if (brokerId == null) {
        throw new StreamsException("Could not find any available broker.");
    }
    return brokerId;
}
Also used : Node(org.apache.kafka.common.Node) StreamsException(org.apache.kafka.streams.errors.StreamsException)

Aggregations

StreamsException (org.apache.kafka.streams.errors.StreamsException)186 Test (org.junit.Test)90 KafkaException (org.apache.kafka.common.KafkaException)41 TopicPartition (org.apache.kafka.common.TopicPartition)38 TimeoutException (org.apache.kafka.common.errors.TimeoutException)36 HashMap (java.util.HashMap)27 Map (java.util.Map)25 HashSet (java.util.HashSet)18 Properties (java.util.Properties)17 TaskId (org.apache.kafka.streams.processor.TaskId)14 ConsumerRecord (org.apache.kafka.clients.consumer.ConsumerRecord)13 StreamsConfig (org.apache.kafka.streams.StreamsConfig)12 ArrayList (java.util.ArrayList)11 ExecutionException (java.util.concurrent.ExecutionException)11 TaskMigratedException (org.apache.kafka.streams.errors.TaskMigratedException)11 IOException (java.io.IOException)10 Set (java.util.Set)10 LogContext (org.apache.kafka.common.utils.LogContext)10 MockTime (org.apache.kafka.common.utils.MockTime)10 StateStore (org.apache.kafka.streams.processor.StateStore)10