Search in sources :

Example 21 with StreamsException

use of org.apache.kafka.streams.errors.StreamsException in project apache-kafka-on-k8s by banzaicloud.

the class GlobalStateManagerImplTest method shouldReleaseLockIfExceptionWhenLoadingCheckpoints.

@Test
public void shouldReleaseLockIfExceptionWhenLoadingCheckpoints() throws IOException {
    writeCorruptCheckpoint();
    try {
        stateManager.initialize();
    } catch (StreamsException e) {
    // expected
    }
    final StateDirectory stateDir = new StateDirectory(streamsConfig, new MockTime());
    try {
        // should be able to get the lock now as it should've been released
        assertTrue(stateDir.lockGlobalState());
    } finally {
        stateDir.unlockGlobalState();
    }
}
Also used : StreamsException(org.apache.kafka.streams.errors.StreamsException) MockTime(org.apache.kafka.common.utils.MockTime) Test(org.junit.Test)

Example 22 with StreamsException

use of org.apache.kafka.streams.errors.StreamsException in project apache-kafka-on-k8s by banzaicloud.

the class GlobalStateManagerImplTest method shouldRetryWhenEndOffsetsThrowsTimeoutException.

@Test
public void shouldRetryWhenEndOffsetsThrowsTimeoutException() {
    final int retries = 2;
    final AtomicInteger numberOfCalls = new AtomicInteger(0);
    consumer = new MockConsumer<byte[], byte[]>(OffsetResetStrategy.EARLIEST) {

        @Override
        public synchronized Map<TopicPartition, Long> endOffsets(Collection<org.apache.kafka.common.TopicPartition> partitions) {
            numberOfCalls.incrementAndGet();
            throw new TimeoutException();
        }
    };
    streamsConfig = new StreamsConfig(new Properties() {

        {
            put(StreamsConfig.APPLICATION_ID_CONFIG, "appId");
            put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "dummy:1234");
            put(StreamsConfig.STATE_DIR_CONFIG, TestUtils.tempDirectory().getPath());
            put(StreamsConfig.RETRIES_CONFIG, retries);
        }
    });
    try {
        new GlobalStateManagerImpl(new LogContext("mock"), topology, consumer, stateDirectory, stateRestoreListener, streamsConfig);
    } catch (final StreamsException expected) {
        assertEquals(numberOfCalls.get(), retries);
    }
}
Also used : StreamsException(org.apache.kafka.streams.errors.StreamsException) LogContext(org.apache.kafka.common.utils.LogContext) Properties(java.util.Properties) OffsetCheckpoint(org.apache.kafka.streams.state.internals.OffsetCheckpoint) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) TopicPartition(org.apache.kafka.common.TopicPartition) HashMap(java.util.HashMap) Map(java.util.Map) TimeoutException(org.apache.kafka.common.errors.TimeoutException) StreamsConfig(org.apache.kafka.streams.StreamsConfig) Test(org.junit.Test)

Example 23 with StreamsException

use of org.apache.kafka.streams.errors.StreamsException in project apache-kafka-on-k8s by banzaicloud.

the class GlobalStateManagerImplTest method shouldRetryWhenPartitionsForThrowsTimeoutException.

@Test
public void shouldRetryWhenPartitionsForThrowsTimeoutException() {
    final int retries = 2;
    final AtomicInteger numberOfCalls = new AtomicInteger(0);
    consumer = new MockConsumer<byte[], byte[]>(OffsetResetStrategy.EARLIEST) {

        @Override
        public synchronized List<PartitionInfo> partitionsFor(String topic) {
            numberOfCalls.incrementAndGet();
            throw new TimeoutException();
        }
    };
    streamsConfig = new StreamsConfig(new Properties() {

        {
            put(StreamsConfig.APPLICATION_ID_CONFIG, "appId");
            put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "dummy:1234");
            put(StreamsConfig.STATE_DIR_CONFIG, TestUtils.tempDirectory().getPath());
            put(StreamsConfig.RETRIES_CONFIG, retries);
        }
    });
    try {
        new GlobalStateManagerImpl(new LogContext("mock"), topology, consumer, stateDirectory, stateRestoreListener, streamsConfig);
    } catch (final StreamsException expected) {
        assertEquals(numberOfCalls.get(), retries);
    }
}
Also used : AtomicInteger(java.util.concurrent.atomic.AtomicInteger) StreamsException(org.apache.kafka.streams.errors.StreamsException) LogContext(org.apache.kafka.common.utils.LogContext) ArrayList(java.util.ArrayList) List(java.util.List) Properties(java.util.Properties) OffsetCheckpoint(org.apache.kafka.streams.state.internals.OffsetCheckpoint) TimeoutException(org.apache.kafka.common.errors.TimeoutException) StreamsConfig(org.apache.kafka.streams.StreamsConfig) Test(org.junit.Test)

Example 24 with StreamsException

use of org.apache.kafka.streams.errors.StreamsException in project apache-kafka-on-k8s by banzaicloud.

the class RecordCollectorTest method shouldThrowStreamsExceptionOnSubsequentCallIfASendFailsWithDefaultExceptionHandler.

@SuppressWarnings("unchecked")
@Test
public void shouldThrowStreamsExceptionOnSubsequentCallIfASendFailsWithDefaultExceptionHandler() {
    final RecordCollector collector = new RecordCollectorImpl(new MockProducer(cluster, true, new DefaultPartitioner(), byteArraySerializer, byteArraySerializer) {

        @Override
        public synchronized Future<RecordMetadata> send(final ProducerRecord record, final Callback callback) {
            callback.onCompletion(null, new Exception());
            return null;
        }
    }, "test", logContext, new DefaultProductionExceptionHandler());
    collector.send("topic1", "3", "0", null, stringSerializer, stringSerializer, streamPartitioner);
    try {
        collector.send("topic1", "3", "0", null, stringSerializer, stringSerializer, streamPartitioner);
        fail("Should have thrown StreamsException");
    } catch (final StreamsException expected) {
    /* ok */
    }
}
Also used : MockProducer(org.apache.kafka.clients.producer.MockProducer) Callback(org.apache.kafka.clients.producer.Callback) DefaultPartitioner(org.apache.kafka.clients.producer.internals.DefaultPartitioner) DefaultProductionExceptionHandler(org.apache.kafka.streams.errors.DefaultProductionExceptionHandler) ProducerRecord(org.apache.kafka.clients.producer.ProducerRecord) StreamsException(org.apache.kafka.streams.errors.StreamsException) Future(java.util.concurrent.Future) KafkaException(org.apache.kafka.common.KafkaException) StreamsException(org.apache.kafka.streams.errors.StreamsException) Test(org.junit.Test)

Example 25 with StreamsException

use of org.apache.kafka.streams.errors.StreamsException in project apache-kafka-on-k8s by banzaicloud.

the class RecordCollectorTest method shouldThrowStreamsExceptionOnFlushIfASendFailedWithDefaultExceptionHandler.

@SuppressWarnings("unchecked")
@Test
public void shouldThrowStreamsExceptionOnFlushIfASendFailedWithDefaultExceptionHandler() {
    final RecordCollector collector = new RecordCollectorImpl(new MockProducer(cluster, true, new DefaultPartitioner(), byteArraySerializer, byteArraySerializer) {

        @Override
        public synchronized Future<RecordMetadata> send(final ProducerRecord record, final Callback callback) {
            callback.onCompletion(null, new Exception());
            return null;
        }
    }, "test", logContext, new DefaultProductionExceptionHandler());
    collector.send("topic1", "3", "0", null, stringSerializer, stringSerializer, streamPartitioner);
    try {
        collector.flush();
        fail("Should have thrown StreamsException");
    } catch (final StreamsException expected) {
    /* ok */
    }
}
Also used : MockProducer(org.apache.kafka.clients.producer.MockProducer) Callback(org.apache.kafka.clients.producer.Callback) DefaultPartitioner(org.apache.kafka.clients.producer.internals.DefaultPartitioner) DefaultProductionExceptionHandler(org.apache.kafka.streams.errors.DefaultProductionExceptionHandler) ProducerRecord(org.apache.kafka.clients.producer.ProducerRecord) StreamsException(org.apache.kafka.streams.errors.StreamsException) Future(java.util.concurrent.Future) KafkaException(org.apache.kafka.common.KafkaException) StreamsException(org.apache.kafka.streams.errors.StreamsException) Test(org.junit.Test)

Aggregations

StreamsException (org.apache.kafka.streams.errors.StreamsException)186 Test (org.junit.Test)90 KafkaException (org.apache.kafka.common.KafkaException)41 TopicPartition (org.apache.kafka.common.TopicPartition)38 TimeoutException (org.apache.kafka.common.errors.TimeoutException)36 HashMap (java.util.HashMap)27 Map (java.util.Map)25 HashSet (java.util.HashSet)18 Properties (java.util.Properties)17 TaskId (org.apache.kafka.streams.processor.TaskId)14 ConsumerRecord (org.apache.kafka.clients.consumer.ConsumerRecord)13 StreamsConfig (org.apache.kafka.streams.StreamsConfig)12 ArrayList (java.util.ArrayList)11 ExecutionException (java.util.concurrent.ExecutionException)11 TaskMigratedException (org.apache.kafka.streams.errors.TaskMigratedException)11 IOException (java.io.IOException)10 Set (java.util.Set)10 LogContext (org.apache.kafka.common.utils.LogContext)10 MockTime (org.apache.kafka.common.utils.MockTime)10 StateStore (org.apache.kafka.streams.processor.StateStore)10