Search in sources :

Example 16 with LogContext

use of org.apache.kafka.common.utils.LogContext in project apache-kafka-on-k8s by banzaicloud.

the class StreamThreadTest method shouldNotCauseExceptionIfNothingCommited.

@SuppressWarnings({ "unchecked", "ThrowableNotThrown" })
@Test
public void shouldNotCauseExceptionIfNothingCommited() {
    final long commitInterval = 1000L;
    final Properties props = configProps(false);
    props.setProperty(StreamsConfig.STATE_DIR_CONFIG, stateDir);
    props.setProperty(StreamsConfig.COMMIT_INTERVAL_MS_CONFIG, Long.toString(commitInterval));
    final StreamsConfig config = new StreamsConfig(props);
    final Consumer<byte[], byte[]> consumer = EasyMock.createNiceMock(Consumer.class);
    final TaskManager taskManager = mockTaskManagerCommit(consumer, 1, 0);
    StreamThread.StreamsMetricsThreadImpl streamsMetrics = new StreamThread.StreamsMetricsThreadImpl(metrics, "", "", Collections.<String, String>emptyMap());
    final StreamThread thread = new StreamThread(mockTime, config, consumer, consumer, null, taskManager, streamsMetrics, internalTopologyBuilder, clientId, new LogContext(""));
    thread.maybeCommit(mockTime.milliseconds());
    mockTime.sleep(commitInterval - 10L);
    thread.maybeCommit(mockTime.milliseconds());
    EasyMock.verify(taskManager);
}
Also used : LogContext(org.apache.kafka.common.utils.LogContext) Properties(java.util.Properties) StreamsConfig(org.apache.kafka.streams.StreamsConfig) InternalStreamsBuilderTest(org.apache.kafka.streams.kstream.internals.InternalStreamsBuilderTest) Test(org.junit.Test)

Example 17 with LogContext

use of org.apache.kafka.common.utils.LogContext in project apache-kafka-on-k8s by banzaicloud.

the class StreamThreadTest method shouldShutdownTaskManagerOnCloseWithoutStart.

@SuppressWarnings("unchecked")
@Test
public void shouldShutdownTaskManagerOnCloseWithoutStart() {
    final Consumer<byte[], byte[]> consumer = EasyMock.createNiceMock(Consumer.class);
    final TaskManager taskManager = EasyMock.createNiceMock(TaskManager.class);
    taskManager.shutdown(true);
    EasyMock.expectLastCall();
    EasyMock.replay(taskManager, consumer);
    final StreamThread.StreamsMetricsThreadImpl streamsMetrics = new StreamThread.StreamsMetricsThreadImpl(metrics, "", "", Collections.<String, String>emptyMap());
    final StreamThread thread = new StreamThread(mockTime, config, consumer, consumer, null, taskManager, streamsMetrics, internalTopologyBuilder, clientId, new LogContext(""));
    thread.shutdown();
    EasyMock.verify(taskManager);
}
Also used : LogContext(org.apache.kafka.common.utils.LogContext) InternalStreamsBuilderTest(org.apache.kafka.streams.kstream.internals.InternalStreamsBuilderTest) Test(org.junit.Test)

Example 18 with LogContext

use of org.apache.kafka.common.utils.LogContext in project apache-kafka-on-k8s by banzaicloud.

the class StreamThreadTest method shouldOnlyShutdownOnce.

@SuppressWarnings("unchecked")
@Test
public void shouldOnlyShutdownOnce() {
    final Consumer<byte[], byte[]> consumer = EasyMock.createNiceMock(Consumer.class);
    final TaskManager taskManager = EasyMock.createNiceMock(TaskManager.class);
    taskManager.shutdown(true);
    EasyMock.expectLastCall();
    EasyMock.replay(taskManager, consumer);
    final StreamThread.StreamsMetricsThreadImpl streamsMetrics = new StreamThread.StreamsMetricsThreadImpl(metrics, "", "", Collections.<String, String>emptyMap());
    final StreamThread thread = new StreamThread(mockTime, config, consumer, consumer, null, taskManager, streamsMetrics, internalTopologyBuilder, clientId, new LogContext(""));
    thread.shutdown();
    // Execute the run method. Verification of the mock will check that shutdown was only done once
    thread.run();
    EasyMock.verify(taskManager);
}
Also used : LogContext(org.apache.kafka.common.utils.LogContext) InternalStreamsBuilderTest(org.apache.kafka.streams.kstream.internals.InternalStreamsBuilderTest) Test(org.junit.Test)

Example 19 with LogContext

use of org.apache.kafka.common.utils.LogContext in project apache-kafka-on-k8s by banzaicloud.

the class StreamThreadTest method shouldCommitAfterTheCommitInterval.

@SuppressWarnings("unchecked")
@Test
public void shouldCommitAfterTheCommitInterval() {
    final long commitInterval = 1000L;
    final Properties props = configProps(false);
    props.setProperty(StreamsConfig.STATE_DIR_CONFIG, stateDir);
    props.setProperty(StreamsConfig.COMMIT_INTERVAL_MS_CONFIG, Long.toString(commitInterval));
    final StreamsConfig config = new StreamsConfig(props);
    final Consumer<byte[], byte[]> consumer = EasyMock.createNiceMock(Consumer.class);
    final TaskManager taskManager = mockTaskManagerCommit(consumer, 2, 1);
    StreamThread.StreamsMetricsThreadImpl streamsMetrics = new StreamThread.StreamsMetricsThreadImpl(metrics, "", "", Collections.<String, String>emptyMap());
    final StreamThread thread = new StreamThread(mockTime, config, consumer, consumer, null, taskManager, streamsMetrics, internalTopologyBuilder, clientId, new LogContext(""));
    thread.maybeCommit(mockTime.milliseconds());
    mockTime.sleep(commitInterval + 1);
    thread.maybeCommit(mockTime.milliseconds());
    EasyMock.verify(taskManager);
}
Also used : LogContext(org.apache.kafka.common.utils.LogContext) Properties(java.util.Properties) StreamsConfig(org.apache.kafka.streams.StreamsConfig) InternalStreamsBuilderTest(org.apache.kafka.streams.kstream.internals.InternalStreamsBuilderTest) Test(org.junit.Test)

Example 20 with LogContext

use of org.apache.kafka.common.utils.LogContext in project apache-kafka-on-k8s by banzaicloud.

the class GlobalStateManagerImplTest method shouldRetryWhenEndOffsetsThrowsTimeoutException.

@Test
public void shouldRetryWhenEndOffsetsThrowsTimeoutException() {
    final int retries = 2;
    final AtomicInteger numberOfCalls = new AtomicInteger(0);
    consumer = new MockConsumer<byte[], byte[]>(OffsetResetStrategy.EARLIEST) {

        @Override
        public synchronized Map<TopicPartition, Long> endOffsets(Collection<org.apache.kafka.common.TopicPartition> partitions) {
            numberOfCalls.incrementAndGet();
            throw new TimeoutException();
        }
    };
    streamsConfig = new StreamsConfig(new Properties() {

        {
            put(StreamsConfig.APPLICATION_ID_CONFIG, "appId");
            put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "dummy:1234");
            put(StreamsConfig.STATE_DIR_CONFIG, TestUtils.tempDirectory().getPath());
            put(StreamsConfig.RETRIES_CONFIG, retries);
        }
    });
    try {
        new GlobalStateManagerImpl(new LogContext("mock"), topology, consumer, stateDirectory, stateRestoreListener, streamsConfig);
    } catch (final StreamsException expected) {
        assertEquals(numberOfCalls.get(), retries);
    }
}
Also used : StreamsException(org.apache.kafka.streams.errors.StreamsException) LogContext(org.apache.kafka.common.utils.LogContext) Properties(java.util.Properties) OffsetCheckpoint(org.apache.kafka.streams.state.internals.OffsetCheckpoint) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) TopicPartition(org.apache.kafka.common.TopicPartition) HashMap(java.util.HashMap) Map(java.util.Map) TimeoutException(org.apache.kafka.common.errors.TimeoutException) StreamsConfig(org.apache.kafka.streams.StreamsConfig) Test(org.junit.Test)

Aggregations

LogContext (org.apache.kafka.common.utils.LogContext)53 Metrics (org.apache.kafka.common.metrics.Metrics)28 Test (org.junit.Test)27 Before (org.junit.Before)17 InternalMockProcessorContext (org.apache.kafka.test.InternalMockProcessorContext)14 MockStreamsMetrics (org.apache.kafka.streams.processor.internals.MockStreamsMetrics)13 NoOpRecordCollector (org.apache.kafka.test.NoOpRecordCollector)10 HashMap (java.util.HashMap)8 MockTime (org.apache.kafka.common.utils.MockTime)8 InetSocketAddress (java.net.InetSocketAddress)7 Properties (java.util.Properties)7 StreamsConfig (org.apache.kafka.streams.StreamsConfig)7 TopicPartition (org.apache.kafka.common.TopicPartition)6 File (java.io.File)5 NetworkClient (org.apache.kafka.clients.NetworkClient)5 InternalStreamsBuilderTest (org.apache.kafka.streams.kstream.internals.InternalStreamsBuilderTest)5 ServerSocketChannel (java.nio.channels.ServerSocketChannel)4 SocketChannel (java.nio.channels.SocketChannel)4 Metadata (org.apache.kafka.clients.Metadata)4 MockClient (org.apache.kafka.clients.MockClient)4