Search in sources :

Example 56 with ProcessorTopology

use of org.apache.kafka.streams.processor.internals.ProcessorTopology in project apache-kafka-on-k8s by banzaicloud.

the class KStreamBuilderTest method kTableTimestampExtractorShouldBeNull.

@Test
public void kTableTimestampExtractorShouldBeNull() {
    builder.table("topic", "store");
    final ProcessorTopology processorTopology = builder.build(null);
    assertNull(processorTopology.source("topic").getTimestampExtractor());
}
Also used : ProcessorTopology(org.apache.kafka.streams.processor.internals.ProcessorTopology) Test(org.junit.Test)

Example 57 with ProcessorTopology

use of org.apache.kafka.streams.processor.internals.ProcessorTopology in project apache-kafka-on-k8s by banzaicloud.

the class KStreamBuilderTest method shouldBuildSimpleGlobalTableTopology.

@Test
public void shouldBuildSimpleGlobalTableTopology() {
    builder.globalTable("table", "globalTable");
    final ProcessorTopology topology = builder.buildGlobalStateTopology();
    final List<StateStore> stateStores = topology.globalStateStores();
    assertEquals(1, stateStores.size());
    assertEquals("globalTable", stateStores.get(0).name());
}
Also used : ProcessorTopology(org.apache.kafka.streams.processor.internals.ProcessorTopology) StateStore(org.apache.kafka.streams.processor.StateStore) Test(org.junit.Test)

Example 58 with ProcessorTopology

use of org.apache.kafka.streams.processor.internals.ProcessorTopology in project apache-kafka-on-k8s by banzaicloud.

the class KStreamBuilderTest method shouldAddTimestampExtractorToStreamWithOffsetResetPerSource.

@Test
public void shouldAddTimestampExtractorToStreamWithOffsetResetPerSource() {
    builder.stream(null, new MockTimestampExtractor(), null, null, "topic");
    final ProcessorTopology processorTopology = builder.build(null);
    assertThat(processorTopology.source("topic").getTimestampExtractor(), instanceOf(MockTimestampExtractor.class));
}
Also used : ProcessorTopology(org.apache.kafka.streams.processor.internals.ProcessorTopology) MockTimestampExtractor(org.apache.kafka.test.MockTimestampExtractor) Test(org.junit.Test)

Example 59 with ProcessorTopology

use of org.apache.kafka.streams.processor.internals.ProcessorTopology in project apache-kafka-on-k8s by banzaicloud.

the class KStreamBuilderTest method shouldAddGlobalTablesToEachGroup.

@Test
public void shouldAddGlobalTablesToEachGroup() {
    final String one = "globalTable";
    final String two = "globalTable2";
    final GlobalKTable<String, String> globalTable = builder.globalTable("table", one);
    final GlobalKTable<String, String> globalTable2 = builder.globalTable("table2", two);
    builder.table("not-global", "not-global");
    final KeyValueMapper<String, String, String> kvMapper = new KeyValueMapper<String, String, String>() {

        @Override
        public String apply(final String key, final String value) {
            return value;
        }
    };
    final KStream<String, String> stream = builder.stream("t1");
    stream.leftJoin(globalTable, kvMapper, MockValueJoiner.TOSTRING_JOINER);
    final KStream<String, String> stream2 = builder.stream("t2");
    stream2.leftJoin(globalTable2, kvMapper, MockValueJoiner.TOSTRING_JOINER);
    final Map<Integer, Set<String>> nodeGroups = builder.nodeGroups();
    for (Integer groupId : nodeGroups.keySet()) {
        final ProcessorTopology topology = builder.build(groupId);
        final List<StateStore> stateStores = topology.globalStateStores();
        final Set<String> names = new HashSet<>();
        for (StateStore stateStore : stateStores) {
            names.add(stateStore.name());
        }
        assertEquals(2, stateStores.size());
        assertTrue(names.contains(one));
        assertTrue(names.contains(two));
    }
}
Also used : ProcessorTopology(org.apache.kafka.streams.processor.internals.ProcessorTopology) HashSet(java.util.HashSet) Set(java.util.Set) StateStore(org.apache.kafka.streams.processor.StateStore) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 60 with ProcessorTopology

use of org.apache.kafka.streams.processor.internals.ProcessorTopology in project apache-kafka-on-k8s by banzaicloud.

the class InternalStreamsBuilderTest method shouldUseProvidedTimestampExtractor.

@Test
public void shouldUseProvidedTimestampExtractor() throws Exception {
    final ConsumedInternal consumed = new ConsumedInternal<>(Consumed.with(new MockTimestampExtractor()));
    builder.stream(Collections.singleton("topic"), consumed);
    final ProcessorTopology processorTopology = builder.internalTopologyBuilder.build(null);
    assertThat(processorTopology.source("topic").getTimestampExtractor(), instanceOf(MockTimestampExtractor.class));
}
Also used : ProcessorTopology(org.apache.kafka.streams.processor.internals.ProcessorTopology) MockTimestampExtractor(org.apache.kafka.test.MockTimestampExtractor) Test(org.junit.Test)

Aggregations

ProcessorTopology (org.apache.kafka.streams.processor.internals.ProcessorTopology)95 Test (org.junit.Test)89 Bytes (org.apache.kafka.common.utils.Bytes)17 MockTimestampExtractor (org.apache.kafka.test.MockTimestampExtractor)14 StateStore (org.apache.kafka.streams.processor.StateStore)13 StreamsConfig (org.apache.kafka.streams.StreamsConfig)10 KeyValueStore (org.apache.kafka.streams.state.KeyValueStore)8 FailOnInvalidTimestamp (org.apache.kafka.streams.processor.FailOnInvalidTimestamp)6 Pattern (java.util.regex.Pattern)5 StreamsBuilder (org.apache.kafka.streams.StreamsBuilder)5 HashSet (java.util.HashSet)4 Properties (java.util.Properties)4 Set (java.util.Set)4 CoreMatchers.containsString (org.hamcrest.CoreMatchers.containsString)4 List (java.util.List)3 KTable (org.apache.kafka.streams.kstream.KTable)3 InternalTopologyBuilder (org.apache.kafka.streams.processor.internals.InternalTopologyBuilder)3 SourceNode (org.apache.kafka.streams.processor.internals.SourceNode)3 Arrays.asList (java.util.Arrays.asList)2 Collections (java.util.Collections)2