Search in sources :

Example 6 with Record

use of org.apache.kafka.streams.processor.api.Record in project kafka by apache.

the class KTableSuppressProcessorTest method suppressShouldEmitWhenOverRecordCapacity.

@Test
public void suppressShouldEmitWhenOverRecordCapacity() {
    final Harness<String, Long> harness = new Harness<>(untilTimeLimit(Duration.ofDays(100), maxRecords(1)), String(), Long());
    final MockInternalNewProcessorContext<String, Change<Long>> context = harness.context;
    final long timestamp = 100L;
    context.setRecordMetadata("", 0, 0L);
    context.setTimestamp(timestamp);
    final String key = "hey";
    final Change<Long> value = new Change<>(null, ARBITRARY_LONG);
    harness.processor.process(new Record<>(key, value, timestamp));
    context.setRecordMetadata("", 0, 1L);
    context.setTimestamp(timestamp + 1);
    harness.processor.process(new Record<>("dummyKey", value, timestamp + 1));
    assertThat(context.forwarded(), hasSize(1));
    final MockProcessorContext.CapturedForward capturedForward = context.forwarded().get(0);
    assertThat(capturedForward.record(), is(new Record<>(key, value, timestamp)));
}
Also used : Long(org.apache.kafka.common.serialization.Serdes.Long) Record(org.apache.kafka.streams.processor.api.Record) String(org.apache.kafka.common.serialization.Serdes.String) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) Change(org.apache.kafka.streams.kstream.internals.Change) MockProcessorContext(org.apache.kafka.streams.processor.api.MockProcessorContext) Test(org.junit.Test)

Example 7 with Record

use of org.apache.kafka.streams.processor.api.Record in project kafka by apache.

the class KTableSuppressProcessorTest method finalResultsWithZeroGraceAtWindowEndShouldImmediatelyEmit.

@Test
public void finalResultsWithZeroGraceAtWindowEndShouldImmediatelyEmit() {
    final Harness<Windowed<String>, Long> harness = new Harness<>(finalResults(ofMillis(0L)), timeWindowedSerdeFrom(String.class, 100L), Long());
    final MockInternalNewProcessorContext<Windowed<String>, Change<Long>> context = harness.context;
    final long timestamp = 100L;
    context.setRecordMetadata("", 0, 0L);
    context.setTimestamp(timestamp);
    final Windowed<String> key = new Windowed<>("hey", new TimeWindow(0, 100L));
    final Change<Long> value = ARBITRARY_CHANGE;
    harness.processor.process(new Record<>(key, value, timestamp));
    assertThat(context.forwarded(), hasSize(1));
    final MockProcessorContext.CapturedForward capturedForward = context.forwarded().get(0);
    assertThat(capturedForward.record(), is(new Record<>(key, value, timestamp)));
}
Also used : String(org.apache.kafka.common.serialization.Serdes.String) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) Change(org.apache.kafka.streams.kstream.internals.Change) TimeWindow(org.apache.kafka.streams.kstream.internals.TimeWindow) MockProcessorContext(org.apache.kafka.streams.processor.api.MockProcessorContext) Windowed(org.apache.kafka.streams.kstream.Windowed) Long(org.apache.kafka.common.serialization.Serdes.Long) Record(org.apache.kafka.streams.processor.api.Record) Test(org.junit.Test)

Example 8 with Record

use of org.apache.kafka.streams.processor.api.Record in project kafka by apache.

the class KTableSuppressProcessorTest method suppressShouldEmitWhenOverByteCapacity.

@Test
public void suppressShouldEmitWhenOverByteCapacity() {
    final Harness<String, Long> harness = new Harness<>(untilTimeLimit(Duration.ofDays(100), maxBytes(60L)), String(), Long());
    final MockInternalNewProcessorContext<String, Change<Long>> context = harness.context;
    final long timestamp = 100L;
    context.setRecordMetadata("", 0, 0L);
    context.setTimestamp(timestamp);
    final String key = "hey";
    final Change<Long> value = new Change<>(null, ARBITRARY_LONG);
    harness.processor.process(new Record<>(key, value, timestamp));
    context.setRecordMetadata("", 0, 1L);
    context.setTimestamp(timestamp + 1);
    harness.processor.process(new Record<>("dummyKey", value, timestamp + 1));
    assertThat(context.forwarded(), hasSize(1));
    final MockProcessorContext.CapturedForward capturedForward = context.forwarded().get(0);
    assertThat(capturedForward.record(), is(new Record<>(key, value, timestamp)));
}
Also used : Long(org.apache.kafka.common.serialization.Serdes.Long) Record(org.apache.kafka.streams.processor.api.Record) String(org.apache.kafka.common.serialization.Serdes.String) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) Change(org.apache.kafka.streams.kstream.internals.Change) MockProcessorContext(org.apache.kafka.streams.processor.api.MockProcessorContext) Test(org.junit.Test)

Example 9 with Record

use of org.apache.kafka.streams.processor.api.Record in project kafka by apache.

the class KTableSuppressProcessorTest method zeroTimeLimitShouldImmediatelyEmit.

@Test
public void zeroTimeLimitShouldImmediatelyEmit() {
    final Harness<String, Long> harness = new Harness<>(untilTimeLimit(ZERO, unbounded()), String(), Long());
    final MockInternalNewProcessorContext<String, Change<Long>> context = harness.context;
    final long timestamp = ARBITRARY_LONG;
    context.setRecordMetadata("", 0, 0L);
    context.setTimestamp(timestamp);
    final String key = "hey";
    final Change<Long> value = ARBITRARY_CHANGE;
    harness.processor.process(new Record<>(key, value, timestamp));
    assertThat(context.forwarded(), hasSize(1));
    final MockProcessorContext.CapturedForward capturedForward = context.forwarded().get(0);
    assertThat(capturedForward.record(), is(new Record<>(key, value, timestamp)));
}
Also used : Long(org.apache.kafka.common.serialization.Serdes.Long) Record(org.apache.kafka.streams.processor.api.Record) String(org.apache.kafka.common.serialization.Serdes.String) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) Change(org.apache.kafka.streams.kstream.internals.Change) MockProcessorContext(org.apache.kafka.streams.processor.api.MockProcessorContext) Test(org.junit.Test)

Example 10 with Record

use of org.apache.kafka.streams.processor.api.Record in project kafka by apache.

the class KTableSuppressProcessorTest method intermediateSuppressionShouldBufferAndEmitLater.

@Test
public void intermediateSuppressionShouldBufferAndEmitLater() {
    final Harness<String, Long> harness = new Harness<>(untilTimeLimit(ofMillis(1), unbounded()), String(), Long());
    final MockInternalNewProcessorContext<String, Change<Long>> context = harness.context;
    final long timestamp = 0L;
    context.setRecordMetadata("topic", 0, 0);
    context.setTimestamp(timestamp);
    final String key = "hey";
    final Change<Long> value = new Change<>(null, 1L);
    harness.processor.process(new Record<>(key, value, timestamp));
    assertThat(context.forwarded(), hasSize(0));
    context.setRecordMetadata("topic", 0, 1);
    context.setTimestamp(1L);
    harness.processor.process(new Record<>("tick", new Change<>(null, null), 1L));
    assertThat(context.forwarded(), hasSize(1));
    final MockProcessorContext.CapturedForward capturedForward = context.forwarded().get(0);
    assertThat(capturedForward.record(), is(new Record<>(key, value, timestamp)));
}
Also used : Long(org.apache.kafka.common.serialization.Serdes.Long) Record(org.apache.kafka.streams.processor.api.Record) String(org.apache.kafka.common.serialization.Serdes.String) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) Change(org.apache.kafka.streams.kstream.internals.Change) MockProcessorContext(org.apache.kafka.streams.processor.api.MockProcessorContext) Test(org.junit.Test)

Aggregations

Record (org.apache.kafka.streams.processor.api.Record)24 Test (org.junit.Test)18 MockProcessorContext (org.apache.kafka.streams.processor.api.MockProcessorContext)16 Change (org.apache.kafka.streams.kstream.internals.Change)12 CoreMatchers.containsString (org.hamcrest.CoreMatchers.containsString)12 Long (org.apache.kafka.common.serialization.Serdes.Long)11 String (org.apache.kafka.common.serialization.Serdes.String)11 Windowed (org.apache.kafka.streams.kstream.Windowed)7 Processor (org.apache.kafka.streams.processor.api.Processor)6 TimeWindow (org.apache.kafka.streams.kstream.internals.TimeWindow)5 Metrics (org.apache.kafka.common.metrics.Metrics)4 MockTime (org.apache.kafka.common.utils.MockTime)4 ProcessorContext (org.apache.kafka.streams.processor.api.ProcessorContext)4 List (java.util.List)3 Properties (java.util.Properties)3 Serdes (org.apache.kafka.common.serialization.Serdes)3 StreamsConfig (org.apache.kafka.streams.StreamsConfig)3 Test (org.junit.jupiter.api.Test)3 File (java.io.File)2 Collectors (java.util.stream.Collectors)2