Search in sources :

Example 61 with SessionWindow

use of org.apache.kafka.streams.kstream.internals.SessionWindow in project apache-kafka-on-k8s by banzaicloud.

the class CachingSessionStoreTest method shouldClearNamespaceCacheOnClose.

@Test
public void shouldClearNamespaceCacheOnClose() {
    final Windowed<Bytes> a1 = new Windowed<>(keyA, new SessionWindow(0, 0));
    cachingStore.put(a1, "1".getBytes());
    assertEquals(1, cache.size());
    cachingStore.close();
    assertEquals(0, cache.size());
}
Also used : Windowed(org.apache.kafka.streams.kstream.Windowed) Bytes(org.apache.kafka.common.utils.Bytes) SessionWindow(org.apache.kafka.streams.kstream.internals.SessionWindow) Test(org.junit.Test)

Example 62 with SessionWindow

use of org.apache.kafka.streams.kstream.internals.SessionWindow in project apache-kafka-on-k8s by banzaicloud.

the class CachingSessionStoreTest method shouldFetchRangeCorrectlyAcrossSegments.

@Test
public void shouldFetchRangeCorrectlyAcrossSegments() {
    final Windowed<Bytes> a1 = new Windowed<>(keyA, new SessionWindow(0, 0));
    final Windowed<Bytes> aa1 = new Windowed<>(keyAA, new SessionWindow(0, 0));
    final Windowed<Bytes> a2 = new Windowed<>(keyA, new SessionWindow(Segments.MIN_SEGMENT_INTERVAL, Segments.MIN_SEGMENT_INTERVAL));
    final Windowed<Bytes> a3 = new Windowed<>(keyA, new SessionWindow(Segments.MIN_SEGMENT_INTERVAL * 2, Segments.MIN_SEGMENT_INTERVAL * 2));
    final Windowed<Bytes> aa3 = new Windowed<>(keyAA, new SessionWindow(Segments.MIN_SEGMENT_INTERVAL * 2, Segments.MIN_SEGMENT_INTERVAL * 2));
    cachingStore.put(a1, "1".getBytes());
    cachingStore.put(aa1, "1".getBytes());
    cachingStore.put(a2, "2".getBytes());
    cachingStore.put(a3, "3".getBytes());
    cachingStore.put(aa3, "3".getBytes());
    cachingStore.flush();
    final KeyValueIterator<Windowed<Bytes>, byte[]> rangeResults = cachingStore.findSessions(keyA, keyAA, 0, Segments.MIN_SEGMENT_INTERVAL * 2);
    assertEquals(a1, rangeResults.next().key);
    assertEquals(aa1, rangeResults.next().key);
    assertEquals(a2, rangeResults.next().key);
    assertEquals(a3, rangeResults.next().key);
    assertEquals(aa3, rangeResults.next().key);
    assertFalse(rangeResults.hasNext());
}
Also used : Windowed(org.apache.kafka.streams.kstream.Windowed) Bytes(org.apache.kafka.common.utils.Bytes) SessionWindow(org.apache.kafka.streams.kstream.internals.SessionWindow) Test(org.junit.Test)

Example 63 with SessionWindow

use of org.apache.kafka.streams.kstream.internals.SessionWindow in project apache-kafka-on-k8s by banzaicloud.

the class CachingSessionStoreTest method addSingleSession.

private void addSingleSession(final String sessionId, final List<KeyValue<Windowed<Bytes>, byte[]>> allSessions) {
    final int timestamp = allSessions.size() * 10;
    final Windowed<Bytes> key = new Windowed<>(Bytes.wrap(sessionId.getBytes()), new SessionWindow(timestamp, timestamp));
    final byte[] value = "1".getBytes();
    cachingStore.put(key, value);
    allSessions.add(KeyValue.pair(key, value));
}
Also used : Windowed(org.apache.kafka.streams.kstream.Windowed) Bytes(org.apache.kafka.common.utils.Bytes) SessionWindow(org.apache.kafka.streams.kstream.internals.SessionWindow)

Example 64 with SessionWindow

use of org.apache.kafka.streams.kstream.internals.SessionWindow in project apache-kafka-on-k8s by banzaicloud.

the class CompositeReadOnlySessionStoreTest method shouldNotGetValueFromOtherStores.

@Test
public void shouldNotGetValueFromOtherStores() {
    final Windowed<String> expectedKey = new Windowed<>("foo", new SessionWindow(0, 0));
    otherUnderlyingStore.put(new Windowed<>("foo", new SessionWindow(10, 10)), 10L);
    underlyingSessionStore.put(expectedKey, 1L);
    final KeyValueIterator<Windowed<String>, Long> result = sessionStore.fetch("foo");
    assertEquals(KeyValue.pair(expectedKey, 1L), result.next());
    assertFalse(result.hasNext());
}
Also used : Windowed(org.apache.kafka.streams.kstream.Windowed) SessionWindow(org.apache.kafka.streams.kstream.internals.SessionWindow) Test(org.junit.Test)

Example 65 with SessionWindow

use of org.apache.kafka.streams.kstream.internals.SessionWindow in project ksql by confluentinc.

the class SourceBuilderV1Test method shouldAddPseudoColumnsAndSessionWindowedRowKeyColumnsToStream.

@Test
public void shouldAddPseudoColumnsAndSessionWindowedRowKeyColumnsToStream() {
    // Given:
    givenWindowedSourceStream();
    final ValueTransformerWithKey<Windowed<GenericKey>, GenericRow, GenericRow> transformer = getTransformerFromStreamSource(windowedStreamSource);
    final Windowed<GenericKey> key = new Windowed<>(KEY, new SessionWindow(A_WINDOW_START, A_WINDOW_END));
    // When:
    final GenericRow withTimestamp = transformer.transform(key, row);
    // Then:
    assertThat(withTimestamp, equalTo(GenericRow.genericRow("baz", 123, HEADER_DATA, A_ROWTIME, A_ROWPARTITION, A_ROWOFFSET, A_KEY, A_WINDOW_START, A_WINDOW_END)));
}
Also used : Windowed(org.apache.kafka.streams.kstream.Windowed) GenericRow(io.confluent.ksql.GenericRow) GenericKey(io.confluent.ksql.GenericKey) SessionWindow(org.apache.kafka.streams.kstream.internals.SessionWindow) Test(org.junit.Test)

Aggregations

SessionWindow (org.apache.kafka.streams.kstream.internals.SessionWindow)130 Test (org.junit.Test)113 Windowed (org.apache.kafka.streams.kstream.Windowed)112 Bytes (org.apache.kafka.common.utils.Bytes)50 KeyValue (org.apache.kafka.streams.KeyValue)50 ArrayList (java.util.ArrayList)10 Properties (java.util.Properties)9 HashMap (java.util.HashMap)8 CountDownLatch (java.util.concurrent.CountDownLatch)8 StringSerializer (org.apache.kafka.common.serialization.StringSerializer)8 StreamsTestUtils.verifyWindowedKeyValue (org.apache.kafka.test.StreamsTestUtils.verifyWindowedKeyValue)8 LinkedList (java.util.LinkedList)6 ReadOnlySessionStoreStub (org.apache.kafka.test.ReadOnlySessionStoreStub)6 GenericRow (io.confluent.ksql.GenericRow)5 RecordHeaders (org.apache.kafka.common.header.internals.RecordHeaders)4 StringDeserializer (org.apache.kafka.common.serialization.StringDeserializer)4 KeyValueTimestamp (org.apache.kafka.streams.KeyValueTimestamp)4 ProcessorContext (org.apache.kafka.streams.processor.ProcessorContext)4 ProcessorRecordContext (org.apache.kafka.streams.processor.internals.ProcessorRecordContext)4 IntegrationTest (org.apache.kafka.test.IntegrationTest)4