Search in sources :

Example 6 with Position

use of org.apache.kafka.streams.query.Position in project kafka by apache.

the class ChangeLoggingKeyValueBytesStoreTest method shouldLogPositionOnPut.

@Test
public void shouldLogPositionOnPut() {
    context.setRecordContext(new ProcessorRecordContext(-1, INPUT_OFFSET, INPUT_PARTITION, INPUT_TOPIC_NAME, new RecordHeaders()));
    context.setTime(1L);
    store.put(hi, there);
    assertThat(collector.collected().size(), equalTo(1));
    assertThat(collector.collected().get(0).headers(), is(notNullValue()));
    final Header versionHeader = collector.collected().get(0).headers().lastHeader(ChangelogRecordDeserializationHelper.CHANGELOG_VERSION_HEADER_KEY);
    assertThat(versionHeader, is(notNullValue()));
    assertThat(versionHeader.equals(ChangelogRecordDeserializationHelper.CHANGELOG_VERSION_HEADER_RECORD_CONSISTENCY), is(true));
    final Header vectorHeader = collector.collected().get(0).headers().lastHeader(ChangelogRecordDeserializationHelper.CHANGELOG_POSITION_HEADER_KEY);
    assertThat(vectorHeader, is(notNullValue()));
    final Position position = PositionSerde.deserialize(ByteBuffer.wrap(vectorHeader.value()));
    assertThat(position.getPartitionPositions(INPUT_TOPIC_NAME), is(notNullValue()));
    assertThat(position.getPartitionPositions(INPUT_TOPIC_NAME), hasEntry(0, 100L));
}
Also used : RecordHeaders(org.apache.kafka.common.header.internals.RecordHeaders) ProcessorRecordContext(org.apache.kafka.streams.processor.internals.ProcessorRecordContext) Header(org.apache.kafka.common.header.Header) Position(org.apache.kafka.streams.query.Position) Test(org.junit.Test)

Example 7 with Position

use of org.apache.kafka.streams.query.Position in project kafka by apache.

the class ChangeLoggingWindowBytesStoreTest method shouldLogPutsWithPosition.

@Test
public void shouldLogPutsWithPosition() {
    EasyMock.expect(inner.getPosition()).andReturn(POSITION).anyTimes();
    inner.put(bytesKey, value, 0);
    EasyMock.expectLastCall();
    init();
    final Bytes key = WindowKeySchema.toStoreKeyBinary(bytesKey, 0, 0);
    EasyMock.reset(context);
    final RecordMetadata recordContext = new ProcessorRecordContext(0L, 1L, 0, "", new RecordHeaders());
    EasyMock.expect(context.recordMetadata()).andStubReturn(Optional.of(recordContext));
    EasyMock.expect(context.timestamp()).andStubReturn(0L);
    final Position position = Position.fromMap(mkMap(mkEntry("", mkMap(mkEntry(0, 1L)))));
    context.logChange(store.name(), key, value, 0L, position);
    EasyMock.replay(context);
    store.put(bytesKey, value, context.timestamp());
    EasyMock.verify(inner, context);
}
Also used : RecordMetadata(org.apache.kafka.streams.processor.api.RecordMetadata) Bytes(org.apache.kafka.common.utils.Bytes) RecordHeaders(org.apache.kafka.common.header.internals.RecordHeaders) ProcessorRecordContext(org.apache.kafka.streams.processor.internals.ProcessorRecordContext) Position(org.apache.kafka.streams.query.Position) Test(org.junit.Test)

Example 8 with Position

use of org.apache.kafka.streams.query.Position in project kafka by apache.

the class InMemoryWindowStoreTest method shouldMatchPositionAfterPut.

@Test
public void shouldMatchPositionAfterPut() {
    final MeteredWindowStore<Integer, String> meteredSessionStore = (MeteredWindowStore<Integer, String>) windowStore;
    final ChangeLoggingWindowBytesStore changeLoggingSessionBytesStore = (ChangeLoggingWindowBytesStore) meteredSessionStore.wrapped();
    final InMemoryWindowStore inMemoryWindowStore = (InMemoryWindowStore) changeLoggingSessionBytesStore.wrapped();
    context.setRecordContext(new ProcessorRecordContext(0, 1, 0, "", new RecordHeaders()));
    windowStore.put(0, "0", SEGMENT_INTERVAL);
    context.setRecordContext(new ProcessorRecordContext(0, 2, 0, "", new RecordHeaders()));
    windowStore.put(1, "1", SEGMENT_INTERVAL);
    context.setRecordContext(new ProcessorRecordContext(0, 3, 0, "", new RecordHeaders()));
    windowStore.put(2, "2", SEGMENT_INTERVAL);
    context.setRecordContext(new ProcessorRecordContext(0, 4, 0, "", new RecordHeaders()));
    windowStore.put(3, "3", SEGMENT_INTERVAL);
    final Position expected = Position.fromMap(mkMap(mkEntry("", mkMap(mkEntry(0, 4L)))));
    final Position actual = inMemoryWindowStore.getPosition();
    assertEquals(expected, actual);
}
Also used : RecordHeaders(org.apache.kafka.common.header.internals.RecordHeaders) ProcessorRecordContext(org.apache.kafka.streams.processor.internals.ProcessorRecordContext) Position(org.apache.kafka.streams.query.Position) Test(org.junit.Test)

Example 9 with Position

use of org.apache.kafka.streams.query.Position in project kafka by apache.

the class ChangeLoggingTimestampedWindowBytesStoreTest method shouldLogPutsWithPosition.

@Test
public void shouldLogPutsWithPosition() {
    EasyMock.expect(inner.getPosition()).andReturn(POSITION).anyTimes();
    inner.put(bytesKey, valueAndTimestamp, 0);
    EasyMock.expectLastCall();
    init();
    final Bytes key = WindowKeySchema.toStoreKeyBinary(bytesKey, 0, 0);
    EasyMock.reset(context);
    final RecordMetadata recordContext = new ProcessorRecordContext(0L, 1L, 0, "", new RecordHeaders());
    EasyMock.expect(context.recordMetadata()).andStubReturn(Optional.of(recordContext));
    final Position position = Position.fromMap(mkMap(mkEntry("", mkMap(mkEntry(0, 1L)))));
    context.logChange(store.name(), key, value, 42, position);
    EasyMock.replay(context);
    store.put(bytesKey, valueAndTimestamp, context.timestamp());
    EasyMock.verify(inner, context);
}
Also used : RecordMetadata(org.apache.kafka.streams.processor.api.RecordMetadata) Bytes(org.apache.kafka.common.utils.Bytes) RecordHeaders(org.apache.kafka.common.header.internals.RecordHeaders) ProcessorRecordContext(org.apache.kafka.streams.processor.internals.ProcessorRecordContext) Position(org.apache.kafka.streams.query.Position) Test(org.junit.Test)

Example 10 with Position

use of org.apache.kafka.streams.query.Position in project ksql by confluentinc.

the class KsMaterializedTableIQv2Test method getRowResult.

private static StateQueryResult getRowResult(final GenericRow row) {
    final StateQueryResult result = new StateQueryResult<>();
    final QueryResult queryResult = QueryResult.forResult(ValueAndTimestamp.make(row, -1));
    final Position position = Position.emptyPosition();
    position.withComponent(TOPIC, PARTITION, OFFSET);
    queryResult.setPosition(position);
    result.addResult(PARTITION, queryResult);
    return result;
}
Also used : QueryResult(org.apache.kafka.streams.query.QueryResult) StateQueryResult(org.apache.kafka.streams.query.StateQueryResult) Position(org.apache.kafka.streams.query.Position) StateQueryResult(org.apache.kafka.streams.query.StateQueryResult)

Aggregations

Position (org.apache.kafka.streams.query.Position)33 RecordHeaders (org.apache.kafka.common.header.internals.RecordHeaders)17 Test (org.junit.Test)17 ProcessorRecordContext (org.apache.kafka.streams.processor.internals.ProcessorRecordContext)11 Windowed (org.apache.kafka.streams.kstream.Windowed)9 Objects (java.util.Objects)7 ConsumerRecord (org.apache.kafka.clients.consumer.ConsumerRecord)7 Bytes (org.apache.kafka.common.utils.Bytes)7 QueryResult (org.apache.kafka.streams.query.QueryResult)7 KeyValueIterator (org.apache.kafka.streams.state.KeyValueIterator)7 Headers (org.apache.kafka.common.header.Headers)6 PositionBound (org.apache.kafka.streams.query.PositionBound)6 RecordHeader (org.apache.kafka.common.header.internals.RecordHeader)5 KeyValue (org.apache.kafka.streams.KeyValue)5 ValueAndTimestamp (org.apache.kafka.streams.state.ValueAndTimestamp)5 ImmutableList (com.google.common.collect.ImmutableList)4 Streams (com.google.common.collect.Streams)4 GenericKey (io.confluent.ksql.GenericKey)4 GenericRow (io.confluent.ksql.GenericRow)4 MaterializationException (io.confluent.ksql.execution.streams.materialization.MaterializationException)4