Search in sources :

Example 6 with MaterializedWindowedTable

use of io.confluent.ksql.execution.streams.materialization.MaterializedWindowedTable in project ksql by confluentinc.

the class KsMaterializationFunctionalTest method shouldQueryMaterializedTableForHoppingWindowed.

@Test
public void shouldQueryMaterializedTableForHoppingWindowed() {
    // Given:
    final PersistentQueryMetadata query = executeQuery("CREATE TABLE " + output + " AS" + " SELECT USERID, COUNT(*) AS COUNT FROM " + USER_STREAM + " WINDOW HOPPING (SIZE " + WINDOW_SIZE.getSeconds() + " SECONDS," + " ADVANCE BY " + WINDOW_SIZE.getSeconds() + " SECONDS)" + " GROUP BY USERID;");
    final LogicalSchema schema = schema("COUNT", SqlTypes.BIGINT);
    final Map<Windowed<String>, GenericRow> rows = waitForUniqueUserRows(TIME_WINDOWED_DESERIALIZER, schema);
    // When:
    final Materialization materialization = query.getMaterialization(queryId, contextStacker).get();
    // Then:
    assertThat(materialization.windowType(), is(Optional.of(WindowType.HOPPING)));
    final MaterializedWindowedTable table = materialization.windowed();
    rows.forEach((k, v) -> {
        final Window w = Window.of(k.window().startTime(), k.window().endTime());
        final GenericKey key = genericKey(k.key());
        final List<WindowedRow> resultAtWindowStart = withRetry(() -> Lists.newArrayList(table.get(key, PARTITION, Range.singleton(w.start()), Range.all())));
        assertThat("at exact window start", resultAtWindowStart, hasSize(1));
        assertThat(resultAtWindowStart.get(0).schema(), is(schema));
        assertThat(resultAtWindowStart.get(0).window(), is(Optional.of(w)));
        assertThat(resultAtWindowStart.get(0).key(), is(key));
        assertThat(resultAtWindowStart.get(0).value(), is(v));
        final List<WindowedRow> resultAtWindowEnd = withRetry(() -> Lists.newArrayList(table.get(key, PARTITION, Range.all(), Range.singleton(w.end()))));
        assertThat("at exact window end", resultAtWindowEnd, hasSize(1));
        final List<WindowedRow> resultFromRange = withRetry(() -> Lists.newArrayList(table.get(key, PARTITION, Range.closed(w.start().minusMillis(1), w.start().plusMillis(1)), Range.all())));
        assertThat("range including window start", resultFromRange, is(resultAtWindowStart));
        final List<WindowedRow> resultPast = withRetry(() -> Lists.newArrayList(table.get(key, PARTITION, Range.closed(w.start().plusMillis(1), w.start().plusMillis(1)), Range.all())));
        assertThat("past start", resultPast, is(empty()));
    });
}
Also used : Windowed(org.apache.kafka.streams.kstream.Windowed) GenericRow(io.confluent.ksql.GenericRow) Window(io.confluent.ksql.Window) Materialization(io.confluent.ksql.execution.streams.materialization.Materialization) MaterializedWindowedTable(io.confluent.ksql.execution.streams.materialization.MaterializedWindowedTable) LogicalSchema(io.confluent.ksql.schema.ksql.LogicalSchema) GenericKey(io.confluent.ksql.GenericKey) WindowedRow(io.confluent.ksql.execution.streams.materialization.WindowedRow) PersistentQueryMetadata(io.confluent.ksql.util.PersistentQueryMetadata) IntegrationTest(org.apache.kafka.test.IntegrationTest) Test(org.junit.Test)

Aggregations

Materialization (io.confluent.ksql.execution.streams.materialization.Materialization)6 MaterializedWindowedTable (io.confluent.ksql.execution.streams.materialization.MaterializedWindowedTable)6 PersistentQueryMetadata (io.confluent.ksql.util.PersistentQueryMetadata)6 IntegrationTest (org.apache.kafka.test.IntegrationTest)6 Test (org.junit.Test)6 GenericKey (io.confluent.ksql.GenericKey)3 GenericRow (io.confluent.ksql.GenericRow)3 Window (io.confluent.ksql.Window)3 WindowedRow (io.confluent.ksql.execution.streams.materialization.WindowedRow)3 LogicalSchema (io.confluent.ksql.schema.ksql.LogicalSchema)3 Optional (java.util.Optional)3 ConsumerRecord (org.apache.kafka.clients.consumer.ConsumerRecord)3 Windowed (org.apache.kafka.streams.kstream.Windowed)3