use of org.apache.samza.table.descriptors.TableDescriptor in project samza by apache.
the class TestExecutionPlanner method createStreamGraphWithInvalidStreamTableJoin.
private StreamApplicationDescriptorImpl createStreamGraphWithInvalidStreamTableJoin() {
/**
* Example stream-table join that is invalid due to disagreement in partition count
* between the 2 input streams.
*
* input1 (64) -> send-to-table t
*
* join-table t -> output1 (8)
* |
* input2 (16) —————————
*/
return new StreamApplicationDescriptorImpl(appDesc -> {
MessageStream<KV<Object, Object>> messageStream1 = appDesc.getInputStream(input1Descriptor);
MessageStream<KV<Object, Object>> messageStream2 = appDesc.getInputStream(input2Descriptor);
OutputStream<KV<Object, Object>> output1 = appDesc.getOutputStream(output1Descriptor);
TableDescriptor tableDescriptor = new TestLocalTableDescriptor.MockLocalTableDescriptor("table-id", new KVSerde(new StringSerde(), new StringSerde()));
Table table = appDesc.getTable(tableDescriptor);
messageStream1.sendTo(table);
messageStream1.join(table, mock(StreamTableJoinFunction.class)).join(messageStream2, mock(JoinFunction.class), mock(Serde.class), mock(Serde.class), mock(Serde.class), Duration.ofHours(1), "j2").sendTo(output1);
}, config);
}
use of org.apache.samza.table.descriptors.TableDescriptor in project samza by apache.
the class TestExecutionPlanner method createStreamGraphWithStreamTableJoinAndSendToSameTable.
private StreamApplicationDescriptorImpl createStreamGraphWithStreamTableJoinAndSendToSameTable() {
/**
* A special example of stream-table join where a stream is joined with a table, and the result is
* sent to the same table. This example is necessary to ensure {@link ExecutionPlanner} does not
* get stuck traversing the virtual cycle between stream-table-join and send-to-table operator specs
* indefinitely.
*
* The reason such virtual cycle is present is to support computing partitions of intermediate
* streams participating in stream-table joins. Please, refer to SAMZA SEP-16 for more details.
*/
return new StreamApplicationDescriptorImpl(appDesc -> {
MessageStream<KV<Object, Object>> messageStream1 = appDesc.getInputStream(input1Descriptor);
TableDescriptor tableDescriptor = new TestLocalTableDescriptor.MockLocalTableDescriptor("table-id", new KVSerde(new StringSerde(), new StringSerde()));
Table table = appDesc.getTable(tableDescriptor);
messageStream1.join(table, mock(StreamTableJoinFunction.class)).sendTo(table);
}, config);
}
Aggregations