Search in sources :

Example 1 with KeyedStateReaderOperator

use of org.apache.flink.state.api.input.operator.KeyedStateReaderOperator in project flink by apache.

the class KeyedStateInputFormatTest method readInputSplit.

@Nonnull
private List<Integer> readInputSplit(KeyGroupRangeInputSplit split, KeyedStateReaderFunction<Integer, Integer> userFunction) throws IOException {
    KeyedStateInputFormat<Integer, VoidNamespace, Integer> format = new KeyedStateInputFormat<>(new OperatorState(OperatorIDGenerator.fromUid("uid"), 1, 4), new MemoryStateBackend(), new Configuration(), new KeyedStateReaderOperator<>(userFunction, Types.INT));
    List<Integer> data = new ArrayList<>();
    format.setRuntimeContext(new MockStreamingRuntimeContext(false, 1, 0));
    format.openInputFormat();
    format.open(split);
    while (!format.reachedEnd()) {
        data.add(format.nextRecord(0));
    }
    format.close();
    format.closeInputFormat();
    data.sort(Comparator.comparingInt(id -> id));
    return data;
}
Also used : RichFlatMapFunction(org.apache.flink.api.common.functions.RichFlatMapFunction) VoidNamespace(org.apache.flink.runtime.state.VoidNamespace) Arrays(java.util.Arrays) OperatorState(org.apache.flink.runtime.checkpoint.OperatorState) KeyedOneInputStreamOperatorTestHarness(org.apache.flink.streaming.util.KeyedOneInputStreamOperatorTestHarness) KeyedStateReaderOperator(org.apache.flink.state.api.input.operator.KeyedStateReaderOperator) KeyedProcessFunction(org.apache.flink.streaming.api.functions.KeyedProcessFunction) KeyedStateReaderFunction(org.apache.flink.state.api.functions.KeyedStateReaderFunction) ArrayList(java.util.ArrayList) MockStreamingRuntimeContext(org.apache.flink.streaming.util.MockStreamingRuntimeContext) KeyGroupRangeInputSplit(org.apache.flink.state.api.input.splits.KeyGroupRangeInputSplit) KeyedProcessOperator(org.apache.flink.streaming.api.operators.KeyedProcessOperator) Collector(org.apache.flink.util.Collector) Nonnull(javax.annotation.Nonnull) Types(org.apache.flink.api.common.typeinfo.Types) ValueStateDescriptor(org.apache.flink.api.common.state.ValueStateDescriptor) StreamFlatMap(org.apache.flink.streaming.api.operators.StreamFlatMap) Configuration(org.apache.flink.configuration.Configuration) Set(java.util.Set) Test(org.junit.Test) IOException(java.io.IOException) OperatorSubtaskState(org.apache.flink.runtime.checkpoint.OperatorSubtaskState) MemoryStateBackend(org.apache.flink.runtime.state.memory.MemoryStateBackend) List(java.util.List) ValueState(org.apache.flink.api.common.state.ValueState) VoidSerializer(org.apache.flink.api.common.typeutils.base.VoidSerializer) OperatorID(org.apache.flink.runtime.jobgraph.OperatorID) OneInputStreamOperator(org.apache.flink.streaming.api.operators.OneInputStreamOperator) Assert(org.junit.Assert) Comparator(java.util.Comparator) OperatorIDGenerator(org.apache.flink.state.api.runtime.OperatorIDGenerator) MockStreamingRuntimeContext(org.apache.flink.streaming.util.MockStreamingRuntimeContext) Configuration(org.apache.flink.configuration.Configuration) MemoryStateBackend(org.apache.flink.runtime.state.memory.MemoryStateBackend) ArrayList(java.util.ArrayList) VoidNamespace(org.apache.flink.runtime.state.VoidNamespace) OperatorState(org.apache.flink.runtime.checkpoint.OperatorState) Nonnull(javax.annotation.Nonnull)

Aggregations

IOException (java.io.IOException)1 ArrayList (java.util.ArrayList)1 Arrays (java.util.Arrays)1 Comparator (java.util.Comparator)1 List (java.util.List)1 Set (java.util.Set)1 Nonnull (javax.annotation.Nonnull)1 RichFlatMapFunction (org.apache.flink.api.common.functions.RichFlatMapFunction)1 ValueState (org.apache.flink.api.common.state.ValueState)1 ValueStateDescriptor (org.apache.flink.api.common.state.ValueStateDescriptor)1 Types (org.apache.flink.api.common.typeinfo.Types)1 VoidSerializer (org.apache.flink.api.common.typeutils.base.VoidSerializer)1 Configuration (org.apache.flink.configuration.Configuration)1 OperatorState (org.apache.flink.runtime.checkpoint.OperatorState)1 OperatorSubtaskState (org.apache.flink.runtime.checkpoint.OperatorSubtaskState)1 OperatorID (org.apache.flink.runtime.jobgraph.OperatorID)1 VoidNamespace (org.apache.flink.runtime.state.VoidNamespace)1 MemoryStateBackend (org.apache.flink.runtime.state.memory.MemoryStateBackend)1 KeyedStateReaderFunction (org.apache.flink.state.api.functions.KeyedStateReaderFunction)1 KeyedStateReaderOperator (org.apache.flink.state.api.input.operator.KeyedStateReaderOperator)1