Search in sources :

Example 6 with KeyGroupsList

use of org.apache.flink.runtime.state.KeyGroupsList in project beam by apache.

the class DedupingOperator method initializeState.

@Override
public void initializeState(StateInitializationContext context) throws Exception {
    if (getKeyedStateBackend() != null) {
        KeyGroupsList localKeyGroupRange = getKeyedStateBackend().getKeyGroupRange();
        for (KeyGroupStatePartitionStreamProvider streamProvider : context.getRawKeyedStateInputs()) {
            DataInputViewStreamWrapper div = new DataInputViewStreamWrapper(streamProvider.getStream());
            int keyGroupIdx = streamProvider.getKeyGroupId();
            checkArgument(localKeyGroupRange.contains(keyGroupIdx), "Key Group " + keyGroupIdx + " does not belong to the local range.");
            // if (this instanceof KeyGroupRestoringOperator)
            restoreKeyGroupState(keyGroupIdx, div);
        }
    }
}
Also used : KeyGroupsList(org.apache.flink.runtime.state.KeyGroupsList) DataInputViewStreamWrapper(org.apache.flink.core.memory.DataInputViewStreamWrapper) KeyGroupStatePartitionStreamProvider(org.apache.flink.runtime.state.KeyGroupStatePartitionStreamProvider)

Example 7 with KeyGroupsList

use of org.apache.flink.runtime.state.KeyGroupsList in project flink by apache.

the class AbstractStreamOperator method initializeState.

/**
	 * Stream operators with state which can be restored need to override this hook method.
	 *
	 * @param context context that allows to register different states.
	 */
public void initializeState(StateInitializationContext context) throws Exception {
    if (getKeyedStateBackend() != null) {
        KeyGroupsList localKeyGroupRange = getKeyedStateBackend().getKeyGroupRange();
        // and then initialize the timer services
        for (KeyGroupStatePartitionStreamProvider streamProvider : context.getRawKeyedStateInputs()) {
            int keyGroupIdx = streamProvider.getKeyGroupId();
            checkArgument(localKeyGroupRange.contains(keyGroupIdx), "Key Group " + keyGroupIdx + " does not belong to the local range.");
            timeServiceManager.restoreStateForKeyGroup(new DataInputViewStreamWrapper(streamProvider.getStream()), keyGroupIdx, getUserCodeClassloader());
        }
    }
}
Also used : KeyGroupsList(org.apache.flink.runtime.state.KeyGroupsList) DataInputViewStreamWrapper(org.apache.flink.core.memory.DataInputViewStreamWrapper) KeyGroupStatePartitionStreamProvider(org.apache.flink.runtime.state.KeyGroupStatePartitionStreamProvider)

Aggregations

KeyGroupsList (org.apache.flink.runtime.state.KeyGroupsList)7 DataOutputViewStreamWrapper (org.apache.flink.core.memory.DataOutputViewStreamWrapper)4 KeyedStateCheckpointOutputStream (org.apache.flink.runtime.state.KeyedStateCheckpointOutputStream)3 IOException (java.io.IOException)2 DataInputViewStreamWrapper (org.apache.flink.core.memory.DataInputViewStreamWrapper)2 KeyGroupStatePartitionStreamProvider (org.apache.flink.runtime.state.KeyGroupStatePartitionStreamProvider)2 ConcurrentModificationException (java.util.ConcurrentModificationException)1 ExecutionException (java.util.concurrent.ExecutionException)1 KeyGroupRange (org.apache.flink.runtime.state.KeyGroupRange)1 TestProcessingTimeService (org.apache.flink.streaming.runtime.tasks.TestProcessingTimeService)1 Test (org.junit.Test)1