Search in sources :

Example 1 with StateChangelogHandleReader

use of org.apache.flink.runtime.state.changelog.StateChangelogHandleReader in project flink by apache.

the class StateChangelogStorageTest method extract.

private Map<Integer, List<byte[]>> extract(T handle, StateChangelogHandleReader<T> reader) throws Exception {
    Map<Integer, List<byte[]>> changes = new HashMap<>();
    try (CloseableIterator<StateChange> it = reader.getChanges(handle)) {
        while (it.hasNext()) {
            StateChange change = it.next();
            changes.computeIfAbsent(change.getKeyGroup(), k -> new ArrayList<>()).add(change.getChange());
        }
    }
    return changes;
}
Also used : StateChangelogStorage(org.apache.flink.runtime.state.changelog.StateChangelogStorage) HashMap(java.util.HashMap) Random(java.util.Random) ArrayList(java.util.ArrayList) ChangelogStateHandle(org.apache.flink.runtime.state.changelog.ChangelogStateHandle) Collectors.toMap(java.util.stream.Collectors.toMap) Map(java.util.Map) Assert.assertArrayEquals(org.junit.Assert.assertArrayEquals) StateChangelogHandleReader(org.apache.flink.runtime.state.changelog.StateChangelogHandleReader) KeyGroupRange(org.apache.flink.runtime.state.KeyGroupRange) Iterator(java.util.Iterator) StateChange(org.apache.flink.runtime.state.changelog.StateChange) Test(org.junit.Test) IOException(java.io.IOException) Collectors(java.util.stream.Collectors) CloseableIterator(org.apache.flink.util.CloseableIterator) List(java.util.List) Rule(org.junit.Rule) Stream(java.util.stream.Stream) StreamSupport.stream(java.util.stream.StreamSupport.stream) Assert.assertFalse(org.junit.Assert.assertFalse) Function.identity(java.util.function.Function.identity) OperatorID(org.apache.flink.runtime.jobgraph.OperatorID) SequenceNumber(org.apache.flink.runtime.state.changelog.SequenceNumber) StateChangelogWriter(org.apache.flink.runtime.state.changelog.StateChangelogWriter) TemporaryFolder(org.junit.rules.TemporaryFolder) Assert.assertEquals(org.junit.Assert.assertEquals) HashMap(java.util.HashMap) StateChange(org.apache.flink.runtime.state.changelog.StateChange) ArrayList(java.util.ArrayList) ArrayList(java.util.ArrayList) List(java.util.List)

Aggregations

IOException (java.io.IOException)1 ArrayList (java.util.ArrayList)1 HashMap (java.util.HashMap)1 Iterator (java.util.Iterator)1 List (java.util.List)1 Map (java.util.Map)1 Random (java.util.Random)1 Function.identity (java.util.function.Function.identity)1 Collectors (java.util.stream.Collectors)1 Collectors.toMap (java.util.stream.Collectors.toMap)1 Stream (java.util.stream.Stream)1 StreamSupport.stream (java.util.stream.StreamSupport.stream)1 OperatorID (org.apache.flink.runtime.jobgraph.OperatorID)1 KeyGroupRange (org.apache.flink.runtime.state.KeyGroupRange)1 ChangelogStateHandle (org.apache.flink.runtime.state.changelog.ChangelogStateHandle)1 SequenceNumber (org.apache.flink.runtime.state.changelog.SequenceNumber)1 StateChange (org.apache.flink.runtime.state.changelog.StateChange)1 StateChangelogHandleReader (org.apache.flink.runtime.state.changelog.StateChangelogHandleReader)1 StateChangelogStorage (org.apache.flink.runtime.state.changelog.StateChangelogStorage)1 StateChangelogWriter (org.apache.flink.runtime.state.changelog.StateChangelogWriter)1