Search in sources :

Example 6 with VertexParallelismStore

use of org.apache.flink.runtime.scheduler.VertexParallelismStore in project flink by apache.

the class AdaptiveSchedulerTest method testComputeVertexParallelismStoreForExecutionInReactiveMode.

@Test
public void testComputeVertexParallelismStoreForExecutionInReactiveMode() {
    JobVertex v1 = createNoOpVertex("v1", 1, 50);
    JobVertex v2 = createNoOpVertex("v2", 50, 50);
    JobGraph graph = streamingJobGraph(v1, v2);
    VertexParallelismStore parallelismStore = AdaptiveScheduler.computeVertexParallelismStoreForExecution(graph, SchedulerExecutionMode.REACTIVE, SchedulerBase::getDefaultMaxParallelism);
    for (JobVertex vertex : graph.getVertices()) {
        VertexParallelismInformation info = parallelismStore.getParallelismInfo(vertex.getID());
        assertThat(info.getParallelism()).isEqualTo(vertex.getParallelism());
        assertThat(info.getMaxParallelism()).isEqualTo(vertex.getMaxParallelism());
    }
}
Also used : VertexParallelismInformation(org.apache.flink.runtime.scheduler.VertexParallelismInformation) JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) JobGraphTestUtils.streamingJobGraph(org.apache.flink.runtime.jobgraph.JobGraphTestUtils.streamingJobGraph) ArchivedExecutionJobVertex(org.apache.flink.runtime.executiongraph.ArchivedExecutionJobVertex) JobVertex(org.apache.flink.runtime.jobgraph.JobVertex) VertexParallelismStore(org.apache.flink.runtime.scheduler.VertexParallelismStore) SchedulerBase(org.apache.flink.runtime.scheduler.SchedulerBase) Test(org.junit.Test) ArchivedExecutionGraphTest(org.apache.flink.runtime.executiongraph.ArchivedExecutionGraphTest) DefaultSchedulerTest(org.apache.flink.runtime.scheduler.DefaultSchedulerTest)

Example 7 with VertexParallelismStore

use of org.apache.flink.runtime.scheduler.VertexParallelismStore in project flink by apache.

the class AdaptiveSchedulerTest method testComputeVertexParallelismStoreForExecutionInDefaultMode.

@Test
public void testComputeVertexParallelismStoreForExecutionInDefaultMode() {
    JobVertex v1 = createNoOpVertex("v1", 1, 50);
    JobVertex v2 = createNoOpVertex("v2", 50, 50);
    JobGraph graph = streamingJobGraph(v1, v2);
    VertexParallelismStore parallelismStore = AdaptiveScheduler.computeVertexParallelismStoreForExecution(graph, null, SchedulerBase::getDefaultMaxParallelism);
    for (JobVertex vertex : graph.getVertices()) {
        VertexParallelismInformation info = parallelismStore.getParallelismInfo(vertex.getID());
        assertThat(info.getParallelism()).isEqualTo(vertex.getParallelism());
        assertThat(info.getMaxParallelism()).isEqualTo(vertex.getMaxParallelism());
    }
}
Also used : VertexParallelismInformation(org.apache.flink.runtime.scheduler.VertexParallelismInformation) JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) JobGraphTestUtils.streamingJobGraph(org.apache.flink.runtime.jobgraph.JobGraphTestUtils.streamingJobGraph) ArchivedExecutionJobVertex(org.apache.flink.runtime.executiongraph.ArchivedExecutionJobVertex) JobVertex(org.apache.flink.runtime.jobgraph.JobVertex) VertexParallelismStore(org.apache.flink.runtime.scheduler.VertexParallelismStore) SchedulerBase(org.apache.flink.runtime.scheduler.SchedulerBase) Test(org.junit.Test) ArchivedExecutionGraphTest(org.apache.flink.runtime.executiongraph.ArchivedExecutionGraphTest) DefaultSchedulerTest(org.apache.flink.runtime.scheduler.DefaultSchedulerTest)

Aggregations

VertexParallelismStore (org.apache.flink.runtime.scheduler.VertexParallelismStore)7 JobVertex (org.apache.flink.runtime.jobgraph.JobVertex)6 VertexParallelismInformation (org.apache.flink.runtime.scheduler.VertexParallelismInformation)6 SchedulerBase (org.apache.flink.runtime.scheduler.SchedulerBase)5 JobGraph (org.apache.flink.runtime.jobgraph.JobGraph)4 Test (org.junit.Test)3 ArchivedExecutionGraphTest (org.apache.flink.runtime.executiongraph.ArchivedExecutionGraphTest)2 ArchivedExecutionJobVertex (org.apache.flink.runtime.executiongraph.ArchivedExecutionJobVertex)2 IntermediateDataSetID (org.apache.flink.runtime.jobgraph.IntermediateDataSetID)2 JobGraphTestUtils.streamingJobGraph (org.apache.flink.runtime.jobgraph.JobGraphTestUtils.streamingJobGraph)2 DefaultSchedulerTest (org.apache.flink.runtime.scheduler.DefaultSchedulerTest)2 IOException (java.io.IOException)1 InetSocketAddress (java.net.InetSocketAddress)1 Duration (java.time.Duration)1 ArrayList (java.util.ArrayList)1 Collection (java.util.Collection)1 Collections (java.util.Collections)1 Iterator (java.util.Iterator)1 List (java.util.List)1 Optional (java.util.Optional)1