Search in sources :

Example 56 with SchedulerBase

use of org.apache.flink.runtime.scheduler.SchedulerBase in project flink by apache.

the class AdaptiveBatchSchedulerTest method testDecideParallelismForForwardTarget.

@Test
public void testDecideParallelismForForwardTarget() throws Exception {
    JobGraph jobGraph = createJobGraph(true);
    Iterator<JobVertex> jobVertexIterator = jobGraph.getVertices().iterator();
    JobVertex source1 = jobVertexIterator.next();
    JobVertex source2 = jobVertexIterator.next();
    JobVertex sink = jobVertexIterator.next();
    SchedulerBase scheduler = createScheduler(jobGraph);
    final DefaultExecutionGraph graph = (DefaultExecutionGraph) scheduler.getExecutionGraph();
    final ExecutionJobVertex sinkExecutionJobVertex = graph.getJobVertex(sink.getID());
    scheduler.startScheduling();
    assertThat(sinkExecutionJobVertex.getParallelism(), is(-1));
    // trigger source1 finished.
    transitionExecutionsState(scheduler, ExecutionState.FINISHED, source1);
    assertThat(sinkExecutionJobVertex.getParallelism(), is(-1));
    // trigger source2 finished.
    transitionExecutionsState(scheduler, ExecutionState.FINISHED, source2);
    assertThat(sinkExecutionJobVertex.getParallelism(), is(SOURCE_PARALLELISM_1));
    // check that the jobGraph is updated
    assertThat(sink.getParallelism(), is(SOURCE_PARALLELISM_1));
}
Also used : JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) JobVertex(org.apache.flink.runtime.jobgraph.JobVertex) ExecutionJobVertex(org.apache.flink.runtime.executiongraph.ExecutionJobVertex) ExecutionJobVertex(org.apache.flink.runtime.executiongraph.ExecutionJobVertex) SchedulerBase(org.apache.flink.runtime.scheduler.SchedulerBase) DefaultExecutionGraph(org.apache.flink.runtime.executiongraph.DefaultExecutionGraph) Test(org.junit.Test)

Aggregations

SchedulerBase (org.apache.flink.runtime.scheduler.SchedulerBase)56 Test (org.junit.Test)49 JobVertex (org.apache.flink.runtime.jobgraph.JobVertex)33 JobGraph (org.apache.flink.runtime.jobgraph.JobGraph)19 JobVertexID (org.apache.flink.runtime.jobgraph.JobVertexID)10 CompletableFuture (java.util.concurrent.CompletableFuture)8 IOException (java.io.IOException)7 TestingPhysicalSlotProvider (org.apache.flink.runtime.scheduler.TestingPhysicalSlotProvider)7 TestRestartBackoffTimeStrategy (org.apache.flink.runtime.executiongraph.failover.flip1.TestRestartBackoffTimeStrategy)6 TaskExecutionState (org.apache.flink.runtime.taskmanager.TaskExecutionState)6 ArrayList (java.util.ArrayList)5 JobStatus (org.apache.flink.api.common.JobStatus)5 IntermediateResultPartitionID (org.apache.flink.runtime.jobgraph.IntermediateResultPartitionID)5 ExecutionGraph (org.apache.flink.runtime.executiongraph.ExecutionGraph)4 ResultPartitionID (org.apache.flink.runtime.io.network.partition.ResultPartitionID)4 SlotPool (org.apache.flink.runtime.jobmaster.slotpool.SlotPool)4 TestingPhysicalSlot (org.apache.flink.runtime.scheduler.TestingPhysicalSlot)4 VertexParallelismInformation (org.apache.flink.runtime.scheduler.VertexParallelismInformation)4 VertexParallelismStore (org.apache.flink.runtime.scheduler.VertexParallelismStore)4 TaskManagerLocation (org.apache.flink.runtime.taskmanager.TaskManagerLocation)4