Search in sources :

Example 26 with ArchivedExecutionGraphBuilder

use of org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder in project flink by apache.

the class ExecutionGraphInfoStoreTestUtils method generateTerminalExecutionGraphInfos.

/**
 * Generate a specified of ExecutionGraphInfo.
 *
 * @param number the given number
 * @return the result ExecutionGraphInfo collection
 */
static Collection<ExecutionGraphInfo> generateTerminalExecutionGraphInfos(int number) {
    final Collection<ExecutionGraphInfo> executionGraphInfos = new ArrayList<>(number);
    for (int i = 0; i < number; i++) {
        final JobStatus state = GLOBALLY_TERMINAL_JOB_STATUS.get(ThreadLocalRandom.current().nextInt(GLOBALLY_TERMINAL_JOB_STATUS.size()));
        executionGraphInfos.add(new ExecutionGraphInfo(new ArchivedExecutionGraphBuilder().setState(state).build()));
    }
    return executionGraphInfos;
}
Also used : JobStatus(org.apache.flink.api.common.JobStatus) ExecutionGraphInfo(org.apache.flink.runtime.scheduler.ExecutionGraphInfo) ArrayList(java.util.ArrayList) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder)

Example 27 with ArchivedExecutionGraphBuilder

use of org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder in project flink by apache.

the class MemoryExecutionGraphInfoStoreTest method assertPutJobGraphWithStatus.

private void assertPutJobGraphWithStatus(JobStatus jobStatus) throws IOException {
    final ExecutionGraphInfo dummyExecutionGraphInfo = new ExecutionGraphInfo(new ArchivedExecutionGraphBuilder().setState(jobStatus).build());
    try (final MemoryExecutionGraphInfoStore executionGraphStore = createMemoryExecutionGraphInfoStore()) {
        // check that the graph store is empty
        assertThat(executionGraphStore.size(), Matchers.equalTo(0));
        executionGraphStore.put(dummyExecutionGraphInfo);
        // check that we have persisted the given execution graph
        assertThat(executionGraphStore.size(), Matchers.equalTo(1));
        assertThat(executionGraphStore.get(dummyExecutionGraphInfo.getJobId()), new ExecutionGraphInfoStoreTestUtils.PartialExecutionGraphInfoMatcher(dummyExecutionGraphInfo));
    }
}
Also used : ExecutionGraphInfo(org.apache.flink.runtime.scheduler.ExecutionGraphInfo) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder)

Example 28 with ArchivedExecutionGraphBuilder

use of org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder in project flink by apache.

the class MemoryExecutionGraphInfoStoreTest method testExecutionGraphExpiration.

/**
 * Tests that an expired execution graph is removed from the execution graph store.
 */
@Test
public void testExecutionGraphExpiration() throws Exception {
    final Time expirationTime = Time.milliseconds(1L);
    final ManuallyTriggeredScheduledExecutor scheduledExecutor = new ManuallyTriggeredScheduledExecutor();
    final ManualTicker manualTicker = new ManualTicker();
    try (final MemoryExecutionGraphInfoStore executionGraphInfoStore = new MemoryExecutionGraphInfoStore(expirationTime, Integer.MAX_VALUE, scheduledExecutor, manualTicker)) {
        final ExecutionGraphInfo executionGraphInfo = new ExecutionGraphInfo(new ArchivedExecutionGraphBuilder().setState(JobStatus.FINISHED).build());
        executionGraphInfoStore.put(executionGraphInfo);
        // there should one execution graph
        assertThat(executionGraphInfoStore.size(), Matchers.equalTo(1));
        manualTicker.advanceTime(expirationTime.toMilliseconds(), TimeUnit.MILLISECONDS);
        // this should trigger the cleanup after expiration
        scheduledExecutor.triggerScheduledTasks();
        assertThat(executionGraphInfoStore.size(), Matchers.equalTo(0));
        assertThat(executionGraphInfoStore.get(executionGraphInfo.getJobId()), Matchers.nullValue());
        // check that the store is empty
        assertThat(executionGraphInfoStore.size(), Matchers.equalTo(0));
    }
}
Also used : ExecutionGraphInfo(org.apache.flink.runtime.scheduler.ExecutionGraphInfo) Time(org.apache.flink.api.common.time.Time) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder) ManualTicker(org.apache.flink.runtime.util.ManualTicker) ManuallyTriggeredScheduledExecutor(org.apache.flink.util.concurrent.ManuallyTriggeredScheduledExecutor) Test(org.junit.Test)

Example 29 with ArchivedExecutionGraphBuilder

use of org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder in project flink by apache.

the class AdaptiveSchedulerTest method testGoToFinished.

@Test
public void testGoToFinished() throws Exception {
    final AdaptiveScheduler scheduler = new AdaptiveSchedulerBuilder(createJobGraph(), mainThreadExecutor).build();
    final ArchivedExecutionGraph archivedExecutionGraph = new ArchivedExecutionGraphBuilder().setState(JobStatus.FAILED).build();
    scheduler.goToFinished(archivedExecutionGraph);
    assertThat(scheduler.getState()).isInstanceOf(Finished.class);
}
Also used : ArchivedExecutionGraph(org.apache.flink.runtime.executiongraph.ArchivedExecutionGraph) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder) Test(org.junit.Test) ArchivedExecutionGraphTest(org.apache.flink.runtime.executiongraph.ArchivedExecutionGraphTest) DefaultSchedulerTest(org.apache.flink.runtime.scheduler.DefaultSchedulerTest)

Aggregations

ArchivedExecutionGraphBuilder (org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder)29 ExecutionGraphInfo (org.apache.flink.runtime.scheduler.ExecutionGraphInfo)23 Test (org.junit.Test)22 JobID (org.apache.flink.api.common.JobID)12 CompletableFuture (java.util.concurrent.CompletableFuture)8 Time (org.apache.flink.api.common.time.Time)6 ErrorInfo (org.apache.flink.runtime.executiongraph.ErrorInfo)6 FlinkException (org.apache.flink.util.FlinkException)6 JobStatus (org.apache.flink.api.common.JobStatus)5 TestingJobManagerRunner (org.apache.flink.runtime.jobmaster.TestingJobManagerRunner)5 File (java.io.File)4 ArrayList (java.util.ArrayList)4 ExecutionException (java.util.concurrent.ExecutionException)4 ArchivedExecutionGraph (org.apache.flink.runtime.executiongraph.ArchivedExecutionGraph)4 ArrayDeque (java.util.ArrayDeque)3 Arrays (java.util.Arrays)3 TimeUnit (java.util.concurrent.TimeUnit)3 Configuration (org.apache.flink.configuration.Configuration)3 BlobServer (org.apache.flink.runtime.blob.BlobServer)3 IOException (java.io.IOException)2