Examples with ResourceOverview org.apache.flink.runtime.resourcemanager.ResourceOverview used on opensource projects

Example 1 with ResourceOverview

use of org.apache.flink.runtime.resourcemanager.ResourceOverview in project flink by apache.

the class MiniClusterResource method cancelAllJobs.

private void cancelAllJobs(boolean waitUntilSlotsAreFreed) {
    try {
        final long shutdownTimeout = miniClusterResourceConfiguration.getShutdownTimeout().toMilliseconds();
        final Deadline jobCancellationDeadline = Deadline.fromNow(Duration.ofMillis(shutdownTimeout));
        final List<CompletableFuture<Acknowledge>> jobCancellationFutures = miniCluster.listJobs().get(jobCancellationDeadline.timeLeft().toMillis(), TimeUnit.MILLISECONDS).stream().filter(status -> !status.getJobState().isGloballyTerminalState()).map(status -> miniCluster.cancelJob(status.getJobId())).collect(Collectors.toList());
        FutureUtils.waitForAll(jobCancellationFutures).get(jobCancellationDeadline.timeLeft().toMillis(), TimeUnit.MILLISECONDS);
        CommonTestUtils.waitUntilCondition(() -> {
            final long unfinishedJobs = miniCluster.listJobs().get(jobCancellationDeadline.timeLeft().toMillis(), TimeUnit.MILLISECONDS).stream().filter(status -> !status.getJobState().isGloballyTerminalState()).count();
            return unfinishedJobs == 0;
        }, jobCancellationDeadline);
        if (waitUntilSlotsAreFreed) {
            CommonTestUtils.waitUntilCondition(() -> {
                final ResourceOverview resourceOverview = miniCluster.getResourceOverview().get();
                return resourceOverview.getNumberRegisteredSlots() == resourceOverview.getNumberFreeSlots();
            }, jobCancellationDeadline);
        }
    } catch (Exception e) {
        log.warn("Exception while shutting down remaining jobs.", e);
    }
}

Example 2 with ResourceOverview

use of org.apache.flink.runtime.resourcemanager.ResourceOverview in project flink by apache.

the class Dispatcher method requestClusterOverview.

@Override
public CompletableFuture<ClusterOverview> requestClusterOverview(Time timeout) {
    CompletableFuture<ResourceOverview> taskManagerOverviewFuture = runResourceManagerCommand(resourceManagerGateway -> resourceManagerGateway.requestResourceOverview(timeout));
    final List<CompletableFuture<Optional<JobStatus>>> optionalJobInformation = queryJobMastersForInformation(jobManagerRunner -> jobManagerRunner.requestJobStatus(timeout));
    CompletableFuture<Collection<Optional<JobStatus>>> allOptionalJobsFuture = FutureUtils.combineAll(optionalJobInformation);
    CompletableFuture<Collection<JobStatus>> allJobsFuture = allOptionalJobsFuture.thenApply(this::flattenOptionalCollection);
    final JobsOverview completedJobsOverview = executionGraphInfoStore.getStoredJobsOverview();
    return allJobsFuture.thenCombine(taskManagerOverviewFuture, (Collection<JobStatus> runningJobsStatus, ResourceOverview resourceOverview) -> {
        final JobsOverview allJobsOverview = JobsOverview.create(runningJobsStatus).combine(completedJobsOverview);
        return new ClusterOverview(resourceOverview, allJobsOverview);
    });
}