Search in sources :

Example 1 with ClusterDescriptor

use of org.apache.flink.client.deployment.ClusterDescriptor in project flink by apache.

the class AbstractSessionClusterExecutor method execute.

@Override
public CompletableFuture<JobClient> execute(@Nonnull final Pipeline pipeline, @Nonnull final Configuration configuration, @Nonnull final ClassLoader userCodeClassloader) throws Exception {
    final JobGraph jobGraph = PipelineExecutorUtils.getJobGraph(pipeline, configuration);
    try (final ClusterDescriptor<ClusterID> clusterDescriptor = clusterClientFactory.createClusterDescriptor(configuration)) {
        final ClusterID clusterID = clusterClientFactory.getClusterId(configuration);
        checkState(clusterID != null);
        final ClusterClientProvider<ClusterID> clusterClientProvider = clusterDescriptor.retrieve(clusterID);
        ClusterClient<ClusterID> clusterClient = clusterClientProvider.getClusterClient();
        return clusterClient.submitJob(jobGraph).thenApplyAsync(FunctionUtils.uncheckedFunction(jobId -> {
            ClientUtils.waitUntilJobInitializationFinished(() -> clusterClient.getJobStatus(jobId).get(), () -> clusterClient.requestJobResult(jobId).get(), userCodeClassloader);
            return jobId;
        })).thenApplyAsync(jobID -> (JobClient) new ClusterClientJobClientAdapter<>(clusterClientProvider, jobID, userCodeClassloader)).whenCompleteAsync((ignored1, ignored2) -> clusterClient.close());
    }
}
Also used : Preconditions.checkState(org.apache.flink.util.Preconditions.checkState) ClientUtils(org.apache.flink.client.ClientUtils) ClusterClientFactory(org.apache.flink.client.deployment.ClusterClientFactory) Pipeline(org.apache.flink.api.dag.Pipeline) Configuration(org.apache.flink.configuration.Configuration) JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) CompletableFuture(java.util.concurrent.CompletableFuture) PipelineExecutor(org.apache.flink.core.execution.PipelineExecutor) JobClient(org.apache.flink.core.execution.JobClient) ClusterClientJobClientAdapter(org.apache.flink.client.deployment.ClusterClientJobClientAdapter) ClusterDescriptor(org.apache.flink.client.deployment.ClusterDescriptor) ClusterClient(org.apache.flink.client.program.ClusterClient) FunctionUtils(org.apache.flink.util.function.FunctionUtils) Internal(org.apache.flink.annotation.Internal) ClusterClientProvider(org.apache.flink.client.program.ClusterClientProvider) Preconditions.checkNotNull(org.apache.flink.util.Preconditions.checkNotNull) Nonnull(javax.annotation.Nonnull) JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) JobClient(org.apache.flink.core.execution.JobClient)

Aggregations

CompletableFuture (java.util.concurrent.CompletableFuture)1 Nonnull (javax.annotation.Nonnull)1 Internal (org.apache.flink.annotation.Internal)1 Pipeline (org.apache.flink.api.dag.Pipeline)1 ClientUtils (org.apache.flink.client.ClientUtils)1 ClusterClientFactory (org.apache.flink.client.deployment.ClusterClientFactory)1 ClusterClientJobClientAdapter (org.apache.flink.client.deployment.ClusterClientJobClientAdapter)1 ClusterDescriptor (org.apache.flink.client.deployment.ClusterDescriptor)1 ClusterClient (org.apache.flink.client.program.ClusterClient)1 ClusterClientProvider (org.apache.flink.client.program.ClusterClientProvider)1 Configuration (org.apache.flink.configuration.Configuration)1 JobClient (org.apache.flink.core.execution.JobClient)1 PipelineExecutor (org.apache.flink.core.execution.PipelineExecutor)1 JobGraph (org.apache.flink.runtime.jobgraph.JobGraph)1 Preconditions.checkNotNull (org.apache.flink.util.Preconditions.checkNotNull)1 Preconditions.checkState (org.apache.flink.util.Preconditions.checkState)1 FunctionUtils (org.apache.flink.util.function.FunctionUtils)1