Search in sources :

Example 1 with Dispatcher

use of org.apache.flink.runtime.dispatcher.Dispatcher in project flink by apache.

the class ApplicationDispatcherGatewayServiceFactory method create.

@Override
public AbstractDispatcherLeaderProcess.DispatcherGatewayService create(DispatcherId fencingToken, Collection<JobGraph> recoveredJobs, Collection<JobResult> recoveredDirtyJobResults, JobGraphWriter jobGraphWriter, JobResultStore jobResultStore) {
    final List<JobID> recoveredJobIds = getRecoveredJobIds(recoveredJobs);
    final Dispatcher dispatcher;
    try {
        dispatcher = dispatcherFactory.createDispatcher(rpcService, fencingToken, recoveredJobs, recoveredDirtyJobResults, (dispatcherGateway, scheduledExecutor, errorHandler) -> new ApplicationDispatcherBootstrap(application, recoveredJobIds, configuration, dispatcherGateway, scheduledExecutor, errorHandler), PartialDispatcherServicesWithJobPersistenceComponents.from(partialDispatcherServices, jobGraphWriter, jobResultStore));
    } catch (Exception e) {
        throw new FlinkRuntimeException("Could not create the Dispatcher rpc endpoint.", e);
    }
    dispatcher.start();
    return DefaultDispatcherGatewayService.from(dispatcher);
}
Also used : DispatcherId(org.apache.flink.runtime.dispatcher.DispatcherId) Dispatcher(org.apache.flink.runtime.dispatcher.Dispatcher) PartialDispatcherServices(org.apache.flink.runtime.dispatcher.PartialDispatcherServices) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) PartialDispatcherServicesWithJobPersistenceComponents(org.apache.flink.runtime.dispatcher.PartialDispatcherServicesWithJobPersistenceComponents) Collection(java.util.Collection) Configuration(org.apache.flink.configuration.Configuration) JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) AbstractDispatcherLeaderProcess(org.apache.flink.runtime.dispatcher.runner.AbstractDispatcherLeaderProcess) Collectors(java.util.stream.Collectors) JobResult(org.apache.flink.runtime.jobmaster.JobResult) List(java.util.List) JobID(org.apache.flink.api.common.JobID) RpcService(org.apache.flink.runtime.rpc.RpcService) Internal(org.apache.flink.annotation.Internal) PackagedProgram(org.apache.flink.client.program.PackagedProgram) DispatcherFactory(org.apache.flink.runtime.dispatcher.DispatcherFactory) JobResultStore(org.apache.flink.runtime.highavailability.JobResultStore) Preconditions.checkNotNull(org.apache.flink.util.Preconditions.checkNotNull) JobGraphWriter(org.apache.flink.runtime.jobmanager.JobGraphWriter) DefaultDispatcherGatewayService(org.apache.flink.runtime.dispatcher.runner.DefaultDispatcherGatewayService) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) Dispatcher(org.apache.flink.runtime.dispatcher.Dispatcher) JobID(org.apache.flink.api.common.JobID) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException)

Example 2 with Dispatcher

use of org.apache.flink.runtime.dispatcher.Dispatcher in project flink by apache.

the class DefaultDispatcherGatewayServiceFactory method create.

@Override
public AbstractDispatcherLeaderProcess.DispatcherGatewayService create(DispatcherId fencingToken, Collection<JobGraph> recoveredJobs, Collection<JobResult> recoveredDirtyJobResults, JobGraphWriter jobGraphWriter, JobResultStore jobResultStore) {
    final Dispatcher dispatcher;
    try {
        dispatcher = dispatcherFactory.createDispatcher(rpcService, fencingToken, recoveredJobs, recoveredDirtyJobResults, (dispatcherGateway, scheduledExecutor, errorHandler) -> new NoOpDispatcherBootstrap(), PartialDispatcherServicesWithJobPersistenceComponents.from(partialDispatcherServices, jobGraphWriter, jobResultStore));
    } catch (Exception e) {
        throw new FlinkRuntimeException("Could not create the Dispatcher rpc endpoint.", e);
    }
    dispatcher.start();
    return DefaultDispatcherGatewayService.from(dispatcher);
}
Also used : DispatcherId(org.apache.flink.runtime.dispatcher.DispatcherId) Dispatcher(org.apache.flink.runtime.dispatcher.Dispatcher) PartialDispatcherServices(org.apache.flink.runtime.dispatcher.PartialDispatcherServices) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) PartialDispatcherServicesWithJobPersistenceComponents(org.apache.flink.runtime.dispatcher.PartialDispatcherServicesWithJobPersistenceComponents) Collection(java.util.Collection) JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) JobResult(org.apache.flink.runtime.jobmaster.JobResult) RpcService(org.apache.flink.runtime.rpc.RpcService) NoOpDispatcherBootstrap(org.apache.flink.runtime.dispatcher.NoOpDispatcherBootstrap) DispatcherFactory(org.apache.flink.runtime.dispatcher.DispatcherFactory) JobResultStore(org.apache.flink.runtime.highavailability.JobResultStore) JobGraphWriter(org.apache.flink.runtime.jobmanager.JobGraphWriter) NoOpDispatcherBootstrap(org.apache.flink.runtime.dispatcher.NoOpDispatcherBootstrap) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) Dispatcher(org.apache.flink.runtime.dispatcher.Dispatcher) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException)

Aggregations

Collection (java.util.Collection)2 Dispatcher (org.apache.flink.runtime.dispatcher.Dispatcher)2 DispatcherFactory (org.apache.flink.runtime.dispatcher.DispatcherFactory)2 DispatcherId (org.apache.flink.runtime.dispatcher.DispatcherId)2 PartialDispatcherServices (org.apache.flink.runtime.dispatcher.PartialDispatcherServices)2 PartialDispatcherServicesWithJobPersistenceComponents (org.apache.flink.runtime.dispatcher.PartialDispatcherServicesWithJobPersistenceComponents)2 JobResultStore (org.apache.flink.runtime.highavailability.JobResultStore)2 JobGraph (org.apache.flink.runtime.jobgraph.JobGraph)2 JobGraphWriter (org.apache.flink.runtime.jobmanager.JobGraphWriter)2 JobResult (org.apache.flink.runtime.jobmaster.JobResult)2 RpcService (org.apache.flink.runtime.rpc.RpcService)2 FlinkRuntimeException (org.apache.flink.util.FlinkRuntimeException)2 List (java.util.List)1 Collectors (java.util.stream.Collectors)1 Internal (org.apache.flink.annotation.Internal)1 JobID (org.apache.flink.api.common.JobID)1 PackagedProgram (org.apache.flink.client.program.PackagedProgram)1 Configuration (org.apache.flink.configuration.Configuration)1 NoOpDispatcherBootstrap (org.apache.flink.runtime.dispatcher.NoOpDispatcherBootstrap)1 AbstractDispatcherLeaderProcess (org.apache.flink.runtime.dispatcher.runner.AbstractDispatcherLeaderProcess)1