Search in sources :

Example 1 with SlotSharingSlotAllocator

use of org.apache.flink.runtime.scheduler.adaptive.allocator.SlotSharingSlotAllocator in project flink by apache.

the class AdaptiveSchedulerFactory method createInstance.

@Override
public SchedulerNG createInstance(Logger log, JobGraph jobGraph, Executor ioExecutor, Configuration jobMasterConfiguration, SlotPoolService slotPoolService, ScheduledExecutorService futureExecutor, ClassLoader userCodeLoader, CheckpointRecoveryFactory checkpointRecoveryFactory, Time rpcTimeout, BlobWriter blobWriter, JobManagerJobMetricGroup jobManagerJobMetricGroup, Time slotRequestTimeout, ShuffleMaster<?> shuffleMaster, JobMasterPartitionTracker partitionTracker, ExecutionDeploymentTracker executionDeploymentTracker, long initializationTimestamp, ComponentMainThreadExecutor mainThreadExecutor, FatalErrorHandler fatalErrorHandler, JobStatusListener jobStatusListener) throws Exception {
    final DeclarativeSlotPool declarativeSlotPool = slotPoolService.castInto(DeclarativeSlotPool.class).orElseThrow(() -> new IllegalStateException("The AdaptiveScheduler requires a DeclarativeSlotPool."));
    final RestartBackoffTimeStrategy restartBackoffTimeStrategy = RestartBackoffTimeStrategyFactoryLoader.createRestartBackoffTimeStrategyFactory(jobGraph.getSerializedExecutionConfig().deserializeValue(userCodeLoader).getRestartStrategy(), jobMasterConfiguration, jobGraph.isCheckpointingEnabled()).create();
    log.info("Using restart back off time strategy {} for {} ({}).", restartBackoffTimeStrategy, jobGraph.getName(), jobGraph.getJobID());
    final SlotSharingSlotAllocator slotAllocator = createSlotSharingSlotAllocator(declarativeSlotPool);
    final ExecutionGraphFactory executionGraphFactory = new DefaultExecutionGraphFactory(jobMasterConfiguration, userCodeLoader, executionDeploymentTracker, futureExecutor, ioExecutor, rpcTimeout, jobManagerJobMetricGroup, blobWriter, shuffleMaster, partitionTracker);
    return new AdaptiveScheduler(jobGraph, jobMasterConfiguration, declarativeSlotPool, slotAllocator, ioExecutor, userCodeLoader, new CheckpointsCleaner(), checkpointRecoveryFactory, initialResourceAllocationTimeout, resourceStabilizationTimeout, jobManagerJobMetricGroup, restartBackoffTimeStrategy, initializationTimestamp, mainThreadExecutor, fatalErrorHandler, jobStatusListener, executionGraphFactory);
}
Also used : DeclarativeSlotPool(org.apache.flink.runtime.jobmaster.slotpool.DeclarativeSlotPool) RestartBackoffTimeStrategy(org.apache.flink.runtime.executiongraph.failover.flip1.RestartBackoffTimeStrategy) CheckpointsCleaner(org.apache.flink.runtime.checkpoint.CheckpointsCleaner) DefaultExecutionGraphFactory(org.apache.flink.runtime.scheduler.DefaultExecutionGraphFactory) SlotSharingSlotAllocator(org.apache.flink.runtime.scheduler.adaptive.allocator.SlotSharingSlotAllocator) ExecutionGraphFactory(org.apache.flink.runtime.scheduler.ExecutionGraphFactory) DefaultExecutionGraphFactory(org.apache.flink.runtime.scheduler.DefaultExecutionGraphFactory)

Aggregations

CheckpointsCleaner (org.apache.flink.runtime.checkpoint.CheckpointsCleaner)1 RestartBackoffTimeStrategy (org.apache.flink.runtime.executiongraph.failover.flip1.RestartBackoffTimeStrategy)1 DeclarativeSlotPool (org.apache.flink.runtime.jobmaster.slotpool.DeclarativeSlotPool)1 DefaultExecutionGraphFactory (org.apache.flink.runtime.scheduler.DefaultExecutionGraphFactory)1 ExecutionGraphFactory (org.apache.flink.runtime.scheduler.ExecutionGraphFactory)1 SlotSharingSlotAllocator (org.apache.flink.runtime.scheduler.adaptive.allocator.SlotSharingSlotAllocator)1