Search in sources :

Example 6 with FlinkRuntimeException

use of org.apache.flink.util.FlinkRuntimeException in project flink by apache.

the class AbstractFileSource method createEnumerator.

@Override
public SplitEnumerator<SplitT, PendingSplitsCheckpoint<SplitT>> createEnumerator(SplitEnumeratorContext<SplitT> enumContext) {
    final FileEnumerator enumerator = enumeratorFactory.create();
    // read the initial set of splits (which is also the total set of splits for bounded
    // sources)
    final Collection<FileSourceSplit> splits;
    try {
        // TODO - in the next cleanup pass, we should try to remove the need to "wrap unchecked"
        // here
        splits = enumerator.enumerateSplits(inputPaths, enumContext.currentParallelism());
    } catch (IOException e) {
        throw new FlinkRuntimeException("Could not enumerate file splits", e);
    }
    return createSplitEnumerator(enumContext, enumerator, splits, null);
}
Also used : FileEnumerator(org.apache.flink.connector.file.src.enumerate.FileEnumerator) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) IOException(java.io.IOException)

Example 7 with FlinkRuntimeException

use of org.apache.flink.util.FlinkRuntimeException in project flink by apache.

the class FutureUtilsTest method testStopAtNonRetryableException.

/**
 * Test that {@link FutureUtils#retry} should stop at non-retryable exception.
 */
@Test
public void testStopAtNonRetryableException() {
    final int retries = 10;
    final int notRetry = 3;
    final AtomicInteger atomicInteger = new AtomicInteger(0);
    final FlinkRuntimeException nonRetryableException = new FlinkRuntimeException("Non-retryable exception");
    CompletableFuture<Boolean> retryFuture = FutureUtils.retry(() -> CompletableFuture.supplyAsync(() -> {
        if (atomicInteger.incrementAndGet() == notRetry) {
            // throw non-retryable exception
            throw new CompletionException(nonRetryableException);
        } else {
            throw new CompletionException(new FlinkException("Test exception"));
        }
    }, TestingUtils.defaultExecutor()), retries, throwable -> ExceptionUtils.findThrowable(throwable, FlinkException.class).isPresent(), TestingUtils.defaultExecutor());
    try {
        retryFuture.get();
        fail("Exception should be thrown.");
    } catch (Exception ex) {
        assertThat(ex, FlinkMatchers.containsCause(nonRetryableException));
    }
    assertThat(atomicInteger.get(), is(notRetry));
}
Also used : AtomicInteger(java.util.concurrent.atomic.AtomicInteger) CompletionException(java.util.concurrent.CompletionException) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) FlinkException(org.apache.flink.util.FlinkException) FlinkException(org.apache.flink.util.FlinkException) TimeoutException(java.util.concurrent.TimeoutException) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) CompletionException(java.util.concurrent.CompletionException) ExecutionException(java.util.concurrent.ExecutionException) Test(org.junit.Test)

Example 8 with FlinkRuntimeException

use of org.apache.flink.util.FlinkRuntimeException in project flink by apache.

the class JobDispatcherLeaderProcessFactoryFactory method createFactory.

@Override
public JobDispatcherLeaderProcessFactory createFactory(JobPersistenceComponentFactory jobPersistenceComponentFactory, Executor ioExecutor, RpcService rpcService, PartialDispatcherServices partialDispatcherServices, FatalErrorHandler fatalErrorHandler) {
    final JobGraph jobGraph;
    try {
        jobGraph = Preconditions.checkNotNull(jobGraphRetriever.retrieveJobGraph(partialDispatcherServices.getConfiguration()));
    } catch (FlinkException e) {
        throw new FlinkRuntimeException("Could not retrieve the JobGraph.", e);
    }
    final JobResultStore jobResultStore = jobPersistenceComponentFactory.createJobResultStore();
    final Collection<JobResult> recoveredDirtyJobResults = getDirtyJobResults(jobResultStore);
    final Optional<JobResult> maybeRecoveredDirtyJobResult = extractDirtyJobResult(recoveredDirtyJobResults, jobGraph);
    final Optional<JobGraph> maybeJobGraph = getJobGraphBasedOnDirtyJobResults(jobGraph, recoveredDirtyJobResults);
    final DefaultDispatcherGatewayServiceFactory defaultDispatcherServiceFactory = new DefaultDispatcherGatewayServiceFactory(JobDispatcherFactory.INSTANCE, rpcService, partialDispatcherServices);
    return new JobDispatcherLeaderProcessFactory(defaultDispatcherServiceFactory, maybeJobGraph.orElse(null), maybeRecoveredDirtyJobResult.orElse(null), jobResultStore, fatalErrorHandler);
}
Also used : JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) JobResult(org.apache.flink.runtime.jobmaster.JobResult) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) JobResultStore(org.apache.flink.runtime.highavailability.JobResultStore) FlinkException(org.apache.flink.util.FlinkException)

Example 9 with FlinkRuntimeException

use of org.apache.flink.util.FlinkRuntimeException in project flink by apache.

the class NFA method findFinalStateAfterProceed.

private State<T> findFinalStateAfterProceed(ConditionContext context, State<T> state, T event) {
    final Stack<State<T>> statesToCheck = new Stack<>();
    statesToCheck.push(state);
    try {
        while (!statesToCheck.isEmpty()) {
            final State<T> currentState = statesToCheck.pop();
            for (StateTransition<T> transition : currentState.getStateTransitions()) {
                if (transition.getAction() == StateTransitionAction.PROCEED && checkFilterCondition(context, transition.getCondition(), event)) {
                    if (transition.getTargetState().isFinal()) {
                        return transition.getTargetState();
                    } else {
                        statesToCheck.push(transition.getTargetState());
                    }
                }
            }
        }
    } catch (Exception e) {
        throw new FlinkRuntimeException("Failure happened in filter function.", e);
    }
    return null;
}
Also used : FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) IOException(java.io.IOException) Stack(java.util.Stack)

Example 10 with FlinkRuntimeException

use of org.apache.flink.util.FlinkRuntimeException in project flink by apache.

the class ApplicationDispatcherGatewayServiceFactory method create.

@Override
public AbstractDispatcherLeaderProcess.DispatcherGatewayService create(DispatcherId fencingToken, Collection<JobGraph> recoveredJobs, Collection<JobResult> recoveredDirtyJobResults, JobGraphWriter jobGraphWriter, JobResultStore jobResultStore) {
    final List<JobID> recoveredJobIds = getRecoveredJobIds(recoveredJobs);
    final Dispatcher dispatcher;
    try {
        dispatcher = dispatcherFactory.createDispatcher(rpcService, fencingToken, recoveredJobs, recoveredDirtyJobResults, (dispatcherGateway, scheduledExecutor, errorHandler) -> new ApplicationDispatcherBootstrap(application, recoveredJobIds, configuration, dispatcherGateway, scheduledExecutor, errorHandler), PartialDispatcherServicesWithJobPersistenceComponents.from(partialDispatcherServices, jobGraphWriter, jobResultStore));
    } catch (Exception e) {
        throw new FlinkRuntimeException("Could not create the Dispatcher rpc endpoint.", e);
    }
    dispatcher.start();
    return DefaultDispatcherGatewayService.from(dispatcher);
}
Also used : DispatcherId(org.apache.flink.runtime.dispatcher.DispatcherId) Dispatcher(org.apache.flink.runtime.dispatcher.Dispatcher) PartialDispatcherServices(org.apache.flink.runtime.dispatcher.PartialDispatcherServices) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) PartialDispatcherServicesWithJobPersistenceComponents(org.apache.flink.runtime.dispatcher.PartialDispatcherServicesWithJobPersistenceComponents) Collection(java.util.Collection) Configuration(org.apache.flink.configuration.Configuration) JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) AbstractDispatcherLeaderProcess(org.apache.flink.runtime.dispatcher.runner.AbstractDispatcherLeaderProcess) Collectors(java.util.stream.Collectors) JobResult(org.apache.flink.runtime.jobmaster.JobResult) List(java.util.List) JobID(org.apache.flink.api.common.JobID) RpcService(org.apache.flink.runtime.rpc.RpcService) Internal(org.apache.flink.annotation.Internal) PackagedProgram(org.apache.flink.client.program.PackagedProgram) DispatcherFactory(org.apache.flink.runtime.dispatcher.DispatcherFactory) JobResultStore(org.apache.flink.runtime.highavailability.JobResultStore) Preconditions.checkNotNull(org.apache.flink.util.Preconditions.checkNotNull) JobGraphWriter(org.apache.flink.runtime.jobmanager.JobGraphWriter) DefaultDispatcherGatewayService(org.apache.flink.runtime.dispatcher.runner.DefaultDispatcherGatewayService) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) Dispatcher(org.apache.flink.runtime.dispatcher.Dispatcher) JobID(org.apache.flink.api.common.JobID) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException)

Aggregations

FlinkRuntimeException (org.apache.flink.util.FlinkRuntimeException)78 IOException (java.io.IOException)28 Test (org.junit.Test)13 JobID (org.apache.flink.api.common.JobID)10 HashMap (java.util.HashMap)8 ArrayList (java.util.ArrayList)7 CompletableFuture (java.util.concurrent.CompletableFuture)7 ExecutionException (java.util.concurrent.ExecutionException)7 Nonnull (javax.annotation.Nonnull)7 Configuration (org.apache.flink.configuration.Configuration)6 Collectors (java.util.stream.Collectors)5 JobGraph (org.apache.flink.runtime.jobgraph.JobGraph)5 JobResultStore (org.apache.flink.runtime.highavailability.JobResultStore)4 RocksDBException (org.rocksdb.RocksDBException)4 List (java.util.List)3 Map (java.util.Map)3 CheckpointMetrics (org.apache.flink.runtime.checkpoint.CheckpointMetrics)3 TaskStateSnapshot (org.apache.flink.runtime.checkpoint.TaskStateSnapshot)3 ExecutionAttemptID (org.apache.flink.runtime.executiongraph.ExecutionAttemptID)3 JobResult (org.apache.flink.runtime.jobmaster.JobResult)3