Search in sources :

Example 16 with FlinkRuntimeException

use of org.apache.flink.util.FlinkRuntimeException in project flink by apache.

the class HadoopRecoverableFsDataOutputStream method ensureTruncateInitialized.

private static void ensureTruncateInitialized() throws FlinkRuntimeException {
    if (HadoopUtils.isMinHadoopVersion(2, 7) && truncateHandle == null) {
        Method truncateMethod;
        try {
            truncateMethod = FileSystem.class.getMethod("truncate", Path.class, long.class);
        } catch (NoSuchMethodException e) {
            throw new FlinkRuntimeException("Could not find a public truncate method on the Hadoop File System.");
        }
        if (!Modifier.isPublic(truncateMethod.getModifiers())) {
            throw new FlinkRuntimeException("Could not find a public truncate method on the Hadoop File System.");
        }
        truncateHandle = truncateMethod;
    }
}
Also used : Path(org.apache.hadoop.fs.Path) DistributedFileSystem(org.apache.hadoop.hdfs.DistributedFileSystem) FileSystem(org.apache.hadoop.fs.FileSystem) ViewFileSystem(org.apache.hadoop.fs.viewfs.ViewFileSystem) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) Method(java.lang.reflect.Method)

Example 17 with FlinkRuntimeException

use of org.apache.flink.util.FlinkRuntimeException in project flink by apache.

the class VertexFinishedStateChecker method checkPredecessorsOfPartiallyFinishedVertex.

private void checkPredecessorsOfPartiallyFinishedVertex(ExecutionJobVertex vertex, VerticesFinishedStatusCache verticesFinishedStatusCache) {
    // Computes the distribution pattern from each predecessor. If there are multiple edges
    // from a single predecessor, ALL_TO_ALL edges would have a higher priority since it
    // implies stricter limitation (must be fully finished).
    Map<JobVertexID, DistributionPattern> predecessorDistribution = new HashMap<>();
    for (JobEdge jobEdge : vertex.getJobVertex().getInputs()) {
        predecessorDistribution.compute(jobEdge.getSource().getProducer().getID(), (k, v) -> v == DistributionPattern.ALL_TO_ALL ? v : jobEdge.getDistributionPattern());
    }
    for (IntermediateResult dataset : vertex.getInputs()) {
        ExecutionJobVertex predecessor = dataset.getProducer();
        VertexFinishedState predecessorState = verticesFinishedStatusCache.getOrUpdate(predecessor);
        DistributionPattern distribution = predecessorDistribution.get(predecessor.getJobVertexId());
        if (distribution == DistributionPattern.ALL_TO_ALL && predecessorState != VertexFinishedState.FULLY_FINISHED) {
            throw new FlinkRuntimeException("Illegal JobGraph modification. Cannot run a program with partially finished" + " vertices predeceased with running or partially finished ones and" + " connected via the ALL_TO_ALL edges. Task vertex " + vertex.getName() + "(" + vertex.getJobVertexId() + ")" + " has a " + (predecessorState == VertexFinishedState.ALL_RUNNING ? "all running" : "partially finished") + " predecessor");
        } else if (distribution == DistributionPattern.POINTWISE && predecessorState == VertexFinishedState.ALL_RUNNING) {
            throw new FlinkRuntimeException("Illegal JobGraph modification. Cannot run a program with partially finished" + " vertices predeceased with all running ones. Task vertex " + vertex.getName() + "(" + vertex.getJobVertexId() + ")" + " has a all running predecessor");
        }
    }
}
Also used : IntermediateResult(org.apache.flink.runtime.executiongraph.IntermediateResult) HashMap(java.util.HashMap) ExecutionJobVertex(org.apache.flink.runtime.executiongraph.ExecutionJobVertex) JobEdge(org.apache.flink.runtime.jobgraph.JobEdge) JobVertexID(org.apache.flink.runtime.jobgraph.JobVertexID) DistributionPattern(org.apache.flink.runtime.jobgraph.DistributionPattern) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException)

Example 18 with FlinkRuntimeException

use of org.apache.flink.util.FlinkRuntimeException in project flink by apache.

the class FileRegionBuffer method getNioBufferReadable.

/**
 * This method is only called by tests and by event-deserialization, like checkpoint barriers.
 * Because such events are not used for bounded intermediate results, this method currently
 * executes only in tests.
 */
@Override
public ByteBuffer getNioBufferReadable() {
    try {
        final ByteBuffer buffer = ByteBuffer.allocateDirect(bufferSize());
        BufferReaderWriterUtil.readByteBufferFully(fileChannel, buffer, position());
        buffer.flip();
        return buffer;
    } catch (IOException e) {
        // to declare IOExceptions, as would be necessary for a proper "lazy buffer".
        throw new FlinkRuntimeException(e.getMessage(), e);
    }
}
Also used : FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) IOException(java.io.IOException) ByteBuffer(java.nio.ByteBuffer)

Example 19 with FlinkRuntimeException

use of org.apache.flink.util.FlinkRuntimeException in project flink by apache.

the class XaFacadeImpl method execute.

private <T> T execute(Command<T> cmd) throws FlinkRuntimeException {
    Preconditions.checkState(isOpen(), "not connected");
    LOG.debug("{}, xid={}", cmd.name, cmd.xid);
    try {
        T result = cmd.callable.call();
        LOG.trace("{} succeeded , xid={}", cmd.name, cmd.xid);
        return result;
    } catch (XAException e) {
        if (HEUR_ERR_CODES.contains(e.errorCode)) {
            cmd.xid.ifPresent(this::forget);
        }
        return cmd.recover.apply(e).orElseThrow(() -> wrapException(cmd.name, cmd.xid, e));
    } catch (FlinkRuntimeException e) {
        throw e;
    } catch (Exception e) {
        throw wrapException(cmd.name, cmd.xid, e);
    }
}
Also used : XAException(javax.transaction.xa.XAException) XA_RBTRANSIENT(javax.transaction.xa.XAException.XA_RBTRANSIENT) XA_RBTIMEOUT(javax.transaction.xa.XAException.XA_RBTIMEOUT) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) SQLException(java.sql.SQLException) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) XAException(javax.transaction.xa.XAException)

Example 20 with FlinkRuntimeException

use of org.apache.flink.util.FlinkRuntimeException in project flink by apache.

the class KubernetesMultipleComponentLeaderElectionHaServices method getOrInitializeSingleLeaderElectionService.

private DefaultMultipleComponentLeaderElectionService getOrInitializeSingleLeaderElectionService() {
    synchronized (lock) {
        if (multipleComponentLeaderElectionService == null) {
            try {
                final KubernetesLeaderElectionConfiguration leaderElectionConfiguration = new KubernetesLeaderElectionConfiguration(getClusterConfigMap(), lockIdentity, configuration);
                multipleComponentLeaderElectionService = new DefaultMultipleComponentLeaderElectionService(fatalErrorHandler, new KubernetesMultipleComponentLeaderElectionDriverFactory(kubeClient, leaderElectionConfiguration, configMapSharedWatcher, watchExecutorService, fatalErrorHandler));
            } catch (Exception e) {
                throw new FlinkRuntimeException("Could not initialize the default single leader election service.", e);
            }
        }
        return multipleComponentLeaderElectionService;
    }
}
Also used : FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) KubernetesLeaderElectionConfiguration(org.apache.flink.kubernetes.configuration.KubernetesLeaderElectionConfiguration) DefaultMultipleComponentLeaderElectionService(org.apache.flink.runtime.leaderelection.DefaultMultipleComponentLeaderElectionService) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) IOException(java.io.IOException)

Aggregations

FlinkRuntimeException (org.apache.flink.util.FlinkRuntimeException)78 IOException (java.io.IOException)28 Test (org.junit.Test)13 JobID (org.apache.flink.api.common.JobID)10 HashMap (java.util.HashMap)8 ArrayList (java.util.ArrayList)7 CompletableFuture (java.util.concurrent.CompletableFuture)7 ExecutionException (java.util.concurrent.ExecutionException)7 Nonnull (javax.annotation.Nonnull)7 Configuration (org.apache.flink.configuration.Configuration)6 Collectors (java.util.stream.Collectors)5 JobGraph (org.apache.flink.runtime.jobgraph.JobGraph)5 JobResultStore (org.apache.flink.runtime.highavailability.JobResultStore)4 RocksDBException (org.rocksdb.RocksDBException)4 List (java.util.List)3 Map (java.util.Map)3 CheckpointMetrics (org.apache.flink.runtime.checkpoint.CheckpointMetrics)3 TaskStateSnapshot (org.apache.flink.runtime.checkpoint.TaskStateSnapshot)3 ExecutionAttemptID (org.apache.flink.runtime.executiongraph.ExecutionAttemptID)3 JobResult (org.apache.flink.runtime.jobmaster.JobResult)3