Search in sources :

Example 16 with SerializedThrowable

use of org.apache.flink.util.SerializedThrowable in project flink by apache.

the class SavepointWriterWindowITCase method submitJob.

private void submitJob(String savepointPath, StreamExecutionEnvironment sEnv) {
    JobGraph jobGraph = sEnv.getStreamGraph().getJobGraph();
    jobGraph.setSavepointRestoreSettings(SavepointRestoreSettings.forPath(savepointPath, true));
    ClusterClient<?> client = MINI_CLUSTER_RESOURCE.getClusterClient();
    try {
        Optional<SerializedThrowable> serializedThrowable = client.submitJob(jobGraph).thenCompose(client::requestJobResult).get().getSerializedThrowable();
        Assert.assertFalse(serializedThrowable.isPresent());
    } catch (Throwable t) {
        Assert.fail("Failed to submit job");
    }
}
Also used : JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) SerializedThrowable(org.apache.flink.util.SerializedThrowable) SerializedThrowable(org.apache.flink.util.SerializedThrowable)

Example 17 with SerializedThrowable

use of org.apache.flink.util.SerializedThrowable in project flink by apache.

the class WritableSavepointITCase method validateBootstrap.

private void validateBootstrap(StateBackend backend, String savepointPath) throws Exception {
    StreamExecutionEnvironment sEnv = StreamExecutionEnvironment.getExecutionEnvironment();
    sEnv.setStateBackend(backend);
    DataStream<Account> stream = sEnv.fromCollection(accounts).keyBy(acc -> acc.id).flatMap(new UpdateAndGetAccount()).uid(ACCOUNT_UID);
    CompletableFuture<Collection<Account>> results = collector.collect(stream);
    sEnv.fromCollection(currencyRates).connect(sEnv.fromCollection(currencyRates).broadcast(descriptor)).process(new CurrencyValidationFunction()).uid(CURRENCY_UID).addSink(new DiscardingSink<>());
    JobGraph jobGraph = sEnv.getStreamGraph().getJobGraph();
    jobGraph.setSavepointRestoreSettings(SavepointRestoreSettings.forPath(savepointPath, false));
    ClusterClient<?> client = MINI_CLUSTER_RESOURCE.getClusterClient();
    Optional<SerializedThrowable> serializedThrowable = client.submitJob(jobGraph).thenCompose(client::requestJobResult).get().getSerializedThrowable();
    serializedThrowable.ifPresent(t -> {
        throw new AssertionError("Unexpected exception during bootstrapping", t);
    });
    Assert.assertEquals("Unexpected output", 3, results.get().size());
}
Also used : JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) Collection(java.util.Collection) StreamExecutionEnvironment(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment) SerializedThrowable(org.apache.flink.util.SerializedThrowable)

Example 18 with SerializedThrowable

use of org.apache.flink.util.SerializedThrowable in project flink by apache.

the class WritableSavepointWindowITCase method submitJob.

private void submitJob(String savepointPath, StreamExecutionEnvironment sEnv) {
    JobGraph jobGraph = sEnv.getStreamGraph().getJobGraph();
    jobGraph.setSavepointRestoreSettings(SavepointRestoreSettings.forPath(savepointPath, true));
    ClusterClient<?> client = MINI_CLUSTER_RESOURCE.getClusterClient();
    try {
        Optional<SerializedThrowable> serializedThrowable = client.submitJob(jobGraph).thenCompose(client::requestJobResult).get().getSerializedThrowable();
        Assert.assertFalse(serializedThrowable.isPresent());
    } catch (Throwable t) {
        Assert.fail("Failed to submit job");
    }
}
Also used : JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) SerializedThrowable(org.apache.flink.util.SerializedThrowable) SerializedThrowable(org.apache.flink.util.SerializedThrowable)

Example 19 with SerializedThrowable

use of org.apache.flink.util.SerializedThrowable in project flink by apache.

the class SavepointWriterITCase method validateBootstrap.

private void validateBootstrap(StateBackend backend, String savepointPath) throws Exception {
    StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
    if (backend != null) {
        env.setStateBackend(backend);
    }
    DataStream<Account> stream = env.fromCollection(accounts).keyBy(acc -> acc.id).flatMap(new UpdateAndGetAccount()).uid(ACCOUNT_UID);
    CompletableFuture<Collection<Account>> results = collector.collect(stream);
    env.fromCollection(currencyRates).connect(env.fromCollection(currencyRates).broadcast(descriptor)).process(new CurrencyValidationFunction()).uid(CURRENCY_UID).addSink(new DiscardingSink<>());
    JobGraph jobGraph = env.getStreamGraph().getJobGraph();
    jobGraph.setSavepointRestoreSettings(SavepointRestoreSettings.forPath(savepointPath, false));
    ClusterClient<?> client = MINI_CLUSTER_RESOURCE.getClusterClient();
    Optional<SerializedThrowable> serializedThrowable = client.submitJob(jobGraph).thenCompose(client::requestJobResult).get().getSerializedThrowable();
    serializedThrowable.ifPresent(t -> {
        throw new AssertionError("Unexpected exception during bootstrapping", t);
    });
    Assert.assertEquals("Unexpected output", 3, results.get().size());
}
Also used : JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) Collection(java.util.Collection) StreamExecutionEnvironment(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment) SerializedThrowable(org.apache.flink.util.SerializedThrowable)

Example 20 with SerializedThrowable

use of org.apache.flink.util.SerializedThrowable in project flink by apache.

the class DefaultJobMasterServiceProcessTest method assertInitializationException.

private static void assertInitializationException(SerializedThrowable actualException, Throwable expectedCause, long actualTimestamp, long expectedLowerTimestampThreshold, long expectedUpperTimestampThreshold) {
    final Throwable deserializedException = actualException.deserializeError(Thread.currentThread().getContextClassLoader());
    assertThat(deserializedException).isInstanceOf(JobInitializationException.class).hasCause(expectedCause);
    assertThat(actualTimestamp).isGreaterThanOrEqualTo(expectedLowerTimestampThreshold).isLessThanOrEqualTo(expectedUpperTimestampThreshold);
}
Also used : SerializedThrowable(org.apache.flink.util.SerializedThrowable) JobInitializationException(org.apache.flink.runtime.client.JobInitializationException)

Aggregations

SerializedThrowable (org.apache.flink.util.SerializedThrowable)30 Test (org.junit.Test)16 JobGraph (org.apache.flink.runtime.jobgraph.JobGraph)6 FlinkException (org.apache.flink.util.FlinkException)6 ExecutionException (java.util.concurrent.ExecutionException)5 Collection (java.util.Collection)4 StreamExecutionEnvironment (org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)4 Optional (java.util.Optional)3 CompletableFuture (java.util.concurrent.CompletableFuture)3 UnavailableDispatcherOperationException (org.apache.flink.runtime.dispatcher.UnavailableDispatcherOperationException)3 JobResult (org.apache.flink.runtime.jobmaster.JobResult)3 ArrayList (java.util.ArrayList)2 Arrays (java.util.Arrays)2 HashSet (java.util.HashSet)2 List (java.util.List)2 Objects (java.util.Objects)2 Set (java.util.Set)2 RichFlatMapFunction (org.apache.flink.api.common.functions.RichFlatMapFunction)2 RichMapFunction (org.apache.flink.api.common.functions.RichMapFunction)2 ListState (org.apache.flink.api.common.state.ListState)2