Search in sources :

Example 46 with Configuration

use of org.apache.flink.configuration.Configuration in project flink by apache.

the class RpcConnectionTest method testConnectFailure.

@Test
public void testConnectFailure() {
    ActorSystem actorSystem = null;
    RpcService rpcService = null;
    try {
        actorSystem = AkkaUtils.createActorSystem(new Configuration(), Option.apply(new Tuple2<String, Object>("localhost", 0)));
        // we start the RPC service with a very long timeout to ensure that the test
        // can only pass if the connection problem is not recognized merely via a timeout
        rpcService = new AkkaRpcService(actorSystem, Time.of(10000000, TimeUnit.SECONDS));
        Future<TaskExecutorGateway> future = rpcService.connect("foo.bar.com.test.invalid", TaskExecutorGateway.class);
        future.get(10000000, TimeUnit.SECONDS);
        fail("should never complete normally");
    } catch (TimeoutException e) {
        fail("should not fail with a generic timeout exception");
    } catch (ExecutionException e) {
        // that is what we want
        assertTrue(e.getCause() instanceof RpcConnectionException);
        assertTrue("wrong error message", e.getCause().getMessage().contains("foo.bar.com.test.invalid"));
    } catch (Throwable t) {
        fail("wrong exception: " + t);
    } finally {
        if (rpcService != null) {
            rpcService.stopService();
        }
        if (actorSystem != null) {
            actorSystem.shutdown();
        }
    }
}
Also used : ActorSystem(akka.actor.ActorSystem) Configuration(org.apache.flink.configuration.Configuration) TaskExecutorGateway(org.apache.flink.runtime.taskexecutor.TaskExecutorGateway) RpcConnectionException(org.apache.flink.runtime.rpc.exceptions.RpcConnectionException) AkkaRpcService(org.apache.flink.runtime.rpc.akka.AkkaRpcService) AkkaRpcService(org.apache.flink.runtime.rpc.akka.AkkaRpcService) ExecutionException(java.util.concurrent.ExecutionException) TimeoutException(java.util.concurrent.TimeoutException) Test(org.junit.Test)

Example 47 with Configuration

use of org.apache.flink.configuration.Configuration in project flink by apache.

the class StateBackendLoadingTest method testInstantiateMemoryBackendByDefault.

@Test
public void testInstantiateMemoryBackendByDefault() throws Exception {
    StateBackend backend = AbstractStateBackend.loadStateBackendFromConfigOrCreateDefault(new Configuration(), cl, null);
    assertTrue(backend instanceof MemoryStateBackend);
}
Also used : Configuration(org.apache.flink.configuration.Configuration) MemoryStateBackend(org.apache.flink.runtime.state.memory.MemoryStateBackend) FsStateBackend(org.apache.flink.runtime.state.filesystem.FsStateBackend) MemoryStateBackend(org.apache.flink.runtime.state.memory.MemoryStateBackend) Test(org.junit.Test)

Example 48 with Configuration

use of org.apache.flink.configuration.Configuration in project flink by apache.

the class StateBackendLoadingTest method testLoadingFails.

/**
	 * This test makes sure that failures properly manifest when the state backend could not be loaded.
	 */
@Test
public void testLoadingFails() throws Exception {
    final Configuration config = new Configuration();
    // try a value that is neither recognized as a name, nor corresponds to a class
    config.setString(backendKey, "does.not.exist");
    try {
        AbstractStateBackend.loadStateBackendFromConfigOrCreateDefault(config, cl, null);
        fail("should fail with an exception");
    } catch (DynamicCodeLoadingException ignored) {
    // expected
    }
    // try a class that is not a factory
    config.setString(backendKey, java.io.File.class.getName());
    try {
        AbstractStateBackend.loadStateBackendFromConfigOrCreateDefault(config, cl, null);
        fail("should fail with an exception");
    } catch (DynamicCodeLoadingException ignored) {
    // expected
    }
    // a factory that fails
    config.setString(backendKey, FailingFactory.class.getName());
    try {
        AbstractStateBackend.loadStateBackendFromConfigOrCreateDefault(config, cl, null);
        fail("should fail with an exception");
    } catch (IOException ignored) {
    // expected
    }
}
Also used : DynamicCodeLoadingException(org.apache.flink.util.DynamicCodeLoadingException) Configuration(org.apache.flink.configuration.Configuration) IOException(java.io.IOException) Test(org.junit.Test)

Example 49 with Configuration

use of org.apache.flink.configuration.Configuration in project flink by apache.

the class StateBackendLoadingTest method testLoadMemoryStateBackend.

@Test
public void testLoadMemoryStateBackend() throws Exception {
    // we configure with the explicit string (rather than AbstractStateBackend#X_STATE_BACKEND_NAME)
    // to guard against config-breaking changes of the name 
    final Configuration config = new Configuration();
    config.setString(backendKey, "jobmanager");
    StateBackend backend = AbstractStateBackend.loadStateBackendFromConfigOrCreateDefault(new Configuration(), cl, null);
    assertTrue(backend instanceof MemoryStateBackend);
}
Also used : Configuration(org.apache.flink.configuration.Configuration) MemoryStateBackend(org.apache.flink.runtime.state.memory.MemoryStateBackend) FsStateBackend(org.apache.flink.runtime.state.filesystem.FsStateBackend) MemoryStateBackend(org.apache.flink.runtime.state.memory.MemoryStateBackend) Test(org.junit.Test)

Example 50 with Configuration

use of org.apache.flink.configuration.Configuration in project flink by apache.

the class SecurityUtilsTest method testSecurityContext.

@Test
public void testSecurityContext() throws Exception {
    SecurityUtils.SecurityConfiguration sc = new SecurityUtils.SecurityConfiguration(new Configuration(), new org.apache.hadoop.conf.Configuration(), Collections.singletonList(TestSecurityModule.class));
    SecurityUtils.install(sc);
    assertEquals(HadoopSecurityContext.class, SecurityUtils.getInstalledContext().getClass());
    SecurityUtils.uninstall();
    assertEquals(NoOpSecurityContext.class, SecurityUtils.getInstalledContext().getClass());
}
Also used : Configuration(org.apache.flink.configuration.Configuration) Test(org.junit.Test)

Aggregations

Configuration (org.apache.flink.configuration.Configuration)630 Test (org.junit.Test)452 IOException (java.io.IOException)137 FileInputSplit (org.apache.flink.core.fs.FileInputSplit)93 File (java.io.File)92 JobID (org.apache.flink.api.common.JobID)74 ExecutionConfig (org.apache.flink.api.common.ExecutionConfig)68 JobVertex (org.apache.flink.runtime.jobgraph.JobVertex)49 ActorGateway (org.apache.flink.runtime.instance.ActorGateway)46 JobGraph (org.apache.flink.runtime.jobgraph.JobGraph)45 Path (org.apache.flink.core.fs.Path)44 ActorRef (akka.actor.ActorRef)43 ArrayList (java.util.ArrayList)43 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)39 FiniteDuration (scala.concurrent.duration.FiniteDuration)38 LocalFlinkMiniCluster (org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster)36 BeforeClass (org.junit.BeforeClass)35 AkkaActorGateway (org.apache.flink.runtime.instance.AkkaActorGateway)33 MetricRegistry (org.apache.flink.runtime.metrics.MetricRegistry)33 JobVertexID (org.apache.flink.runtime.jobgraph.JobVertexID)32