Search in sources :

Example 46 with ActorSystem

use of akka.actor.ActorSystem in project flink by apache.

the class WebRuntimeMonitorITCase method testLeaderNotAvailable.

@Test
public void testLeaderNotAvailable() throws Exception {
    final Deadline deadline = TestTimeout.fromNow();
    ActorSystem actorSystem = null;
    WebRuntimeMonitor webRuntimeMonitor = null;
    try (TestingServer zooKeeper = new TestingServer()) {
        File logDir = temporaryFolder.newFolder();
        Path logFile = Files.createFile(new File(logDir, "jobmanager.log").toPath());
        Files.createFile(new File(logDir, "jobmanager.out").toPath());
        final Configuration config = new Configuration();
        config.setInteger(ConfigConstants.JOB_MANAGER_WEB_PORT_KEY, 0);
        config.setString(ConfigConstants.JOB_MANAGER_WEB_LOG_PATH_KEY, logFile.toString());
        config.setString(HighAvailabilityOptions.HA_MODE, "ZOOKEEPER");
        config.setString(HighAvailabilityOptions.HA_ZOOKEEPER_QUORUM, zooKeeper.getConnectString());
        actorSystem = AkkaUtils.createDefaultActorSystem();
        LeaderRetrievalService leaderRetrievalService = mock(LeaderRetrievalService.class);
        webRuntimeMonitor = new WebRuntimeMonitor(config, leaderRetrievalService, actorSystem);
        webRuntimeMonitor.start("akka://schmakka");
        try (HttpTestClient client = new HttpTestClient("localhost", webRuntimeMonitor.getServerPort())) {
            client.sendGetRequest("index.html", deadline.timeLeft());
            HttpTestClient.SimpleHttpResponse response = client.getNextResponse();
            assertEquals(HttpResponseStatus.SERVICE_UNAVAILABLE, response.getStatus());
            assertEquals(MimeTypes.getMimeTypeForExtension("txt"), response.getType());
            assertTrue(response.getContent().contains("refresh"));
        }
    } finally {
        if (actorSystem != null) {
            actorSystem.shutdown();
        }
        if (webRuntimeMonitor != null) {
            webRuntimeMonitor.stop();
        }
    }
}
Also used : ActorSystem(akka.actor.ActorSystem) TestingServer(org.apache.curator.test.TestingServer) Path(java.nio.file.Path) HttpTestClient(org.apache.flink.runtime.webmonitor.testutils.HttpTestClient) Configuration(org.apache.flink.configuration.Configuration) LeaderRetrievalService(org.apache.flink.runtime.leaderretrieval.LeaderRetrievalService) Deadline(scala.concurrent.duration.Deadline) File(java.io.File) Test(org.junit.Test)

Example 47 with ActorSystem

use of akka.actor.ActorSystem in project flink by apache.

the class BootstrapTools method startActorSystem.

/**
	 * Starts an Actor System at a specific port.
	 * @param configuration The Flink configuration.
	 * @param listeningAddress The address to listen at.
	 * @param listeningPort The port to listen at.
	 * @param logger the logger to output log information.
	 * @return The ActorSystem which has been started.
	 * @throws Exception
	 */
public static ActorSystem startActorSystem(Configuration configuration, String listeningAddress, int listeningPort, Logger logger) throws Exception {
    String hostPortUrl = listeningAddress + ':' + listeningPort;
    logger.info("Trying to start actor system at {}", hostPortUrl);
    try {
        Config akkaConfig = AkkaUtils.getAkkaConfig(configuration, new scala.Some<>(new scala.Tuple2<String, Object>(listeningAddress, listeningPort)));
        logger.debug("Using akka configuration\n {}", akkaConfig);
        ActorSystem actorSystem = AkkaUtils.createActorSystem(akkaConfig);
        logger.info("Actor system started at {}", AkkaUtils.getAddress(actorSystem));
        return actorSystem;
    } catch (Throwable t) {
        if (t instanceof org.jboss.netty.channel.ChannelException) {
            Throwable cause = t.getCause();
            if (cause != null && t.getCause() instanceof java.net.BindException) {
                throw new IOException("Unable to create ActorSystem at address " + hostPortUrl + " : " + cause.getMessage(), t);
            }
        }
        throw new Exception("Could not create actor system", t);
    }
}
Also used : ActorSystem(akka.actor.ActorSystem) Config(com.typesafe.config.Config) BindException(java.net.BindException) IOException(java.io.IOException) BindException(java.net.BindException) IOException(java.io.IOException)

Example 48 with ActorSystem

use of akka.actor.ActorSystem in project flink by apache.

the class FlinkClient method getJobManager.

private ActorRef getJobManager() throws IOException {
    final Configuration configuration = GlobalConfiguration.loadConfiguration();
    ActorSystem actorSystem;
    try {
        final scala.Tuple2<String, Object> systemEndpoint = new scala.Tuple2<String, Object>("", 0);
        actorSystem = AkkaUtils.createActorSystem(configuration, new Some<scala.Tuple2<String, Object>>(systemEndpoint));
    } catch (final Exception e) {
        throw new RuntimeException("Could not start actor system to communicate with JobManager", e);
    }
    return JobManager.getJobManagerActorRef(AkkaUtils.getAkkaProtocol(configuration), NetUtils.unresolvedHostAndPortToNormalizedString(this.jobManagerHost, this.jobManagerPort), actorSystem, AkkaUtils.getLookupTimeout(configuration));
}
Also used : ActorSystem(akka.actor.ActorSystem) Some(scala.Some) Configuration(org.apache.flink.configuration.Configuration) GlobalConfiguration(org.apache.flink.configuration.GlobalConfiguration) ProgramInvocationException(org.apache.flink.client.program.ProgramInvocationException) AlreadyAliveException(org.apache.storm.generated.AlreadyAliveException) IOException(java.io.IOException) NotAliveException(org.apache.storm.generated.NotAliveException) InvalidTopologyException(org.apache.storm.generated.InvalidTopologyException)

Example 49 with ActorSystem

use of akka.actor.ActorSystem in project torodb by torodb.

the class DefaultOplogApplier method apply.

@Override
public ApplyingJob apply(OplogFetcher fetcher, ApplierContext applierContext) {
    Materializer materializer = ActorMaterializer.create(actorSystem);
    RunnableGraph<Pair<UniqueKillSwitch, CompletionStage<Done>>> graph = createOplogSource(fetcher).async().via(createBatcherFlow(applierContext)).viaMat(KillSwitches.single(), Keep.right()).async().map(analyzedElem -> {
        for (AnalyzedOplogBatch analyzedOplogBatch : analyzedElem.analyzedBatch) {
            batchExecutor.apply(analyzedOplogBatch, applierContext);
        }
        return analyzedElem;
    }).map(this::metricExecution).toMat(Sink.foreach(this::storeLastAppliedOp), (killSwitch, completionStage) -> new Pair<>(killSwitch, completionStage));
    Pair<UniqueKillSwitch, CompletionStage<Done>> pair = graph.run(materializer);
    UniqueKillSwitch killSwitch = pair.first();
    CompletableFuture<Empty> whenComplete = pair.second().toCompletableFuture().thenApply(done -> Empty.getInstance()).whenComplete((done, t) -> {
        fetcher.close();
        if (done != null) {
            LOGGER.trace("Oplog replication stream finished normally");
        } else {
            Throwable cause;
            if (t instanceof CompletionException) {
                cause = t.getCause();
            } else {
                cause = t;
            }
            //the completable future has been cancelled
            if (cause instanceof CancellationException) {
                LOGGER.debug("Oplog replication stream has been cancelled");
                killSwitch.shutdown();
            } else {
                //in this case the exception should came from the stream
                cause = Throwables.getRootCause(cause);
                LOGGER.error("Oplog replication stream finished exceptionally: " + cause.getLocalizedMessage(), cause);
                //the stream should be finished exceptionally, but just in case we
                //notify the kill switch to stop the stream.
                killSwitch.shutdown();
            }
        }
    });
    return new DefaultApplyingJob(killSwitch, whenComplete);
}
Also used : AnalyzedOplogBatch(com.torodb.mongodb.repl.oplogreplier.batch.AnalyzedOplogBatch) BatchAnalyzerFactory(com.torodb.mongodb.repl.oplogreplier.batch.BatchAnalyzer.BatchAnalyzerFactory) BiFunction(java.util.function.BiFunction) Flow(akka.stream.javadsl.Flow) Source(akka.stream.javadsl.Source) Supplier(com.google.common.base.Supplier) KillSwitch(akka.stream.KillSwitch) Materializer(akka.stream.Materializer) CompletableFuture(java.util.concurrent.CompletableFuture) UniqueKillSwitch(akka.stream.UniqueKillSwitch) OplogManager(com.torodb.mongodb.repl.OplogManager) BatchAnalyzer(com.torodb.mongodb.repl.oplogreplier.batch.BatchAnalyzer) Inject(javax.inject.Inject) ActorMaterializer(akka.stream.ActorMaterializer) Keep(akka.stream.javadsl.Keep) RunnableGraph(akka.stream.javadsl.RunnableGraph) OplogOperation(com.eightkdata.mongowp.server.api.oplog.OplogOperation) Empty(com.eightkdata.mongowp.server.api.tools.Empty) OplogManagerPersistException(com.torodb.mongodb.repl.OplogManager.OplogManagerPersistException) AnalyzedOplogBatchExecutor(com.torodb.mongodb.repl.oplogreplier.batch.AnalyzedOplogBatchExecutor) ConcurrentToolsFactory(com.torodb.core.concurrent.ConcurrentToolsFactory) AnalyzedOplogBatch(com.torodb.mongodb.repl.oplogreplier.batch.AnalyzedOplogBatch) OplogFetcher(com.torodb.mongodb.repl.oplogreplier.fetcher.OplogFetcher) Shutdowner(com.torodb.core.Shutdowner) Done(akka.Done) CancellationException(java.util.concurrent.CancellationException) FiniteDuration(scala.concurrent.duration.FiniteDuration) Predicate(java.util.function.Predicate) ToIntFunction(java.util.function.ToIntFunction) Sink(akka.stream.javadsl.Sink) Throwables(com.google.common.base.Throwables) CompletionException(java.util.concurrent.CompletionException) KillSwitches(akka.stream.KillSwitches) WriteOplogTransaction(com.torodb.mongodb.repl.OplogManager.WriteOplogTransaction) ExecutionContexts(akka.dispatch.ExecutionContexts) Pair(akka.japi.Pair) TimeUnit(java.util.concurrent.TimeUnit) Duration(scala.concurrent.duration.Duration) List(java.util.List) Logger(org.apache.logging.log4j.Logger) CompletionStage(java.util.concurrent.CompletionStage) NotUsed(akka.NotUsed) BatchFlow(com.torodb.concurrent.akka.BatchFlow) ActorSystem(akka.actor.ActorSystem) Optional(java.util.Optional) LogManager(org.apache.logging.log4j.LogManager) Await(scala.concurrent.Await) Done(akka.Done) UniqueKillSwitch(akka.stream.UniqueKillSwitch) Empty(com.eightkdata.mongowp.server.api.tools.Empty) CancellationException(java.util.concurrent.CancellationException) CompletionException(java.util.concurrent.CompletionException) Materializer(akka.stream.Materializer) ActorMaterializer(akka.stream.ActorMaterializer) CompletionStage(java.util.concurrent.CompletionStage) Pair(akka.japi.Pair)

Example 50 with ActorSystem

use of akka.actor.ActorSystem in project flink by apache.

the class SlotPoolRpcTest method setup.

// ------------------------------------------------------------------------
//  setup
// ------------------------------------------------------------------------
@BeforeClass
public static void setup() {
    ActorSystem actorSystem = AkkaUtils.createLocalActorSystem(new Configuration());
    rpcService = new AkkaRpcService(actorSystem, Time.seconds(10));
}
Also used : ActorSystem(akka.actor.ActorSystem) AkkaRpcService(org.apache.flink.runtime.rpc.akka.AkkaRpcService) Configuration(org.apache.flink.configuration.Configuration) BeforeClass(org.junit.BeforeClass)

Aggregations

ActorSystem (akka.actor.ActorSystem)91 ActorRef (akka.actor.ActorRef)54 Test (org.junit.Test)51 Configuration (org.apache.flink.configuration.Configuration)27 FiniteDuration (scala.concurrent.duration.FiniteDuration)12 File (java.io.File)11 ActorGateway (org.apache.flink.runtime.instance.ActorGateway)11 LeaderRetrievalService (org.apache.flink.runtime.leaderretrieval.LeaderRetrievalService)11 Props (akka.actor.Props)10 JobGraph (org.apache.flink.runtime.jobgraph.JobGraph)10 TestActorRef (akka.testkit.TestActorRef)8 IOException (java.io.IOException)8 AkkaActorGateway (org.apache.flink.runtime.instance.AkkaActorGateway)8 JobVertex (org.apache.flink.runtime.jobgraph.JobVertex)8 Deadline (scala.concurrent.duration.Deadline)8 AddressFromURIString (akka.actor.AddressFromURIString)7 ActorMaterializer (akka.stream.ActorMaterializer)7 Materializer (akka.stream.Materializer)7 Sink (akka.stream.javadsl.Sink)7 Source (akka.stream.javadsl.Source)7