Search in sources :

Example 16 with Timeout

use of akka.util.Timeout in project flink by apache.

the class TestBaseUtils method stopCluster.

public static void stopCluster(LocalFlinkMiniCluster executor, FiniteDuration timeout) throws Exception {
    if (logDir != null) {
        FileUtils.deleteDirectory(logDir);
    }
    if (executor != null) {
        int numUnreleasedBCVars = 0;
        int numActiveConnections = 0;
        if (executor.running()) {
            List<ActorRef> tms = executor.getTaskManagersAsJava();
            List<Future<Object>> bcVariableManagerResponseFutures = new ArrayList<>();
            List<Future<Object>> numActiveConnectionsResponseFutures = new ArrayList<>();
            for (ActorRef tm : tms) {
                bcVariableManagerResponseFutures.add(Patterns.ask(tm, TaskManagerMessages.getRequestBroadcastVariablesWithReferences(), new Timeout(timeout)));
                numActiveConnectionsResponseFutures.add(Patterns.ask(tm, TaskManagerMessages.getRequestNumActiveConnections(), new Timeout(timeout)));
            }
            Future<Iterable<Object>> bcVariableManagerFutureResponses = Futures.sequence(bcVariableManagerResponseFutures, defaultExecutionContext());
            Iterable<Object> responses = Await.result(bcVariableManagerFutureResponses, timeout);
            for (Object response : responses) {
                numUnreleasedBCVars += ((TaskManagerMessages.ResponseBroadcastVariablesWithReferences) response).number();
            }
            Future<Iterable<Object>> numActiveConnectionsFutureResponses = Futures.sequence(numActiveConnectionsResponseFutures, defaultExecutionContext());
            responses = Await.result(numActiveConnectionsFutureResponses, timeout);
            for (Object response : responses) {
                numActiveConnections += ((TaskManagerMessages.ResponseNumActiveConnections) response).number();
            }
        }
        executor.stop();
        FileSystem.closeAll();
        System.gc();
        Assert.assertEquals("Not all broadcast variables were released.", 0, numUnreleasedBCVars);
        Assert.assertEquals("Not all TCP connections were released.", 0, numActiveConnections);
    }
}
Also used : TaskManagerMessages(org.apache.flink.runtime.messages.TaskManagerMessages) ActorRef(akka.actor.ActorRef) Timeout(akka.util.Timeout) ArrayList(java.util.ArrayList) Future(scala.concurrent.Future)

Example 17 with Timeout

use of akka.util.Timeout in project flink by apache.

the class ProcessFailureCancelingITCase method waitUntilNumTaskManagersAreRegistered.

private void waitUntilNumTaskManagersAreRegistered(ActorRef jobManager, int numExpected, long maxDelay) throws Exception {
    final long deadline = System.currentTimeMillis() + maxDelay;
    while (true) {
        long remaining = deadline - System.currentTimeMillis();
        if (remaining <= 0) {
            fail("The TaskManagers did not register within the expected time (" + maxDelay + "msecs)");
        }
        FiniteDuration timeout = new FiniteDuration(remaining, TimeUnit.MILLISECONDS);
        try {
            Future<?> result = Patterns.ask(jobManager, JobManagerMessages.getRequestNumberRegisteredTaskManager(), new Timeout(timeout));
            Integer numTMs = (Integer) Await.result(result, timeout);
            if (numTMs == numExpected) {
                break;
            }
        } catch (TimeoutException e) {
        // ignore and retry
        } catch (ClassCastException e) {
            fail("Wrong response: " + e.getMessage());
        }
    }
}
Also used : Timeout(akka.util.Timeout) FiniteDuration(scala.concurrent.duration.FiniteDuration) TimeoutException(java.util.concurrent.TimeoutException)

Example 18 with Timeout

use of akka.util.Timeout in project flink by apache.

the class ProcessFailureCancelingITCase method cancelRunningJob.

private void cancelRunningJob(ActorRef jobManager) throws Exception {
    final FiniteDuration askTimeout = new FiniteDuration(10, TimeUnit.SECONDS);
    // try at most for 30 seconds
    final long deadline = System.currentTimeMillis() + 30000;
    JobID jobId = null;
    do {
        Future<Object> response = Patterns.ask(jobManager, JobManagerMessages.getRequestRunningJobsStatus(), new Timeout(askTimeout));
        Object result;
        try {
            result = Await.result(response, askTimeout);
        } catch (Exception e) {
            throw new Exception("Could not retrieve running jobs from the JobManager.", e);
        }
        if (result instanceof JobManagerMessages.RunningJobsStatus) {
            List<JobStatusMessage> jobs = ((JobManagerMessages.RunningJobsStatus) result).getStatusMessages();
            if (jobs.size() == 1) {
                jobId = jobs.get(0).getJobId();
                break;
            }
        }
    } while (System.currentTimeMillis() < deadline);
    if (jobId == null) {
        // we never found it running, must have failed already
        return;
    }
    // tell the JobManager to cancel the job
    jobManager.tell(new JobManagerMessages.CancelJob(jobId), ActorRef.noSender());
}
Also used : Timeout(akka.util.Timeout) JobStatusMessage(org.apache.flink.runtime.client.JobStatusMessage) JobManagerMessages(org.apache.flink.runtime.messages.JobManagerMessages) FiniteDuration(scala.concurrent.duration.FiniteDuration) JobID(org.apache.flink.api.common.JobID) ProgramInvocationException(org.apache.flink.client.program.ProgramInvocationException) TimeoutException(java.util.concurrent.TimeoutException)

Aggregations

Timeout (akka.util.Timeout)18 ActorRef (akka.actor.ActorRef)11 FiniteDuration (scala.concurrent.duration.FiniteDuration)11 Props (akka.actor.Props)8 TestingLeaderRetrievalService (org.apache.flink.runtime.leaderelection.TestingLeaderRetrievalService)7 JobClientMessages (org.apache.flink.runtime.messages.JobClientMessages)7 Test (org.junit.Test)7 UUID (java.util.UUID)5 IOException (java.io.IOException)4 ProgramInvocationException (org.apache.flink.client.program.ProgramInvocationException)4 TimeoutException (java.util.concurrent.TimeoutException)3 AttachToJobAndWait (org.apache.flink.runtime.messages.JobClientMessages.AttachToJobAndWait)3 ArrayList (java.util.ArrayList)2 JobStatusMessage (org.apache.flink.runtime.client.JobStatusMessage)2 YarnException (org.apache.hadoop.yarn.exceptions.YarnException)2 ActorSelection (akka.actor.ActorSelection)1 JavaTestKit (akka.testkit.JavaTestKit)1 File (java.io.File)1 JobID (org.apache.flink.api.common.JobID)1 Configuration (org.apache.flink.configuration.Configuration)1