Search in sources :

Example 1 with Timeout

use of org.mockito.verification.Timeout in project flink by apache.

the class ExecutionGraphSchedulingTest method testScheduleSourceBeforeTarget.

// ------------------------------------------------------------------------
//  Tests
// ------------------------------------------------------------------------
/**
	 * Tests that with scheduling futures and pipelined deployment, the target vertex will
	 * not deploy its task before the source vertex does.
	 */
@Test
public void testScheduleSourceBeforeTarget() throws Exception {
    //                                            [pipelined]
    //  we construct a simple graph    (source) ----------------> (target)
    final int parallelism = 1;
    final JobVertex sourceVertex = new JobVertex("source");
    sourceVertex.setParallelism(parallelism);
    sourceVertex.setInvokableClass(NoOpInvokable.class);
    final JobVertex targetVertex = new JobVertex("target");
    targetVertex.setParallelism(parallelism);
    targetVertex.setInvokableClass(NoOpInvokable.class);
    targetVertex.connectNewDataSetAsInput(sourceVertex, DistributionPattern.ALL_TO_ALL, ResultPartitionType.PIPELINED);
    final JobID jobId = new JobID();
    final JobGraph jobGraph = new JobGraph(jobId, "test", sourceVertex, targetVertex);
    final FlinkCompletableFuture<SimpleSlot> sourceFuture = new FlinkCompletableFuture<>();
    final FlinkCompletableFuture<SimpleSlot> targetFuture = new FlinkCompletableFuture<>();
    ProgrammedSlotProvider slotProvider = new ProgrammedSlotProvider(parallelism);
    slotProvider.addSlot(sourceVertex.getID(), 0, sourceFuture);
    slotProvider.addSlot(targetVertex.getID(), 0, targetFuture);
    final ExecutionGraph eg = createExecutionGraph(jobGraph, slotProvider);
    //  set up two TaskManager gateways and slots
    final TaskManagerGateway gatewaySource = createTaskManager();
    final TaskManagerGateway gatewayTarget = createTaskManager();
    final SimpleSlot sourceSlot = createSlot(gatewaySource, jobId);
    final SimpleSlot targetSlot = createSlot(gatewayTarget, jobId);
    eg.setScheduleMode(ScheduleMode.EAGER);
    eg.setQueuedSchedulingAllowed(true);
    eg.scheduleForExecution();
    // job should be running
    assertEquals(JobStatus.RUNNING, eg.getState());
    // we fulfill the target slot before the source slot
    // that should not cause a deployment or deployment related failure
    targetFuture.complete(targetSlot);
    verify(gatewayTarget, new Timeout(50, times(0))).submitTask(any(TaskDeploymentDescriptor.class), any(Time.class));
    assertEquals(JobStatus.RUNNING, eg.getState());
    // now supply the source slot
    sourceFuture.complete(sourceSlot);
    // by now, all deployments should have happened
    verify(gatewaySource, timeout(1000)).submitTask(any(TaskDeploymentDescriptor.class), any(Time.class));
    verify(gatewayTarget, timeout(1000)).submitTask(any(TaskDeploymentDescriptor.class), any(Time.class));
    assertEquals(JobStatus.RUNNING, eg.getState());
}
Also used : Timeout(org.mockito.verification.Timeout) TaskManagerGateway(org.apache.flink.runtime.jobmanager.slots.TaskManagerGateway) Time(org.apache.flink.api.common.time.Time) FlinkCompletableFuture(org.apache.flink.runtime.concurrent.impl.FlinkCompletableFuture) SimpleSlot(org.apache.flink.runtime.instance.SimpleSlot) JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) JobVertex(org.apache.flink.runtime.jobgraph.JobVertex) TaskDeploymentDescriptor(org.apache.flink.runtime.deployment.TaskDeploymentDescriptor) JobID(org.apache.flink.api.common.JobID) Test(org.junit.Test)

Example 2 with Timeout

use of org.mockito.verification.Timeout in project flink by apache.

the class ExecutionGraphSchedulingTest method testOneSlotFailureAbortsDeploy.

/**
	 * This test verifies that if one slot future fails, the deployment will be aborted.
	 */
@Test
public void testOneSlotFailureAbortsDeploy() throws Exception {
    //                                            [pipelined]
    //  we construct a simple graph    (source) ----------------> (target)
    final int parallelism = 6;
    final JobVertex sourceVertex = new JobVertex("source");
    sourceVertex.setParallelism(parallelism);
    sourceVertex.setInvokableClass(NoOpInvokable.class);
    final JobVertex targetVertex = new JobVertex("target");
    targetVertex.setParallelism(parallelism);
    targetVertex.setInvokableClass(NoOpInvokable.class);
    targetVertex.connectNewDataSetAsInput(sourceVertex, DistributionPattern.POINTWISE, ResultPartitionType.PIPELINED);
    final JobID jobId = new JobID();
    final JobGraph jobGraph = new JobGraph(jobId, "test", sourceVertex, targetVertex);
    //
    //  Create the slots, futures, and the slot provider
    final TaskManagerGateway taskManager = mock(TaskManagerGateway.class);
    final SlotOwner slotOwner = mock(SlotOwner.class);
    final SimpleSlot[] sourceSlots = new SimpleSlot[parallelism];
    final SimpleSlot[] targetSlots = new SimpleSlot[parallelism];
    @SuppressWarnings({ "unchecked", "rawtypes" }) final FlinkCompletableFuture<SimpleSlot>[] sourceFutures = new FlinkCompletableFuture[parallelism];
    @SuppressWarnings({ "unchecked", "rawtypes" }) final FlinkCompletableFuture<SimpleSlot>[] targetFutures = new FlinkCompletableFuture[parallelism];
    for (int i = 0; i < parallelism; i++) {
        sourceSlots[i] = createSlot(taskManager, jobId, slotOwner);
        targetSlots[i] = createSlot(taskManager, jobId, slotOwner);
        sourceFutures[i] = new FlinkCompletableFuture<>();
        targetFutures[i] = new FlinkCompletableFuture<>();
    }
    ProgrammedSlotProvider slotProvider = new ProgrammedSlotProvider(parallelism);
    slotProvider.addSlots(sourceVertex.getID(), sourceFutures);
    slotProvider.addSlots(targetVertex.getID(), targetFutures);
    final ExecutionGraph eg = createExecutionGraph(jobGraph, slotProvider);
    TerminalJobStatusListener testListener = new TerminalJobStatusListener();
    eg.registerJobStatusListener(testListener);
    for (int i = 0; i < parallelism; i += 2) {
        sourceFutures[i].complete(sourceSlots[i]);
        targetFutures[i + 1].complete(targetSlots[i + 1]);
    }
    //
    //  kick off the scheduling
    eg.setScheduleMode(ScheduleMode.EAGER);
    eg.setQueuedSchedulingAllowed(true);
    eg.scheduleForExecution();
    // fail one slot
    sourceFutures[1].completeExceptionally(new TestRuntimeException());
    // wait until the job failed as a whole
    testListener.waitForTerminalState(2000);
    // wait until all slots are back
    verify(slotOwner, new Timeout(2000, times(6))).returnAllocatedSlot(any(Slot.class));
    // no deployment calls must have happened
    verify(taskManager, times(0)).submitTask(any(TaskDeploymentDescriptor.class), any(Time.class));
    // all completed futures must have been returns
    for (int i = 0; i < parallelism; i += 2) {
        assertTrue(sourceSlots[i].isCanceled());
        assertTrue(targetSlots[i + 1].isCanceled());
    }
}
Also used : Timeout(org.mockito.verification.Timeout) TaskManagerGateway(org.apache.flink.runtime.jobmanager.slots.TaskManagerGateway) Time(org.apache.flink.api.common.time.Time) SimpleSlot(org.apache.flink.runtime.instance.SimpleSlot) FlinkCompletableFuture(org.apache.flink.runtime.concurrent.impl.FlinkCompletableFuture) SlotOwner(org.apache.flink.runtime.jobmanager.slots.SlotOwner) JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) JobVertex(org.apache.flink.runtime.jobgraph.JobVertex) Slot(org.apache.flink.runtime.instance.Slot) SimpleSlot(org.apache.flink.runtime.instance.SimpleSlot) AllocatedSlot(org.apache.flink.runtime.jobmanager.slots.AllocatedSlot) TaskDeploymentDescriptor(org.apache.flink.runtime.deployment.TaskDeploymentDescriptor) JobID(org.apache.flink.api.common.JobID) Test(org.junit.Test)

Example 3 with Timeout

use of org.mockito.verification.Timeout in project mockito by mockito.

the class VerificationWithTimeoutTest method should_return_formatted_output_from_toString_using_wrapped_verification_mode.

@Test
public void should_return_formatted_output_from_toString_using_wrapped_verification_mode() {
    VerificationMode timeoutAndAtLeastOnce = new Timeout(9, new DummyVerificationMode());
    assertThat(timeoutAndAtLeastOnce).hasToString("Wanted after at most 9 ms: [Dummy verification mode]");
}
Also used : Timeout(org.mockito.verification.Timeout) DummyVerificationMode(org.mockito.internal.verification.DummyVerificationMode) VerificationMode(org.mockito.verification.VerificationMode) DummyVerificationMode(org.mockito.internal.verification.DummyVerificationMode) Test(org.junit.Test)

Example 4 with Timeout

use of org.mockito.verification.Timeout in project geode by apache.

the class GMSJoinLeaveJUnitTest method testViewIgnoredAfterShutdown.

@Test
public void testViewIgnoredAfterShutdown() throws Exception {
    try {
        initMocks(true);
        System.setProperty(GMSJoinLeave.BYPASS_DISCOVERY_PROPERTY, "true");
        gmsJoinLeave.join();
        installView(1, gmsJoinLeaveMemberId, createMemberList(mockMembers[0], mockMembers[1], mockMembers[2], gmsJoinLeaveMemberId, mockMembers[3]));
        gmsJoinLeave.stop();
        for (int i = 1; i < 4; i++) {
            RemoveMemberMessage msg = new RemoveMemberMessage(gmsJoinLeaveMemberId, mockMembers[i], "crashed");
            msg.setSender(gmsJoinLeaveMemberId);
            gmsJoinLeave.processMessage(msg);
        }
        Timeout to = new Timeout(2 * ServiceConfig.MEMBER_REQUEST_COLLECTION_INTERVAL, never());
        verify(messenger, to).send(isA(NetworkPartitionMessage.class));
    } finally {
        System.getProperties().remove(GMSJoinLeave.BYPASS_DISCOVERY_PROPERTY);
    }
}
Also used : NetworkPartitionMessage(org.apache.geode.distributed.internal.membership.gms.messages.NetworkPartitionMessage) RemoveMemberMessage(org.apache.geode.distributed.internal.membership.gms.messages.RemoveMemberMessage) Timeout(org.mockito.verification.Timeout) Test(org.junit.Test) MembershipTest(org.apache.geode.test.junit.categories.MembershipTest) IntegrationTest(org.apache.geode.test.junit.categories.IntegrationTest)

Example 5 with Timeout

use of org.mockito.verification.Timeout in project geode by apache.

the class GMSJoinLeaveJUnitTest method testNetworkPartionMessage.

// With the removal of the JoinResponse message from GMSJoinLeave.processJoinRequest (GEODE-870)
// This test now seems to be invalid
// @Test
// public void testJoinResponseMsgWithBecomeCoordinator() throws Exception {
// initMocks(false);
// gmsJoinLeaveMemberId.getNetMember().setPreferredForCoordinator(false);
// JoinRequestMessage reqMsg = new JoinRequestMessage(gmsJoinLeaveMemberId, mockMembers[0], null,
// 56734);
// InternalDistributedMember ids = new InternalDistributedMember("localhost", 97898);
// ids.getNetMember().setPreferredForCoordinator(true);
// gmsJoinLeave.processMessage(reqMsg);
// ArgumentCaptor<JoinResponseMessage> ac = ArgumentCaptor.forClass(JoinResponseMessage.class);
// verify(messenger).send(ac.capture());
//
// assertTrue("Should have asked for becoming a coordinator",
// ac.getValue().getBecomeCoordinator());
// }
@Test
public void testNetworkPartionMessage() throws Exception {
    try {
        initMocks(true);
        System.setProperty(GMSJoinLeave.BYPASS_DISCOVERY_PROPERTY, "true");
        gmsJoinLeave.join();
        installView(1, gmsJoinLeaveMemberId, createMemberList(mockMembers[0], mockMembers[1], mockMembers[2], gmsJoinLeaveMemberId, mockMembers[3]));
        for (int i = 1; i < 4; i++) {
            RemoveMemberMessage msg = new RemoveMemberMessage(gmsJoinLeaveMemberId, mockMembers[i], "crashed");
            msg.setSender(gmsJoinLeaveMemberId);
            gmsJoinLeave.processMessage(msg);
        }
        Timeout to = new Timeout(3 * ServiceConfig.MEMBER_REQUEST_COLLECTION_INTERVAL, new Times(1));
        verify(messenger, to).send(isA(NetworkPartitionMessage.class));
    } finally {
        System.getProperties().remove(GMSJoinLeave.BYPASS_DISCOVERY_PROPERTY);
    }
}
Also used : NetworkPartitionMessage(org.apache.geode.distributed.internal.membership.gms.messages.NetworkPartitionMessage) RemoveMemberMessage(org.apache.geode.distributed.internal.membership.gms.messages.RemoveMemberMessage) Timeout(org.mockito.verification.Timeout) Times(org.mockito.internal.verification.Times) Test(org.junit.Test) MembershipTest(org.apache.geode.test.junit.categories.MembershipTest) IntegrationTest(org.apache.geode.test.junit.categories.IntegrationTest)

Aggregations

Test (org.junit.Test)6 Timeout (org.mockito.verification.Timeout)6 JobID (org.apache.flink.api.common.JobID)3 Time (org.apache.flink.api.common.time.Time)3 FlinkCompletableFuture (org.apache.flink.runtime.concurrent.impl.FlinkCompletableFuture)3 TaskDeploymentDescriptor (org.apache.flink.runtime.deployment.TaskDeploymentDescriptor)3 SimpleSlot (org.apache.flink.runtime.instance.SimpleSlot)3 JobGraph (org.apache.flink.runtime.jobgraph.JobGraph)3 JobVertex (org.apache.flink.runtime.jobgraph.JobVertex)3 TaskManagerGateway (org.apache.flink.runtime.jobmanager.slots.TaskManagerGateway)3 Slot (org.apache.flink.runtime.instance.Slot)2 AllocatedSlot (org.apache.flink.runtime.jobmanager.slots.AllocatedSlot)2 SlotOwner (org.apache.flink.runtime.jobmanager.slots.SlotOwner)2 NetworkPartitionMessage (org.apache.geode.distributed.internal.membership.gms.messages.NetworkPartitionMessage)2 RemoveMemberMessage (org.apache.geode.distributed.internal.membership.gms.messages.RemoveMemberMessage)2 IntegrationTest (org.apache.geode.test.junit.categories.IntegrationTest)2 MembershipTest (org.apache.geode.test.junit.categories.MembershipTest)2 DummyVerificationMode (org.mockito.internal.verification.DummyVerificationMode)1 Times (org.mockito.internal.verification.Times)1 VerificationMode (org.mockito.verification.VerificationMode)1