Examples with LogicalSlot org.apache.flink.runtime.jobmaster.LogicalSlot used on opensource projects

Example 41 with LogicalSlot

use of org.apache.flink.runtime.jobmaster.LogicalSlot in project flink by apache.

the class SharedSlotTest method tesDuplicatedReturnLogicalSlotFails.

@Test
public void tesDuplicatedReturnLogicalSlotFails() {
    CompletableFuture<PhysicalSlot> slotContextFuture = CompletableFuture.completedFuture(new TestingPhysicalSlot(RP, new AllocationID()));
    AtomicInteger released = new AtomicInteger(0);
    SharedSlot sharedSlot = SharedSlotBuilder.newBuilder().withSlotContextFuture(slotContextFuture).withExternalReleaseCallback(g -> released.incrementAndGet()).build();
    LogicalSlot logicalSlot = sharedSlot.allocateLogicalSlot(EV1).join();
    sharedSlot.returnLogicalSlot(logicalSlot);
    try {
        sharedSlot.returnLogicalSlot(logicalSlot);
        fail("Duplicated 'returnLogicalSlot' call should fail with IllegalStateException");
    } catch (IllegalStateException e) {
    // expected
    }
}

Example 42 with LogicalSlot

use of org.apache.flink.runtime.jobmaster.LogicalSlot in project flink by apache.

the class SlotSharingExecutionSlotAllocatorTest method testFailedPhysicalSlotRequestFailsLogicalSlotFuturesAndRemovesSharedSlot.

@Test
public void testFailedPhysicalSlotRequestFailsLogicalSlotFuturesAndRemovesSharedSlot() {
    AllocationContext context = AllocationContext.newBuilder().addGroup(EV1).withPhysicalSlotProvider(TestingPhysicalSlotProvider.createWithoutImmediatePhysicalSlotCreation()).build();
    CompletableFuture<LogicalSlot> logicalSlotFuture = context.allocateSlotsFor(EV1).get(0).getLogicalSlotFuture();
    SlotRequestId slotRequestId = context.getSlotProvider().getFirstRequestOrFail().getSlotRequestId();
    assertThat(logicalSlotFuture.isDone(), is(false));
    context.getSlotProvider().getResponses().get(slotRequestId).completeExceptionally(new Throwable());
    assertThat(logicalSlotFuture.isCompletedExceptionally(), is(true));
    // next allocation allocates new shared slot
    context.allocateSlotsFor(EV1);
    assertThat(context.getSlotProvider().getRequests().keySet(), hasSize(2));
}

Example 43 with LogicalSlot

use of org.apache.flink.runtime.jobmaster.LogicalSlot in project flink by apache.

the class SlotSharingExecutionSlotAllocatorTest method testRequestBulkCancel.

@Test
public void testRequestBulkCancel() {
    TestingPhysicalSlotRequestBulkChecker bulkChecker = new TestingPhysicalSlotRequestBulkChecker();
    AllocationContext context = createBulkCheckerContextWithEv12GroupAndEv3Group(bulkChecker);
    // allocate 2 physical slots for 2 groups
    List<SlotExecutionVertexAssignment> assignments1 = context.allocateSlotsFor(EV1, EV3);
    fulfilOneOfTwoSlotRequestsAndGetPendingProfile(context, new AllocationID());
    PhysicalSlotRequestBulk bulk1 = bulkChecker.getBulk();
    List<SlotExecutionVertexAssignment> assignments2 = context.allocateSlotsFor(EV2);
    // cancelling of (EV1, EV3) releases assignments1 and only one physical slot for EV3
    // the second physical slot is held by sharing EV2 from the next bulk
    bulk1.cancel(new Throwable());
    // return completed logical slot to clear shared slot and release physical slot
    CompletableFuture<LogicalSlot> ev1slot = assignments1.get(0).getLogicalSlotFuture();
    boolean ev1failed = ev1slot.isCompletedExceptionally();
    CompletableFuture<LogicalSlot> ev3slot = assignments1.get(1).getLogicalSlotFuture();
    boolean ev3failed = ev3slot.isCompletedExceptionally();
    LogicalSlot slot = ev1failed ? ev3slot.join() : ev1slot.join();
    releaseLogicalSlot(slot);
    // EV3 needs again a physical slot, therefore there are 3 requests overall
    context.allocateSlotsFor(EV1, EV3);
    assertThat(context.getSlotProvider().getRequests().values(), hasSize(3));
    // either EV1 or EV3 logical slot future is fulfilled before cancellation
    assertThat(ev1failed != ev3failed, is(true));
    assertThat(assignments2.get(0).getLogicalSlotFuture().isCompletedExceptionally(), is(false));
}

Example 44 with LogicalSlot

use of org.apache.flink.runtime.jobmaster.LogicalSlot in project flink by apache.

the class TestExecutionSlotAllocator method completePendingRequest.

public LogicalSlot completePendingRequest(final ExecutionVertexID executionVertexId) {
    final LogicalSlot slot = logicalSlotBuilder.setSlotOwner(this).createTestingLogicalSlot();
    final SlotExecutionVertexAssignment slotVertexAssignment = removePendingRequest(executionVertexId);
    checkState(slotVertexAssignment != null);
    slotVertexAssignment.getLogicalSlotFuture().complete(slot);
    return slot;
}

Example 45 with LogicalSlot

use of org.apache.flink.runtime.jobmaster.LogicalSlot in project flink by apache.

the class AdaptiveScheduler method assignSlotsToExecutionGraph.

@Nonnull
private ExecutionGraph assignSlotsToExecutionGraph(ExecutionGraph executionGraph, ReservedSlots reservedSlots) {
    for (ExecutionVertex executionVertex : executionGraph.getAllExecutionVertices()) {
        final LogicalSlot assignedSlot = reservedSlots.getSlotFor(executionVertex.getID());
        final CompletableFuture<Void> registrationFuture = executionVertex.getCurrentExecutionAttempt().registerProducedPartitions(assignedSlot.getTaskManagerLocation(), false);
        Preconditions.checkState(registrationFuture.isDone(), "Partition registration must be completed immediately for reactive mode");
        executionVertex.tryAssignResource(assignedSlot);
    }
    return executionGraph;
}