Search in sources :

Example 1 with TaskExecutorPartitionInfo

use of org.apache.flink.runtime.io.network.partition.TaskExecutorPartitionInfo in project flink by apache.

the class TaskExecutorPartitionLifecycleTest method testBlockingLocalPartitionReleaseDoesNotBlockTaskExecutor.

@Test
public void testBlockingLocalPartitionReleaseDoesNotBlockTaskExecutor() throws Exception {
    BlockerSync sync = new BlockerSync();
    ResultPartitionManager blockingResultPartitionManager = new ResultPartitionManager() {

        @Override
        public void releasePartition(ResultPartitionID partitionId, Throwable cause) {
            sync.blockNonInterruptible();
            super.releasePartition(partitionId, cause);
        }
    };
    NettyShuffleEnvironment shuffleEnvironment = new NettyShuffleEnvironmentBuilder().setResultPartitionManager(blockingResultPartitionManager).setIoExecutor(TEST_EXECUTOR_SERVICE_RESOURCE.getExecutor()).build();
    final CompletableFuture<ResultPartitionID> startTrackingFuture = new CompletableFuture<>();
    final TaskExecutorPartitionTracker partitionTracker = new TaskExecutorPartitionTrackerImpl(shuffleEnvironment) {

        @Override
        public void startTrackingPartition(JobID producingJobId, TaskExecutorPartitionInfo partitionInfo) {
            super.startTrackingPartition(producingJobId, partitionInfo);
            startTrackingFuture.complete(partitionInfo.getResultPartitionId());
        }
    };
    try {
        internalTestPartitionRelease(partitionTracker, shuffleEnvironment, startTrackingFuture, (jobId, resultPartitionDeploymentDescriptor, taskExecutor, taskExecutorGateway) -> {
            final IntermediateDataSetID dataSetId = resultPartitionDeploymentDescriptor.getResultId();
            taskExecutorGateway.releaseClusterPartitions(Collections.singleton(dataSetId), timeout);
            // execute some operation to check whether the TaskExecutor is blocked
            taskExecutorGateway.canBeReleased().get(5, TimeUnit.SECONDS);
        });
    } finally {
        sync.releaseBlocker();
    }
}
Also used : BlockerSync(org.apache.flink.core.testutils.BlockerSync) TaskExecutorPartitionTracker(org.apache.flink.runtime.io.network.partition.TaskExecutorPartitionTracker) TestingTaskExecutorPartitionTracker(org.apache.flink.runtime.io.network.partition.TestingTaskExecutorPartitionTracker) NettyShuffleEnvironmentBuilder(org.apache.flink.runtime.io.network.NettyShuffleEnvironmentBuilder) NettyShuffleEnvironment(org.apache.flink.runtime.io.network.NettyShuffleEnvironment) ResultPartitionManager(org.apache.flink.runtime.io.network.partition.ResultPartitionManager) TaskExecutorPartitionTrackerImpl(org.apache.flink.runtime.io.network.partition.TaskExecutorPartitionTrackerImpl) CompletableFuture(java.util.concurrent.CompletableFuture) ResultPartitionID(org.apache.flink.runtime.io.network.partition.ResultPartitionID) IntermediateDataSetID(org.apache.flink.runtime.jobgraph.IntermediateDataSetID) TaskExecutorPartitionInfo(org.apache.flink.runtime.io.network.partition.TaskExecutorPartitionInfo) JobID(org.apache.flink.api.common.JobID) Test(org.junit.Test)

Aggregations

CompletableFuture (java.util.concurrent.CompletableFuture)1 JobID (org.apache.flink.api.common.JobID)1 BlockerSync (org.apache.flink.core.testutils.BlockerSync)1 NettyShuffleEnvironment (org.apache.flink.runtime.io.network.NettyShuffleEnvironment)1 NettyShuffleEnvironmentBuilder (org.apache.flink.runtime.io.network.NettyShuffleEnvironmentBuilder)1 ResultPartitionID (org.apache.flink.runtime.io.network.partition.ResultPartitionID)1 ResultPartitionManager (org.apache.flink.runtime.io.network.partition.ResultPartitionManager)1 TaskExecutorPartitionInfo (org.apache.flink.runtime.io.network.partition.TaskExecutorPartitionInfo)1 TaskExecutorPartitionTracker (org.apache.flink.runtime.io.network.partition.TaskExecutorPartitionTracker)1 TaskExecutorPartitionTrackerImpl (org.apache.flink.runtime.io.network.partition.TaskExecutorPartitionTrackerImpl)1 TestingTaskExecutorPartitionTracker (org.apache.flink.runtime.io.network.partition.TestingTaskExecutorPartitionTracker)1 IntermediateDataSetID (org.apache.flink.runtime.jobgraph.IntermediateDataSetID)1 Test (org.junit.Test)1