Search in sources :

Example 6 with TestingFatalErrorHandler

use of org.apache.flink.runtime.util.TestingFatalErrorHandler in project flink by apache.

the class ZooKeeperLeaderElectionTest method testUnExpectedErrorForwarding.

/**
 * Test that background errors in the {@link LeaderElectionDriver} are correctly forwarded to
 * the {@link FatalErrorHandler}.
 */
@Test
public void testUnExpectedErrorForwarding() throws Exception {
    LeaderElectionDriver leaderElectionDriver = null;
    final TestingLeaderElectionEventHandler electionEventHandler = new TestingLeaderElectionEventHandler(LEADER_ADDRESS);
    final TestingFatalErrorHandler fatalErrorHandler = new TestingFatalErrorHandler();
    final FlinkRuntimeException testException = new FlinkRuntimeException("testUnExpectedErrorForwarding");
    final CuratorFrameworkFactory.Builder curatorFrameworkBuilder = CuratorFrameworkFactory.builder().connectString(testingServer.getConnectString()).retryPolicy(new ExponentialBackoffRetry(1, 0)).aclProvider(new ACLProvider() {

        // trigger background exception
        @Override
        public List<ACL> getDefaultAcl() {
            throw testException;
        }

        @Override
        public List<ACL> getAclForPath(String s) {
            throw testException;
        }
    }).namespace("flink");
    try (CuratorFrameworkWithUnhandledErrorListener curatorFrameworkWrapper = ZooKeeperUtils.startCuratorFramework(curatorFrameworkBuilder, fatalErrorHandler)) {
        CuratorFramework clientWithErrorHandler = curatorFrameworkWrapper.asCuratorFramework();
        assertFalse(fatalErrorHandler.getErrorFuture().isDone());
        leaderElectionDriver = createAndInitLeaderElectionDriver(clientWithErrorHandler, electionEventHandler);
        assertThat(fatalErrorHandler.getErrorFuture().join(), FlinkMatchers.containsCause(testException));
    } finally {
        electionEventHandler.close();
        if (leaderElectionDriver != null) {
            leaderElectionDriver.close();
        }
    }
}
Also used : TestingFatalErrorHandler(org.apache.flink.runtime.util.TestingFatalErrorHandler) ACLProvider(org.apache.flink.shaded.curator5.org.apache.curator.framework.api.ACLProvider) CuratorFramework(org.apache.flink.shaded.curator5.org.apache.curator.framework.CuratorFramework) CuratorFrameworkFactory(org.apache.flink.shaded.curator5.org.apache.curator.framework.CuratorFrameworkFactory) ExponentialBackoffRetry(org.apache.flink.shaded.curator5.org.apache.curator.retry.ExponentialBackoffRetry) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) CuratorFrameworkWithUnhandledErrorListener(org.apache.flink.runtime.highavailability.zookeeper.CuratorFrameworkWithUnhandledErrorListener) ACL(org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.data.ACL) Mockito.anyString(org.mockito.Mockito.anyString) Test(org.junit.Test)

Example 7 with TestingFatalErrorHandler

use of org.apache.flink.runtime.util.TestingFatalErrorHandler in project flink by apache.

the class ResourceManagerServiceImplTest method setupClass.

@BeforeClass
public static void setupClass() {
    rpcService = new TestingRpcService();
    haService = new TestingHighAvailabilityServices();
    fatalErrorHandler = new TestingFatalErrorHandler();
}
Also used : TestingFatalErrorHandler(org.apache.flink.runtime.util.TestingFatalErrorHandler) TestingHighAvailabilityServices(org.apache.flink.runtime.highavailability.TestingHighAvailabilityServices) TestingRpcService(org.apache.flink.runtime.rpc.TestingRpcService) BeforeClass(org.junit.BeforeClass)

Example 8 with TestingFatalErrorHandler

use of org.apache.flink.runtime.util.TestingFatalErrorHandler in project flink by apache.

the class WebMonitorEndpointTest method cleansUpExpiredExecutionGraphs.

@Test
public void cleansUpExpiredExecutionGraphs() throws Exception {
    final Configuration configuration = new Configuration();
    configuration.setString(RestOptions.ADDRESS, "localhost");
    configuration.setLong(WebOptions.REFRESH_INTERVAL, 5L);
    final ScheduledExecutorService executor = Executors.newScheduledThreadPool(1);
    final long timeout = 10000L;
    final OneShotLatch cleanupLatch = new OneShotLatch();
    final TestingExecutionGraphCache executionGraphCache = TestingExecutionGraphCache.newBuilder().setCleanupRunnable(cleanupLatch::trigger).build();
    try (final WebMonitorEndpoint<RestfulGateway> webMonitorEndpoint = new WebMonitorEndpoint<>(CompletableFuture::new, configuration, RestHandlerConfiguration.fromConfiguration(configuration), CompletableFuture::new, NoOpTransientBlobService.INSTANCE, executor, VoidMetricFetcher.INSTANCE, new TestingLeaderElectionService(), executionGraphCache, new TestingFatalErrorHandler())) {
        webMonitorEndpoint.start();
        // check that the cleanup will be triggered
        cleanupLatch.await(timeout, TimeUnit.MILLISECONDS);
    } finally {
        ExecutorUtils.gracefulShutdown(timeout, TimeUnit.MILLISECONDS, executor);
    }
}
Also used : TestingFatalErrorHandler(org.apache.flink.runtime.util.TestingFatalErrorHandler) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) CompletableFuture(java.util.concurrent.CompletableFuture) TestingLeaderElectionService(org.apache.flink.runtime.leaderelection.TestingLeaderElectionService) Configuration(org.apache.flink.configuration.Configuration) RestHandlerConfiguration(org.apache.flink.runtime.rest.handler.RestHandlerConfiguration) OneShotLatch(org.apache.flink.core.testutils.OneShotLatch) Test(org.junit.Test)

Example 9 with TestingFatalErrorHandler

use of org.apache.flink.runtime.util.TestingFatalErrorHandler in project flink by apache.

the class TaskExecutorTest method setup.

@Before
public void setup() throws IOException {
    rpc = new TestingRpcService();
    configuration = new Configuration();
    TaskExecutorResourceUtils.adjustForLocalExecution(configuration);
    unresolvedTaskManagerLocation = new LocalUnresolvedTaskManagerLocation();
    jobId = new JobID();
    jobId2 = new JobID();
    testingFatalErrorHandler = new TestingFatalErrorHandler();
    haServices = new TestingHighAvailabilityServices();
    resourceManagerLeaderRetriever = new SettableLeaderRetrievalService();
    jobManagerLeaderRetriever = new SettableLeaderRetrievalService();
    jobManagerLeaderRetriever2 = new SettableLeaderRetrievalService();
    haServices.setResourceManagerLeaderRetriever(resourceManagerLeaderRetriever);
    haServices.setJobMasterLeaderRetriever(jobId, jobManagerLeaderRetriever);
    haServices.setJobMasterLeaderRetriever(jobId2, jobManagerLeaderRetriever2);
    nettyShuffleEnvironment = new NettyShuffleEnvironmentBuilder().build();
}
Also used : TestingFatalErrorHandler(org.apache.flink.runtime.util.TestingFatalErrorHandler) TestingHighAvailabilityServices(org.apache.flink.runtime.highavailability.TestingHighAvailabilityServices) RetryingRegistrationConfiguration(org.apache.flink.runtime.registration.RetryingRegistrationConfiguration) Configuration(org.apache.flink.configuration.Configuration) SettableLeaderRetrievalService(org.apache.flink.runtime.leaderretrieval.SettableLeaderRetrievalService) TestingRpcService(org.apache.flink.runtime.rpc.TestingRpcService) LocalUnresolvedTaskManagerLocation(org.apache.flink.runtime.taskmanager.LocalUnresolvedTaskManagerLocation) NettyShuffleEnvironmentBuilder(org.apache.flink.runtime.io.network.NettyShuffleEnvironmentBuilder) JobID(org.apache.flink.api.common.JobID) Before(org.junit.Before)

Example 10 with TestingFatalErrorHandler

use of org.apache.flink.runtime.util.TestingFatalErrorHandler in project flink by apache.

the class JobMasterPartitionReleaseTest method testPartitionTableCleanupOnDisconnect.

@Test
public void testPartitionTableCleanupOnDisconnect() throws Exception {
    final CompletableFuture<JobID> disconnectTaskExecutorFuture = new CompletableFuture<>();
    final TestingTaskExecutorGateway testingTaskExecutorGateway = new TestingTaskExecutorGatewayBuilder().setDisconnectJobManagerConsumer((jobID, throwable) -> disconnectTaskExecutorFuture.complete(jobID)).createTestingTaskExecutorGateway();
    try (final TestSetup testSetup = new TestSetup(rpcService, testingFatalErrorHandler, testingTaskExecutorGateway)) {
        final JobMasterGateway jobMasterGateway = testSetup.jobMaster.getSelfGateway(JobMasterGateway.class);
        jobMasterGateway.disconnectTaskManager(testSetup.getTaskExecutorResourceID(), new Exception("test"));
        disconnectTaskExecutorFuture.get();
        assertThat(testSetup.getStopTrackingPartitionsTargetResourceId().get(), equalTo(testSetup.getTaskExecutorResourceID()));
    }
}
Also used : TestingTaskExecutorGateway(org.apache.flink.runtime.taskexecutor.TestingTaskExecutorGateway) Arrays(java.util.Arrays) TestingRpcService(org.apache.flink.runtime.rpc.TestingRpcService) JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) JobMasterBuilder(org.apache.flink.runtime.jobmaster.utils.JobMasterBuilder) TaskExecutorGateway(org.apache.flink.runtime.taskexecutor.TaskExecutorGateway) ResultPartitionID(org.apache.flink.runtime.io.network.partition.ResultPartitionID) SettableLeaderRetrievalService(org.apache.flink.runtime.leaderretrieval.SettableLeaderRetrievalService) TestingFatalErrorHandler(org.apache.flink.runtime.util.TestingFatalErrorHandler) After(org.junit.After) TestLogger(org.apache.flink.util.TestLogger) ClassRule(org.junit.ClassRule) BlobServerOptions(org.apache.flink.configuration.BlobServerOptions) AfterClass(org.junit.AfterClass) Collection(java.util.Collection) AbstractPartitionTrackerTest(org.apache.flink.runtime.io.network.partition.AbstractPartitionTrackerTest) Acknowledge(org.apache.flink.runtime.messages.Acknowledge) ResourceProfile(org.apache.flink.runtime.clusterframework.types.ResourceProfile) HeartbeatServices(org.apache.flink.runtime.heartbeat.HeartbeatServices) SlotOffer(org.apache.flink.runtime.taskexecutor.slot.SlotOffer) TestingUtils(org.apache.flink.testutils.TestingUtils) Matchers.containsInAnyOrder(org.hamcrest.Matchers.containsInAnyOrder) ResultPartitionDeploymentDescriptor(org.apache.flink.runtime.deployment.ResultPartitionDeploymentDescriptor) Matchers.equalTo(org.hamcrest.Matchers.equalTo) Time(org.apache.flink.api.common.time.Time) AllocationID(org.apache.flink.runtime.clusterframework.types.AllocationID) BeforeClass(org.junit.BeforeClass) CompletableFuture(java.util.concurrent.CompletableFuture) Function(java.util.function.Function) TaskDeploymentDescriptor(org.apache.flink.runtime.deployment.TaskDeploymentDescriptor) TestingJobMasterPartitionTracker(org.apache.flink.runtime.io.network.partition.TestingJobMasterPartitionTracker) JobGraphTestUtils(org.apache.flink.runtime.jobgraph.JobGraphTestUtils) FatalErrorHandler(org.apache.flink.runtime.rpc.FatalErrorHandler) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) ResourceID(org.apache.flink.runtime.clusterframework.types.ResourceID) Before(org.junit.Before) LocalUnresolvedTaskManagerLocation(org.apache.flink.runtime.taskmanager.LocalUnresolvedTaskManagerLocation) Configuration(org.apache.flink.configuration.Configuration) ExecutionState(org.apache.flink.runtime.execution.ExecutionState) Test(org.junit.Test) IOException(java.io.IOException) RpcUtils(org.apache.flink.runtime.rpc.RpcUtils) ExecutionException(java.util.concurrent.ExecutionException) JobID(org.apache.flink.api.common.JobID) StandaloneCheckpointRecoveryFactory(org.apache.flink.runtime.checkpoint.StandaloneCheckpointRecoveryFactory) TestingTaskExecutorGatewayBuilder(org.apache.flink.runtime.taskexecutor.TestingTaskExecutorGatewayBuilder) TaskExecutionState(org.apache.flink.runtime.taskmanager.TaskExecutionState) TestingHighAvailabilityServices(org.apache.flink.runtime.highavailability.TestingHighAvailabilityServices) Collections(java.util.Collections) TemporaryFolder(org.junit.rules.TemporaryFolder) CompletableFuture(java.util.concurrent.CompletableFuture) TestingTaskExecutorGatewayBuilder(org.apache.flink.runtime.taskexecutor.TestingTaskExecutorGatewayBuilder) TestingTaskExecutorGateway(org.apache.flink.runtime.taskexecutor.TestingTaskExecutorGateway) JobID(org.apache.flink.api.common.JobID) IOException(java.io.IOException) ExecutionException(java.util.concurrent.ExecutionException) AbstractPartitionTrackerTest(org.apache.flink.runtime.io.network.partition.AbstractPartitionTrackerTest) Test(org.junit.Test)

Aggregations

TestingFatalErrorHandler (org.apache.flink.runtime.util.TestingFatalErrorHandler)24 Test (org.junit.Test)15 Configuration (org.apache.flink.configuration.Configuration)12 TestingHighAvailabilityServices (org.apache.flink.runtime.highavailability.TestingHighAvailabilityServices)11 Before (org.junit.Before)10 JobID (org.apache.flink.api.common.JobID)8 HeartbeatServices (org.apache.flink.runtime.heartbeat.HeartbeatServices)8 TestingLeaderElectionService (org.apache.flink.runtime.leaderelection.TestingLeaderElectionService)7 AllocationID (org.apache.flink.runtime.clusterframework.types.AllocationID)6 ResourceID (org.apache.flink.runtime.clusterframework.types.ResourceID)6 UUID (java.util.UUID)5 CompletableFuture (java.util.concurrent.CompletableFuture)5 Time (org.apache.flink.api.common.time.Time)5 SettableLeaderRetrievalService (org.apache.flink.runtime.leaderretrieval.SettableLeaderRetrievalService)5 TaskManagerLocation (org.apache.flink.runtime.taskmanager.TaskManagerLocation)5 StandaloneCheckpointRecoveryFactory (org.apache.flink.runtime.checkpoint.StandaloneCheckpointRecoveryFactory)4 ResourceProfile (org.apache.flink.runtime.clusterframework.types.ResourceProfile)4 TaskDeploymentDescriptor (org.apache.flink.runtime.deployment.TaskDeploymentDescriptor)4 JobGraph (org.apache.flink.runtime.jobgraph.JobGraph)4 TestingLeaderRetrievalService (org.apache.flink.runtime.leaderelection.TestingLeaderRetrievalService)4