Examples with CuratorFrameworkWithUnhandledErrorListener org.apache.flink.runtime.highavailability.zookeeper.CuratorFrameworkWithUnhandledErrorListener used on opensource projects

Example 1 with CuratorFrameworkWithUnhandledErrorListener

use of org.apache.flink.runtime.highavailability.zookeeper.CuratorFrameworkWithUnhandledErrorListener in project flink by apache.

the class HighAvailabilityServicesUtils method createZooKeeperHaServices.

private static HighAvailabilityServices createZooKeeperHaServices(Configuration configuration, Executor executor, FatalErrorHandler fatalErrorHandler) throws Exception {
    final boolean useOldHaServices = configuration.get(HighAvailabilityOptions.USE_OLD_HA_SERVICES);
    BlobStoreService blobStoreService = BlobUtils.createBlobStoreFromConfig(configuration);
    final CuratorFrameworkWithUnhandledErrorListener curatorFrameworkWrapper = ZooKeeperUtils.startCuratorFramework(configuration, fatalErrorHandler);
    if (useOldHaServices) {
        return new ZooKeeperHaServices(curatorFrameworkWrapper, executor, configuration, blobStoreService);
    } else {
        return new ZooKeeperMultipleComponentLeaderElectionHaServices(curatorFrameworkWrapper, configuration, executor, blobStoreService, fatalErrorHandler);
    }
}

Example 2 with CuratorFrameworkWithUnhandledErrorListener

use of org.apache.flink.runtime.highavailability.zookeeper.CuratorFrameworkWithUnhandledErrorListener in project flink by apache.

the class ZKCheckpointIDCounterMultiServersTest method testRecoveredAfterConnectionLoss.

/**
 * Tests that {@link ZooKeeperCheckpointIDCounter} can be recovered after a connection loss
 * exception from ZooKeeper ensemble.
 *
 * <p>See also FLINK-14091.
 */
@Test
public void testRecoveredAfterConnectionLoss() throws Exception {
    final Configuration configuration = new Configuration();
    configuration.setString(HighAvailabilityOptions.HA_ZOOKEEPER_QUORUM, zooKeeperResource.getConnectString());
    final CuratorFrameworkWithUnhandledErrorListener curatorFrameworkWrapper = ZooKeeperUtils.startCuratorFramework(configuration, NoOpFatalErrorHandler.INSTANCE);
    try {
        OneShotLatch connectionLossLatch = new OneShotLatch();
        OneShotLatch reconnectedLatch = new OneShotLatch();
        TestingLastStateConnectionStateListener listener = new TestingLastStateConnectionStateListener(connectionLossLatch, reconnectedLatch);
        ZooKeeperCheckpointIDCounter idCounter = new ZooKeeperCheckpointIDCounter(curatorFrameworkWrapper.asCuratorFramework(), listener);
        idCounter.start();
        AtomicLong localCounter = new AtomicLong(1L);
        assertThat("ZooKeeperCheckpointIDCounter doesn't properly work.", idCounter.getAndIncrement(), is(localCounter.getAndIncrement()));
        zooKeeperResource.restart();
        connectionLossLatch.await();
        reconnectedLatch.await();
        assertThat("ZooKeeperCheckpointIDCounter doesn't properly work after reconnected.", idCounter.getAndIncrement(), is(localCounter.getAndIncrement()));
    } finally {
        curatorFrameworkWrapper.close();
    }
}

Example 3 with CuratorFrameworkWithUnhandledErrorListener

use of org.apache.flink.runtime.highavailability.zookeeper.CuratorFrameworkWithUnhandledErrorListener in project flink by apache.

the class ZooKeeperCompletedCheckpointStoreTest method testDiscardingCheckpointsAtShutDown.

/**
 * Tests that checkpoints are discarded when the completed checkpoint store is shut down with a
 * globally terminal state.
 */
@Test
public void testDiscardingCheckpointsAtShutDown() throws Exception {
    final SharedStateRegistry sharedStateRegistry = new SharedStateRegistryImpl();
    final Configuration configuration = new Configuration();
    configuration.setString(HighAvailabilityOptions.HA_ZOOKEEPER_QUORUM, zooKeeperResource.getConnectString());
    final CuratorFrameworkWithUnhandledErrorListener curatorFrameworkWrapper = ZooKeeperUtils.startCuratorFramework(configuration, NoOpFatalErrorHandler.INSTANCE);
    final CompletedCheckpointStore checkpointStore = createZooKeeperCheckpointStore(curatorFrameworkWrapper.asCuratorFramework());
    try {
        final CompletedCheckpointStoreTest.TestCompletedCheckpoint checkpoint1 = CompletedCheckpointStoreTest.createCheckpoint(0, sharedStateRegistry);
        checkpointStore.addCheckpointAndSubsumeOldestOne(checkpoint1, new CheckpointsCleaner(), () -> {
        });
        assertThat(checkpointStore.getAllCheckpoints(), Matchers.contains(checkpoint1));
        checkpointStore.shutdown(JobStatus.FINISHED, new CheckpointsCleaner());
        // verify that the checkpoint is discarded
        CompletedCheckpointStoreTest.verifyCheckpointDiscarded(checkpoint1);
    } finally {
        curatorFrameworkWrapper.close();
    }
}

Example 4 with CuratorFrameworkWithUnhandledErrorListener

use of org.apache.flink.runtime.highavailability.zookeeper.CuratorFrameworkWithUnhandledErrorListener in project flink by apache.

the class ZooKeeperUtils method startCuratorFramework.

/**
 * Starts a {@link CuratorFramework} instance and connects it to the given ZooKeeper quorum from
 * a builder.
 *
 * @param builder {@link CuratorFrameworkFactory.Builder} A builder for curatorFramework.
 * @param fatalErrorHandler {@link FatalErrorHandler} fatalErrorHandler to handle unexpected
 *     errors of {@link CuratorFramework}
 * @return {@link CuratorFrameworkWithUnhandledErrorListener} instance
 */
@VisibleForTesting
public static CuratorFrameworkWithUnhandledErrorListener startCuratorFramework(CuratorFrameworkFactory.Builder builder, FatalErrorHandler fatalErrorHandler) {
    CuratorFramework cf = builder.build();
    UnhandledErrorListener unhandledErrorListener = (message, throwable) -> {
        LOG.error("Unhandled error in curator framework, error message: {}", message, throwable);
        // The exception thrown in UnhandledErrorListener will be caught by
        // CuratorFramework. So we mostly trigger exit process or interact with main
        // thread to inform the failure in FatalErrorHandler.
        fatalErrorHandler.onFatalError(throwable);
    };
    cf.getUnhandledErrorListenable().addListener(unhandledErrorListener);
    cf.start();
    return new CuratorFrameworkWithUnhandledErrorListener(cf, unhandledErrorListener);
}

Example 5 with CuratorFrameworkWithUnhandledErrorListener

use of org.apache.flink.runtime.highavailability.zookeeper.CuratorFrameworkWithUnhandledErrorListener in project flink by apache.

the class ZooKeeperLeaderElectionTest method testUnExpectedErrorForwarding.

/**
 * Test that background errors in the {@link LeaderElectionDriver} are correctly forwarded to
 * the {@link FatalErrorHandler}.
 */
@Test
public void testUnExpectedErrorForwarding() throws Exception {
    LeaderElectionDriver leaderElectionDriver = null;
    final TestingLeaderElectionEventHandler electionEventHandler = new TestingLeaderElectionEventHandler(LEADER_ADDRESS);
    final TestingFatalErrorHandler fatalErrorHandler = new TestingFatalErrorHandler();
    final FlinkRuntimeException testException = new FlinkRuntimeException("testUnExpectedErrorForwarding");
    final CuratorFrameworkFactory.Builder curatorFrameworkBuilder = CuratorFrameworkFactory.builder().connectString(testingServer.getConnectString()).retryPolicy(new ExponentialBackoffRetry(1, 0)).aclProvider(new ACLProvider() {

        // trigger background exception
        @Override
        public List<ACL> getDefaultAcl() {
            throw testException;
        }

        @Override
        public List<ACL> getAclForPath(String s) {
            throw testException;
        }
    }).namespace("flink");
    try (CuratorFrameworkWithUnhandledErrorListener curatorFrameworkWrapper = ZooKeeperUtils.startCuratorFramework(curatorFrameworkBuilder, fatalErrorHandler)) {
        CuratorFramework clientWithErrorHandler = curatorFrameworkWrapper.asCuratorFramework();
        assertFalse(fatalErrorHandler.getErrorFuture().isDone());
        leaderElectionDriver = createAndInitLeaderElectionDriver(clientWithErrorHandler, electionEventHandler);
        assertThat(fatalErrorHandler.getErrorFuture().join(), FlinkMatchers.containsCause(testException));
    } finally {
        electionEventHandler.close();
        if (leaderElectionDriver != null) {
            leaderElectionDriver.close();
        }
    }
}