Search in sources :

Example 1 with DefaultExhibitorRestClient

use of org.apache.curator.ensemble.exhibitor.DefaultExhibitorRestClient in project storm by apache.

the class Utils method setupBuilder.

protected static void setupBuilder(CuratorFrameworkFactory.Builder builder, final String zkStr, Map conf, ZookeeperAuthInfo auth) {
    List<String> exhibitorServers = getStrings(conf.get(Config.STORM_EXHIBITOR_SERVERS));
    if (!exhibitorServers.isEmpty()) {
        // use exhibitor servers
        builder.ensembleProvider(new ExhibitorEnsembleProvider(new Exhibitors(exhibitorServers, Utils.getInt(conf.get(Config.STORM_EXHIBITOR_PORT)), new Exhibitors.BackupConnectionStringProvider() {

            @Override
            public String getBackupConnectionString() throws Exception {
                // use zk servers as backup if they exist
                return zkStr;
            }
        }), new DefaultExhibitorRestClient(), Utils.getString(conf.get(Config.STORM_EXHIBITOR_URIPATH)), Utils.getInt(conf.get(Config.STORM_EXHIBITOR_POLL)), new StormBoundedExponentialBackoffRetry(Utils.getInt(conf.get(Config.STORM_EXHIBITOR_RETRY_INTERVAL)), Utils.getInt(conf.get(Config.STORM_EXHIBITOR_RETRY_INTERVAL_CEILING)), Utils.getInt(conf.get(Config.STORM_EXHIBITOR_RETRY_TIMES)))));
    } else {
        builder.connectString(zkStr);
    }
    builder.connectionTimeoutMs(Utils.getInt(conf.get(Config.STORM_ZOOKEEPER_CONNECTION_TIMEOUT))).sessionTimeoutMs(Utils.getInt(conf.get(Config.STORM_ZOOKEEPER_SESSION_TIMEOUT))).retryPolicy(new StormBoundedExponentialBackoffRetry(Utils.getInt(conf.get(Config.STORM_ZOOKEEPER_RETRY_INTERVAL)), Utils.getInt(conf.get(Config.STORM_ZOOKEEPER_RETRY_INTERVAL_CEILING)), Utils.getInt(conf.get(Config.STORM_ZOOKEEPER_RETRY_TIMES))));
    if (auth != null && auth.scheme != null && auth.payload != null) {
        builder.authorization(auth.scheme, auth.payload);
    }
}
Also used : Exhibitors(org.apache.curator.ensemble.exhibitor.Exhibitors) DefaultExhibitorRestClient(org.apache.curator.ensemble.exhibitor.DefaultExhibitorRestClient) ExhibitorEnsembleProvider(org.apache.curator.ensemble.exhibitor.ExhibitorEnsembleProvider) AuthorizationException(org.apache.storm.generated.AuthorizationException) TException(org.apache.thrift.TException) IOException(java.io.IOException) UnknownHostException(java.net.UnknownHostException) ExecuteException(org.apache.commons.exec.ExecuteException) FileNotFoundException(java.io.FileNotFoundException) KeyNotFoundException(org.apache.storm.generated.KeyNotFoundException) InvalidTopologyException(org.apache.storm.generated.InvalidTopologyException) ParseException(org.json.simple.parser.ParseException)

Example 2 with DefaultExhibitorRestClient

use of org.apache.curator.ensemble.exhibitor.DefaultExhibitorRestClient in project druid by druid-io.

the class CuratorModule method makeEnsembleProvider.

@Provides
@LazySingleton
public EnsembleProvider makeEnsembleProvider(CuratorConfig config, ExhibitorConfig exConfig) {
    if (exConfig.getHosts().isEmpty()) {
        return new FixedEnsembleProvider(config.getZkHosts());
    }
    return new ExhibitorEnsembleProvider(new Exhibitors(exConfig.getHosts(), exConfig.getRestPort(), newBackupProvider(config.getZkHosts())), new DefaultExhibitorRestClient(exConfig.getUseSsl()), exConfig.getRestUriPath(), exConfig.getPollingMs(), new BoundedExponentialBackoffRetry(BASE_SLEEP_TIME_MS, MAX_SLEEP_TIME_MS, MAX_RETRIES)) {

        @Override
        public void start() throws Exception {
            log.info("Poll the list of zookeeper servers for initial ensemble");
            this.pollForInitialEnsemble();
            super.start();
        }
    };
}
Also used : Exhibitors(org.apache.curator.ensemble.exhibitor.Exhibitors) DefaultExhibitorRestClient(org.apache.curator.ensemble.exhibitor.DefaultExhibitorRestClient) ExhibitorEnsembleProvider(org.apache.curator.ensemble.exhibitor.ExhibitorEnsembleProvider) FixedEnsembleProvider(org.apache.curator.ensemble.fixed.FixedEnsembleProvider) BoundedExponentialBackoffRetry(org.apache.curator.retry.BoundedExponentialBackoffRetry) LazySingleton(io.druid.guice.LazySingleton) Provides(com.google.inject.Provides)

Example 3 with DefaultExhibitorRestClient

use of org.apache.curator.ensemble.exhibitor.DefaultExhibitorRestClient in project nakadi by zalando.

the class ZooKeeperHolder method initExhibitor.

private void initExhibitor() throws Exception {
    final RetryPolicy retryPolicy = new ExponentialBackoffRetry(EXHIBITOR_RETRY_TIME, EXHIBITOR_RETRY_MAX);
    final EnsembleProvider ensembleProvider;
    if (exhibitorAddresses != null) {
        final Collection<String> exhibitorHosts = Arrays.asList(exhibitorAddresses.split("\\s*,\\s*"));
        final Exhibitors exhibitors = new Exhibitors(exhibitorHosts, exhibitorPort, () -> zookeeperBrokers + zookeeperKafkaNamespace);
        final ExhibitorRestClient exhibitorRestClient = new DefaultExhibitorRestClient();
        ensembleProvider = new ExhibitorEnsembleProvider(exhibitors, exhibitorRestClient, "/exhibitor/v1/cluster/list", EXHIBITOR_POLLING_MS, retryPolicy);
        ((ExhibitorEnsembleProvider) ensembleProvider).pollForInitialEnsemble();
    } else {
        ensembleProvider = new FixedEnsembleProvider(zookeeperBrokers + zookeeperKafkaNamespace);
    }
    zooKeeper = CuratorFrameworkFactory.builder().ensembleProvider(ensembleProvider).retryPolicy(retryPolicy).build();
    zooKeeper.start();
}
Also used : ExponentialBackoffRetry(org.apache.curator.retry.ExponentialBackoffRetry) Exhibitors(org.apache.curator.ensemble.exhibitor.Exhibitors) DefaultExhibitorRestClient(org.apache.curator.ensemble.exhibitor.DefaultExhibitorRestClient) ExhibitorRestClient(org.apache.curator.ensemble.exhibitor.ExhibitorRestClient) DefaultExhibitorRestClient(org.apache.curator.ensemble.exhibitor.DefaultExhibitorRestClient) RetryPolicy(org.apache.curator.RetryPolicy) FixedEnsembleProvider(org.apache.curator.ensemble.fixed.FixedEnsembleProvider) FixedEnsembleProvider(org.apache.curator.ensemble.fixed.FixedEnsembleProvider) EnsembleProvider(org.apache.curator.ensemble.EnsembleProvider)

Example 4 with DefaultExhibitorRestClient

use of org.apache.curator.ensemble.exhibitor.DefaultExhibitorRestClient in project druid by druid-io.

the class CuratorModule method makeEnsembleProvider.

@Provides
@LazySingleton
public EnsembleProvider makeEnsembleProvider(CuratorConfig config, ExhibitorConfig exConfig) {
    if (exConfig.getHosts().isEmpty()) {
        return new FixedEnsembleProvider(config.getZkHosts());
    }
    RetryPolicy retryPolicy = new BoundedExponentialBackoffRetry(BASE_SLEEP_TIME_MS, MAX_SLEEP_TIME_MS, MAX_RETRIES);
    return new ExhibitorEnsembleProvider(new Exhibitors(exConfig.getHosts(), exConfig.getRestPort(), newBackupProvider(config.getZkHosts())), new DefaultExhibitorRestClient(exConfig.getUseSsl()), exConfig.getRestUriPath(), exConfig.getPollingMs(), retryPolicy) {

        @Override
        public void start() throws Exception {
            log.debug("Polling the list of ZooKeeper servers for the initial ensemble");
            this.pollForInitialEnsemble();
            super.start();
        }
    };
}
Also used : Exhibitors(org.apache.curator.ensemble.exhibitor.Exhibitors) DefaultExhibitorRestClient(org.apache.curator.ensemble.exhibitor.DefaultExhibitorRestClient) ExhibitorEnsembleProvider(org.apache.curator.ensemble.exhibitor.ExhibitorEnsembleProvider) RetryPolicy(org.apache.curator.RetryPolicy) FixedEnsembleProvider(org.apache.curator.ensemble.fixed.FixedEnsembleProvider) BoundedExponentialBackoffRetry(org.apache.curator.retry.BoundedExponentialBackoffRetry) LazySingleton(org.apache.druid.guice.LazySingleton) Provides(com.google.inject.Provides)

Example 5 with DefaultExhibitorRestClient

use of org.apache.curator.ensemble.exhibitor.DefaultExhibitorRestClient in project exhibitor by soabase.

the class ExhibitorCreator method makeCurator.

private CuratorFramework makeCurator(final String connectString, int baseSleepTimeMs, int maxRetries, int exhibitorPort, String exhibitorRestPath, int pollingMs) {
    List<String> hostnames = Lists.newArrayList();
    String[] parts = connectString.split(",");
    for (String spec : parts) {
        String[] subParts = spec.split(":");
        try {
            if (subParts.length != 2) {
                log.error("Bad connection string: " + connectString);
                return null;
            }
        } catch (NumberFormatException e) {
            log.error("Bad connection string: " + connectString);
            return null;
        }
        hostnames.add(subParts[0]);
    }
    ExponentialBackoffRetry retryPolicy = new ExponentialBackoffRetry(baseSleepTimeMs, maxRetries);
    Exhibitors.BackupConnectionStringProvider backupConnectionStringProvider = new Exhibitors.BackupConnectionStringProvider() {

        @Override
        public String getBackupConnectionString() throws Exception {
            return connectString;
        }
    };
    CuratorFrameworkFactory.Builder builder = CuratorFrameworkFactory.builder().connectString(connectString).retryPolicy(retryPolicy);
    if (exhibitorPort > 0) {
        Exhibitors exhibitors = new Exhibitors(hostnames, exhibitorPort, backupConnectionStringProvider);
        ExhibitorEnsembleProvider ensembleProvider = new ExhibitorEnsembleProvider(exhibitors, new DefaultExhibitorRestClient(), exhibitorRestPath + "exhibitor/v1/cluster/list", pollingMs, retryPolicy);
        builder = builder.ensembleProvider(ensembleProvider);
    } else {
        log.warn("Exhibitor on the shared ZooKeeper config ensemble is not being used.");
    }
    return builder.build();
}
Also used : CuratorFrameworkFactory(org.apache.curator.framework.CuratorFrameworkFactory) ExponentialBackoffRetry(org.apache.curator.retry.ExponentialBackoffRetry) Exhibitors(org.apache.curator.ensemble.exhibitor.Exhibitors) DefaultExhibitorRestClient(org.apache.curator.ensemble.exhibitor.DefaultExhibitorRestClient) ExhibitorEnsembleProvider(org.apache.curator.ensemble.exhibitor.ExhibitorEnsembleProvider)

Aggregations

DefaultExhibitorRestClient (org.apache.curator.ensemble.exhibitor.DefaultExhibitorRestClient)5 Exhibitors (org.apache.curator.ensemble.exhibitor.Exhibitors)5 ExhibitorEnsembleProvider (org.apache.curator.ensemble.exhibitor.ExhibitorEnsembleProvider)4 FixedEnsembleProvider (org.apache.curator.ensemble.fixed.FixedEnsembleProvider)3 Provides (com.google.inject.Provides)2 RetryPolicy (org.apache.curator.RetryPolicy)2 BoundedExponentialBackoffRetry (org.apache.curator.retry.BoundedExponentialBackoffRetry)2 ExponentialBackoffRetry (org.apache.curator.retry.ExponentialBackoffRetry)2 LazySingleton (io.druid.guice.LazySingleton)1 FileNotFoundException (java.io.FileNotFoundException)1 IOException (java.io.IOException)1 UnknownHostException (java.net.UnknownHostException)1 ExecuteException (org.apache.commons.exec.ExecuteException)1 EnsembleProvider (org.apache.curator.ensemble.EnsembleProvider)1 ExhibitorRestClient (org.apache.curator.ensemble.exhibitor.ExhibitorRestClient)1 CuratorFrameworkFactory (org.apache.curator.framework.CuratorFrameworkFactory)1 LazySingleton (org.apache.druid.guice.LazySingleton)1 AuthorizationException (org.apache.storm.generated.AuthorizationException)1 InvalidTopologyException (org.apache.storm.generated.InvalidTopologyException)1 KeyNotFoundException (org.apache.storm.generated.KeyNotFoundException)1