Search in sources :

Example 6 with ACLProvider

use of org.apache.flink.shaded.curator5.org.apache.curator.framework.api.ACLProvider in project flink by apache.

the class ZooKeeperLeaderElectionTest method testUnExpectedErrorForwarding.

/**
 * Test that background errors in the {@link LeaderElectionDriver} are correctly forwarded to
 * the {@link FatalErrorHandler}.
 */
@Test
public void testUnExpectedErrorForwarding() throws Exception {
    LeaderElectionDriver leaderElectionDriver = null;
    final TestingLeaderElectionEventHandler electionEventHandler = new TestingLeaderElectionEventHandler(LEADER_ADDRESS);
    final TestingFatalErrorHandler fatalErrorHandler = new TestingFatalErrorHandler();
    final FlinkRuntimeException testException = new FlinkRuntimeException("testUnExpectedErrorForwarding");
    final CuratorFrameworkFactory.Builder curatorFrameworkBuilder = CuratorFrameworkFactory.builder().connectString(testingServer.getConnectString()).retryPolicy(new ExponentialBackoffRetry(1, 0)).aclProvider(new ACLProvider() {

        // trigger background exception
        @Override
        public List<ACL> getDefaultAcl() {
            throw testException;
        }

        @Override
        public List<ACL> getAclForPath(String s) {
            throw testException;
        }
    }).namespace("flink");
    try (CuratorFrameworkWithUnhandledErrorListener curatorFrameworkWrapper = ZooKeeperUtils.startCuratorFramework(curatorFrameworkBuilder, fatalErrorHandler)) {
        CuratorFramework clientWithErrorHandler = curatorFrameworkWrapper.asCuratorFramework();
        assertFalse(fatalErrorHandler.getErrorFuture().isDone());
        leaderElectionDriver = createAndInitLeaderElectionDriver(clientWithErrorHandler, electionEventHandler);
        assertThat(fatalErrorHandler.getErrorFuture().join(), FlinkMatchers.containsCause(testException));
    } finally {
        electionEventHandler.close();
        if (leaderElectionDriver != null) {
            leaderElectionDriver.close();
        }
    }
}
Also used : TestingFatalErrorHandler(org.apache.flink.runtime.util.TestingFatalErrorHandler) ACLProvider(org.apache.flink.shaded.curator5.org.apache.curator.framework.api.ACLProvider) CuratorFramework(org.apache.flink.shaded.curator5.org.apache.curator.framework.CuratorFramework) CuratorFrameworkFactory(org.apache.flink.shaded.curator5.org.apache.curator.framework.CuratorFrameworkFactory) ExponentialBackoffRetry(org.apache.flink.shaded.curator5.org.apache.curator.retry.ExponentialBackoffRetry) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) CuratorFrameworkWithUnhandledErrorListener(org.apache.flink.runtime.highavailability.zookeeper.CuratorFrameworkWithUnhandledErrorListener) ACL(org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.data.ACL) Mockito.anyString(org.mockito.Mockito.anyString) Test(org.junit.Test)

Example 7 with ACLProvider

use of org.apache.flink.shaded.curator5.org.apache.curator.framework.api.ACLProvider in project helios by spotify.

the class ZooKeeperAclInitializer method initializeAcl.

static void initializeAcl(final String zooKeeperConnectionString, final String zooKeeperClusterId, final String masterUser, final String masterPassword, final String agentUser, final String agentPassword) throws KeeperException {
    final ACLProvider aclProvider = heliosAclProvider(masterUser, digest(masterUser, masterPassword), agentUser, digest(agentUser, agentPassword));
    final List<AuthInfo> authorization = Lists.newArrayList(new AuthInfo("digest", String.format("%s:%s", masterUser, masterPassword).getBytes()));
    final RetryPolicy zooKeeperRetryPolicy = new ExponentialBackoffRetry(1000, 3);
    final CuratorFramework curator = new CuratorClientFactoryImpl().newClient(zooKeeperConnectionString, (int) TimeUnit.SECONDS.toMillis(60), (int) TimeUnit.SECONDS.toMillis(15), zooKeeperRetryPolicy, aclProvider, authorization);
    final ZooKeeperClient client = new DefaultZooKeeperClient(curator, zooKeeperClusterId);
    try {
        client.start();
        initializeAclRecursive(client, "/", aclProvider);
    } finally {
        client.close();
    }
}
Also used : ACLProvider(org.apache.curator.framework.api.ACLProvider) AuthInfo(org.apache.curator.framework.AuthInfo) CuratorFramework(org.apache.curator.framework.CuratorFramework) ZooKeeperClient(com.spotify.helios.servicescommon.coordination.ZooKeeperClient) DefaultZooKeeperClient(com.spotify.helios.servicescommon.coordination.DefaultZooKeeperClient) ExponentialBackoffRetry(org.apache.curator.retry.ExponentialBackoffRetry) CuratorClientFactoryImpl(com.spotify.helios.servicescommon.coordination.CuratorClientFactoryImpl) DefaultZooKeeperClient(com.spotify.helios.servicescommon.coordination.DefaultZooKeeperClient) RetryPolicy(org.apache.curator.RetryPolicy)

Example 8 with ACLProvider

use of org.apache.flink.shaded.curator5.org.apache.curator.framework.api.ACLProvider in project helios by spotify.

the class MasterService method setupZookeeperClient.

/**
 * Create a Zookeeper client and create the control and state nodes if needed.
 *
 * @param config The service configuration.
 *
 * @return A zookeeper client.
 */
private ZooKeeperClient setupZookeeperClient(final MasterConfig config) {
    ACLProvider aclProvider = null;
    List<AuthInfo> authorization = null;
    final String masterUser = config.getZookeeperAclMasterUser();
    final String masterPassword = config.getZooKeeperAclMasterPassword();
    final String agentUser = config.getZookeeperAclAgentUser();
    final String agentDigest = config.getZooKeeperAclAgentDigest();
    if (!isNullOrEmpty(masterPassword)) {
        if (isNullOrEmpty(masterUser)) {
            throw new HeliosRuntimeException("Master username must be set if a password is set");
        }
        authorization = Lists.newArrayList(new AuthInfo("digest", String.format("%s:%s", masterUser, masterPassword).getBytes()));
    }
    if (config.isZooKeeperEnableAcls()) {
        if (isNullOrEmpty(masterUser) || isNullOrEmpty(masterPassword)) {
            throw new HeliosRuntimeException("ZooKeeper ACLs enabled but master username and/or password not set");
        }
        if (isNullOrEmpty(agentUser) || isNullOrEmpty(agentDigest)) {
            throw new HeliosRuntimeException("ZooKeeper ACLs enabled but agent username and/or digest not set");
        }
        aclProvider = heliosAclProvider(masterUser, digest(masterUser, masterPassword), agentUser, agentDigest);
    }
    final RetryPolicy zooKeeperRetryPolicy = new ExponentialBackoffRetry(1000, 3);
    final CuratorFramework curator = curatorClientFactory.newClient(config.getZooKeeperConnectionString(), config.getZooKeeperSessionTimeoutMillis(), config.getZooKeeperConnectionTimeoutMillis(), zooKeeperRetryPolicy, aclProvider, authorization);
    final ZooKeeperClient client = new DefaultZooKeeperClient(curator, config.getZooKeeperClusterId());
    client.start();
    zkRegistrar = ZooKeeperRegistrarService.newBuilder().setZooKeeperClient(client).setZooKeeperRegistrar(new MasterZooKeeperRegistrar(config.getName())).build();
    // place where we have access to the ACL provider.
    if (aclProvider != null) {
        // effects are limited to a spurious log line.
        try {
            final List<ACL> curAcls = client.getAcl("/");
            final List<ACL> wantedAcls = aclProvider.getAclForPath("/");
            if (!Sets.newHashSet(curAcls).equals(Sets.newHashSet(wantedAcls))) {
                log.info("Current ACL's on the zookeeper root node differ from desired, updating: {} -> {}", curAcls, wantedAcls);
                client.getCuratorFramework().setACL().withACL(wantedAcls).forPath("/");
            }
        } catch (Exception e) {
            log.error("Failed to get/set ACLs on the zookeeper root node", e);
        }
    }
    return client;
}
Also used : ACLProvider(org.apache.curator.framework.api.ACLProvider) AuthInfo(org.apache.curator.framework.AuthInfo) ExponentialBackoffRetry(org.apache.curator.retry.ExponentialBackoffRetry) HeliosRuntimeException(com.spotify.helios.common.HeliosRuntimeException) ACL(org.apache.zookeeper.data.ACL) DefaultZooKeeperClient(com.spotify.helios.servicescommon.coordination.DefaultZooKeeperClient) HeliosRuntimeException(com.spotify.helios.common.HeliosRuntimeException) ConfigurationException(io.dropwizard.configuration.ConfigurationException) IOException(java.io.IOException) CuratorFramework(org.apache.curator.framework.CuratorFramework) ZooKeeperClient(com.spotify.helios.servicescommon.coordination.ZooKeeperClient) DefaultZooKeeperClient(com.spotify.helios.servicescommon.coordination.DefaultZooKeeperClient) RetryPolicy(org.apache.curator.RetryPolicy)

Example 9 with ACLProvider

use of org.apache.flink.shaded.curator5.org.apache.curator.framework.api.ACLProvider in project exhibitor by soabase.

the class ExhibitorCreator method getAclProvider.

private ACLProvider getAclProvider(ExhibitorCLI cli, String aclId, String aclScheme, String aclPerms) throws ExhibitorCreatorExit {
    int perms;
    if (notNullOrEmpty(aclPerms)) {
        perms = 0;
        for (String verb : aclPerms.split(",")) {
            verb = verb.trim();
            if (verb.equalsIgnoreCase("read")) {
                perms |= ZooDefs.Perms.READ;
            } else if (verb.equalsIgnoreCase("write")) {
                perms |= ZooDefs.Perms.WRITE;
            } else if (verb.equalsIgnoreCase("create")) {
                perms |= ZooDefs.Perms.CREATE;
            } else if (verb.equalsIgnoreCase("delete")) {
                perms |= ZooDefs.Perms.DELETE;
            } else if (verb.equalsIgnoreCase("admin")) {
                perms |= ZooDefs.Perms.ADMIN;
            } else {
                log.error("Unknown ACL perm value: " + verb);
                throw new ExhibitorCreatorExit(cli);
            }
        }
    } else {
        perms = ZooDefs.Perms.ALL;
    }
    if (aclId == null) {
        aclId = "";
    }
    if (aclScheme == null) {
        aclScheme = "";
    }
    final ACL acl = new ACL(perms, new Id(aclScheme, aclId));
    return new ACLProvider() {

        @Override
        public List<ACL> getDefaultAcl() {
            return Collections.singletonList(acl);
        }

        @Override
        public List<ACL> getAclForPath(String path) {
            return Collections.singletonList(acl);
        }
    };
}
Also used : ACLProvider(org.apache.curator.framework.api.ACLProvider) ACL(org.apache.zookeeper.data.ACL) Id(org.apache.zookeeper.data.Id) Constraint(org.mortbay.jetty.security.Constraint)

Example 10 with ACLProvider

use of org.apache.flink.shaded.curator5.org.apache.curator.framework.api.ACLProvider in project incubator-atlas by apache.

the class CuratorFactoryTest method shouldAddAclProviderWithRightACL.

@Test
public void shouldAddAclProviderWithRightACL() {
    when(zookeeperProperties.hasAcl()).thenReturn(true);
    when(zookeeperProperties.getAcl()).thenReturn("sasl:myclient@EXAMPLE.COM");
    when(zookeeperProperties.hasAuth()).thenReturn(false);
    CuratorFactory curatorFactory = new CuratorFactory(configuration) {

        @Override
        protected void initializeCuratorFramework() {
        }
    };
    curatorFactory.enhanceBuilderWithSecurityParameters(zookeeperProperties, builder);
    verify(builder).aclProvider(argThat(new ArgumentMatcher<ACLProvider>() {

        @Override
        public boolean matches(Object o) {
            ACLProvider aclProvider = (ACLProvider) o;
            ACL acl = aclProvider.getDefaultAcl().get(0);
            return acl.getId().getId().equals("myclient@EXAMPLE.COM") && acl.getId().getScheme().equals("sasl");
        }
    }));
}
Also used : ACLProvider(org.apache.curator.framework.api.ACLProvider) ArgumentMatcher(org.mockito.ArgumentMatcher) ACL(org.apache.zookeeper.data.ACL) Test(org.testng.annotations.Test)

Aggregations

ACLProvider (org.apache.curator.framework.api.ACLProvider)26 ACL (org.apache.zookeeper.data.ACL)15 ExponentialBackoffRetry (org.apache.curator.retry.ExponentialBackoffRetry)12 CuratorFramework (org.apache.curator.framework.CuratorFramework)10 DefaultACLProvider (org.apache.curator.framework.imps.DefaultACLProvider)8 RetryPolicy (org.apache.curator.RetryPolicy)7 Test (org.junit.Test)6 List (java.util.List)5 AuthInfo (org.apache.curator.framework.AuthInfo)5 ZooKeeperClientConfig (org.apache.nifi.controller.cluster.ZooKeeperClientConfig)4 NiFiProperties (org.apache.nifi.util.NiFiProperties)4 Id (org.apache.zookeeper.data.Id)4 Test (org.testng.annotations.Test)4 DefaultZooKeeperClient (com.spotify.helios.servicescommon.coordination.DefaultZooKeeperClient)3 ZooKeeperClient (com.spotify.helios.servicescommon.coordination.ZooKeeperClient)3 CuratorFrameworkFactory (org.apache.curator.framework.CuratorFrameworkFactory)3 VisibleForTesting (com.google.common.annotations.VisibleForTesting)2 HeliosRuntimeException (com.spotify.helios.common.HeliosRuntimeException)2 CuratorClientFactoryImpl (com.spotify.helios.servicescommon.coordination.CuratorClientFactoryImpl)2 ArrayList (java.util.ArrayList)2