Search in sources :

Example 6 with PulsarServerException

use of org.apache.pulsar.broker.PulsarServerException in project incubator-pulsar by apache.

the class LocalZooKeeperCacheService method createPolicies.

/**
 * Create LocalPolicies with bundle-data in LocalZookeeper by fetching it from GlobalZookeeper
 *
 * @param path
 *            znode path
 * @param readFromGlobal
 *            if true copy policies from global zk to local zk else create a new znode with empty {@link Policies}
 * @throws Exception
 */
@SuppressWarnings("deprecation")
public CompletableFuture<Optional<LocalPolicies>> createPolicies(String path, boolean readFromGlobal) {
    CompletableFuture<Optional<LocalPolicies>> future = new CompletableFuture<>();
    if (path == null || !path.startsWith(LOCAL_POLICIES_ROOT)) {
        future.completeExceptionally(new IllegalArgumentException("Invalid path of local policies " + path));
        return future;
    }
    if (LOG.isDebugEnabled()) {
        LOG.debug("Creating local namespace policies for {} - readFromGlobal: {}", path, readFromGlobal);
    }
    CompletableFuture<Optional<LocalPolicies>> readFromGlobalFuture = new CompletableFuture<>();
    if (readFromGlobal) {
        String globalPath = joinPath(POLICIES_ROOT, path.substring(path.indexOf(LOCAL_POLICIES_ROOT) + LOCAL_POLICIES_ROOT.length() + 1));
        checkNotNull(configurationCacheService);
        checkNotNull(configurationCacheService.policiesCache());
        checkNotNull(configurationCacheService.policiesCache().getAsync(globalPath));
        configurationCacheService.policiesCache().getAsync(globalPath).thenAccept(policies -> {
            if (policies.isPresent()) {
                // Copying global bundles information to local policies
                LocalPolicies localPolicies = new LocalPolicies();
                localPolicies.bundles = policies.get().bundles;
                readFromGlobalFuture.complete(Optional.of(localPolicies));
            } else {
                // Policies are not present in global zk
                if (LOG.isDebugEnabled()) {
                    LOG.debug("Global policies not found at {}", globalPath);
                }
                readFromGlobalFuture.complete(Optional.empty());
            }
        }).exceptionally(ex -> {
            future.completeExceptionally(ex);
            return null;
        });
    } else {
        // Use default local policies
        readFromGlobalFuture.complete(Optional.of(new LocalPolicies()));
    }
    readFromGlobalFuture.thenAccept(localPolicies -> {
        if (!localPolicies.isPresent()) {
            future.complete(Optional.empty());
        }
        // When we have the updated localPolicies, we can write them back in local ZK
        byte[] content;
        try {
            content = ObjectMapperFactory.getThreadLocal().writeValueAsBytes(localPolicies.get());
        } catch (Throwable t) {
            // Failed to serialize to json
            future.completeExceptionally(t);
            return;
        }
        ZkUtils.asyncCreateFullPathOptimistic(cache.getZooKeeper(), path, content, Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT, (rc, path1, ctx, name) -> {
            if (rc == KeeperException.Code.OK.intValue() || rc == KeeperException.Code.NODEEXISTS.intValue()) {
                LOG.info("Successfully copyied bundles data to local zk at {}", path);
                future.complete(localPolicies);
            } else {
                LOG.error("Failed to create policies for {} in local zookeeper: {}", path, KeeperException.Code.get(rc));
                future.completeExceptionally(new PulsarServerException(KeeperException.create(rc)));
            }
        }, null);
    }).exceptionally(ex -> {
        future.completeExceptionally(ex);
        return null;
    });
    return future;
}
Also used : CreateMode(org.apache.zookeeper.CreateMode) ZooKeeper(org.apache.zookeeper.ZooKeeper) Logger(org.slf4j.Logger) KeeperException(org.apache.zookeeper.KeeperException) Ids(org.apache.zookeeper.ZooDefs.Ids) ObjectMapperFactory(org.apache.pulsar.common.util.ObjectMapperFactory) Preconditions.checkNotNull(com.google.common.base.Preconditions.checkNotNull) LoggerFactory(org.slf4j.LoggerFactory) NamespaceEphemeralData(org.apache.pulsar.broker.namespace.NamespaceEphemeralData) LocalPolicies(org.apache.pulsar.common.policies.data.LocalPolicies) ZooKeeperChildrenCache(org.apache.pulsar.zookeeper.ZooKeeperChildrenCache) CompletableFuture(java.util.concurrent.CompletableFuture) Stat(org.apache.zookeeper.data.Stat) PulsarWebResource.joinPath(org.apache.pulsar.broker.web.PulsarWebResource.joinPath) Maps(com.google.common.collect.Maps) ZkUtils(org.apache.bookkeeper.util.ZkUtils) Policies(org.apache.pulsar.common.policies.data.Policies) PulsarServerException(org.apache.pulsar.broker.PulsarServerException) ZooKeeperCache(org.apache.pulsar.zookeeper.ZooKeeperCache) ZooKeeperDataCache(org.apache.pulsar.zookeeper.ZooKeeperDataCache) Entry(java.util.Map.Entry) Optional(java.util.Optional) POLICIES_ROOT(org.apache.pulsar.broker.cache.ConfigurationCacheService.POLICIES_ROOT) PulsarServerException(org.apache.pulsar.broker.PulsarServerException) CompletableFuture(java.util.concurrent.CompletableFuture) Optional(java.util.Optional) LocalPolicies(org.apache.pulsar.common.policies.data.LocalPolicies)

Example 7 with PulsarServerException

use of org.apache.pulsar.broker.PulsarServerException in project incubator-pulsar by apache.

the class WebSocketService method start.

public void start() throws PulsarServerException, PulsarClientException, MalformedURLException, ServletException, DeploymentException {
    if (isNotBlank(config.getGlobalZookeeperServers())) {
        this.globalZkCache = new GlobalZooKeeperCache(getZooKeeperClientFactory(), (int) config.getZooKeeperSessionTimeoutMillis(), config.getGlobalZookeeperServers(), this.orderedExecutor, this.executor);
        try {
            this.globalZkCache.start();
        } catch (IOException e) {
            throw new PulsarServerException(e);
        }
        this.configurationCacheService = new ConfigurationCacheService(getGlobalZkCache());
        log.info("Global Zookeeper cache started");
    }
    // start authorizationService
    if (config.isAuthorizationEnabled()) {
        if (configurationCacheService == null) {
            throw new PulsarServerException("Failed to initialize authorization manager due to empty GlobalZookeeperServers");
        }
        authorizationService = new AuthorizationService(this.config, configurationCacheService);
    }
    // start authentication service
    authenticationService = new AuthenticationService(this.config);
    log.info("Pulsar WebSocket Service started");
}
Also used : PulsarServerException(org.apache.pulsar.broker.PulsarServerException) GlobalZooKeeperCache(org.apache.pulsar.zookeeper.GlobalZooKeeperCache) AuthorizationService(org.apache.pulsar.broker.authorization.AuthorizationService) ConfigurationCacheService(org.apache.pulsar.broker.cache.ConfigurationCacheService) IOException(java.io.IOException) AuthenticationService(org.apache.pulsar.broker.authentication.AuthenticationService)

Example 8 with PulsarServerException

use of org.apache.pulsar.broker.PulsarServerException in project incubator-pulsar by apache.

the class AdminApiTest method namespaces.

@Test(invocationCount = 1)
public void namespaces() throws PulsarAdminException, PulsarServerException, Exception {
    admin.clusters().createCluster("usw", new ClusterData());
    PropertyAdmin propertyAdmin = new PropertyAdmin(Lists.newArrayList("role1", "role2"), Sets.newHashSet("use", "usw"));
    admin.properties().updateProperty("prop-xyz", propertyAdmin);
    assertEquals(admin.namespaces().getPolicies("prop-xyz/use/ns1").bundles, Policies.defaultBundle());
    admin.namespaces().createNamespace("prop-xyz/use/ns2");
    admin.namespaces().createNamespace("prop-xyz/use/ns3", 4);
    assertEquals(admin.namespaces().getPolicies("prop-xyz/use/ns3").bundles.numBundles, 4);
    assertEquals(admin.namespaces().getPolicies("prop-xyz/use/ns3").bundles.boundaries.size(), 5);
    admin.namespaces().deleteNamespace("prop-xyz/use/ns3");
    try {
        admin.namespaces().createNamespace("non-existing/usw/ns1");
        fail("Should not have passed");
    } catch (NotFoundException e) {
    // Ok
    }
    assertEquals(admin.namespaces().getNamespaces("prop-xyz"), Lists.newArrayList("prop-xyz/use/ns1", "prop-xyz/use/ns2"));
    assertEquals(admin.namespaces().getNamespaces("prop-xyz", "use"), Lists.newArrayList("prop-xyz/use/ns1", "prop-xyz/use/ns2"));
    try {
        admin.namespaces().createNamespace("prop-xyz/usc/ns1");
        fail("Should not have passed");
    } catch (NotAuthorizedException e) {
    // Ok, got the non authorized exception since usc cluster is not in the allowed clusters list.
    }
    admin.namespaces().grantPermissionOnNamespace("prop-xyz/use/ns1", "my-role", EnumSet.allOf(AuthAction.class));
    Policies policies = new Policies();
    policies.auth_policies.namespace_auth.put("my-role", EnumSet.allOf(AuthAction.class));
    assertEquals(admin.namespaces().getPolicies("prop-xyz/use/ns1"), policies);
    assertEquals(admin.namespaces().getPermissions("prop-xyz/use/ns1"), policies.auth_policies.namespace_auth);
    assertEquals(admin.namespaces().getTopics("prop-xyz/use/ns1"), Lists.newArrayList());
    admin.namespaces().revokePermissionsOnNamespace("prop-xyz/use/ns1", "my-role");
    policies.auth_policies.namespace_auth.remove("my-role");
    assertEquals(admin.namespaces().getPolicies("prop-xyz/use/ns1"), policies);
    assertEquals(admin.namespaces().getPersistence("prop-xyz/use/ns1"), new PersistencePolicies(1, 1, 1, 0.0));
    admin.namespaces().setPersistence("prop-xyz/use/ns1", new PersistencePolicies(3, 2, 1, 10.0));
    assertEquals(admin.namespaces().getPersistence("prop-xyz/use/ns1"), new PersistencePolicies(3, 2, 1, 10.0));
    // Force topic creation and namespace being loaded
    Producer<byte[]> producer = pulsarClient.newProducer().topic("persistent://prop-xyz/use/ns1/my-topic").create();
    producer.close();
    admin.persistentTopics().delete("persistent://prop-xyz/use/ns1/my-topic");
    admin.namespaces().unloadNamespaceBundle("prop-xyz/use/ns1", "0x00000000_0xffffffff");
    NamespaceName ns = NamespaceName.get("prop-xyz/use/ns1");
    // Now, w/ bundle policies, we will use default bundle
    NamespaceBundle defaultBundle = bundleFactory.getFullBundle(ns);
    int i = 0;
    for (; i < 10; i++) {
        Optional<NamespaceEphemeralData> data1 = pulsar.getNamespaceService().getOwnershipCache().getOwnerAsync(defaultBundle).get();
        if (!data1.isPresent()) {
            // Already unloaded
            break;
        }
        LOG.info("Waiting for unload namespace {} to complete. Current service unit isDisabled: {}", defaultBundle, data1.get().isDisabled());
        Thread.sleep(1000);
    }
    assertTrue(i < 10);
    admin.namespaces().deleteNamespace("prop-xyz/use/ns1");
    assertEquals(admin.namespaces().getNamespaces("prop-xyz", "use"), Lists.newArrayList("prop-xyz/use/ns2"));
    try {
        admin.namespaces().unload("prop-xyz/use/ns1");
        fail("should have raised exception");
    } catch (Exception e) {
    // OK excepted
    }
    // Force topic creation and namespace being loaded
    producer = pulsarClient.newProducer().topic("persistent://prop-xyz/use/ns2/my-topic").create();
    producer.close();
    admin.persistentTopics().delete("persistent://prop-xyz/use/ns2/my-topic");
// both unload and delete should succeed for ns2 on other broker with a redirect
// otheradmin.namespaces().unload("prop-xyz/use/ns2");
}
Also used : NamespaceBundle(org.apache.pulsar.common.naming.NamespaceBundle) PersistencePolicies(org.apache.pulsar.common.policies.data.PersistencePolicies) RetentionPolicies(org.apache.pulsar.common.policies.data.RetentionPolicies) Policies(org.apache.pulsar.common.policies.data.Policies) PersistencePolicies(org.apache.pulsar.common.policies.data.PersistencePolicies) NotFoundException(org.apache.pulsar.client.admin.PulsarAdminException.NotFoundException) NotAuthorizedException(org.apache.pulsar.client.admin.PulsarAdminException.NotAuthorizedException) NotFoundException(org.apache.pulsar.client.admin.PulsarAdminException.NotFoundException) PreconditionFailedException(org.apache.pulsar.client.admin.PulsarAdminException.PreconditionFailedException) ConflictException(org.apache.pulsar.client.admin.PulsarAdminException.ConflictException) PulsarAdminException(org.apache.pulsar.client.admin.PulsarAdminException) NotAuthorizedException(org.apache.pulsar.client.admin.PulsarAdminException.NotAuthorizedException) PulsarServerException(org.apache.pulsar.broker.PulsarServerException) AuthAction(org.apache.pulsar.common.policies.data.AuthAction) NamespaceName(org.apache.pulsar.common.naming.NamespaceName) ClusterData(org.apache.pulsar.common.policies.data.ClusterData) PropertyAdmin(org.apache.pulsar.common.policies.data.PropertyAdmin) NamespaceEphemeralData(org.apache.pulsar.broker.namespace.NamespaceEphemeralData) Test(org.testng.annotations.Test) MockedPulsarServiceBaseTest(org.apache.pulsar.broker.auth.MockedPulsarServiceBaseTest)

Example 9 with PulsarServerException

use of org.apache.pulsar.broker.PulsarServerException in project incubator-pulsar by apache.

the class WebService method close.

@Override
public void close() throws PulsarServerException {
    try {
        server.stop();
        webServiceExecutor.shutdown();
        log.info("Web service closed");
    } catch (Exception e) {
        throw new PulsarServerException(e);
    }
}
Also used : PulsarServerException(org.apache.pulsar.broker.PulsarServerException) GeneralSecurityException(java.security.GeneralSecurityException) PulsarServerException(org.apache.pulsar.broker.PulsarServerException)

Example 10 with PulsarServerException

use of org.apache.pulsar.broker.PulsarServerException in project incubator-pulsar by apache.

the class ServerConnection method sendLookupResponse.

private void sendLookupResponse(long requestId) {
    try {
        LoadManagerReport availableBroker = service.getDiscoveryProvider().nextBroker();
        ctx.writeAndFlush(Commands.newLookupResponse(availableBroker.getPulsarServiceUrl(), availableBroker.getPulsarServiceUrlTls(), false, Redirect, requestId, false));
    } catch (PulsarServerException e) {
        LOG.warn("[{}] Failed to get next active broker {}", remoteAddress, e.getMessage(), e);
        ctx.writeAndFlush(Commands.newLookupErrorResponse(ServerError.ServiceNotReady, e.getMessage(), requestId));
    }
}
Also used : LoadManagerReport(org.apache.pulsar.policies.data.loadbalancer.LoadManagerReport) PulsarServerException(org.apache.pulsar.broker.PulsarServerException)

Aggregations

PulsarServerException (org.apache.pulsar.broker.PulsarServerException)21 KeeperException (org.apache.zookeeper.KeeperException)7 CompletableFuture (java.util.concurrent.CompletableFuture)6 NamespaceName (org.apache.pulsar.common.naming.NamespaceName)6 Policies (org.apache.pulsar.common.policies.data.Policies)5 IOException (java.io.IOException)4 Logger (org.slf4j.Logger)4 LoggerFactory (org.slf4j.LoggerFactory)4 Preconditions.checkNotNull (com.google.common.base.Preconditions.checkNotNull)3 ByteBuf (io.netty.buffer.ByteBuf)3 GeneralSecurityException (java.security.GeneralSecurityException)3 Map (java.util.Map)3 Set (java.util.Set)3 ServiceConfiguration (org.apache.pulsar.broker.ServiceConfiguration)3 NamespaceBundle (org.apache.pulsar.common.naming.NamespaceBundle)3 Preconditions.checkArgument (com.google.common.base.Preconditions.checkArgument)2 ChannelHandlerContext (io.netty.channel.ChannelHandlerContext)2 Optional (java.util.Optional)2 PositionImpl (org.apache.bookkeeper.mledger.impl.PositionImpl)2 StringUtils.isNotBlank (org.apache.commons.lang3.StringUtils.isNotBlank)2