Search in sources :

Example 16 with Policies

use of org.apache.pulsar.common.policies.data.Policies in project incubator-pulsar by apache.

the class LocalZooKeeperCacheService method createPolicies.

/**
 * Create LocalPolicies with bundle-data in LocalZookeeper by fetching it from GlobalZookeeper
 *
 * @param path
 *            znode path
 * @param readFromGlobal
 *            if true copy policies from global zk to local zk else create a new znode with empty {@link Policies}
 * @throws Exception
 */
@SuppressWarnings("deprecation")
public CompletableFuture<Optional<LocalPolicies>> createPolicies(String path, boolean readFromGlobal) {
    CompletableFuture<Optional<LocalPolicies>> future = new CompletableFuture<>();
    if (path == null || !path.startsWith(LOCAL_POLICIES_ROOT)) {
        future.completeExceptionally(new IllegalArgumentException("Invalid path of local policies " + path));
        return future;
    }
    if (LOG.isDebugEnabled()) {
        LOG.debug("Creating local namespace policies for {} - readFromGlobal: {}", path, readFromGlobal);
    }
    CompletableFuture<Optional<LocalPolicies>> readFromGlobalFuture = new CompletableFuture<>();
    if (readFromGlobal) {
        String globalPath = joinPath(POLICIES_ROOT, path.substring(path.indexOf(LOCAL_POLICIES_ROOT) + LOCAL_POLICIES_ROOT.length() + 1));
        checkNotNull(configurationCacheService);
        checkNotNull(configurationCacheService.policiesCache());
        checkNotNull(configurationCacheService.policiesCache().getAsync(globalPath));
        configurationCacheService.policiesCache().getAsync(globalPath).thenAccept(policies -> {
            if (policies.isPresent()) {
                // Copying global bundles information to local policies
                LocalPolicies localPolicies = new LocalPolicies();
                localPolicies.bundles = policies.get().bundles;
                readFromGlobalFuture.complete(Optional.of(localPolicies));
            } else {
                // Policies are not present in global zk
                if (LOG.isDebugEnabled()) {
                    LOG.debug("Global policies not found at {}", globalPath);
                }
                readFromGlobalFuture.complete(Optional.empty());
            }
        }).exceptionally(ex -> {
            future.completeExceptionally(ex);
            return null;
        });
    } else {
        // Use default local policies
        readFromGlobalFuture.complete(Optional.of(new LocalPolicies()));
    }
    readFromGlobalFuture.thenAccept(localPolicies -> {
        if (!localPolicies.isPresent()) {
            future.complete(Optional.empty());
        }
        // When we have the updated localPolicies, we can write them back in local ZK
        byte[] content;
        try {
            content = ObjectMapperFactory.getThreadLocal().writeValueAsBytes(localPolicies.get());
        } catch (Throwable t) {
            // Failed to serialize to json
            future.completeExceptionally(t);
            return;
        }
        ZkUtils.asyncCreateFullPathOptimistic(cache.getZooKeeper(), path, content, Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT, (rc, path1, ctx, name) -> {
            if (rc == KeeperException.Code.OK.intValue() || rc == KeeperException.Code.NODEEXISTS.intValue()) {
                LOG.info("Successfully copyied bundles data to local zk at {}", path);
                future.complete(localPolicies);
            } else {
                LOG.error("Failed to create policies for {} in local zookeeper: {}", path, KeeperException.Code.get(rc));
                future.completeExceptionally(new PulsarServerException(KeeperException.create(rc)));
            }
        }, null);
    }).exceptionally(ex -> {
        future.completeExceptionally(ex);
        return null;
    });
    return future;
}
Also used : CreateMode(org.apache.zookeeper.CreateMode) ZooKeeper(org.apache.zookeeper.ZooKeeper) Logger(org.slf4j.Logger) KeeperException(org.apache.zookeeper.KeeperException) Ids(org.apache.zookeeper.ZooDefs.Ids) ObjectMapperFactory(org.apache.pulsar.common.util.ObjectMapperFactory) Preconditions.checkNotNull(com.google.common.base.Preconditions.checkNotNull) LoggerFactory(org.slf4j.LoggerFactory) NamespaceEphemeralData(org.apache.pulsar.broker.namespace.NamespaceEphemeralData) LocalPolicies(org.apache.pulsar.common.policies.data.LocalPolicies) ZooKeeperChildrenCache(org.apache.pulsar.zookeeper.ZooKeeperChildrenCache) CompletableFuture(java.util.concurrent.CompletableFuture) Stat(org.apache.zookeeper.data.Stat) PulsarWebResource.joinPath(org.apache.pulsar.broker.web.PulsarWebResource.joinPath) Maps(com.google.common.collect.Maps) ZkUtils(org.apache.bookkeeper.util.ZkUtils) Policies(org.apache.pulsar.common.policies.data.Policies) PulsarServerException(org.apache.pulsar.broker.PulsarServerException) ZooKeeperCache(org.apache.pulsar.zookeeper.ZooKeeperCache) ZooKeeperDataCache(org.apache.pulsar.zookeeper.ZooKeeperDataCache) Entry(java.util.Map.Entry) Optional(java.util.Optional) POLICIES_ROOT(org.apache.pulsar.broker.cache.ConfigurationCacheService.POLICIES_ROOT) PulsarServerException(org.apache.pulsar.broker.PulsarServerException) CompletableFuture(java.util.concurrent.CompletableFuture) Optional(java.util.Optional) LocalPolicies(org.apache.pulsar.common.policies.data.LocalPolicies)

Example 17 with Policies

use of org.apache.pulsar.common.policies.data.Policies in project incubator-pulsar by apache.

the class Namespaces method getNamespaceMessageTTL.

@GET
@Path("/{property}/{namespace}/messageTTL")
@ApiOperation(value = "Get the message TTL for the namespace")
@ApiResponses(value = { @ApiResponse(code = 403, message = "Don't have admin permission"), @ApiResponse(code = 404, message = "Property or cluster or namespace doesn't exist") })
public int getNamespaceMessageTTL(@PathParam("property") String property, @PathParam("namespace") String namespace) {
    validateAdminAccessOnProperty(property);
    validateNamespaceName(property, namespace);
    Policies policies = getNamespacePolicies(namespaceName);
    return policies.message_ttl_in_seconds;
}
Also used : RetentionPolicies(org.apache.pulsar.common.policies.data.RetentionPolicies) PersistencePolicies(org.apache.pulsar.common.policies.data.PersistencePolicies) Policies(org.apache.pulsar.common.policies.data.Policies) Path(javax.ws.rs.Path) GET(javax.ws.rs.GET) ApiOperation(io.swagger.annotations.ApiOperation) ApiResponses(io.swagger.annotations.ApiResponses)

Example 18 with Policies

use of org.apache.pulsar.common.policies.data.Policies in project incubator-pulsar by apache.

the class Namespaces method getBacklogQuotaMap.

@GET
@Path("/{property}/{namespace}/backlogQuotaMap")
@ApiOperation(value = "Get backlog quota map on a namespace.")
@ApiResponses(value = { @ApiResponse(code = 403, message = "Don't have admin permission"), @ApiResponse(code = 404, message = "Namespace does not exist") })
public Map<BacklogQuotaType, BacklogQuota> getBacklogQuotaMap(@PathParam("property") String property, @PathParam("namespace") String namespace) {
    validateAdminAccessOnProperty(property);
    validateNamespaceName(property, namespace);
    Policies policies = getNamespacePolicies(namespaceName);
    return policies.backlog_quota_map;
}
Also used : RetentionPolicies(org.apache.pulsar.common.policies.data.RetentionPolicies) PersistencePolicies(org.apache.pulsar.common.policies.data.PersistencePolicies) Policies(org.apache.pulsar.common.policies.data.Policies) Path(javax.ws.rs.Path) GET(javax.ws.rs.GET) ApiOperation(io.swagger.annotations.ApiOperation) ApiResponses(io.swagger.annotations.ApiResponses)

Example 19 with Policies

use of org.apache.pulsar.common.policies.data.Policies in project incubator-pulsar by apache.

the class Namespaces method getPermissions.

@GET
@Path("/{property}/{namespace}/permissions")
@ApiOperation(value = "Retrieve the permissions for a namespace.")
@ApiResponses(value = { @ApiResponse(code = 403, message = "Don't have admin permission"), @ApiResponse(code = 404, message = "Property or cluster or namespace doesn't exist"), @ApiResponse(code = 409, message = "Namespace is not empty") })
public Map<String, Set<AuthAction>> getPermissions(@PathParam("property") String property, @PathParam("cluster") String cluster, @PathParam("namespace") String namespace) {
    validateAdminAccessOnProperty(property);
    validateNamespaceName(property, namespace);
    Policies policies = getNamespacePolicies(namespaceName);
    return policies.auth_policies.namespace_auth;
}
Also used : RetentionPolicies(org.apache.pulsar.common.policies.data.RetentionPolicies) PersistencePolicies(org.apache.pulsar.common.policies.data.PersistencePolicies) Policies(org.apache.pulsar.common.policies.data.Policies) Path(javax.ws.rs.Path) GET(javax.ws.rs.GET) ApiOperation(io.swagger.annotations.ApiOperation) ApiResponses(io.swagger.annotations.ApiResponses)

Example 20 with Policies

use of org.apache.pulsar.common.policies.data.Policies in project incubator-pulsar by apache.

the class ServerCnxTest method testSendFailureOnEncryptionRequiredTopic.

@Test(timeOut = 30000)
public void testSendFailureOnEncryptionRequiredTopic() throws Exception {
    resetChannel();
    setChannelConnected();
    // Set encryption_required to true
    ZooKeeperDataCache<Policies> zkDataCache = mock(ZooKeeperDataCache.class);
    Policies policies = mock(Policies.class);
    policies.encryption_required = true;
    policies.clusterDispatchRate = Maps.newHashMap();
    doReturn(Optional.of(policies)).when(zkDataCache).get(AdminResource.path(POLICIES, TopicName.get(encryptionRequiredTopicName).getNamespace()));
    doReturn(CompletableFuture.completedFuture(Optional.of(policies))).when(zkDataCache).getAsync(AdminResource.path(POLICIES, TopicName.get(encryptionRequiredTopicName).getNamespace()));
    doReturn(zkDataCache).when(configCacheService).policiesCache();
    ByteBuf clientCommand = Commands.newProducer(encryptionRequiredTopicName, 1, /* producer id */
    1, /* request id */
    "prod-name", true, null);
    channel.writeInbound(clientCommand);
    assertTrue(getResponse() instanceof CommandProducerSuccess);
    // test failure case: unencrypted messages cannot be published
    MessageMetadata messageMetadata = MessageMetadata.newBuilder().setPublishTime(System.currentTimeMillis()).setProducerName("prod-name").setSequenceId(0).build();
    ByteBuf data = Unpooled.buffer(1024);
    clientCommand = ByteBufPair.coalesce(Commands.newSend(1, 0, 1, ChecksumType.None, messageMetadata, data));
    channel.writeInbound(Unpooled.copiedBuffer(clientCommand));
    clientCommand.release();
    assertTrue(getResponse() instanceof CommandSendError);
    channel.finish();
}
Also used : MessageMetadata(org.apache.pulsar.common.api.proto.PulsarApi.MessageMetadata) Policies(org.apache.pulsar.common.policies.data.Policies) CommandProducerSuccess(org.apache.pulsar.common.api.proto.PulsarApi.CommandProducerSuccess) ByteBuf(io.netty.buffer.ByteBuf) CommandSendError(org.apache.pulsar.common.api.proto.PulsarApi.CommandSendError) Test(org.testng.annotations.Test)

Aggregations

Policies (org.apache.pulsar.common.policies.data.Policies)93 KeeperException (org.apache.zookeeper.KeeperException)43 RetentionPolicies (org.apache.pulsar.common.policies.data.RetentionPolicies)40 PersistencePolicies (org.apache.pulsar.common.policies.data.PersistencePolicies)39 RestException (org.apache.pulsar.broker.web.RestException)34 PulsarServerException (org.apache.pulsar.broker.PulsarServerException)30 SubscriptionBusyException (org.apache.pulsar.broker.service.BrokerServiceException.SubscriptionBusyException)28 Stat (org.apache.zookeeper.data.Stat)26 WebApplicationException (javax.ws.rs.WebApplicationException)24 ExecutionException (java.util.concurrent.ExecutionException)23 PulsarAdminException (org.apache.pulsar.client.admin.PulsarAdminException)23 Test (org.testng.annotations.Test)21 NamespaceBundle (org.apache.pulsar.common.naming.NamespaceBundle)15 TopicName (org.apache.pulsar.common.naming.TopicName)14 ApiOperation (io.swagger.annotations.ApiOperation)13 ApiResponses (io.swagger.annotations.ApiResponses)13 Path (javax.ws.rs.Path)13 ManagedLedgerException (org.apache.bookkeeper.mledger.ManagedLedgerException)13 NotAllowedException (org.apache.pulsar.broker.service.BrokerServiceException.NotAllowedException)11 NamespaceName (org.apache.pulsar.common.naming.NamespaceName)11