Search in sources :

Example 11 with PulsarServerException

use of org.apache.pulsar.broker.PulsarServerException in project incubator-pulsar by apache.

the class NonPersistentTopics method getList.

@GET
@Path("/{property}/{cluster}/{namespace}")
@ApiOperation(value = "Get the list of non-persistent topics under a namespace.", response = String.class, responseContainer = "List")
@ApiResponses(value = { @ApiResponse(code = 403, message = "Don't have admin permission"), @ApiResponse(code = 404, message = "Namespace doesn't exist") })
public List<String> getList(@PathParam("property") String property, @PathParam("cluster") String cluster, @PathParam("namespace") String namespace) {
    log.info("[{}] list of topics on namespace {}/{}/{}/{}", clientAppId(), property, cluster, namespace);
    validateAdminAccessOnProperty(property);
    Policies policies = getNamespacePolicies(property, cluster, namespace);
    NamespaceName nsName = NamespaceName.get(property, cluster, namespace);
    if (!cluster.equals(Constants.GLOBAL_CLUSTER)) {
        validateClusterOwnership(cluster);
        validateClusterForProperty(property, cluster);
    } else {
        // check cluster ownership for a given global namespace: redirect if peer-cluster owns it
        validateGlobalNamespaceOwnership(nsName);
    }
    final List<CompletableFuture<List<String>>> futures = Lists.newArrayList();
    final List<String> boundaries = policies.bundles.getBoundaries();
    for (int i = 0; i < boundaries.size() - 1; i++) {
        final String bundle = String.format("%s_%s", boundaries.get(i), boundaries.get(i + 1));
        try {
            futures.add(pulsar().getAdminClient().nonPersistentTopics().getListInBundleAsync(nsName.toString(), bundle));
        } catch (PulsarServerException e) {
            log.error(String.format("[%s] Failed to get list of topics under namespace %s/%s/%s/%s", clientAppId(), property, cluster, namespace, bundle), e);
            throw new RestException(e);
        }
    }
    final List<String> topics = Lists.newArrayList();
    try {
        FutureUtil.waitForAll(futures).get();
        futures.forEach(topicListFuture -> {
            try {
                if (topicListFuture.isDone() && topicListFuture.get() != null) {
                    topics.addAll(topicListFuture.get());
                }
            } catch (InterruptedException | ExecutionException e) {
                log.error(String.format("[%s] Failed to get list of topics under namespace %s/%s/%s", clientAppId(), property, cluster, namespace), e);
            }
        });
    } catch (InterruptedException | ExecutionException e) {
        log.error(String.format("[%s] Failed to get list of topics under namespace %s/%s/%s", clientAppId(), property, cluster, namespace), e);
        throw new RestException(e instanceof ExecutionException ? e.getCause() : e);
    }
    return topics;
}
Also used : PulsarServerException(org.apache.pulsar.broker.PulsarServerException) NamespaceName(org.apache.pulsar.common.naming.NamespaceName) CompletableFuture(java.util.concurrent.CompletableFuture) Policies(org.apache.pulsar.common.policies.data.Policies) RestException(org.apache.pulsar.broker.web.RestException) ExecutionException(java.util.concurrent.ExecutionException) Path(javax.ws.rs.Path) GET(javax.ws.rs.GET) ApiOperation(io.swagger.annotations.ApiOperation) ApiResponses(io.swagger.annotations.ApiResponses)

Example 12 with PulsarServerException

use of org.apache.pulsar.broker.PulsarServerException in project incubator-pulsar by apache.

the class PulsarAuthorizationProvider method canConsumeAsync.

/**
 * Check if the specified role has permission to receive messages from the specified fully qualified topic
 * name.
 *
 * @param topicName
 *            the fully qualified topic name associated with the topic.
 * @param role
 *            the app id used to receive messages from the topic.
 * @param subscription
 *            the subscription name defined by the client
 */
@Override
public CompletableFuture<Boolean> canConsumeAsync(TopicName topicName, String role, AuthenticationDataSource authenticationData, String subscription) {
    CompletableFuture<Boolean> permissionFuture = new CompletableFuture<>();
    try {
        configCache.policiesCache().getAsync(POLICY_ROOT + topicName.getNamespace()).thenAccept(policies -> {
            if (!policies.isPresent()) {
                if (log.isDebugEnabled()) {
                    log.debug("Policies node couldn't be found for topic : {}", topicName);
                }
            } else {
                if (isNotBlank(subscription) && !isSuperUser(role)) {
                    switch(policies.get().subscription_auth_mode) {
                        case Prefix:
                            if (!subscription.startsWith(role)) {
                                PulsarServerException ex = new PulsarServerException(String.format("Failed to create consumer - The subscription name needs to be prefixed by the authentication role, like %s-xxxx for topic: %s", role, topicName));
                                permissionFuture.completeExceptionally(ex);
                                return;
                            }
                            break;
                        default:
                            break;
                    }
                }
            }
            checkAuthorization(topicName, role, AuthAction.consume).thenAccept(isAuthorized -> {
                permissionFuture.complete(isAuthorized);
            });
        }).exceptionally(ex -> {
            log.warn("Client with Role - {} failed to get permissions for topic - {}. {}", role, topicName, ex.getMessage());
            permissionFuture.completeExceptionally(ex);
            return null;
        });
    } catch (Exception e) {
        log.warn("Client  with Role - {} failed to get permissions for topic - {}. {}", role, topicName, e.getMessage());
        permissionFuture.completeExceptionally(e);
    }
    return permissionFuture;
}
Also used : ZooKeeper(org.apache.zookeeper.ZooKeeper) TopicName(org.apache.pulsar.common.naming.TopicName) Logger(org.slf4j.Logger) KeeperException(org.apache.zookeeper.KeeperException) ServiceConfiguration(org.apache.pulsar.broker.ServiceConfiguration) Preconditions.checkNotNull(com.google.common.base.Preconditions.checkNotNull) ObjectMapperFactory.getThreadLocal(org.apache.pulsar.common.util.ObjectMapperFactory.getThreadLocal) LoggerFactory(org.slf4j.LoggerFactory) Set(java.util.Set) IOException(java.io.IOException) CompletableFuture(java.util.concurrent.CompletableFuture) ConfigurationCacheService(org.apache.pulsar.broker.cache.ConfigurationCacheService) Stat(org.apache.zookeeper.data.Stat) AuthenticationDataSource(org.apache.pulsar.broker.authentication.AuthenticationDataSource) States(org.apache.zookeeper.ZooKeeper.States) Policies(org.apache.pulsar.common.policies.data.Policies) StringUtils.isNotBlank(org.apache.commons.lang3.StringUtils.isNotBlank) POLICIES(org.apache.pulsar.broker.cache.ConfigurationCacheService.POLICIES) PulsarServerException(org.apache.pulsar.broker.PulsarServerException) AuthAction(org.apache.pulsar.common.policies.data.AuthAction) ZooKeeperCache(org.apache.pulsar.zookeeper.ZooKeeperCache) Map(java.util.Map) NamespaceName(org.apache.pulsar.common.naming.NamespaceName) PulsarServerException(org.apache.pulsar.broker.PulsarServerException) CompletableFuture(java.util.concurrent.CompletableFuture) KeeperException(org.apache.zookeeper.KeeperException) IOException(java.io.IOException) PulsarServerException(org.apache.pulsar.broker.PulsarServerException)

Example 13 with PulsarServerException

use of org.apache.pulsar.broker.PulsarServerException in project incubator-pulsar by apache.

the class ResourceQuotaCache method initZK.

/**
 * Initialize the resource quota root node in ZooKeeper.
 */
public void initZK() throws PulsarServerException {
    String zpath = ResourceQuotaCache.path(null);
    ResourceQuota quota = this.readQuotaFromZnode(zpath);
    if (!quota.isValid()) {
        quota = ResourceQuotaCache.getInitialQuotaValue();
        try {
            this.saveQuotaToZnode(zpath, quota);
        } catch (Exception e) {
            throw new PulsarServerException(e);
        }
    }
}
Also used : PulsarServerException(org.apache.pulsar.broker.PulsarServerException) ResourceQuota(org.apache.pulsar.common.policies.data.ResourceQuota) KeeperException(org.apache.zookeeper.KeeperException) PulsarServerException(org.apache.pulsar.broker.PulsarServerException)

Example 14 with PulsarServerException

use of org.apache.pulsar.broker.PulsarServerException in project incubator-pulsar by apache.

the class ProxyServer method start.

public void start() throws PulsarServerException {
    log.info("Starting web socket proxy at port {}", conf.getWebServicePort());
    try {
        RequestLogHandler requestLogHandler = new RequestLogHandler();
        Slf4jRequestLog requestLog = new Slf4jRequestLog();
        requestLog.setExtended(true);
        requestLog.setLogTimeZone(TimeZone.getDefault().getID());
        requestLog.setLogLatency(true);
        requestLogHandler.setRequestLog(requestLog);
        handlers.add(0, new ContextHandlerCollection());
        handlers.add(requestLogHandler);
        ContextHandlerCollection contexts = new ContextHandlerCollection();
        contexts.setHandlers(handlers.toArray(new Handler[handlers.size()]));
        HandlerCollection handlerCollection = new HandlerCollection();
        handlerCollection.setHandlers(new Handler[] { contexts, new DefaultHandler(), requestLogHandler });
        server.setHandler(handlerCollection);
        server.start();
    } catch (Exception e) {
        throw new PulsarServerException(e);
    }
}
Also used : Slf4jRequestLog(org.eclipse.jetty.server.Slf4jRequestLog) PulsarServerException(org.apache.pulsar.broker.PulsarServerException) RequestLogHandler(org.eclipse.jetty.server.handler.RequestLogHandler) ServletContextHandler(org.eclipse.jetty.servlet.ServletContextHandler) Handler(org.eclipse.jetty.server.Handler) DefaultHandler(org.eclipse.jetty.server.handler.DefaultHandler) RequestLogHandler(org.eclipse.jetty.server.handler.RequestLogHandler) ContextHandlerCollection(org.eclipse.jetty.server.handler.ContextHandlerCollection) ContextHandlerCollection(org.eclipse.jetty.server.handler.ContextHandlerCollection) HandlerCollection(org.eclipse.jetty.server.handler.HandlerCollection) ServletException(javax.servlet.ServletException) DeploymentException(javax.websocket.DeploymentException) GeneralSecurityException(java.security.GeneralSecurityException) PulsarClientException(org.apache.pulsar.client.api.PulsarClientException) MalformedURLException(java.net.MalformedURLException) PulsarServerException(org.apache.pulsar.broker.PulsarServerException) DefaultHandler(org.eclipse.jetty.server.handler.DefaultHandler)

Example 15 with PulsarServerException

use of org.apache.pulsar.broker.PulsarServerException in project incubator-pulsar by apache.

the class Consumer method sendMessages.

/**
 * Dispatch a list of entries to the consumer. <br/>
 * <b>It is also responsible to release entries data and recycle entries object.</b>
 *
 * @return a promise that can be use to track when all the data has been written into the socket
 */
public SendMessageInfo sendMessages(final List<Entry> entries) {
    final ChannelHandlerContext ctx = cnx.ctx();
    final SendMessageInfo sentMessages = new SendMessageInfo();
    final ChannelPromise writePromise = ctx.newPromise();
    sentMessages.channelPromse = writePromise;
    if (entries.isEmpty()) {
        if (log.isDebugEnabled()) {
            log.debug("[{}-{}] List of messages is empty, triggering write future immediately for consumerId {}", topicName, subscription, consumerId);
        }
        writePromise.setSuccess();
        sentMessages.totalSentMessages = 0;
        sentMessages.totalSentMessageBytes = 0;
        return sentMessages;
    }
    try {
        updatePermitsAndPendingAcks(entries, sentMessages);
    } catch (PulsarServerException pe) {
        log.warn("[{}] [{}] consumer doesn't support batch-message {}", subscription, consumerId, cnx.getRemoteEndpointProtocolVersion());
        subscription.markTopicWithBatchMessagePublished();
        sentMessages.totalSentMessages = 0;
        sentMessages.totalSentMessageBytes = 0;
        // disconnect consumer: it will update dispatcher's availablePermits and resend pendingAck-messages of this
        // consumer to other consumer
        disconnect();
        return sentMessages;
    }
    ctx.channel().eventLoop().execute(() -> {
        for (int i = 0; i < entries.size(); i++) {
            Entry entry = entries.get(i);
            PositionImpl pos = (PositionImpl) entry.getPosition();
            MessageIdData.Builder messageIdBuilder = MessageIdData.newBuilder();
            MessageIdData messageId = messageIdBuilder.setLedgerId(pos.getLedgerId()).setEntryId(pos.getEntryId()).build();
            ByteBuf metadataAndPayload = entry.getDataBuffer();
            // increment ref-count of data and release at the end of process: so, we can get chance to call entry.release
            metadataAndPayload.retain();
            // skip checksum by incrementing reader-index if consumer-client doesn't support checksum verification
            if (cnx.getRemoteEndpointProtocolVersion() < ProtocolVersion.v11.getNumber()) {
                Commands.skipChecksumIfPresent(metadataAndPayload);
            }
            if (log.isDebugEnabled()) {
                log.debug("[{}-{}] Sending message to consumerId {}, entry id {}", topicName, subscription, consumerId, pos.getEntryId());
            }
            // We only want to pass the "real" promise on the last entry written
            ChannelPromise promise = ctx.voidPromise();
            if (i == (entries.size() - 1)) {
                promise = writePromise;
            }
            ctx.write(Commands.newMessage(consumerId, messageId, metadataAndPayload), promise);
            messageId.recycle();
            messageIdBuilder.recycle();
            entry.release();
        }
        ctx.flush();
    });
    return sentMessages;
}
Also used : PulsarServerException(org.apache.pulsar.broker.PulsarServerException) Entry(org.apache.bookkeeper.mledger.Entry) MessageIdData(org.apache.pulsar.common.api.proto.PulsarApi.MessageIdData) PositionImpl(org.apache.bookkeeper.mledger.impl.PositionImpl) ChannelHandlerContext(io.netty.channel.ChannelHandlerContext) ChannelPromise(io.netty.channel.ChannelPromise) ByteBuf(io.netty.buffer.ByteBuf)

Aggregations

PulsarServerException (org.apache.pulsar.broker.PulsarServerException)21 KeeperException (org.apache.zookeeper.KeeperException)7 CompletableFuture (java.util.concurrent.CompletableFuture)6 NamespaceName (org.apache.pulsar.common.naming.NamespaceName)6 Policies (org.apache.pulsar.common.policies.data.Policies)5 IOException (java.io.IOException)4 Logger (org.slf4j.Logger)4 LoggerFactory (org.slf4j.LoggerFactory)4 Preconditions.checkNotNull (com.google.common.base.Preconditions.checkNotNull)3 ByteBuf (io.netty.buffer.ByteBuf)3 GeneralSecurityException (java.security.GeneralSecurityException)3 Map (java.util.Map)3 Set (java.util.Set)3 ServiceConfiguration (org.apache.pulsar.broker.ServiceConfiguration)3 NamespaceBundle (org.apache.pulsar.common.naming.NamespaceBundle)3 Preconditions.checkArgument (com.google.common.base.Preconditions.checkArgument)2 ChannelHandlerContext (io.netty.channel.ChannelHandlerContext)2 Optional (java.util.Optional)2 PositionImpl (org.apache.bookkeeper.mledger.impl.PositionImpl)2 StringUtils.isNotBlank (org.apache.commons.lang3.StringUtils.isNotBlank)2