Search in sources :

Example 6 with PulsarAdmin

use of org.apache.pulsar.client.admin.PulsarAdmin in project incubator-pulsar by apache.

the class PulsarAdminTool method setupCommands.

private void setupCommands(BiFunction<URL, ClientConfigurationData, ? extends PulsarAdmin> adminFactory) {
    try {
        URL url = new URL(serviceUrl);
        config.setAuthentication(AuthenticationFactory.create(authPluginClassName, authParams));
        PulsarAdmin admin = adminFactory.apply(url, config);
        for (Map.Entry<String, Class> c : commandMap.entrySet()) {
            jcommander.addCommand(c.getKey(), c.getValue().getConstructor(PulsarAdmin.class).newInstance(admin));
        }
    } catch (MalformedURLException e) {
        System.err.println("Invalid serviceUrl: '" + serviceUrl + "'");
        System.exit(1);
    } catch (Exception e) {
        System.err.println(e.getClass() + ": " + e.getMessage());
        System.exit(1);
    }
}
Also used : MalformedURLException(java.net.MalformedURLException) PulsarAdmin(org.apache.pulsar.client.admin.PulsarAdmin) HashMap(java.util.HashMap) Map(java.util.Map) URL(java.net.URL) MalformedURLException(java.net.MalformedURLException)

Example 7 with PulsarAdmin

use of org.apache.pulsar.client.admin.PulsarAdmin in project incubator-pulsar by apache.

the class MembershipManager method getCurrentMembership.

public List<WorkerInfo> getCurrentMembership() {
    List<WorkerInfo> workerIds = new LinkedList<>();
    PersistentTopicStats persistentTopicStats = null;
    PulsarAdmin pulsarAdmin = this.getPulsarAdminClient();
    try {
        persistentTopicStats = pulsarAdmin.persistentTopics().getStats(this.workerConfig.getClusterCoordinationTopic());
    } catch (PulsarAdminException e) {
        log.error("Failed to get status of coordinate topic {}", this.workerConfig.getClusterCoordinationTopic(), e);
        throw new RuntimeException(e);
    }
    for (ConsumerStats consumerStats : persistentTopicStats.subscriptions.get(COORDINATION_TOPIC_SUBSCRIPTION).consumers) {
        WorkerInfo workerInfo = WorkerInfo.parseFrom(consumerStats.metadata.get(WORKER_IDENTIFIER));
        workerIds.add(workerInfo);
    }
    return workerIds;
}
Also used : PulsarAdmin(org.apache.pulsar.client.admin.PulsarAdmin) ConsumerStats(org.apache.pulsar.common.policies.data.ConsumerStats) PulsarAdminException(org.apache.pulsar.client.admin.PulsarAdminException) PersistentTopicStats(org.apache.pulsar.common.policies.data.PersistentTopicStats) LinkedList(java.util.LinkedList)

Example 8 with PulsarAdmin

use of org.apache.pulsar.client.admin.PulsarAdmin in project incubator-pulsar by apache.

the class Utils method getPulsarAdminClient.

public static PulsarAdmin getPulsarAdminClient(String pulsarWebServiceUrl) {
    URL url = null;
    try {
        url = new URL(pulsarWebServiceUrl);
    } catch (MalformedURLException e) {
        log.error("Error when parsing pulsar web service url", e);
        throw new RuntimeException(e);
    }
    PulsarAdmin admin = null;
    try {
        admin = new PulsarAdmin(url, new org.apache.pulsar.client.api.ClientConfiguration());
    } catch (PulsarClientException e) {
        log.error("Error creating pulsar admin client", e);
        throw new RuntimeException(e);
    }
    return admin;
}
Also used : MalformedURLException(java.net.MalformedURLException) PulsarAdmin(org.apache.pulsar.client.admin.PulsarAdmin) PulsarClientException(org.apache.pulsar.client.api.PulsarClientException) URL(java.net.URL)

Example 9 with PulsarAdmin

use of org.apache.pulsar.client.admin.PulsarAdmin in project incubator-pulsar by apache.

the class FunctionMetadataSetup method setupFunctionMetadata.

/**
 * Setup function metadata.
 *
 * @param workerConfig worker config
 * @return dlog uri for store function jars
 * @throws InterruptedException interrupted at setting up metadata
 * @throws PulsarAdminException when encountering exception on pulsar admin operation
 * @throws IOException when create dlog namespace for storing function jars.
 */
public static URI setupFunctionMetadata(WorkerConfig workerConfig) throws InterruptedException, PulsarAdminException, IOException {
    // initializing pulsar functions namespace
    PulsarAdmin admin = Utils.getPulsarAdminClient(workerConfig.getPulsarWebServiceUrl());
    InternalConfigurationData internalConf;
    // make sure pulsar broker is up
    log.info("Checking if pulsar service at {} is up...", workerConfig.getPulsarWebServiceUrl());
    int maxRetries = workerConfig.getInitialBrokerReconnectMaxRetries();
    int retries = 0;
    while (true) {
        try {
            admin.clusters().getClusters();
            break;
        } catch (PulsarAdminException e) {
            log.warn("Failed to retrieve clusters from pulsar service", e);
            log.warn("Retry to connect to Pulsar service at {}", workerConfig.getPulsarWebServiceUrl());
            if (retries >= maxRetries) {
                log.error("Failed to connect to Pulsar service at {} after {} attempts", workerConfig.getPulsarFunctionsNamespace(), maxRetries);
                throw e;
            }
            retries++;
            Thread.sleep(1000);
        }
    }
    // getting namespace policy
    log.info("Initializing Pulsar Functions namespace...");
    try {
        try {
            admin.namespaces().getPolicies(workerConfig.getPulsarFunctionsNamespace());
        } catch (PulsarAdminException e) {
            if (e.getStatusCode() == Response.Status.NOT_FOUND.getStatusCode()) {
                // if not found than create
                try {
                    admin.namespaces().createNamespace(workerConfig.getPulsarFunctionsNamespace());
                } catch (PulsarAdminException e1) {
                    // prevent race condition with other workers starting up
                    if (e1.getStatusCode() != Response.Status.CONFLICT.getStatusCode()) {
                        log.error("Failed to create namespace {} for pulsar functions", workerConfig.getPulsarFunctionsNamespace(), e1);
                        throw e1;
                    }
                }
                try {
                    admin.namespaces().setRetention(workerConfig.getPulsarFunctionsNamespace(), new RetentionPolicies(Integer.MAX_VALUE, Integer.MAX_VALUE));
                } catch (PulsarAdminException e1) {
                    log.error("Failed to set retention policy for pulsar functions namespace", e);
                    throw new RuntimeException(e1);
                }
            } else {
                log.error("Failed to get retention policy for pulsar function namespace {}", workerConfig.getPulsarFunctionsNamespace(), e);
                throw e;
            }
        }
        try {
            internalConf = admin.brokers().getInternalConfigurationData();
        } catch (PulsarAdminException e) {
            log.error("Failed to retrieve broker internal configuration", e);
            throw e;
        }
    } finally {
        admin.close();
    }
    // TODO: move this as part of pulsar cluster initialization later
    try {
        return Utils.initializeDlogNamespace(internalConf.getZookeeperServers(), internalConf.getLedgersRootPath());
    } catch (IOException ioe) {
        log.error("Failed to initialize dlog namespace at zookeeper {} for storing function packages", internalConf.getZookeeperServers(), ioe);
        throw ioe;
    }
}
Also used : RetentionPolicies(org.apache.pulsar.common.policies.data.RetentionPolicies) PulsarAdmin(org.apache.pulsar.client.admin.PulsarAdmin) InternalConfigurationData(org.apache.pulsar.common.conf.InternalConfigurationData) PulsarAdminException(org.apache.pulsar.client.admin.PulsarAdminException) IOException(java.io.IOException)

Example 10 with PulsarAdmin

use of org.apache.pulsar.client.admin.PulsarAdmin in project incubator-pulsar by apache.

the class PersistentTopicsBase method internalCreateSubscription.

protected void internalCreateSubscription(String subscriptionName, MessageIdImpl messageId, boolean authoritative) {
    if (topicName.isGlobal()) {
        validateGlobalNamespaceOwnership(namespaceName);
    }
    log.info("[{}][{}] Creating subscription {} at message id {}", clientAppId(), topicName, subscriptionName, messageId);
    PartitionedTopicMetadata partitionMetadata = getPartitionedTopicMetadata(topicName, authoritative);
    try {
        if (partitionMetadata.partitions > 0) {
            // Create the subscription on each partition
            List<CompletableFuture<Void>> futures = Lists.newArrayList();
            PulsarAdmin admin = pulsar().getAdminClient();
            for (int i = 0; i < partitionMetadata.partitions; i++) {
                futures.add(admin.persistentTopics().createSubscriptionAsync(topicName.getPartition(i).toString(), subscriptionName, messageId));
            }
            FutureUtil.waitForAll(futures).join();
        } else {
            validateAdminOperationOnTopic(authoritative);
            PersistentTopic topic = (PersistentTopic) getOrCreateTopic(topicName);
            if (topic.getSubscriptions().containsKey(subscriptionName)) {
                throw new RestException(Status.CONFLICT, "Subscription already exists for topic");
            }
            PersistentSubscription subscription = (PersistentSubscription) topic.createSubscription(subscriptionName, InitialPosition.Latest).get();
            subscription.resetCursor(PositionImpl.get(messageId.getLedgerId(), messageId.getEntryId())).get();
            log.info("[{}][{}] Successfully created subscription {} at message id {}", clientAppId(), topicName, subscriptionName, messageId);
        }
    } catch (Exception e) {
        Throwable t = e.getCause();
        log.warn("[{}] [{}] Failed to create subscription {} at message id {}", clientAppId(), topicName, subscriptionName, messageId, e);
        if (t instanceof SubscriptionInvalidCursorPosition) {
            throw new RestException(Status.PRECONDITION_FAILED, "Unable to find position for position specified: " + t.getMessage());
        } else {
            throw new RestException(e);
        }
    }
}
Also used : SubscriptionInvalidCursorPosition(org.apache.pulsar.broker.service.BrokerServiceException.SubscriptionInvalidCursorPosition) CompletableFuture(java.util.concurrent.CompletableFuture) PulsarAdmin(org.apache.pulsar.client.admin.PulsarAdmin) PersistentTopic(org.apache.pulsar.broker.service.persistent.PersistentTopic) RestException(org.apache.pulsar.broker.web.RestException) PartitionedTopicMetadata(org.apache.pulsar.common.partition.PartitionedTopicMetadata) PersistentSubscription(org.apache.pulsar.broker.service.persistent.PersistentSubscription) NotAllowedException(org.apache.pulsar.broker.service.BrokerServiceException.NotAllowedException) NotFoundException(org.apache.pulsar.client.admin.PulsarAdminException.NotFoundException) PreconditionFailedException(org.apache.pulsar.client.admin.PulsarAdminException.PreconditionFailedException) RestException(org.apache.pulsar.broker.web.RestException) PulsarClientException(org.apache.pulsar.client.api.PulsarClientException) ManagedLedgerException(org.apache.bookkeeper.mledger.ManagedLedgerException) SubscriptionBusyException(org.apache.pulsar.broker.service.BrokerServiceException.SubscriptionBusyException) WebApplicationException(javax.ws.rs.WebApplicationException) KeeperException(org.apache.zookeeper.KeeperException) PulsarAdminException(org.apache.pulsar.client.admin.PulsarAdminException) IOException(java.io.IOException) ExecutionException(java.util.concurrent.ExecutionException) TopicBusyException(org.apache.pulsar.broker.service.BrokerServiceException.TopicBusyException) PulsarServerException(org.apache.pulsar.broker.PulsarServerException)

Aggregations

PulsarAdmin (org.apache.pulsar.client.admin.PulsarAdmin)39 URL (java.net.URL)15 Test (org.testng.annotations.Test)14 PulsarService (org.apache.pulsar.broker.PulsarService)12 Authentication (org.apache.pulsar.client.api.Authentication)11 PropertyAdmin (org.apache.pulsar.common.policies.data.PropertyAdmin)11 ClusterData (org.apache.pulsar.common.policies.data.ClusterData)10 ServiceConfiguration (org.apache.pulsar.broker.ServiceConfiguration)9 LocalBookkeeperEnsemble (org.apache.pulsar.zookeeper.LocalBookkeeperEnsemble)9 BeforeMethod (org.testng.annotations.BeforeMethod)7 PulsarAdminException (org.apache.pulsar.client.admin.PulsarAdminException)6 AuthenticationTls (org.apache.pulsar.client.impl.auth.AuthenticationTls)6 IOException (java.io.IOException)4 HashMap (java.util.HashMap)4 URI (java.net.URI)3 ModularLoadManagerImpl (org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl)3 SimpleLoadManagerImpl (org.apache.pulsar.broker.loadbalance.impl.SimpleLoadManagerImpl)3 MalformedURLException (java.net.MalformedURLException)2 Brokers (org.apache.pulsar.client.admin.Brokers)2 ClientConfiguration (org.apache.pulsar.client.api.ClientConfiguration)2