Search in sources :

Example 1 with ResourceGroup

use of org.apache.pulsar.broker.resourcegroup.ResourceGroup in project pulsar by apache.

the class AbstractTopic method updateResourceGroupLimiter.

public void updateResourceGroupLimiter(Optional<Policies> optPolicies) {
    Policies policies;
    try {
        policies = optPolicies.orElseGet(() -> brokerService.pulsar().getPulsarResources().getNamespaceResources().getPoliciesIfCached(TopicName.get(topic).getNamespaceObject()).orElseGet(Policies::new));
    } catch (Exception e) {
        log.warn("[{}] Error getting policies {} and publish throttling will be disabled", topic, e.getMessage());
        policies = new Policies();
    }
    // attach the resource-group level rate limiters, if set
    String rgName = policies.resource_group_name != null ? policies.resource_group_name : null;
    if (rgName != null) {
        final ResourceGroup resourceGroup = brokerService.getPulsar().getResourceGroupServiceManager().resourceGroupGet(rgName);
        if (resourceGroup != null) {
            this.resourceGroupRateLimitingEnabled = true;
            this.resourceGroupPublishLimiter = resourceGroup.getResourceGroupPublishLimiter();
            this.resourceGroupPublishLimiter.registerRateLimitFunction(this.getName(), () -> this.enableCnxAutoRead());
            log.info("Using resource group {} rate limiter for topic {}", rgName, topic);
            return;
        }
    } else {
        if (this.resourceGroupRateLimitingEnabled) {
            this.resourceGroupPublishLimiter.unregisterRateLimitFunction(this.getName());
            this.resourceGroupPublishLimiter = null;
            this.resourceGroupRateLimitingEnabled = false;
        }
        /* Namespace detached from resource group. Enable the producer read */
        enableProducerReadForPublishRateLimiting();
    }
}
Also used : DelayedDeliveryPolicies(org.apache.pulsar.common.policies.data.DelayedDeliveryPolicies) InactiveTopicPolicies(org.apache.pulsar.common.policies.data.InactiveTopicPolicies) HierarchyTopicPolicies(org.apache.pulsar.common.policies.data.HierarchyTopicPolicies) RetentionPolicies(org.apache.pulsar.common.policies.data.RetentionPolicies) TopicPolicies(org.apache.pulsar.common.policies.data.TopicPolicies) Policies(org.apache.pulsar.common.policies.data.Policies) ProducerBusyException(org.apache.pulsar.broker.service.BrokerServiceException.ProducerBusyException) TopicTerminatedException(org.apache.pulsar.broker.service.BrokerServiceException.TopicTerminatedException) IncompatibleSchemaException(org.apache.pulsar.broker.service.schema.exceptions.IncompatibleSchemaException) ProducerFencedException(org.apache.pulsar.broker.service.BrokerServiceException.ProducerFencedException) ConsumerBusyException(org.apache.pulsar.broker.service.BrokerServiceException.ConsumerBusyException) ResourceGroup(org.apache.pulsar.broker.resourcegroup.ResourceGroup)

Example 2 with ResourceGroup

use of org.apache.pulsar.broker.resourcegroup.ResourceGroup in project incubator-pulsar by apache.

the class AbstractTopic method updateResourceGroupLimiter.

public void updateResourceGroupLimiter(Optional<Policies> optPolicies) {
    Policies policies;
    try {
        policies = optPolicies.orElseGet(() -> brokerService.pulsar().getPulsarResources().getNamespaceResources().getPoliciesIfCached(TopicName.get(topic).getNamespaceObject()).orElseGet(Policies::new));
    } catch (Exception e) {
        log.warn("[{}] Error getting policies {} and publish throttling will be disabled", topic, e.getMessage());
        policies = new Policies();
    }
    // attach the resource-group level rate limiters, if set
    String rgName = policies.resource_group_name;
    if (rgName != null) {
        final ResourceGroup resourceGroup = brokerService.getPulsar().getResourceGroupServiceManager().resourceGroupGet(rgName);
        if (resourceGroup != null) {
            this.resourceGroupRateLimitingEnabled = true;
            this.resourceGroupPublishLimiter = resourceGroup.getResourceGroupPublishLimiter();
            this.resourceGroupPublishLimiter.registerRateLimitFunction(this.getName(), () -> this.enableCnxAutoRead());
            log.info("Using resource group {} rate limiter for topic {}", rgName, topic);
            return;
        }
    } else {
        if (this.resourceGroupRateLimitingEnabled) {
            this.resourceGroupPublishLimiter.unregisterRateLimitFunction(this.getName());
            this.resourceGroupPublishLimiter = null;
            this.resourceGroupRateLimitingEnabled = false;
        }
        /* Namespace detached from resource group. Enable the producer read */
        enableProducerReadForPublishRateLimiting();
    }
}
Also used : DelayedDeliveryPolicies(org.apache.pulsar.common.policies.data.DelayedDeliveryPolicies) SubscriptionPolicies(org.apache.pulsar.common.policies.data.SubscriptionPolicies) InactiveTopicPolicies(org.apache.pulsar.common.policies.data.InactiveTopicPolicies) HierarchyTopicPolicies(org.apache.pulsar.common.policies.data.HierarchyTopicPolicies) RetentionPolicies(org.apache.pulsar.common.policies.data.RetentionPolicies) TopicPolicies(org.apache.pulsar.common.policies.data.TopicPolicies) Policies(org.apache.pulsar.common.policies.data.Policies) ProducerBusyException(org.apache.pulsar.broker.service.BrokerServiceException.ProducerBusyException) TopicTerminatedException(org.apache.pulsar.broker.service.BrokerServiceException.TopicTerminatedException) IncompatibleSchemaException(org.apache.pulsar.broker.service.schema.exceptions.IncompatibleSchemaException) ProducerFencedException(org.apache.pulsar.broker.service.BrokerServiceException.ProducerFencedException) ConsumerBusyException(org.apache.pulsar.broker.service.BrokerServiceException.ConsumerBusyException) ResourceGroup(org.apache.pulsar.broker.resourcegroup.ResourceGroup)

Example 3 with ResourceGroup

use of org.apache.pulsar.broker.resourcegroup.ResourceGroup in project pulsar by yahoo.

the class AbstractTopic method updateResourceGroupLimiter.

public void updateResourceGroupLimiter(Optional<Policies> optPolicies) {
    Policies policies;
    try {
        policies = optPolicies.orElseGet(() -> brokerService.pulsar().getPulsarResources().getNamespaceResources().getPoliciesIfCached(TopicName.get(topic).getNamespaceObject()).orElseGet(Policies::new));
    } catch (Exception e) {
        log.warn("[{}] Error getting policies {} and publish throttling will be disabled", topic, e.getMessage());
        policies = new Policies();
    }
    // attach the resource-group level rate limiters, if set
    String rgName = policies.resource_group_name;
    if (rgName != null) {
        final ResourceGroup resourceGroup = brokerService.getPulsar().getResourceGroupServiceManager().resourceGroupGet(rgName);
        if (resourceGroup != null) {
            this.resourceGroupRateLimitingEnabled = true;
            this.resourceGroupPublishLimiter = resourceGroup.getResourceGroupPublishLimiter();
            this.resourceGroupPublishLimiter.registerRateLimitFunction(this.getName(), () -> this.enableCnxAutoRead());
            log.info("Using resource group {} rate limiter for topic {}", rgName, topic);
            return;
        }
    } else {
        if (this.resourceGroupRateLimitingEnabled) {
            this.resourceGroupPublishLimiter.unregisterRateLimitFunction(this.getName());
            this.resourceGroupPublishLimiter = null;
            this.resourceGroupRateLimitingEnabled = false;
        }
        /* Namespace detached from resource group. Enable the producer read */
        enableProducerReadForPublishRateLimiting();
    }
}
Also used : DelayedDeliveryPolicies(org.apache.pulsar.common.policies.data.DelayedDeliveryPolicies) SubscriptionPolicies(org.apache.pulsar.common.policies.data.SubscriptionPolicies) InactiveTopicPolicies(org.apache.pulsar.common.policies.data.InactiveTopicPolicies) HierarchyTopicPolicies(org.apache.pulsar.common.policies.data.HierarchyTopicPolicies) RetentionPolicies(org.apache.pulsar.common.policies.data.RetentionPolicies) TopicPolicies(org.apache.pulsar.common.policies.data.TopicPolicies) Policies(org.apache.pulsar.common.policies.data.Policies) ProducerBusyException(org.apache.pulsar.broker.service.BrokerServiceException.ProducerBusyException) TopicTerminatedException(org.apache.pulsar.broker.service.BrokerServiceException.TopicTerminatedException) IncompatibleSchemaException(org.apache.pulsar.broker.service.schema.exceptions.IncompatibleSchemaException) ProducerFencedException(org.apache.pulsar.broker.service.BrokerServiceException.ProducerFencedException) ConsumerBusyException(org.apache.pulsar.broker.service.BrokerServiceException.ConsumerBusyException) ResourceGroup(org.apache.pulsar.broker.resourcegroup.ResourceGroup)

Aggregations

ResourceGroup (org.apache.pulsar.broker.resourcegroup.ResourceGroup)3 ConsumerBusyException (org.apache.pulsar.broker.service.BrokerServiceException.ConsumerBusyException)3 ProducerBusyException (org.apache.pulsar.broker.service.BrokerServiceException.ProducerBusyException)3 ProducerFencedException (org.apache.pulsar.broker.service.BrokerServiceException.ProducerFencedException)3 TopicTerminatedException (org.apache.pulsar.broker.service.BrokerServiceException.TopicTerminatedException)3 IncompatibleSchemaException (org.apache.pulsar.broker.service.schema.exceptions.IncompatibleSchemaException)3 DelayedDeliveryPolicies (org.apache.pulsar.common.policies.data.DelayedDeliveryPolicies)3 HierarchyTopicPolicies (org.apache.pulsar.common.policies.data.HierarchyTopicPolicies)3 InactiveTopicPolicies (org.apache.pulsar.common.policies.data.InactiveTopicPolicies)3 Policies (org.apache.pulsar.common.policies.data.Policies)3 RetentionPolicies (org.apache.pulsar.common.policies.data.RetentionPolicies)3 TopicPolicies (org.apache.pulsar.common.policies.data.TopicPolicies)3 SubscriptionPolicies (org.apache.pulsar.common.policies.data.SubscriptionPolicies)2