Search in sources :

Example 1 with TopicFencedException

use of com.yahoo.pulsar.broker.service.BrokerServiceException.TopicFencedException in project pulsar by yahoo.

the class PersistentTopic method close.

/**
     * Close this topic - close all producers and subscriptions associated with this topic
     *
     * @return Completable future indicating completion of close operation
     */
@Override
public CompletableFuture<Void> close() {
    CompletableFuture<Void> closeFuture = new CompletableFuture<>();
    lock.writeLock().lock();
    try {
        if (!isFenced) {
            isFenced = true;
        } else {
            log.warn("[{}] Topic is already being closed or deleted", topic);
            closeFuture.completeExceptionally(new TopicFencedException("Topic is already fenced"));
            return closeFuture;
        }
    } finally {
        lock.writeLock().unlock();
    }
    List<CompletableFuture<Void>> futures = Lists.newArrayList();
    replicators.forEach((cluster, replicator) -> futures.add(replicator.disconnect()));
    producers.forEach(producer -> futures.add(producer.disconnect()));
    subscriptions.forEach((s, sub) -> futures.add(sub.disconnect()));
    FutureUtil.waitForAll(futures).thenRun(() -> {
        // After having disconnected all producers/consumers, close the managed ledger
        ledger.asyncClose(new CloseCallback() {

            @Override
            public void closeComplete(Object ctx) {
                // Everything is now closed, remove the topic from map
                brokerService.removeTopicFromCache(topic);
                log.info("[{}] Topic closed", topic);
                closeFuture.complete(null);
            }

            @Override
            public void closeFailed(ManagedLedgerException exception, Object ctx) {
                log.error("[{}] Failed to close managed ledger, proceeding anyway.", topic, exception);
                brokerService.removeTopicFromCache(topic);
                closeFuture.complete(null);
            }
        }, null);
    }).exceptionally(exception -> {
        log.error("[{}] Error closing topic", topic, exception);
        isFenced = false;
        closeFuture.completeExceptionally(exception);
        return null;
    });
    return closeFuture;
}
Also used : CompletableFuture(java.util.concurrent.CompletableFuture) ManagedLedgerException(org.apache.bookkeeper.mledger.ManagedLedgerException) TopicFencedException(com.yahoo.pulsar.broker.service.BrokerServiceException.TopicFencedException) CloseCallback(org.apache.bookkeeper.mledger.AsyncCallbacks.CloseCallback)

Example 2 with TopicFencedException

use of com.yahoo.pulsar.broker.service.BrokerServiceException.TopicFencedException in project pulsar by yahoo.

the class PersistentTopic method delete.

/**
     * Delete the managed ledger associated with this topic
     *
     * @param failIfHasSubscriptions
     *            Flag indicating whether delete should succeed if topic still has unconnected subscriptions. Set to
     *            false when called from admin API (it will delete the subs too), and set to true when called from GC
     *            thread
     *
     * @return Completable future indicating completion of delete operation Completed exceptionally with:
     *         IllegalStateException if topic is still active ManagedLedgerException if ledger delete operation fails
     */
private CompletableFuture<Void> delete(boolean failIfHasSubscriptions) {
    CompletableFuture<Void> deleteFuture = new CompletableFuture<>();
    lock.writeLock().lock();
    try {
        if (isFenced) {
            log.warn("[{}] Topic is already being closed or deleted", topic);
            deleteFuture.completeExceptionally(new TopicFencedException("Topic is already fenced"));
            return deleteFuture;
        }
        if (USAGE_COUNT_UPDATER.get(this) == 0) {
            isFenced = true;
            List<CompletableFuture<Void>> futures = Lists.newArrayList();
            if (failIfHasSubscriptions) {
                if (!subscriptions.isEmpty()) {
                    isFenced = false;
                    deleteFuture.completeExceptionally(new TopicBusyException("Topic has subscriptions"));
                    return deleteFuture;
                }
            } else {
                subscriptions.forEach((s, sub) -> futures.add(sub.delete()));
            }
            FutureUtil.waitForAll(futures).whenComplete((v, ex) -> {
                if (ex != null) {
                    log.error("[{}] Error deleting topic", topic, ex);
                    isFenced = false;
                    deleteFuture.completeExceptionally(ex);
                } else {
                    ledger.asyncDelete(new AsyncCallbacks.DeleteLedgerCallback() {

                        @Override
                        public void deleteLedgerComplete(Object ctx) {
                            brokerService.removeTopicFromCache(topic);
                            log.info("[{}] Topic deleted", topic);
                            deleteFuture.complete(null);
                        }

                        @Override
                        public void deleteLedgerFailed(ManagedLedgerException exception, Object ctx) {
                            isFenced = false;
                            log.error("[{}] Error deleting topic", topic, exception);
                            deleteFuture.completeExceptionally(new PersistenceException(exception));
                        }
                    }, null);
                }
            });
        } else {
            deleteFuture.completeExceptionally(new TopicBusyException("Topic has " + USAGE_COUNT_UPDATER.get(this) + " connected producers/consumers"));
        }
    } finally {
        lock.writeLock().unlock();
    }
    return deleteFuture;
}
Also used : TopicBusyException(com.yahoo.pulsar.broker.service.BrokerServiceException.TopicBusyException) CompletableFuture(java.util.concurrent.CompletableFuture) ManagedLedgerException(org.apache.bookkeeper.mledger.ManagedLedgerException) TopicFencedException(com.yahoo.pulsar.broker.service.BrokerServiceException.TopicFencedException) PersistenceException(com.yahoo.pulsar.broker.service.BrokerServiceException.PersistenceException) AsyncCallbacks(org.apache.bookkeeper.mledger.AsyncCallbacks)

Example 3 with TopicFencedException

use of com.yahoo.pulsar.broker.service.BrokerServiceException.TopicFencedException in project pulsar by yahoo.

the class PersistentTopic method subscribe.

@Override
public CompletableFuture<Consumer> subscribe(final ServerCnx cnx, String subscriptionName, long consumerId, SubType subType, int priorityLevel, String consumerName) {
    final CompletableFuture<Consumer> future = new CompletableFuture<>();
    if (hasBatchMessagePublished && !cnx.isBatchMessageCompatibleVersion()) {
        if (log.isDebugEnabled()) {
            log.debug("[{}] Consumer doesn't support batch-message {}", topic, subscriptionName);
        }
        future.completeExceptionally(new UnsupportedVersionException("Consumer doesn't support batch-message"));
        return future;
    }
    if (subscriptionName.startsWith(replicatorPrefix)) {
        log.warn("[{}] Failed to create subscription for {}", topic, subscriptionName);
        future.completeExceptionally(new NamingException("Subscription with reserved subscription name attempted"));
        return future;
    }
    lock.readLock().lock();
    try {
        if (isFenced) {
            log.warn("[{}] Attempting to subscribe to a fenced topic", topic);
            future.completeExceptionally(new TopicFencedException("Topic is temporarily unavailable"));
            return future;
        }
        USAGE_COUNT_UPDATER.incrementAndGet(this);
        if (log.isDebugEnabled()) {
            log.debug("[{}] [{}] [{}] Added consumer -- count: {}", topic, subscriptionName, consumerName, USAGE_COUNT_UPDATER.get(this));
        }
    } finally {
        lock.readLock().unlock();
    }
    ledger.asyncOpenCursor(Codec.encode(subscriptionName), new OpenCursorCallback() {

        @Override
        public void openCursorComplete(ManagedCursor cursor, Object ctx) {
            if (log.isDebugEnabled()) {
                log.debug("[{}][{}] Opened cursor for {} {}", topic, subscriptionName, consumerId, consumerName);
            }
            try {
                PersistentSubscription subscription = subscriptions.computeIfAbsent(subscriptionName, name -> new PersistentSubscription(PersistentTopic.this, cursor));
                Consumer consumer = new Consumer(subscription, subType, consumerId, priorityLevel, consumerName, brokerService.pulsar().getConfiguration().getMaxUnackedMessagesPerConsumer(), cnx, cnx.getRole());
                subscription.addConsumer(consumer);
                if (!cnx.isActive()) {
                    consumer.close();
                    if (log.isDebugEnabled()) {
                        log.debug("[{}] [{}] [{}] Subscribe failed -- count: {}", topic, subscriptionName, consumer.consumerName(), USAGE_COUNT_UPDATER.get(PersistentTopic.this));
                    }
                    future.completeExceptionally(new BrokerServiceException("Connection was closed while the opening the cursor "));
                } else {
                    log.info("[{}][{}] Created new subscription for {}", topic, subscriptionName, consumerId);
                    future.complete(consumer);
                }
            } catch (BrokerServiceException e) {
                if (e instanceof ConsumerBusyException) {
                    log.warn("[{}][{}] Consumer {} {} already connected", topic, subscriptionName, consumerId, consumerName);
                } else if (e instanceof SubscriptionBusyException) {
                    log.warn("[{}][{}] {}", topic, subscriptionName, e.getMessage());
                }
                USAGE_COUNT_UPDATER.decrementAndGet(PersistentTopic.this);
                future.completeExceptionally(e);
            }
        }

        @Override
        public void openCursorFailed(ManagedLedgerException exception, Object ctx) {
            log.warn("[{}] Failed to create subscription for {}", topic, subscriptionName);
            USAGE_COUNT_UPDATER.decrementAndGet(PersistentTopic.this);
            future.completeExceptionally(new PersistenceException(exception));
        }
    }, null);
    return future;
}
Also used : ReplicationMetrics(com.yahoo.pulsar.broker.stats.ReplicationMetrics) SubType(com.yahoo.pulsar.common.api.proto.PulsarApi.CommandSubscribe.SubType) PersistentSubscriptionStats(com.yahoo.pulsar.common.policies.data.PersistentSubscriptionStats) LedgerInfo(com.yahoo.pulsar.common.policies.data.PersistentTopicInternalStats.LedgerInfo) NamingException(com.yahoo.pulsar.broker.service.BrokerServiceException.NamingException) ConsumerStats(com.yahoo.pulsar.common.policies.data.ConsumerStats) CloseCallback(org.apache.bookkeeper.mledger.AsyncCallbacks.CloseCallback) PersistentTopicInternalStats(com.yahoo.pulsar.common.policies.data.PersistentTopicInternalStats) LoggerFactory(org.slf4j.LoggerFactory) ObjectObjectHashMap(com.carrotsearch.hppc.ObjectObjectHashMap) NamespaceBundleStats(com.yahoo.pulsar.common.policies.data.loadbalancer.NamespaceBundleStats) Policies(com.yahoo.pulsar.common.policies.data.Policies) BacklogQuota(com.yahoo.pulsar.common.policies.data.BacklogQuota) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) OpenCursorCallback(org.apache.bookkeeper.mledger.AsyncCallbacks.OpenCursorCallback) ManagedCursor(org.apache.bookkeeper.mledger.ManagedCursor) TopicBusyException(com.yahoo.pulsar.broker.service.BrokerServiceException.TopicBusyException) ManagedLedger(org.apache.bookkeeper.mledger.ManagedLedger) ReplicatorStats(com.yahoo.pulsar.common.policies.data.ReplicatorStats) DeleteCursorCallback(org.apache.bookkeeper.mledger.AsyncCallbacks.DeleteCursorCallback) FutureUtil(com.yahoo.pulsar.client.util.FutureUtil) Objects(com.google.common.base.Objects) PositionImpl(org.apache.bookkeeper.mledger.impl.PositionImpl) DestinationName(com.yahoo.pulsar.common.naming.DestinationName) PersistenceException(com.yahoo.pulsar.broker.service.BrokerServiceException.PersistenceException) CursorStats(com.yahoo.pulsar.common.policies.data.PersistentTopicInternalStats.CursorStats) Set(java.util.Set) Position(org.apache.bookkeeper.mledger.Position) Instant(java.time.Instant) IndividualDeletedEntries(org.apache.bookkeeper.mledger.ManagedCursor.IndividualDeletedEntries) TopicFencedException(com.yahoo.pulsar.broker.service.BrokerServiceException.TopicFencedException) Lists(com.beust.jcommander.internal.Lists) ZoneId(java.time.ZoneId) Sets(com.google.common.collect.Sets) AddEntryCallback(org.apache.bookkeeper.mledger.AsyncCallbacks.AddEntryCallback) BrokerService(com.yahoo.pulsar.broker.service.BrokerService) ManagedLedgerException(org.apache.bookkeeper.mledger.ManagedLedgerException) Topic(com.yahoo.pulsar.broker.service.Topic) ManagedCursorImpl(org.apache.bookkeeper.mledger.impl.ManagedCursorImpl) Consumer(com.yahoo.pulsar.broker.service.Consumer) List(java.util.List) ManagedLedgerFencedException(org.apache.bookkeeper.mledger.ManagedLedgerException.ManagedLedgerFencedException) AdminResource(com.yahoo.pulsar.broker.admin.AdminResource) AsyncCallbacks(org.apache.bookkeeper.mledger.AsyncCallbacks) StatsOutputStream(com.yahoo.pulsar.utils.StatsOutputStream) Entry(org.apache.bookkeeper.mledger.Entry) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) CompletableFuture(java.util.concurrent.CompletableFuture) UnsupportedVersionException(com.yahoo.pulsar.broker.service.BrokerServiceException.UnsupportedVersionException) ReentrantReadWriteLock(java.util.concurrent.locks.ReentrantReadWriteLock) SubscriptionBusyException(com.yahoo.pulsar.broker.service.BrokerServiceException.SubscriptionBusyException) MessageImpl(com.yahoo.pulsar.client.impl.MessageImpl) PersistentTopicStats(com.yahoo.pulsar.common.policies.data.PersistentTopicStats) ServerCnx(com.yahoo.pulsar.broker.service.ServerCnx) ByteBuf(io.netty.buffer.ByteBuf) FastThreadLocal(io.netty.util.concurrent.FastThreadLocal) ConcurrentOpenHashSet(com.yahoo.pulsar.common.util.collections.ConcurrentOpenHashSet) ConsumerBusyException(com.yahoo.pulsar.broker.service.BrokerServiceException.ConsumerBusyException) ConcurrentOpenHashMap(com.yahoo.pulsar.common.util.collections.ConcurrentOpenHashMap) Codec(com.yahoo.pulsar.common.util.Codec) ManagedLedgerImpl(org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl) ClusterReplicationMetrics(com.yahoo.pulsar.broker.stats.ClusterReplicationMetrics) Logger(org.slf4j.Logger) KeeperException(org.apache.zookeeper.KeeperException) BrokerServiceException(com.yahoo.pulsar.broker.service.BrokerServiceException) PublisherStats(com.yahoo.pulsar.common.policies.data.PublisherStats) Producer(com.yahoo.pulsar.broker.service.Producer) ServerMetadataException(com.yahoo.pulsar.broker.service.BrokerServiceException.ServerMetadataException) AtomicLongFieldUpdater(java.util.concurrent.atomic.AtomicLongFieldUpdater) NamespaceStats(com.yahoo.pulsar.broker.stats.NamespaceStats) Maps(com.google.common.collect.Maps) TimeUnit(java.util.concurrent.TimeUnit) DateTimeFormatter(java.time.format.DateTimeFormatter) Collections(java.util.Collections) TopicFencedException(com.yahoo.pulsar.broker.service.BrokerServiceException.TopicFencedException) BrokerServiceException(com.yahoo.pulsar.broker.service.BrokerServiceException) ManagedCursor(org.apache.bookkeeper.mledger.ManagedCursor) CompletableFuture(java.util.concurrent.CompletableFuture) OpenCursorCallback(org.apache.bookkeeper.mledger.AsyncCallbacks.OpenCursorCallback) ConsumerBusyException(com.yahoo.pulsar.broker.service.BrokerServiceException.ConsumerBusyException) ManagedLedgerException(org.apache.bookkeeper.mledger.ManagedLedgerException) Consumer(com.yahoo.pulsar.broker.service.Consumer) PersistenceException(com.yahoo.pulsar.broker.service.BrokerServiceException.PersistenceException) NamingException(com.yahoo.pulsar.broker.service.BrokerServiceException.NamingException) SubscriptionBusyException(com.yahoo.pulsar.broker.service.BrokerServiceException.SubscriptionBusyException) UnsupportedVersionException(com.yahoo.pulsar.broker.service.BrokerServiceException.UnsupportedVersionException)

Aggregations

TopicFencedException (com.yahoo.pulsar.broker.service.BrokerServiceException.TopicFencedException)3 CompletableFuture (java.util.concurrent.CompletableFuture)3 ManagedLedgerException (org.apache.bookkeeper.mledger.ManagedLedgerException)3 PersistenceException (com.yahoo.pulsar.broker.service.BrokerServiceException.PersistenceException)2 TopicBusyException (com.yahoo.pulsar.broker.service.BrokerServiceException.TopicBusyException)2 AsyncCallbacks (org.apache.bookkeeper.mledger.AsyncCallbacks)2 Lists (com.beust.jcommander.internal.Lists)1 ObjectObjectHashMap (com.carrotsearch.hppc.ObjectObjectHashMap)1 Objects (com.google.common.base.Objects)1 Preconditions.checkArgument (com.google.common.base.Preconditions.checkArgument)1 Maps (com.google.common.collect.Maps)1 Sets (com.google.common.collect.Sets)1 AdminResource (com.yahoo.pulsar.broker.admin.AdminResource)1 BrokerService (com.yahoo.pulsar.broker.service.BrokerService)1 BrokerServiceException (com.yahoo.pulsar.broker.service.BrokerServiceException)1 ConsumerBusyException (com.yahoo.pulsar.broker.service.BrokerServiceException.ConsumerBusyException)1 NamingException (com.yahoo.pulsar.broker.service.BrokerServiceException.NamingException)1 ServerMetadataException (com.yahoo.pulsar.broker.service.BrokerServiceException.ServerMetadataException)1 SubscriptionBusyException (com.yahoo.pulsar.broker.service.BrokerServiceException.SubscriptionBusyException)1 UnsupportedVersionException (com.yahoo.pulsar.broker.service.BrokerServiceException.UnsupportedVersionException)1