Search in sources :

Example 1 with MQConsumerInner

use of org.apache.rocketmq.client.impl.consumer.MQConsumerInner in project rocketmq-rocketmq-all-4.1.0-incubating by lirenzuo.

the class MQClientInstance method adjustThreadPool.

public void adjustThreadPool() {
    Iterator<Entry<String, MQConsumerInner>> it = this.consumerTable.entrySet().iterator();
    while (it.hasNext()) {
        Entry<String, MQConsumerInner> entry = it.next();
        MQConsumerInner impl = entry.getValue();
        if (impl != null) {
            try {
                if (impl instanceof DefaultMQPushConsumerImpl) {
                    DefaultMQPushConsumerImpl dmq = (DefaultMQPushConsumerImpl) impl;
                    dmq.adjustThreadPool();
                }
            } catch (Exception e) {
            }
        }
    }
}
Also used : MQConsumerInner(org.apache.rocketmq.client.impl.consumer.MQConsumerInner) DefaultMQPushConsumerImpl(org.apache.rocketmq.client.impl.consumer.DefaultMQPushConsumerImpl) Entry(java.util.Map.Entry) MQClientException(org.apache.rocketmq.client.exception.MQClientException) UnsupportedEncodingException(java.io.UnsupportedEncodingException) RemotingException(org.apache.rocketmq.remoting.exception.RemotingException) MQBrokerException(org.apache.rocketmq.client.exception.MQBrokerException)

Example 2 with MQConsumerInner

use of org.apache.rocketmq.client.impl.consumer.MQConsumerInner in project rocketmq-rocketmq-all-4.1.0-incubating by lirenzuo.

the class MQClientInstance method prepareHeartbeatData.

private HeartbeatData prepareHeartbeatData() {
    HeartbeatData heartbeatData = new HeartbeatData();
    // clientID
    heartbeatData.setClientID(this.clientId);
    // Consumer
    for (Map.Entry<String, MQConsumerInner> entry : this.consumerTable.entrySet()) {
        MQConsumerInner impl = entry.getValue();
        if (impl != null) {
            ConsumerData consumerData = new ConsumerData();
            consumerData.setGroupName(impl.groupName());
            consumerData.setConsumeType(impl.consumeType());
            consumerData.setMessageModel(impl.messageModel());
            consumerData.setConsumeFromWhere(impl.consumeFromWhere());
            consumerData.getSubscriptionDataSet().addAll(impl.subscriptions());
            consumerData.setUnitMode(impl.isUnitMode());
            heartbeatData.getConsumerDataSet().add(consumerData);
        }
    }
    // Producer
    for (Map.Entry<String, MQProducerInner> /* group */
    entry : this.producerTable.entrySet()) {
        MQProducerInner impl = entry.getValue();
        if (impl != null) {
            ProducerData producerData = new ProducerData();
            producerData.setGroupName(entry.getKey());
            heartbeatData.getProducerDataSet().add(producerData);
        }
    }
    return heartbeatData;
}
Also used : MQConsumerInner(org.apache.rocketmq.client.impl.consumer.MQConsumerInner) MQProducerInner(org.apache.rocketmq.client.impl.producer.MQProducerInner) ConsumerData(org.apache.rocketmq.common.protocol.heartbeat.ConsumerData) HeartbeatData(org.apache.rocketmq.common.protocol.heartbeat.HeartbeatData) Map(java.util.Map) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) HashMap(java.util.HashMap) ConcurrentMap(java.util.concurrent.ConcurrentMap) ProducerData(org.apache.rocketmq.common.protocol.heartbeat.ProducerData)

Example 3 with MQConsumerInner

use of org.apache.rocketmq.client.impl.consumer.MQConsumerInner in project rocketmq-rocketmq-all-4.1.0-incubating by lirenzuo.

the class MQClientInstance method checkClientInBroker.

public void checkClientInBroker() throws MQClientException {
    Iterator<Entry<String, MQConsumerInner>> it = this.consumerTable.entrySet().iterator();
    while (it.hasNext()) {
        Entry<String, MQConsumerInner> entry = it.next();
        Set<SubscriptionData> subscriptionInner = entry.getValue().subscriptions();
        if (subscriptionInner == null || subscriptionInner.isEmpty()) {
            return;
        }
        for (SubscriptionData subscriptionData : subscriptionInner) {
            if (ExpressionType.isTagType(subscriptionData.getExpressionType())) {
                continue;
            }
            // may need to check one broker every cluster...
            // assume that the configs of every broker in cluster are the the same.
            String addr = findBrokerAddrByTopic(subscriptionData.getTopic());
            if (addr != null) {
                try {
                    this.getMQClientAPIImpl().checkClientInBroker(addr, entry.getKey(), this.clientId, subscriptionData, 3 * 1000);
                } catch (Exception e) {
                    if (e instanceof MQClientException) {
                        throw (MQClientException) e;
                    } else {
                        throw new MQClientException("Check client in broker error, maybe because you use " + subscriptionData.getExpressionType() + " to filter message, but server has not been upgraded to support!" + "This error would not affect the launch of consumer, but may has impact on message receiving if you " + "have use the new features which are not supported by server, please check the log!", e);
                    }
                }
            }
        }
    }
}
Also used : MQConsumerInner(org.apache.rocketmq.client.impl.consumer.MQConsumerInner) Entry(java.util.Map.Entry) SubscriptionData(org.apache.rocketmq.common.protocol.heartbeat.SubscriptionData) MQClientException(org.apache.rocketmq.client.exception.MQClientException) UnsupportedEncodingException(java.io.UnsupportedEncodingException) RemotingException(org.apache.rocketmq.remoting.exception.RemotingException) MQBrokerException(org.apache.rocketmq.client.exception.MQBrokerException) MQClientException(org.apache.rocketmq.client.exception.MQClientException)

Example 4 with MQConsumerInner

use of org.apache.rocketmq.client.impl.consumer.MQConsumerInner in project rocketmq-rocketmq-all-4.1.0-incubating by lirenzuo.

the class MQClientInstance method persistAllConsumerOffset.

private void persistAllConsumerOffset() {
    Iterator<Entry<String, MQConsumerInner>> it = this.consumerTable.entrySet().iterator();
    while (it.hasNext()) {
        Entry<String, MQConsumerInner> entry = it.next();
        MQConsumerInner impl = entry.getValue();
        impl.persistConsumerOffset();
    }
}
Also used : MQConsumerInner(org.apache.rocketmq.client.impl.consumer.MQConsumerInner) Entry(java.util.Map.Entry)

Example 5 with MQConsumerInner

use of org.apache.rocketmq.client.impl.consumer.MQConsumerInner in project rocketmq-rocketmq-all-4.1.0-incubating by lirenzuo.

the class MQClientInstance method resetOffset.

public void resetOffset(String topic, String group, Map<MessageQueue, Long> offsetTable) {
    DefaultMQPushConsumerImpl consumer = null;
    try {
        MQConsumerInner impl = this.consumerTable.get(group);
        if (impl != null && impl instanceof DefaultMQPushConsumerImpl) {
            consumer = (DefaultMQPushConsumerImpl) impl;
        } else {
            log.info("[reset-offset] consumer dose not exist. group={}", group);
            return;
        }
        consumer.suspend();
        ConcurrentMap<MessageQueue, ProcessQueue> processQueueTable = consumer.getRebalanceImpl().getProcessQueueTable();
        for (Map.Entry<MessageQueue, ProcessQueue> entry : processQueueTable.entrySet()) {
            MessageQueue mq = entry.getKey();
            if (topic.equals(mq.getTopic()) && offsetTable.containsKey(mq)) {
                ProcessQueue pq = entry.getValue();
                pq.setDropped(true);
                pq.clear();
            }
        }
        try {
            TimeUnit.SECONDS.sleep(10);
        } catch (InterruptedException e) {
        // 
        }
        Iterator<MessageQueue> iterator = processQueueTable.keySet().iterator();
        while (iterator.hasNext()) {
            MessageQueue mq = iterator.next();
            Long offset = offsetTable.get(mq);
            if (topic.equals(mq.getTopic()) && offset != null) {
                try {
                    consumer.updateConsumeOffset(mq, offset);
                    consumer.getRebalanceImpl().removeUnnecessaryMessageQueue(mq, processQueueTable.get(mq));
                    iterator.remove();
                } catch (Exception e) {
                    log.warn("reset offset failed. group={}, {}", group, mq, e);
                }
            }
        }
    } finally {
        if (consumer != null) {
            consumer.resume();
        }
    }
}
Also used : MQConsumerInner(org.apache.rocketmq.client.impl.consumer.MQConsumerInner) DefaultMQPushConsumerImpl(org.apache.rocketmq.client.impl.consumer.DefaultMQPushConsumerImpl) MessageQueue(org.apache.rocketmq.common.message.MessageQueue) ProcessQueue(org.apache.rocketmq.client.impl.consumer.ProcessQueue) AtomicLong(java.util.concurrent.atomic.AtomicLong) Map(java.util.Map) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) HashMap(java.util.HashMap) ConcurrentMap(java.util.concurrent.ConcurrentMap) MQClientException(org.apache.rocketmq.client.exception.MQClientException) UnsupportedEncodingException(java.io.UnsupportedEncodingException) RemotingException(org.apache.rocketmq.remoting.exception.RemotingException) MQBrokerException(org.apache.rocketmq.client.exception.MQBrokerException)

Aggregations

MQConsumerInner (org.apache.rocketmq.client.impl.consumer.MQConsumerInner)18 UnsupportedEncodingException (java.io.UnsupportedEncodingException)10 Entry (java.util.Map.Entry)10 MQBrokerException (org.apache.rocketmq.client.exception.MQBrokerException)10 MQClientException (org.apache.rocketmq.client.exception.MQClientException)10 RemotingException (org.apache.rocketmq.remoting.exception.RemotingException)10 DefaultMQPushConsumerImpl (org.apache.rocketmq.client.impl.consumer.DefaultMQPushConsumerImpl)6 HashMap (java.util.HashMap)4 Map (java.util.Map)4 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)4 ConcurrentMap (java.util.concurrent.ConcurrentMap)4 MQProducerInner (org.apache.rocketmq.client.impl.producer.MQProducerInner)4 SubscriptionData (org.apache.rocketmq.common.protocol.heartbeat.SubscriptionData)4 HashSet (java.util.HashSet)2 Iterator (java.util.Iterator)2 Set (java.util.Set)2 AtomicLong (java.util.concurrent.atomic.AtomicLong)2 ProcessQueue (org.apache.rocketmq.client.impl.consumer.ProcessQueue)2 TopicPublishInfo (org.apache.rocketmq.client.impl.producer.TopicPublishInfo)2 MessageQueue (org.apache.rocketmq.common.message.MessageQueue)2