Search in sources :

Example 1 with PullResult

use of com.alibaba.rocketmq.client.consumer.PullResult in project jstorm by alibaba.

the class MetaSimpleClient method fetchOneBatch.

public List<MessageExt> fetchOneBatch() {
    List<MessageExt> ret = new ArrayList<MessageExt>();
    String subexpress = metaSpoutConfig.getSubExpress();
    for (Entry<MessageQueue, Long> entry : currentOffsets.entrySet()) {
        MessageQueue mq = entry.getKey();
        Long offset = entry.getValue();
        int fetchSize = 0;
        int oneFetchSize = Math.min(oneQueueFetchSize, 32);
        while (fetchSize < oneQueueFetchSize) {
            PullResult pullResult = null;
            try {
                pullResult = consumer.pullBlockIfNotFound(mq, subexpress, offset, oneFetchSize);
                offset = pullResult.getNextBeginOffset();
                PullStatus status = pullResult.getPullStatus();
                if (status == PullStatus.FOUND) {
                    List<MessageExt> msgList = pullResult.getMsgFoundList();
                    ret.addAll(msgList);
                    fetchSize += msgList.size();
                    continue;
                } else if (status == PullStatus.NO_MATCHED_MSG) {
                    continue;
                } else if (status == PullStatus.NO_NEW_MSG) {
                    break;
                } else if (status == PullStatus.OFFSET_ILLEGAL) {
                    break;
                } else {
                    break;
                }
            } catch (Exception e) {
                LOG.warn("Failed to fetch messages of " + mq + ":" + offset, e);
                break;
            }
        }
        backendOffset.put(mq, offset);
    }
    return ret;
}
Also used : MessageExt(com.alibaba.rocketmq.common.message.MessageExt) MessageQueue(com.alibaba.rocketmq.common.message.MessageQueue) ArrayList(java.util.ArrayList) PullStatus(com.alibaba.rocketmq.client.consumer.PullStatus) PullResult(com.alibaba.rocketmq.client.consumer.PullResult) FailedException(backtype.storm.topology.FailedException) MQClientException(com.alibaba.rocketmq.client.exception.MQClientException)

Aggregations

FailedException (backtype.storm.topology.FailedException)1 PullResult (com.alibaba.rocketmq.client.consumer.PullResult)1 PullStatus (com.alibaba.rocketmq.client.consumer.PullStatus)1 MQClientException (com.alibaba.rocketmq.client.exception.MQClientException)1 MessageExt (com.alibaba.rocketmq.common.message.MessageExt)1 MessageQueue (com.alibaba.rocketmq.common.message.MessageQueue)1 ArrayList (java.util.ArrayList)1