Search in sources :

Example 1 with StandardThreadFactory

use of com.mendmix.common.async.StandardThreadExecutor.StandardThreadFactory in project jeesuite-libs by vakinge.

the class MNSConsumer method start.

private void start() {
    CloudQueue queue = MNSClientInstance.createQueueIfAbsent(queueName);
    initTopicHanlders();
    fetchExecutor = new StandardThreadExecutor(1, 1, 0, TimeUnit.SECONDS, 1, new StandardThreadFactory("mns-Fetch-Executor"));
    int maxThread = ResourceUtils.getInt("aliyun.mns.consumer.processThreads", 50);
    semaphore = new Semaphore(maxThread);
    defaultProcessExecutor = new StandardThreadExecutor(1, maxThread, 60, TimeUnit.SECONDS, 1, new StandardThreadFactory("mns-defaultProcess-Executor"));
    fetchExecutor.submit(new Worker(queue));
    logger.info("start work for queue Ok -> queue:{}", queue.getQueueURL());
}
Also used : StandardThreadExecutor(com.mendmix.common.async.StandardThreadExecutor) StandardThreadFactory(com.mendmix.common.async.StandardThreadExecutor.StandardThreadFactory) Semaphore(java.util.concurrent.Semaphore) CloudQueue(com.aliyun.mns.client.CloudQueue)

Example 2 with StandardThreadFactory

use of com.mendmix.common.async.StandardThreadExecutor.StandardThreadFactory in project jeesuite-libs by vakinge.

the class MQContext method getLogHandleExecutor.

private static ThreadPoolExecutor getLogHandleExecutor() {
    if (context.logHandleExecutor != null)
        return context.logHandleExecutor;
    if (!context.loghandlerEnabled)
        return null;
    synchronized (context) {
        StandardThreadFactory threadFactory = new StandardThreadFactory("mqLogHandleExecutor");
        int nThread = ResourceUtils.getInt("mendmix.amqp.loghandler.threads", 2);
        int capacity = ResourceUtils.getInt("mendmix.amqp.loghandler.queueSize", 1000);
        context.logHandleExecutor = new ThreadPoolExecutor(nThread, nThread, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>(capacity), threadFactory);
        // 忽略topic列表
        if (ResourceUtils.containsProperty("mendmix.amqp.loghandler.ignoreTopics")) {
            String[] topics = ResourceUtils.getProperty("mendmix.amqp.loghandler.ignoreTopics").split(",|;");
            for (String topic : topics) {
                ignoreLogTopics.add(rebuildWithNamespace(topic));
            }
        }
    }
    return context.logHandleExecutor;
}
Also used : StandardThreadFactory(com.mendmix.common.async.StandardThreadExecutor.StandardThreadFactory) ThreadPoolExecutor(java.util.concurrent.ThreadPoolExecutor) LinkedBlockingQueue(java.util.concurrent.LinkedBlockingQueue)

Example 3 with StandardThreadFactory

use of com.mendmix.common.async.StandardThreadExecutor.StandardThreadFactory in project jeesuite-libs by vakinge.

the class RedisConsumerAdapter method start.

@Override
public void start() throws Exception {
    int maxThread = MQContext.getMaxProcessThreads();
    this.fetchExecutor = new ThreadPoolExecutor(1, 1, 0, TimeUnit.SECONDS, new LinkedBlockingQueue<Runnable>(), new StandardThreadFactory("messageFetcher"));
    this.asyncProcessExecutor = new StandardThreadExecutor(1, maxThread, 60, TimeUnit.SECONDS, 1000, new StandardThreadFactory("messageAsyncProcessor"));
    container.setConnectionFactory(connectionFactory);
    container.setSubscriptionExecutor(fetchExecutor);
    container.setTaskExecutor(asyncProcessExecutor);
    // 
    Set<String> topics = messageHandlers.keySet();
    MessageListenerAdapter listener;
    for (String topic : topics) {
        MessageHandlerDelegate delegate = new MessageHandlerDelegate(topic, messageHandlers.get(topic));
        listener = new MessageListenerAdapter(delegate, "onMessage");
        listener.afterPropertiesSet();
        container.addMessageListener(listener, new PatternTopic(topic));
    }
    container.afterPropertiesSet();
    container.start();
}
Also used : MessageListenerAdapter(org.springframework.data.redis.listener.adapter.MessageListenerAdapter) PatternTopic(org.springframework.data.redis.listener.PatternTopic) StandardThreadFactory(com.mendmix.common.async.StandardThreadExecutor.StandardThreadFactory) StandardThreadExecutor(com.mendmix.common.async.StandardThreadExecutor) ThreadPoolExecutor(java.util.concurrent.ThreadPoolExecutor) LinkedBlockingQueue(java.util.concurrent.LinkedBlockingQueue)

Example 4 with StandardThreadFactory

use of com.mendmix.common.async.StandardThreadExecutor.StandardThreadFactory in project jeesuite-libs by vakinge.

the class AbstractConsumer method startWorker.

protected void startWorker() {
    // 异步处理拉取线程默认1
    int fetchCoreThreads = 1;
    int fetchMaxThreads = fetchCoreThreads;
    // 异步处理
    if (MQContext.isAsyncConsumeEnabled()) {
        int maxThread = MQContext.getMaxProcessThreads();
        semaphore = new Semaphore(maxThread);
        this.asyncProcessExecutor = new StandardThreadExecutor(1, maxThread, 60, TimeUnit.SECONDS, maxThread, new StandardThreadFactory("messageAsyncProcessor"));
        // 
        fetchMaxThreads = maxThread;
        logger.info("init asyncProcessExecutor finish -> maxThread:{}", maxThread);
    }
    // 
    this.fetchExecutor = new StandardThreadExecutor(fetchCoreThreads, fetchMaxThreads, 0, TimeUnit.SECONDS, fetchMaxThreads * 10, new StandardThreadFactory("messageFetcher"));
    fetchExecutor.execute(new Worker());
    logger.info("init fetchExecutor finish -> fetchMaxThreads:{}", fetchMaxThreads);
}
Also used : StandardThreadExecutor(com.mendmix.common.async.StandardThreadExecutor) StandardThreadFactory(com.mendmix.common.async.StandardThreadExecutor.StandardThreadFactory) Semaphore(java.util.concurrent.Semaphore)

Example 5 with StandardThreadFactory

use of com.mendmix.common.async.StandardThreadExecutor.StandardThreadFactory in project jeesuite-libs by vakinge.

the class CosProviderServiceFacade method afterPropertiesSet.

@Override
public void afterPropertiesSet() throws Exception {
    // 
    if (defaultBucket == null) {
        defaultBucket = ResourceUtils.getProperty("cos.defaultBucket");
    }
    if (type == null) {
        type = ResourceUtils.getAndValidateProperty("cos.provider");
    }
    if (config == null) {
        config = new CosProviderConfig();
        config.setAccessKey(ResourceUtils.getProperty("cos.accessKey"));
        config.setSecretKey(ResourceUtils.getProperty("cos.secretKey"));
        config.setAppId(ResourceUtils.getProperty("cos.appId"));
        config.setRegionName(ResourceUtils.getProperty("cos.regionName"));
        config.setMaxConnectionsCount(ResourceUtils.getInt("cos.maxConnections", 200));
    }
    if (AliyunProvider.NAME.equals(type)) {
        provider = new AliyunProvider(config);
    } else if (QcloudProvider.NAME.equals(type)) {
        provider = new QcloudProvider(config);
    } else if (QiniuProvider.NAME.equals(type)) {
        provider = new QiniuProvider(config);
    } else {
        throw new JeesuiteBaseException("cos[" + type + "] not support");
    }
    if (defaultBucket != null) {
        BucketConfig bucketConfig = provider.getBucketConfig(defaultBucket);
        bucketConfig.setUrlPrefix(ResourceUtils.getProperty("cos.defaultUrlPrefix"));
        ((AbstractProvider) provider).addBucketConfig(bucketConfig);
    } else {
        Map<String, String> urlPrefixMappings = ResourceUtils.getMappingValues("cos.bucket.urlPrefix.mapping");
        if (urlPrefixMappings != null) {
            urlPrefixMappings.forEach((bucket, urlPrefix) -> {
                BucketConfig bucketConfig = provider.getBucketConfig(defaultBucket);
                bucketConfig.setUrlPrefix(ResourceUtils.getProperty("cos.defaultUrlPrefix"));
                ((AbstractProvider) provider).addBucketConfig(bucketConfig);
            });
        }
    }
    logUrl = ResourceUtils.getProperty("cos.loghandler.url");
    if (logUrl != null && Boolean.parseBoolean(ResourceUtils.getProperty("cos.loghandler.enabled", "true"))) {
        int nThread = ResourceUtils.getInt("cos.loghandler.threads", 1);
        int capacity = ResourceUtils.getInt("cos.loghandler.queueSize", 1000);
        logHandleExecutor = new ThreadPoolExecutor(nThread, nThread, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>(capacity), new StandardThreadFactory("cosLogHandleExecutor"));
        logger.info("init logHandleExecutor OK ,nThread:{},queue:{}", nThread, capacity);
    }
}
Also used : AbstractProvider(com.mendmix.cos.provider.AbstractProvider) AliyunProvider(com.mendmix.cos.provider.aliyun.AliyunProvider) StandardThreadFactory(com.mendmix.common.async.StandardThreadExecutor.StandardThreadFactory) QiniuProvider(com.mendmix.cos.provider.qiniu.QiniuProvider) QcloudProvider(com.mendmix.cos.provider.qcloud.QcloudProvider) LinkedBlockingQueue(java.util.concurrent.LinkedBlockingQueue) JeesuiteBaseException(com.mendmix.common.JeesuiteBaseException) ThreadPoolExecutor(java.util.concurrent.ThreadPoolExecutor)

Aggregations

StandardThreadFactory (com.mendmix.common.async.StandardThreadExecutor.StandardThreadFactory)5 StandardThreadExecutor (com.mendmix.common.async.StandardThreadExecutor)3 LinkedBlockingQueue (java.util.concurrent.LinkedBlockingQueue)3 ThreadPoolExecutor (java.util.concurrent.ThreadPoolExecutor)3 Semaphore (java.util.concurrent.Semaphore)2 CloudQueue (com.aliyun.mns.client.CloudQueue)1 JeesuiteBaseException (com.mendmix.common.JeesuiteBaseException)1 AbstractProvider (com.mendmix.cos.provider.AbstractProvider)1 AliyunProvider (com.mendmix.cos.provider.aliyun.AliyunProvider)1 QcloudProvider (com.mendmix.cos.provider.qcloud.QcloudProvider)1 QiniuProvider (com.mendmix.cos.provider.qiniu.QiniuProvider)1 PatternTopic (org.springframework.data.redis.listener.PatternTopic)1 MessageListenerAdapter (org.springframework.data.redis.listener.adapter.MessageListenerAdapter)1