Search in sources :

Example 1 with ExpiredCallback

use of com.alibaba.jstorm.utils.ExpiredCallback in project jstorm by alibaba.

the class NimbusData method createFileHandler.

public void createFileHandler() {
    ExpiredCallback<Object, Object> expiredCallback = new ExpiredCallback<Object, Object>() {

        @Override
        public void expire(Object key, Object val) {
            try {
                LOG.info("Close file " + String.valueOf(key));
                if (val != null) {
                    if (val instanceof Channel) {
                        Channel channel = (Channel) val;
                        channel.close();
                    } else if (val instanceof BufferFileInputStream) {
                        BufferFileInputStream is = (BufferFileInputStream) val;
                        is.close();
                    }
                }
            } catch (IOException e) {
                LOG.error(e.getMessage(), e);
            }
        }
    };
    int file_copy_expiration_secs = JStormUtils.parseInt(conf.get(Config.NIMBUS_FILE_COPY_EXPIRATION_SECS), 30);
    uploaders = new TimeCacheMap<Object, Object>(file_copy_expiration_secs, expiredCallback);
    downloaders = new TimeCacheMap<Object, Object>(file_copy_expiration_secs, expiredCallback);
}
Also used : ExpiredCallback(com.alibaba.jstorm.utils.ExpiredCallback) Channel(java.nio.channels.Channel) IOException(java.io.IOException) BufferFileInputStream(backtype.storm.utils.BufferFileInputStream)

Example 2 with ExpiredCallback

use of com.alibaba.jstorm.utils.ExpiredCallback in project jstorm by alibaba.

the class NimbusData method mkBlobCacheMap.

public void mkBlobCacheMap() {
    ExpiredCallback<Object, Object> expiredCallback = new ExpiredCallback<Object, Object>() {

        @Override
        public void expire(Object key, Object val) {
            try {
                LOG.debug("Close blob file " + String.valueOf(key));
                if (val != null) {
                    if (val instanceof AtomicOutputStream) {
                        AtomicOutputStream stream = (AtomicOutputStream) val;
                        stream.cancel();
                        stream.close();
                    } else if (val instanceof BufferInputStream) {
                        BufferInputStream is = (BufferInputStream) val;
                        is.close();
                    }
                }
            } catch (IOException e) {
                LOG.error(e.getMessage(), e);
            }
        }
    };
    int expiration_secs = JStormUtils.parseInt(conf.get(Config.NIMBUS_FILE_COPY_EXPIRATION_SECS), 30);
    blobUploaders = new TimeCacheMap<Object, Object>(expiration_secs, expiredCallback);
    blobDownloaders = new TimeCacheMap<Object, Object>(expiration_secs, expiredCallback);
    blobListers = new TimeCacheMap<Object, Object>(expiration_secs, null);
}
Also used : BufferInputStream(backtype.storm.utils.BufferInputStream) ExpiredCallback(com.alibaba.jstorm.utils.ExpiredCallback) AtomicOutputStream(com.alibaba.jstorm.blobstore.AtomicOutputStream) IOException(java.io.IOException)

Aggregations

ExpiredCallback (com.alibaba.jstorm.utils.ExpiredCallback)2 IOException (java.io.IOException)2 BufferFileInputStream (backtype.storm.utils.BufferFileInputStream)1 BufferInputStream (backtype.storm.utils.BufferInputStream)1 AtomicOutputStream (com.alibaba.jstorm.blobstore.AtomicOutputStream)1 Channel (java.nio.channels.Channel)1