Search in sources :

Example 36 with HazelcastException

use of com.hazelcast.core.HazelcastException in project hazelcast by hazelcast.

the class MapProxyImpl method aggregate.

@Override
public <SuppliedValue, Result> Result aggregate(Supplier<K, V, SuppliedValue> supplier, Aggregation<K, SuppliedValue, Result> aggregation, JobTracker jobTracker) {
    checkTrue(NATIVE != mapConfig.getInMemoryFormat(), "NATIVE storage format is not supported for MapReduce");
    try {
        isNotNull(jobTracker, "jobTracker");
        KeyValueSource<K, V> keyValueSource = KeyValueSource.fromMap(this);
        Job<K, V> job = jobTracker.newJob(keyValueSource);
        Mapper mapper = aggregation.getMapper(supplier);
        CombinerFactory combinerFactory = aggregation.getCombinerFactory();
        ReducerFactory reducerFactory = aggregation.getReducerFactory();
        Collator collator = aggregation.getCollator();
        MappingJob mappingJob = job.mapper(mapper);
        ReducingSubmittableJob reducingJob;
        if (combinerFactory == null) {
            reducingJob = mappingJob.reducer(reducerFactory);
        } else {
            reducingJob = mappingJob.combiner(combinerFactory).reducer(reducerFactory);
        }
        ICompletableFuture<Result> future = reducingJob.submit(collator);
        return future.get();
    } catch (Exception e) {
        // TODO: not what we want, because it can lead to wrapping of HazelcastException
        throw new HazelcastException(e);
    }
}
Also used : MappingJob(com.hazelcast.mapreduce.MappingJob) HazelcastException(com.hazelcast.core.HazelcastException) ReducerFactory(com.hazelcast.mapreduce.ReducerFactory) HazelcastException(com.hazelcast.core.HazelcastException) Collator(com.hazelcast.mapreduce.Collator) AggregationResult(com.hazelcast.map.impl.query.AggregationResult) QueryResult(com.hazelcast.map.impl.query.QueryResult) CombinerFactory(com.hazelcast.mapreduce.CombinerFactory) Mapper(com.hazelcast.mapreduce.Mapper) ReducingSubmittableJob(com.hazelcast.mapreduce.ReducingSubmittableJob)

Example 37 with HazelcastException

use of com.hazelcast.core.HazelcastException in project hazelcast by hazelcast.

the class ClientMapProxy method aggregate.

@Override
public <SuppliedValue, Result> Result aggregate(Supplier<K, V, SuppliedValue> supplier, Aggregation<K, SuppliedValue, Result> aggregation, JobTracker jobTracker) {
    try {
        Preconditions.isNotNull(jobTracker, "jobTracker");
        KeyValueSource<K, V> keyValueSource = KeyValueSource.fromMap(this);
        Job<K, V> job = jobTracker.newJob(keyValueSource);
        Mapper mapper = aggregation.getMapper(supplier);
        CombinerFactory combinerFactory = aggregation.getCombinerFactory();
        ReducerFactory reducerFactory = aggregation.getReducerFactory();
        Collator collator = aggregation.getCollator();
        MappingJob mappingJob = job.mapper(mapper);
        ReducingSubmittableJob reducingJob;
        if (combinerFactory != null) {
            reducingJob = mappingJob.combiner(combinerFactory).reducer(reducerFactory);
        } else {
            reducingJob = mappingJob.reducer(reducerFactory);
        }
        ICompletableFuture<Result> future = reducingJob.submit(collator);
        return future.get();
    } catch (Exception e) {
        throw new HazelcastException(e);
    }
}
Also used : MappingJob(com.hazelcast.mapreduce.MappingJob) HazelcastException(com.hazelcast.core.HazelcastException) ReducerFactory(com.hazelcast.mapreduce.ReducerFactory) HazelcastException(com.hazelcast.core.HazelcastException) Collator(com.hazelcast.mapreduce.Collator) CombinerFactory(com.hazelcast.mapreduce.CombinerFactory) Mapper(com.hazelcast.mapreduce.Mapper) ReducingSubmittableJob(com.hazelcast.mapreduce.ReducingSubmittableJob)

Example 38 with HazelcastException

use of com.hazelcast.core.HazelcastException in project hazelcast by hazelcast.

the class QueueContainer method compareAndRemove.

/**
     * Compares if the queue contains the items in the dataList and removes them according to the retain parameter. If
     * the retain parameter is true, it will remove items which are not in the dataList (retaining the items which are in the
     * list). If the retain parameter is false, it will remove items which are in the dataList (retaining all other items which
     * are not in the list).
     *
     * Note : this method will trigger store load.
     *
     * @param dataList the list of items which are to be retained in the queue or which are to be removed from the queue
     * @param retain does the method retain the items in the list (true) or remove them from the queue (false)
     * @return map of removed items by id
     */
public Map<Long, Data> compareAndRemove(Collection<Data> dataList, boolean retain) {
    final LinkedHashMap<Long, Data> map = new LinkedHashMap<Long, Data>();
    for (QueueItem item : getItemQueue()) {
        if (item.getData() == null && store.isEnabled()) {
            try {
                load(item);
            } catch (Exception e) {
                throw new HazelcastException(e);
            }
        }
        boolean contains = dataList.contains(item.getData());
        if ((retain && !contains) || (!retain && contains)) {
            map.put(item.getItemId(), item.getData());
        }
    }
    mapIterateAndRemove(map);
    return map;
}
Also used : HazelcastException(com.hazelcast.core.HazelcastException) Data(com.hazelcast.nio.serialization.Data) TxQueueItem(com.hazelcast.collection.impl.txnqueue.TxQueueItem) TransactionException(com.hazelcast.transaction.TransactionException) HazelcastException(com.hazelcast.core.HazelcastException) IOException(java.io.IOException) LinkedHashMap(java.util.LinkedHashMap)

Example 39 with HazelcastException

use of com.hazelcast.core.HazelcastException in project hazelcast by hazelcast.

the class QueueContainer method addAll.

/**
     * Adds all items from the {@code dataList} to the queue. The data will be stored in the queue store if configured and
     * enabled. If the store is enabled, only {@link QueueStoreWrapper#getMemoryLimit()} item data will be stored in memory.
     * Cancels the eviction if one is scheduled.
     *
     * @param dataList the items to be added to the queue and stored in the queue store
     * @return map of item ID and items added
     */
public Map<Long, Data> addAll(Collection<Data> dataList) {
    final Map<Long, Data> map = new HashMap<Long, Data>(dataList.size());
    final List<QueueItem> list = new ArrayList<QueueItem>(dataList.size());
    for (Data data : dataList) {
        final QueueItem item = new QueueItem(this, nextId(), null);
        if (!store.isEnabled() || store.getMemoryLimit() > getItemQueue().size()) {
            item.setData(data);
        }
        map.put(item.getItemId(), data);
        list.add(item);
    }
    if (store.isEnabled() && !map.isEmpty()) {
        try {
            store.storeAll(map);
        } catch (Exception e) {
            throw new HazelcastException(e);
        }
    }
    if (!list.isEmpty()) {
        getItemQueue().addAll(list);
        cancelEvictionIfExists();
    }
    return map;
}
Also used : HazelcastException(com.hazelcast.core.HazelcastException) HashMap(java.util.HashMap) LinkedHashMap(java.util.LinkedHashMap) ArrayList(java.util.ArrayList) Data(com.hazelcast.nio.serialization.Data) TxQueueItem(com.hazelcast.collection.impl.txnqueue.TxQueueItem) TransactionException(com.hazelcast.transaction.TransactionException) HazelcastException(com.hazelcast.core.HazelcastException) IOException(java.io.IOException)

Example 40 with HazelcastException

use of com.hazelcast.core.HazelcastException in project hazelcast by hazelcast.

the class QueueContainer method mapDrainIterator.

public void mapDrainIterator(int maxSize, Map map) {
    Iterator<QueueItem> iter = getItemQueue().iterator();
    for (int i = 0; i < maxSize; i++) {
        QueueItem item = iter.next();
        if (store.isEnabled() && item.getData() == null) {
            try {
                load(item);
            } catch (Exception e) {
                throw new HazelcastException(e);
            }
        }
        map.put(item.getItemId(), item.getData());
    }
}
Also used : HazelcastException(com.hazelcast.core.HazelcastException) TxQueueItem(com.hazelcast.collection.impl.txnqueue.TxQueueItem) TransactionException(com.hazelcast.transaction.TransactionException) HazelcastException(com.hazelcast.core.HazelcastException) IOException(java.io.IOException)

Aggregations

HazelcastException (com.hazelcast.core.HazelcastException)57 IOException (java.io.IOException)18 TxQueueItem (com.hazelcast.collection.impl.txnqueue.TxQueueItem)11 TransactionException (com.hazelcast.transaction.TransactionException)11 QuickTest (com.hazelcast.test.annotation.QuickTest)10 Test (org.junit.Test)10 HazelcastInstanceNotActiveException (com.hazelcast.core.HazelcastInstanceNotActiveException)8 ParallelTest (com.hazelcast.test.annotation.ParallelTest)8 OOfflineNodeException (com.orientechnologies.common.concur.OOfflineNodeException)8 File (java.io.File)8 ONeedRetryException (com.orientechnologies.common.concur.ONeedRetryException)7 OException (com.orientechnologies.common.exception.OException)7 OIOException (com.orientechnologies.common.io.OIOException)7 ODistributedRedirectException (com.orientechnologies.orient.enterprise.channel.binary.ODistributedRedirectException)7 OCallable (com.orientechnologies.common.util.OCallable)6 FileInputStream (java.io.FileInputStream)5 Callable (java.util.concurrent.Callable)5 Collator (com.hazelcast.mapreduce.Collator)4 CombinerFactory (com.hazelcast.mapreduce.CombinerFactory)4 Mapper (com.hazelcast.mapreduce.Mapper)4