Search in sources :

Example 56 with Long2DoubleOpenHashMap

use of it.unimi.dsi.fastutil.longs.Long2DoubleOpenHashMap in project angel by Tencent.

the class SparseLongKeyDoubleVector method resize.

private void resize(int newSize) {
    if (indexToValueMap.size() < newSize) {
        Long2DoubleOpenHashMap oldMap = indexToValueMap;
        indexToValueMap = new Long2DoubleOpenHashMap(newSize);
        indexToValueMap.putAll(oldMap);
    }
}
Also used : Long2DoubleOpenHashMap(it.unimi.dsi.fastutil.longs.Long2DoubleOpenHashMap)

Example 57 with Long2DoubleOpenHashMap

use of it.unimi.dsi.fastutil.longs.Long2DoubleOpenHashMap in project angel by Tencent.

the class ServerSparseDoubleLongKeyRow method readFrom.

@Override
public void readFrom(DataInputStream input) throws IOException {
    try {
        lock.writeLock().lock();
        super.readFrom(input);
        int nnz = input.readInt();
        if (index2ValueMap.size() < nnz) {
            index2ValueMap = new Long2DoubleOpenHashMap(nnz);
        }
        for (int i = 0; i < nnz; i++) {
            index2ValueMap.addTo(input.readLong(), input.readDouble());
        }
    } finally {
        lock.writeLock().unlock();
    }
}
Also used : Long2DoubleOpenHashMap(it.unimi.dsi.fastutil.longs.Long2DoubleOpenHashMap)

Example 58 with Long2DoubleOpenHashMap

use of it.unimi.dsi.fastutil.longs.Long2DoubleOpenHashMap in project angel by Tencent.

the class ServerSparseDoubleLongKeyRow method resizeHashMap.

private void resizeHashMap(int size) {
    if (index2ValueMap.size() < size) {
        Long2DoubleOpenHashMap oldMap = index2ValueMap;
        index2ValueMap = new Long2DoubleOpenHashMap(size);
        index2ValueMap.putAll(oldMap);
    }
}
Also used : Long2DoubleOpenHashMap(it.unimi.dsi.fastutil.longs.Long2DoubleOpenHashMap)

Example 59 with Long2DoubleOpenHashMap

use of it.unimi.dsi.fastutil.longs.Long2DoubleOpenHashMap in project presto by prestodb.

the class KHyperLogLog method uniquenessDistribution.

public Long2DoubleMap uniquenessDistribution(long histogramSize) {
    Long2DoubleMap out = new Long2DoubleOpenHashMap();
    PrimitiveIterator.OfLong iterator = LongStream.rangeClosed(1, histogramSize).iterator();
    while (iterator.hasNext()) {
        // Initialize all entries to zero
        out.put(iterator.nextLong(), 0D);
    }
    int size = minhash.size();
    for (HyperLogLog hll : minhash.values()) {
        long bucket = Math.min(hll.cardinality(), histogramSize);
        out.merge(bucket, (double) 1 / size, Double::sum);
    }
    return out;
}
Also used : PrimitiveIterator(java.util.PrimitiveIterator) Long2DoubleMap(it.unimi.dsi.fastutil.longs.Long2DoubleMap) Long2DoubleOpenHashMap(it.unimi.dsi.fastutil.longs.Long2DoubleOpenHashMap) HyperLogLog(com.facebook.airlift.stats.cardinality.HyperLogLog)

Aggregations

Long2DoubleOpenHashMap (it.unimi.dsi.fastutil.longs.Long2DoubleOpenHashMap)59 Map (java.util.Map)29 LongSet (it.unimi.dsi.fastutil.longs.LongSet)7 ServerSparseDoubleLongKeyRow (com.tencent.angel.ps.impl.matrix.ServerSparseDoubleLongKeyRow)3 ArrayList (java.util.ArrayList)2 HyperLogLog (com.facebook.airlift.stats.cardinality.HyperLogLog)1 CommonParam (com.tencent.angel.ml.matrix.psf.common.CommonParam)1 PartitionGetResult (com.tencent.angel.ml.matrix.psf.get.base.PartitionGetResult)1 RowOffset (com.tencent.angel.model.output.format.ModelPartitionMeta.RowOffset)1 ServerLongAnyRow (com.tencent.angel.ps.storage.vector.ServerLongAnyRow)1 ILongKeyPartOp (com.tencent.angel.psagent.matrix.transport.router.operator.ILongKeyPartOp)1 Int2DoubleOpenHashMap (it.unimi.dsi.fastutil.ints.Int2DoubleOpenHashMap)1 Int2FloatOpenHashMap (it.unimi.dsi.fastutil.ints.Int2FloatOpenHashMap)1 Int2IntOpenHashMap (it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap)1 Long2DoubleMap (it.unimi.dsi.fastutil.longs.Long2DoubleMap)1 PrimitiveIterator (java.util.PrimitiveIterator)1 Configuration (org.apache.hadoop.conf.Configuration)1 Path (org.apache.hadoop.fs.Path)1