Search in sources :

Example 6 with LongArrayElement

use of com.tencent.angel.ps.storage.vector.element.LongArrayElement in project angel by Tencent.

the class SampleNeighbor method partitionGet.

@Override
public PartitionGetResult partitionGet(PartitionGetParam partParam) {
    PartSampleNeighborParam param = (PartSampleNeighborParam) partParam;
    ServerMatrix matrix = psContext.getMatrixStorageManager().getMatrix(partParam.getMatrixId());
    ServerPartition part = matrix.getPartition(partParam.getPartKey().getPartitionId());
    ServerLongAnyRow row = (ServerLongAnyRow) (((RowBasedPartition) part).getRow(0));
    long[] nodeIds = param.getNodeIds();
    long[][] neighbors = new long[nodeIds.length][];
    int count = param.getCount();
    Random r = new Random();
    for (int i = 0; i < nodeIds.length; i++) {
        long nodeId = nodeIds[i];
        // Get node neighbor number
        LongArrayElement element = (LongArrayElement) (row.get(nodeId));
        if (element == null) {
            neighbors[i] = null;
        } else {
            long[] nodeNeighbors = element.getData();
            if (nodeNeighbors == null || nodeNeighbors.length == 0) {
                neighbors[i] = null;
            } else if (count <= 0 || nodeNeighbors.length <= count) {
                neighbors[i] = nodeNeighbors;
            } else {
                neighbors[i] = new long[count];
                // If the neighbor number > count, just copy a range of neighbors to the result array, the copy position is random
                int startPos = Math.abs(r.nextInt()) % nodeNeighbors.length;
                if (startPos + count <= nodeNeighbors.length) {
                    System.arraycopy(nodeNeighbors, startPos, neighbors[i], 0, count);
                } else {
                    System.arraycopy(nodeNeighbors, startPos, neighbors[i], 0, nodeNeighbors.length - startPos);
                    System.arraycopy(nodeNeighbors, 0, neighbors[i], nodeNeighbors.length - startPos, count - (nodeNeighbors.length - startPos));
                }
            }
        }
    }
    return new PartSampleNeighborResult(part.getPartitionKey().getPartitionId(), neighbors);
}
Also used : Random(java.util.Random) ServerMatrix(com.tencent.angel.ps.storage.matrix.ServerMatrix) LongArrayElement(com.tencent.angel.ps.storage.vector.element.LongArrayElement) RowBasedPartition(com.tencent.angel.ps.storage.partition.RowBasedPartition) ServerLongAnyRow(com.tencent.angel.ps.storage.vector.ServerLongAnyRow) ServerPartition(com.tencent.angel.ps.storage.partition.ServerPartition)

Example 7 with LongArrayElement

use of com.tencent.angel.ps.storage.vector.element.LongArrayElement in project angel by Tencent.

the class PullMaxDegree method partitionGet.

@Override
public PartitionGetResult partitionGet(PartitionGetParam partParam) {
    ServerLongAnyRow row = (ServerLongAnyRow) psContext.getMatrixStorageManager().getRow(partParam.getPartKey(), 0);
    int partResult = Integer.MIN_VALUE;
    ObjectIterator<Long2ObjectMap.Entry<IElement>> iter = row.iterator();
    while (iter.hasNext()) {
        Long2ObjectMap.Entry<IElement> entry = iter.next();
        LongArrayElement value = (LongArrayElement) entry.getValue();
        int length = value.getData().length;
        if (length > partResult) {
            partResult = length;
        }
    }
    return new PullMaxDegreePartitionResult(partResult);
}
Also used : IElement(com.tencent.angel.ps.storage.vector.element.IElement) Long2ObjectMap(it.unimi.dsi.fastutil.longs.Long2ObjectMap) LongArrayElement(com.tencent.angel.ps.storage.vector.element.LongArrayElement) ServerLongAnyRow(com.tencent.angel.ps.storage.vector.ServerLongAnyRow)

Example 8 with LongArrayElement

use of com.tencent.angel.ps.storage.vector.element.LongArrayElement in project angel by Tencent.

the class PushNeighbor method partitionUpdate.

@Override
public void partitionUpdate(PartitionUpdateParam partParam) {
    PushNeighborPartitionParam pparam = (PushNeighborPartitionParam) partParam;
    ServerLongAnyRow row = (ServerLongAnyRow) psContext.getMatrixStorageManager().getRow(pparam.getPartKey(), 0);
    Long2ObjectOpenHashMap<long[]> nodeIdToNeighborIndices = pparam.getNodeIdToNeighborIndices();
    ObjectIterator<Long2ObjectMap.Entry<long[]>> iter = nodeIdToNeighborIndices.long2ObjectEntrySet().iterator();
    row.startWrite();
    try {
        while (iter.hasNext()) {
            Long2ObjectMap.Entry<long[]> entry = iter.next();
            row.set(entry.getLongKey(), new LongArrayElement(entry.getValue()));
        }
    } finally {
        row.endWrite();
    }
}
Also used : Long2ObjectMap(it.unimi.dsi.fastutil.longs.Long2ObjectMap) LongArrayElement(com.tencent.angel.ps.storage.vector.element.LongArrayElement) ServerLongAnyRow(com.tencent.angel.ps.storage.vector.ServerLongAnyRow)

Aggregations

ServerLongAnyRow (com.tencent.angel.ps.storage.vector.ServerLongAnyRow)8 LongArrayElement (com.tencent.angel.ps.storage.vector.element.LongArrayElement)8 Long2ObjectMap (it.unimi.dsi.fastutil.longs.Long2ObjectMap)5 IElement (com.tencent.angel.ps.storage.vector.element.IElement)4 ServerMatrix (com.tencent.angel.ps.storage.matrix.ServerMatrix)2 RowBasedPartition (com.tencent.angel.ps.storage.partition.RowBasedPartition)2 Random (java.util.Random)2 PartitionKey (com.tencent.angel.PartitionKey)1 WalkPath (com.tencent.angel.graph.client.node2vec.data.WalkPath)1 PartitionGetParamWithIds (com.tencent.angel.graph.client.node2vec.params.PartitionGetParamWithIds)1 DynamicNeighborElement (com.tencent.angel.graph.model.neighbor.dynamic.DynamicNeighborElement)1 GeneralPartUpdateParam (com.tencent.angel.ml.matrix.psf.update.base.GeneralPartUpdateParam)1 ServerPartition (com.tencent.angel.ps.storage.partition.ServerPartition)1 ILongKeyAnyValuePartOp (com.tencent.angel.psagent.matrix.transport.router.operator.ILongKeyAnyValuePartOp)1 Long2ObjectOpenHashMap (it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap)1 ArrayList (java.util.ArrayList)1