Search in sources :

Example 26 with IElement

use of com.tencent.angel.ps.storage.vector.element.IElement in project angel by Tencent.

the class IntArrayElementStorage method deserialize.

@Override
public void deserialize(ByteBuf buf) {
    super.deserialize(buf);
    // Array len
    int len = buf.readInt();
    data = new IElement[len];
    // Valid element number
    int elementNum = buf.readInt();
    for (int i = 0; i < elementNum; i++) {
        IElement element = newElement();
        data[buf.readInt()] = element;
        element.deserialize(buf);
    }
}
Also used : IElement(com.tencent.angel.ps.storage.vector.element.IElement)

Example 27 with IElement

use of com.tencent.angel.ps.storage.vector.element.IElement in project angel by Tencent.

the class GeneralInit method partitionUpdate.

@Override
public void partitionUpdate(PartitionUpdateParam partParam) {
    GeneralPartUpdateParam initParam = (GeneralPartUpdateParam) partParam;
    ServerLongAnyRow row = GraphMatrixUtils.getPSLongKeyRow(psContext, initParam);
    // Get nodes and features
    ILongKeyAnyValuePartOp split = (ILongKeyAnyValuePartOp) initParam.getKeyValuePart();
    long[] nodeIds = split.getKeys();
    IElement[] neighbors = split.getValues();
    row.startWrite();
    try {
        for (int i = 0; i < nodeIds.length; i++) {
            row.set(nodeIds[i], neighbors[i]);
        }
    } finally {
        row.endWrite();
    }
}
Also used : IElement(com.tencent.angel.ps.storage.vector.element.IElement) GeneralPartUpdateParam(com.tencent.angel.ml.matrix.psf.update.base.GeneralPartUpdateParam) ServerLongAnyRow(com.tencent.angel.ps.storage.vector.ServerLongAnyRow) ILongKeyAnyValuePartOp(com.tencent.angel.psagent.matrix.transport.router.operator.ILongKeyAnyValuePartOp)

Example 28 with IElement

use of com.tencent.angel.ps.storage.vector.element.IElement in project angel by Tencent.

the class GetNeighbor method merge.

@Override
public GetResult merge(List<PartitionGetResult> partResults) {
    int resultSize = 0;
    for (PartitionGetResult result : partResults) {
        resultSize += ((PartGeneralGetResult) result).getNodeIds().length;
    }
    Long2ObjectOpenHashMap<long[]> nodeIdToNeighbors = new Long2ObjectOpenHashMap<>(resultSize);
    for (PartitionGetResult result : partResults) {
        PartGeneralGetResult partResult = (PartGeneralGetResult) result;
        long[] nodeIds = partResult.getNodeIds();
        IElement[] neighbors = partResult.getData();
        for (int i = 0; i < nodeIds.length; i++) {
            if (neighbors[i] != null) {
                byte[] nbrs = ((DynamicNeighborElement) neighbors[i]).getData();
                nodeIdToNeighbors.put(nodeIds[i], ScalaKryoInstantiator.defaultPool().fromBytes(nbrs, long[].class));
            } else {
                nodeIdToNeighbors.put(nodeIds[i], Constents.emptyLongs);
            }
        }
    }
    return new GetLongsResult(nodeIdToNeighbors);
}
Also used : IElement(com.tencent.angel.ps.storage.vector.element.IElement) PartGeneralGetResult(com.tencent.angel.graph.model.general.get.PartGeneralGetResult) DynamicNeighborElement(com.tencent.angel.graph.model.neighbor.dynamic.DynamicNeighborElement) Long2ObjectOpenHashMap(it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap) PartitionGetResult(com.tencent.angel.ml.matrix.psf.get.base.PartitionGetResult) GetLongsResult(com.tencent.angel.graph.common.psf.result.GetLongsResult)

Example 29 with IElement

use of com.tencent.angel.ps.storage.vector.element.IElement in project angel by Tencent.

the class GetLongNeighbor method merge.

@Override
public GetResult merge(List<PartitionGetResult> partResults) {
    int resultSize = 0;
    for (PartitionGetResult result : partResults) {
        resultSize += ((PartGeneralGetResult) result).getNodeIds().length;
    }
    Long2ObjectOpenHashMap<long[]> nodeIdToNeighbors = new Long2ObjectOpenHashMap<>(resultSize);
    for (PartitionGetResult result : partResults) {
        PartGeneralGetResult partResult = (PartGeneralGetResult) result;
        long[] nodeIds = partResult.getNodeIds();
        IElement[] neighbors = partResult.getData();
        for (int i = 0; i < nodeIds.length; i++) {
            if (neighbors[i] != null) {
                nodeIdToNeighbors.put(nodeIds[i], ((LongArrayElement) neighbors[i]).getData());
            } else {
                nodeIdToNeighbors.put(nodeIds[i], Constents.emptyLongs);
            }
        }
    }
    return new GetLongsResult(nodeIdToNeighbors);
}
Also used : IElement(com.tencent.angel.ps.storage.vector.element.IElement) PartGeneralGetResult(com.tencent.angel.graph.model.general.get.PartGeneralGetResult) Long2ObjectOpenHashMap(it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap) PartitionGetResult(com.tencent.angel.ml.matrix.psf.get.base.PartitionGetResult) GetLongsResult(com.tencent.angel.graph.common.psf.result.GetLongsResult)

Example 30 with IElement

use of com.tencent.angel.ps.storage.vector.element.IElement in project angel by Tencent.

the class InitEdgeTypes method partitionUpdate.

@Override
public void partitionUpdate(PartitionUpdateParam partParam) {
    GeneralPartUpdateParam param = (GeneralPartUpdateParam) partParam;
    ServerLongAnyRow row = GraphMatrixUtils.getPSLongKeyRow(psContext, param);
    ILongKeyAnyValuePartOp keyValuePart = (ILongKeyAnyValuePartOp) param.getKeyValuePart();
    long[] nodeIds = keyValuePart.getKeys();
    IElement[] neighbors = keyValuePart.getValues();
    row.startWrite();
    try {
        for (int i = 0; i < nodeIds.length; i++) {
            GraphNode graphNode = (GraphNode) row.get(nodeIds[i]);
            if (graphNode == null) {
                graphNode = new GraphNode();
                row.set(nodeIds[i], graphNode);
            }
            graphNode.setEdgeTypes(((EdgeType) neighbors[i]).getTypes());
        }
    } finally {
        row.endWrite();
    }
}
Also used : IElement(com.tencent.angel.ps.storage.vector.element.IElement) GeneralPartUpdateParam(com.tencent.angel.ml.matrix.psf.update.base.GeneralPartUpdateParam) GraphNode(com.tencent.angel.graph.data.GraphNode) ServerLongAnyRow(com.tencent.angel.ps.storage.vector.ServerLongAnyRow) ILongKeyAnyValuePartOp(com.tencent.angel.psagent.matrix.transport.router.operator.ILongKeyAnyValuePartOp)

Aggregations

IElement (com.tencent.angel.ps.storage.vector.element.IElement)38 ServerLongAnyRow (com.tencent.angel.ps.storage.vector.ServerLongAnyRow)19 ILongKeyAnyValuePartOp (com.tencent.angel.psagent.matrix.transport.router.operator.ILongKeyAnyValuePartOp)13 GeneralPartUpdateParam (com.tencent.angel.ml.matrix.psf.update.base.GeneralPartUpdateParam)12 GraphNode (com.tencent.angel.graph.data.GraphNode)7 PartGeneralGetResult (com.tencent.angel.graph.model.general.get.PartGeneralGetResult)6 PartitionGetResult (com.tencent.angel.ml.matrix.psf.get.base.PartitionGetResult)6 Long2ObjectMap (it.unimi.dsi.fastutil.longs.Long2ObjectMap)6 Long2ObjectOpenHashMap (it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap)6 LongArrayElement (com.tencent.angel.ps.storage.vector.element.LongArrayElement)4 Entry (java.util.Map.Entry)4 GetLongsResult (com.tencent.angel.graph.common.psf.result.GetLongsResult)3 DynamicNeighborElement (com.tencent.angel.graph.model.neighbor.dynamic.DynamicNeighborElement)3 Object2ObjectMap (it.unimi.dsi.fastutil.objects.Object2ObjectMap)2 HyperLogLogPlus (com.clearspring.analytics.stream.cardinality.HyperLogLogPlus)1 PartitionKey (com.tencent.angel.PartitionKey)1 AngelException (com.tencent.angel.exception.AngelException)1 WalkPath (com.tencent.angel.graph.client.node2vec.data.WalkPath)1 GetElementResult (com.tencent.angel.graph.common.psf.result.GetElementResult)1 GetFloatsResult (com.tencent.angel.graph.common.psf.result.GetFloatsResult)1