Search in sources :

Example 21 with IElement

use of com.tencent.angel.ps.storage.vector.element.IElement in project angel by Tencent.

the class ComplexRowFormat method save.

private void save(ServerAnyAnyRow row, PSMatrixSaveContext saveContext, MatrixPartitionMeta partMeta, DataOutputStream output) throws IOException {
    ElementElementStorage storage = row.getStorage();
    ObjectIterator<Object2ObjectMap.Entry<IElement, IElement>> iter = storage.iterator();
    while (iter.hasNext()) {
        Object2ObjectMap.Entry<IElement, IElement> entry = iter.next();
        save(entry.getKey(), entry.getValue(), output);
    }
}
Also used : ElementElementStorage(com.tencent.angel.ps.storage.vector.storage.ElementElementStorage) Entry(java.util.Map.Entry) IElement(com.tencent.angel.ps.storage.vector.element.IElement) Object2ObjectMap(it.unimi.dsi.fastutil.objects.Object2ObjectMap)

Example 22 with IElement

use of com.tencent.angel.ps.storage.vector.element.IElement in project angel by Tencent.

the class PullMaxDegree method partitionGet.

@Override
public PartitionGetResult partitionGet(PartitionGetParam partParam) {
    ServerLongAnyRow row = (ServerLongAnyRow) psContext.getMatrixStorageManager().getRow(partParam.getPartKey(), 0);
    int partResult = Integer.MIN_VALUE;
    ObjectIterator<Long2ObjectMap.Entry<IElement>> iter = row.iterator();
    while (iter.hasNext()) {
        Long2ObjectMap.Entry<IElement> entry = iter.next();
        LongArrayElement value = (LongArrayElement) entry.getValue();
        int length = value.getData().length;
        if (length > partResult) {
            partResult = length;
        }
    }
    return new PullMaxDegreePartitionResult(partResult);
}
Also used : IElement(com.tencent.angel.ps.storage.vector.element.IElement) Long2ObjectMap(it.unimi.dsi.fastutil.longs.Long2ObjectMap) LongArrayElement(com.tencent.angel.ps.storage.vector.element.LongArrayElement) ServerLongAnyRow(com.tencent.angel.ps.storage.vector.ServerLongAnyRow)

Example 23 with IElement

use of com.tencent.angel.ps.storage.vector.element.IElement in project angel by Tencent.

the class ElementElementMapStorage method deserialize.

@Override
public void deserialize(DataInputStream input) throws IOException {
    super.deserialize(input);
    // Valid element number
    int elementNum = input.readInt();
    data = new Object2ObjectOpenHashMap<>(elementNum);
    // Deserialize the data
    for (int i = 0; i < elementNum; i++) {
        IElement elementKey = newElement();
        IElement elementValue = newElement();
        data.put(elementKey, elementValue);
        elementKey.deserialize(input);
        elementValue.deserialize(input);
    }
}
Also used : IElement(com.tencent.angel.ps.storage.vector.element.IElement)

Example 24 with IElement

use of com.tencent.angel.ps.storage.vector.element.IElement in project angel by Tencent.

the class GeneralGetUtils method partitionGet.

public static PartitionGetResult partitionGet(PSContext psContext, PartitionGetParam partParam) {
    GeneralPartGetParam param = (GeneralPartGetParam) partParam;
    KeyPart keyPart = param.getIndicesPart();
    // Long type node id
    long[] nodeIds = ((ILongKeyPartOp) keyPart).getKeys();
    ServerLongAnyRow row = GraphMatrixUtils.getPSLongKeyRow(psContext, param);
    // Get data
    IElement[] data = new IElement[nodeIds.length];
    for (int i = 0; i < nodeIds.length; i++) {
        data[i] = row.get(nodeIds[i]);
    }
    MatrixMeta meta = psContext.getMatrixMetaManager().getMatrixMeta(param.getMatrixId());
    try {
        return new PartGeneralGetResult(meta.getValueClass(), nodeIds, data);
    } catch (ClassNotFoundException e) {
        throw new AngelException("Can not get value class ");
    }
}
Also used : AngelException(com.tencent.angel.exception.AngelException) GeneralPartGetParam(com.tencent.angel.ml.matrix.psf.get.base.GeneralPartGetParam) IElement(com.tencent.angel.ps.storage.vector.element.IElement) PartGeneralGetResult(com.tencent.angel.graph.model.general.get.PartGeneralGetResult) MatrixMeta(com.tencent.angel.ml.matrix.MatrixMeta) KeyPart(com.tencent.angel.psagent.matrix.transport.router.KeyPart) ServerLongAnyRow(com.tencent.angel.ps.storage.vector.ServerLongAnyRow) ILongKeyPartOp(com.tencent.angel.psagent.matrix.transport.router.operator.ILongKeyPartOp)

Example 25 with IElement

use of com.tencent.angel.ps.storage.vector.element.IElement in project angel by Tencent.

the class IntArrayElementStorage method deserialize.

@Override
public void deserialize(DataInputStream input) throws IOException {
    super.deserialize(input);
    // Array len
    int len = input.readInt();
    data = new IElement[len];
    // Valid element number
    int elementNum = input.readInt();
    for (int i = 0; i < elementNum; i++) {
        IElement element = newElement();
        data[input.readInt()] = element;
        element.deserialize(input);
    }
}
Also used : IElement(com.tencent.angel.ps.storage.vector.element.IElement)

Aggregations

IElement (com.tencent.angel.ps.storage.vector.element.IElement)38 ServerLongAnyRow (com.tencent.angel.ps.storage.vector.ServerLongAnyRow)19 ILongKeyAnyValuePartOp (com.tencent.angel.psagent.matrix.transport.router.operator.ILongKeyAnyValuePartOp)13 GeneralPartUpdateParam (com.tencent.angel.ml.matrix.psf.update.base.GeneralPartUpdateParam)12 GraphNode (com.tencent.angel.graph.data.GraphNode)7 PartGeneralGetResult (com.tencent.angel.graph.model.general.get.PartGeneralGetResult)6 PartitionGetResult (com.tencent.angel.ml.matrix.psf.get.base.PartitionGetResult)6 Long2ObjectMap (it.unimi.dsi.fastutil.longs.Long2ObjectMap)6 Long2ObjectOpenHashMap (it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap)6 LongArrayElement (com.tencent.angel.ps.storage.vector.element.LongArrayElement)4 Entry (java.util.Map.Entry)4 GetLongsResult (com.tencent.angel.graph.common.psf.result.GetLongsResult)3 DynamicNeighborElement (com.tencent.angel.graph.model.neighbor.dynamic.DynamicNeighborElement)3 Object2ObjectMap (it.unimi.dsi.fastutil.objects.Object2ObjectMap)2 HyperLogLogPlus (com.clearspring.analytics.stream.cardinality.HyperLogLogPlus)1 PartitionKey (com.tencent.angel.PartitionKey)1 AngelException (com.tencent.angel.exception.AngelException)1 WalkPath (com.tencent.angel.graph.client.node2vec.data.WalkPath)1 GetElementResult (com.tencent.angel.graph.common.psf.result.GetElementResult)1 GetFloatsResult (com.tencent.angel.graph.common.psf.result.GetFloatsResult)1