Search in sources :

Example 46 with PartitionKey

use of com.tencent.angel.PartitionKey in project angel by Tencent.

the class BinaryAggrParam method split.

@Override
public List<PartitionGetParam> split() {
    List<PartitionKey> parts = PSAgentContext.get().getMatrixMetaManager().getPartitions(matrixId);
    int size = parts.size();
    List<PartitionGetParam> partParams = new ArrayList<PartitionGetParam>(size);
    for (PartitionKey part : parts) {
        partParams.add(new BinaryPartitionAggrParam(matrixId, part, rowId1, rowId2));
    }
    return partParams;
}
Also used : ArrayList(java.util.ArrayList) PartitionKey(com.tencent.angel.PartitionKey) PartitionGetParam(com.tencent.angel.ml.matrix.psf.get.base.PartitionGetParam)

Example 47 with PartitionKey

use of com.tencent.angel.PartitionKey in project angel by Tencent.

the class FullAggrFunc method partitionGet.

@Override
public PartitionGetResult partitionGet(PartitionGetParam partKey) {
    ServerPartition part = psContext.getMatrixStorageManager().getPart(partKey.getMatrixId(), partKey.getPartKey().getPartitionId());
    double[][] result = null;
    if (part != null) {
        result = process(part, partKey.getPartKey());
    }
    PartitionKey key = partKey.getPartKey();
    int[] partInfo = new int[] { key.getStartRow(), key.getEndRow(), (int) key.getStartCol(), (int) key.getEndCol() };
    return new FullPartitionAggrResult(result, partInfo);
}
Also used : PartitionKey(com.tencent.angel.PartitionKey) ServerPartition(com.tencent.angel.ps.impl.matrix.ServerPartition)

Example 48 with PartitionKey

use of com.tencent.angel.PartitionKey in project angel by Tencent.

the class FullAggrParam method split.

@Override
public List<PartitionGetParam> split() {
    List<PartitionKey> parts = PSAgentContext.get().getMatrixMetaManager().getPartitions(matrixId);
    int size = parts.size();
    List<PartitionGetParam> partParams = new ArrayList<PartitionGetParam>(size);
    for (PartitionKey part : parts) {
        partParams.add(new FullPartitionAggrParam(matrixId, part));
    }
    return partParams;
}
Also used : ArrayList(java.util.ArrayList) PartitionKey(com.tencent.angel.PartitionKey) PartitionGetParam(com.tencent.angel.ml.matrix.psf.get.base.PartitionGetParam)

Example 49 with PartitionKey

use of com.tencent.angel.PartitionKey in project angel by Tencent.

the class MultiAggrParam method split.

@Override
public List<PartitionGetParam> split() {
    List<PartitionKey> parts = PSAgentContext.get().getMatrixMetaManager().getPartitions(matrixId);
    int size = parts.size();
    List<PartitionGetParam> partParams = new ArrayList<PartitionGetParam>(size);
    for (PartitionKey part : parts) {
        partParams.add(new MultiPartitionAggrParam(matrixId, part, rowIds));
    }
    return partParams;
}
Also used : ArrayList(java.util.ArrayList) PartitionKey(com.tencent.angel.PartitionKey) PartitionGetParam(com.tencent.angel.ml.matrix.psf.get.base.PartitionGetParam)

Example 50 with PartitionKey

use of com.tencent.angel.PartitionKey in project angel by Tencent.

the class CompressUpdateParam method split.

@Override
public List<PartitionUpdateParam> split() {
    List<PartitionKey> partList = PSAgentContext.get().getMatrixMetaManager().getPartitions(matrixId, rowId);
    int size = partList.size();
    List<PartitionUpdateParam> partParams = new ArrayList<PartitionUpdateParam>(size);
    for (PartitionKey part : partList) {
        if (rowId < part.getStartRow() || rowId >= part.getEndRow()) {
            throw new RuntimeException("Wrong rowId!");
        }
        partParams.add(new CompressPartitionUpdateParam(matrixId, part, rowId, (int) part.getStartCol(), (int) part.getEndCol(), array, bitsPerItem));
    }
    return partParams;
}
Also used : ArrayList(java.util.ArrayList) PartitionKey(com.tencent.angel.PartitionKey)

Aggregations

PartitionKey (com.tencent.angel.PartitionKey)80 ArrayList (java.util.ArrayList)17 ByteBuf (io.netty.buffer.ByteBuf)12 Test (org.junit.Test)9 PartitionGetResult (com.tencent.angel.ml.matrix.psf.get.base.PartitionGetResult)8 MatrixMeta (com.tencent.angel.ml.matrix.MatrixMeta)7 PartitionGetParam (com.tencent.angel.ml.matrix.psf.get.base.PartitionGetParam)7 PartitionLocation (com.tencent.angel.ml.matrix.PartitionLocation)4 ServerRow (com.tencent.angel.ps.impl.matrix.ServerRow)4 ParameterServerId (com.tencent.angel.ps.ParameterServerId)3 RecoverPartKey (com.tencent.angel.ps.recovery.ha.RecoverPartKey)3 FutureResult (com.tencent.angel.psagent.matrix.transport.FutureResult)3 Map (java.util.Map)3 Location (com.tencent.angel.common.location.Location)2 TVector (com.tencent.angel.ml.math.TVector)2 RowType (com.tencent.angel.ml.matrix.RowType)2 PSLocation (com.tencent.angel.ml.matrix.transport.PSLocation)2 MatrixStorageManager (com.tencent.angel.ps.impl.MatrixStorageManager)2 ClockCache (com.tencent.angel.psagent.clock.ClockCache)2 Worker (com.tencent.angel.worker.Worker)2