Search in sources :

Example 6 with VoidResult

use of com.tencent.angel.ml.matrix.psf.update.base.VoidResult in project angel by Tencent.

the class UserRequestAdapter method update.

/**
 * Update matrix use a udf.
 *
 * @param updateFunc update udf function
 * @return Future<VoidResult> update future result
 */
public Future<VoidResult> update(UpdateFunc updateFunc) {
    MatrixTransportClient matrixClient = PSAgentContext.get().getMatrixTransportClient();
    UpdateParam param = updateFunc.getParam();
    // Split the param use matrix partitions
    List<PartitionUpdateParam> partParams = param.split();
    int size = partParams.size();
    UpdatePSFRequest request = new UpdatePSFRequest(updateFunc);
    ResponseCache cache = new MapResponseCache(size);
    FutureResult<VoidResult> result = new FutureResult<>();
    int requestId = request.getRequestId();
    requests.put(requestId, request);
    requestIdToResponseCache.put(requestId, cache);
    requestIdToResultMap.put(requestId, result);
    // Send request to PSS
    for (int i = 0; i < size; i++) {
        sendUpdateUDFRequest(matrixClient, requestId, partParams.get(i).getMatrixId(), partParams.get(i).getPartKey().getPartitionId(), updateFunc, partParams.get(i));
    }
    return result;
}
Also used : MatrixTransportClient(com.tencent.angel.psagent.matrix.transport.MatrixTransportClient) VoidResult(com.tencent.angel.ml.matrix.psf.update.base.VoidResult) FutureResult(com.tencent.angel.psagent.matrix.transport.FutureResult) PartitionUpdateParam(com.tencent.angel.ml.matrix.psf.update.base.PartitionUpdateParam) PartitionUpdateParam(com.tencent.angel.ml.matrix.psf.update.base.PartitionUpdateParam) UpdateParam(com.tencent.angel.ml.matrix.psf.update.base.UpdateParam) MapResponseCache(com.tencent.angel.psagent.matrix.transport.response.MapResponseCache) ResponseCache(com.tencent.angel.psagent.matrix.transport.response.ResponseCache) MapResponseCache(com.tencent.angel.psagent.matrix.transport.response.MapResponseCache)

Example 7 with VoidResult

use of com.tencent.angel.ml.matrix.psf.update.base.VoidResult in project angel by Tencent.

the class UpdateHandler method handle.

@Override
public void handle(FutureResult finalResult, UserRequest userRequest, ResponseCache responseCache) {
    MapResponseCache cache = (MapResponseCache) responseCache;
    // Check update result
    boolean success = true;
    String detail = "";
    for (Response response : cache.getResponses().values()) {
        success = success && (response.getResponseType() == ResponseType.SUCCESS);
        if (!success) {
            detail = response.getDetail();
            break;
        }
    }
    // Set the final result
    if (success) {
        finalResult.set(new VoidResult(com.tencent.angel.psagent.matrix.ResponseType.SUCCESS));
    } else {
        finalResult.set(new VoidResult(com.tencent.angel.psagent.matrix.ResponseType.FAILED, detail));
    }
}
Also used : Response(com.tencent.angel.ps.server.data.response.Response) UpdateResponse(com.tencent.angel.ps.server.data.response.UpdateResponse) VoidResult(com.tencent.angel.ml.matrix.psf.update.base.VoidResult)

Example 8 with VoidResult

use of com.tencent.angel.ml.matrix.psf.update.base.VoidResult in project angel by Tencent.

the class MatrixTransportClient method update.

@Override
public FutureResult<VoidResult> update(UpdateFunc func, PartitionUpdateParam param) {
    // Request header
    RequestHeader header = createRequestHeader(-1, TransportMethod.UPDATE_PSF, param.getMatrixId(), param.getPartKey().getPartitionId());
    // Request body
    UpdateUDFRequest requestData = new UpdateUDFRequest(func.getClass().getName(), param);
    // Request
    Request request = new Request(header, requestData);
    FutureResult<VoidResult> resultFuture = new FutureResult<>();
    requestToResultMap.put(request, resultFuture);
    // Send the request
    sendUpdateRequest(request);
    return resultFuture;
}
Also used : VoidResult(com.tencent.angel.ml.matrix.psf.update.base.VoidResult) UserRequest(com.tencent.angel.psagent.matrix.transport.adapter.UserRequest) Request(com.tencent.angel.ps.server.data.request.Request) UpdateUDFRequest(com.tencent.angel.ps.server.data.request.UpdateUDFRequest) GetUDFRequest(com.tencent.angel.ps.server.data.request.GetUDFRequest) RequestHeader(com.tencent.angel.ps.server.data.request.RequestHeader) UpdateUDFRequest(com.tencent.angel.ps.server.data.request.UpdateUDFRequest)

Example 9 with VoidResult

use of com.tencent.angel.ml.matrix.psf.update.base.VoidResult in project angel by Tencent.

the class UserRequestAdapter method update.

public Future<VoidResult> update(int matrixId, int[] rowIds, Vector[] rows, UpdateOp op) {
    assert rowIds.length == rows.length;
    checkParams(matrixId, rowIds);
    for (int i = 0; i < rowIds.length; i++) {
        rows[i].setRowId(rowIds[i]);
    }
    MatrixMeta matrixMeta = PSAgentContext.get().getMatrixMetaManager().getMatrixMeta(matrixId);
    PartitionKey[] parts = matrixMeta.getPartitionKeys();
    CompStreamKeyValuePart[] splits = RouterUtils.splitStream(matrixMeta, rows);
    int needRequestPartNum = noEmptyPartNum(splits);
    FutureResult<VoidResult> result = new FutureResult<>();
    if (needRequestPartNum == 0) {
        result.set(null);
        return result;
    }
    UpdateRowsRequest request = new UpdateRowsRequest(matrixId, op);
    ResponseCache cache = new MapResponseCache(needRequestPartNum);
    int requestId = request.getRequestId();
    requestIdToResponseCache.put(requestId, cache);
    requestIdToResultMap.put(requestId, result);
    requests.put(requestId, request);
    MatrixTransportClient matrixClient = PSAgentContext.get().getMatrixTransportClient();
    for (int i = 0; i < splits.length; i++) {
        if (splits[i] != null && splits[i].size() > 0) {
            sendUpdateRequest(matrixClient, requestId, matrixId, parts[i].getPartitionId(), splits[i], op);
        }
    }
    return result;
}
Also used : VoidResult(com.tencent.angel.ml.matrix.psf.update.base.VoidResult) MatrixMeta(com.tencent.angel.ml.matrix.MatrixMeta) MapResponseCache(com.tencent.angel.psagent.matrix.transport.response.MapResponseCache) MatrixTransportClient(com.tencent.angel.psagent.matrix.transport.MatrixTransportClient) FutureResult(com.tencent.angel.psagent.matrix.transport.FutureResult) PartitionKey(com.tencent.angel.PartitionKey) MapResponseCache(com.tencent.angel.psagent.matrix.transport.response.MapResponseCache) ResponseCache(com.tencent.angel.psagent.matrix.transport.response.ResponseCache) CompStreamKeyValuePart(com.tencent.angel.psagent.matrix.transport.router.CompStreamKeyValuePart)

Example 10 with VoidResult

use of com.tencent.angel.ml.matrix.psf.update.base.VoidResult in project angel by Tencent.

the class UserRequestAdapter method update.

public Future<VoidResult> update(int matrixId, Matrix delta, UpdateOp op) {
    checkParams(matrixId);
    delta.setMatrixId(matrixId);
    MatrixMeta matrixMeta = PSAgentContext.get().getMatrixMetaManager().getMatrixMeta(matrixId);
    PartitionKey[] parts = matrixMeta.getPartitionKeys();
    CompStreamKeyValuePart[] splits = RouterUtils.splitStream(matrixMeta, delta);
    int needRequestPartNum = noEmptyPartNum(splits);
    FutureResult<VoidResult> result = new FutureResult<>();
    if (needRequestPartNum == 0) {
        result.set(null);
        return result;
    }
    UpdateMatrixRequest request = new UpdateMatrixRequest(matrixId, op);
    ResponseCache cache = new MapResponseCache(needRequestPartNum);
    int requestId = request.getRequestId();
    requestIdToResponseCache.put(requestId, cache);
    requestIdToResultMap.put(requestId, result);
    requests.put(requestId, request);
    MatrixTransportClient matrixClient = PSAgentContext.get().getMatrixTransportClient();
    for (int i = 0; i < splits.length; i++) {
        if (splits[i] != null && splits[i].size() > 0) {
            sendUpdateRequest(matrixClient, requestId, matrixId, parts[i].getPartitionId(), splits[i], op);
        }
    }
    return result;
}
Also used : VoidResult(com.tencent.angel.ml.matrix.psf.update.base.VoidResult) MatrixMeta(com.tencent.angel.ml.matrix.MatrixMeta) MapResponseCache(com.tencent.angel.psagent.matrix.transport.response.MapResponseCache) MatrixTransportClient(com.tencent.angel.psagent.matrix.transport.MatrixTransportClient) FutureResult(com.tencent.angel.psagent.matrix.transport.FutureResult) PartitionKey(com.tencent.angel.PartitionKey) MapResponseCache(com.tencent.angel.psagent.matrix.transport.response.MapResponseCache) ResponseCache(com.tencent.angel.psagent.matrix.transport.response.ResponseCache) CompStreamKeyValuePart(com.tencent.angel.psagent.matrix.transport.router.CompStreamKeyValuePart)

Aggregations

VoidResult (com.tencent.angel.ml.matrix.psf.update.base.VoidResult)14 FutureResult (com.tencent.angel.psagent.matrix.transport.FutureResult)8 MatrixTransportClient (com.tencent.angel.psagent.matrix.transport.MatrixTransportClient)4 MapResponseCache (com.tencent.angel.psagent.matrix.transport.response.MapResponseCache)4 ResponseCache (com.tencent.angel.psagent.matrix.transport.response.ResponseCache)4 PartitionKey (com.tencent.angel.PartitionKey)3 CSRPartUpdateParam (com.tencent.angel.ml.lda.psf.CSRPartUpdateParam)3 UpdatePartFunc (com.tencent.angel.ml.lda.psf.UpdatePartFunc)3 MatrixMeta (com.tencent.angel.ml.matrix.MatrixMeta)3 CompStreamKeyValuePart (com.tencent.angel.psagent.matrix.transport.router.CompStreamKeyValuePart)3 Int2IntOpenHashMap (it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap)3 AngelException (com.tencent.angel.exception.AngelException)2 Request (com.tencent.angel.ps.server.data.request.Request)2 Response (com.tencent.angel.ps.server.data.response.Response)2 UserRequest (com.tencent.angel.psagent.matrix.transport.adapter.UserRequest)2 Random (java.util.Random)2 PartitionUpdateParam (com.tencent.angel.ml.matrix.psf.update.base.PartitionUpdateParam)1 UpdateParam (com.tencent.angel.ml.matrix.psf.update.base.UpdateParam)1 GetUDFRequest (com.tencent.angel.ps.server.data.request.GetUDFRequest)1 RequestHeader (com.tencent.angel.ps.server.data.request.RequestHeader)1