Search in sources :

Example 26 with PSAttemptId

use of com.tencent.angel.ps.PSAttemptId in project angel by Tencent.

the class MasterService method psDone.

/**
 * notify a parameter server run over successfully
 * @param controller rpc controller of protobuf
 * @param request parameter server attempt id
 * @throws ServiceException
 */
@SuppressWarnings("unchecked")
@Override
public PSDoneResponse psDone(RpcController controller, PSDoneRequest request) throws ServiceException {
    PSAttemptId psAttemptId = ProtobufUtil.convertToId(request.getPsAttemptId());
    LOG.info("psAttempt " + psAttemptId + " is done");
    // remove this parameter server attempt from monitor set
    psLastHeartbeatTS.remove(psAttemptId);
    context.getEventHandler().handle(new PSAttemptEvent(PSAttemptEventType.PA_SUCCESS, psAttemptId));
    return PSDoneResponse.newBuilder().build();
}
Also used : PSAttemptId(com.tencent.angel.ps.PSAttemptId)

Example 27 with PSAttemptId

use of com.tencent.angel.ps.PSAttemptId in project angel by Tencent.

the class YarnContainerAllocator method containerAssigned.

@SuppressWarnings("unchecked")
private void containerAssigned(Container allocated, ContainerRequest assigned) {
    // Update resource requests
    decContainerReq(assigned);
    // send the container-assigned event to task attempt
    if (assigned.id instanceof PSAttemptId) {
        context.getEventHandler().handle(new PSAttemptContainerAssignedEvent((PSAttemptId) assigned.id, allocated));
    } else if (assigned.id instanceof PSAgentAttemptId) {
        context.getEventHandler().handle(new PSAgentAttemptContainerAssignedEvent((PSAgentAttemptId) assigned.id, allocated));
    } else if (assigned.id instanceof WorkerAttemptId) {
        context.getEventHandler().handle(new WorkerAttemptContainerAssignedEvent((WorkerAttemptId) assigned.id, allocated));
    }
    assignedContainerToIDMap.put(allocated.getId(), assigned.id);
    idToContainerMap.put(assigned.id, allocated);
    LOG.info("Assigned container (" + allocated + ") " + " to task " + assigned.id + " on node " + allocated.getNodeId().toString());
}
Also used : PSAttemptContainerAssignedEvent(com.tencent.angel.master.ps.attempt.PSAttemptContainerAssignedEvent) PSAttemptId(com.tencent.angel.ps.PSAttemptId) WorkerAttemptId(com.tencent.angel.worker.WorkerAttemptId) PSAgentAttemptContainerAssignedEvent(com.tencent.angel.master.psagent.PSAgentAttemptContainerAssignedEvent) PSAgentAttemptId(com.tencent.angel.psagent.PSAgentAttemptId) WorkerAttemptContainerAssignedEvent(com.tencent.angel.master.worker.attempt.WorkerAttemptContainerAssignedEvent)

Aggregations

PSAttemptId (com.tencent.angel.ps.PSAttemptId)27 ParameterServerId (com.tencent.angel.ps.ParameterServerId)15 WorkerAttemptId (com.tencent.angel.worker.WorkerAttemptId)15 WorkerGroupId (com.tencent.angel.worker.WorkerGroupId)11 WorkerId (com.tencent.angel.worker.WorkerId)11 Configuration (org.apache.hadoop.conf.Configuration)11 MatrixContext (com.tencent.angel.ml.matrix.MatrixContext)10 CombineTextInputFormat (org.apache.hadoop.mapreduce.lib.input.CombineTextInputFormat)10 TaskId (com.tencent.angel.worker.task.TaskId)8 Before (org.junit.Before)6 Test (org.junit.Test)6 AngelException (com.tencent.angel.exception.AngelException)5 AMParameterServer (com.tencent.angel.master.ps.ps.AMParameterServer)5 DummyTask (com.tencent.angel.master.DummyTask)4 PSAttempt (com.tencent.angel.master.ps.attempt.PSAttempt)4 ParameterServer (com.tencent.angel.ps.impl.ParameterServer)4 PSAgentAttemptId (com.tencent.angel.psagent.PSAgentAttemptId)4 Id (com.tencent.angel.common.Id)3 Location (com.tencent.angel.common.location.Location)3 ParameterServerManager (com.tencent.angel.master.ps.ParameterServerManager)3