Search in sources :

Example 6 with InvalidParameterException

use of com.tencent.angel.exception.InvalidParameterException in project angel by Tencent.

the class ConfUtils method parseArgs.

private static Map<String, String> parseArgs(String[] args) throws InvalidParameterException {
    Map<String, String> kvMap = new HashMap<String, String>();
    int seg = 0;
    for (int i = 0; i < args.length; i++) {
        if (args[i].startsWith("-D")) {
            seg = args[i].indexOf("=");
            if (seg > 0) {
                kvMap.put(args[i].substring(2, seg), args[i].substring(seg + 1));
            } else {
                throw new InvalidParameterException("unvalid parameter " + args[i]);
            }
        } else if (args[i].startsWith("--")) {
            String key = args[i].substring(2);
            i++;
            if (i < args.length) {
                String value = args[i];
                kvMap.put(key, value);
            } else {
                throw new InvalidParameterException("there is no value for parameter " + key);
            }
        } else if ((seg = args[i].indexOf(":")) > 0) {
            kvMap.put(args[i].substring(0, seg), args[i].substring(seg + 1));
        } else {
            switch(args[i]) {
                case "jar":
                    {
                        if (i == args.length - 1) {
                            throw new InvalidParameterException("there must be a jar file after jar commond");
                        } else {
                            i++;
                            kvMap.put(AngelConf.ANGEL_JOB_JAR, args[i]);
                        }
                        break;
                    }
                default:
                    {
                        throw new InvalidParameterException("unvalid parameter " + args[i]);
                    }
            }
        }
    }
    return kvMap;
}
Also used : InvalidParameterException(com.tencent.angel.exception.InvalidParameterException)

Example 7 with InvalidParameterException

use of com.tencent.angel.exception.InvalidParameterException in project angel by Tencent.

the class GetNodeTypes method partitionGet.

@Override
public PartitionGetResult partitionGet(PartitionGetParam partParam) {
    GeneralPartGetParam param = (GeneralPartGetParam) partParam;
    KeyPart keyPart = param.getIndicesPart();
    switch(keyPart.getKeyType()) {
        case LONG:
            {
                // Long type node id
                long[] nodeIds = ((ILongKeyPartOp) keyPart).getKeys();
                ServerLongAnyRow row = GraphMatrixUtils.getPSLongKeyRow(psContext, param);
                Long2ObjectOpenHashMap<int[]> nodeIdToTypes = new Long2ObjectOpenHashMap<>(nodeIds.length);
                for (long nodeId : nodeIds) {
                    if (row.get(nodeId) == null) {
                        // If node not exist, just skip
                        continue;
                    }
                    int[] types = ((GraphNode) (row.get(nodeId))).getTypes();
                    if (types != null) {
                        nodeIdToTypes.put(nodeId, types);
                    }
                }
                return new PartGetIntArrayAttrsResult(param.getPartKey().getPartitionId(), nodeIdToTypes);
            }
        default:
            {
                // TODO: support String, Int, and Any type node id
                throw new InvalidParameterException("Unsupport index type " + keyPart.getKeyType());
            }
    }
}
Also used : PartGetIntArrayAttrsResult(com.tencent.angel.graph.client.psf.get.utils.PartGetIntArrayAttrsResult) InvalidParameterException(com.tencent.angel.exception.InvalidParameterException) GeneralPartGetParam(com.tencent.angel.ml.matrix.psf.get.base.GeneralPartGetParam) KeyPart(com.tencent.angel.psagent.matrix.transport.router.KeyPart) Long2ObjectOpenHashMap(it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap) ServerLongAnyRow(com.tencent.angel.ps.storage.vector.ServerLongAnyRow)

Example 8 with InvalidParameterException

use of com.tencent.angel.exception.InvalidParameterException in project angel by Tencent.

the class GetEdgeFeats method partitionGet.

@Override
public PartitionGetResult partitionGet(PartitionGetParam partParam) {
    GeneralPartGetParam param = (GeneralPartGetParam) partParam;
    KeyPart keyPart = param.getIndicesPart();
    switch(keyPart.getKeyType()) {
        case LONG:
            {
                // Long type node id
                long[] nodeIds = ((ILongKeyPartOp) keyPart).getKeys();
                ServerLongAnyRow row = GraphMatrixUtils.getPSLongKeyRow(psContext, param);
                Long2ObjectOpenHashMap<IntFloatVector[]> nodeIdToFeats = new Long2ObjectOpenHashMap<>(nodeIds.length);
                for (long nodeId : nodeIds) {
                    if (row.get(nodeId) == null) {
                        // If node not exist, just skip
                        continue;
                    }
                    IntFloatVector[] edgeFeats = ((GraphNode) (row.get(nodeId))).getEdgeFeatures();
                    if (edgeFeats != null) {
                        nodeIdToFeats.put(nodeId, edgeFeats);
                    }
                }
                return new PartGetEdgeFeatsResult(param.getPartKey().getPartitionId(), nodeIdToFeats);
            }
        default:
            {
                // TODO: support String, Int, and Any type node id
                throw new InvalidParameterException("Unsupport index type " + keyPart.getKeyType());
            }
    }
}
Also used : InvalidParameterException(com.tencent.angel.exception.InvalidParameterException) KeyPart(com.tencent.angel.psagent.matrix.transport.router.KeyPart) Long2ObjectOpenHashMap(it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap) ServerLongAnyRow(com.tencent.angel.ps.storage.vector.ServerLongAnyRow) IntFloatVector(com.tencent.angel.ml.math2.vector.IntFloatVector)

Example 9 with InvalidParameterException

use of com.tencent.angel.exception.InvalidParameterException in project angel by Tencent.

the class GetEdgeTypes method partitionGet.

@Override
public PartitionGetResult partitionGet(PartitionGetParam partParam) {
    GeneralPartGetParam param = (GeneralPartGetParam) partParam;
    KeyPart keyPart = param.getIndicesPart();
    switch(keyPart.getKeyType()) {
        case LONG:
            {
                // Long type node id
                long[] nodeIds = ((ILongKeyPartOp) keyPart).getKeys();
                ServerLongAnyRow row = GraphMatrixUtils.getPSLongKeyRow(psContext, param);
                Long2ObjectOpenHashMap<int[]> nodeIdToTypes = new Long2ObjectOpenHashMap<>(nodeIds.length);
                for (long nodeId : nodeIds) {
                    if (row.get(nodeId) == null) {
                        // If node not exist, just skip
                        continue;
                    }
                    int[] types = ((GraphNode) (row.get(nodeId))).getEdgeTypes();
                    if (types != null) {
                        nodeIdToTypes.put(nodeId, types);
                    }
                }
                return new PartGetIntArrayAttrsResult(param.getPartKey().getPartitionId(), nodeIdToTypes);
            }
        default:
            {
                // TODO: support String, Int, and Any type node id
                throw new InvalidParameterException("Unsupport index type " + keyPart.getKeyType());
            }
    }
}
Also used : PartGetIntArrayAttrsResult(com.tencent.angel.graph.client.psf.get.utils.PartGetIntArrayAttrsResult) InvalidParameterException(com.tencent.angel.exception.InvalidParameterException) KeyPart(com.tencent.angel.psagent.matrix.transport.router.KeyPart) Long2ObjectOpenHashMap(it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap) ServerLongAnyRow(com.tencent.angel.ps.storage.vector.ServerLongAnyRow)

Example 10 with InvalidParameterException

use of com.tencent.angel.exception.InvalidParameterException in project angel by Tencent.

the class GetEdgeWeights method partitionGet.

@Override
public PartitionGetResult partitionGet(PartitionGetParam partParam) {
    GeneralPartGetParam param = (GeneralPartGetParam) partParam;
    KeyPart keyPart = param.getIndicesPart();
    switch(keyPart.getKeyType()) {
        case LONG:
            {
                // Long type node id
                long[] nodeIds = ((ILongKeyPartOp) keyPart).getKeys();
                ServerLongAnyRow row = GraphMatrixUtils.getPSLongKeyRow(psContext, param);
                Long2ObjectOpenHashMap<float[]> nodeIdToWeights = new Long2ObjectOpenHashMap<>(nodeIds.length);
                for (long nodeId : nodeIds) {
                    if (row.get(nodeId) == null) {
                        // If node not exist, just skip
                        continue;
                    }
                    float[] weights = ((GraphNode) (row.get(nodeId))).getWeights();
                    if (weights != null) {
                        nodeIdToWeights.put(nodeId, weights);
                    }
                }
                return new PartGetFloatArrayAttrsResult(param.getPartKey().getPartitionId(), nodeIdToWeights);
            }
        default:
            {
                // TODO: support String, Int, and Any type node id
                throw new InvalidParameterException("Unsupport index type " + keyPart.getKeyType());
            }
    }
}
Also used : InvalidParameterException(com.tencent.angel.exception.InvalidParameterException) KeyPart(com.tencent.angel.psagent.matrix.transport.router.KeyPart) Long2ObjectOpenHashMap(it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap) PartGetFloatArrayAttrsResult(com.tencent.angel.graph.client.psf.get.utils.PartGetFloatArrayAttrsResult) ServerLongAnyRow(com.tencent.angel.ps.storage.vector.ServerLongAnyRow)

Aggregations

InvalidParameterException (com.tencent.angel.exception.InvalidParameterException)24 KeyPart (com.tencent.angel.psagent.matrix.transport.router.KeyPart)8 ServerLongAnyRow (com.tencent.angel.ps.storage.vector.ServerLongAnyRow)7 Long2ObjectOpenHashMap (it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap)7 AngelException (com.tencent.angel.exception.AngelException)6 IOException (java.io.IOException)6 Path (org.apache.hadoop.fs.Path)4 MatrixMeta (com.tencent.angel.ml.matrix.MatrixMeta)3 GeneralPartGetParam (com.tencent.angel.ml.matrix.psf.get.base.GeneralPartGetParam)3 MatrixClient (com.tencent.angel.psagent.matrix.MatrixClient)3 PartGetFloatArrayAttrsResult (com.tencent.angel.graph.client.psf.get.utils.PartGetFloatArrayAttrsResult)2 PartGetIntArrayAttrsResult (com.tencent.angel.graph.client.psf.get.utils.PartGetIntArrayAttrsResult)2 IntFloatVector (com.tencent.angel.ml.math2.vector.IntFloatVector)2 LongFloatVector (com.tencent.angel.ml.math2.vector.LongFloatVector)2 MatrixContext (com.tencent.angel.ml.matrix.MatrixContext)2 ModelLineConvert (com.tencent.angel.tools.ModelLineConvert)2 TextModelLineConvert (com.tencent.angel.tools.TextModelLineConvert)2 FileNotFoundException (java.io.FileNotFoundException)2 ArrayList (java.util.ArrayList)2 Configuration (org.apache.hadoop.conf.Configuration)2