Search in sources :

Example 1 with GraphFormatException

use of edu.cmu.ml.proppr.graph.GraphFormatException in project ProPPR by TeamCohen.

the class RWExampleParser method parse.

public PosNegRWExample parse(String line, LearningGraphBuilder builder, SRW learner) throws GraphFormatException {
    //String[] parts = line.trim().split(MAJOR_DELIM,5);
    // first parse the query metadata
    //LearningGraphBuilder.split(line,'\t',4);
    String[] parts = new String[4];
    int last = 0, i = 0;
    for (int next = last; i < parts.length; last = next + 1, i++) {
        if (next == -1)
            throw new GraphFormatException("Need 8 distinct tsv fields in the grounded example:" + line);
        next = line.indexOf(MAJOR_DELIM, last);
        parts[i] = next < 0 ? line.substring(last) : line.substring(last, next);
    }
    TIntDoubleMap queryVec = new TIntDoubleHashMap();
    //for(String u : parts[1].split(MINOR_DELIM)) queryVec.put(Integer.parseInt(u), 1.0);
    for (int u : parseNodes(parts[1])) queryVec.put(u, 1.0);
    int[] posList, negList;
    if (//stringToInt(parts[2].split(MINOR_DELIM));
    parts[2].length() > 0)
        //stringToInt(parts[2].split(MINOR_DELIM));
        posList = parseNodes(parts[2]);
    else
        posList = new int[0];
    if (//stringToInt(parts[3].split(MINOR_DELIM));
    parts[3].length() > 0)
        //stringToInt(parts[3].split(MINOR_DELIM));
        negList = parseNodes(parts[3]);
    else
        negList = new int[0];
    LearningGraph g = builder.deserialize(line.substring(last));
    return learner.makeExample(parts[0], g, queryVec, posList, negList);
}
Also used : TIntDoubleHashMap(gnu.trove.map.hash.TIntDoubleHashMap) TIntDoubleMap(gnu.trove.map.TIntDoubleMap) GraphFormatException(edu.cmu.ml.proppr.graph.GraphFormatException) LearningGraph(edu.cmu.ml.proppr.graph.LearningGraph)

Example 2 with GraphFormatException

use of edu.cmu.ml.proppr.graph.GraphFormatException in project ProPPR by TeamCohen.

the class CachingTrainer method train.

@Override
public ParamVector<String, ?> train(SymbolTable<String> masterFeatures, Iterable<String> exampleFile, LearningGraphBuilder builder, ParamVector<String, ?> initialParamVec, int numEpochs) {
    ArrayList<PosNegRWExample> examples = new ArrayList<PosNegRWExample>();
    RWExampleParser parser = new RWExampleParser();
    if (masterFeatures.size() > 0)
        LearningGraphBuilder.setFeatures(masterFeatures);
    int id = 0;
    StatusLogger stattime = new StatusLogger();
    TrainingStatistics total = new TrainingStatistics();
    boolean logged = false;
    for (String s : exampleFile) {
        total.updateReadingStatistics(stattime.sinceLast());
        id++;
        try {
            stattime.tick();
            PosNegRWExample ex = parser.parse(s, builder, masterLearner);
            total.updateParsingStatistics(stattime.sinceLast());
            examples.add(ex);
            if (status.due()) {
                log.info("Parsed " + id + " ...");
                logged = true;
            }
        } catch (GraphFormatException e) {
            log.error("Trouble with #" + id, e);
        }
        stattime.tick();
    }
    if (logged)
        log.info("Total parsed: " + id);
    return trainCached(examples, builder, initialParamVec, numEpochs, total);
}
Also used : StatusLogger(edu.cmu.ml.proppr.util.StatusLogger) PosNegRWExample(edu.cmu.ml.proppr.examples.PosNegRWExample) ArrayList(java.util.ArrayList) GraphFormatException(edu.cmu.ml.proppr.graph.GraphFormatException) RWExampleParser(edu.cmu.ml.proppr.learn.tools.RWExampleParser)

Aggregations

GraphFormatException (edu.cmu.ml.proppr.graph.GraphFormatException)2 PosNegRWExample (edu.cmu.ml.proppr.examples.PosNegRWExample)1 LearningGraph (edu.cmu.ml.proppr.graph.LearningGraph)1 RWExampleParser (edu.cmu.ml.proppr.learn.tools.RWExampleParser)1 StatusLogger (edu.cmu.ml.proppr.util.StatusLogger)1 TIntDoubleMap (gnu.trove.map.TIntDoubleMap)1 TIntDoubleHashMap (gnu.trove.map.hash.TIntDoubleHashMap)1 ArrayList (java.util.ArrayList)1