Examples with Preprocessor edu.illinois.cs.cogcomp.depparse.io.Preprocessor used on opensource projects

Example 1 with Preprocessor

use of edu.illinois.cs.cogcomp.depparse.io.Preprocessor in project cogcomp-nlp by CogComp.

the class MainClass method getStructuredData.

private static SLProblem getStructuredData(String filepath, LabeledChuLiuEdmondsDecoder infSolver) throws Exception {
    CONLLReader depReader = new CONLLReader(new Preprocessor(), useGoldPOS, conllIndexOffset);
    depReader.startReading(filepath);
    SLProblem problem = new SLProblem();
    DepInst instance = depReader.getNext();
    while (instance != null) {
        infSolver.updateInferenceSolver(instance);
        Pair<IInstance, IStructure> pair = getSLPair(instance);
        problem.addExample(pair.getFirst(), pair.getSecond());
        instance = depReader.getNext();
    }
    logger.info("{} of dependency instances.", problem.size());
    return problem;
}

Example 2 with Preprocessor

use of edu.illinois.cs.cogcomp.depparse.io.Preprocessor in project cogcomp-nlp by CogComp.

the class MainClass method annotate.

private static void annotate(String filepath) throws IOException {
    DepAnnotator annotator = new DepAnnotator();
    TextAnnotationBuilder taBuilder = new TokenizerTextAnnotationBuilder(new StatefulTokenizer(true));
    Preprocessor preprocessor = new Preprocessor();
    Files.lines(Paths.get(filepath)).forEach(line -> {
        TextAnnotation ta = taBuilder.createTextAnnotation(line);
        try {
            preprocessor.annotate(ta);
            annotator.addView(ta);
            System.out.println(ta.getView(annotator.getViewName()).toString());
        } catch (AnnotatorException e) {
            e.printStackTrace();
        }
    });
}