Search in sources :

Example 1 with ShiftReduceParser

use of edu.stanford.nlp.parser.shiftreduce.ShiftReduceParser in project CoreNLP by stanfordnlp.

the class TaggerParserPosTagCompatibilityITest method testTagSet4.

private static void testTagSet4(String[] lexParsers, String[] maxentTaggers, String[] srParsers, String[] nnDepParsers) {
    LexicalizedParser lp = LexicalizedParser.loadModel(lexParsers[0]);
    Set<String> tagSet = lp.getLexicon().tagSet(lp.treebankLanguagePack().getBasicCategoryFunction());
    for (String name : maxentTaggers) {
        MaxentTagger tagger = new MaxentTagger(name);
        assertEquals(lexParsers[0] + " vs. " + name + " tag set mismatch:\n" + "left - right: " + Sets.diff(tagSet, tagger.tagSet()) + "; right - left: " + Sets.diff(tagger.tagSet(), tagSet) + "\n", tagSet, tagger.tagSet());
    }
    for (String name : lexParsers) {
        LexicalizedParser lp2 = LexicalizedParser.loadModel(name);
        assertEquals(lexParsers[0] + " vs. " + name + " tag set mismatch:\n" + "left - right: " + Sets.diff(tagSet, lp2.getLexicon().tagSet(lp.treebankLanguagePack().getBasicCategoryFunction())) + "; right - left: " + Sets.diff(lp2.getLexicon().tagSet(lp.treebankLanguagePack().getBasicCategoryFunction()), tagSet) + "\n", tagSet, lp2.getLexicon().tagSet(lp.treebankLanguagePack().getBasicCategoryFunction()));
    }
    for (String name : srParsers) {
        ShiftReduceParser srp = ShiftReduceParser.loadModel(name);
        assertEquals(lexParsers[0] + " vs. " + name + " tag set mismatch:\n" + "left - right: " + Sets.diff(tagSet, srp.tagSet()) + "; right - left: " + Sets.diff(srp.tagSet(), tagSet) + "\n", tagSet, srp.tagSet());
    }
    for (String name : nnDepParsers) {
        DependencyParser dp = DependencyParser.loadFromModelFile(name);
        assertEquals(lexParsers[0] + " vs. " + name + " tag set mismatch:\n" + "left - right: " + Sets.diff(tagSet, dp.getPosSet()) + "; right - left: " + Sets.diff(dp.getPosSet(), tagSet) + "\n", tagSet, dp.getPosSet());
    }
}
Also used : MaxentTagger(edu.stanford.nlp.tagger.maxent.MaxentTagger) ShiftReduceParser(edu.stanford.nlp.parser.shiftreduce.ShiftReduceParser) DependencyParser(edu.stanford.nlp.parser.nndep.DependencyParser) LexicalizedParser(edu.stanford.nlp.parser.lexparser.LexicalizedParser)

Example 2 with ShiftReduceParser

use of edu.stanford.nlp.parser.shiftreduce.ShiftReduceParser in project CoreNLP by stanfordnlp.

the class ShiftReduceDemo method main.

public static void main(String[] args) {
    String modelPath = "edu/stanford/nlp/models/srparser/englishSR.ser.gz";
    String taggerPath = "edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger";
    for (int argIndex = 0; argIndex < args.length; ) {
        switch(args[argIndex]) {
            case "-tagger":
                taggerPath = args[argIndex + 1];
                argIndex += 2;
                break;
            case "-model":
                modelPath = args[argIndex + 1];
                argIndex += 2;
                break;
            default:
                throw new RuntimeException("Unknown argument " + args[argIndex]);
        }
    }
    String text = "My dog likes to shake his stuffed chickadee toy.";
    MaxentTagger tagger = new MaxentTagger(taggerPath);
    ShiftReduceParser model = ShiftReduceParser.loadModel(modelPath);
    DocumentPreprocessor tokenizer = new DocumentPreprocessor(new StringReader(text));
    for (List<HasWord> sentence : tokenizer) {
        List<TaggedWord> tagged = tagger.tagSentence(sentence);
        Tree tree = model.apply(tagged);
        log.info(tree);
    }
}
Also used : HasWord(edu.stanford.nlp.ling.HasWord) TaggedWord(edu.stanford.nlp.ling.TaggedWord) MaxentTagger(edu.stanford.nlp.tagger.maxent.MaxentTagger) ShiftReduceParser(edu.stanford.nlp.parser.shiftreduce.ShiftReduceParser) StringReader(java.io.StringReader) Tree(edu.stanford.nlp.trees.Tree) DocumentPreprocessor(edu.stanford.nlp.process.DocumentPreprocessor)

Aggregations

ShiftReduceParser (edu.stanford.nlp.parser.shiftreduce.ShiftReduceParser)2 MaxentTagger (edu.stanford.nlp.tagger.maxent.MaxentTagger)2 HasWord (edu.stanford.nlp.ling.HasWord)1 TaggedWord (edu.stanford.nlp.ling.TaggedWord)1 LexicalizedParser (edu.stanford.nlp.parser.lexparser.LexicalizedParser)1 DependencyParser (edu.stanford.nlp.parser.nndep.DependencyParser)1 DocumentPreprocessor (edu.stanford.nlp.process.DocumentPreprocessor)1 Tree (edu.stanford.nlp.trees.Tree)1 StringReader (java.io.StringReader)1