Search in sources :

Example 1 with NERCombinerAnnotator

use of edu.stanford.nlp.pipeline.NERCombinerAnnotator in project CoreNLP by stanfordnlp.

the class NERBenchmarkSlowITest method setUp.

// TODO: Consider using NERFromConllAnnotator format
@Before
public void setUp() throws IOException {
    if (conllNERAnnotator == null || ontoNERAnnotator == null) {
        // Default properties are fine but need to provide a properties object in factory method
        Properties nerProps = new Properties();
        nerProps.setProperty("ner.useSUTime", "false");
        nerProps.setProperty("ner.applyNumericClassifiers", "false");
        nerProps.setProperty("ner.model", DefaultPaths.DEFAULT_NER_CONLL_MODEL);
        nerProps.setProperty("applyNumericClassifiers", "false");
        conllNERAnnotator = new NERCombinerAnnotator(nerProps);
        // Change NER model for different dataset -- everything else is the same
        nerProps.setProperty("ner.model", DefaultPaths.DEFAULT_NER_THREECLASS_MODEL);
        ontoNERAnnotator = new NERCombinerAnnotator(nerProps);
        // Set up conll pipeline
        Properties tokenizerProps = new Properties();
        tokenizerProps.setProperty("tokenize.whitespace", "true");
        conllNERAnnotationPipeline = new AnnotationPipeline();
        conllNERAnnotationPipeline.addAnnotator(new TokenizerAnnotator(false, tokenizerProps));
        conllNERAnnotationPipeline.addAnnotator(new WordsToSentencesAnnotator(false));
        conllNERAnnotationPipeline.addAnnotator(conllNERAnnotator);
        // Set up onto pipeline
        ontoNERAnnotationPipeline = new AnnotationPipeline();
        ontoNERAnnotationPipeline.addAnnotator(new TokenizerAnnotator(false, tokenizerProps));
        ontoNERAnnotationPipeline.addAnnotator(new WordsToSentencesAnnotator(false));
        ontoNERAnnotationPipeline.addAnnotator(ontoNERAnnotator);
    }
}
Also used : NERCombinerAnnotator(edu.stanford.nlp.pipeline.NERCombinerAnnotator) AnnotationPipeline(edu.stanford.nlp.pipeline.AnnotationPipeline) WordsToSentencesAnnotator(edu.stanford.nlp.pipeline.WordsToSentencesAnnotator) Properties(java.util.Properties) TokenizerAnnotator(edu.stanford.nlp.pipeline.TokenizerAnnotator) Before(org.junit.Before)

Example 2 with NERCombinerAnnotator

use of edu.stanford.nlp.pipeline.NERCombinerAnnotator in project CoreNLP by stanfordnlp.

the class PatternsSimpleThreadedITest method setUp.

@BeforeClass
public static void setUp() {
    nlpPipeline = new AnnotationPipeline();
    // We assume the input is already tokenized, so we use a cheap whitespace tokenizer.
    // The original code uses this property for the tokenizer:
    // props.setProperty("tokenize.options", "ptb3Escaping=false,normalizeParentheses=false,escapeForwardSlashAsterisk=false");
    nlpPipeline.addAnnotator(new TokenizerAnnotator(false, TokenizerType.Whitespace));
    nlpPipeline.addAnnotator(new WordsToSentencesAnnotator(false));
    nlpPipeline.addAnnotator(new POSTaggerAnnotator());
    nlpPipeline.addAnnotator(new MorphaAnnotator(false));
    Properties nerAnnotatorProperties = new Properties();
    nerAnnotatorProperties.setProperty("ner.useSUTime", Boolean.toString(false));
    nerAnnotatorProperties.setProperty("ner.applyFineGrained", Boolean.toString(false));
    // nerAnnotatorProperties.setProperty("ner.fine.regexner.mapping", spiedProperties.getProperty("fineGrainedRegexnerMapping"));
    try {
        nlpPipeline.addAnnotator(new NERCombinerAnnotator(nerAnnotatorProperties));
    } catch (IOException e) {
        throw new RuntimeIOException(e);
    }
}
Also used : POSTaggerAnnotator(edu.stanford.nlp.pipeline.POSTaggerAnnotator) RuntimeIOException(edu.stanford.nlp.io.RuntimeIOException) MorphaAnnotator(edu.stanford.nlp.pipeline.MorphaAnnotator) NERCombinerAnnotator(edu.stanford.nlp.pipeline.NERCombinerAnnotator) AnnotationPipeline(edu.stanford.nlp.pipeline.AnnotationPipeline) WordsToSentencesAnnotator(edu.stanford.nlp.pipeline.WordsToSentencesAnnotator) RuntimeIOException(edu.stanford.nlp.io.RuntimeIOException) IOException(java.io.IOException) TokenizerAnnotator(edu.stanford.nlp.pipeline.TokenizerAnnotator) Properties(java.util.Properties) BeforeClass(org.junit.BeforeClass)

Aggregations

AnnotationPipeline (edu.stanford.nlp.pipeline.AnnotationPipeline)2 NERCombinerAnnotator (edu.stanford.nlp.pipeline.NERCombinerAnnotator)2 TokenizerAnnotator (edu.stanford.nlp.pipeline.TokenizerAnnotator)2 WordsToSentencesAnnotator (edu.stanford.nlp.pipeline.WordsToSentencesAnnotator)2 Properties (java.util.Properties)2 RuntimeIOException (edu.stanford.nlp.io.RuntimeIOException)1 MorphaAnnotator (edu.stanford.nlp.pipeline.MorphaAnnotator)1 POSTaggerAnnotator (edu.stanford.nlp.pipeline.POSTaggerAnnotator)1 IOException (java.io.IOException)1 Before (org.junit.Before)1 BeforeClass (org.junit.BeforeClass)1