Search in sources :

Example 21 with PipelineSpecification

use of com.graphaware.nlp.dsl.request.PipelineSpecification in project neo4j-nlp by graphaware.

the class TextRank method initializePipelineWithoutNEs.

private void initializePipelineWithoutNEs() {
    // System.out.println(" >>> default processor: " + NLPManager.getInstance().getTextProcessorsManager().getDefaultProcessor().getAlias());
    Map<String, Object> params = new HashMap<>();
    params.put("tokenize", true);
    params.put("ner", false);
    PipelineSpecification ps = new PipelineSpecification(PIPELINE_WITHOUT_NER, null);
    ps.setProcessingSteps(params);
    NLPManager.getInstance().getTextProcessorsManager().getDefaultProcessor().createPipeline(ps);
}
Also used : PipelineSpecification(com.graphaware.nlp.dsl.request.PipelineSpecification) TfIdfObject(com.graphaware.nlp.domain.TfIdfObject)

Example 22 with PipelineSpecification

use of com.graphaware.nlp.dsl.request.PipelineSpecification in project neo4j-nlp by graphaware.

the class ConceptNet5Importer method getDefaultPipeline.

private PipelineSpecification getDefaultPipeline() {
    NLPManager manager = NLPManager.getInstance();
    String pipeline = manager.getPipeline(null);
    PipelineSpecification pipelineSpecification = manager.getConfiguration().loadPipeline(pipeline);
    if (pipelineSpecification == null) {
        throw new RuntimeException("No default pipeline");
    }
    return pipelineSpecification;
}
Also used : PipelineSpecification(com.graphaware.nlp.dsl.request.PipelineSpecification) NLPManager(com.graphaware.nlp.NLPManager)

Example 23 with PipelineSpecification

use of com.graphaware.nlp.dsl.request.PipelineSpecification in project neo4j-nlp by graphaware.

the class TextProcessorsProcedure method setDefaultPipeline.

@Procedure(value = "ga.nlp.processor.pipeline.default", mode = Mode.WRITE)
@Description("Specify the pipeline to be used by default")
public Stream<SingleResult> setDefaultPipeline(@Name("name") String name) {
    PipelineSpecification pipelineSpecification = getConfiguration().loadPipeline(name);
    if (null == pipelineSpecification) {
        throw new RuntimeException("Pipeline " + name + " does not exist");
    }
    getConfiguration().updateInternalSetting(SettingsConstants.DEFAULT_PIPELINE, name);
    return Stream.of(SingleResult.success());
}
Also used : PipelineSpecification(com.graphaware.nlp.dsl.request.PipelineSpecification) Description(org.neo4j.procedure.Description) Procedure(org.neo4j.procedure.Procedure)

Example 24 with PipelineSpecification

use of com.graphaware.nlp.dsl.request.PipelineSpecification in project neo4j-nlp by graphaware.

the class NLPManager method annotateTextAndPersist.

public Node annotateTextAndPersist(String text, String id, String textProcessor, String pipelineName, boolean force, boolean checkForLanguage) {
    String lang = checkTextLanguage(text, checkForLanguage);
    String pipeline = getPipeline(pipelineName);
    PipelineSpecification pipelineSpecification = getConfiguration().loadPipeline(pipeline);
    if (null == pipelineSpecification) {
        throw new RuntimeException("No pipeline " + pipelineName + " found.");
    }
    TextProcessor processor = textProcessorsManager.getTextProcessor(pipelineSpecification.getTextProcessor());
    AnnotatedText at = processor.annotateText(text, lang, pipelineSpecification);
    return processAnnotationPersist(id, text, at, pipelineSpecification);
}
Also used : PipelineSpecification(com.graphaware.nlp.dsl.request.PipelineSpecification) TextProcessor(com.graphaware.nlp.processor.TextProcessor) AnnotatedText(com.graphaware.nlp.domain.AnnotatedText)

Example 25 with PipelineSpecification

use of com.graphaware.nlp.dsl.request.PipelineSpecification in project neo4j-nlp by graphaware.

the class NLPManager method setDefaultPipeline.

public void setDefaultPipeline(String pipeline) {
    PipelineSpecification pipelineSpecification = configuration.loadPipeline(pipeline);
    if (null == pipelineSpecification) {
        throw new RuntimeException("No pipeline " + pipeline + " exist");
    }
    configuration.updateInternalSetting(SettingsConstants.DEFAULT_PIPELINE, pipeline);
}
Also used : PipelineSpecification(com.graphaware.nlp.dsl.request.PipelineSpecification)

Aggregations

PipelineSpecification (com.graphaware.nlp.dsl.request.PipelineSpecification)49 Test (org.junit.Test)22 StubTextProcessor (com.graphaware.nlp.stub.StubTextProcessor)13 StanfordTextProcessor (com.graphaware.nlp.processor.stanford.StanfordTextProcessor)11 NLPIntegrationTest (com.graphaware.nlp.NLPIntegrationTest)8 TestAnnotatedText (com.graphaware.nlp.util.TestAnnotatedText)8 Transaction (org.neo4j.graphdb.Transaction)8 AbstractEmbeddedTest (com.graphaware.nlp.AbstractEmbeddedTest)5 NLPTextProcessor (com.graphaware.nlp.annotation.NLPTextProcessor)4 TextProcessor (com.graphaware.nlp.processor.TextProcessor)3 CoreAnnotations (edu.stanford.nlp.ling.CoreAnnotations)3 Annotation (edu.stanford.nlp.pipeline.Annotation)3 StanfordCoreNLP (edu.stanford.nlp.pipeline.StanfordCoreNLP)3 SemanticGraph (edu.stanford.nlp.semgraph.SemanticGraph)3 SemanticGraphCoreAnnotations (edu.stanford.nlp.semgraph.SemanticGraphCoreAnnotations)3 SemanticGraphEdge (edu.stanford.nlp.semgraph.SemanticGraphEdge)3 CoreMap (edu.stanford.nlp.util.CoreMap)3 Description (org.neo4j.procedure.Description)3 LoggerFactory (com.graphaware.common.log.LoggerFactory)2 NLPManager (com.graphaware.nlp.NLPManager)2