Search in sources :

Example 46 with PipelineSpecification

use of com.graphaware.nlp.dsl.request.PipelineSpecification in project neo4j-nlp by graphaware.

the class AnnotateFunction method getAnnotation.

@UserFunction("ga.nlp.processor.annotate")
@Description("Perform the annotation on the given text, returns the produced annotation domain")
public Map<String, Object> getAnnotation(@Name("text") String text, @Name("pipelineSpecification") Map<String, Object> specificationInput) {
    if (!specificationInput.containsKey("name")) {
        throw new RuntimeException("You mast specify the name of the pipeline");
    }
    PipelineSpecification spec = getNLPManager().getTextProcessorsManager().getPipelineSpecification((String) specificationInput.get("name"));
    TextProcessor processor = getNLPManager().getTextProcessorsManager().getTextProcessor(spec.getTextProcessor());
    AnnotatedText annotatedText = processor.annotateText(text, spec);
    ObjectMapper mapper = new ObjectMapper();
    mapper.configure(SerializationConfig.Feature.FAIL_ON_EMPTY_BEANS, false);
    Map map = mapper.convertValue(annotatedText, Map.class);
    return map;
}
Also used : PipelineSpecification(com.graphaware.nlp.dsl.request.PipelineSpecification) TextProcessor(com.graphaware.nlp.processor.TextProcessor) AnnotatedText(com.graphaware.nlp.domain.AnnotatedText) Map(java.util.Map) ObjectMapper(org.codehaus.jackson.map.ObjectMapper) Description(org.neo4j.procedure.Description) UserFunction(org.neo4j.procedure.UserFunction)

Example 47 with PipelineSpecification

use of com.graphaware.nlp.dsl.request.PipelineSpecification in project neo4j-nlp by graphaware.

the class TextProcessorsProcedure method refreshPipeline.

@Procedure(value = "ga.nlp.refreshPipeline", mode = Mode.WRITE)
@Description("Refresh pipeline for the given name")
public Stream<PipelineSpecification> refreshPipeline(@Name(value = "pipeline") String pipeline) {
    try {
        PipelineSpecification pipelineSpecification = getNLPManager().getConfiguration().loadPipeline(pipeline);
        getNLPManager().getTextProcessorsManager().removePipeline(pipelineSpecification.getName(), pipelineSpecification.getTextProcessor());
        PipelineSpecification newPipeline = new PipelineSpecification(pipelineSpecification.getName(), pipelineSpecification.getLanguage(), pipelineSpecification.getTextProcessor(), pipelineSpecification.getProcessingStepsAsStrings(), pipelineSpecification.getStopWords(), pipelineSpecification.getThreadNumber(), pipelineSpecification.getExcludedNER(), pipelineSpecification.getExcludedPOS());
        getNLPManager().getTextProcessorsManager().addPipeline(newPipeline);
        return Stream.of(newPipeline);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}
Also used : PipelineSpecification(com.graphaware.nlp.dsl.request.PipelineSpecification) Description(org.neo4j.procedure.Description) Procedure(org.neo4j.procedure.Procedure)

Example 48 with PipelineSpecification

use of com.graphaware.nlp.dsl.request.PipelineSpecification in project neo4j-nlp by graphaware.

the class MigrationHandler method doMigrate.

private void doMigrate(String k) throws Exception {
    String specStr = graphKeyValueStore.get(k).toString();
    if (!specStr.contains("\"@class\"")) {
        PipelineSpecification pipelineSpecification = mapper.readValue(specStr, PipelineSpecification.class);
        graphKeyValueStore.remove(k);
        configuration.storeCustomPipeline(pipelineSpecification);
    }
}
Also used : PipelineSpecification(com.graphaware.nlp.dsl.request.PipelineSpecification)

Example 49 with PipelineSpecification

use of com.graphaware.nlp.dsl.request.PipelineSpecification in project neo4j-nlp by graphaware.

the class DynamicConfiguration method removePipeline.

public void removePipeline(String name, String textProcessor) {
    Map<String, Object> config = getAllConfigValuesFromStore();
    config.keySet().forEach(k -> {
        if (k.startsWith(PIPELINE_KEY_PREFIX)) {
            try {
                PipelineSpecification pipelineSpecification = mapper.readValue(config.get(k).toString(), PipelineSpecification.class);
                if (pipelineSpecification.getName().equals(name) && pipelineSpecification.getTextProcessor().equals(textProcessor)) {
                    removeKey(STORE_KEY + k);
                }
            } catch (Exception e) {
                throw new RuntimeException(e.getMessage());
            }
        }
    });
}
Also used : PipelineSpecification(com.graphaware.nlp.dsl.request.PipelineSpecification) IOException(java.io.IOException)

Aggregations

PipelineSpecification (com.graphaware.nlp.dsl.request.PipelineSpecification)49 Test (org.junit.Test)22 StubTextProcessor (com.graphaware.nlp.stub.StubTextProcessor)13 StanfordTextProcessor (com.graphaware.nlp.processor.stanford.StanfordTextProcessor)11 NLPIntegrationTest (com.graphaware.nlp.NLPIntegrationTest)8 TestAnnotatedText (com.graphaware.nlp.util.TestAnnotatedText)8 Transaction (org.neo4j.graphdb.Transaction)8 AbstractEmbeddedTest (com.graphaware.nlp.AbstractEmbeddedTest)5 NLPTextProcessor (com.graphaware.nlp.annotation.NLPTextProcessor)4 TextProcessor (com.graphaware.nlp.processor.TextProcessor)3 CoreAnnotations (edu.stanford.nlp.ling.CoreAnnotations)3 Annotation (edu.stanford.nlp.pipeline.Annotation)3 StanfordCoreNLP (edu.stanford.nlp.pipeline.StanfordCoreNLP)3 SemanticGraph (edu.stanford.nlp.semgraph.SemanticGraph)3 SemanticGraphCoreAnnotations (edu.stanford.nlp.semgraph.SemanticGraphCoreAnnotations)3 SemanticGraphEdge (edu.stanford.nlp.semgraph.SemanticGraphEdge)3 CoreMap (edu.stanford.nlp.util.CoreMap)3 Description (org.neo4j.procedure.Description)3 LoggerFactory (com.graphaware.common.log.LoggerFactory)2 NLPManager (com.graphaware.nlp.NLPManager)2