Search in sources :

Example 1 with PersistenceRegistry

use of com.graphaware.nlp.persistence.PersistenceRegistry in project neo4j-nlp by graphaware.

the class NLPManager method init.

public void init(GraphDatabaseService database, DynamicConfiguration configuration) {
    if (initialized) {
        return;
    }
    this.configuration = configuration;
    this.languageManager = new LanguageManager();
    this.database = database;
    this.persistenceRegistry = new PersistenceRegistry(database);
    this.enrichmentRegistry = buildAndRegisterEnrichers();
    this.eventDispatcher = new EventDispatcher();
    loadExtensions();
    if (textProcessorsManager == null) {
        this.textProcessorsManager = new TextProcessorsManager(configuration);
    }
    this.textProcessorsManager.registerPipelinesFromConfig();
    loadVectorComputationProcesses();
    loadSummarizers();
    registerWord2VecModelFromConfig();
    initialized = true;
}
Also used : PersistenceRegistry(com.graphaware.nlp.persistence.PersistenceRegistry) EventDispatcher(com.graphaware.nlp.event.EventDispatcher) TextProcessorsManager(com.graphaware.nlp.processor.TextProcessorsManager) LanguageManager(com.graphaware.nlp.language.LanguageManager)

Example 2 with PersistenceRegistry

use of com.graphaware.nlp.persistence.PersistenceRegistry in project neo4j-nlp by graphaware.

the class ConceptNet5EnricherIntegrationTest method testConceptWorksAfterImmediateRemoval.

@Test
public void testConceptWorksAfterImmediateRemoval() {
    PersistenceRegistry registry = new PersistenceRegistry(getDatabase());
    ConceptNet5Enricher enricher = new ConceptNet5Enricher(getDatabase(), registry, new TextProcessorsManager(new DynamicConfiguration(getDatabase())));
    clearDb();
    executeInTransaction("CALL ga.nlp.annotate({text: 'tension mounted as eclipse time approached.', id: 'test-proc', pipeline: 'tokenizer'})", (result -> {
    // 
    }));
    executeInTransaction("MATCH (n:Tag) CALL ga.nlp.enrich.concept({tag: n, depth: 2, language: 'en', admittedRelationships:['IsA','PartOf']}) YIELD result return result", (result -> {
        assertTrue(result.hasNext());
    }));
    executeInTransaction("MATCH (n)-[r:IS_RELATED_TO]->(x) DELETE r", (result -> {
    }));
    executeInTransaction("MATCH (n:Tag) WHERE size((n)--()) = 0 DELETE n", (result -> {
    }));
    executeInTransaction("MATCH (n:Tag) CALL ga.nlp.enrich.concept({tag: n, depth: 2, language: 'en', admittedRelationships:['IsA','PartOf']}) YIELD result return result", (result -> {
        assertTrue(result.hasNext());
    }));
    executeInTransaction("MATCH (n)-[r:IS_RELATED_TO]->(x) RETURN r", (result -> {
        assertTrue(result.hasNext());
    }));
}
Also used : Arrays(java.util.Arrays) DynamicConfiguration(com.graphaware.nlp.configuration.DynamicConfiguration) PersistenceRegistry(com.graphaware.nlp.persistence.PersistenceRegistry) Label(org.neo4j.graphdb.Label) EnricherAbstractTest(com.graphaware.nlp.enrich.EnricherAbstractTest) TextProcessor(com.graphaware.nlp.processor.TextProcessor) ConceptRequest(com.graphaware.nlp.dsl.request.ConceptRequest) Test(org.junit.Test) StubTextProcessor(com.graphaware.nlp.stub.StubTextProcessor) TextProcessorsManager(com.graphaware.nlp.processor.TextProcessorsManager) TestNLPGraph(com.graphaware.nlp.util.TestNLPGraph) Ignore(org.junit.Ignore) Transaction(org.neo4j.graphdb.Transaction) Assert(org.junit.Assert) Collections(java.util.Collections) Before(org.junit.Before) PersistenceRegistry(com.graphaware.nlp.persistence.PersistenceRegistry) DynamicConfiguration(com.graphaware.nlp.configuration.DynamicConfiguration) TextProcessorsManager(com.graphaware.nlp.processor.TextProcessorsManager) EnricherAbstractTest(com.graphaware.nlp.enrich.EnricherAbstractTest) Test(org.junit.Test)

Example 3 with PersistenceRegistry

use of com.graphaware.nlp.persistence.PersistenceRegistry in project neo4j-nlp by graphaware.

the class ConceptNet5EnricherIntegrationTest method testTagsCanBeEnrichedWithConceptNet5.

@Test
public void testTagsCanBeEnrichedWithConceptNet5() {
    PersistenceRegistry registry = new PersistenceRegistry(getDatabase());
    ConceptNet5Enricher enricher = new ConceptNet5Enricher(getDatabase(), registry, new TextProcessorsManager(new DynamicConfiguration(getDatabase())));
    clearDb();
    executeInTransaction("CALL ga.nlp.config.set('SETTING_fallbackLanguage','en')", emptyConsumer());
    executeInTransaction("CALL ga.nlp.annotate({text: 'kill cats', id: 'test-proc', pipeline: 'tokenizer'})", emptyConsumer());
    try (Transaction tx = getDatabase().beginTx()) {
        getDatabase().findNodes(Label.label("AnnotatedText")).stream().forEach(node -> {
            ConceptRequest request = new ConceptRequest();
            request.setAnnotatedNode(node);
            request.setDepth(1);
            request.setProcessor(StubTextProcessor.class.getName());
            request.setAdmittedRelationships(Arrays.asList("RelatedTo", "IsA"));
            request.setFilterByLanguage(true);
            request.setSplitTag(false);
            request.setRelDirection("both");
            request.setOutputLanguages(Arrays.asList("en"));
            enricher.importConcept(request);
            tx.success();
        });
    }
    debugTagsRelations();
    TestNLPGraph tester = new TestNLPGraph(getDatabase());
    tester.assertTagWithValueExist("cats");
    // tester.assertTagHasRelatedTag("cats", "cat");
    tester.assertTagHasRelatedTag("kill", "death");
}
Also used : PersistenceRegistry(com.graphaware.nlp.persistence.PersistenceRegistry) DynamicConfiguration(com.graphaware.nlp.configuration.DynamicConfiguration) Transaction(org.neo4j.graphdb.Transaction) TextProcessorsManager(com.graphaware.nlp.processor.TextProcessorsManager) TestNLPGraph(com.graphaware.nlp.util.TestNLPGraph) ConceptRequest(com.graphaware.nlp.dsl.request.ConceptRequest) StubTextProcessor(com.graphaware.nlp.stub.StubTextProcessor) EnricherAbstractTest(com.graphaware.nlp.enrich.EnricherAbstractTest) Test(org.junit.Test)

Example 4 with PersistenceRegistry

use of com.graphaware.nlp.persistence.PersistenceRegistry in project neo4j-nlp by graphaware.

the class ConceptNet5EnricherIntegrationTest method testRequestWithRelationshipsConstraintDoNotGetThem.

@Test
public void testRequestWithRelationshipsConstraintDoNotGetThem() {
    PersistenceRegistry registry = new PersistenceRegistry(getDatabase());
    ConceptNet5Enricher enricher = new ConceptNet5Enricher(getDatabase(), registry, new TextProcessorsManager(new DynamicConfiguration(getDatabase())));
    clearDb();
    executeInTransaction("CALL ga.nlp.annotate({text: 'tension mounted as the eclipse time approached.', id: 'test-proc', pipeline: 'tokenizer'})", (result -> {
    // 
    }));
    try (Transaction tx = getDatabase().beginTx()) {
        getDatabase().findNodes(Label.label("Tag")).stream().forEach(node -> {
            ConceptRequest request = new ConceptRequest();
            request.setTag(node);
            // request.setLanguage("en");
            request.setDepth(2);
            request.setProcessor(StubTextProcessor.class.getName());
            request.setAdmittedRelationships(Arrays.asList("IsA", "PartOf"));
            request.setFilterByLanguage(true);
            request.setSplitTag(false);
            enricher.importConcept(request);
            tx.success();
        });
    }
    executeInTransaction("MATCH (n)-[r:IS_RELATED_TO]->() WHERE r.type = 'AtLocation' RETURN n, r", (result -> {
        assertFalse(result.hasNext());
    }));
    debugTagsRelations();
}
Also used : Arrays(java.util.Arrays) DynamicConfiguration(com.graphaware.nlp.configuration.DynamicConfiguration) PersistenceRegistry(com.graphaware.nlp.persistence.PersistenceRegistry) Label(org.neo4j.graphdb.Label) EnricherAbstractTest(com.graphaware.nlp.enrich.EnricherAbstractTest) TextProcessor(com.graphaware.nlp.processor.TextProcessor) ConceptRequest(com.graphaware.nlp.dsl.request.ConceptRequest) Test(org.junit.Test) StubTextProcessor(com.graphaware.nlp.stub.StubTextProcessor) TextProcessorsManager(com.graphaware.nlp.processor.TextProcessorsManager) TestNLPGraph(com.graphaware.nlp.util.TestNLPGraph) Ignore(org.junit.Ignore) Transaction(org.neo4j.graphdb.Transaction) Assert(org.junit.Assert) Collections(java.util.Collections) Before(org.junit.Before) PersistenceRegistry(com.graphaware.nlp.persistence.PersistenceRegistry) DynamicConfiguration(com.graphaware.nlp.configuration.DynamicConfiguration) Transaction(org.neo4j.graphdb.Transaction) TextProcessorsManager(com.graphaware.nlp.processor.TextProcessorsManager) ConceptRequest(com.graphaware.nlp.dsl.request.ConceptRequest) StubTextProcessor(com.graphaware.nlp.stub.StubTextProcessor) EnricherAbstractTest(com.graphaware.nlp.enrich.EnricherAbstractTest) Test(org.junit.Test)

Example 5 with PersistenceRegistry

use of com.graphaware.nlp.persistence.PersistenceRegistry in project neo4j-nlp by graphaware.

the class ConceptNet5EnricherIntegrationTest method testConceptEnrichmentWithRelConstraintViaProcedure.

@Test
public void testConceptEnrichmentWithRelConstraintViaProcedure() {
    PersistenceRegistry registry = new PersistenceRegistry(getDatabase());
    ConceptNet5Enricher enricher = new ConceptNet5Enricher(getDatabase(), registry, new TextProcessorsManager(new DynamicConfiguration(getDatabase())));
    clearDb();
    executeInTransaction("CALL ga.nlp.annotate({text: 'tension mounted as eclipse time approached.', id: 'test-proc', pipeline: 'tokenizer'})", (result -> {
    // 
    }));
    executeInTransaction("MATCH (n:Tag) CALL ga.nlp.enrich.concept({tag: n, depth: 2, language: 'en', admittedRelationships:['IsA','PartOf']}) YIELD result return result", (result -> {
        assertTrue(result.hasNext());
    }));
    executeInTransaction("MATCH (n)-[r:IS_RELATED_TO]->() WHERE r.type = 'AtLocation' RETURN n, r", (result -> {
        assertFalse(result.hasNext());
    }));
    debugTagsRelations();
}
Also used : Arrays(java.util.Arrays) DynamicConfiguration(com.graphaware.nlp.configuration.DynamicConfiguration) PersistenceRegistry(com.graphaware.nlp.persistence.PersistenceRegistry) Label(org.neo4j.graphdb.Label) EnricherAbstractTest(com.graphaware.nlp.enrich.EnricherAbstractTest) TextProcessor(com.graphaware.nlp.processor.TextProcessor) ConceptRequest(com.graphaware.nlp.dsl.request.ConceptRequest) Test(org.junit.Test) StubTextProcessor(com.graphaware.nlp.stub.StubTextProcessor) TextProcessorsManager(com.graphaware.nlp.processor.TextProcessorsManager) TestNLPGraph(com.graphaware.nlp.util.TestNLPGraph) Ignore(org.junit.Ignore) Transaction(org.neo4j.graphdb.Transaction) Assert(org.junit.Assert) Collections(java.util.Collections) Before(org.junit.Before) PersistenceRegistry(com.graphaware.nlp.persistence.PersistenceRegistry) DynamicConfiguration(com.graphaware.nlp.configuration.DynamicConfiguration) TextProcessorsManager(com.graphaware.nlp.processor.TextProcessorsManager) EnricherAbstractTest(com.graphaware.nlp.enrich.EnricherAbstractTest) Test(org.junit.Test)

Aggregations

PersistenceRegistry (com.graphaware.nlp.persistence.PersistenceRegistry)9 TextProcessorsManager (com.graphaware.nlp.processor.TextProcessorsManager)9 DynamicConfiguration (com.graphaware.nlp.configuration.DynamicConfiguration)7 EnricherAbstractTest (com.graphaware.nlp.enrich.EnricherAbstractTest)7 Test (org.junit.Test)7 ConceptRequest (com.graphaware.nlp.dsl.request.ConceptRequest)6 StubTextProcessor (com.graphaware.nlp.stub.StubTextProcessor)6 Transaction (org.neo4j.graphdb.Transaction)6 TestNLPGraph (com.graphaware.nlp.util.TestNLPGraph)5 TextProcessor (com.graphaware.nlp.processor.TextProcessor)3 Arrays (java.util.Arrays)3 Collections (java.util.Collections)3 Assert (org.junit.Assert)3 Before (org.junit.Before)3 Ignore (org.junit.Ignore)3 Label (org.neo4j.graphdb.Label)3 EventDispatcher (com.graphaware.nlp.event.EventDispatcher)2 LanguageManager (com.graphaware.nlp.language.LanguageManager)1 QueryBasedVectorComputation (com.graphaware.nlp.vector.QueryBasedVectorComputation)1