Search in sources :

Example 61 with SailRepositoryConnection

use of org.eclipse.rdf4j.repository.sail.SailRepositoryConnection in project incubator-rya by apache.

the class RyaInputIncrementalUpdateIT method historicThenStreamedResults.

/**
 * Simulates the case where a Triple is added to Rya, a new query that
 * includes that triple as a historic match is inserted into Fluo, and then
 * some new triple that matches the query is streamed into Fluo. The query's
 * results must include both the historic result and the newly streamed
 * result.
 */
@Test
public void historicThenStreamedResults() throws Exception {
    // A query that finds people who talk to Eve and work at Chipotle.
    final String sparql = "SELECT ?x " + "WHERE { " + "?x <http://talksTo> <http://Eve>. " + "?x <http://worksAt> <http://Chipotle>." + "}";
    // Triples that are loaded into Rya before the PCJ is created.
    final ValueFactory vf = SimpleValueFactory.getInstance();
    final Set<Statement> historicTriples = Sets.newHashSet(vf.createStatement(vf.createIRI("http://Alice"), vf.createIRI("http://talksTo"), vf.createIRI("http://Eve")), vf.createStatement(vf.createIRI("http://Alice"), vf.createIRI("http://worksAt"), vf.createIRI("http://Chipotle")), vf.createStatement(vf.createIRI("http://Joe"), vf.createIRI("http://worksAt"), vf.createIRI("http://Chipotle")));
    // Triples that will be streamed into Fluo after the PCJ has been
    final Set<Statement> streamedTriples = Sets.newHashSet(vf.createStatement(vf.createIRI("http://Frank"), vf.createIRI("http://talksTo"), vf.createIRI("http://Eve")), vf.createStatement(vf.createIRI("http://Joe"), vf.createIRI("http://talksTo"), vf.createIRI("http://Eve")), vf.createStatement(vf.createIRI("http://Frank"), vf.createIRI("http://worksAt"), vf.createIRI("http://Chipotle")));
    // Load the historic data into Rya.
    final SailRepositoryConnection ryaConn = super.getRyaSailRepository().getConnection();
    for (final Statement triple : historicTriples) {
        ryaConn.add(triple);
    }
    // Create the PCJ table.
    final Connector accumuloConn = super.getAccumuloConnector();
    final PrecomputedJoinStorage pcjStorage = new AccumuloPcjStorage(accumuloConn, getRyaInstanceName());
    final String pcjId = pcjStorage.createPcj(sparql);
    try (FluoClient fluoClient = FluoFactory.newClient(super.getFluoConfiguration())) {
        // Tell the Fluo app to maintain the PCJ.
        new CreateFluoPcj().withRyaIntegration(pcjId, pcjStorage, fluoClient, accumuloConn, getRyaInstanceName());
        super.getMiniFluo().waitForObservers();
        // Load the streaming data into Rya.
        for (final Statement triple : streamedTriples) {
            ryaConn.add(triple);
        }
        // Ensure Alice is a match.
        super.getMiniFluo().waitForObservers();
        final Set<BindingSet> expected = new HashSet<>();
        MapBindingSet bs = new MapBindingSet();
        bs.addBinding("x", vf.createIRI("http://Alice"));
        expected.add(bs);
        bs = new MapBindingSet();
        bs.addBinding("x", vf.createIRI("http://Frank"));
        expected.add(bs);
        bs = new MapBindingSet();
        bs.addBinding("x", vf.createIRI("http://Joe"));
        expected.add(bs);
        final Set<BindingSet> results = new HashSet<>();
        try (CloseableIterator<BindingSet> resultIt = pcjStorage.listResults(pcjId)) {
            while (resultIt.hasNext()) {
                results.add(resultIt.next());
            }
        }
        assertEquals(expected, results);
    }
}
Also used : Connector(org.apache.accumulo.core.client.Connector) BindingSet(org.eclipse.rdf4j.query.BindingSet) MapBindingSet(org.eclipse.rdf4j.query.impl.MapBindingSet) FluoClient(org.apache.fluo.api.client.FluoClient) AccumuloPcjStorage(org.apache.rya.indexing.pcj.storage.accumulo.AccumuloPcjStorage) Statement(org.eclipse.rdf4j.model.Statement) CreateFluoPcj(org.apache.rya.indexing.pcj.fluo.api.CreateFluoPcj) ValueFactory(org.eclipse.rdf4j.model.ValueFactory) SimpleValueFactory(org.eclipse.rdf4j.model.impl.SimpleValueFactory) SailRepositoryConnection(org.eclipse.rdf4j.repository.sail.SailRepositoryConnection) PrecomputedJoinStorage(org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage) MapBindingSet(org.eclipse.rdf4j.query.impl.MapBindingSet) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 62 with SailRepositoryConnection

use of org.eclipse.rdf4j.repository.sail.SailRepositoryConnection in project incubator-rya by apache.

the class InputIT method historicAndStreamConflict.

/**
 * Simulates the case where a Triple is added to Rya, a new query that
 * includes the triple as a historic match is inserted into Fluo, and then
 * the same triple is streamed into Fluo. The query's results will already
 * include the Triple because they were added while the query was being
 * created. This case should not fail or effect the end results in any way.
 */
@Test
public void historicAndStreamConflict() throws Exception {
    // A query that finds people who talk to Eve and work at Chipotle.
    final String sparql = "SELECT ?x WHERE { " + "?x <http://talksTo> <http://Eve>. " + "?x <http://worksAt> <http://Chipotle>." + "}";
    // Triples that are loaded into Rya before the PCJ is created.
    final ValueFactory vf = SimpleValueFactory.getInstance();
    final Set<Statement> historicTriples = Sets.newHashSet(vf.createStatement(vf.createIRI("http://Alice"), vf.createIRI("http://talksTo"), vf.createIRI("http://Eve")), vf.createStatement(vf.createIRI("http://Alice"), vf.createIRI("http://worksAt"), vf.createIRI("http://Chipotle")));
    // Triples that will be streamed into Fluo after the PCJ has been created.
    final Set<RyaStatement> streamedTriples = Sets.newHashSet(new RyaStatement(new RyaIRI("http://Alice"), new RyaIRI("http://talksTo"), new RyaIRI("http://Eve")), new RyaStatement(new RyaIRI("http://Alice"), new RyaIRI("http://worksAt"), new RyaIRI("http://Chipotle")));
    // The expected final result.
    final Set<BindingSet> expected = new HashSet<>();
    final MapBindingSet bs = new MapBindingSet();
    bs.addBinding("x", vf.createIRI("http://Alice"));
    expected.add(bs);
    // Load the historic data into Rya.
    final SailRepositoryConnection ryaConn = super.getRyaSailRepository().getConnection();
    for (final Statement triple : historicTriples) {
        ryaConn.add(triple);
    }
    ryaConn.close();
    // Create the PCJ table.
    final Connector accumuloConn = super.getAccumuloConnector();
    final PrecomputedJoinStorage pcjStorage = new AccumuloPcjStorage(accumuloConn, getRyaInstanceName());
    final String pcjId = pcjStorage.createPcj(sparql);
    try (FluoClient fluoClient = FluoFactory.newClient(super.getFluoConfiguration())) {
        // Tell the Fluo app to maintain the PCJ.
        new CreateFluoPcj().withRyaIntegration(pcjId, pcjStorage, fluoClient, accumuloConn, getRyaInstanceName());
        // Ensure Alice is a match.
        super.getMiniFluo().waitForObservers();
        Set<BindingSet> results = new HashSet<>();
        try (CloseableIterator<BindingSet> resultsIt = pcjStorage.listResults(pcjId)) {
            while (resultsIt.hasNext()) {
                results.add(resultsIt.next());
            }
        }
        assertEquals(expected, results);
        // Stream the same Alice triple into Fluo.
        new InsertTriples().insert(fluoClient, streamedTriples, Optional.absent());
        // Verify the end results of the query is stiill only Alice.
        super.getMiniFluo().waitForObservers();
        results = new HashSet<>();
        try (CloseableIterator<BindingSet> resultsIt = pcjStorage.listResults(pcjId)) {
            while (resultsIt.hasNext()) {
                results.add(resultsIt.next());
            }
        }
        assertEquals(expected, results);
    }
}
Also used : BindingSet(org.eclipse.rdf4j.query.BindingSet) MapBindingSet(org.eclipse.rdf4j.query.impl.MapBindingSet) Connector(org.apache.accumulo.core.client.Connector) FluoClient(org.apache.fluo.api.client.FluoClient) AccumuloPcjStorage(org.apache.rya.indexing.pcj.storage.accumulo.AccumuloPcjStorage) RyaIRI(org.apache.rya.api.domain.RyaIRI) InsertTriples(org.apache.rya.indexing.pcj.fluo.api.InsertTriples) RyaStatement(org.apache.rya.api.domain.RyaStatement) Statement(org.eclipse.rdf4j.model.Statement) RyaStatement(org.apache.rya.api.domain.RyaStatement) CreateFluoPcj(org.apache.rya.indexing.pcj.fluo.api.CreateFluoPcj) ValueFactory(org.eclipse.rdf4j.model.ValueFactory) SimpleValueFactory(org.eclipse.rdf4j.model.impl.SimpleValueFactory) SailRepositoryConnection(org.eclipse.rdf4j.repository.sail.SailRepositoryConnection) PrecomputedJoinStorage(org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage) MapBindingSet(org.eclipse.rdf4j.query.impl.MapBindingSet) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 63 with SailRepositoryConnection

use of org.eclipse.rdf4j.repository.sail.SailRepositoryConnection in project incubator-rya by apache.

the class InputIT method historicResults.

/**
 * Ensure historic matches are included in the result.
 */
@Test
public void historicResults() throws Exception {
    // A query that finds people who talk to Eve and work at Chipotle.
    final String sparql = "SELECT ?x WHERE { " + "?x <http://talksTo> <http://Eve>. " + "?x <http://worksAt> <http://Chipotle>." + "}";
    // Triples that are loaded into Rya before the PCJ is created.
    final ValueFactory vf = SimpleValueFactory.getInstance();
    final Set<Statement> historicTriples = Sets.newHashSet(vf.createStatement(vf.createIRI("http://Alice"), vf.createIRI("http://talksTo"), vf.createIRI("http://Eve")), vf.createStatement(vf.createIRI("http://Bob"), vf.createIRI("http://talksTo"), vf.createIRI("http://Eve")), vf.createStatement(vf.createIRI("http://Charlie"), vf.createIRI("http://talksTo"), vf.createIRI("http://Eve")), vf.createStatement(vf.createIRI("http://Eve"), vf.createIRI("http://helps"), vf.createIRI("http://Kevin")), vf.createStatement(vf.createIRI("http://Bob"), vf.createIRI("http://worksAt"), vf.createIRI("http://Chipotle")), vf.createStatement(vf.createIRI("http://Charlie"), vf.createIRI("http://worksAt"), vf.createIRI("http://Chipotle")), vf.createStatement(vf.createIRI("http://Eve"), vf.createIRI("http://worksAt"), vf.createIRI("http://Chipotle")), vf.createStatement(vf.createIRI("http://David"), vf.createIRI("http://worksAt"), vf.createIRI("http://Chipotle")));
    // The expected results of the SPARQL query once the PCJ has been computed.
    final Set<BindingSet> expected = new HashSet<>();
    MapBindingSet bs = new MapBindingSet();
    bs.addBinding("x", vf.createIRI("http://Bob"));
    expected.add(bs);
    bs = new MapBindingSet();
    bs.addBinding("x", vf.createIRI("http://Charlie"));
    expected.add(bs);
    // Load the historic data into Rya.
    final SailRepositoryConnection ryaConn = super.getRyaSailRepository().getConnection();
    for (final Statement triple : historicTriples) {
        ryaConn.add(triple);
    }
    ryaConn.close();
    // Create the PCJ table.
    final Connector accumuloConn = super.getAccumuloConnector();
    final PrecomputedJoinStorage pcjStorage = new AccumuloPcjStorage(accumuloConn, getRyaInstanceName());
    final String pcjId = pcjStorage.createPcj(sparql);
    try (FluoClient fluoClient = FluoFactory.newClient(super.getFluoConfiguration())) {
        // Tell the Fluo app to maintain the PCJ.
        new CreateFluoPcj().withRyaIntegration(pcjId, pcjStorage, fluoClient, accumuloConn, getRyaInstanceName());
        // Verify the end results of the query match the expected results.
        super.getMiniFluo().waitForObservers();
        final Set<BindingSet> results = new HashSet<>();
        try (CloseableIterator<BindingSet> resultsIt = pcjStorage.listResults(pcjId)) {
            while (resultsIt.hasNext()) {
                results.add(resultsIt.next());
            }
        }
        assertEquals(expected, results);
    }
}
Also used : BindingSet(org.eclipse.rdf4j.query.BindingSet) MapBindingSet(org.eclipse.rdf4j.query.impl.MapBindingSet) Connector(org.apache.accumulo.core.client.Connector) FluoClient(org.apache.fluo.api.client.FluoClient) AccumuloPcjStorage(org.apache.rya.indexing.pcj.storage.accumulo.AccumuloPcjStorage) RyaStatement(org.apache.rya.api.domain.RyaStatement) Statement(org.eclipse.rdf4j.model.Statement) CreateFluoPcj(org.apache.rya.indexing.pcj.fluo.api.CreateFluoPcj) ValueFactory(org.eclipse.rdf4j.model.ValueFactory) SimpleValueFactory(org.eclipse.rdf4j.model.impl.SimpleValueFactory) SailRepositoryConnection(org.eclipse.rdf4j.repository.sail.SailRepositoryConnection) PrecomputedJoinStorage(org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage) MapBindingSet(org.eclipse.rdf4j.query.impl.MapBindingSet) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 64 with SailRepositoryConnection

use of org.eclipse.rdf4j.repository.sail.SailRepositoryConnection in project AJAN-service by aantakli.

the class SPARQLUtil method queryModel.

public static List<BindingSet> queryModel(final Model model, final ParsedTupleQuery query) throws QueryEvaluationException {
    SailRepository repo = createRepository(model);
    List<BindingSet> resultModel;
    try (SailRepositoryConnection conn = repo.getConnection()) {
        conn.begin();
        SailQueryPreparer preparer = new SailQueryPreparer(conn, false);
        TupleQueryResult results = preparer.prepare(query).evaluate();
        resultModel = getBindingSetList(results);
        conn.commit();
    }
    repo.shutDown();
    return resultModel;
}
Also used : BindingSet(org.eclipse.rdf4j.query.BindingSet) SailRepository(org.eclipse.rdf4j.repository.sail.SailRepository) SailQueryPreparer(org.eclipse.rdf4j.repository.sail.SailQueryPreparer) SailRepositoryConnection(org.eclipse.rdf4j.repository.sail.SailRepositoryConnection) TupleQueryResult(org.eclipse.rdf4j.query.TupleQueryResult)

Example 65 with SailRepositoryConnection

use of org.eclipse.rdf4j.repository.sail.SailRepositoryConnection in project AJAN-service by aantakli.

the class SPARQLUtil method queryModel.

public static Model queryModel(final Model model, final ParsedGraphQuery query) throws QueryEvaluationException {
    SailRepository repo = createRepository(model);
    Model resultModel;
    try (SailRepositoryConnection conn = repo.getConnection()) {
        conn.begin();
        SailQueryPreparer preparer = new SailQueryPreparer(conn, false);
        GraphQueryResult results = preparer.prepare(query).evaluate();
        resultModel = QueryResults.asModel(results);
        conn.commit();
    }
    repo.shutDown();
    return resultModel;
}
Also used : SailRepository(org.eclipse.rdf4j.repository.sail.SailRepository) Model(org.eclipse.rdf4j.model.Model) SailQueryPreparer(org.eclipse.rdf4j.repository.sail.SailQueryPreparer) SailRepositoryConnection(org.eclipse.rdf4j.repository.sail.SailRepositoryConnection) GraphQueryResult(org.eclipse.rdf4j.query.GraphQueryResult)

Aggregations

SailRepositoryConnection (org.eclipse.rdf4j.repository.sail.SailRepositoryConnection)65 SailRepository (org.eclipse.rdf4j.repository.sail.SailRepository)44 Sail (org.eclipse.rdf4j.sail.Sail)37 Test (org.junit.Test)35 BindingSet (org.eclipse.rdf4j.query.BindingSet)24 HashSet (java.util.HashSet)17 MapBindingSet (org.eclipse.rdf4j.query.impl.MapBindingSet)15 TupleQueryResult (org.eclipse.rdf4j.query.TupleQueryResult)13 Statement (org.eclipse.rdf4j.model.Statement)11 ValueFactory (org.eclipse.rdf4j.model.ValueFactory)9 ArrayList (java.util.ArrayList)8 Connector (org.apache.accumulo.core.client.Connector)8 Configuration (org.apache.hadoop.conf.Configuration)8 PrecomputedJoinStorage (org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage)8 RepositoryException (org.eclipse.rdf4j.repository.RepositoryException)8 RyaStatement (org.apache.rya.api.domain.RyaStatement)7 AccumuloPcjStorage (org.apache.rya.indexing.pcj.storage.accumulo.AccumuloPcjStorage)7 StatefulMongoDBRdfConfiguration (org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration)7 SimpleValueFactory (org.eclipse.rdf4j.model.impl.SimpleValueFactory)7 RyaIRI (org.apache.rya.api.domain.RyaIRI)6