Search in sources :

Example 31 with Sail

use of org.eclipse.rdf4j.sail.Sail in project incubator-rya by apache.

the class RyaDirectExample method main.

public static void main(final String[] args) throws Exception {
    final Configuration conf = getConf();
    conf.set(PrecomputedJoinIndexerConfig.PCJ_STORAGE_TYPE, PrecomputedJoinStorageType.ACCUMULO.name());
    conf.setBoolean(ConfigUtils.DISPLAY_QUERY_PLAN, PRINT_QUERIES);
    log.info("Creating the tables as root.");
    SailRepository repository = null;
    SailRepositoryConnection conn = null;
    try {
        log.info("Creating PCJ Tables");
        createPCJ(conf);
        log.info("Connecting to Indexing Sail Repository.");
        final Sail extSail = RyaSailFactory.getInstance(conf);
        repository = new SailRepository(extSail);
        conn = repository.getConnection();
        final long start = System.currentTimeMillis();
        log.info("Running SPARQL Example: Add and Delete");
        testAddAndDelete(conn);
        log.info("Running SAIL/SPARQL Example: PCJ Search");
        testPCJSearch(conn);
        log.info("Running SAIL/SPARQL Example: Add and Temporal Search");
        testAddAndTemporalSearchWithPCJ(conn);
        log.info("Running SAIL/SPARQL Example: Add and Free Text Search with PCJ");
        testAddAndFreeTextSearchWithPCJ(conn);
        // log.info("Running SPARQL Example: Add Point and Geo Search with PCJ");
        // //			testAddPointAndWithinSearchWithPCJ(conn);
        // log.info("Running SPARQL Example: Temporal, Freetext, and Geo Search");
        // testTemporalFreeGeoSearch(conn);
        // log.info("Running SPARQL Example: Geo, Freetext, and PCJ Search");
        // testGeoFreetextWithPCJSearch(conn);
        log.info("Running SPARQL Example: Delete Temporal Data");
        testDeleteTemporalData(conn);
        log.info("Running SPARQL Example: Delete Free Text Data");
        testDeleteFreeTextData(conn);
        // log.info("Running SPARQL Example: Delete Geo Data");
        // testDeleteGeoData(conn);
        log.info("TIME: " + (System.currentTimeMillis() - start) / 1000.);
    } finally {
        log.info("Shutting down");
        closeQuietly(conn);
        closeQuietly(repository);
    }
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) AccumuloIndexingConfiguration(org.apache.rya.indexing.accumulo.AccumuloIndexingConfiguration) AccumuloRdfConfiguration(org.apache.rya.accumulo.AccumuloRdfConfiguration) SailRepository(org.eclipse.rdf4j.repository.sail.SailRepository) Sail(org.eclipse.rdf4j.sail.Sail) SailRepositoryConnection(org.eclipse.rdf4j.repository.sail.SailRepositoryConnection)

Example 32 with Sail

use of org.eclipse.rdf4j.sail.Sail in project incubator-rya by apache.

the class RyaSinkTask method start.

@Override
public void start(final Map<String, String> props) throws ConnectException {
    requireNonNull(props);
    // Ensure the configured Rya Instance is installed within the configured database.
    checkRyaInstanceExists(props);
    // Create the Sail object that is connected to the Rya Instance.
    final Sail sail = makeSail(props);
    sailRepo = new SailRepository(sail);
    conn = sailRepo.getConnection();
}
Also used : SailRepository(org.eclipse.rdf4j.repository.sail.SailRepository) Sail(org.eclipse.rdf4j.sail.Sail)

Example 33 with Sail

use of org.eclipse.rdf4j.sail.Sail in project incubator-rya by apache.

the class RyaSinkTaskTest method flushBetweenPuts.

@Test
public void flushBetweenPuts() {
    // Create the Statements that will be put by the task.
    final ValueFactory vf = SimpleValueFactory.getInstance();
    final Set<Statement> batch1 = Sets.newHashSet(vf.createStatement(vf.createIRI("urn:Alice"), vf.createIRI("urn:WorksAt"), vf.createIRI("urn:Taco Shop"), vf.createIRI("urn:graph1")), vf.createStatement(vf.createIRI("urn:Bob"), vf.createIRI("urn:TalksTo"), vf.createIRI("urn:Charlie"), vf.createIRI("urn:graph2")));
    final Set<Statement> batch2 = Sets.newHashSet(vf.createStatement(vf.createIRI("urn:Eve"), vf.createIRI("urn:ListensTo"), vf.createIRI("urn:Alice"), vf.createIRI("urn:graph1")));
    // Create the task that will be tested.
    final InMemoryRyaSinkTask task = new InMemoryRyaSinkTask();
    // Setup the properties that will be used to configure the task. We don't actually need to set anything
    // here since we're always returning true for ryaInstanceExists(...) and use an in memory RDF store.
    final Map<String, String> props = new HashMap<>();
    try {
        // Start the task.
        task.start(props);
        // Put the statements with flushes between them.
        task.put(Collections.singleton(new SinkRecord("topic", 1, null, "key", null, batch1, 0)));
        task.flush(new HashMap<>());
        task.put(Collections.singleton(new SinkRecord("topic", 1, null, "key", null, batch2, 1)));
        task.flush(new HashMap<>());
        // Fetch the stored Statements to show they match the original set.
        final Set<Statement> fetched = new HashSet<>();
        final Sail sail = task.makeSail(props);
        try (SailConnection conn = sail.getConnection();
            CloseableIteration<? extends Statement, SailException> it = conn.getStatements(null, null, null, false)) {
            while (it.hasNext()) {
                fetched.add(it.next());
            }
        }
        assertEquals(Sets.union(batch1, batch2), fetched);
    } finally {
        // Stop the task.
        task.stop();
    }
}
Also used : HashMap(java.util.HashMap) Statement(org.eclipse.rdf4j.model.Statement) ValueFactory(org.eclipse.rdf4j.model.ValueFactory) SimpleValueFactory(org.eclipse.rdf4j.model.impl.SimpleValueFactory) SailException(org.eclipse.rdf4j.sail.SailException) SinkRecord(org.apache.kafka.connect.sink.SinkRecord) SailConnection(org.eclipse.rdf4j.sail.SailConnection) Sail(org.eclipse.rdf4j.sail.Sail) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 34 with Sail

use of org.eclipse.rdf4j.sail.Sail in project incubator-rya by apache.

the class InferenceExamples method main.

public static void main(final String[] args) throws Exception {
    if (IS_DETAILED_LOGGING_ENABLED) {
        setupLogging();
    }
    final Configuration conf = getConf();
    conf.setBoolean(ConfigUtils.DISPLAY_QUERY_PLAN, PRINT_QUERIES);
    SailRepository repository = null;
    SailRepositoryConnection conn = null;
    try {
        log.info("Connecting to Indexing Sail Repository.");
        final Sail sail = RyaSailFactory.getInstance(conf);
        repository = new SailRepository(sail);
        conn = repository.getConnection();
        final long start = System.currentTimeMillis();
        testInfer(conn, sail);
        testPropertyChainInference(conn, sail);
        testPropertyChainInferenceAltRepresentation(conn, sail);
        testSomeValuesFromInference(conn, sail);
        testAllValuesFromInference(conn, sail);
        testIntersectionOfInference(conn, sail);
        testOneOfInference(conn, sail);
        log.info("TIME: " + (System.currentTimeMillis() - start) / 1000.);
    } finally {
        log.info("Shutting down");
        closeQuietly(conn);
        closeQuietly(repository);
    }
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) MongoIndexingConfiguration(org.apache.rya.indexing.mongodb.MongoIndexingConfiguration) SailRepository(org.eclipse.rdf4j.repository.sail.SailRepository) Sail(org.eclipse.rdf4j.sail.Sail) SailRepositoryConnection(org.eclipse.rdf4j.repository.sail.SailRepositoryConnection)

Example 35 with Sail

use of org.eclipse.rdf4j.sail.Sail in project incubator-rya by apache.

the class RyaSinkTaskTest method multipleRecords.

@Test
public void multipleRecords() {
    // Create the Statements that will be put by the task.
    final ValueFactory vf = SimpleValueFactory.getInstance();
    final Set<Statement> batch1 = Sets.newHashSet(vf.createStatement(vf.createIRI("urn:Alice"), vf.createIRI("urn:WorksAt"), vf.createIRI("urn:Taco Shop"), vf.createIRI("urn:graph1")), vf.createStatement(vf.createIRI("urn:Bob"), vf.createIRI("urn:TalksTo"), vf.createIRI("urn:Charlie"), vf.createIRI("urn:graph2")));
    final Set<Statement> batch2 = Sets.newHashSet(vf.createStatement(vf.createIRI("urn:Eve"), vf.createIRI("urn:ListensTo"), vf.createIRI("urn:Alice"), vf.createIRI("urn:graph1")));
    // Create the task that will be tested.
    final InMemoryRyaSinkTask task = new InMemoryRyaSinkTask();
    // Setup the properties that will be used to configure the task. We don't actually need to set anything
    // here since we're always returning true for ryaInstanceExists(...) and use an in memory RDF store.
    final Map<String, String> props = new HashMap<>();
    try {
        // Start the task.
        task.start(props);
        // Put the statements as SinkRecords.
        final Collection<SinkRecord> records = Sets.newHashSet(new SinkRecord("topic", 1, null, "key", null, batch1, 0), new SinkRecord("topic", 1, null, "key", null, batch2, 1));
        task.put(records);
        // Flush the statements.
        task.flush(new HashMap<>());
        // Fetch the stored Statements to show they match the original set.
        final Set<Statement> fetched = new HashSet<>();
        final Sail sail = task.makeSail(props);
        try (SailConnection conn = sail.getConnection();
            CloseableIteration<? extends Statement, SailException> it = conn.getStatements(null, null, null, false)) {
            while (it.hasNext()) {
                fetched.add(it.next());
            }
        }
        assertEquals(Sets.union(batch1, batch2), fetched);
    } finally {
        // Stop the task.
        task.stop();
    }
}
Also used : HashMap(java.util.HashMap) Statement(org.eclipse.rdf4j.model.Statement) ValueFactory(org.eclipse.rdf4j.model.ValueFactory) SimpleValueFactory(org.eclipse.rdf4j.model.impl.SimpleValueFactory) SailException(org.eclipse.rdf4j.sail.SailException) SinkRecord(org.apache.kafka.connect.sink.SinkRecord) SailConnection(org.eclipse.rdf4j.sail.SailConnection) Sail(org.eclipse.rdf4j.sail.Sail) HashSet(java.util.HashSet) Test(org.junit.Test)

Aggregations

Sail (org.eclipse.rdf4j.sail.Sail)68 SailRepository (org.eclipse.rdf4j.repository.sail.SailRepository)49 SailRepositoryConnection (org.eclipse.rdf4j.repository.sail.SailRepositoryConnection)37 Test (org.junit.Test)33 AccumuloRdfConfiguration (org.apache.rya.accumulo.AccumuloRdfConfiguration)15 BindingSet (org.eclipse.rdf4j.query.BindingSet)13 TupleQueryResult (org.eclipse.rdf4j.query.TupleQueryResult)13 SailException (org.eclipse.rdf4j.sail.SailException)13 HashSet (java.util.HashSet)11 ValueFactory (org.eclipse.rdf4j.model.ValueFactory)11 SailConnection (org.eclipse.rdf4j.sail.SailConnection)11 RyaClient (org.apache.rya.api.client.RyaClient)9 ArrayList (java.util.ArrayList)8 AccumuloSecurityException (org.apache.accumulo.core.client.AccumuloSecurityException)8 Configuration (org.apache.hadoop.conf.Configuration)8 RyaClientException (org.apache.rya.api.client.RyaClientException)8 MongoDBRdfConfiguration (org.apache.rya.mongodb.MongoDBRdfConfiguration)7 RyaSailRepository (org.apache.rya.rdftriplestore.RyaSailRepository)7 MapBindingSet (org.eclipse.rdf4j.query.impl.MapBindingSet)7 RepositoryException (org.eclipse.rdf4j.repository.RepositoryException)7