Search in sources :

Example 46 with SailRepositoryConnection

use of org.eclipse.rdf4j.repository.sail.SailRepositoryConnection in project incubator-rya by apache.

the class GeowaveDirectExample method main.

public static void main(final String[] args) throws Exception {
    final Configuration conf = getConf();
    conf.set(PrecomputedJoinIndexerConfig.PCJ_STORAGE_TYPE, PrecomputedJoinStorageType.ACCUMULO.name());
    conf.setBoolean(ConfigUtils.DISPLAY_QUERY_PLAN, PRINT_QUERIES);
    conf.setBoolean(OptionalConfigUtils.USE_GEO, true);
    conf.setEnum(OptionalConfigUtils.GEO_INDEXER_TYPE, GeoIndexerType.GEO_WAVE);
    log.info("Creating the tables as root.");
    SailRepository repository = null;
    SailRepositoryConnection conn = null;
    try {
        log.info("Connecting to Geo Sail Repository.");
        final Sail extSail = GeoRyaSailFactory.getInstance(conf);
        repository = new SailRepository(extSail);
        conn = repository.getConnection();
        final long start = System.currentTimeMillis();
        log.info("Running SPARQL Example: Add Point and Geo Search with PCJ");
        testAddPointAndWithinSearchWithPCJ(conn);
        log.info("Running SPARQL Example: Temporal, Freetext, and Geo Search");
        testTemporalFreeGeoSearch(conn);
        log.info("Running SPARQL Example: Geo, Freetext, and PCJ Search");
        testGeoFreetextWithPCJSearch(conn);
        log.info("Running SPARQL Example: Delete Geo Data");
        testDeleteGeoData(conn);
        log.info("TIME: " + (System.currentTimeMillis() - start) / 1000.);
    } finally {
        log.info("Shutting down");
        closeQuietly(conn);
        closeQuietly(repository);
    }
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) AccumuloIndexingConfiguration(org.apache.rya.indexing.accumulo.AccumuloIndexingConfiguration) SailRepository(org.eclipse.rdf4j.repository.sail.SailRepository) Sail(org.eclipse.rdf4j.sail.Sail) SailRepositoryConnection(org.eclipse.rdf4j.repository.sail.SailRepositoryConnection)

Example 47 with SailRepositoryConnection

use of org.eclipse.rdf4j.repository.sail.SailRepositoryConnection in project incubator-rya by apache.

the class MongoSpinIT method executeQuery.

private Set<BindingSet> executeQuery(final URL queryFile) throws Exception {
    final SailRepositoryConnection conn = repository.getConnection();
    try (final InputStream queryIS = queryFile.openStream();
        final BufferedReader br = new BufferedReader(new InputStreamReader(queryIS, StandardCharsets.UTF_8))) {
        final String query = br.lines().collect(Collectors.joining("\n"));
        final TupleQuery tupleQuery = conn.prepareTupleQuery(QueryLanguage.SPARQL, query);
        final Set<BindingSet> solutions = new HashSet<>();
        tupleQuery.evaluate(new AbstractTupleQueryResultHandler() {

            @Override
            public void handleSolution(final BindingSet bindingSet) throws TupleQueryResultHandlerException {
                solutions.add(bindingSet);
            }
        });
        return solutions;
    } finally {
        closeQuietly(conn);
    }
}
Also used : ListBindingSet(org.eclipse.rdf4j.query.impl.ListBindingSet) BindingSet(org.eclipse.rdf4j.query.BindingSet) AbstractTupleQueryResultHandler(org.eclipse.rdf4j.query.AbstractTupleQueryResultHandler) InputStreamReader(java.io.InputStreamReader) TupleQueryResultHandlerException(org.eclipse.rdf4j.query.TupleQueryResultHandlerException) InputStream(java.io.InputStream) BufferedReader(java.io.BufferedReader) TupleQuery(org.eclipse.rdf4j.query.TupleQuery) SailRepositoryConnection(org.eclipse.rdf4j.repository.sail.SailRepositoryConnection) HashSet(java.util.HashSet)

Example 48 with SailRepositoryConnection

use of org.eclipse.rdf4j.repository.sail.SailRepositoryConnection in project incubator-rya by apache.

the class MongoSpinIT method insertDataFile.

private void insertDataFile(final URL dataFile, final String defaultNamespace) throws Exception {
    final RDFFormat format = Rio.getParserFormatForFileName(dataFile.getFile()).get();
    final SailRepositoryConnection conn = repository.getConnection();
    try {
        conn.add(dataFile, defaultNamespace, format);
    } finally {
        closeQuietly(conn);
    }
}
Also used : SailRepositoryConnection(org.eclipse.rdf4j.repository.sail.SailRepositoryConnection) RDFFormat(org.eclipse.rdf4j.rio.RDFFormat)

Example 49 with SailRepositoryConnection

use of org.eclipse.rdf4j.repository.sail.SailRepositoryConnection in project incubator-rya by apache.

the class AccumuloLoadStatementsFile method loadStatements.

@Override
public void loadStatements(final String ryaInstanceName, final Path statementsFile, final RDFFormat format) throws InstanceDoesNotExistException, RyaClientException {
    requireNonNull(ryaInstanceName);
    requireNonNull(statementsFile);
    requireNonNull(format);
    // Ensure the Rya Instance exists.
    if (!instanceExists.exists(ryaInstanceName)) {
        throw new InstanceDoesNotExistException(String.format("There is no Rya instance named '%s'.", ryaInstanceName));
    }
    Sail sail = null;
    SailRepository sailRepo = null;
    SailRepositoryConnection sailRepoConn = null;
    try {
        // Get a Sail object that is connected to the Rya instance.
        final AccumuloRdfConfiguration ryaConf = getAccumuloConnectionDetails().buildAccumuloRdfConfiguration(ryaInstanceName);
        // RYA-327 should address this hardcoded value.
        ryaConf.setFlush(false);
        sail = RyaSailFactory.getInstance(ryaConf);
        // Load the file.
        sailRepo = new SailRepository(sail);
        sailRepoConn = sailRepo.getConnection();
        sailRepoConn.add(statementsFile.toFile(), null, format);
    } catch (final SailException | AccumuloException | AccumuloSecurityException | RyaDAOException | InferenceEngineException e) {
        log.warn("Exception while loading:", e);
        throw new RyaClientException("A problem connecting to the Rya instance named '" + ryaInstanceName + "' has caused the load to fail.", e);
    } catch (final RepositoryException | RDFParseException | UnsupportedRDFormatException | IOException e) {
        log.warn("Exception while loading:", e);
        throw new RyaClientException("A problem processing the RDF file has caused the load into Rya instance named " + ryaInstanceName + "to fail.", e);
    } finally {
        // Shut it all down.
        if (sailRepoConn != null) {
            try {
                sailRepoConn.close();
            } catch (final RepositoryException e) {
                log.warn("Couldn't close the SailRepoConnection that is attached to the Rya instance.", e);
            }
        }
        if (sailRepo != null) {
            try {
                sailRepo.shutDown();
            } catch (final RepositoryException e) {
                log.warn("Couldn't shut down the SailRepository that is attached to the Rya instance.", e);
            }
        }
        if (sail != null) {
            try {
                sail.shutDown();
            } catch (final SailException e) {
                log.warn("Couldn't shut down the Sail that is attached to the Rya instance.", e);
            }
        }
    }
}
Also used : AccumuloException(org.apache.accumulo.core.client.AccumuloException) UnsupportedRDFormatException(org.eclipse.rdf4j.rio.UnsupportedRDFormatException) RyaClientException(org.apache.rya.api.client.RyaClientException) SailRepository(org.eclipse.rdf4j.repository.sail.SailRepository) InferenceEngineException(org.apache.rya.rdftriplestore.inference.InferenceEngineException) RepositoryException(org.eclipse.rdf4j.repository.RepositoryException) InstanceDoesNotExistException(org.apache.rya.api.client.InstanceDoesNotExistException) SailException(org.eclipse.rdf4j.sail.SailException) IOException(java.io.IOException) SailRepositoryConnection(org.eclipse.rdf4j.repository.sail.SailRepositoryConnection) AccumuloRdfConfiguration(org.apache.rya.accumulo.AccumuloRdfConfiguration) Sail(org.eclipse.rdf4j.sail.Sail) RyaDAOException(org.apache.rya.api.persist.RyaDAOException) AccumuloSecurityException(org.apache.accumulo.core.client.AccumuloSecurityException) RDFParseException(org.eclipse.rdf4j.rio.RDFParseException)

Example 50 with SailRepositoryConnection

use of org.eclipse.rdf4j.repository.sail.SailRepositoryConnection in project incubator-rya by apache.

the class MongoBatchUpdatePCJ method updatePCJResults.

private void updatePCJResults(final String ryaInstanceName, final String pcjId, final MongoClient client) throws InstanceDoesNotExistException, PCJDoesNotExistException, RyaClientException {
    // Things that have to be closed before we exit.
    Sail sail = null;
    SailConnection sailConn = null;
    try (final PrecomputedJoinStorage pcjStorage = new MongoPcjStorage(client, ryaInstanceName)) {
        // Create an instance of Sail backed by the Rya instance.
        sail = connectToRya(ryaInstanceName);
        final SailRepository sailRepo = new SailRepository(sail);
        final SailRepositoryConnection sailRepoConn = sailRepo.getConnection();
        // Purge the old results from the PCJ.
        try {
            pcjStorage.purge(pcjId);
        } catch (final PCJStorageException e) {
            throw new RyaClientException("Could not batch update PCJ with ID '" + pcjId + "' because the old " + "results could not be purged from it.", e);
        }
        // Parse the PCJ's SPARQL query.
        final PcjMetadata metadata = pcjStorage.getPcjMetadata(pcjId);
        final String sparql = metadata.getSparql();
        sailConn = sail.getConnection();
        final TupleQuery tupleQuery = sailRepoConn.prepareTupleQuery(QueryLanguage.SPARQL, sparql);
        // Execute the query.
        final List<VisibilityBindingSet> batch = new ArrayList<>(1000);
        tupleQuery.evaluate(new AbstractTupleQueryResultHandler() {

            @Override
            public void handleSolution(final BindingSet bindingSet) throws TupleQueryResultHandlerException {
                final VisibilityBindingSet result = new VisibilityBindingSet(bindingSet, "");
                log.warn("Visibility information on the binding set is lost during a batch update." + "  This can create data leaks.");
                batch.add(result);
                if (batch.size() == 1000) {
                    try {
                        pcjStorage.addResults(pcjId, batch);
                    } catch (final PCJStorageException e) {
                        throw new TupleQueryResultHandlerException("Fail to batch load new results into the PCJ with ID '" + pcjId + "'.", e);
                    }
                    batch.clear();
                }
            }
        });
        if (!batch.isEmpty()) {
            pcjStorage.addResults(pcjId, batch);
            batch.clear();
        }
    } catch (final MalformedQueryException | PCJStorageException | SailException | QueryEvaluationException | RepositoryException | TupleQueryResultHandlerException e) {
        throw new RyaClientException("Fail to batch load new results into the PCJ with ID '" + pcjId + "'.", e);
    } finally {
        if (sailConn != null) {
            try {
                sailConn.close();
            } catch (final SailException e) {
                log.warn(e.getMessage(), e);
            }
        }
        if (sail != null) {
            try {
                sail.shutDown();
            } catch (final SailException e) {
                log.warn(e.getMessage(), e);
            }
        }
    }
}
Also used : VisibilityBindingSet(org.apache.rya.api.model.VisibilityBindingSet) BindingSet(org.eclipse.rdf4j.query.BindingSet) RyaClientException(org.apache.rya.api.client.RyaClientException) VisibilityBindingSet(org.apache.rya.api.model.VisibilityBindingSet) TupleQueryResultHandlerException(org.eclipse.rdf4j.query.TupleQueryResultHandlerException) SailRepository(org.eclipse.rdf4j.repository.sail.SailRepository) ArrayList(java.util.ArrayList) TupleQuery(org.eclipse.rdf4j.query.TupleQuery) RepositoryException(org.eclipse.rdf4j.repository.RepositoryException) RyaDetailsRepositoryException(org.apache.rya.api.instance.RyaDetailsRepository.RyaDetailsRepositoryException) SailException(org.eclipse.rdf4j.sail.SailException) SailRepositoryConnection(org.eclipse.rdf4j.repository.sail.SailRepositoryConnection) MongoPcjStorage(org.apache.rya.indexing.pcj.storage.mongo.MongoPcjStorage) AbstractTupleQueryResultHandler(org.eclipse.rdf4j.query.AbstractTupleQueryResultHandler) SailConnection(org.eclipse.rdf4j.sail.SailConnection) QueryEvaluationException(org.eclipse.rdf4j.query.QueryEvaluationException) Sail(org.eclipse.rdf4j.sail.Sail) PrecomputedJoinStorage(org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage) MalformedQueryException(org.eclipse.rdf4j.query.MalformedQueryException) PcjMetadata(org.apache.rya.indexing.pcj.storage.PcjMetadata) PCJStorageException(org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage.PCJStorageException)

Aggregations

SailRepositoryConnection (org.eclipse.rdf4j.repository.sail.SailRepositoryConnection)65 SailRepository (org.eclipse.rdf4j.repository.sail.SailRepository)44 Sail (org.eclipse.rdf4j.sail.Sail)37 Test (org.junit.Test)35 BindingSet (org.eclipse.rdf4j.query.BindingSet)24 HashSet (java.util.HashSet)17 MapBindingSet (org.eclipse.rdf4j.query.impl.MapBindingSet)15 TupleQueryResult (org.eclipse.rdf4j.query.TupleQueryResult)13 Statement (org.eclipse.rdf4j.model.Statement)11 ValueFactory (org.eclipse.rdf4j.model.ValueFactory)9 ArrayList (java.util.ArrayList)8 Connector (org.apache.accumulo.core.client.Connector)8 Configuration (org.apache.hadoop.conf.Configuration)8 PrecomputedJoinStorage (org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage)8 RepositoryException (org.eclipse.rdf4j.repository.RepositoryException)8 RyaStatement (org.apache.rya.api.domain.RyaStatement)7 AccumuloPcjStorage (org.apache.rya.indexing.pcj.storage.accumulo.AccumuloPcjStorage)7 StatefulMongoDBRdfConfiguration (org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration)7 SimpleValueFactory (org.eclipse.rdf4j.model.impl.SimpleValueFactory)7 RyaIRI (org.apache.rya.api.domain.RyaIRI)6