Search in sources :

Example 11 with InferenceEngineException

use of org.apache.rya.rdftriplestore.inference.InferenceEngineException in project incubator-rya by apache.

the class MergeDriverClient method main.

public static void main(final String[] args) throws ParseException, MergeConfigurationException, UnknownHostException, MergerException, java.text.ParseException, SailException, AccumuloException, AccumuloSecurityException, InferenceEngineException, RepositoryException, MalformedQueryException, UpdateExecutionException {
    final String log4jConfiguration = System.getProperties().getProperty("log4j.configuration");
    if (StringUtils.isNotBlank(log4jConfiguration)) {
        final String parsedConfiguration = PathUtils.clean(StringUtils.removeStart(log4jConfiguration, "file:"));
        final File configFile = new File(parsedConfiguration);
        if (configFile.exists()) {
            DOMConfigurator.configure(parsedConfiguration);
        } else {
            BasicConfigurator.configure();
        }
    }
    final MergeConfigurationCLI config = new MergeConfigurationCLI(args);
    try {
        configuration = config.createConfiguration();
    } catch (final MergeConfigurationException e) {
        LOG.error("Configuration failed.", e);
    }
    final boolean useTimeSync = configuration.getUseNtpServer();
    Optional<Long> offset = Optional.absent();
    if (useTimeSync) {
        final String tomcat = configuration.getChildTomcatUrl();
        final String ntpHost = configuration.getNtpServerHost();
        try {
            offset = Optional.<Long>fromNullable(TimeUtils.getNtpServerAndMachineTimeDifference(ntpHost, tomcat));
        } catch (final IOException e) {
            LOG.error("Unable to get time difference between time server: " + ntpHost + " and the server: " + tomcat, e);
        }
    }
    final StatementStoreFactory storeFactory = new StatementStoreFactory(configuration);
    try {
        final RyaStatementStore parentStore = storeFactory.getParentStatementStore();
        final RyaStatementStore childStore = storeFactory.getChildStatementStore();
        LOG.info("Starting Merge Tool");
        if (configuration.getParentDBType() == ACCUMULO && configuration.getChildDBType() == ACCUMULO) {
            final AccumuloRyaStatementStore childAStore = (AccumuloRyaStatementStore) childStore;
            final AccumuloRyaStatementStore parentAStore = (AccumuloRyaStatementStore) parentStore;
        // do map reduce merging.
        // TODO: Run Merger
        } else {
            if (configuration.getMergePolicy() == TIMESTAMP) {
                final TimestampPolicyMergeConfiguration timeConfig = (TimestampPolicyMergeConfiguration) configuration;
                final Long timeOffset;
                if (offset.isPresent()) {
                    timeOffset = offset.get();
                } else {
                    timeOffset = 0L;
                }
                final MemoryTimeMerger merger = new MemoryTimeMerger(parentStore, childStore, new VisibilityStatementMerger(), timeConfig.getToolStartTime(), configuration.getParentRyaInstanceName(), timeOffset);
                merger.runJob();
            }
        }
    } catch (final Exception e) {
        LOG.error("Something went wrong creating a Rya Statement Store connection.", e);
    }
    Thread.setDefaultUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() {

        @Override
        public void uncaughtException(final Thread thread, final Throwable throwable) {
            LOG.error("Uncaught exception in " + thread.getName(), throwable);
        }
    });
    LOG.info("Finished running Merge Tool");
    System.exit(1);
}
Also used : StatementStoreFactory(org.apache.rya.export.client.merge.StatementStoreFactory) TimestampPolicyMergeConfiguration(org.apache.rya.export.api.conf.policy.TimestampPolicyMergeConfiguration) VisibilityStatementMerger(org.apache.rya.export.client.merge.VisibilityStatementMerger) MemoryTimeMerger(org.apache.rya.export.client.merge.MemoryTimeMerger) IOException(java.io.IOException) MergeConfigurationCLI(org.apache.rya.export.client.conf.MergeConfigurationCLI) InferenceEngineException(org.apache.rya.rdftriplestore.inference.InferenceEngineException) SailException(org.openrdf.sail.SailException) RepositoryException(org.openrdf.repository.RepositoryException) AccumuloSecurityException(org.apache.accumulo.core.client.AccumuloSecurityException) UpdateExecutionException(org.openrdf.query.UpdateExecutionException) MalformedQueryException(org.openrdf.query.MalformedQueryException) IOException(java.io.IOException) UnknownHostException(java.net.UnknownHostException) AccumuloException(org.apache.accumulo.core.client.AccumuloException) MergerException(org.apache.rya.export.api.MergerException) ParseException(org.apache.commons.cli.ParseException) MergeConfigurationException(org.apache.rya.export.api.conf.MergeConfigurationException) MergeConfigurationException(org.apache.rya.export.api.conf.MergeConfigurationException) AccumuloRyaStatementStore(org.apache.rya.export.accumulo.AccumuloRyaStatementStore) File(java.io.File) AccumuloRyaStatementStore(org.apache.rya.export.accumulo.AccumuloRyaStatementStore) RyaStatementStore(org.apache.rya.export.api.store.RyaStatementStore)

Example 12 with InferenceEngineException

use of org.apache.rya.rdftriplestore.inference.InferenceEngineException in project incubator-rya by apache.

the class AccumuloLoadStatementsFile method loadStatements.

@Override
public void loadStatements(final String ryaInstanceName, final Path statementsFile, final RDFFormat format) throws InstanceDoesNotExistException, RyaClientException {
    requireNonNull(ryaInstanceName);
    requireNonNull(statementsFile);
    requireNonNull(format);
    // Ensure the Rya Instance exists.
    if (!instanceExists.exists(ryaInstanceName)) {
        throw new InstanceDoesNotExistException(String.format("There is no Rya instance named '%s'.", ryaInstanceName));
    }
    Sail sail = null;
    SailRepository sailRepo = null;
    SailRepositoryConnection sailRepoConn = null;
    try {
        // Get a Sail object that is connected to the Rya instance.
        final AccumuloRdfConfiguration ryaConf = getAccumuloConnectionDetails().buildAccumuloRdfConfiguration(ryaInstanceName);
        // RYA-327 should address this hardcoded value.
        ryaConf.setFlush(false);
        sail = RyaSailFactory.getInstance(ryaConf);
        // Load the file.
        sailRepo = new SailRepository(sail);
        sailRepoConn = sailRepo.getConnection();
        sailRepoConn.add(statementsFile.toFile(), null, format);
    } catch (final SailException | AccumuloException | AccumuloSecurityException | RyaDAOException | InferenceEngineException e) {
        log.warn("Exception while loading:", e);
        throw new RyaClientException("A problem connecting to the Rya instance named '" + ryaInstanceName + "' has caused the load to fail.", e);
    } catch (final RepositoryException | RDFParseException | UnsupportedRDFormatException | IOException e) {
        log.warn("Exception while loading:", e);
        throw new RyaClientException("A problem processing the RDF file has caused the load into Rya instance named " + ryaInstanceName + "to fail.", e);
    } finally {
        // Shut it all down.
        if (sailRepoConn != null) {
            try {
                sailRepoConn.close();
            } catch (final RepositoryException e) {
                log.warn("Couldn't close the SailRepoConnection that is attached to the Rya instance.", e);
            }
        }
        if (sailRepo != null) {
            try {
                sailRepo.shutDown();
            } catch (final RepositoryException e) {
                log.warn("Couldn't shut down the SailRepository that is attached to the Rya instance.", e);
            }
        }
        if (sail != null) {
            try {
                sail.shutDown();
            } catch (final SailException e) {
                log.warn("Couldn't shut down the Sail that is attached to the Rya instance.", e);
            }
        }
    }
}
Also used : AccumuloException(org.apache.accumulo.core.client.AccumuloException) UnsupportedRDFormatException(org.openrdf.rio.UnsupportedRDFormatException) RyaClientException(org.apache.rya.api.client.RyaClientException) SailRepository(org.openrdf.repository.sail.SailRepository) InferenceEngineException(org.apache.rya.rdftriplestore.inference.InferenceEngineException) RepositoryException(org.openrdf.repository.RepositoryException) InstanceDoesNotExistException(org.apache.rya.api.client.InstanceDoesNotExistException) SailException(org.openrdf.sail.SailException) IOException(java.io.IOException) SailRepositoryConnection(org.openrdf.repository.sail.SailRepositoryConnection) AccumuloRdfConfiguration(org.apache.rya.accumulo.AccumuloRdfConfiguration) Sail(org.openrdf.sail.Sail) RyaDAOException(org.apache.rya.api.persist.RyaDAOException) AccumuloSecurityException(org.apache.accumulo.core.client.AccumuloSecurityException) RDFParseException(org.openrdf.rio.RDFParseException)

Aggregations

InferenceEngineException (org.apache.rya.rdftriplestore.inference.InferenceEngineException)12 AccumuloException (org.apache.accumulo.core.client.AccumuloException)10 AccumuloSecurityException (org.apache.accumulo.core.client.AccumuloSecurityException)10 RyaDAOException (org.apache.rya.api.persist.RyaDAOException)10 SailException (org.openrdf.sail.SailException)10 RyaClientException (org.apache.rya.api.client.RyaClientException)9 RepositoryException (org.openrdf.repository.RepositoryException)7 Sail (org.openrdf.sail.Sail)7 AccumuloRdfConfiguration (org.apache.rya.accumulo.AccumuloRdfConfiguration)6 InstanceDoesNotExistException (org.apache.rya.api.client.InstanceDoesNotExistException)6 SailRepository (org.openrdf.repository.sail.SailRepository)5 SailRepositoryConnection (org.openrdf.repository.sail.SailRepositoryConnection)5 IOException (java.io.IOException)4 MongoDBRdfConfiguration (org.apache.rya.mongodb.MongoDBRdfConfiguration)4 MalformedQueryException (org.openrdf.query.MalformedQueryException)4 ByteArrayOutputStream (java.io.ByteArrayOutputStream)2 DecimalFormat (java.text.DecimalFormat)2 Map (java.util.Map)2 Set (java.util.Set)2 RyaDetails (org.apache.rya.api.instance.RyaDetails)2