Search in sources :

Example 6 with StatefulMongoDBRdfConfiguration

use of org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration in project incubator-rya by apache.

the class GeoTemporalIndexerFactory method getIndexer.

/**
 * Creates and returns a {@link GeoTemporalIndexer}.
 * @param conf - The {@link Configuration} to base the {@link GeoTemporalIndexer} on.
 * @return The created {@link GeoTemporalIndexer}.
 */
public GeoTemporalIndexer getIndexer(final Configuration conf) {
    if (ConfigUtils.getUseMongo(conf)) {
        Preconditions.checkArgument(conf instanceof StatefulMongoDBRdfConfiguration, "The configuration provided must be a StatefulMongoDBRdfConfiguration, found: " + conf.getClass().getSimpleName());
        final StatefulMongoDBRdfConfiguration statefulConf = (StatefulMongoDBRdfConfiguration) conf;
        for (final MongoSecondaryIndex index : statefulConf.getAdditionalIndexers()) {
            if (index instanceof GeoTemporalIndexer) {
                return (GeoTemporalIndexer) index;
            }
        }
        throw new IllegalStateException("Geo Temporal Indexing is not turned on. Check configuration.");
    } else {
        // TODO: add Accumulo here.
        return null;
    }
}
Also used : StatefulMongoDBRdfConfiguration(org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration) MongoSecondaryIndex(org.apache.rya.mongodb.MongoSecondaryIndex)

Example 7 with StatefulMongoDBRdfConfiguration

use of org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration in project incubator-rya by apache.

the class RyaSailFactory method getMongoDAO.

/**
 * Connects to MongoDB and creates a MongoDBRyaDAO.
 * @param config - user configuration
 * @return - MongoDBRyaDAO with Indexers configured according to user's specification
 * @throws RyaDAOException if the DAO can't be initialized
 */
public static MongoDBRyaDAO getMongoDAO(MongoDBRdfConfiguration mongoConfig) throws RyaDAOException {
    // Create the MongoClient that will be used by the Sail object's components.
    final MongoClient client = createMongoClient(mongoConfig);
    // Add the Indexer and Optimizer names to the configuration object that are configured to be used.
    ConfigUtils.setIndexers(mongoConfig);
    // Populate the configuration using previously stored Rya Details if this instance uses them.
    try {
        final MongoRyaInstanceDetailsRepository ryaDetailsRepo = new MongoRyaInstanceDetailsRepository(client, mongoConfig.getRyaInstanceName());
        RyaDetailsToConfiguration.addRyaDetailsToConfiguration(ryaDetailsRepo.getRyaInstanceDetails(), mongoConfig);
    } catch (final RyaDetailsRepositoryException e) {
        LOG.info("Instance does not have a rya details collection, skipping.");
    }
    // Set the configuration to the stateful configuration that is used to pass the constructed objects around.
    final StatefulMongoDBRdfConfiguration statefulConfig = new StatefulMongoDBRdfConfiguration(mongoConfig, client);
    final List<MongoSecondaryIndex> indexers = statefulConfig.getInstances(AccumuloRdfConfiguration.CONF_ADDITIONAL_INDEXERS, MongoSecondaryIndex.class);
    statefulConfig.setIndexers(indexers);
    // Create the DAO that is able to interact with MongoDB.
    final MongoDBRyaDAO mongoDao = new MongoDBRyaDAO();
    mongoDao.setConf(statefulConfig);
    mongoDao.init();
    return mongoDao;
}
Also used : MongoClient(com.mongodb.MongoClient) MongoDBRyaDAO(org.apache.rya.mongodb.MongoDBRyaDAO) StatefulMongoDBRdfConfiguration(org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration) MongoRyaInstanceDetailsRepository(org.apache.rya.mongodb.instance.MongoRyaInstanceDetailsRepository) RyaDetailsRepositoryException(org.apache.rya.api.instance.RyaDetailsRepository.RyaDetailsRepositoryException) MongoSecondaryIndex(org.apache.rya.mongodb.MongoSecondaryIndex)

Example 8 with StatefulMongoDBRdfConfiguration

use of org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration in project incubator-rya by apache.

the class StatementStoreFactory method getBaseMongoStore.

private MongoRyaStatementStore getBaseMongoStore(final String hostname, final int port, final String ryaInstanceName) throws RyaDAOException {
    final MongoClient client = new MongoClient(hostname, port);
    final MongoDBRyaDAO dao = new MongoDBRyaDAO();
    dao.setConf(new StatefulMongoDBRdfConfiguration(MergeConfigHadoopAdapter.getMongoConfiguration(configuration), client));
    dao.init();
    return new MongoRyaStatementStore(client, ryaInstanceName, dao);
}
Also used : MongoClient(com.mongodb.MongoClient) MongoDBRyaDAO(org.apache.rya.mongodb.MongoDBRyaDAO) StatefulMongoDBRdfConfiguration(org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration) MongoRyaStatementStore(org.apache.rya.export.mongo.MongoRyaStatementStore) TimestampPolicyMongoRyaStatementStore(org.apache.rya.export.mongo.policy.TimestampPolicyMongoRyaStatementStore)

Example 9 with StatefulMongoDBRdfConfiguration

use of org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration in project incubator-rya by apache.

the class FilterFunctionOptimizer method init.

private synchronized void init() {
    if (!init) {
        if (ConfigUtils.getUseMongo(conf)) {
            StatefulMongoDBRdfConfiguration stateConf = (StatefulMongoDBRdfConfiguration) conf;
            for (final MongoSecondaryIndex indexer : stateConf.getAdditionalIndexers()) {
                if (indexer instanceof FreeTextIndexer) {
                    freeTextIndexer = (FreeTextIndexer) indexer;
                } else if (indexer instanceof TemporalIndexer) {
                    temporalIndexer = (TemporalIndexer) indexer;
                }
            }
        } else {
            freeTextIndexer = new AccumuloFreeTextIndexer();
            freeTextIndexer.setConf(conf);
            temporalIndexer = new AccumuloTemporalIndexer();
            temporalIndexer.setConf(conf);
        }
        init = true;
    }
}
Also used : StatefulMongoDBRdfConfiguration(org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration) AccumuloFreeTextIndexer(org.apache.rya.indexing.accumulo.freetext.AccumuloFreeTextIndexer) AccumuloTemporalIndexer(org.apache.rya.indexing.accumulo.temporal.AccumuloTemporalIndexer) AccumuloTemporalIndexer(org.apache.rya.indexing.accumulo.temporal.AccumuloTemporalIndexer) MongoSecondaryIndex(org.apache.rya.mongodb.MongoSecondaryIndex) AccumuloFreeTextIndexer(org.apache.rya.indexing.accumulo.freetext.AccumuloFreeTextIndexer)

Example 10 with StatefulMongoDBRdfConfiguration

use of org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration in project incubator-rya by apache.

the class GeoRyaSailFactory method getRyaSail.

private static Sail getRyaSail(final Configuration config) throws InferenceEngineException, RyaDAOException, AccumuloException, AccumuloSecurityException, SailException {
    final RdfCloudTripleStore store = new RdfCloudTripleStore();
    final RyaDAO<?> dao;
    final RdfCloudTripleStoreConfiguration rdfConfig;
    final String user;
    final String pswd;
    // XXX Should(?) be MongoDBRdfConfiguration.MONGO_COLLECTION_PREFIX inside the if below. RYA-135
    final String ryaInstance = config.get(RdfCloudTripleStoreConfiguration.CONF_TBL_PREFIX);
    Objects.requireNonNull(ryaInstance, "RyaInstance or table prefix is missing from configuration." + RdfCloudTripleStoreConfiguration.CONF_TBL_PREFIX);
    if (ConfigUtils.getUseMongo(config)) {
        // Get a reference to a Mongo DB configuration object.
        final MongoDBRdfConfiguration mongoConfig = (config instanceof MongoDBRdfConfiguration) ? (MongoDBRdfConfiguration) config : new MongoDBRdfConfiguration(config);
        // Create the MongoClient that will be used by the Sail object's components.
        final MongoClient client = createMongoClient(mongoConfig);
        // Add the Indexer and Optimizer names to the configuration object that are configured to be used.
        OptionalConfigUtils.setIndexers(mongoConfig);
        // Populate the configuration using previously stored Rya Details if this instance uses them.
        try {
            final MongoRyaInstanceDetailsRepository ryaDetailsRepo = new MongoRyaInstanceDetailsRepository(client, mongoConfig.getRyaInstanceName());
            RyaDetailsToConfiguration.addRyaDetailsToConfiguration(ryaDetailsRepo.getRyaInstanceDetails(), mongoConfig);
        } catch (final RyaDetailsRepositoryException e) {
            LOG.info("Instance does not have a rya details collection, skipping.");
        }
        // Set the configuration to the stateful configuration that is used to pass the constructed objects around.
        final StatefulMongoDBRdfConfiguration statefulConfig = new StatefulMongoDBRdfConfiguration(mongoConfig, client);
        final List<MongoSecondaryIndex> indexers = statefulConfig.getInstances(AccumuloRdfConfiguration.CONF_ADDITIONAL_INDEXERS, MongoSecondaryIndex.class);
        statefulConfig.setIndexers(indexers);
        rdfConfig = statefulConfig;
        // Create the DAO that is able to interact with MongoDB.
        final MongoDBRyaDAO mongoDao = new MongoDBRyaDAO();
        mongoDao.setConf(statefulConfig);
        mongoDao.init();
        dao = mongoDao;
    } else {
        rdfConfig = new AccumuloRdfConfiguration(config);
        user = rdfConfig.get(ConfigUtils.CLOUDBASE_USER);
        pswd = rdfConfig.get(ConfigUtils.CLOUDBASE_PASSWORD);
        Objects.requireNonNull(user, "Accumulo user name is missing from configuration." + ConfigUtils.CLOUDBASE_USER);
        Objects.requireNonNull(pswd, "Accumulo user password is missing from configuration." + ConfigUtils.CLOUDBASE_PASSWORD);
        rdfConfig.setTableLayoutStrategy(new TablePrefixLayoutStrategy(ryaInstance));
        RyaSailFactory.updateAccumuloConfig((AccumuloRdfConfiguration) rdfConfig, user, pswd, ryaInstance);
        dao = getAccumuloDAO((AccumuloRdfConfiguration) rdfConfig);
    }
    store.setRyaDAO(dao);
    rdfConfig.setTablePrefix(ryaInstance);
    if (rdfConfig.isInfer()) {
        final InferenceEngine inferenceEngine = new InferenceEngine();
        inferenceEngine.setConf(rdfConfig);
        inferenceEngine.setRyaDAO(dao);
        inferenceEngine.init();
        store.setInferenceEngine(inferenceEngine);
    }
    store.initialize();
    return store;
}
Also used : RdfCloudTripleStore(org.apache.rya.rdftriplestore.RdfCloudTripleStore) StatefulMongoDBRdfConfiguration(org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration) MongoRyaInstanceDetailsRepository(org.apache.rya.mongodb.instance.MongoRyaInstanceDetailsRepository) AccumuloRdfConfiguration(org.apache.rya.accumulo.AccumuloRdfConfiguration) MongoSecondaryIndex(org.apache.rya.mongodb.MongoSecondaryIndex) MongoClient(com.mongodb.MongoClient) MongoDBRyaDAO(org.apache.rya.mongodb.MongoDBRyaDAO) InferenceEngine(org.apache.rya.rdftriplestore.inference.InferenceEngine) TablePrefixLayoutStrategy(org.apache.rya.api.layout.TablePrefixLayoutStrategy) RyaDetailsRepositoryException(org.apache.rya.api.instance.RyaDetailsRepository.RyaDetailsRepositoryException) RdfCloudTripleStoreConfiguration(org.apache.rya.api.RdfCloudTripleStoreConfiguration) MongoDBRdfConfiguration(org.apache.rya.mongodb.MongoDBRdfConfiguration) StatefulMongoDBRdfConfiguration(org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration)

Aggregations

StatefulMongoDBRdfConfiguration (org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration)17 MongoDBRyaDAO (org.apache.rya.mongodb.MongoDBRyaDAO)8 MongoClient (com.mongodb.MongoClient)6 MongoSecondaryIndex (org.apache.rya.mongodb.MongoSecondaryIndex)5 Test (org.junit.Test)5 LiteralImpl (org.openrdf.model.impl.LiteralImpl)5 URIImpl (org.openrdf.model.impl.URIImpl)5 RdfCloudTripleStore (org.apache.rya.rdftriplestore.RdfCloudTripleStore)4 SailRepositoryConnection (org.openrdf.repository.sail.SailRepositoryConnection)4 HashSet (java.util.HashSet)3 RyaDetailsRepositoryException (org.apache.rya.api.instance.RyaDetailsRepository.RyaDetailsRepositoryException)3 MongoRyaStatementStore (org.apache.rya.export.mongo.MongoRyaStatementStore)3 TimestampPolicyMongoRyaStatementStore (org.apache.rya.export.mongo.policy.TimestampPolicyMongoRyaStatementStore)3 ExternalTupleSet (org.apache.rya.indexing.external.tupleSet.ExternalTupleSet)3 MongoPcjIndexSetProvider (org.apache.rya.indexing.mongodb.pcj.MongoPcjIndexSetProvider)3 PcjMetadata (org.apache.rya.indexing.pcj.storage.PcjMetadata)3 MongoDBRdfConfiguration (org.apache.rya.mongodb.MongoDBRdfConfiguration)3 MongoRyaInstanceDetailsRepository (org.apache.rya.mongodb.instance.MongoRyaInstanceDetailsRepository)3 RyaSailRepository (org.apache.rya.rdftriplestore.RyaSailRepository)3 Statement (org.openrdf.model.Statement)3