Search in sources :

Example 1 with TablePrefixLayoutStrategy

use of org.apache.rya.api.layout.TablePrefixLayoutStrategy in project incubator-rya by apache.

the class RdfCloudTripleStoreConfiguration method setTableLayoutStrategy.

public void setTableLayoutStrategy(final TableLayoutStrategy tableLayoutStrategy) {
    if (tableLayoutStrategy != null) {
        this.tableLayoutStrategy = tableLayoutStrategy;
    } else {
        // default
        this.tableLayoutStrategy = new TablePrefixLayoutStrategy();
    }
    set(CONF_TBL_SPO, this.tableLayoutStrategy.getSpo());
    set(CONF_TBL_PO, this.tableLayoutStrategy.getPo());
    set(CONF_TBL_OSP, this.tableLayoutStrategy.getOsp());
    set(CONF_TBL_NS, this.tableLayoutStrategy.getNs());
    set(CONF_TBL_EVAL, this.tableLayoutStrategy.getEval());
}
Also used : TablePrefixLayoutStrategy(org.apache.rya.api.layout.TablePrefixLayoutStrategy)

Example 2 with TablePrefixLayoutStrategy

use of org.apache.rya.api.layout.TablePrefixLayoutStrategy in project incubator-rya by apache.

the class AccumuloSelectivityEvalDAOTest method init.

@Before
public void init() throws AccumuloException, AccumuloSecurityException, TableNotFoundException, TableExistsException {
    mock = new MockInstance("accumulo");
    PasswordToken pToken = new PasswordToken("pass".getBytes());
    conn = mock.getConnector("user", pToken);
    config = new BatchWriterConfig();
    config.setMaxMemory(1000);
    config.setMaxLatency(1000, TimeUnit.SECONDS);
    config.setMaxWriteThreads(10);
    if (conn.tableOperations().exists("rya_prospects")) {
        conn.tableOperations().delete("rya_prospects");
    }
    if (conn.tableOperations().exists("rya_selectivity")) {
        conn.tableOperations().delete("rya_selectivity");
    }
    arc = new AccumuloRdfConfiguration();
    res = new ProspectorServiceEvalStatsDAO(conn, arc);
    arc.setTableLayoutStrategy(new TablePrefixLayoutStrategy());
    arc.setMaxRangesForScanner(300);
}
Also used : PasswordToken(org.apache.accumulo.core.client.security.tokens.PasswordToken) MockInstance(org.apache.accumulo.core.client.mock.MockInstance) ProspectorServiceEvalStatsDAO(org.apache.rya.prospector.service.ProspectorServiceEvalStatsDAO) TablePrefixLayoutStrategy(org.apache.rya.api.layout.TablePrefixLayoutStrategy) BatchWriterConfig(org.apache.accumulo.core.client.BatchWriterConfig) AccumuloRdfConfiguration(org.apache.rya.accumulo.AccumuloRdfConfiguration) Before(org.junit.Before)

Example 3 with TablePrefixLayoutStrategy

use of org.apache.rya.api.layout.TablePrefixLayoutStrategy in project incubator-rya by apache.

the class QueryJoinSelectOptimizerTest method init.

@Before
public void init() throws AccumuloException, AccumuloSecurityException, TableNotFoundException, TableExistsException {
    mock = new MockInstance("accumulo");
    PasswordToken pToken = new PasswordToken("pass".getBytes());
    conn = mock.getConnector("user", pToken);
    config = new BatchWriterConfig();
    config.setMaxMemory(1000);
    config.setMaxLatency(1000, TimeUnit.SECONDS);
    config.setMaxWriteThreads(10);
    if (conn.tableOperations().exists("rya_prospects")) {
        conn.tableOperations().delete("rya_prospects");
    }
    if (conn.tableOperations().exists("rya_selectivity")) {
        conn.tableOperations().delete("rya_selectivity");
    }
    arc = new AccumuloRdfConfiguration();
    arc.setTableLayoutStrategy(new TablePrefixLayoutStrategy());
    arc.setMaxRangesForScanner(300);
    res = new ProspectorServiceEvalStatsDAO(conn, arc);
}
Also used : PasswordToken(org.apache.accumulo.core.client.security.tokens.PasswordToken) MockInstance(org.apache.accumulo.core.client.mock.MockInstance) TablePrefixLayoutStrategy(org.apache.rya.api.layout.TablePrefixLayoutStrategy) ProspectorServiceEvalStatsDAO(org.apache.rya.prospector.service.ProspectorServiceEvalStatsDAO) BatchWriterConfig(org.apache.accumulo.core.client.BatchWriterConfig) AccumuloRdfConfiguration(org.apache.rya.accumulo.AccumuloRdfConfiguration) Before(org.junit.Before)

Example 4 with TablePrefixLayoutStrategy

use of org.apache.rya.api.layout.TablePrefixLayoutStrategy in project incubator-rya by apache.

the class GeoRyaSailFactory method getRyaSail.

private static Sail getRyaSail(final Configuration config) throws InferenceEngineException, RyaDAOException, AccumuloException, AccumuloSecurityException, SailException {
    final RdfCloudTripleStore store = new RdfCloudTripleStore();
    final RyaDAO<?> dao;
    final RdfCloudTripleStoreConfiguration rdfConfig;
    final String user;
    final String pswd;
    // XXX Should(?) be MongoDBRdfConfiguration.MONGO_COLLECTION_PREFIX inside the if below. RYA-135
    final String ryaInstance = config.get(RdfCloudTripleStoreConfiguration.CONF_TBL_PREFIX);
    Objects.requireNonNull(ryaInstance, "RyaInstance or table prefix is missing from configuration." + RdfCloudTripleStoreConfiguration.CONF_TBL_PREFIX);
    if (ConfigUtils.getUseMongo(config)) {
        // Get a reference to a Mongo DB configuration object.
        final MongoDBRdfConfiguration mongoConfig = (config instanceof MongoDBRdfConfiguration) ? (MongoDBRdfConfiguration) config : new MongoDBRdfConfiguration(config);
        // Create the MongoClient that will be used by the Sail object's components.
        final MongoClient client = createMongoClient(mongoConfig);
        // Add the Indexer and Optimizer names to the configuration object that are configured to be used.
        OptionalConfigUtils.setIndexers(mongoConfig);
        // Populate the configuration using previously stored Rya Details if this instance uses them.
        try {
            final MongoRyaInstanceDetailsRepository ryaDetailsRepo = new MongoRyaInstanceDetailsRepository(client, mongoConfig.getRyaInstanceName());
            RyaDetailsToConfiguration.addRyaDetailsToConfiguration(ryaDetailsRepo.getRyaInstanceDetails(), mongoConfig);
        } catch (final RyaDetailsRepositoryException e) {
            LOG.info("Instance does not have a rya details collection, skipping.");
        }
        // Set the configuration to the stateful configuration that is used to pass the constructed objects around.
        final StatefulMongoDBRdfConfiguration statefulConfig = new StatefulMongoDBRdfConfiguration(mongoConfig, client);
        final List<MongoSecondaryIndex> indexers = statefulConfig.getInstances(AccumuloRdfConfiguration.CONF_ADDITIONAL_INDEXERS, MongoSecondaryIndex.class);
        statefulConfig.setIndexers(indexers);
        rdfConfig = statefulConfig;
        // Create the DAO that is able to interact with MongoDB.
        final MongoDBRyaDAO mongoDao = new MongoDBRyaDAO();
        mongoDao.setConf(statefulConfig);
        mongoDao.init();
        dao = mongoDao;
    } else {
        rdfConfig = new AccumuloRdfConfiguration(config);
        user = rdfConfig.get(ConfigUtils.CLOUDBASE_USER);
        pswd = rdfConfig.get(ConfigUtils.CLOUDBASE_PASSWORD);
        Objects.requireNonNull(user, "Accumulo user name is missing from configuration." + ConfigUtils.CLOUDBASE_USER);
        Objects.requireNonNull(pswd, "Accumulo user password is missing from configuration." + ConfigUtils.CLOUDBASE_PASSWORD);
        rdfConfig.setTableLayoutStrategy(new TablePrefixLayoutStrategy(ryaInstance));
        RyaSailFactory.updateAccumuloConfig((AccumuloRdfConfiguration) rdfConfig, user, pswd, ryaInstance);
        dao = getAccumuloDAO((AccumuloRdfConfiguration) rdfConfig);
    }
    store.setRyaDAO(dao);
    rdfConfig.setTablePrefix(ryaInstance);
    if (rdfConfig.isInfer()) {
        final InferenceEngine inferenceEngine = new InferenceEngine();
        inferenceEngine.setConf(rdfConfig);
        inferenceEngine.setRyaDAO(dao);
        inferenceEngine.init();
        store.setInferenceEngine(inferenceEngine);
    }
    store.initialize();
    return store;
}
Also used : RdfCloudTripleStore(org.apache.rya.rdftriplestore.RdfCloudTripleStore) StatefulMongoDBRdfConfiguration(org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration) MongoRyaInstanceDetailsRepository(org.apache.rya.mongodb.instance.MongoRyaInstanceDetailsRepository) AccumuloRdfConfiguration(org.apache.rya.accumulo.AccumuloRdfConfiguration) MongoSecondaryIndex(org.apache.rya.mongodb.MongoSecondaryIndex) MongoClient(com.mongodb.MongoClient) MongoDBRyaDAO(org.apache.rya.mongodb.MongoDBRyaDAO) InferenceEngine(org.apache.rya.rdftriplestore.inference.InferenceEngine) TablePrefixLayoutStrategy(org.apache.rya.api.layout.TablePrefixLayoutStrategy) RyaDetailsRepositoryException(org.apache.rya.api.instance.RyaDetailsRepository.RyaDetailsRepositoryException) RdfCloudTripleStoreConfiguration(org.apache.rya.api.RdfCloudTripleStoreConfiguration) MongoDBRdfConfiguration(org.apache.rya.mongodb.MongoDBRdfConfiguration) StatefulMongoDBRdfConfiguration(org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration)

Example 5 with TablePrefixLayoutStrategy

use of org.apache.rya.api.layout.TablePrefixLayoutStrategy in project incubator-rya by apache.

the class RdfCloudTripleStoreConfiguration method setTablePrefix.

public void setTablePrefix(final String tablePrefix) {
    Preconditions.checkNotNull(tablePrefix);
    set(CONF_TBL_PREFIX, tablePrefix);
    // TODO: Should we change the layout strategy
    setTableLayoutStrategy(new TablePrefixLayoutStrategy(tablePrefix));
}
Also used : TablePrefixLayoutStrategy(org.apache.rya.api.layout.TablePrefixLayoutStrategy)

Aggregations

TablePrefixLayoutStrategy (org.apache.rya.api.layout.TablePrefixLayoutStrategy)11 AccumuloRdfConfiguration (org.apache.rya.accumulo.AccumuloRdfConfiguration)7 BatchWriterConfig (org.apache.accumulo.core.client.BatchWriterConfig)4 MockInstance (org.apache.accumulo.core.client.mock.MockInstance)4 Before (org.junit.Before)4 PasswordToken (org.apache.accumulo.core.client.security.tokens.PasswordToken)3 RdfCloudTripleStoreConfiguration (org.apache.rya.api.RdfCloudTripleStoreConfiguration)3 ProspectorServiceEvalStatsDAO (org.apache.rya.prospector.service.ProspectorServiceEvalStatsDAO)3 MongoDBRdfConfiguration (org.apache.rya.mongodb.MongoDBRdfConfiguration)2 StatefulMongoDBRdfConfiguration (org.apache.rya.mongodb.StatefulMongoDBRdfConfiguration)2 RdfCloudTripleStore (org.apache.rya.rdftriplestore.RdfCloudTripleStore)2 InferenceEngine (org.apache.rya.rdftriplestore.inference.InferenceEngine)2 MongoClient (com.mongodb.MongoClient)1 ArrayList (java.util.ArrayList)1 Date (java.util.Date)1 TableNotFoundException (org.apache.accumulo.core.client.TableNotFoundException)1 TableOperations (org.apache.accumulo.core.client.admin.TableOperations)1 Configuration (org.apache.hadoop.conf.Configuration)1 FileStatus (org.apache.hadoop.fs.FileStatus)1 Path (org.apache.hadoop.fs.Path)1