Search in sources :

Example 6 with Configuration

use of com.thinkaurelius.titan.diskstorage.configuration.Configuration in project titan by thinkaurelius.

the class ElasticSearchConfigTest method testIndexCreationOptions.

@Test
public void testIndexCreationOptions() throws InterruptedException, BackendException {
    final int shards = 77;
    ElasticsearchRunner esr = new ElasticsearchRunner(".", "indexCreationOptions.yml");
    esr.start();
    CommonsConfiguration cc = new CommonsConfiguration(new BaseConfiguration());
    cc.set("index." + INDEX_NAME + ".elasticsearch.create.ext.number_of_shards", String.valueOf(shards));
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.cluster.name", "indexCreationOptions");
    ModifiableConfiguration config = new ModifiableConfiguration(GraphDatabaseConfiguration.ROOT_NS, cc, BasicConfiguration.Restriction.NONE);
    config.set(INTERFACE, ElasticSearchSetup.NODE.toString(), INDEX_NAME);
    Configuration indexConfig = config.restrictTo(INDEX_NAME);
    IndexProvider idx = new ElasticSearchIndex(indexConfig);
    simpleWriteAndQuery(idx);
    ImmutableSettings.Builder settingsBuilder = ImmutableSettings.settingsBuilder();
    settingsBuilder.put("discovery.zen.ping.multicast.enabled", "false");
    settingsBuilder.put("discovery.zen.ping.unicast.hosts", "localhost,127.0.0.1:9300");
    settingsBuilder.put("cluster.name", "indexCreationOptions");
    NodeBuilder nodeBuilder = NodeBuilder.nodeBuilder().settings(settingsBuilder.build());
    nodeBuilder.client(true).data(false).local(false);
    Node n = nodeBuilder.build().start();
    GetSettingsResponse response = n.client().admin().indices().getSettings(new GetSettingsRequest().indices("titan")).actionGet();
    assertEquals(String.valueOf(shards), response.getSetting("titan", "index.number_of_shards"));
    idx.close();
    n.stop();
    esr.stop();
}
Also used : Configuration(com.thinkaurelius.titan.diskstorage.configuration.Configuration) CommonsConfiguration(com.thinkaurelius.titan.diskstorage.configuration.backend.CommonsConfiguration) BasicConfiguration(com.thinkaurelius.titan.diskstorage.configuration.BasicConfiguration) GraphDatabaseConfiguration(com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration) BaseConfiguration(org.apache.commons.configuration.BaseConfiguration) ModifiableConfiguration(com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration) GetSettingsResponse(org.elasticsearch.action.admin.indices.settings.get.GetSettingsResponse) Node(org.elasticsearch.node.Node) CommonsConfiguration(com.thinkaurelius.titan.diskstorage.configuration.backend.CommonsConfiguration) ModifiableConfiguration(com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration) NodeBuilder(org.elasticsearch.node.NodeBuilder) ImmutableSettings(org.elasticsearch.common.settings.ImmutableSettings) BaseConfiguration(org.apache.commons.configuration.BaseConfiguration) GetSettingsRequest(org.elasticsearch.action.admin.indices.settings.get.GetSettingsRequest) ElasticSearchIndex(com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex) Test(org.junit.Test)

Example 7 with Configuration

use of com.thinkaurelius.titan.diskstorage.configuration.Configuration in project titan by thinkaurelius.

the class ElasticSearchIndexTest method testConfiguration.

@Test
public void testConfiguration() throws BackendException {
    // Test that local-mode has precedence over hostname
    final String index = "es";
    ModifiableConfiguration config = GraphDatabaseConfiguration.buildGraphConfiguration();
    config.set(LOCAL_MODE, true, index);
    config.set(CLIENT_ONLY, true, index);
    config.set(INDEX_HOSTS, new String[] { "10.0.0.1" }, index);
    config.set(GraphDatabaseConfiguration.INDEX_DIRECTORY, StorageSetup.getHomeDir("es"), index);
    Configuration indexConfig = config.restrictTo(index);
    // Shouldn't throw exception
    IndexProvider idx = new ElasticSearchIndex(indexConfig);
    idx.close();
    config = GraphDatabaseConfiguration.buildGraphConfiguration();
    config.set(LOCAL_MODE, false, index);
    config.set(CLIENT_ONLY, true, index);
    config.set(INDEX_HOSTS, new String[] { "10.0.0.1" }, index);
    config.set(GraphDatabaseConfiguration.INDEX_DIRECTORY, StorageSetup.getHomeDir("es"), index);
    indexConfig = config.restrictTo(index);
    RuntimeException expectedException = null;
    try {
        // Should try 10.0.0.1 and throw exception
        idx = new ElasticSearchIndex(indexConfig);
        idx.close();
    } catch (RuntimeException re) {
        expectedException = re;
    }
    assertNotNull(expectedException);
}
Also used : IndexProvider(com.thinkaurelius.titan.diskstorage.indexing.IndexProvider) Configuration(com.thinkaurelius.titan.diskstorage.configuration.Configuration) GraphDatabaseConfiguration(com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration) ModifiableConfiguration(com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration) ModifiableConfiguration(com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration) ElasticSearchIndex(com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex) IndexProviderTest(com.thinkaurelius.titan.diskstorage.indexing.IndexProviderTest) Test(org.junit.Test)

Example 8 with Configuration

use of com.thinkaurelius.titan.diskstorage.configuration.Configuration in project incubator-atlas by apache.

the class HBaseStoreManager method getFeatures.

@Override
public StoreFeatures getFeatures() {
    Configuration c = GraphDatabaseConfiguration.buildConfiguration();
    StandardStoreFeatures.Builder fb = new StandardStoreFeatures.Builder().orderedScan(true).unorderedScan(true).batchMutation(true).multiQuery(true).distributed(true).keyOrdered(true).storeTTL(true).timestamps(true).preferredTimestamps(PREFERRED_TIMESTAMPS).locking(true).keyConsistent(c);
    try {
        fb.localKeyPartition(getDeployment() == Deployment.LOCAL);
    } catch (Exception e) {
        logger.warn("Unexpected exception during getDeployment()", e);
    }
    return fb.build();
}
Also used : StandardStoreFeatures(com.thinkaurelius.titan.diskstorage.keycolumnvalue.StandardStoreFeatures) Configuration(com.thinkaurelius.titan.diskstorage.configuration.Configuration) HBaseConfiguration(org.apache.hadoop.hbase.HBaseConfiguration) GraphDatabaseConfiguration(com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration) TableNotFoundException(org.apache.hadoop.hbase.TableNotFoundException) TemporaryBackendException(com.thinkaurelius.titan.diskstorage.TemporaryBackendException) PermanentBackendException(com.thinkaurelius.titan.diskstorage.PermanentBackendException) BackendException(com.thinkaurelius.titan.diskstorage.BackendException) MasterNotRunningException(org.apache.hadoop.hbase.MasterNotRunningException) TableNotEnabledException(org.apache.hadoop.hbase.TableNotEnabledException) TitanException(com.thinkaurelius.titan.core.TitanException) ZooKeeperConnectionException(org.apache.hadoop.hbase.ZooKeeperConnectionException) IOException(java.io.IOException)

Example 9 with Configuration

use of com.thinkaurelius.titan.diskstorage.configuration.Configuration in project titan by thinkaurelius.

the class ExpectedValueCheckingStoreManager method beginTransaction.

@Override
public ExpectedValueCheckingTransaction beginTransaction(BaseTransactionConfig configuration) throws BackendException {
    // Get a transaction without any guarantees about strong consistency
    StoreTransaction inconsistentTx = manager.beginTransaction(configuration);
    // Get a transaction that provides global strong consistency
    Configuration customOptions = new MergedConfiguration(storeFeatures.getKeyConsistentTxConfig(), configuration.getCustomOptions());
    BaseTransactionConfig consistentTxCfg = new StandardBaseTransactionConfig.Builder(configuration).customOptions(customOptions).build();
    StoreTransaction strongConsistentTx = manager.beginTransaction(consistentTxCfg);
    // Return a wrapper around both the inconsistent and consistent store transactions
    ExpectedValueCheckingTransaction wrappedTx = new ExpectedValueCheckingTransaction(inconsistentTx, strongConsistentTx, maxReadTime);
    return wrappedTx;
}
Also used : MergedConfiguration(com.thinkaurelius.titan.diskstorage.configuration.MergedConfiguration) Configuration(com.thinkaurelius.titan.diskstorage.configuration.Configuration) MergedConfiguration(com.thinkaurelius.titan.diskstorage.configuration.MergedConfiguration) StandardBaseTransactionConfig(com.thinkaurelius.titan.diskstorage.util.StandardBaseTransactionConfig) BaseTransactionConfig(com.thinkaurelius.titan.diskstorage.BaseTransactionConfig)

Example 10 with Configuration

use of com.thinkaurelius.titan.diskstorage.configuration.Configuration in project titan by thinkaurelius.

the class ElasticSearchConfigTest method testLocalNodeUsingExt.

@Test
public void testLocalNodeUsingExt() throws BackendException, InterruptedException {
    String baseDir = Joiner.on(File.separator).join("target", "es", "jvmlocal_ext");
    assertFalse(new File(baseDir + File.separator + "data").exists());
    CommonsConfiguration cc = new CommonsConfiguration(new BaseConfiguration());
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.node.data", "true");
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.node.client", "false");
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.node.local", "true");
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.path.data", baseDir + File.separator + "data");
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.path.work", baseDir + File.separator + "work");
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.path.logs", baseDir + File.separator + "logs");
    ModifiableConfiguration config = new ModifiableConfiguration(GraphDatabaseConfiguration.ROOT_NS, cc, BasicConfiguration.Restriction.NONE);
    config.set(INTERFACE, ElasticSearchSetup.NODE.toString(), INDEX_NAME);
    Configuration indexConfig = config.restrictTo(INDEX_NAME);
    IndexProvider idx = new ElasticSearchIndex(indexConfig);
    simpleWriteAndQuery(idx);
    idx.close();
    assertTrue(new File(baseDir + File.separator + "data").exists());
}
Also used : BaseConfiguration(org.apache.commons.configuration.BaseConfiguration) Configuration(com.thinkaurelius.titan.diskstorage.configuration.Configuration) CommonsConfiguration(com.thinkaurelius.titan.diskstorage.configuration.backend.CommonsConfiguration) BasicConfiguration(com.thinkaurelius.titan.diskstorage.configuration.BasicConfiguration) GraphDatabaseConfiguration(com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration) BaseConfiguration(org.apache.commons.configuration.BaseConfiguration) ModifiableConfiguration(com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration) CommonsConfiguration(com.thinkaurelius.titan.diskstorage.configuration.backend.CommonsConfiguration) ModifiableConfiguration(com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration) ElasticSearchIndex(com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex) File(java.io.File) Test(org.junit.Test)

Aggregations

Configuration (com.thinkaurelius.titan.diskstorage.configuration.Configuration)17 GraphDatabaseConfiguration (com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration)14 ModifiableConfiguration (com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration)10 Test (org.junit.Test)10 ElasticSearchIndex (com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex)9 BaseConfiguration (org.apache.commons.configuration.BaseConfiguration)8 BasicConfiguration (com.thinkaurelius.titan.diskstorage.configuration.BasicConfiguration)7 CommonsConfiguration (com.thinkaurelius.titan.diskstorage.configuration.backend.CommonsConfiguration)7 PermanentBackendException (com.thinkaurelius.titan.diskstorage.PermanentBackendException)3 StandardStoreFeatures (com.thinkaurelius.titan.diskstorage.keycolumnvalue.StandardStoreFeatures)3 File (java.io.File)3 TitanException (com.thinkaurelius.titan.core.TitanException)2 BackendException (com.thinkaurelius.titan.diskstorage.BackendException)2 TemporaryBackendException (com.thinkaurelius.titan.diskstorage.TemporaryBackendException)2 MergedConfiguration (com.thinkaurelius.titan.diskstorage.configuration.MergedConfiguration)2 IndexProviderTest (com.thinkaurelius.titan.diskstorage.indexing.IndexProviderTest)2 IOException (java.io.IOException)2 HBaseConfiguration (org.apache.hadoop.hbase.HBaseConfiguration)2 MasterNotRunningException (org.apache.hadoop.hbase.MasterNotRunningException)2 TableNotEnabledException (org.apache.hadoop.hbase.TableNotEnabledException)2