Search in sources :

Example 11 with Configuration

use of com.thinkaurelius.titan.diskstorage.configuration.Configuration in project titan by thinkaurelius.

the class HadoopScanMapper method setup.

@Override
protected void setup(Context context) throws IOException, InterruptedException {
    super.setup(context);
    org.apache.hadoop.conf.Configuration hadoopConf = DEFAULT_COMPAT.getContextConfiguration(context);
    ModifiableHadoopConfiguration scanConf = ModifiableHadoopConfiguration.of(TitanHadoopConfiguration.MAPRED_NS, hadoopConf);
    job = getJob(scanConf);
    metrics = new HadoopContextScanMetrics(context);
    Configuration graphConf = getTitanConfiguration(context);
    finishSetup(scanConf, graphConf);
}
Also used : Configuration(com.thinkaurelius.titan.diskstorage.configuration.Configuration) ModifiableHadoopConfiguration(com.thinkaurelius.titan.hadoop.config.ModifiableHadoopConfiguration) TitanHadoopConfiguration(com.thinkaurelius.titan.hadoop.config.TitanHadoopConfiguration) ModifiableConfiguration(com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration) ModifiableHadoopConfiguration(com.thinkaurelius.titan.hadoop.config.ModifiableHadoopConfiguration)

Example 12 with Configuration

use of com.thinkaurelius.titan.diskstorage.configuration.Configuration in project titan by thinkaurelius.

the class ElasticSearchConfigTest method testTransportClient.

@Test
public void testTransportClient() throws BackendException, InterruptedException {
    ElasticsearchRunner esr = new ElasticsearchRunner(".", "transportClient.yml");
    esr.start();
    ModifiableConfiguration config = GraphDatabaseConfiguration.buildGraphConfiguration();
    config.set(INTERFACE, ElasticSearchSetup.TRANSPORT_CLIENT.toString(), INDEX_NAME);
    config.set(INDEX_HOSTS, new String[] { "127.0.0.1" }, INDEX_NAME);
    Configuration indexConfig = config.restrictTo(INDEX_NAME);
    IndexProvider idx = new ElasticSearchIndex(indexConfig);
    simpleWriteAndQuery(idx);
    idx.close();
    config = GraphDatabaseConfiguration.buildGraphConfiguration();
    config.set(INTERFACE, ElasticSearchSetup.TRANSPORT_CLIENT.toString(), INDEX_NAME);
    config.set(INDEX_HOSTS, new String[] { "10.11.12.13" }, INDEX_NAME);
    indexConfig = config.restrictTo(INDEX_NAME);
    Throwable failure = null;
    try {
        idx = new ElasticSearchIndex(indexConfig);
    } catch (Throwable t) {
        failure = t;
    }
    // idx.close();
    Assert.assertNotNull("ES client failed to throw exception on connection failure", failure);
    esr.stop();
}
Also used : Configuration(com.thinkaurelius.titan.diskstorage.configuration.Configuration) CommonsConfiguration(com.thinkaurelius.titan.diskstorage.configuration.backend.CommonsConfiguration) BasicConfiguration(com.thinkaurelius.titan.diskstorage.configuration.BasicConfiguration) GraphDatabaseConfiguration(com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration) BaseConfiguration(org.apache.commons.configuration.BaseConfiguration) ModifiableConfiguration(com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration) ModifiableConfiguration(com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration) ElasticSearchIndex(com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex) Test(org.junit.Test)

Example 13 with Configuration

use of com.thinkaurelius.titan.diskstorage.configuration.Configuration in project titan by thinkaurelius.

the class ElasticSearchConfigTest method testNetworkNodeUsingYaml.

@Test
public void testNetworkNodeUsingYaml() throws BackendException, InterruptedException {
    ElasticsearchRunner esr = new ElasticsearchRunner(".", "networkNodeUsingYaml.yml");
    esr.start();
    ModifiableConfiguration config = GraphDatabaseConfiguration.buildGraphConfiguration();
    config.set(INTERFACE, ElasticSearchSetup.NODE.toString(), INDEX_NAME);
    config.set(INDEX_CONF_FILE, Joiner.on(File.separator).join("target", "test-classes", "es_cfg_nodeclient.yml"), INDEX_NAME);
    Configuration indexConfig = config.restrictTo(INDEX_NAME);
    IndexProvider idx = new ElasticSearchIndex(indexConfig);
    simpleWriteAndQuery(idx);
    idx.close();
    config = GraphDatabaseConfiguration.buildGraphConfiguration();
    config.set(INTERFACE, ElasticSearchSetup.NODE.toString(), INDEX_NAME);
    config.set(HEALTH_REQUEST_TIMEOUT, "5s", INDEX_NAME);
    config.set(INDEX_CONF_FILE, Joiner.on(File.separator).join("target", "test-classes", "es_cfg_bogus_nodeclient.yml"), INDEX_NAME);
    indexConfig = config.restrictTo(INDEX_NAME);
    Throwable failure = null;
    try {
        idx = new ElasticSearchIndex(indexConfig);
    } catch (Throwable t) {
        failure = t;
    }
    //idx.close();
    Assert.assertNotNull("ES client failed to throw exception on connection failure", failure);
    esr.stop();
}
Also used : Configuration(com.thinkaurelius.titan.diskstorage.configuration.Configuration) CommonsConfiguration(com.thinkaurelius.titan.diskstorage.configuration.backend.CommonsConfiguration) BasicConfiguration(com.thinkaurelius.titan.diskstorage.configuration.BasicConfiguration) GraphDatabaseConfiguration(com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration) BaseConfiguration(org.apache.commons.configuration.BaseConfiguration) ModifiableConfiguration(com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration) ModifiableConfiguration(com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration) ElasticSearchIndex(com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex) Test(org.junit.Test)

Example 14 with Configuration

use of com.thinkaurelius.titan.diskstorage.configuration.Configuration in project titan by thinkaurelius.

the class ElasticSearchConfigTest method testNetworkNodeUsingExt.

@Test
public void testNetworkNodeUsingExt() throws BackendException, InterruptedException {
    ElasticsearchRunner esr = new ElasticsearchRunner(".", "networkNodeUsingExt.yml");
    esr.start();
    CommonsConfiguration cc = new CommonsConfiguration(new BaseConfiguration());
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.node.data", "false");
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.node.client", "true");
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.cluster.name", "networkNodeUsingExt");
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.discovery.zen.ping.multicast.enabled", "false");
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.discovery.zen.ping.unicast.hosts", "localhost,127.0.0.1:9300");
    ModifiableConfiguration config = new ModifiableConfiguration(GraphDatabaseConfiguration.ROOT_NS, cc, BasicConfiguration.Restriction.NONE);
    config.set(INTERFACE, ElasticSearchSetup.NODE.toString(), INDEX_NAME);
    Configuration indexConfig = config.restrictTo(INDEX_NAME);
    IndexProvider idx = new ElasticSearchIndex(indexConfig);
    simpleWriteAndQuery(idx);
    idx.close();
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.discovery.zen.ping.unicast.hosts", "10.11.12.13");
    config = new ModifiableConfiguration(GraphDatabaseConfiguration.ROOT_NS, cc, BasicConfiguration.Restriction.NONE);
    config.set(INTERFACE, ElasticSearchSetup.NODE.toString(), INDEX_NAME);
    config.set(HEALTH_REQUEST_TIMEOUT, "5s", INDEX_NAME);
    indexConfig = config.restrictTo(INDEX_NAME);
    Throwable failure = null;
    try {
        idx = new ElasticSearchIndex(indexConfig);
    } catch (Throwable t) {
        failure = t;
    }
    // idx.close();
    Assert.assertNotNull("ES client failed to throw exception on connection failure", failure);
    esr.stop();
}
Also used : BaseConfiguration(org.apache.commons.configuration.BaseConfiguration) Configuration(com.thinkaurelius.titan.diskstorage.configuration.Configuration) CommonsConfiguration(com.thinkaurelius.titan.diskstorage.configuration.backend.CommonsConfiguration) BasicConfiguration(com.thinkaurelius.titan.diskstorage.configuration.BasicConfiguration) GraphDatabaseConfiguration(com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration) BaseConfiguration(org.apache.commons.configuration.BaseConfiguration) ModifiableConfiguration(com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration) CommonsConfiguration(com.thinkaurelius.titan.diskstorage.configuration.backend.CommonsConfiguration) ModifiableConfiguration(com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration) ElasticSearchIndex(com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex) Test(org.junit.Test)

Example 15 with Configuration

use of com.thinkaurelius.titan.diskstorage.configuration.Configuration in project titan by thinkaurelius.

the class ElasticSearchConfigTest method testLocalNodeUsingExtAndIndexDirectory.

@Test
public void testLocalNodeUsingExtAndIndexDirectory() throws BackendException, InterruptedException {
    String baseDir = Joiner.on(File.separator).join("target", "es", "jvmlocal_ext2");
    assertFalse(new File(baseDir + File.separator + "data").exists());
    CommonsConfiguration cc = new CommonsConfiguration(new BaseConfiguration());
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.node.data", "true");
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.node.client", "false");
    cc.set("index." + INDEX_NAME + ".elasticsearch.ext.node.local", "true");
    ModifiableConfiguration config = new ModifiableConfiguration(GraphDatabaseConfiguration.ROOT_NS, cc, BasicConfiguration.Restriction.NONE);
    config.set(INTERFACE, ElasticSearchSetup.NODE.toString(), INDEX_NAME);
    config.set(INDEX_DIRECTORY, baseDir, INDEX_NAME);
    Configuration indexConfig = config.restrictTo(INDEX_NAME);
    IndexProvider idx = new ElasticSearchIndex(indexConfig);
    simpleWriteAndQuery(idx);
    idx.close();
    assertTrue(new File(baseDir + File.separator + "data").exists());
}
Also used : BaseConfiguration(org.apache.commons.configuration.BaseConfiguration) Configuration(com.thinkaurelius.titan.diskstorage.configuration.Configuration) CommonsConfiguration(com.thinkaurelius.titan.diskstorage.configuration.backend.CommonsConfiguration) BasicConfiguration(com.thinkaurelius.titan.diskstorage.configuration.BasicConfiguration) GraphDatabaseConfiguration(com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration) BaseConfiguration(org.apache.commons.configuration.BaseConfiguration) ModifiableConfiguration(com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration) CommonsConfiguration(com.thinkaurelius.titan.diskstorage.configuration.backend.CommonsConfiguration) ModifiableConfiguration(com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration) ElasticSearchIndex(com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex) File(java.io.File) Test(org.junit.Test)

Aggregations

Configuration (com.thinkaurelius.titan.diskstorage.configuration.Configuration)17 GraphDatabaseConfiguration (com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration)14 ModifiableConfiguration (com.thinkaurelius.titan.diskstorage.configuration.ModifiableConfiguration)10 Test (org.junit.Test)10 ElasticSearchIndex (com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex)9 BaseConfiguration (org.apache.commons.configuration.BaseConfiguration)8 BasicConfiguration (com.thinkaurelius.titan.diskstorage.configuration.BasicConfiguration)7 CommonsConfiguration (com.thinkaurelius.titan.diskstorage.configuration.backend.CommonsConfiguration)7 PermanentBackendException (com.thinkaurelius.titan.diskstorage.PermanentBackendException)3 StandardStoreFeatures (com.thinkaurelius.titan.diskstorage.keycolumnvalue.StandardStoreFeatures)3 File (java.io.File)3 TitanException (com.thinkaurelius.titan.core.TitanException)2 BackendException (com.thinkaurelius.titan.diskstorage.BackendException)2 TemporaryBackendException (com.thinkaurelius.titan.diskstorage.TemporaryBackendException)2 MergedConfiguration (com.thinkaurelius.titan.diskstorage.configuration.MergedConfiguration)2 IndexProviderTest (com.thinkaurelius.titan.diskstorage.indexing.IndexProviderTest)2 IOException (java.io.IOException)2 HBaseConfiguration (org.apache.hadoop.hbase.HBaseConfiguration)2 MasterNotRunningException (org.apache.hadoop.hbase.MasterNotRunningException)2 TableNotEnabledException (org.apache.hadoop.hbase.TableNotEnabledException)2