Search in sources :

Example 6 with IndicesExistsRequest

use of org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequest in project storm-elastic-search by hmsonline.

the class ElasticSearchState method createIndices.

public void createIndices(TridentElasticSearchMapper mapper, List<TridentTuple> tuples) {
    BulkRequestBuilder bulkRequest = client.prepareBulk();
    Set<String> existingIndex = new HashSet<String>();
    for (TridentTuple tuple : tuples) {
        String indexName = mapper.mapToIndex(tuple);
        String type = mapper.mapToType(tuple);
        String key = mapper.mapToKey(tuple);
        Map<String, Object> data = mapper.mapToData(tuple);
        String parentId = mapper.mapToParentId(tuple);
        if (!existingIndex.contains(indexName) && !client.admin().indices().exists(new IndicesExistsRequest(indexName)).actionGet().isExists()) {
            createIndex(bulkRequest, indexName, mapper.mapToIndexSettings(tuple));
            createMapping(bulkRequest, indexName, type, mapper.mapToMappingSettings(tuple));
            existingIndex.add(indexName);
        }
        if (StringUtils.isBlank(parentId)) {
            bulkRequest.add(client.prepareIndex(indexName, type, key).setSource(data));
        } else {
            LOGGER.debug("parent: " + parentId);
            bulkRequest.add(client.prepareIndex(indexName, type, key).setSource(data).setParent(parentId));
        }
    }
    try {
        BulkResponse bulkResponse = bulkRequest.execute().actionGet();
        if (bulkResponse.hasFailures()) {
            // Index failed. Retry!
            throw new FailedException("Cannot create index via ES: " + bulkResponse.buildFailureMessage());
        }
    } catch (ElasticSearchException e) {
        StormElasticSearchUtils.handleElasticSearchException(getClass(), e);
    }
}
Also used : FailedException(backtype.storm.topology.FailedException) BulkResponse(org.elasticsearch.action.bulk.BulkResponse) BulkRequestBuilder(org.elasticsearch.action.bulk.BulkRequestBuilder) IndicesExistsRequest(org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequest) ElasticSearchException(org.elasticsearch.ElasticSearchException) HashSet(java.util.HashSet) TridentTuple(storm.trident.tuple.TridentTuple)

Example 7 with IndicesExistsRequest

use of org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequest in project titan by thinkaurelius.

the class ElasticSearchIndex method checkForOrCreateIndex.

/**
     * If ES already contains this instance's target index, then do nothing.
     * Otherwise, create the index, then wait {@link #CREATE_SLEEP}.
     * <p>
     * The {@code client} field must point to a live, connected client.
     * The {@code indexName} field must be non-null and point to the name
     * of the index to check for existence or create.
     *
     * @param config the config for this ElasticSearchIndex
     * @throws java.lang.IllegalArgumentException if the index could not be created
     */
private void checkForOrCreateIndex(Configuration config) {
    Preconditions.checkState(null != client);
    //Create index if it does not already exist
    IndicesExistsResponse response = client.admin().indices().exists(new IndicesExistsRequest(indexName)).actionGet();
    if (!response.isExists()) {
        ImmutableSettings.Builder settings = ImmutableSettings.settingsBuilder();
        ElasticSearchSetup.applySettingsFromTitanConf(settings, config, ES_CREATE_EXTRAS_NS);
        CreateIndexResponse create = client.admin().indices().prepareCreate(indexName).setSettings(settings.build()).execute().actionGet();
        try {
            final long sleep = config.get(CREATE_SLEEP);
            log.debug("Sleeping {} ms after {} index creation returned from actionGet()", sleep, indexName);
            Thread.sleep(sleep);
        } catch (InterruptedException e) {
            throw new TitanException("Interrupted while waiting for index to settle in", e);
        }
        if (!create.isAcknowledged())
            throw new IllegalArgumentException("Could not create index: " + indexName);
    }
}
Also used : IndicesExistsResponse(org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsResponse) TitanException(com.thinkaurelius.titan.core.TitanException) CreateIndexResponse(org.elasticsearch.action.admin.indices.create.CreateIndexResponse) ImmutableSettings(org.elasticsearch.common.settings.ImmutableSettings) IndicesExistsRequest(org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequest)

Example 8 with IndicesExistsRequest

use of org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequest in project graylog2-server by Graylog2.

the class IndicesTest method testDelete.

@Test
@UsingDataSet(loadStrategy = LoadStrategyEnum.CLEAN_INSERT)
public void testDelete() throws Exception {
    final IndicesExistsRequest beforeRequest = client.admin().indices().prepareExists(INDEX_NAME).request();
    final IndicesExistsResponse beforeResponse = client.admin().indices().exists(beforeRequest).actionGet(ES_TIMEOUT);
    assertThat(beforeResponse.isExists()).isTrue();
    indices.delete(INDEX_NAME);
    final IndicesExistsRequest request = client.admin().indices().prepareExists(INDEX_NAME).request();
    final IndicesExistsResponse response = client.admin().indices().exists(request).actionGet(ES_TIMEOUT);
    assertThat(response.isExists()).isFalse();
}
Also used : IndicesExistsResponse(org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsResponse) IndicesExistsRequest(org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequest) UsingDataSet(com.lordofthejars.nosqlunit.annotation.UsingDataSet) Test(org.junit.Test)

Example 9 with IndicesExistsRequest

use of org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequest in project crate by crate.

the class DDLIntegrationTest method testCreateTableWithIndex.

@Test
public void testCreateTableWithIndex() throws Exception {
    execute("create table quotes (quote string, " + "index quote_fulltext using fulltext(quote) with (analyzer='english')) with (number_of_replicas = 0)");
    ensureYellow();
    assertTrue(client().admin().indices().exists(new IndicesExistsRequest("quotes")).actionGet().isExists());
    String quote = "Would it save you a lot of time if I just gave up and went mad now?";
    execute("insert into quotes values (?)", new Object[] { quote });
    refresh();
    execute("select quote from quotes where match(quote_fulltext, 'time')");
    assertEquals(1L, response.rowCount());
    assertEquals(quote, response.rows()[0][0]);
    // filtering on the actual value does still work
    execute("select quote from quotes where quote = ?", new Object[] { quote });
    assertEquals(1L, response.rowCount());
}
Also used : Matchers.containsString(org.hamcrest.Matchers.containsString) IndicesExistsRequest(org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequest) Test(org.junit.Test)

Example 10 with IndicesExistsRequest

use of org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequest in project crate by crate.

the class DDLIntegrationTest method testCreateTableWithStrictColumnPolicy.

@Test
public void testCreateTableWithStrictColumnPolicy() throws Exception {
    execute("create table test (col1 integer primary key, col2 string) " + "clustered into 5 shards " + "with (column_policy='strict', number_of_replicas = 0)");
    assertTrue(client().admin().indices().exists(new IndicesExistsRequest("test")).actionGet().isExists());
    String expectedMapping = "{\"default\":{" + "\"dynamic\":\"strict\",\"_meta\":{\"routing_hash_function\":\"org.elasticsearch.cluster.routing.Murmur3HashFunction\",\"primary_keys\":[\"col1\"]," + "\"version\":{\"created\":{\"elasticsearch\":" + Version.CURRENT.esVersion.id + ",\"cratedb\":" + Version.CURRENT.id + "}}}," + "\"_all\":{\"enabled\":false}," + "\"dynamic_templates\":[{\"strings\":{\"mapping\":{\"index\":\"not_analyzed\",\"store\":false,\"type\":\"string\",\"doc_values\":true},\"match_mapping_type\":\"string\"}}]," + "\"properties\":{" + "\"col1\":{\"type\":\"integer\"}," + "\"col2\":{\"type\":\"string\",\"index\":\"not_analyzed\"}}}}";
    String expectedSettings = "{\"test\":{" + "\"settings\":{" + "\"index.number_of_replicas\":\"0\"," + "\"index.number_of_shards\":\"5\"," + "\"index.version.created\":\"" + Version.CURRENT.esVersion.id + "\"" + "}}}";
    assertEquals(expectedMapping, getIndexMapping("test"));
    JSONAssert.assertEquals(expectedSettings, getIndexSettings("test"), false);
}
Also used : Matchers.containsString(org.hamcrest.Matchers.containsString) IndicesExistsRequest(org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequest) Test(org.junit.Test)

Aggregations

IndicesExistsRequest (org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequest)17 Test (org.junit.Test)13 Matchers.containsString (org.hamcrest.Matchers.containsString)9 IndicesExistsResponse (org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsResponse)4 UsingDataSet (com.lordofthejars.nosqlunit.annotation.UsingDataSet)2 SQLTransportIntegrationTest (io.crate.integrationtests.SQLTransportIntegrationTest)2 PartitionName (io.crate.metadata.PartitionName)2 BytesRef (org.apache.lucene.util.BytesRef)2 FailedException (backtype.storm.topology.FailedException)1 TitanException (com.thinkaurelius.titan.core.TitanException)1 HashSet (java.util.HashSet)1 Set (java.util.Set)1 ElasticSearchException (org.elasticsearch.ElasticSearchException)1 Alias (org.elasticsearch.action.admin.indices.alias.Alias)1 CreateIndexResponse (org.elasticsearch.action.admin.indices.create.CreateIndexResponse)1 IndicesExistsRequestBuilder (org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequestBuilder)1 BulkRequestBuilder (org.elasticsearch.action.bulk.BulkRequestBuilder)1 BulkResponse (org.elasticsearch.action.bulk.BulkResponse)1 IndicesAdminClient (org.elasticsearch.client.IndicesAdminClient)1 ImmutableSettings (org.elasticsearch.common.settings.ImmutableSettings)1