Search in sources :

Example 1 with BulkIndexRequest

use of org.graylog.testing.elasticsearch.BulkIndexRequest in project graylog2-server by Graylog2.

the class ClientES7 method bulkIndex.

@Override
public void bulkIndex(BulkIndexRequest bulkIndexRequest) {
    final BulkRequest bulkRequest = new BulkRequest();
    bulkRequest.setRefreshPolicy(WriteRequest.RefreshPolicy.IMMEDIATE);
    bulkIndexRequest.requests().forEach((indexName, documents) -> documents.forEach(doc -> bulkRequest.add(createIndexRequest(indexName, doc))));
    client.execute((c, requestOptions) -> c.bulk(bulkRequest, requestOptions));
}
Also used : ElasticsearchClient(org.graylog.storage.elasticsearch7.ElasticsearchClient) IndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.index.IndexRequest) Retryer(com.github.rholder.retry.Retryer) RetryerBuilder(com.github.rholder.retry.RetryerBuilder) LoggerFactory(org.slf4j.LoggerFactory) Callable(java.util.concurrent.Callable) WriteRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.support.WriteRequest) CloseIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.CloseIndexRequest) ClusterHealthStatus(org.graylog.shaded.elasticsearch7.org.elasticsearch.cluster.health.ClusterHealthStatus) PutIndexTemplateRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.PutIndexTemplateRequest) GetIndexTemplatesRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.GetIndexTemplatesRequest) RefreshRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.refresh.RefreshRequest) Map(java.util.Map) JsonNode(com.fasterxml.jackson.databind.JsonNode) AliasActions(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest.AliasActions) Response(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Response) WaitStrategies(com.github.rholder.retry.WaitStrategies) GetIndexTemplatesResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.GetIndexTemplatesResponse) GetIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.GetIndexRequest) BulkIndexRequest(org.graylog.testing.elasticsearch.BulkIndexRequest) Logger(org.slf4j.Logger) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) DeleteIndexTemplateRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.template.delete.DeleteIndexTemplateRequest) Request(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Request) Client(org.graylog.testing.elasticsearch.Client) ObjectMapperProvider(org.graylog2.shared.bindings.providers.ObjectMapperProvider) Streams(com.google.common.collect.Streams) ClusterHealthRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.health.ClusterHealthRequest) Collectors(java.util.stream.Collectors) DeleteIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest) CreateIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.CreateIndexRequest) UpdateSettingsRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.settings.put.UpdateSettingsRequest) TimeUnit(java.util.concurrent.TimeUnit) ClusterUpdateSettingsRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsRequest) Stream(java.util.stream.Stream) StopStrategies(com.github.rholder.retry.StopStrategies) Settings(org.graylog.shaded.elasticsearch7.org.elasticsearch.common.settings.Settings) IndicesAliasesRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest) BulkRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.bulk.BulkRequest) IndexTemplateMetadata(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.IndexTemplateMetadata) Collections(java.util.Collections) BulkRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.bulk.BulkRequest)

Example 2 with BulkIndexRequest

use of org.graylog.testing.elasticsearch.BulkIndexRequest in project graylog2-server by Graylog2.

the class CountsIT method totalReturnsNumberOfMessages.

@ContainerMatrixTest
public void totalReturnsNumberOfMessages() {
    final BulkIndexRequest bulkIndexRequest = new BulkIndexRequest();
    final int count1 = 10;
    for (int i = 0; i < count1; i++) {
        final Map<String, Object> source = ImmutableMap.of("foo", "bar", "counter", i);
        bulkIndexRequest.addRequest(INDEX_NAME_1, source);
    }
    final int count2 = 5;
    for (int i = 0; i < count2; i++) {
        final Map<String, Object> source = ImmutableMap.of("foo", "bar", "counter", i);
        bulkIndexRequest.addRequest(INDEX_NAME_2, source);
    }
    client().bulkIndex(bulkIndexRequest);
    assertThat(counts.total()).isEqualTo(count1 + count2);
    assertThat(counts.total(indexSet1)).isEqualTo(count1);
    assertThat(counts.total(indexSet2)).isEqualTo(count2);
}
Also used : BulkIndexRequest(org.graylog.testing.elasticsearch.BulkIndexRequest) ContainerMatrixTest(org.graylog.testing.containermatrix.annotations.ContainerMatrixTest)

Example 3 with BulkIndexRequest

use of org.graylog.testing.elasticsearch.BulkIndexRequest in project graylog2-server by Graylog2.

the class CountsIT method totalReturnsZeroWithNoIndices.

@ContainerMatrixTest
@Order(2)
public void totalReturnsZeroWithNoIndices() {
    final BulkIndexRequest bulkIndexRequest = new BulkIndexRequest();
    for (int i = 0; i < 10; i++) {
        final Map<String, Object> source = ImmutableMap.of("foo", "bar", "counter", i);
        bulkIndexRequest.addRequest(INDEX_NAME_3, source);
    }
    client().bulkIndex(bulkIndexRequest);
    // Simulate no indices for the second index set.
    when(indexSet2.getManagedIndices()).thenReturn(new String[0]);
    assertThat(counts.total(indexSet1)).isEqualTo(0L);
    assertThat(counts.total(indexSet2)).isEqualTo(0L);
    assertThat(counts.total(indexSet3)).isEqualTo(10L);
    // Simulate no indices for all index sets.
    when(indexSetRegistry.getManagedIndices()).thenReturn(new String[0]);
    assertThat(counts.total()).isEqualTo(0L);
}
Also used : BulkIndexRequest(org.graylog.testing.elasticsearch.BulkIndexRequest) Order(org.junit.jupiter.api.Order) TestMethodOrder(org.junit.jupiter.api.TestMethodOrder) ContainerMatrixTest(org.graylog.testing.containermatrix.annotations.ContainerMatrixTest)

Aggregations

BulkIndexRequest (org.graylog.testing.elasticsearch.BulkIndexRequest)3 ContainerMatrixTest (org.graylog.testing.containermatrix.annotations.ContainerMatrixTest)2 JsonNode (com.fasterxml.jackson.databind.JsonNode)1 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1 Retryer (com.github.rholder.retry.Retryer)1 RetryerBuilder (com.github.rholder.retry.RetryerBuilder)1 StopStrategies (com.github.rholder.retry.StopStrategies)1 WaitStrategies (com.github.rholder.retry.WaitStrategies)1 Streams (com.google.common.collect.Streams)1 Collections (java.util.Collections)1 Map (java.util.Map)1 Callable (java.util.concurrent.Callable)1 TimeUnit (java.util.concurrent.TimeUnit)1 Collectors (java.util.stream.Collectors)1 Stream (java.util.stream.Stream)1 ClusterHealthRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.health.ClusterHealthRequest)1 ClusterUpdateSettingsRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsRequest)1 IndicesAliasesRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest)1 AliasActions (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest.AliasActions)1 DeleteIndexRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest)1