Search in sources :

Example 46 with RestHighLevelClient

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient in project jnosql-diana-driver by eclipse.

the class ElasticsearchDocumentConfiguration method get.

/**
 * returns an {@link ElasticsearchDocumentCollectionManagerFactory} instance
 *
 * @param builder the builder {@link RestClientBuilder}
 * @return a manager factory instance
 * @throws NullPointerException when builder is null
 */
public ElasticsearchDocumentCollectionManagerFactory get(RestClientBuilder builder) {
    Objects.requireNonNull(builder, "builder is required");
    RestHighLevelClient client = new RestHighLevelClient(builder);
    return new ElasticsearchDocumentCollectionManagerFactory(client);
}
Also used : RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient)

Example 47 with RestHighLevelClient

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient in project jnosql-diana-driver by eclipse.

the class EntityConverter method executeId.

private static void executeId(DocumentQuery query, RestHighLevelClient client, String index, QueryConverterResult select, List<DocumentEntity> entities) throws IOException {
    String type = query.getDocumentCollection();
    MultiGetRequest multiGetRequest = new MultiGetRequest();
    select.getIds().stream().map(id -> new MultiGetRequest.Item(index, type, id)).forEach(multiGetRequest::add);
    MultiGetResponse responses = client.multiGet(multiGetRequest);
    Stream.of(responses.getResponses()).map(MultiGetItemResponse::getResponse).map(ElasticsearchEntry::of).filter(ElasticsearchEntry::isNotEmpty).map(ElasticsearchEntry::toEntity).forEach(entities::add);
}
Also used : Document(org.jnosql.diana.api.document.Document) MultiGetResponse(org.elasticsearch.action.get.MultiGetResponse) MultiGetRequest(org.elasticsearch.action.get.MultiGetRequest) IOException(java.io.IOException) HashMap(java.util.HashMap) MultiGetItemResponse(org.elasticsearch.action.get.MultiGetItemResponse) SearchRequest(org.elasticsearch.action.search.SearchRequest) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) DocumentQuery(org.jnosql.diana.api.document.DocumentQuery) ArrayList(java.util.ArrayList) Consumer(java.util.function.Consumer) List(java.util.List) Collectors.toList(java.util.stream.Collectors.toList) Stream(java.util.stream.Stream) DocumentEntity(org.jnosql.diana.api.document.DocumentEntity) Map(java.util.Map) SearchResponse(org.elasticsearch.action.search.SearchResponse) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder) StreamSupport(java.util.stream.StreamSupport) Collections.singletonMap(java.util.Collections.singletonMap) ValueUtil(org.jnosql.diana.driver.ValueUtil) MultiGetResponse(org.elasticsearch.action.get.MultiGetResponse) MultiGetItemResponse(org.elasticsearch.action.get.MultiGetItemResponse) MultiGetRequest(org.elasticsearch.action.get.MultiGetRequest)

Example 48 with RestHighLevelClient

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient in project logging-log4j2 by apache.

the class LogstashIT method test_newlines.

@Test
void test_newlines() throws IOException {
    // Create two log events containing new lines.
    final Level level = Level.DEBUG;
    final String loggerFqcn = "f.q.c.n";
    final String loggerName = "A";
    final SimpleMessage message1 = new SimpleMessage("line1\nline2\r\nline3");
    final long instantMillis1 = Instant.EPOCH.toEpochMilli();
    final LogEvent logEvent1 = Log4jLogEvent.newBuilder().setLoggerName(loggerName).setLoggerFqcn(loggerFqcn).setLevel(level).setMessage(message1).setTimeMillis(instantMillis1).build();
    final SimpleMessage message2 = new SimpleMessage("line4\nline5\r\nline6");
    final long instantMillis2 = instantMillis1 + Duration.ofDays(1).toMillis();
    final LogEvent logEvent2 = Log4jLogEvent.newBuilder().setLoggerName(loggerName).setLoggerFqcn(loggerFqcn).setLevel(level).setMessage(message2).setTimeMillis(instantMillis2).build();
    try (final RestHighLevelClient client = createClient()) {
        final Appender appender = createStartedAppender(JSON_TEMPLATE_GELF_LAYOUT, MavenHardcodedConstants.LS_GELF_INPUT_PORT);
        try {
            // Append the event.
            LOGGER.info("appending events");
            appender.append(logEvent1);
            appender.append(logEvent2);
            LOGGER.info("completed appending events");
            // Wait the message to arrive.
            Awaitility.await().atMost(Duration.ofSeconds(60)).pollDelay(Duration.ofSeconds(2)).until(() -> queryDocumentCount(client) == 2);
            // Verify indexed messages.
            final Set<String> expectedMessages = Stream.of(logEvent1, logEvent2).map(LogstashIT::expectedLogstashMessageField).collect(Collectors.toSet());
            final Set<String> actualMessages = queryDocuments(client).stream().map(source -> (String) source.get(ES_INDEX_MESSAGE_FIELD_NAME)).filter(Objects::nonNull).collect(Collectors.toSet());
            Assertions.assertThat(actualMessages).isEqualTo(expectedMessages);
        } finally {
            appender.stop();
        }
    }
}
Also used : Appender(org.apache.logging.log4j.core.Appender) SocketAppender(org.apache.logging.log4j.core.appender.SocketAppender) LogEvent(org.apache.logging.log4j.core.LogEvent) Log4jLogEvent(org.apache.logging.log4j.core.impl.Log4jLogEvent) SimpleMessage(org.apache.logging.log4j.message.SimpleMessage) Level(org.apache.logging.log4j.Level) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) Test(org.junit.jupiter.api.Test)

Example 49 with RestHighLevelClient

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient in project presto by prestodb.

the class ElasticsearchClient method createClient.

private static RestHighLevelClient createClient(ElasticsearchConfig config, Optional<AwsSecurityConfig> awsSecurityConfig) {
    RestClientBuilder builder = RestClient.builder(new HttpHost(config.getHost(), config.getPort(), config.isTlsEnabled() ? "https" : "http")).setMaxRetryTimeoutMillis((int) config.getMaxRetryTime().toMillis());
    builder.setHttpClientConfigCallback(ignored -> {
        RequestConfig requestConfig = RequestConfig.custom().setConnectTimeout(toIntExact(config.getConnectTimeout().toMillis())).setSocketTimeout(toIntExact(config.getRequestTimeout().toMillis())).build();
        IOReactorConfig reactorConfig = IOReactorConfig.custom().setIoThreadCount(config.getHttpThreadCount()).build();
        // the client builder passed to the call-back is configured to use system properties, which makes it
        // impossible to configure concurrency settings, so we need to build a new one from scratch
        HttpAsyncClientBuilder clientBuilder = HttpAsyncClientBuilder.create().setDefaultRequestConfig(requestConfig).setDefaultIOReactorConfig(reactorConfig).setMaxConnPerRoute(config.getMaxHttpConnections()).setMaxConnTotal(config.getMaxHttpConnections());
        if (config.isTlsEnabled()) {
            buildSslContext(config.getKeystorePath(), config.getKeystorePassword(), config.getTrustStorePath(), config.getTruststorePassword()).ifPresent(clientBuilder::setSSLContext);
            if (config.isVerifyHostnames()) {
                clientBuilder.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE);
            }
        }
        awsSecurityConfig.ifPresent(securityConfig -> clientBuilder.addInterceptorLast(new AwsRequestSigner(securityConfig.getRegion(), getAwsCredentialsProvider(securityConfig))));
        return clientBuilder;
    });
    return new RestHighLevelClient(builder);
}
Also used : IOReactorConfig(org.apache.http.impl.nio.reactor.IOReactorConfig) RequestConfig(org.apache.http.client.config.RequestConfig) HttpHost(org.apache.http.HttpHost) RestClientBuilder(org.elasticsearch.client.RestClientBuilder) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) HttpAsyncClientBuilder(org.apache.http.impl.nio.client.HttpAsyncClientBuilder)

Example 50 with RestHighLevelClient

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient in project flink by apache.

the class ElasticsearchSinkBaseITCase method createRestHighLevelClient.

private RestHighLevelClient createRestHighLevelClient() {
    final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
    credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(ELASTICSEARCH_USER, ELASTICSEARCH_PASSWORD));
    return new RestHighLevelClient(RestClient.builder(HttpHost.create(getElasticsearchHttpHostAddress())).setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)));
}
Also used : RestClient(org.elasticsearch.client.RestClient) DeliveryGuarantee(org.apache.flink.connector.base.DeliveryGuarantee) BeforeEach(org.junit.jupiter.api.BeforeEach) Tuple2(org.apache.flink.api.java.tuple.Tuple2) BiFunction(java.util.function.BiFunction) LoggerFactory(org.slf4j.LoggerFactory) EnumSource(org.junit.jupiter.params.provider.EnumSource) RestartStrategies(org.apache.flink.api.common.restartstrategy.RestartStrategies) MapFunction(org.apache.flink.api.common.functions.MapFunction) LocalStreamEnvironment(org.apache.flink.streaming.api.environment.LocalStreamEnvironment) Lists(org.apache.flink.shaded.guava30.com.google.common.collect.Lists) TestLoggerExtension(org.apache.flink.util.TestLoggerExtension) BasicCredentialsProvider(org.apache.http.impl.client.BasicCredentialsProvider) ExtendWith(org.junit.jupiter.api.extension.ExtendWith) CheckpointListener(org.apache.flink.api.common.state.CheckpointListener) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) Nullable(javax.annotation.Nullable) MethodSource(org.junit.jupiter.params.provider.MethodSource) Logger(org.slf4j.Logger) IOException(java.io.IOException) UUID(java.util.UUID) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) Assertions.assertSame(org.junit.jupiter.api.Assertions.assertSame) DataStream(org.apache.flink.streaming.api.datastream.DataStream) UsernamePasswordCredentials(org.apache.http.auth.UsernamePasswordCredentials) Test(org.junit.jupiter.api.Test) AfterEach(org.junit.jupiter.api.AfterEach) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) List(java.util.List) AuthScope(org.apache.http.auth.AuthScope) Assertions.assertTrue(org.junit.jupiter.api.Assertions.assertTrue) CredentialsProvider(org.apache.http.client.CredentialsProvider) HttpHost(org.apache.http.HttpHost) StreamExecutionEnvironment(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment) BasicCredentialsProvider(org.apache.http.impl.client.BasicCredentialsProvider) BasicCredentialsProvider(org.apache.http.impl.client.BasicCredentialsProvider) CredentialsProvider(org.apache.http.client.CredentialsProvider) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) UsernamePasswordCredentials(org.apache.http.auth.UsernamePasswordCredentials)

Aggregations

RestHighLevelClient (org.elasticsearch.client.RestHighLevelClient)61 HttpHost (org.apache.http.HttpHost)23 RestClientBuilder (org.elasticsearch.client.RestClientBuilder)21 IOException (java.io.IOException)14 RestClient (org.elasticsearch.client.RestClient)13 HashMap (java.util.HashMap)10 IndexRequest (org.elasticsearch.action.index.IndexRequest)8 RequestOptions (org.elasticsearch.client.RequestOptions)7 Test (org.junit.jupiter.api.Test)7 CredentialsProvider (org.apache.http.client.CredentialsProvider)6 SearchRequest (org.elasticsearch.action.search.SearchRequest)6 SearchResponse (org.elasticsearch.action.search.SearchResponse)6 Test (org.junit.Test)6 ArrayList (java.util.ArrayList)5 List (java.util.List)5 Map (java.util.Map)5 UsernamePasswordCredentials (org.apache.http.auth.UsernamePasswordCredentials)5 BasicCredentialsProvider (org.apache.http.impl.client.BasicCredentialsProvider)5 DeleteIndexRequest (org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest)5 BulkRequest (org.elasticsearch.action.bulk.BulkRequest)5