Search in sources :

Example 21 with RestHighLevelClient

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient in project ranger by apache.

the class ElasticSearchAuditDestination method newClient.

private RestHighLevelClient newClient() {
    try {
        if (StringUtils.isNotBlank(user) && StringUtils.isNotBlank(password) && password.contains("keytab") && new File(password).exists()) {
            subject = CredentialsProviderUtil.login(user, password);
        }
        RestClientBuilder restClientBuilder = getRestClientBuilder(hosts, protocol, user, password, port);
        RestHighLevelClient restHighLevelClient = new RestHighLevelClient(restClientBuilder);
        if (LOG.isDebugEnabled()) {
            LOG.debug("Initialized client");
        }
        boolean exits = false;
        try {
            exits = restHighLevelClient.indices().open(new OpenIndexRequest(this.index), RequestOptions.DEFAULT).isShardsAcknowledged();
        } catch (Exception e) {
            LOG.warn("Error validating index " + this.index);
        }
        if (exits) {
            if (LOG.isDebugEnabled()) {
                LOG.debug("Index exists");
            }
        } else {
            LOG.info("Index does not exist");
        }
        return restHighLevelClient;
    } catch (Throwable t) {
        lastLoggedAt.updateAndGet(lastLoggedAt -> {
            long now = System.currentTimeMillis();
            long elapsed = now - lastLoggedAt;
            if (elapsed > TimeUnit.MINUTES.toMillis(1)) {
                LOG.error("Can't connect to ElasticSearch server: " + connectionString(), t);
                return now;
            } else {
                return lastLoggedAt;
            }
        });
        return null;
    }
}
Also used : RestClient(org.elasticsearch.client.RestClient) AuthSchemeProvider(org.apache.http.auth.AuthSchemeProvider) Arrays(java.util.Arrays) StringUtils(org.apache.commons.lang.StringUtils) RegistryBuilder(org.apache.http.config.RegistryBuilder) RestClientBuilder(org.elasticsearch.client.RestClientBuilder) Date(java.util.Date) LoggerFactory(org.slf4j.LoggerFactory) HashMap(java.util.HashMap) AuthSchemes(org.apache.http.client.config.AuthSchemes) OpenIndexRequest(org.elasticsearch.action.admin.indices.open.OpenIndexRequest) ArrayList(java.util.ArrayList) IndexRequest(org.elasticsearch.action.index.IndexRequest) KerberosCredentialsProvider(org.apache.ranger.authorization.credutils.kerberos.KerberosCredentialsProvider) Locale(java.util.Locale) Map(java.util.Map) Lookup(org.apache.http.config.Lookup) RequestOptions(org.elasticsearch.client.RequestOptions) SPNegoSchemeFactory(org.apache.http.impl.auth.SPNegoSchemeFactory) AuthzAuditEvent(org.apache.ranger.audit.model.AuthzAuditEvent) PrivilegedActionException(java.security.PrivilegedActionException) Properties(java.util.Properties) Logger(org.slf4j.Logger) BulkItemResponse(org.elasticsearch.action.bulk.BulkItemResponse) Collection(java.util.Collection) BulkResponse(org.elasticsearch.action.bulk.BulkResponse) KerberosTicket(javax.security.auth.kerberos.KerberosTicket) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) File(java.io.File) Subject(javax.security.auth.Subject) TimeUnit(java.util.concurrent.TimeUnit) AtomicLong(java.util.concurrent.atomic.AtomicLong) MiscUtil(org.apache.ranger.audit.provider.MiscUtil) CredentialsProviderUtil(org.apache.ranger.authorization.credutils.CredentialsProviderUtil) AuditEventBase(org.apache.ranger.audit.model.AuditEventBase) CredentialsProvider(org.apache.http.client.CredentialsProvider) HttpHost(org.apache.http.HttpHost) BulkRequest(org.elasticsearch.action.bulk.BulkRequest) OpenIndexRequest(org.elasticsearch.action.admin.indices.open.OpenIndexRequest) RestClientBuilder(org.elasticsearch.client.RestClientBuilder) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) File(java.io.File) PrivilegedActionException(java.security.PrivilegedActionException)

Example 22 with RestHighLevelClient

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient in project pancm_project by xuwujing.

the class EsScriptSearchTest method init.

/*
     * 初始化服务
     */
private static void init() {
    RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(elasticIp, elasticPort));
    client = new RestHighLevelClient(restClientBuilder);
}
Also used : HttpHost(org.apache.http.HttpHost) RestClientBuilder(org.elasticsearch.client.RestClientBuilder) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient)

Example 23 with RestHighLevelClient

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient in project elasticsearch by elastic.

the class DeleteDocumentationIT method testDelete.

/**
     * This test documents docs/java-rest/high-level/document/delete.asciidoc
     */
public void testDelete() throws IOException {
    RestHighLevelClient client = highLevelClient();
    // tag::delete-request[]
    DeleteRequest request = new DeleteRequest(// <1>
    "index", // <2>
    "type", // <3>
    "id");
    // end::delete-request[]
    // tag::delete-request-props[]
    // <1>
    request.timeout(TimeValue.timeValueSeconds(1));
    // <2>
    request.timeout("1s");
    // <3>
    request.setRefreshPolicy(WriteRequest.RefreshPolicy.WAIT_UNTIL);
    // <4>
    request.setRefreshPolicy("wait_for");
    // <5>
    request.version(2);
    // <6>
    request.versionType(VersionType.EXTERNAL);
    // end::delete-request-props[]
    // tag::delete-execute[]
    DeleteResponse response = client.delete(request);
    try {
        // tag::delete-notfound[]
        if (response.getResult().equals(DocWriteResponse.Result.NOT_FOUND)) {
            // <1>
            throw new Exception("Can't find document to be removed");
        }
    // end::delete-notfound[]
    } catch (Exception ignored) {
    }
    // tag::delete-execute-async[]
    client.deleteAsync(request, new ActionListener<DeleteResponse>() {

        @Override
        public void onResponse(DeleteResponse deleteResponse) {
        // <1>
        }

        @Override
        public void onFailure(Exception e) {
        // <2>
        }
    });
    // tag::delete-conflict[]
    try {
        client.delete(request);
    } catch (ElasticsearchException exception) {
        if (exception.status().equals(RestStatus.CONFLICT)) {
        // <1>
        }
    }
// end::delete-conflict[]
}
Also used : DeleteResponse(org.elasticsearch.action.delete.DeleteResponse) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) ElasticsearchException(org.elasticsearch.ElasticsearchException) DeleteRequest(org.elasticsearch.action.delete.DeleteRequest) ElasticsearchException(org.elasticsearch.ElasticsearchException) IOException(java.io.IOException)

Example 24 with RestHighLevelClient

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient in project elastest-torm by elastest.

the class ElasticsearchService method init.

@PostConstruct
private void init() {
    URL url;
    try {
        url = new URL(this.esApiUrl);
        this.host = url.getHost();
        this.port = url.getPort();
        logger.debug("Elasticsearch API host: {} / port: {}", this.host, this.port);
        this.esClient = new RestHighLevelClient(RestClient.builder(new HttpHost(this.host, this.port, "http")));
    } catch (MalformedURLException e) {
        logger.error("Cannot get Elasticsearch url by given: {}", this.esApiUrl);
    }
}
Also used : MalformedURLException(java.net.MalformedURLException) HttpHost(org.apache.http.HttpHost) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) URL(java.net.URL) PostConstruct(javax.annotation.PostConstruct)

Example 25 with RestHighLevelClient

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestHighLevelClient in project hazelcast by hazelcast.

the class ElasticSinkBuilder method build.

/**
 * Create a sink that writes data into Elasticsearch based on this builder configuration
 */
@Nonnull
public Sink<T> build() {
    requireNonNull(clientFn, "clientFn is not set");
    requireNonNull(mapToRequestFn, "mapToRequestFn is not set");
    return SinkBuilder.sinkBuilder(DEFAULT_NAME, ctx -> new BulkContext(new RestHighLevelClient(clientFn.get()), bulkRequestFn, optionsFn, retries, ctx.logger())).<T>receiveFn((bulkContext, item) -> bulkContext.add(mapToRequestFn.apply(item))).flushFn(BulkContext::flush).destroyFn(BulkContext::close).preferredLocalParallelism(DEFAULT_LOCAL_PARALLELISM).build();
}
Also used : FunctionEx(com.hazelcast.function.FunctionEx) ActionRequest(org.elasticsearch.action.ActionRequest) RestClientBuilder(org.elasticsearch.client.RestClientBuilder) Util.checkNonNullAndSerializable(com.hazelcast.jet.impl.util.Util.checkNonNullAndSerializable) BulkResponse(org.elasticsearch.action.bulk.BulkResponse) IOException(java.io.IOException) DocWriteRequest(org.elasticsearch.action.DocWriteRequest) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) SupplierEx(com.hazelcast.function.SupplierEx) Serializable(java.io.Serializable) JetException(com.hazelcast.jet.JetException) ILogger(com.hazelcast.logging.ILogger) Objects.requireNonNull(java.util.Objects.requireNonNull) RequestOptions(org.elasticsearch.client.RequestOptions) SinkBuilder(com.hazelcast.jet.pipeline.SinkBuilder) BulkRequest(org.elasticsearch.action.bulk.BulkRequest) Nonnull(javax.annotation.Nonnull) Sink(com.hazelcast.jet.pipeline.Sink) RetryUtils.withRetry(com.hazelcast.jet.elastic.impl.RetryUtils.withRetry) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) Nonnull(javax.annotation.Nonnull)

Aggregations

RestHighLevelClient (org.elasticsearch.client.RestHighLevelClient)56 HttpHost (org.apache.http.HttpHost)22 RestClientBuilder (org.elasticsearch.client.RestClientBuilder)19 IOException (java.io.IOException)12 RestClient (org.elasticsearch.client.RestClient)11 HashMap (java.util.HashMap)9 IndexRequest (org.elasticsearch.action.index.IndexRequest)8 Test (org.junit.jupiter.api.Test)7 SearchRequest (org.elasticsearch.action.search.SearchRequest)6 SearchResponse (org.elasticsearch.action.search.SearchResponse)6 RequestOptions (org.elasticsearch.client.RequestOptions)6 ArrayList (java.util.ArrayList)5 List (java.util.List)5 Map (java.util.Map)5 CredentialsProvider (org.apache.http.client.CredentialsProvider)5 DeleteIndexRequest (org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest)5 BulkRequest (org.elasticsearch.action.bulk.BulkRequest)5 BulkResponse (org.elasticsearch.action.bulk.BulkResponse)5 Test (org.junit.Test)5 TableEnvironment (org.apache.flink.table.api.TableEnvironment)4