Search in sources :

Example 1 with RestClient

use of org.elasticsearch.client.RestClient in project elasticsearch by elastic.

the class RemoteScrollableHitSourceTests method sourceWithMockedClient.

private RemoteScrollableHitSource sourceWithMockedClient(boolean mockRemoteVersion, CloseableHttpAsyncClient httpClient) throws Exception {
    HttpAsyncClientBuilder clientBuilder = mock(HttpAsyncClientBuilder.class);
    when(clientBuilder.build()).thenReturn(httpClient);
    RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).setHttpClientConfigCallback(httpClientBuilder -> clientBuilder).build();
    TestRemoteScrollableHitSource hitSource = new TestRemoteScrollableHitSource(restClient) {

        @Override
        void lookupRemoteVersion(Consumer<Version> onVersion) {
            if (mockRemoteVersion) {
                onVersion.accept(Version.CURRENT);
            } else {
                super.lookupRemoteVersion(onVersion);
            }
        }
    };
    if (mockRemoteVersion) {
        hitSource.remoteVersion = Version.CURRENT;
    }
    return hitSource;
}
Also used : Response(org.elasticsearch.action.bulk.byscroll.ScrollableHitSource.Response) ByteSizeUnit(org.elasticsearch.common.unit.ByteSizeUnit) ScheduledFuture(java.util.concurrent.ScheduledFuture) URL(java.net.URL) HttpClientContext(org.apache.http.client.protocol.HttpClientContext) StatusLine(org.apache.http.StatusLine) HeapBufferedAsyncResponseConsumer(org.elasticsearch.client.HeapBufferedAsyncResponseConsumer) Future(java.util.concurrent.Future) After(org.junit.After) ElasticsearchStatusException(org.elasticsearch.ElasticsearchStatusException) Streams(org.elasticsearch.common.io.Streams) ThreadPool(org.elasticsearch.threadpool.ThreadPool) HttpAsyncRequestProducer(org.apache.http.nio.protocol.HttpAsyncRequestProducer) ByteSizeValue(org.elasticsearch.common.unit.ByteSizeValue) HttpEntity(org.apache.http.HttpEntity) ContentType(org.apache.http.entity.ContentType) StringEntity(org.apache.http.entity.StringEntity) CloseableHttpAsyncClient(org.apache.http.impl.nio.client.CloseableHttpAsyncClient) StandardCharsets(java.nio.charset.StandardCharsets) Matchers.instanceOf(org.hamcrest.Matchers.instanceOf) Matchers.any(org.mockito.Matchers.any) HttpAsyncClientBuilder(org.apache.http.impl.nio.client.HttpAsyncClientBuilder) Version(org.elasticsearch.Version) RestStatus(org.elasticsearch.rest.RestStatus) TimeValue.timeValueMinutes(org.elasticsearch.common.unit.TimeValue.timeValueMinutes) Matchers.containsString(org.hamcrest.Matchers.containsString) Mockito.mock(org.mockito.Mockito.mock) RestClient(org.elasticsearch.client.RestClient) ContentTooLongException(org.apache.http.ContentTooLongException) ParsingException(org.elasticsearch.common.ParsingException) BasicStatusLine(org.apache.http.message.BasicStatusLine) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) SearchRequest(org.elasticsearch.action.search.SearchRequest) BackoffPolicy(org.elasticsearch.action.bulk.BackoffPolicy) BytesArray(org.elasticsearch.common.bytes.BytesArray) HttpAsyncResponseConsumer(org.apache.http.nio.protocol.HttpAsyncResponseConsumer) Answer(org.mockito.stubbing.Answer) BasicHttpResponse(org.apache.http.message.BasicHttpResponse) InvocationOnMock(org.mockito.invocation.InvocationOnMock) TimeValue(org.elasticsearch.common.unit.TimeValue) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder) TimeValue.timeValueMillis(org.elasticsearch.common.unit.TimeValue.timeValueMillis) Matchers.hasSize(org.hamcrest.Matchers.hasSize) ESTestCase(org.elasticsearch.test.ESTestCase) ExecutorService(java.util.concurrent.ExecutorService) Before(org.junit.Before) HttpEntityEnclosingRequest(org.apache.http.HttpEntityEnclosingRequest) FileSystemUtils(org.elasticsearch.common.io.FileSystemUtils) TestThreadPool(org.elasticsearch.threadpool.TestThreadPool) Matchers.empty(org.hamcrest.Matchers.empty) EsExecutors(org.elasticsearch.common.util.concurrent.EsExecutors) FutureCallback(org.apache.http.concurrent.FutureCallback) IOException(java.io.IOException) Mockito.when(org.mockito.Mockito.when) InputStreamReader(java.io.InputStreamReader) Consumer(java.util.function.Consumer) ProtocolVersion(org.apache.http.ProtocolVersion) EsRejectedExecutionException(org.elasticsearch.common.util.concurrent.EsRejectedExecutionException) HttpResponse(org.apache.http.HttpResponse) InputStreamEntity(org.apache.http.entity.InputStreamEntity) HttpHost(org.apache.http.HttpHost) HeapBufferedAsyncResponseConsumer(org.elasticsearch.client.HeapBufferedAsyncResponseConsumer) HttpAsyncResponseConsumer(org.apache.http.nio.protocol.HttpAsyncResponseConsumer) Consumer(java.util.function.Consumer) HttpHost(org.apache.http.HttpHost) RestClient(org.elasticsearch.client.RestClient) HttpAsyncClientBuilder(org.apache.http.impl.nio.client.HttpAsyncClientBuilder)

Example 2 with RestClient

use of org.elasticsearch.client.RestClient in project elasticsearch by elastic.

the class ReindexFromRemoteBuildRestClientTests method testBuildRestClient.

public void testBuildRestClient() throws Exception {
    RemoteInfo remoteInfo = new RemoteInfo("https", "localhost", 9200, new BytesArray("ignored"), null, null, emptyMap(), RemoteInfo.DEFAULT_SOCKET_TIMEOUT, RemoteInfo.DEFAULT_CONNECT_TIMEOUT);
    long taskId = randomLong();
    List<Thread> threads = synchronizedList(new ArrayList<>());
    RestClient client = TransportReindexAction.buildRestClient(remoteInfo, taskId, threads);
    try {
        assertBusy(() -> assertThat(threads, hasSize(2)));
        int i = 0;
        for (Thread thread : threads) {
            assertEquals("es-client-" + taskId + "-" + i, thread.getName());
            i++;
        }
    } finally {
        client.close();
    }
}
Also used : BytesArray(org.elasticsearch.common.bytes.BytesArray) RestClient(org.elasticsearch.client.RestClient) RemoteInfo(org.elasticsearch.index.reindex.remote.RemoteInfo)

Example 3 with RestClient

use of org.elasticsearch.client.RestClient in project elasticsearch by elastic.

the class IndexingIT method testSeqNoCheckpoints.

public void testSeqNoCheckpoints() throws Exception {
    Nodes nodes = buildNodeAndVersions();
    assumeFalse("new nodes is empty", nodes.getNewNodes().isEmpty());
    logger.info("cluster discovered: {}", nodes.toString());
    final List<String> bwcNamesList = nodes.getBWCNodes().stream().map(Node::getNodeName).collect(Collectors.toList());
    final String bwcNames = bwcNamesList.stream().collect(Collectors.joining(","));
    Settings.Builder settings = Settings.builder().put(IndexMetaData.INDEX_NUMBER_OF_SHARDS_SETTING.getKey(), 1).put(IndexMetaData.INDEX_NUMBER_OF_REPLICAS_SETTING.getKey(), 2).put("index.routing.allocation.include._name", bwcNames);
    final boolean checkGlobalCheckpoints = nodes.getMaster().getVersion().onOrAfter(Version.V_6_0_0_alpha1_UNRELEASED);
    logger.info("master version is [{}], global checkpoints will be [{}]", nodes.getMaster().getVersion(), checkGlobalCheckpoints ? "checked" : "not be checked");
    if (checkGlobalCheckpoints) {
        settings.put(IndexSettings.INDEX_SEQ_NO_CHECKPOINT_SYNC_INTERVAL.getKey(), "100ms");
    }
    final String index = "test";
    createIndex(index, settings.build());
    try (RestClient newNodeClient = buildClient(restClientSettings(), nodes.getNewNodes().stream().map(Node::getPublishAddress).toArray(HttpHost[]::new))) {
        int numDocs = 0;
        final int numberOfInitialDocs = 1 + randomInt(5);
        logger.info("indexing [{}] docs initially", numberOfInitialDocs);
        numDocs += indexDocs(index, 0, numberOfInitialDocs);
        assertSeqNoOnShards(nodes, checkGlobalCheckpoints, 0, newNodeClient);
        logger.info("allowing shards on all nodes");
        updateIndexSetting(index, Settings.builder().putNull("index.routing.allocation.include._name"));
        ensureGreen();
        assertOK(client().performRequest("POST", index + "/_refresh"));
        for (final String bwcName : bwcNamesList) {
            assertCount(index, "_only_nodes:" + bwcName, numDocs);
        }
        final int numberOfDocsAfterAllowingShardsOnAllNodes = 1 + randomInt(5);
        logger.info("indexing [{}] docs after allowing shards on all nodes", numberOfDocsAfterAllowingShardsOnAllNodes);
        numDocs += indexDocs(index, numDocs, numberOfDocsAfterAllowingShardsOnAllNodes);
        assertSeqNoOnShards(nodes, checkGlobalCheckpoints, 0, newNodeClient);
        Shard primary = buildShards(nodes, newNodeClient).stream().filter(Shard::isPrimary).findFirst().get();
        logger.info("moving primary to new node by excluding {}", primary.getNode().getNodeName());
        updateIndexSetting(index, Settings.builder().put("index.routing.allocation.exclude._name", primary.getNode().getNodeName()));
        ensureGreen();
        int numDocsOnNewPrimary = 0;
        final int numberOfDocsAfterMovingPrimary = 1 + randomInt(5);
        logger.info("indexing [{}] docs after moving primary", numberOfDocsAfterMovingPrimary);
        numDocsOnNewPrimary += indexDocs(index, numDocs, numberOfDocsAfterMovingPrimary);
        numDocs += numberOfDocsAfterMovingPrimary;
        assertSeqNoOnShards(nodes, checkGlobalCheckpoints, numDocsOnNewPrimary, newNodeClient);
        /*
             * Dropping the number of replicas to zero, and then increasing it to one triggers a recovery thus exercising any BWC-logic in
             * the recovery code.
             */
        logger.info("setting number of replicas to 0");
        updateIndexSetting(index, Settings.builder().put("index.number_of_replicas", 0));
        final int numberOfDocsAfterDroppingReplicas = 1 + randomInt(5);
        logger.info("indexing [{}] docs after setting number of replicas to 0", numberOfDocsAfterDroppingReplicas);
        numDocsOnNewPrimary += indexDocs(index, numDocs, numberOfDocsAfterDroppingReplicas);
        numDocs += numberOfDocsAfterDroppingReplicas;
        logger.info("setting number of replicas to 1");
        updateIndexSetting(index, Settings.builder().put("index.number_of_replicas", 1));
        ensureGreen();
        assertOK(client().performRequest("POST", index + "/_refresh"));
        // the number of documents on the primary and on the recovered replica should match the number of indexed documents
        assertCount(index, "_primary", numDocs);
        assertCount(index, "_replica", numDocs);
        assertSeqNoOnShards(nodes, checkGlobalCheckpoints, numDocsOnNewPrimary, newNodeClient);
    }
}
Also used : HttpHost(org.apache.http.HttpHost) RestClient(org.elasticsearch.client.RestClient) Settings(org.elasticsearch.common.settings.Settings) IndexSettings(org.elasticsearch.index.IndexSettings)

Example 4 with RestClient

use of org.elasticsearch.client.RestClient in project elasticsearch by elastic.

the class HttpCompressionIT method testUncompressedResponseByDefault.

public void testUncompressedResponseByDefault() throws IOException {
    RestClient client = client();
    Response response = client.performRequest("GET", "/");
    assertEquals(200, response.getStatusLine().getStatusCode());
    assertNull(response.getHeader(HttpHeaders.CONTENT_ENCODING));
    response = client.performRequest("POST", "/company/employees/1", Collections.emptyMap(), SAMPLE_DOCUMENT);
    assertEquals(201, response.getStatusLine().getStatusCode());
    assertNull(response.getHeader(HttpHeaders.CONTENT_ENCODING));
}
Also used : Response(org.elasticsearch.client.Response) RestClient(org.elasticsearch.client.RestClient)

Example 5 with RestClient

use of org.elasticsearch.client.RestClient in project elasticsearch by elastic.

the class ElasticsearchHostsSnifferTests method testConstructorValidation.

public void testConstructorValidation() throws IOException {
    try {
        new ElasticsearchHostsSniffer(null, 1, ElasticsearchHostsSniffer.Scheme.HTTP);
        fail("should have failed");
    } catch (NullPointerException e) {
        assertEquals("restClient cannot be null", e.getMessage());
    }
    HttpHost httpHost = new HttpHost(httpServer.getAddress().getHostString(), httpServer.getAddress().getPort());
    try (RestClient restClient = RestClient.builder(httpHost).build()) {
        try {
            new ElasticsearchHostsSniffer(restClient, 1, null);
            fail("should have failed");
        } catch (NullPointerException e) {
            assertEquals(e.getMessage(), "scheme cannot be null");
        }
        try {
            new ElasticsearchHostsSniffer(restClient, RandomNumbers.randomIntBetween(getRandom(), Integer.MIN_VALUE, 0), ElasticsearchHostsSniffer.Scheme.HTTP);
            fail("should have failed");
        } catch (IllegalArgumentException e) {
            assertEquals(e.getMessage(), "sniffRequestTimeoutMillis must be greater than 0");
        }
    }
}
Also used : HttpHost(org.apache.http.HttpHost) RestClient(org.elasticsearch.client.RestClient)

Aggregations

RestClient (org.elasticsearch.client.RestClient)12 HttpHost (org.apache.http.HttpHost)9 Response (org.elasticsearch.client.Response)3 Settings (org.elasticsearch.common.settings.Settings)3 IOException (java.io.IOException)2 BasicHeader (org.apache.http.message.BasicHeader)2 Version (org.elasticsearch.Version)2 BackoffPolicy (org.elasticsearch.action.bulk.BackoffPolicy)2 ResponseException (org.elasticsearch.client.ResponseException)2 BytesArray (org.elasticsearch.common.bytes.BytesArray)2 InputStreamReader (java.io.InputStreamReader)1 UncheckedIOException (java.io.UncheckedIOException)1 URL (java.net.URL)1 StandardCharsets (java.nio.charset.StandardCharsets)1 FileSystem (java.nio.file.FileSystem)1 ArrayList (java.util.ArrayList)1 Collections.emptyList (java.util.Collections.emptyList)1 Collections.synchronizedList (java.util.Collections.synchronizedList)1 List (java.util.List)1 Map (java.util.Map)1