Search in sources :

Example 1 with Tuple

use of org.opensearch.common.collect.Tuple in project OpenSearch by opensearch-project.

the class RequestConvertersTests method testRethrottle.

public void testRethrottle() {
    TaskId taskId = new TaskId(randomAlphaOfLength(10), randomIntBetween(1, 100));
    RethrottleRequest rethrottleRequest;
    Float requestsPerSecond;
    Map<String, String> expectedParams = new HashMap<>();
    if (frequently()) {
        requestsPerSecond = (float) randomDoubleBetween(0.0, 100.0, true);
        rethrottleRequest = new RethrottleRequest(taskId, requestsPerSecond);
        expectedParams.put(RethrottleRequest.REQUEST_PER_SECOND_PARAMETER, Float.toString(requestsPerSecond));
    } else {
        rethrottleRequest = new RethrottleRequest(taskId);
        expectedParams.put(RethrottleRequest.REQUEST_PER_SECOND_PARAMETER, "-1");
    }
    expectedParams.put("group_by", "none");
    List<Tuple<String, Supplier<Request>>> variants = new ArrayList<>();
    variants.add(new Tuple<String, Supplier<Request>>("_reindex", () -> RequestConverters.rethrottleReindex(rethrottleRequest)));
    variants.add(new Tuple<String, Supplier<Request>>("_update_by_query", () -> RequestConverters.rethrottleUpdateByQuery(rethrottleRequest)));
    variants.add(new Tuple<String, Supplier<Request>>("_delete_by_query", () -> RequestConverters.rethrottleDeleteByQuery(rethrottleRequest)));
    for (Tuple<String, Supplier<Request>> variant : variants) {
        Request request = variant.v2().get();
        assertEquals("/" + variant.v1() + "/" + taskId + "/_rethrottle", request.getEndpoint());
        assertEquals(HttpPost.METHOD_NAME, request.getMethod());
        assertEquals(expectedParams, request.getParameters());
        assertNull(request.getEntity());
    }
    // test illegal RethrottleRequest values
    Exception e = expectThrows(NullPointerException.class, () -> new RethrottleRequest(null, 1.0f));
    assertEquals("taskId cannot be null", e.getMessage());
    e = expectThrows(IllegalArgumentException.class, () -> new RethrottleRequest(new TaskId("taskId", 1), -5.0f));
    assertEquals("requestsPerSecond needs to be positive value but was [-5.0]", e.getMessage());
}
Also used : TaskId(org.opensearch.tasks.TaskId) HashMap(java.util.HashMap) RandomSearchRequestGenerator.randomSearchRequest(org.opensearch.search.RandomSearchRequestGenerator.randomSearchRequest) MasterNodeRequest(org.opensearch.action.support.master.MasterNodeRequest) WriteRequest(org.opensearch.action.support.WriteRequest) AbstractBulkByScrollRequest(org.opensearch.index.reindex.AbstractBulkByScrollRequest) RatedRequest(org.opensearch.index.rankeval.RatedRequest) DeleteRequest(org.opensearch.action.delete.DeleteRequest) TermVectorsRequest(org.opensearch.client.core.TermVectorsRequest) AcknowledgedRequest(org.opensearch.action.support.master.AcknowledgedRequest) FieldCapabilitiesRequest(org.opensearch.action.fieldcaps.FieldCapabilitiesRequest) UpdateRequest(org.opensearch.action.update.UpdateRequest) GetSourceRequest(org.opensearch.client.core.GetSourceRequest) MultiSearchRequest(org.opensearch.action.search.MultiSearchRequest) DocWriteRequest(org.opensearch.action.DocWriteRequest) SearchScrollRequest(org.opensearch.action.search.SearchScrollRequest) ExplainRequest(org.opensearch.action.explain.ExplainRequest) SearchRequest(org.opensearch.action.search.SearchRequest) PutStoredScriptRequest(org.opensearch.action.admin.cluster.storedscripts.PutStoredScriptRequest) DeleteByQueryRequest(org.opensearch.index.reindex.DeleteByQueryRequest) MultiGetRequest(org.opensearch.action.get.MultiGetRequest) BulkRequest(org.opensearch.action.bulk.BulkRequest) GetStoredScriptRequest(org.opensearch.action.admin.cluster.storedscripts.GetStoredScriptRequest) ReplicationRequest(org.opensearch.action.support.replication.ReplicationRequest) AnalyzeRequest(org.opensearch.client.indices.AnalyzeRequest) CountRequest(org.opensearch.client.core.CountRequest) BulkShardRequest(org.opensearch.action.bulk.BulkShardRequest) SearchTemplateRequest(org.opensearch.script.mustache.SearchTemplateRequest) RankEvalRequest(org.opensearch.index.rankeval.RankEvalRequest) DeleteStoredScriptRequest(org.opensearch.action.admin.cluster.storedscripts.DeleteStoredScriptRequest) GetRequest(org.opensearch.action.get.GetRequest) MultiSearchTemplateRequest(org.opensearch.script.mustache.MultiSearchTemplateRequest) UpdateByQueryRequest(org.opensearch.index.reindex.UpdateByQueryRequest) MultiTermVectorsRequest(org.opensearch.client.core.MultiTermVectorsRequest) ClearScrollRequest(org.opensearch.action.search.ClearScrollRequest) IndexRequest(org.opensearch.action.index.IndexRequest) ReindexRequest(org.opensearch.index.reindex.ReindexRequest) ArrayList(java.util.ArrayList) IOException(java.io.IOException) Supplier(java.util.function.Supplier) Tuple(org.opensearch.common.collect.Tuple)

Example 2 with Tuple

use of org.opensearch.common.collect.Tuple in project OpenSearch by opensearch-project.

the class RestHighLevelClientTests method testApiNamingConventions.

public void testApiNamingConventions() throws Exception {
    // this list should be empty once the high-level client is feature complete
    String[] notYetSupportedApi = new String[] { "create", "get_script_context", "get_script_languages", "indices.exists_type", "indices.get_upgrade", "indices.put_alias", "render_search_template", "scripts_painless_execute", "indices.simulate_template", "indices.resolve_index", "indices.add_block" };
    // These API are not required for high-level client feature completeness
    String[] notRequiredApi = new String[] { "cluster.allocation_explain", "cluster.pending_tasks", "cluster.reroute", "cluster.state", "cluster.stats", "cluster.post_voting_config_exclusions", "cluster.delete_voting_config_exclusions", "dangling_indices.delete_dangling_index", "dangling_indices.import_dangling_index", "dangling_indices.list_dangling_indices", "indices.shard_stores", "indices.upgrade", "indices.recovery", "indices.segments", "indices.stats", "ingest.processor_grok", "nodes.info", "nodes.stats", "nodes.hot_threads", "nodes.usage", "nodes.reload_secure_settings", "search_shards" };
    List<String> booleanReturnMethods = Arrays.asList("security.enable_user", "security.disable_user", "security.change_password");
    Set<String> deprecatedMethods = new HashSet<>();
    deprecatedMethods.add("indices.force_merge");
    deprecatedMethods.add("multi_get");
    deprecatedMethods.add("multi_search");
    deprecatedMethods.add("search_scroll");
    ClientYamlSuiteRestSpec restSpec = ClientYamlSuiteRestSpec.load("/rest-api-spec/api");
    Set<String> apiSpec = restSpec.getApis().stream().map(ClientYamlSuiteRestApi::getName).collect(Collectors.toSet());
    Set<String> apiUnsupported = new HashSet<>(apiSpec);
    Set<String> apiNotFound = new HashSet<>();
    Set<String> topLevelMethodsExclusions = new HashSet<>();
    topLevelMethodsExclusions.add("getLowLevelClient");
    topLevelMethodsExclusions.add("close");
    Map<String, Set<Method>> methods = Arrays.stream(RestHighLevelClient.class.getMethods()).filter(method -> method.getDeclaringClass().equals(RestHighLevelClient.class) && topLevelMethodsExclusions.contains(method.getName()) == false).map(method -> Tuple.tuple(toSnakeCase(method.getName()), method)).flatMap(tuple -> tuple.v2().getReturnType().getName().endsWith("Client") ? getSubClientMethods(tuple.v1(), tuple.v2().getReturnType()) : Stream.of(tuple)).filter(tuple -> tuple.v2().getAnnotation(Deprecated.class) == null).collect(Collectors.groupingBy(Tuple::v1, Collectors.mapping(Tuple::v2, Collectors.toSet())));
    // TODO remove in 8.0 - we will undeprecate indices.get_template because the current getIndexTemplate
    // impl will replace the existing getTemplate method.
    // The above general-purpose code ignores all deprecated methods which in this case leaves `getTemplate`
    // looking like it doesn't have a valid implementatation when it does.
    apiUnsupported.remove("indices.get_template");
    // Synced flush is deprecated
    apiUnsupported.remove("indices.flush_synced");
    for (Map.Entry<String, Set<Method>> entry : methods.entrySet()) {
        String apiName = entry.getKey();
        for (Method method : entry.getValue()) {
            assertTrue("method [" + apiName + "] is not final", Modifier.isFinal(method.getClass().getModifiers()) || Modifier.isFinal(method.getModifiers()));
            assertTrue("method [" + method + "] should be public", Modifier.isPublic(method.getModifiers()));
            // we convert all the method names to snake case, hence we need to look for the '_async' suffix rather than 'Async'
            if (apiName.endsWith("_async")) {
                assertAsyncMethod(methods, method, apiName);
            } else if (isSubmitTaskMethod(apiName)) {
                assertSubmitTaskMethod(methods, method, apiName, restSpec);
            } else {
                assertSyncMethod(method, apiName, booleanReturnMethods);
                apiUnsupported.remove(apiName);
                if (apiSpec.contains(apiName) == false) {
                    if (deprecatedMethods.contains(apiName)) {
                        assertTrue("method [" + method.getName() + "], api [" + apiName + "] should be deprecated", method.isAnnotationPresent(Deprecated.class));
                    } else {
                        if (// can get rid of 7.0's deprecated "getTemplate"
                        apiName.equals("indices.get_index_template") == false) {
                            apiNotFound.add(apiName);
                        }
                    }
                }
            }
        }
    }
    assertThat("Some client method doesn't match a corresponding API defined in the REST spec: " + apiNotFound, apiNotFound.size(), equalTo(0));
    // we decided not to support cat API in the high-level REST client, they are supposed to be used from a low-level client
    apiUnsupported.removeIf(api -> api.startsWith("cat."));
    Stream.concat(Arrays.stream(notYetSupportedApi), Arrays.stream(notRequiredApi)).forEach(api -> assertTrue(api + " API is either not defined in the spec or already supported by the high-level client", apiUnsupported.remove(api)));
    assertThat("Some API are not supported but they should be: " + apiUnsupported, apiUnsupported.size(), equalTo(0));
}
Also used : InternalAggregationTestCase(org.opensearch.test.InternalAggregationTestCase) Arrays(java.util.Arrays) Aggregation(org.opensearch.search.aggregations.Aggregation) MainResponse(org.opensearch.client.core.MainResponse) ToXContentFragment(org.opensearch.common.xcontent.ToXContentFragment) CheckedFunction(org.opensearch.common.CheckedFunction) ToXContent(org.opensearch.common.xcontent.ToXContent) CoreMatchers.endsWith(org.hamcrest.CoreMatchers.endsWith) ChildrenAggregationBuilder(org.opensearch.join.aggregations.ChildrenAggregationBuilder) ActionRequest(org.opensearch.action.ActionRequest) OpenSearchException(org.opensearch.OpenSearchException) StatusLine(org.apache.http.StatusLine) NByteArrayEntity(org.apache.http.nio.entity.NByteArrayEntity) XContentParser(org.opensearch.common.xcontent.XContentParser) CoreMatchers.instanceOf(org.hamcrest.CoreMatchers.instanceOf) EvaluationMetric(org.opensearch.index.rankeval.EvaluationMetric) RequestLine(org.apache.http.RequestLine) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) InternalAggregations(org.opensearch.search.aggregations.InternalAggregations) Map(java.util.Map) NStringEntity(org.apache.http.nio.entity.NStringEntity) ActionListener(org.opensearch.action.ActionListener) JsonParseException(com.fasterxml.jackson.core.JsonParseException) Method(java.lang.reflect.Method) PrecisionAtK(org.opensearch.index.rankeval.PrecisionAtK) ClientYamlSuiteRestSpec(org.opensearch.test.rest.yaml.restspec.ClientYamlSuiteRestSpec) MetricDetail(org.opensearch.index.rankeval.MetricDetail) OpenSearchTestCase(org.opensearch.test.OpenSearchTestCase) HttpEntity(org.apache.http.HttpEntity) ContentType(org.apache.http.entity.ContentType) Set(java.util.Set) RestStatus(org.opensearch.rest.RestStatus) Collectors(java.util.stream.Collectors) Tuple(org.opensearch.common.collect.Tuple) RecallAtK(org.opensearch.index.rankeval.RecallAtK) List(java.util.List) Stream(java.util.stream.Stream) HttpGet(org.apache.http.client.methods.HttpGet) Modifier(java.lang.reflect.Modifier) Optional(java.util.Optional) MeanReciprocalRank(org.opensearch.index.rankeval.MeanReciprocalRank) Mockito.any(org.mockito.Mockito.any) SmileXContent(org.opensearch.common.xcontent.smile.SmileXContent) Mockito.mock(org.mockito.Mockito.mock) BytesReference(org.opensearch.common.bytes.BytesReference) BasicStatusLine(org.apache.http.message.BasicStatusLine) CoreMatchers.equalTo(org.hamcrest.CoreMatchers.equalTo) CborXContent(org.opensearch.common.xcontent.cbor.CborXContent) ExpectedReciprocalRank(org.opensearch.index.rankeval.ExpectedReciprocalRank) HashMap(java.util.HashMap) SearchHits(org.opensearch.search.SearchHits) AtomicReference(java.util.concurrent.atomic.AtomicReference) ActionRequestValidationException(org.opensearch.action.ActionRequestValidationException) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) BasicHttpResponse(org.apache.http.message.BasicHttpResponse) XContentHelper.toXContent(org.opensearch.common.xcontent.XContentHelper.toXContent) SearchScrollRequest(org.opensearch.action.search.SearchScrollRequest) SocketTimeoutException(java.net.SocketTimeoutException) ClearScrollResponse(org.opensearch.action.search.ClearScrollResponse) SearchResponse(org.opensearch.action.search.SearchResponse) ClientYamlSuiteRestApi(org.opensearch.test.rest.yaml.restspec.ClientYamlSuiteRestApi) MainRequest(org.opensearch.client.core.MainRequest) Before(org.junit.Before) MatrixStatsAggregationBuilder(org.opensearch.search.aggregations.matrix.stats.MatrixStatsAggregationBuilder) BasicRequestLine(org.apache.http.message.BasicRequestLine) Matchers(org.hamcrest.Matchers) IOException(java.io.IOException) Mockito.times(org.mockito.Mockito.times) Mockito.when(org.mockito.Mockito.when) Mockito.verify(org.mockito.Mockito.verify) XContentBuilder(org.opensearch.common.xcontent.XContentBuilder) DiscountedCumulativeGain(org.opensearch.index.rankeval.DiscountedCumulativeGain) ProtocolVersion(org.apache.http.ProtocolVersion) Sets(org.opensearch.common.util.set.Sets) SearchResponseSections(org.opensearch.action.search.SearchResponseSections) Suggest(org.opensearch.search.suggest.Suggest) ClearScrollRequest(org.opensearch.action.search.ClearScrollRequest) ShardSearchFailure(org.opensearch.action.search.ShardSearchFailure) NamedXContentRegistry(org.opensearch.common.xcontent.NamedXContentRegistry) HttpResponse(org.apache.http.HttpResponse) HttpHost(org.apache.http.HttpHost) Collections(java.util.Collections) Set(java.util.Set) HashSet(java.util.HashSet) Method(java.lang.reflect.Method) ClientYamlSuiteRestSpec(org.opensearch.test.rest.yaml.restspec.ClientYamlSuiteRestSpec) Map(java.util.Map) HashMap(java.util.HashMap) Tuple(org.opensearch.common.collect.Tuple) HashSet(java.util.HashSet)

Example 3 with Tuple

use of org.opensearch.common.collect.Tuple in project OpenSearch by opensearch-project.

the class AddFileKeyStoreCommandTests method testAddMultipleFiles.

public void testAddMultipleFiles() throws Exception {
    final String password = "keystorepassword";
    createKeystore(password);
    final int n = randomIntBetween(1, 8);
    final List<Tuple<String, Path>> settingFilePairs = new ArrayList<>(n);
    for (int i = 0; i < n; i++) {
        settingFilePairs.add(Tuple.tuple("foo" + i, createRandomFile()));
    }
    terminal.addSecretInput(password);
    execute(settingFilePairs.stream().flatMap(t -> Stream.of(t.v1(), t.v2().toString())).toArray(String[]::new));
    for (int i = 0; i < n; i++) {
        assertSecureFile(settingFilePairs.get(i).v1(), settingFilePairs.get(i).v2(), password);
    }
}
Also used : ArrayList(java.util.ArrayList) Matchers.containsString(org.hamcrest.Matchers.containsString) Tuple(org.opensearch.common.collect.Tuple)

Example 4 with Tuple

use of org.opensearch.common.collect.Tuple in project OpenSearch by opensearch-project.

the class UpgradeCli method execute.

/**
 * Executes the upgrade task. This retrieves an instance of {@link UpgradeTask} which is composed
 * of smaller individual tasks that perform specific operations as part of the overall process.
 *
 * @param terminal current terminal the command is running
 * @param options  options supplied to the command
 * @param env      current environment in which this cli tool is running.
 * @throws UserException if any exception is thrown from the tasks
 */
@Override
protected void execute(final Terminal terminal, final OptionSet options, final Environment env) throws UserException {
    try {
        final Tuple<TaskInput, Terminal> input = new Tuple<>(new TaskInput(env), terminal);
        UpgradeTask.getTask().accept(input);
        terminal.println("Done!");
        terminal.println("Next Steps: ");
        terminal.println(" Stop the running elasticsearch on this node.");
        terminal.println(" Start OpenSearch on this node.");
    } catch (RuntimeException ex) {
        throw new UserException(ExitCodes.DATA_ERROR, ex.getMessage());
    }
}
Also used : UserException(org.opensearch.cli.UserException) Terminal(org.opensearch.cli.Terminal) Tuple(org.opensearch.common.collect.Tuple)

Example 5 with Tuple

use of org.opensearch.common.collect.Tuple in project OpenSearch by opensearch-project.

the class DetectEsInstallationTaskTests method testTaskExecution.

@SuppressForbidden(reason = "Read config directory from test resources.")
public void testTaskExecution() throws Exception {
    Path esConfig = new File(getClass().getResource("/config").getPath()).toPath();
    // path for es_home
    terminal.addTextInput(esConfig.getParent().toString());
    // path for es_config
    terminal.addTextInput(esConfig.toString());
    TaskInput taskInput = new TaskInput(env);
    Tuple<TaskInput, Terminal> input = new Tuple<>(taskInput, terminal);
    task.accept(input);
    assertThat(taskInput.getEsConfig(), is(esConfig));
    assertThat(taskInput.getBaseUrl(), is("http://localhost:42123"));
    assertThat(taskInput.getPlugins(), hasSize(0));
    assertThat(taskInput.getNode(), is("node-x"));
    assertThat(taskInput.getCluster(), is("my-cluster"));
}
Also used : Path(java.nio.file.Path) File(java.io.File) Terminal(org.opensearch.cli.Terminal) MockTerminal(org.opensearch.cli.MockTerminal) Tuple(org.opensearch.common.collect.Tuple) SuppressForbidden(org.opensearch.common.SuppressForbidden)

Aggregations

Tuple (org.opensearch.common.collect.Tuple)141 ArrayList (java.util.ArrayList)59 IOException (java.io.IOException)44 List (java.util.List)44 Collections (java.util.Collections)40 HashMap (java.util.HashMap)40 Map (java.util.Map)39 Settings (org.opensearch.common.settings.Settings)37 ClusterState (org.opensearch.cluster.ClusterState)34 ShardId (org.opensearch.index.shard.ShardId)28 Arrays (java.util.Arrays)27 HashSet (java.util.HashSet)27 Collectors (java.util.stream.Collectors)26 Index (org.opensearch.index.Index)25 Set (java.util.Set)24 OpenSearchTestCase (org.opensearch.test.OpenSearchTestCase)24 Matchers.containsString (org.hamcrest.Matchers.containsString)23 BytesReference (org.opensearch.common.bytes.BytesReference)23 Version (org.opensearch.Version)21 Strings (org.opensearch.common.Strings)21