Search in sources :

Example 1 with SearchVersion

use of org.graylog2.storage.SearchVersion in project graylog2-server by Graylog2.

the class V20170607164210_MigrateReopenedIndicesToAliases method getReopenedIndices.

private Set<String> getReopenedIndices(final Collection<String> indices) {
    final SearchVersion elasticsearchVersion = node.getVersion().orElseThrow(() -> new ElasticsearchException("Unable to retrieve Elasticsearch version."));
    final JsonNode clusterStateJson = clusterState.getForIndices(indices);
    final JsonNode indicesJson = clusterStateJson.path("metadata").path("indices");
    final ImmutableSet.Builder<String> reopenedIndices = ImmutableSet.builder();
    if (indicesJson.isMissingNode()) {
        LOG.error("Retrieved cluster state is invalid (no metadata.indices key).");
        LOG.debug("Received cluster state was: {}", clusterStateJson.toString());
        return Collections.emptySet();
    }
    for (Iterator<Map.Entry<String, JsonNode>> it = indicesJson.fields(); it.hasNext(); ) {
        final Map.Entry<String, JsonNode> entry = it.next();
        final String indexName = entry.getKey();
        final JsonNode value = entry.getValue();
        final JsonNode indexSettings = value.path("settings");
        if (indexSettings.isMissingNode()) {
            LOG.error("Unable to retrieve index settings from metadata for index {} - skipping.", indexName);
            LOG.debug("Index metadata was: {}", value.toString());
            continue;
        }
        if (checkForReopened(indexSettings, elasticsearchVersion)) {
            LOG.debug("Adding {} to list of indices to be migrated.", indexName);
            reopenedIndices.add(indexName);
        }
    }
    return reopenedIndices.build();
}
Also used : ImmutableSet(com.google.common.collect.ImmutableSet) JsonNode(com.fasterxml.jackson.databind.JsonNode) ElasticsearchException(org.graylog2.indexer.ElasticsearchException) SearchVersion(org.graylog2.storage.SearchVersion) Map(java.util.Map)

Example 2 with SearchVersion

use of org.graylog2.storage.SearchVersion in project graylog2-server by Graylog2.

the class NodeAdapterES7 method version.

@Override
public Optional<SearchVersion> version() {
    final Request request = new Request("GET", "/");
    final Optional<JsonNode> resp = Optional.of(jsonApi.perform(request, "Unable to retrieve cluster information"));
    final Optional<String> version = resp.map(r -> r.path("version")).map(r -> r.path("number")).map(JsonNode::textValue);
    final SearchVersion.Distribution distribution = resp.map(r -> r.path("version")).map(r -> r.path("distribution")).map(JsonNode::textValue).map(d -> d.toUpperCase(Locale.ROOT)).map(SearchVersion.Distribution::valueOf).orElse(SearchVersion.Distribution.ELASTICSEARCH);
    return version.map(this::parseVersion).map(v -> SearchVersion.create(distribution, v));
}
Also used : Inject(javax.inject.Inject) Locale(java.util.Locale) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Request(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Request) Optional(java.util.Optional) JsonNode(com.fasterxml.jackson.databind.JsonNode) SearchVersion(org.graylog2.storage.SearchVersion) NodeAdapter(org.graylog2.indexer.cluster.NodeAdapter) Request(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Request) JsonNode(com.fasterxml.jackson.databind.JsonNode) SearchVersion(org.graylog2.storage.SearchVersion)

Example 3 with SearchVersion

use of org.graylog2.storage.SearchVersion in project graylog2-server by Graylog2.

the class CmdLineTool method annotateInjectorExceptions.

protected void annotateInjectorExceptions(Collection<Message> messages) {
    for (Message message : messages) {
        // noinspection ThrowableResultOfMethodCallIgnored
        final Throwable rootCause = ExceptionUtils.getRootCause(message.getCause());
        if (rootCause instanceof NodeIdPersistenceException) {
            LOG.error(UI.wallString("Unable to read or persist your NodeId file. This means your node id file (" + configuration.getNodeIdFile() + ") is not readable or writable by the current user. The following exception might give more information: " + message));
            System.exit(-1);
        } else if (rootCause instanceof AccessDeniedException) {
            LOG.error(UI.wallString("Unable to access file " + rootCause.getMessage()));
            System.exit(-2);
        } else if (rootCause instanceof UnsupportedSearchException) {
            final SearchVersion search = ((UnsupportedSearchException) rootCause).getSearchMajorVersion();
            LOG.error(UI.wallString("Unsupported search version: " + search, DocsHelper.PAGE_ES_VERSIONS.toString()));
            System.exit(-3);
        } else if (rootCause instanceof ElasticsearchProbeException) {
            LOG.error(UI.wallString(rootCause.getMessage(), DocsHelper.PAGE_ES_CONFIGURATION.toString()));
            System.exit(-4);
        } else {
            // other guice error, still print the raw messages
            // TODO this could potentially print duplicate messages depending on what a subclass does...
            LOG.error("Guice error (more detail on log level debug): {}", message.getMessage());
            if (rootCause != null) {
                LOG.debug("Stacktrace:", rootCause);
            }
        }
    }
}
Also used : ElasticsearchProbeException(org.graylog2.storage.versionprobe.ElasticsearchProbeException) AccessDeniedException(java.nio.file.AccessDeniedException) Message(com.google.inject.spi.Message) UnsupportedSearchException(org.graylog2.storage.UnsupportedSearchException) SearchVersion(org.graylog2.storage.SearchVersion) NodeIdPersistenceException(org.graylog2.plugin.system.NodeIdPersistenceException)

Example 4 with SearchVersion

use of org.graylog2.storage.SearchVersion in project graylog2-server by Graylog2.

the class SearchDbPreflightCheck method runCheck.

@Override
public void runCheck() throws PreflightCheckException {
    try {
        final SearchVersion searchVersion = elasticVersionProbe.probe(elasticsearchHosts).orElseThrow(() -> new PreflightCheckException("Could not get Elasticsearch version"));
        if (SUPPORTED_ES_VERSIONS.stream().noneMatch(searchVersion::satisfies)) {
            throw new PreflightCheckException(StringUtils.f("Unsupported (Elastic/Open)Search version <%s>. Supported versions: <%s>", searchVersion, SUPPORTED_ES_VERSIONS));
        }
        LOG.info("Connected to (Elastic/Open)Search version <{}>", searchVersion);
    } catch (ElasticsearchProbeException e) {
        throw new PreflightCheckException(e);
    }
}
Also used : ElasticsearchProbeException(org.graylog2.storage.versionprobe.ElasticsearchProbeException) SearchVersion(org.graylog2.storage.SearchVersion)

Example 5 with SearchVersion

use of org.graylog2.storage.SearchVersion in project graylog2-server by Graylog2.

the class VersionProbe method probeSingleHost.

private Optional<SearchVersion> probeSingleHost(URI host) {
    final Retrofit retrofit;
    try {
        retrofit = new Retrofit.Builder().baseUrl(host.toURL()).addConverterFactory(JacksonConverterFactory.create(objectMapper)).client(addAuthenticationIfPresent(host, okHttpClient)).build();
    } catch (MalformedURLException e) {
        LOG.error("Elasticsearch node URL is invalid: " + host.toString(), e);
        return Optional.empty();
    }
    final RootRoute root = retrofit.create(RootRoute.class);
    final Converter<ResponseBody, ErrorResponse> errorResponseConverter = retrofit.responseBodyConverter(ErrorResponse.class, new Annotation[0]);
    final Consumer<ResponseBody> errorLogger = (responseBody) -> {
        try {
            final ErrorResponse errorResponse = errorResponseConverter.convert(responseBody);
            LOG.error("Unable to retrieve version from Elasticsearch node {}:{}: {}", host.getHost(), host.getPort(), errorResponse);
        } catch (IOException e) {
            LOG.error("Unable to retrieve version from Elasticsearch node {}:{}: unknown error - an exception occurred while deserializing error response: {}", host.getHost(), host.getPort(), e);
        }
    };
    return rootResponse(root, errorLogger).map(RootResponse::version).flatMap(this::parseVersion);
}
Also used : RetryerBuilder(com.github.rholder.retry.RetryerBuilder) RetryException(com.github.rholder.retry.RetryException) LoggerFactory(org.slf4j.LoggerFactory) Response(retrofit2.Response) Inject(javax.inject.Inject) Strings(com.google.common.base.Strings) Duration(com.github.joschi.jadconfig.util.Duration) Named(javax.inject.Named) URI(java.net.URI) Version(org.graylog2.plugin.Version) ResponseBody(okhttp3.ResponseBody) WaitStrategies(com.github.rholder.retry.WaitStrategies) ExceptionUtils(org.graylog2.shared.utilities.ExceptionUtils) Converter(retrofit2.Converter) Request(okhttp3.Request) Logger(org.slf4j.Logger) MalformedURLException(java.net.MalformedURLException) Collection(java.util.Collection) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Attempt(com.github.rholder.retry.Attempt) IOException(java.io.IOException) Credentials(okhttp3.Credentials) SearchVersion(org.graylog2.storage.SearchVersion) RetryListener(com.github.rholder.retry.RetryListener) Retrofit(retrofit2.Retrofit) ExecutionException(java.util.concurrent.ExecutionException) Consumer(java.util.function.Consumer) OkHttpClient(okhttp3.OkHttpClient) StopStrategies(com.github.rholder.retry.StopStrategies) JacksonConverterFactory(retrofit2.converter.jackson.JacksonConverterFactory) Annotation(java.lang.annotation.Annotation) Optional(java.util.Optional) Retrofit(retrofit2.Retrofit) MalformedURLException(java.net.MalformedURLException) IOException(java.io.IOException) ResponseBody(okhttp3.ResponseBody)

Aggregations

SearchVersion (org.graylog2.storage.SearchVersion)16 Optional (java.util.Optional)3 Inject (javax.inject.Inject)3 ElasticsearchProbeException (org.graylog2.storage.versionprobe.ElasticsearchProbeException)3 JsonNode (com.fasterxml.jackson.databind.JsonNode)2 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)2 JestResult (io.searchbox.client.JestResult)2 Ping (io.searchbox.core.Ping)2 URI (java.net.URI)2 List (java.util.List)2 ExecutionException (java.util.concurrent.ExecutionException)2 Named (javax.inject.Named)2 MongodbServer (org.graylog.testing.containermatrix.MongodbServer)2 ContainerMatrixTestClassDescriptor (org.junit.jupiter.engine.descriptor.ContainerMatrixTestClassDescriptor)2 ContainerMatrixTestsDescriptor (org.junit.jupiter.engine.descriptor.ContainerMatrixTestsDescriptor)2 Logger (org.slf4j.Logger)2 LoggerFactory (org.slf4j.LoggerFactory)2 Duration (com.github.joschi.jadconfig.util.Duration)1 Attempt (com.github.rholder.retry.Attempt)1 RetryException (com.github.rholder.retry.RetryException)1