Search in sources :

Example 91 with ElasticSearchException

use of org.elasticsearch.ElasticSearchException in project elasticsearch-indexing-proxy by codelibs.

the class RequestUtils method createBulkRequest.

public static BulkRequestBuilder createBulkRequest(final Client client, final StreamInput streamInput, final String index) throws IOException {
    final BulkRequestBuilder builder = client.prepareBulk();
    final BulkRequest request = builder.request();
    request.readFrom(streamInput);
    if (index != null) {
        request.requests().stream().forEach(req -> {
            if (req instanceof DeleteRequest) {
                ((DeleteRequest) req).index(index);
            } else if (req instanceof DeleteByQueryRequest) {
                ((DeleteByQueryRequest) req).indices(index);
            } else if (req instanceof IndexRequest) {
                ((IndexRequest) req).index(index);
            } else if (req instanceof UpdateRequest) {
                ((UpdateRequest) req).index(index);
            } else if (req instanceof UpdateByQueryRequest) {
                ((UpdateByQueryRequest) req).indices(index);
            } else {
                throw new ElasticsearchException("Unsupported request in bulk: " + req);
            }
        });
    }
    return builder;
}
Also used : UpdateRequest(org.elasticsearch.action.update.UpdateRequest) BulkRequest(org.elasticsearch.action.bulk.BulkRequest) DeleteByQueryRequest(org.elasticsearch.index.reindex.DeleteByQueryRequest) BulkRequestBuilder(org.elasticsearch.action.bulk.BulkRequestBuilder) ElasticsearchException(org.elasticsearch.ElasticsearchException) IndexRequest(org.elasticsearch.action.index.IndexRequest) DeleteRequest(org.elasticsearch.action.delete.DeleteRequest) UpdateByQueryRequest(org.elasticsearch.index.reindex.UpdateByQueryRequest)

Example 92 with ElasticSearchException

use of org.elasticsearch.ElasticSearchException in project opencast by opencast.

the class AbstractElasticsearchIndex method createIndex.

/**
 * Prepares Elasticsearch index to store data for the types (or mappings) as returned by {@link #getDocumenTypes()}.
 *
 * @param idx
 *          the index name
 *
 * @throws SearchIndexException
 *           if index and type creation fails
 * @throws IOException
 *           if loading of the type definitions fails
 */
private void createIndex(String idx) throws SearchIndexException, IOException {
    // Make sure the site index exists
    try {
        logger.debug("Trying to create index for '{}'", idx);
        CreateIndexRequest indexCreateRequest = new CreateIndexRequest(idx);
        String settings = getIndexSettings(idx);
        if (settings != null)
            indexCreateRequest.settings(settings);
        CreateIndexResponse siteidxResponse = nodeClient.admin().indices().create(indexCreateRequest).actionGet();
        if (!siteidxResponse.isAcknowledged()) {
            throw new SearchIndexException("Unable to create index for '" + idx + "'");
        }
    } catch (IndexAlreadyExistsException e) {
        logger.info("Detected existing index '{}'", idx);
    }
    // Store the correct mapping
    for (String type : getDocumenTypes()) {
        PutMappingRequest siteMappingRequest = new PutMappingRequest(idx);
        siteMappingRequest.source(getIndexTypeDefinition(idx, type));
        siteMappingRequest.type(type);
        PutMappingResponse siteMappingResponse = nodeClient.admin().indices().putMapping(siteMappingRequest).actionGet();
        if (!siteMappingResponse.isAcknowledged()) {
            throw new SearchIndexException("Unable to install '" + type + "' mapping for index '" + idx + "'");
        }
    }
    // See if the index version exists and check if it matches. The request will
    // fail if there is no version index
    boolean versionIndexExists = false;
    GetRequestBuilder getRequestBuilder = nodeClient.prepareGet(idx, VERSION_TYPE, ROOT_ID);
    try {
        GetResponse response = getRequestBuilder.execute().actionGet();
        if (response.isExists() && response.getField(VERSION) != null) {
            int actualIndexVersion = Integer.parseInt((String) response.getField(VERSION).getValue());
            if (indexVersion != actualIndexVersion)
                throw new SearchIndexException("Search index is at version " + actualIndexVersion + ", but codebase expects " + indexVersion);
            versionIndexExists = true;
            logger.debug("Search index version is {}", indexVersion);
        }
    } catch (ElasticsearchException e) {
        logger.debug("Version index has not been created");
    }
    // The index does not exist, let's create it
    if (!versionIndexExists) {
        logger.debug("Creating version index for site '{}'", idx);
        IndexRequestBuilder requestBuilder = nodeClient.prepareIndex(idx, VERSION_TYPE, ROOT_ID);
        logger.debug("Index version of site '{}' is {}", idx, indexVersion);
        requestBuilder = requestBuilder.setSource(VERSION, Integer.toString(indexVersion));
        requestBuilder.execute().actionGet();
    }
    preparedIndices.add(idx);
}
Also used : SearchIndexException(org.opencastproject.matterhorn.search.SearchIndexException) PutMappingRequest(org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest) ElasticsearchException(org.elasticsearch.ElasticsearchException) GetResponse(org.elasticsearch.action.get.GetResponse) IndexRequestBuilder(org.elasticsearch.action.index.IndexRequestBuilder) IndexAlreadyExistsException(org.elasticsearch.indices.IndexAlreadyExistsException) PutMappingResponse(org.elasticsearch.action.admin.indices.mapping.put.PutMappingResponse) CreateIndexRequest(org.elasticsearch.action.admin.indices.create.CreateIndexRequest) CreateIndexResponse(org.elasticsearch.action.admin.indices.create.CreateIndexResponse) GetRequestBuilder(org.elasticsearch.action.get.GetRequestBuilder)

Example 93 with ElasticSearchException

use of org.elasticsearch.ElasticSearchException in project arctic-sea by 52North.

the class ElasticsearchAdminHandler method addUuidToMetadataIfNeeded.

@SuppressWarnings("unchecked")
private void addUuidToMetadataIfNeeded(String uuid) throws ElasticsearchException {
    GetResponse resp = client.prepareGet(settings.getIndexId(), MetadataDataMapping.METADATA_TYPE_NAME, MetadataDataMapping.METADATA_ROW_ID).setOperationThreaded(false).get();
    Object retValues = resp.getSourceAsMap().get(MetadataDataMapping.METADATA_UUIDS_FIELD.getName());
    List<String> values;
    if (retValues instanceof String) {
        values = new LinkedList<>();
        values.add((String) retValues);
    } else if (retValues instanceof List<?>) {
        values = (List<String>) retValues;
    } else {
        throw new ConfigurationError("Invalid %s field type %s should have String or java.util.Collection<String>", MetadataDataMapping.METADATA_UUIDS_FIELD, retValues.getClass());
    }
    // add new uuid if needed
    if (!values.stream().anyMatch(m -> m.equals(uuid))) {
        Map<String, Object> uuids = new HashMap<>();
        values.add(uuid);
        uuids.put(MetadataDataMapping.METADATA_UUIDS_FIELD.getName(), values);
        uuids.put(MetadataDataMapping.METADATA_UPDATE_TIME_FIELD.getName(), Calendar.getInstance(DateTimeZone.UTC.toTimeZone()));
        client.prepareUpdate(settings.getIndexId(), MetadataDataMapping.METADATA_TYPE_NAME, "1").setDoc(uuids).get();
        logger.info("UUID {} is added to the {} type", uuid, MetadataDataMapping.METADATA_TYPE_NAME);
    }
}
Also used : ConfigurationError(org.n52.faroe.ConfigurationError) ElasticsearchException(org.elasticsearch.ElasticsearchException) GetResponse(org.elasticsearch.action.get.GetResponse) DateTimeZone(org.joda.time.DateTimeZone) ConfigurationError(org.n52.faroe.ConfigurationError) KibanaImporter(org.n52.iceland.statistics.api.utils.KibanaImporter) NodeBuilder(org.elasticsearch.node.NodeBuilder) TransportClient(org.elasticsearch.client.transport.TransportClient) LoggerFactory(org.slf4j.LoggerFactory) MetadataDataMapping(org.n52.iceland.statistics.api.mappings.MetadataDataMapping) HashMap(java.util.HashMap) InetAddress(java.net.InetAddress) Inject(javax.inject.Inject) Calendar(java.util.Calendar) Settings(org.elasticsearch.common.settings.Settings) Map(java.util.Map) EmbeddedElasticsearch(org.n52.iceland.statistics.impl.server.EmbeddedElasticsearch) Node(org.elasticsearch.node.Node) ElasticsearchSettings(org.n52.iceland.statistics.api.ElasticsearchSettings) ElasticsearchSettingsKeys(org.n52.iceland.statistics.api.ElasticsearchSettingsKeys) LinkedList(java.util.LinkedList) JsonParseException(com.fasterxml.jackson.core.JsonParseException) Logger(org.slf4j.Logger) InetSocketTransportAddress(org.elasticsearch.common.transport.InetSocketTransportAddress) Client(org.elasticsearch.client.Client) IOException(java.io.IOException) FileInputStream(java.io.FileInputStream) UnknownHostException(java.net.UnknownHostException) StandardCharsets(java.nio.charset.StandardCharsets) CreateIndexResponse(org.elasticsearch.action.admin.indices.create.CreateIndexResponse) IndicesAdminClient(org.elasticsearch.client.IndicesAdminClient) Objects(java.util.Objects) IOUtils(org.apache.commons.io.IOUtils) List(java.util.List) ServletContext(javax.servlet.ServletContext) JsonMappingException(com.fasterxml.jackson.databind.JsonMappingException) IAdminDataHandler(org.n52.iceland.statistics.api.interfaces.datahandler.IAdminDataHandler) DefaultElasticsearchSchemas(org.n52.iceland.statistics.impl.schemabuilders.DefaultElasticsearchSchemas) Joiner(com.google.common.base.Joiner) HashMap(java.util.HashMap) LinkedList(java.util.LinkedList) List(java.util.List) GetResponse(org.elasticsearch.action.get.GetResponse)

Example 94 with ElasticSearchException

use of org.elasticsearch.ElasticSearchException in project core-ng-project by neowu.

the class ElasticSearchImpl method indices.

@Override
public List<ElasticSearchIndex> indices() {
    StopWatch watch = new StopWatch();
    try {
        AdminClient adminClient = client().admin();
        ClusterStateResponse response = adminClient.cluster().state(new ClusterStateRequest().clear().metaData(true)).actionGet();
        ImmutableOpenMap<String, IndexMetaData> indices = response.getState().getMetaData().indices();
        List<ElasticSearchIndex> results = new ArrayList<>(indices.size());
        for (ObjectObjectCursor<String, IndexMetaData> cursor : indices) {
            IndexMetaData metaData = cursor.value;
            ElasticSearchIndex index = new ElasticSearchIndex();
            index.index = metaData.getIndex().getName();
            index.state = metaData.getState();
            results.add(index);
        }
        return results;
    } catch (ElasticsearchException e) {
        // due to elastic search uses async executor to run, we have to wrap the exception to retain the original place caused the exception
        throw new SearchException(e);
    } finally {
        logger.info("indices, elapsedTime={}", watch.elapsedTime());
    }
}
Also used : ClusterStateResponse(org.elasticsearch.action.admin.cluster.state.ClusterStateResponse) ClusterStateRequest(org.elasticsearch.action.admin.cluster.state.ClusterStateRequest) ArrayList(java.util.ArrayList) SearchException(core.framework.search.SearchException) ElasticsearchException(org.elasticsearch.ElasticsearchException) StopWatch(core.framework.util.StopWatch) IndexMetaData(org.elasticsearch.cluster.metadata.IndexMetaData) ElasticSearchIndex(core.framework.search.ElasticSearchIndex) AdminClient(org.elasticsearch.client.AdminClient)

Example 95 with ElasticSearchException

use of org.elasticsearch.ElasticSearchException in project core-ng-project by neowu.

the class ElasticSearchTypeImpl method search.

@Override
public SearchResponse<T> search(SearchRequest request) {
    validate(request);
    StopWatch watch = new StopWatch();
    long esTookTime = 0;
    String index = request.index == null ? this.index : request.index;
    long hits = 0;
    try {
        SearchRequestBuilder builder = client().prepareSearch(index).setQuery(request.query);
        if (request.type != null)
            builder.setSearchType(request.type);
        request.aggregations.forEach(builder::addAggregation);
        request.sorts.forEach(builder::addSort);
        if (request.skip != null)
            builder.setFrom(request.skip);
        if (request.limit != null)
            builder.setSize(request.limit);
        logger.debug("search, index={}, type={}, request={}", index, type, builder);
        org.elasticsearch.action.search.SearchResponse searchResponse = builder.get();
        hits = searchResponse.getHits().getTotalHits();
        esTookTime = searchResponse.getTook().nanos();
        if (searchResponse.getFailedShards() > 0)
            logger.warn("some shard failed, response={}", searchResponse);
        return searchResponse(searchResponse);
    } catch (ElasticsearchException e) {
        // due to elastic search uses async executor to run, we have to wrap the exception to retain the original place caused the exception
        throw new SearchException(e);
    } finally {
        long elapsedTime = watch.elapsedTime();
        ActionLogContext.track("elasticsearch", elapsedTime, (int) hits, 0);
        logger.debug("search, hits={}, esTookTime={}, elapsedTime={}", hits, esTookTime, elapsedTime);
        checkSlowOperation(elapsedTime);
    }
}
Also used : SearchRequestBuilder(org.elasticsearch.action.search.SearchRequestBuilder) SearchException(core.framework.search.SearchException) ElasticsearchException(org.elasticsearch.ElasticsearchException) StopWatch(core.framework.util.StopWatch)

Aggregations

ElasticsearchException (org.elasticsearch.ElasticsearchException)309 IOException (java.io.IOException)127 Settings (org.elasticsearch.common.settings.Settings)32 XContentBuilder (org.elasticsearch.common.xcontent.XContentBuilder)30 HashMap (java.util.HashMap)29 ClusterState (org.elasticsearch.cluster.ClusterState)29 ArrayList (java.util.ArrayList)28 Matchers.containsString (org.hamcrest.Matchers.containsString)25 List (java.util.List)20 Map (java.util.Map)20 AtomicReference (java.util.concurrent.atomic.AtomicReference)20 ParameterizedMessage (org.apache.logging.log4j.message.ParameterizedMessage)18 XContentParser (org.elasticsearch.common.xcontent.XContentParser)17 Path (java.nio.file.Path)16 Test (org.junit.Test)16 ActionListener (org.elasticsearch.action.ActionListener)15 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)14 Version (org.elasticsearch.Version)14 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)13 ResourceNotFoundException (org.elasticsearch.ResourceNotFoundException)13