Search in sources :

Example 76 with ElasticSearchException

use of org.elasticsearch.ElasticSearchException in project graylog2-server by Graylog2.

the class Indices method indexCreationDate.

@Nullable
public DateTime indexCreationDate(String index) {
    final GetIndexRequest indexRequest = c.admin().indices().prepareGetIndex().addFeatures(GetIndexRequest.Feature.SETTINGS).addIndices(index).request();
    try {
        final GetIndexResponse response = c.admin().indices().getIndex(indexRequest).actionGet();
        final Settings settings = response.settings().get(index);
        if (settings == null) {
            return null;
        }
        return new DateTime(settings.getAsLong("index.creation_date", 0L), DateTimeZone.UTC);
    } catch (ElasticsearchException e) {
        LOG.warn("Unable to read creation_date for index " + index, e.getRootCause());
        return null;
    }
}
Also used : GetIndexResponse(org.elasticsearch.action.admin.indices.get.GetIndexResponse) GetIndexRequest(org.elasticsearch.action.admin.indices.get.GetIndexRequest) ElasticsearchException(org.elasticsearch.ElasticsearchException) Settings(org.elasticsearch.common.settings.Settings) DateTime(org.joda.time.DateTime) Nullable(javax.annotation.Nullable)

Example 77 with ElasticSearchException

use of org.elasticsearch.ElasticSearchException in project graylog2-server by Graylog2.

the class Indices method indexRangeStatsOfIndex.

/**
     * Calculate min and max message timestamps in the given index.
     *
     * @param index Name of the index to query.
     * @return the timestamp stats in the given index, or {@code null} if they couldn't be calculated.
     * @see org.elasticsearch.search.aggregations.metrics.stats.Stats
     */
public IndexRangeStats indexRangeStatsOfIndex(String index) {
    final FilterAggregationBuilder builder = AggregationBuilders.filter("agg").filter(QueryBuilders.existsQuery("timestamp")).subAggregation(AggregationBuilders.min("ts_min").field("timestamp")).subAggregation(AggregationBuilders.max("ts_max").field("timestamp")).subAggregation(AggregationBuilders.terms("streams").field("streams"));
    final SearchRequestBuilder srb = c.prepareSearch().setIndices(index).setSearchType(SearchType.QUERY_THEN_FETCH).setSize(0).addAggregation(builder);
    final SearchResponse response;
    try {
        final SearchRequest request = srb.request();
        if (LOG.isDebugEnabled()) {
            LOG.debug("Index range query: _search/{}: {}", index, XContentHelper.convertToJson(request.source(), false));
        }
        response = c.search(request).actionGet();
    } catch (IndexClosedException e) {
        throw e;
    } catch (org.elasticsearch.index.IndexNotFoundException e) {
        LOG.error("Error while calculating timestamp stats in index <" + index + ">", e);
        throw e;
    } catch (ElasticsearchException e) {
        LOG.error("Error while calculating timestamp stats in index <" + index + ">", e);
        throw new org.elasticsearch.index.IndexNotFoundException("Index " + index + " not found", e);
    } catch (IOException e) {
        // the index range aggregation query on DEBUG (via XContentHelper)
        throw new RuntimeException(e);
    }
    final Filter f = response.getAggregations().get("agg");
    if (f.getDocCount() == 0L) {
        LOG.debug("No documents with attribute \"timestamp\" found in index <{}>", index);
        return IndexRangeStats.EMPTY;
    }
    final Min minAgg = f.getAggregations().get("ts_min");
    final DateTime min = new DateTime((long) minAgg.getValue(), DateTimeZone.UTC);
    final Max maxAgg = f.getAggregations().get("ts_max");
    final DateTime max = new DateTime((long) maxAgg.getValue(), DateTimeZone.UTC);
    // make sure we return an empty list, so we can differentiate between old indices that don't have this information
    // and newer ones that simply have no streams.
    ImmutableList.Builder<String> streamIds = ImmutableList.builder();
    final Terms streams = f.getAggregations().get("streams");
    if (!streams.getBuckets().isEmpty()) {
        streamIds.addAll(streams.getBuckets().stream().map(Terms.Bucket::getKeyAsString).collect(toSet()));
    }
    return IndexRangeStats.create(min, max, streamIds.build());
}
Also used : SearchRequest(org.elasticsearch.action.search.SearchRequest) FilterAggregationBuilder(org.elasticsearch.search.aggregations.bucket.filter.FilterAggregationBuilder) SearchRequestBuilder(org.elasticsearch.action.search.SearchRequestBuilder) Max(org.elasticsearch.search.aggregations.metrics.max.Max) ImmutableList(com.google.common.collect.ImmutableList) Terms(org.elasticsearch.search.aggregations.bucket.terms.Terms) ElasticsearchException(org.elasticsearch.ElasticsearchException) IOException(java.io.IOException) DateTime(org.joda.time.DateTime) SearchResponse(org.elasticsearch.action.search.SearchResponse) Min(org.elasticsearch.search.aggregations.metrics.min.Min) IndexClosedException(org.elasticsearch.indices.IndexClosedException) Filter(org.elasticsearch.search.aggregations.bucket.filter.Filter)

Example 78 with ElasticSearchException

use of org.elasticsearch.ElasticSearchException in project graylog2-server by Graylog2.

the class Indices method getIndicesStats.

public Set<IndexStatistics> getIndicesStats(final IndexSet indexSet) {
    final Map<String, IndexStats> responseIndices;
    try {
        responseIndices = getAll(indexSet);
    } catch (ElasticsearchException e) {
        return Collections.emptySet();
    }
    final ImmutableSet.Builder<IndexStatistics> result = ImmutableSet.builder();
    for (IndexStats indexStats : responseIndices.values()) {
        final ImmutableList.Builder<ShardRouting> shardRouting = ImmutableList.builder();
        for (ShardStats shardStats : indexStats.getShards()) {
            shardRouting.add(shardStats.getShardRouting());
        }
        final IndexStatistics stats = IndexStatistics.create(indexStats.getIndex(), indexStats.getPrimaries(), indexStats.getTotal(), shardRouting.build());
        result.add(stats);
    }
    return result.build();
}
Also used : ShardStats(org.elasticsearch.action.admin.indices.stats.ShardStats) ImmutableSet(com.google.common.collect.ImmutableSet) ImmutableList(com.google.common.collect.ImmutableList) ElasticsearchException(org.elasticsearch.ElasticsearchException) IndexStats(org.elasticsearch.action.admin.indices.stats.IndexStats) ShardRouting(org.elasticsearch.cluster.routing.ShardRouting)

Example 79 with ElasticSearchException

use of org.elasticsearch.ElasticSearchException in project pentaho-kettle by pentaho.

the class ElasticSearchBulk method processBatch.

private boolean processBatch(boolean makeNew) throws KettleStepException {
    ListenableActionFuture<BulkResponse> actionFuture = currentRequest.execute();
    boolean responseOk = false;
    BulkResponse response = null;
    try {
        if (timeout != null && timeoutUnit != null) {
            response = actionFuture.actionGet(timeout, timeoutUnit);
        } else {
            response = actionFuture.actionGet();
        }
    } catch (ElasticsearchException e) {
        String msg = BaseMessages.getString(PKG, "ElasticSearchBulk.Error.BatchExecuteFail", e.getLocalizedMessage());
        if (e instanceof ElasticsearchTimeoutException) {
            msg = BaseMessages.getString(PKG, "ElasticSearchBulk.Error.Timeout");
        }
        logError(msg);
        rejectAllRows(msg);
    }
    if (response != null) {
        responseOk = handleResponse(response);
        requestsBuffer.clear();
    } else {
        // have to assume all failed
        numberOfErrors += currentRequest.numberOfActions();
        setErrors(numberOfErrors);
    }
    if (makeNew) {
        currentRequest = client.prepareBulk();
        data.nextBufferRowIdx = 0;
        data.inputRowBuffer = new Object[batchSize][];
    } else {
        currentRequest = null;
        data.inputRowBuffer = null;
    }
    return responseOk;
}
Also used : ElasticsearchTimeoutException(org.elasticsearch.ElasticsearchTimeoutException) BulkResponse(org.elasticsearch.action.bulk.BulkResponse) ElasticsearchException(org.elasticsearch.ElasticsearchException)

Example 80 with ElasticSearchException

use of org.elasticsearch.ElasticSearchException in project fess by codelibs.

the class FessEsClient method store.

public boolean store(final String index, final String type, final Object obj) {
    final FessConfig fessConfig = ComponentUtil.getFessConfig();
    @SuppressWarnings("unchecked") final Map<String, Object> source = obj instanceof Map ? (Map<String, Object>) obj : BeanUtil.copyBeanToNewMap(obj);
    final String id = (String) source.remove(fessConfig.getIndexFieldId());
    final Long version = (Long) source.remove(fessConfig.getIndexFieldVersion());
    IndexResponse response;
    try {
        if (id == null) {
            // create
            response = client.prepareIndex(index, type).setSource(new DocMap(source)).setRefreshPolicy(RefreshPolicy.IMMEDIATE).setOpType(OpType.CREATE).execute().actionGet(fessConfig.getIndexIndexTimeout());
        } else {
            // create or update
            final IndexRequestBuilder builder = client.prepareIndex(index, type, id).setSource(new DocMap(source)).setRefreshPolicy(RefreshPolicy.IMMEDIATE).setOpType(OpType.INDEX);
            if (version != null && version.longValue() > 0) {
                builder.setVersion(version);
            }
            response = builder.execute().actionGet(fessConfig.getIndexIndexTimeout());
        }
        final Result result = response.getResult();
        return result == Result.CREATED || result == Result.UPDATED;
    } catch (final ElasticsearchException e) {
        throw new FessEsClientException("Failed to store: " + obj, e);
    }
}
Also used : ElasticsearchException(org.elasticsearch.ElasticsearchException) FessConfig(org.codelibs.fess.mylasta.direction.FessConfig) Result(org.elasticsearch.action.DocWriteResponse.Result) IndexRequestBuilder(org.elasticsearch.action.index.IndexRequestBuilder) CreateIndexResponse(org.elasticsearch.action.admin.indices.create.CreateIndexResponse) GetIndexResponse(org.elasticsearch.action.admin.indices.get.GetIndexResponse) IndexResponse(org.elasticsearch.action.index.IndexResponse) DocMap(org.codelibs.fess.util.DocMap) Map(java.util.Map) DocMap(org.codelibs.fess.util.DocMap) ImmutableOpenMap(org.elasticsearch.common.collect.ImmutableOpenMap) HashMap(java.util.HashMap)

Aggregations

ElasticsearchException (org.elasticsearch.ElasticsearchException)309 IOException (java.io.IOException)127 Settings (org.elasticsearch.common.settings.Settings)32 XContentBuilder (org.elasticsearch.common.xcontent.XContentBuilder)30 HashMap (java.util.HashMap)29 ClusterState (org.elasticsearch.cluster.ClusterState)29 ArrayList (java.util.ArrayList)28 Matchers.containsString (org.hamcrest.Matchers.containsString)25 List (java.util.List)20 Map (java.util.Map)20 AtomicReference (java.util.concurrent.atomic.AtomicReference)20 ParameterizedMessage (org.apache.logging.log4j.message.ParameterizedMessage)18 XContentParser (org.elasticsearch.common.xcontent.XContentParser)17 Path (java.nio.file.Path)16 Test (org.junit.Test)16 ActionListener (org.elasticsearch.action.ActionListener)15 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)14 Version (org.elasticsearch.Version)14 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)13 ResourceNotFoundException (org.elasticsearch.ResourceNotFoundException)13