Search in sources :

Example 16 with FlushRequest

use of org.elasticsearch.action.admin.indices.flush.FlushRequest in project elasticsearch by elastic.

the class IndicesRequestIT method testFlush.

public void testFlush() {
    String[] indexShardActions = new String[] { TransportShardFlushAction.NAME, TransportShardFlushAction.NAME + "[r]", TransportShardFlushAction.NAME + "[p]" };
    interceptTransportActions(indexShardActions);
    FlushRequest flushRequest = new FlushRequest(randomIndicesOrAliases());
    internalCluster().coordOnlyNodeClient().admin().indices().flush(flushRequest).actionGet();
    clearInterceptedActions();
    String[] indices = new IndexNameExpressionResolver(Settings.EMPTY).concreteIndexNames(client().admin().cluster().prepareState().get().getState(), flushRequest);
    assertIndicesSubset(Arrays.asList(indices), indexShardActions);
}
Also used : FlushRequest(org.elasticsearch.action.admin.indices.flush.FlushRequest) IndexNameExpressionResolver(org.elasticsearch.cluster.metadata.IndexNameExpressionResolver)

Example 17 with FlushRequest

use of org.elasticsearch.action.admin.indices.flush.FlushRequest in project elasticsearch by elastic.

the class RecoveryDuringReplicationTests method testRecoveryOfDisconnectedReplica.

public void testRecoveryOfDisconnectedReplica() throws Exception {
    try (ReplicationGroup shards = createGroup(1)) {
        shards.startAll();
        int docs = shards.indexDocs(randomInt(50));
        shards.flush();
        shards.getPrimary().updateGlobalCheckpointOnPrimary();
        final IndexShard originalReplica = shards.getReplicas().get(0);
        long replicaCommittedLocalCheckpoint = docs - 1;
        boolean replicaHasDocsSinceLastFlushedCheckpoint = false;
        for (int i = 0; i < randomInt(2); i++) {
            final int indexedDocs = shards.indexDocs(randomInt(5));
            docs += indexedDocs;
            if (indexedDocs > 0) {
                replicaHasDocsSinceLastFlushedCheckpoint = true;
            }
            final boolean flush = randomBoolean();
            if (flush) {
                originalReplica.flush(new FlushRequest());
                replicaHasDocsSinceLastFlushedCheckpoint = false;
                replicaCommittedLocalCheckpoint = docs - 1;
            }
            final boolean sync = randomBoolean();
            if (sync) {
                shards.getPrimary().updateGlobalCheckpointOnPrimary();
            }
        }
        shards.removeReplica(originalReplica);
        final int missingOnReplica = shards.indexDocs(randomInt(5));
        docs += missingOnReplica;
        replicaHasDocsSinceLastFlushedCheckpoint |= missingOnReplica > 0;
        if (randomBoolean()) {
            shards.getPrimary().updateGlobalCheckpointOnPrimary();
        }
        final boolean flushPrimary = randomBoolean();
        if (flushPrimary) {
            shards.flush();
        }
        originalReplica.close("disconnected", false);
        IOUtils.close(originalReplica.store());
        final IndexShard recoveredReplica = shards.addReplicaWithExistingPath(originalReplica.shardPath(), originalReplica.routingEntry().currentNodeId());
        shards.recoverReplica(recoveredReplica);
        if (flushPrimary && replicaHasDocsSinceLastFlushedCheckpoint) {
            // replica has something to catch up with, but since we flushed the primary, we should fall back to full recovery
            assertThat(recoveredReplica.recoveryState().getIndex().fileDetails(), not(empty()));
        } else {
            assertThat(recoveredReplica.recoveryState().getIndex().fileDetails(), empty());
            assertThat(recoveredReplica.recoveryState().getTranslog().recoveredOperations(), equalTo(Math.toIntExact(docs - (replicaCommittedLocalCheckpoint + 1))));
        }
        docs += shards.indexDocs(randomInt(5));
        shards.assertAllEqual(docs);
    }
}
Also used : FlushRequest(org.elasticsearch.action.admin.indices.flush.FlushRequest) IndexShard(org.elasticsearch.index.shard.IndexShard)

Example 18 with FlushRequest

use of org.elasticsearch.action.admin.indices.flush.FlushRequest in project graylog2-server by Graylog2.

the class Indices method flush.

public void flush(String index) {
    FlushRequest flush = new FlushRequest(index);
    // Just flushes. Even if it is not necessary.
    flush.force(true);
    c.admin().indices().flush(new FlushRequest(index).force(true)).actionGet();
}
Also used : FlushRequest(org.elasticsearch.action.admin.indices.flush.FlushRequest)

Example 19 with FlushRequest

use of org.elasticsearch.action.admin.indices.flush.FlushRequest in project zipkin by openzipkin.

the class ElasticsearchDependenciesTest method writeDependencyLinks.

protected void writeDependencyLinks(List<DependencyLink> links, long timestampMillis) {
    long midnight = Util.midnightUTC(timestampMillis);
    TransportClient client = ((NativeClient) storage().client()).client;
    BulkRequestBuilder request = client.prepareBulk();
    for (DependencyLink link : links) {
        request.add(client.prepareIndex(storage().indexNameFormatter.indexNameForTimestamp(midnight), ElasticsearchConstants.DEPENDENCY_LINK).setId(// Unique constraint
        link.parent + "|" + link.child).setSource("parent", link.parent, "child", link.child, "callCount", link.callCount));
    }
    request.execute().actionGet();
    client.admin().indices().flush(new FlushRequest()).actionGet();
}
Also used : TransportClient(org.elasticsearch.client.transport.TransportClient) FlushRequest(org.elasticsearch.action.admin.indices.flush.FlushRequest) BulkRequestBuilder(org.elasticsearch.action.bulk.BulkRequestBuilder) DependencyLink(zipkin.DependencyLink)

Example 20 with FlushRequest

use of org.elasticsearch.action.admin.indices.flush.FlushRequest in project ranger by apache.

the class RequestUtils method getIndexFromRequest.

// To support all kinds of request in elasticsearch
public static <Request extends ActionRequest> List<String> getIndexFromRequest(Request request) {
    List<String> indexs = new ArrayList<>();
    if (request instanceof SingleShardRequest) {
        indexs.add(((SingleShardRequest<?>) request).index());
        return indexs;
    }
    if (request instanceof ReplicationRequest) {
        indexs.add(((ReplicationRequest<?>) request).index());
        return indexs;
    }
    if (request instanceof InstanceShardOperationRequest) {
        indexs.add(((InstanceShardOperationRequest<?>) request).index());
        return indexs;
    }
    if (request instanceof CreateIndexRequest) {
        indexs.add(((CreateIndexRequest) request).index());
        return indexs;
    }
    if (request instanceof PutMappingRequest) {
        if (((PutMappingRequest) request).getConcreteIndex() != null) {
            indexs.add(((PutMappingRequest) request).getConcreteIndex().getName());
            return indexs;
        } else {
            return Arrays.asList(((PutMappingRequest) request).indices());
        }
    }
    if (request instanceof SearchRequest) {
        return Arrays.asList(((SearchRequest) request).indices());
    }
    if (request instanceof IndicesStatsRequest) {
        return Arrays.asList(((IndicesStatsRequest) request).indices());
    }
    if (request instanceof OpenIndexRequest) {
        return Arrays.asList(((OpenIndexRequest) request).indices());
    }
    if (request instanceof DeleteIndexRequest) {
        return Arrays.asList(((DeleteIndexRequest) request).indices());
    }
    if (request instanceof BulkRequest) {
        @SuppressWarnings("rawtypes") List<DocWriteRequest<?>> requests = ((BulkRequest) request).requests();
        if (CollectionUtils.isNotEmpty(requests)) {
            for (DocWriteRequest<?> docWriteRequest : requests) {
                indexs.add(docWriteRequest.index());
            }
            return indexs;
        }
    }
    if (request instanceof MultiGetRequest) {
        List<Item> items = ((MultiGetRequest) request).getItems();
        if (CollectionUtils.isNotEmpty(items)) {
            for (Item item : items) {
                indexs.add(item.index());
            }
            return indexs;
        }
    }
    if (request instanceof GetMappingsRequest) {
        return Arrays.asList(((GetMappingsRequest) request).indices());
    }
    if (request instanceof GetSettingsRequest) {
        return Arrays.asList(((GetSettingsRequest) request).indices());
    }
    if (request instanceof IndicesExistsRequest) {
        return Arrays.asList(((IndicesExistsRequest) request).indices());
    }
    if (request instanceof GetAliasesRequest) {
        return Arrays.asList(((GetAliasesRequest) request).indices());
    }
    if (request instanceof GetIndexRequest) {
        return Arrays.asList(((GetIndexRequest) request).indices());
    }
    if (request instanceof GetFieldMappingsRequest) {
        return Arrays.asList(((GetFieldMappingsRequest) request).indices());
    }
    if (request instanceof TypesExistsRequest) {
        return Arrays.asList(((TypesExistsRequest) request).indices());
    }
    if (request instanceof ValidateQueryRequest) {
        return Arrays.asList(((ValidateQueryRequest) request).indices());
    }
    if (request instanceof RecoveryRequest) {
        return Arrays.asList(((RecoveryRequest) request).indices());
    }
    if (request instanceof IndicesSegmentsRequest) {
        return Arrays.asList(((IndicesSegmentsRequest) request).indices());
    }
    if (request instanceof IndicesShardStoresRequest) {
        return Arrays.asList(((IndicesShardStoresRequest) request).indices());
    }
    if (request instanceof UpgradeStatusRequest) {
        return Arrays.asList(((UpgradeStatusRequest) request).indices());
    }
    if (request instanceof ClusterSearchShardsRequest) {
        return Arrays.asList(((ClusterSearchShardsRequest) request).indices());
    }
    if (request instanceof IndicesAliasesRequest) {
        List<IndicesAliasesRequest.AliasActions> aliasActions = ((IndicesAliasesRequest) request).getAliasActions();
        if (CollectionUtils.isNotEmpty(aliasActions)) {
            for (IndicesAliasesRequest.AliasActions action : aliasActions) {
                indexs.addAll(Arrays.asList(action.indices()));
            }
            return indexs;
        }
    }
    if (request instanceof ClearIndicesCacheRequest) {
        return Arrays.asList(((ClearIndicesCacheRequest) request).indices());
    }
    if (request instanceof CloseIndexRequest) {
        return Arrays.asList(((CloseIndexRequest) request).indices());
    }
    if (request instanceof FlushRequest) {
        return Arrays.asList(((FlushRequest) request).indices());
    }
    if (request instanceof SyncedFlushRequest) {
        return Arrays.asList(((SyncedFlushRequest) request).indices());
    }
    if (request instanceof ForceMergeRequest) {
        return Arrays.asList(((ForceMergeRequest) request).indices());
    }
    if (request instanceof RefreshRequest) {
        return Arrays.asList(((RefreshRequest) request).indices());
    }
    if (request instanceof RolloverRequest) {
        return Arrays.asList(((RolloverRequest) request).indices());
    }
    if (request instanceof UpdateSettingsRequest) {
        return Arrays.asList(((UpdateSettingsRequest) request).indices());
    }
    if (request instanceof ResizeRequest) {
        return Arrays.asList(((ResizeRequest) request).indices());
    }
    if (request instanceof DeleteIndexTemplateRequest) {
        indexs.add(((DeleteIndexTemplateRequest) request).name());
        return indexs;
    }
    if (request instanceof GetIndexTemplatesRequest) {
        return Arrays.asList(((GetIndexTemplatesRequest) request).names());
    }
    if (request instanceof PutIndexTemplateRequest) {
        indexs.add(((PutIndexTemplateRequest) request).name());
        return indexs;
    }
    if (request instanceof UpgradeRequest) {
        return Arrays.asList(((UpgradeRequest) request).indices());
    }
    if (request instanceof FieldCapabilitiesRequest) {
        return Arrays.asList(((FieldCapabilitiesRequest) request).indices());
    }
    if (request instanceof MultiSearchRequest) {
        List<SearchRequest> searchRequests = ((MultiSearchRequest) request).requests();
        if (CollectionUtils.isNotEmpty(searchRequests)) {
            for (SearchRequest singleRequest : searchRequests) {
                indexs.addAll(Arrays.asList(singleRequest.indices()));
            }
            return indexs;
        }
    }
    if (request instanceof MultiTermVectorsRequest) {
        List<TermVectorsRequest> termVectorsRequests = ((MultiTermVectorsRequest) request).getRequests();
        if (CollectionUtils.isNotEmpty(termVectorsRequests)) {
            for (TermVectorsRequest singleRequest : termVectorsRequests) {
                indexs.addAll(Arrays.asList(singleRequest.indices()));
            }
            return indexs;
        }
    }
    if (request instanceof UpdateByQueryRequest) {
        return Arrays.asList(((UpdateByQueryRequest) request).indices());
    }
    if (request instanceof DeleteByQueryRequest) {
        return Arrays.asList(((DeleteByQueryRequest) request).indices());
    }
    if (request instanceof ReindexRequest) {
        indexs.addAll(Arrays.asList(((ReindexRequest) request).getSearchRequest().indices()));
        indexs.addAll(Arrays.asList(((ReindexRequest) request).getDestination().indices()));
        return indexs;
    }
    // ClearScrollRequest does not carry any index, so return empty List
    if (request instanceof ClearScrollRequest) {
        return indexs;
    }
    // No matched request type to find specific index , set default value *
    indexs.add("*");
    return indexs;
}
Also used : ForceMergeRequest(org.elasticsearch.action.admin.indices.forcemerge.ForceMergeRequest) IndicesShardStoresRequest(org.elasticsearch.action.admin.indices.shards.IndicesShardStoresRequest) TypesExistsRequest(org.elasticsearch.action.admin.indices.exists.types.TypesExistsRequest) ClearIndicesCacheRequest(org.elasticsearch.action.admin.indices.cache.clear.ClearIndicesCacheRequest) ArrayList(java.util.ArrayList) DeleteIndexRequest(org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest) ReplicationRequest(org.elasticsearch.action.support.replication.ReplicationRequest) DeleteIndexTemplateRequest(org.elasticsearch.action.admin.indices.template.delete.DeleteIndexTemplateRequest) SyncedFlushRequest(org.elasticsearch.action.admin.indices.flush.SyncedFlushRequest) InstanceShardOperationRequest(org.elasticsearch.action.support.single.instance.InstanceShardOperationRequest) MultiGetRequest(org.elasticsearch.action.get.MultiGetRequest) Item(org.elasticsearch.action.get.MultiGetRequest.Item) ReindexRequest(org.elasticsearch.index.reindex.ReindexRequest) SingleShardRequest(org.elasticsearch.action.support.single.shard.SingleShardRequest) RolloverRequest(org.elasticsearch.action.admin.indices.rollover.RolloverRequest) GetFieldMappingsRequest(org.elasticsearch.action.admin.indices.mapping.get.GetFieldMappingsRequest) IndicesSegmentsRequest(org.elasticsearch.action.admin.indices.segments.IndicesSegmentsRequest) ClusterSearchShardsRequest(org.elasticsearch.action.admin.cluster.shards.ClusterSearchShardsRequest) IndicesAliasesRequest(org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest) DocWriteRequest(org.elasticsearch.action.DocWriteRequest) RefreshRequest(org.elasticsearch.action.admin.indices.refresh.RefreshRequest) TermVectorsRequest(org.elasticsearch.action.termvectors.TermVectorsRequest) MultiTermVectorsRequest(org.elasticsearch.action.termvectors.MultiTermVectorsRequest) SearchRequest(org.elasticsearch.action.search.SearchRequest) MultiSearchRequest(org.elasticsearch.action.search.MultiSearchRequest) PutMappingRequest(org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest) UpdateSettingsRequest(org.elasticsearch.action.admin.indices.settings.put.UpdateSettingsRequest) UpgradeRequest(org.elasticsearch.action.admin.indices.upgrade.post.UpgradeRequest) GetIndexTemplatesRequest(org.elasticsearch.action.admin.indices.template.get.GetIndexTemplatesRequest) PutIndexTemplateRequest(org.elasticsearch.action.admin.indices.template.put.PutIndexTemplateRequest) RecoveryRequest(org.elasticsearch.action.admin.indices.recovery.RecoveryRequest) SyncedFlushRequest(org.elasticsearch.action.admin.indices.flush.SyncedFlushRequest) FlushRequest(org.elasticsearch.action.admin.indices.flush.FlushRequest) UpgradeStatusRequest(org.elasticsearch.action.admin.indices.upgrade.get.UpgradeStatusRequest) GetIndexRequest(org.elasticsearch.action.admin.indices.get.GetIndexRequest) GetAliasesRequest(org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest) ClearScrollRequest(org.elasticsearch.action.search.ClearScrollRequest) OpenIndexRequest(org.elasticsearch.action.admin.indices.open.OpenIndexRequest) DeleteByQueryRequest(org.elasticsearch.index.reindex.DeleteByQueryRequest) ValidateQueryRequest(org.elasticsearch.action.admin.indices.validate.query.ValidateQueryRequest) GetMappingsRequest(org.elasticsearch.action.admin.indices.mapping.get.GetMappingsRequest) UpdateByQueryRequest(org.elasticsearch.index.reindex.UpdateByQueryRequest) GetSettingsRequest(org.elasticsearch.action.admin.indices.settings.get.GetSettingsRequest) CloseIndexRequest(org.elasticsearch.action.admin.indices.close.CloseIndexRequest) BulkRequest(org.elasticsearch.action.bulk.BulkRequest) MultiTermVectorsRequest(org.elasticsearch.action.termvectors.MultiTermVectorsRequest) MultiSearchRequest(org.elasticsearch.action.search.MultiSearchRequest) FieldCapabilitiesRequest(org.elasticsearch.action.fieldcaps.FieldCapabilitiesRequest) CreateIndexRequest(org.elasticsearch.action.admin.indices.create.CreateIndexRequest) IndicesExistsRequest(org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequest) ResizeRequest(org.elasticsearch.action.admin.indices.shrink.ResizeRequest) IndicesStatsRequest(org.elasticsearch.action.admin.indices.stats.IndicesStatsRequest)

Aggregations

FlushRequest (org.elasticsearch.action.admin.indices.flush.FlushRequest)22 IOException (java.io.IOException)7 Engine (org.elasticsearch.index.engine.Engine)6 IndexShard (org.elasticsearch.index.shard.IndexShard)5 ClosedByInterruptException (java.nio.channels.ClosedByInterruptException)3 ArrayList (java.util.ArrayList)3 TimeoutException (java.util.concurrent.TimeoutException)3 ElasticsearchException (org.elasticsearch.ElasticsearchException)3 ForceMergeRequest (org.elasticsearch.action.admin.indices.forcemerge.ForceMergeRequest)3 BytesArray (org.elasticsearch.common.bytes.BytesArray)3 Settings (org.elasticsearch.common.settings.Settings)3 AbstractRunnable (org.elasticsearch.common.util.concurrent.AbstractRunnable)3 IndexNotFoundException (org.elasticsearch.index.IndexNotFoundException)3 Term (org.apache.lucene.index.Term)2 AlreadyClosedException (org.apache.lucene.store.AlreadyClosedException)2 ThreadInterruptedException (org.apache.lucene.util.ThreadInterruptedException)2 UpgradeRequest (org.elasticsearch.action.admin.indices.upgrade.post.UpgradeRequest)2 EngineException (org.elasticsearch.index.engine.EngineException)2 RefreshFailedEngineException (org.elasticsearch.index.engine.RefreshFailedEngineException)2 ParsedDocument (org.elasticsearch.index.mapper.ParsedDocument)2