Search in sources :

Example 1 with BulkItemRequest

use of org.opensearch.action.bulk.BulkItemRequest in project OpenSearch by opensearch-project.

the class IndexingPressureServiceTests method testCoordinatingOperationForShardIndexingPressure.

public void testCoordinatingOperationForShardIndexingPressure() {
    IndexingPressureService service = new IndexingPressureService(settings, clusterService);
    Index index = new Index("IndexName", "UUID");
    ShardId shardId = new ShardId(index, 0);
    BulkItemRequest[] items = new BulkItemRequest[1];
    DocWriteRequest<IndexRequest> writeRequest = new IndexRequest("index").id("id").source(Requests.INDEX_CONTENT_TYPE, "foo", "bar");
    items[0] = new BulkItemRequest(0, writeRequest);
    BulkShardRequest bulkShardRequest = new BulkShardRequest(shardId, WriteRequest.RefreshPolicy.NONE, items);
    Releasable releasable = service.markCoordinatingOperationStarted(shardId, bulkShardRequest::ramBytesUsed, false);
    IndexingPressurePerShardStats shardStats = service.shardStats(CommonStatsFlags.ALL).getIndexingPressureShardStats(shardId);
    assertEquals(bulkShardRequest.ramBytesUsed(), shardStats.getCurrentCoordinatingBytes());
    releasable.close();
}
Also used : ShardId(org.opensearch.index.shard.ShardId) BulkShardRequest(org.opensearch.action.bulk.BulkShardRequest) BulkItemRequest(org.opensearch.action.bulk.BulkItemRequest) Releasable(org.opensearch.common.lease.Releasable) IndexRequest(org.opensearch.action.index.IndexRequest) IndexingPressurePerShardStats(org.opensearch.index.stats.IndexingPressurePerShardStats)

Example 2 with BulkItemRequest

use of org.opensearch.action.bulk.BulkItemRequest in project OpenSearch by opensearch-project.

the class OpenSearchIndexLevelReplicationTestCase method executeShardBulkOnPrimary.

private void executeShardBulkOnPrimary(IndexShard primary, BulkShardRequest request, ActionListener<TransportWriteAction.WritePrimaryResult<BulkShardRequest, BulkShardResponse>> listener) {
    for (BulkItemRequest itemRequest : request.items()) {
        if (itemRequest.request() instanceof IndexRequest) {
            ((IndexRequest) itemRequest.request()).process(Version.CURRENT, null, index.getName());
        }
    }
    final PlainActionFuture<Releasable> permitAcquiredFuture = new PlainActionFuture<>();
    primary.acquirePrimaryOperationPermit(permitAcquiredFuture, ThreadPool.Names.SAME, request);
    try (Releasable ignored = permitAcquiredFuture.actionGet()) {
        MappingUpdatePerformer noopMappingUpdater = (update, shardId, listener1) -> {
        };
        TransportShardBulkAction.performOnPrimary(request, primary, null, System::currentTimeMillis, noopMappingUpdater, null, ActionTestUtils.assertNoFailureListener(result -> {
            TransportWriteActionTestHelper.performPostWriteActions(primary, request, ((TransportWriteAction.WritePrimaryResult<BulkShardRequest, BulkShardResponse>) result).location, logger);
            listener.onResponse((TransportWriteAction.WritePrimaryResult<BulkShardRequest, BulkShardResponse>) result);
        }), threadPool, Names.WRITE);
    } catch (Exception e) {
        listener.onFailure(e);
    }
}
Also used : ResyncReplicationResponse(org.opensearch.action.resync.ResyncReplicationResponse) ReplicatedWriteRequest(org.opensearch.action.support.replication.ReplicatedWriteRequest) BiFunction(java.util.function.BiFunction) AlreadyClosedException(org.apache.lucene.store.AlreadyClosedException) BulkShardResponse(org.opensearch.action.bulk.BulkShardResponse) Version(org.opensearch.Version) Releasables(org.opensearch.common.lease.Releasables) PrimaryReplicaSyncer(org.opensearch.index.shard.PrimaryReplicaSyncer) DiscoveryNode(org.opensearch.cluster.node.DiscoveryNode) IndexShardTestCase(org.opensearch.index.shard.IndexShardTestCase) Future(java.util.concurrent.Future) WriteRequest(org.opensearch.action.support.WriteRequest) PlainActionFuture(org.opensearch.action.support.PlainActionFuture) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) BulkItemResponse(org.opensearch.action.bulk.BulkItemResponse) RecoveryState(org.opensearch.indices.recovery.RecoveryState) RetentionLeaseSyncAction(org.opensearch.index.seqno.RetentionLeaseSyncAction) Map(java.util.Map) ReplicationRequest(org.opensearch.action.support.replication.ReplicationRequest) ActionListener(org.opensearch.action.ActionListener) BulkItemRequest(org.opensearch.action.bulk.BulkItemRequest) DeleteRequest(org.opensearch.action.delete.DeleteRequest) IndexShardRoutingTable(org.opensearch.cluster.routing.IndexShardRoutingTable) TimeValue(org.opensearch.common.unit.TimeValue) Iterators(org.opensearch.common.collect.Iterators) Index(org.opensearch.index.Index) BulkShardRequest(org.opensearch.action.bulk.BulkShardRequest) TransportResyncReplicationAction(org.opensearch.action.resync.TransportResyncReplicationAction) Set(java.util.Set) Settings(org.opensearch.common.settings.Settings) DiscoveryNodeRole(org.opensearch.cluster.node.DiscoveryNodeRole) Collectors(java.util.stream.Collectors) TaskManager(org.opensearch.tasks.TaskManager) List(java.util.List) EngineConfigFactory(org.opensearch.index.engine.EngineConfigFactory) ActionTestUtils(org.opensearch.action.support.ActionTestUtils) GlobalCheckpointSyncAction(org.opensearch.index.seqno.GlobalCheckpointSyncAction) Matchers.equalTo(org.hamcrest.Matchers.equalTo) IndexSettings(org.opensearch.index.IndexSettings) ReplicationResponse(org.opensearch.action.support.replication.ReplicationResponse) DocWriteResponse(org.opensearch.action.DocWriteResponse) XContentType(org.opensearch.common.xcontent.XContentType) ShardRoutingHelper(org.opensearch.cluster.routing.ShardRoutingHelper) Names(org.opensearch.threadpool.ThreadPool.Names) CopyOnWriteArrayList(java.util.concurrent.CopyOnWriteArrayList) IndexMetadata(org.opensearch.cluster.metadata.IndexMetadata) FlushRequest(org.opensearch.action.admin.indices.flush.FlushRequest) TransportWriteActionTestHelper(org.opensearch.action.support.replication.TransportWriteActionTestHelper) ThreadPool(org.opensearch.threadpool.ThreadPool) DocWriteRequest(org.opensearch.action.DocWriteRequest) FutureTask(java.util.concurrent.FutureTask) Releasable(org.opensearch.common.lease.Releasable) Function(java.util.function.Function) ArrayList(java.util.ArrayList) RecoverySource(org.opensearch.cluster.routing.RecoverySource) HashSet(java.util.HashSet) DocIdSeqNoAndSource(org.opensearch.index.engine.DocIdSeqNoAndSource) TransportShardBulkAction(org.opensearch.action.bulk.TransportShardBulkAction) IndexShard(org.opensearch.index.shard.IndexShard) ReplicationOperation(org.opensearch.action.support.replication.ReplicationOperation) ShardRoutingState(org.opensearch.cluster.routing.ShardRoutingState) BiConsumer(java.util.function.BiConsumer) Translog(org.opensearch.index.translog.Translog) TransportWriteAction(org.opensearch.action.support.replication.TransportWriteAction) StreamSupport(java.util.stream.StreamSupport) ShardPath(org.opensearch.index.shard.ShardPath) EngineFactory(org.opensearch.index.engine.EngineFactory) RetentionLease(org.opensearch.index.seqno.RetentionLease) StreamInput(org.opensearch.common.io.stream.StreamInput) InternalEngineFactory(org.opensearch.index.engine.InternalEngineFactory) Matchers.empty(org.hamcrest.Matchers.empty) Matchers.greaterThanOrEqualTo(org.hamcrest.Matchers.greaterThanOrEqualTo) RecoveryTarget(org.opensearch.indices.recovery.RecoveryTarget) Iterator(java.util.Iterator) PendingReplicationActions(org.opensearch.action.support.replication.PendingReplicationActions) AllocationId(org.opensearch.cluster.routing.AllocationId) MappingUpdatePerformer(org.opensearch.action.bulk.MappingUpdatePerformer) IOException(java.io.IOException) TransportReplicationAction(org.opensearch.action.support.replication.TransportReplicationAction) BytesStreamOutput(org.opensearch.common.io.stream.BytesStreamOutput) ReplicaResponse(org.opensearch.action.support.replication.TransportReplicationAction.ReplicaResponse) ShardRouting(org.opensearch.cluster.routing.ShardRouting) ShardId(org.opensearch.index.shard.ShardId) TestShardRouting(org.opensearch.cluster.routing.TestShardRouting) RetentionLeases(org.opensearch.index.seqno.RetentionLeases) ResyncReplicationRequest(org.opensearch.action.resync.ResyncReplicationRequest) IndexRequest(org.opensearch.action.index.IndexRequest) RetentionLeaseSyncer(org.opensearch.index.seqno.RetentionLeaseSyncer) Collections(java.util.Collections) BulkItemRequest(org.opensearch.action.bulk.BulkItemRequest) MappingUpdatePerformer(org.opensearch.action.bulk.MappingUpdatePerformer) PlainActionFuture(org.opensearch.action.support.PlainActionFuture) Releasable(org.opensearch.common.lease.Releasable) IndexRequest(org.opensearch.action.index.IndexRequest) AlreadyClosedException(org.apache.lucene.store.AlreadyClosedException) IOException(java.io.IOException)

Example 3 with BulkItemRequest

use of org.opensearch.action.bulk.BulkItemRequest in project OpenSearch by opensearch-project.

the class OpenSearchIndexLevelReplicationTestCase method executeReplicationRequestOnPrimary.

private <Request extends ReplicatedWriteRequest & DocWriteRequest> BulkShardRequest executeReplicationRequestOnPrimary(IndexShard primary, Request request) throws Exception {
    final BulkShardRequest bulkShardRequest = new BulkShardRequest(shardId, request.getRefreshPolicy(), new BulkItemRequest[] { new BulkItemRequest(0, request) });
    final PlainActionFuture<BulkShardRequest> res = new PlainActionFuture<>();
    executeShardBulkOnPrimary(primary, bulkShardRequest, ActionListener.map(res, TransportReplicationAction.PrimaryResult::replicaRequest));
    return res.get();
}
Also used : BulkShardRequest(org.opensearch.action.bulk.BulkShardRequest) BulkItemRequest(org.opensearch.action.bulk.BulkItemRequest) PlainActionFuture(org.opensearch.action.support.PlainActionFuture) TransportReplicationAction(org.opensearch.action.support.replication.TransportReplicationAction)

Aggregations

BulkItemRequest (org.opensearch.action.bulk.BulkItemRequest)3 BulkShardRequest (org.opensearch.action.bulk.BulkShardRequest)3 IndexRequest (org.opensearch.action.index.IndexRequest)2 PlainActionFuture (org.opensearch.action.support.PlainActionFuture)2 TransportReplicationAction (org.opensearch.action.support.replication.TransportReplicationAction)2 IOException (java.io.IOException)1 ArrayList (java.util.ArrayList)1 Collections (java.util.Collections)1 HashSet (java.util.HashSet)1 Iterator (java.util.Iterator)1 List (java.util.List)1 Map (java.util.Map)1 Set (java.util.Set)1 CopyOnWriteArrayList (java.util.concurrent.CopyOnWriteArrayList)1 Future (java.util.concurrent.Future)1 FutureTask (java.util.concurrent.FutureTask)1 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)1 BiConsumer (java.util.function.BiConsumer)1 BiFunction (java.util.function.BiFunction)1 Function (java.util.function.Function)1