Search in sources :

Example 6 with IndexingPressureService

use of org.opensearch.index.IndexingPressureService in project OpenSearch by opensearch-project.

the class TransportWriteActionForIndexingPressureTests method testIndexingPressureOperationStartedForPrimaryNode.

public void testIndexingPressureOperationStartedForPrimaryNode() {
    final ShardId shardId = new ShardId("test", "_na_", 0);
    final ClusterState state = state(shardId.getIndexName(), true, ShardRoutingState.STARTED, ShardRoutingState.STARTED);
    setState(clusterService, state);
    final ShardRouting replicaRouting = state.getRoutingTable().shardRoutingTable(shardId).replicaShards().get(0);
    final ReplicationTask task = maybeTask();
    final Settings settings = Settings.builder().put(ShardIndexingPressureSettings.SHARD_INDEXING_PRESSURE_ENABLED.getKey(), false).build();
    this.indexingPressureService = new IndexingPressureService(settings, clusterService);
    TestAction action = new TestAction(settings, "internal:testActionWithExceptions", transportService, clusterService, shardStateAction, threadPool);
    action.handlePrimaryRequest(new TransportReplicationAction.ConcreteReplicaRequest<>(new TestRequest(), replicaRouting.allocationId().getId(), randomNonNegativeLong(), randomNonNegativeLong(), randomNonNegativeLong()), createTransportChannel(new PlainActionFuture<>()), task);
    IndexingPressurePerShardStats shardStats = this.indexingPressureService.shardStats(CommonStatsFlags.ALL).getIndexingPressureShardStats(shardId);
    assertPhase(task, "finished");
    assertTrue(Objects.isNull(shardStats));
}
Also used : ShardId(org.opensearch.index.shard.ShardId) ClusterState(org.opensearch.cluster.ClusterState) IndexingPressureService(org.opensearch.index.IndexingPressureService) PlainActionFuture(org.opensearch.action.support.PlainActionFuture) ShardRouting(org.opensearch.cluster.routing.ShardRouting) ShardIndexingPressureSettings(org.opensearch.index.ShardIndexingPressureSettings) Settings(org.opensearch.common.settings.Settings) ClusterSettings(org.opensearch.common.settings.ClusterSettings) IndexingPressurePerShardStats(org.opensearch.index.stats.IndexingPressurePerShardStats)

Example 7 with IndexingPressureService

use of org.opensearch.index.IndexingPressureService in project OpenSearch by opensearch-project.

the class TransportWriteActionForIndexingPressureTests method testIndexingPressureOperationStartedForLocalPrimaryShard.

public void testIndexingPressureOperationStartedForLocalPrimaryShard() {
    final ShardId shardId = new ShardId("test", "_na_", 0);
    final ClusterState state = state(shardId.getIndexName(), true, ShardRoutingState.STARTED, ShardRoutingState.STARTED);
    setState(clusterService, state);
    final ShardRouting replicaRouting = state.getRoutingTable().shardRoutingTable(shardId).replicaShards().get(0);
    final ReplicationTask task = maybeTask();
    final Settings settings = Settings.builder().put(ShardIndexingPressureSettings.SHARD_INDEXING_PRESSURE_ENABLED.getKey(), true).build();
    this.indexingPressureService = new IndexingPressureService(settings, clusterService);
    TestAction action = new TestAction(settings, "internal:testAction", transportService, clusterService, shardStateAction, threadPool);
    action.handlePrimaryRequest(new TransportReplicationAction.ConcreteShardRequest<>(new TestRequest(), replicaRouting.allocationId().getId(), randomNonNegativeLong(), true, true), createTransportChannel(new PlainActionFuture<>()), task);
    CommonStatsFlags statsFlag = new CommonStatsFlags();
    statsFlag.includeAllShardIndexingPressureTrackers(true);
    IndexingPressurePerShardStats shardStats = this.indexingPressureService.shardStats(statsFlag).getIndexingPressureShardStats(shardId);
    assertPhase(task, "finished");
    assertTrue(!Objects.isNull(shardStats));
}
Also used : ClusterState(org.opensearch.cluster.ClusterState) IndexingPressureService(org.opensearch.index.IndexingPressureService) CommonStatsFlags(org.opensearch.action.admin.indices.stats.CommonStatsFlags) ShardId(org.opensearch.index.shard.ShardId) PlainActionFuture(org.opensearch.action.support.PlainActionFuture) ShardRouting(org.opensearch.cluster.routing.ShardRouting) ShardIndexingPressureSettings(org.opensearch.index.ShardIndexingPressureSettings) Settings(org.opensearch.common.settings.Settings) ClusterSettings(org.opensearch.common.settings.ClusterSettings) IndexingPressurePerShardStats(org.opensearch.index.stats.IndexingPressurePerShardStats)

Example 8 with IndexingPressureService

use of org.opensearch.index.IndexingPressureService in project OpenSearch by opensearch-project.

the class TransportWriteActionForIndexingPressureTests method testIndexingPressureOperationStartedForReplicaShard.

public void testIndexingPressureOperationStartedForReplicaShard() {
    final ShardId shardId = new ShardId("test", "_na_", 0);
    final ClusterState state = state(shardId.getIndexName(), true, ShardRoutingState.STARTED, ShardRoutingState.STARTED);
    setState(clusterService, state);
    final ShardRouting replicaRouting = state.getRoutingTable().shardRoutingTable(shardId).replicaShards().get(0);
    final ReplicationTask task = maybeTask();
    final Settings settings = Settings.builder().put(ShardIndexingPressureSettings.SHARD_INDEXING_PRESSURE_ENABLED.getKey(), true).build();
    this.indexingPressureService = new IndexingPressureService(settings, clusterService);
    TestAction action = new TestAction(settings, "internal:testAction", transportService, clusterService, shardStateAction, threadPool);
    action.handleReplicaRequest(new TransportReplicationAction.ConcreteReplicaRequest<>(new TestRequest(), replicaRouting.allocationId().getId(), randomNonNegativeLong(), randomNonNegativeLong(), randomNonNegativeLong()), createTransportChannel(new PlainActionFuture<>()), task);
    CommonStatsFlags statsFlag = new CommonStatsFlags();
    statsFlag.includeAllShardIndexingPressureTrackers(true);
    IndexingPressurePerShardStats shardStats = this.indexingPressureService.shardStats(statsFlag).getIndexingPressureShardStats(shardId);
    assertPhase(task, "finished");
    assertTrue(!Objects.isNull(shardStats));
    assertEquals(100, shardStats.getTotalReplicaBytes());
}
Also used : ClusterState(org.opensearch.cluster.ClusterState) IndexingPressureService(org.opensearch.index.IndexingPressureService) CommonStatsFlags(org.opensearch.action.admin.indices.stats.CommonStatsFlags) ShardId(org.opensearch.index.shard.ShardId) PlainActionFuture(org.opensearch.action.support.PlainActionFuture) ShardRouting(org.opensearch.cluster.routing.ShardRouting) ShardIndexingPressureSettings(org.opensearch.index.ShardIndexingPressureSettings) Settings(org.opensearch.common.settings.Settings) ClusterSettings(org.opensearch.common.settings.ClusterSettings) IndexingPressurePerShardStats(org.opensearch.index.stats.IndexingPressurePerShardStats)

Example 9 with IndexingPressureService

use of org.opensearch.index.IndexingPressureService in project OpenSearch by opensearch-project.

the class RetentionLeaseSyncActionTests method testRetentionLeaseSyncActionOnReplica.

public void testRetentionLeaseSyncActionOnReplica() throws Exception {
    final IndicesService indicesService = mock(IndicesService.class);
    final Index index = new Index("index", "uuid");
    final IndexService indexService = mock(IndexService.class);
    when(indicesService.indexServiceSafe(index)).thenReturn(indexService);
    final int id = randomIntBetween(0, 4);
    final IndexShard indexShard = mock(IndexShard.class);
    when(indexService.getShard(id)).thenReturn(indexShard);
    final ShardId shardId = new ShardId(index, id);
    when(indexShard.shardId()).thenReturn(shardId);
    final RetentionLeaseSyncAction action = new RetentionLeaseSyncAction(Settings.EMPTY, transportService, clusterService, indicesService, threadPool, shardStateAction, new ActionFilters(Collections.emptySet()), new IndexingPressureService(Settings.EMPTY, clusterService), new SystemIndices(emptyMap()));
    final RetentionLeases retentionLeases = mock(RetentionLeases.class);
    final RetentionLeaseSyncAction.Request request = new RetentionLeaseSyncAction.Request(indexShard.shardId(), retentionLeases);
    PlainActionFuture<TransportReplicationAction.ReplicaResult> listener = PlainActionFuture.newFuture();
    action.dispatchedShardOperationOnReplica(request, indexShard, listener);
    final TransportReplicationAction.ReplicaResult result = listener.actionGet();
    // the retention leases on the shard should be updated
    verify(indexShard).updateRetentionLeasesOnReplica(retentionLeases);
    // the retention leases on the shard should be persisted
    verify(indexShard).persistRetentionLeases();
    // the result should indicate success
    final AtomicBoolean success = new AtomicBoolean();
    result.runPostReplicaActions(ActionListener.wrap(r -> success.set(true), e -> fail(e.toString())));
    assertTrue(success.get());
}
Also used : ThreadPool(org.opensearch.threadpool.ThreadPool) TestThreadPool(org.opensearch.threadpool.TestThreadPool) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) IndexShard(org.opensearch.index.shard.IndexShard) PlainActionFuture(org.opensearch.action.support.PlainActionFuture) ActionListener(org.opensearch.action.ActionListener) ShardStateAction(org.opensearch.cluster.action.shard.ShardStateAction) Collections.emptyMap(java.util.Collections.emptyMap) ClusterServiceUtils.createClusterService(org.opensearch.test.ClusterServiceUtils.createClusterService) Index(org.opensearch.index.Index) IndexingPressureService(org.opensearch.index.IndexingPressureService) OpenSearchTestCase(org.opensearch.test.OpenSearchTestCase) IndicesService(org.opensearch.indices.IndicesService) Settings(org.opensearch.common.settings.Settings) Mockito.when(org.mockito.Mockito.when) TransportReplicationAction(org.opensearch.action.support.replication.TransportReplicationAction) IndexService(org.opensearch.index.IndexService) TransportService(org.opensearch.transport.TransportService) Mockito.verify(org.mockito.Mockito.verify) IOUtils(org.opensearch.core.internal.io.IOUtils) ShardId(org.opensearch.index.shard.ShardId) ActionFilters(org.opensearch.action.support.ActionFilters) SystemIndices(org.opensearch.indices.SystemIndices) ActionTestUtils(org.opensearch.action.support.ActionTestUtils) Matchers.sameInstance(org.hamcrest.Matchers.sameInstance) ClusterService(org.opensearch.cluster.service.ClusterService) CapturingTransport(org.opensearch.test.transport.CapturingTransport) Collections(java.util.Collections) Mockito.mock(org.mockito.Mockito.mock) IndexingPressureService(org.opensearch.index.IndexingPressureService) IndexService(org.opensearch.index.IndexService) IndexShard(org.opensearch.index.shard.IndexShard) IndicesService(org.opensearch.indices.IndicesService) Index(org.opensearch.index.Index) ActionFilters(org.opensearch.action.support.ActionFilters) TransportReplicationAction(org.opensearch.action.support.replication.TransportReplicationAction) ShardId(org.opensearch.index.shard.ShardId) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) SystemIndices(org.opensearch.indices.SystemIndices)

Example 10 with IndexingPressureService

use of org.opensearch.index.IndexingPressureService in project OpenSearch by opensearch-project.

the class RetentionLeaseSyncActionTests method testBlocks.

public void testBlocks() {
    final IndicesService indicesService = mock(IndicesService.class);
    final Index index = new Index("index", "uuid");
    final IndexService indexService = mock(IndexService.class);
    when(indicesService.indexServiceSafe(index)).thenReturn(indexService);
    final int id = randomIntBetween(0, 4);
    final IndexShard indexShard = mock(IndexShard.class);
    when(indexService.getShard(id)).thenReturn(indexShard);
    final ShardId shardId = new ShardId(index, id);
    when(indexShard.shardId()).thenReturn(shardId);
    final RetentionLeaseSyncAction action = new RetentionLeaseSyncAction(Settings.EMPTY, transportService, clusterService, indicesService, threadPool, shardStateAction, new ActionFilters(Collections.emptySet()), new IndexingPressureService(Settings.EMPTY, clusterService), new SystemIndices(emptyMap()));
    assertNull(action.indexBlockLevel());
}
Also used : ShardId(org.opensearch.index.shard.ShardId) IndexingPressureService(org.opensearch.index.IndexingPressureService) IndexService(org.opensearch.index.IndexService) IndexShard(org.opensearch.index.shard.IndexShard) IndicesService(org.opensearch.indices.IndicesService) Index(org.opensearch.index.Index) ActionFilters(org.opensearch.action.support.ActionFilters) SystemIndices(org.opensearch.indices.SystemIndices)

Aggregations

IndexingPressureService (org.opensearch.index.IndexingPressureService)11 ShardId (org.opensearch.index.shard.ShardId)10 PlainActionFuture (org.opensearch.action.support.PlainActionFuture)9 Settings (org.opensearch.common.settings.Settings)8 ClusterState (org.opensearch.cluster.ClusterState)7 ShardRouting (org.opensearch.cluster.routing.ShardRouting)7 ClusterSettings (org.opensearch.common.settings.ClusterSettings)7 ShardIndexingPressureSettings (org.opensearch.index.ShardIndexingPressureSettings)6 IndexingPressurePerShardStats (org.opensearch.index.stats.IndexingPressurePerShardStats)6 ActionFilters (org.opensearch.action.support.ActionFilters)5 SystemIndices (org.opensearch.indices.SystemIndices)5 ActionListener (org.opensearch.action.ActionListener)4 ClusterService (org.opensearch.cluster.service.ClusterService)4 Index (org.opensearch.index.Index)4 IndexService (org.opensearch.index.IndexService)4 IndexShard (org.opensearch.index.shard.IndexShard)4 IndicesService (org.opensearch.indices.IndicesService)4 Collections.emptyMap (java.util.Collections.emptyMap)3 CommonStatsFlags (org.opensearch.action.admin.indices.stats.CommonStatsFlags)3 ShardStateAction (org.opensearch.cluster.action.shard.ShardStateAction)3