Search in sources :

Example 36 with Releasable

use of org.opensearch.common.lease.Releasable in project OpenSearch by opensearch-project.

the class IndexingPressureTests method testAvoidDoubleAccounting.

public void testAvoidDoubleAccounting() {
    IndexingPressure indexingPressure = new IndexingPressure(settings);
    try (Releasable coordinating = indexingPressure.markCoordinatingOperationStarted(10, false);
        Releasable primary = indexingPressure.markPrimaryOperationLocalToCoordinatingNodeStarted(15)) {
        IndexingPressureStats stats = indexingPressure.stats();
        assertEquals(10, stats.getCurrentCoordinatingBytes());
        assertEquals(15, stats.getCurrentPrimaryBytes());
        assertEquals(10, stats.getCurrentCombinedCoordinatingAndPrimaryBytes());
    }
    IndexingPressureStats stats = indexingPressure.stats();
    assertEquals(0, stats.getCurrentCoordinatingBytes());
    assertEquals(0, stats.getCurrentPrimaryBytes());
    assertEquals(0, stats.getCurrentCombinedCoordinatingAndPrimaryBytes());
    assertEquals(10, stats.getTotalCoordinatingBytes());
    assertEquals(15, stats.getTotalPrimaryBytes());
    assertEquals(10, stats.getTotalCombinedCoordinatingAndPrimaryBytes());
}
Also used : Releasable(org.opensearch.common.lease.Releasable) IndexingPressureStats(org.opensearch.index.stats.IndexingPressureStats)

Example 37 with Releasable

use of org.opensearch.common.lease.Releasable in project OpenSearch by opensearch-project.

the class IndexingPressureServiceTests method testPrimaryOperationForShardIndexingPressure.

public void testPrimaryOperationForShardIndexingPressure() {
    IndexingPressureService service = new IndexingPressureService(settings, clusterService);
    Index index = new Index("IndexName", "UUID");
    ShardId shardId = new ShardId(index, 0);
    Releasable releasable = service.markPrimaryOperationStarted(shardId, 1024, false);
    IndexingPressurePerShardStats shardStats = service.shardStats(CommonStatsFlags.ALL).getIndexingPressureShardStats(shardId);
    assertEquals(1024, shardStats.getCurrentPrimaryBytes());
    releasable.close();
}
Also used : ShardId(org.opensearch.index.shard.ShardId) Releasable(org.opensearch.common.lease.Releasable) IndexingPressurePerShardStats(org.opensearch.index.stats.IndexingPressurePerShardStats)

Example 38 with Releasable

use of org.opensearch.common.lease.Releasable in project OpenSearch by opensearch-project.

the class IndexingPressureServiceTests method testReplicaOperationForShardIndexingPressure.

public void testReplicaOperationForShardIndexingPressure() {
    IndexingPressureService service = new IndexingPressureService(settings, clusterService);
    Index index = new Index("IndexName", "UUID");
    ShardId shardId = new ShardId(index, 0);
    Releasable releasable = service.markReplicaOperationStarted(shardId, 1024, false);
    IndexingPressurePerShardStats shardStats = service.shardStats(CommonStatsFlags.ALL).getIndexingPressureShardStats(shardId);
    assertEquals(1024, shardStats.getCurrentReplicaBytes());
    releasable.close();
}
Also used : ShardId(org.opensearch.index.shard.ShardId) Releasable(org.opensearch.common.lease.Releasable) IndexingPressurePerShardStats(org.opensearch.index.stats.IndexingPressurePerShardStats)

Example 39 with Releasable

use of org.opensearch.common.lease.Releasable in project OpenSearch by opensearch-project.

the class IndexingPressureServiceTests method testCoordinatingOperationForShardIndexingPressure.

public void testCoordinatingOperationForShardIndexingPressure() {
    IndexingPressureService service = new IndexingPressureService(settings, clusterService);
    Index index = new Index("IndexName", "UUID");
    ShardId shardId = new ShardId(index, 0);
    BulkItemRequest[] items = new BulkItemRequest[1];
    DocWriteRequest<IndexRequest> writeRequest = new IndexRequest("index").id("id").source(Requests.INDEX_CONTENT_TYPE, "foo", "bar");
    items[0] = new BulkItemRequest(0, writeRequest);
    BulkShardRequest bulkShardRequest = new BulkShardRequest(shardId, WriteRequest.RefreshPolicy.NONE, items);
    Releasable releasable = service.markCoordinatingOperationStarted(shardId, bulkShardRequest::ramBytesUsed, false);
    IndexingPressurePerShardStats shardStats = service.shardStats(CommonStatsFlags.ALL).getIndexingPressureShardStats(shardId);
    assertEquals(bulkShardRequest.ramBytesUsed(), shardStats.getCurrentCoordinatingBytes());
    releasable.close();
}
Also used : ShardId(org.opensearch.index.shard.ShardId) BulkShardRequest(org.opensearch.action.bulk.BulkShardRequest) BulkItemRequest(org.opensearch.action.bulk.BulkItemRequest) Releasable(org.opensearch.common.lease.Releasable) IndexRequest(org.opensearch.action.index.IndexRequest) IndexingPressurePerShardStats(org.opensearch.index.stats.IndexingPressurePerShardStats)

Example 40 with Releasable

use of org.opensearch.common.lease.Releasable in project OpenSearch by opensearch-project.

the class IndexingPressureServiceTests method testLocalPrimaryOperationForIndexingPressure.

public void testLocalPrimaryOperationForIndexingPressure() {
    IndexingPressureService service = new IndexingPressureService(settings, clusterService);
    Index index = new Index("IndexName", "UUID");
    ShardId shardId = new ShardId(index, 0);
    Settings.Builder updated = Settings.builder();
    clusterSettings.updateDynamicSettings(Settings.builder().put(ShardIndexingPressureSettings.SHARD_INDEXING_PRESSURE_ENABLED.getKey(), false).build(), Settings.builder().put(settings), updated, getTestClass().getName());
    clusterSettings.applySettings(updated.build());
    Releasable releasable = service.markPrimaryOperationLocalToCoordinatingNodeStarted(shardId, 1024);
    IndexingPressurePerShardStats shardStats = service.shardStats(CommonStatsFlags.ALL).getIndexingPressureShardStats(shardId);
    assertNull(shardStats);
    IndexingPressureStats nodeStats = service.nodeStats();
    assertEquals(1024, nodeStats.getCurrentPrimaryBytes());
    releasable.close();
}
Also used : ShardId(org.opensearch.index.shard.ShardId) Releasable(org.opensearch.common.lease.Releasable) IndexingPressureStats(org.opensearch.index.stats.IndexingPressureStats) Settings(org.opensearch.common.settings.Settings) ClusterSettings(org.opensearch.common.settings.ClusterSettings) IndexingPressurePerShardStats(org.opensearch.index.stats.IndexingPressurePerShardStats)

Aggregations

Releasable (org.opensearch.common.lease.Releasable)161 ShardId (org.opensearch.index.shard.ShardId)50 IOException (java.io.IOException)45 CountDownLatch (java.util.concurrent.CountDownLatch)36 Settings (org.opensearch.common.settings.Settings)35 IndexingPressurePerShardStats (org.opensearch.index.stats.IndexingPressurePerShardStats)32 ExecutionException (java.util.concurrent.ExecutionException)30 ArrayList (java.util.ArrayList)28 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)28 AlreadyClosedException (org.apache.lucene.store.AlreadyClosedException)27 OpenSearchException (org.opensearch.OpenSearchException)25 ThreadPool (org.opensearch.threadpool.ThreadPool)25 PlainActionFuture (org.opensearch.action.support.PlainActionFuture)24 ActionListener (org.opensearch.action.ActionListener)23 IndexRequest (org.opensearch.action.index.IndexRequest)22 ShardRouting (org.opensearch.cluster.routing.ShardRouting)22 IndexingPressureStats (org.opensearch.index.stats.IndexingPressureStats)22 BrokenBarrierException (java.util.concurrent.BrokenBarrierException)21 List (java.util.List)20 Translog (org.opensearch.index.translog.Translog)19