Search in sources :

Example 1 with ScalingExecutorBuilder

use of org.opensearch.threadpool.ScalingExecutorBuilder in project asynchronous-search by opensearch-project.

the class AsynchronousSearchActiveContextTests method testInitializeContext.

public void testInitializeContext() {
    TestThreadPool threadPool = null;
    try {
        int writeThreadPoolSize = randomIntBetween(1, 2);
        int writeThreadPoolQueueSize = randomIntBetween(1, 2);
        Settings settings = Settings.builder().put("thread_pool." + AsynchronousSearchPlugin.OPEN_DISTRO_ASYNC_SEARCH_GENERIC_THREAD_POOL_NAME + ".size", writeThreadPoolSize).put("thread_pool." + AsynchronousSearchPlugin.OPEN_DISTRO_ASYNC_SEARCH_GENERIC_THREAD_POOL_NAME + ".queue_size", writeThreadPoolQueueSize).build();
        final int availableProcessors = OpenSearchExecutors.allocatedProcessors(settings);
        ScalingExecutorBuilder scalingExecutorBuilder = new ScalingExecutorBuilder(AsynchronousSearchPlugin.OPEN_DISTRO_ASYNC_SEARCH_GENERIC_THREAD_POOL_NAME, 1, Math.min(2 * availableProcessors, Math.max(128, 512)), TimeValue.timeValueMinutes(30));
        threadPool = new TestThreadPool("Tests", settings, scalingExecutorBuilder);
        String node = UUID.randomUUID().toString();
        AsynchronousSearchProgressListener asProgressListener = mockAsynchronousSearchProgressListener(threadPool);
        AsynchronousSearchContextId asContextId = new AsynchronousSearchContextId(UUID.randomUUID().toString(), randomNonNegativeLong());
        boolean keepOnCompletion = randomBoolean();
        User user = TestClientUtils.randomUser();
        TimeValue keepAlive = TimeValue.timeValueDays(randomInt(100));
        AsynchronousSearchActiveContext context = new AsynchronousSearchActiveContext(asContextId, node, keepAlive, keepOnCompletion, threadPool, threadPool::absoluteTimeInMillis, asProgressListener, user, () -> true);
        assertEquals(AsynchronousSearchState.INIT, context.getAsynchronousSearchState());
        assertNull(context.getTask());
        assertNull(context.getAsynchronousSearchId());
        assertEquals(context.getAsynchronousSearchState(), AsynchronousSearchState.INIT);
    } finally {
        ThreadPool.terminate(threadPool, 30, TimeUnit.SECONDS);
    }
}
Also used : ScalingExecutorBuilder(org.opensearch.threadpool.ScalingExecutorBuilder) User(org.opensearch.commons.authuser.User) AsynchronousSearchContextId(org.opensearch.search.asynchronous.context.AsynchronousSearchContextId) Matchers.containsString(org.hamcrest.Matchers.containsString) TestThreadPool(org.opensearch.threadpool.TestThreadPool) Settings(org.opensearch.common.settings.Settings) TimeValue(org.opensearch.common.unit.TimeValue) AsynchronousSearchProgressListener(org.opensearch.search.asynchronous.listener.AsynchronousSearchProgressListener)

Example 2 with ScalingExecutorBuilder

use of org.opensearch.threadpool.ScalingExecutorBuilder in project asynchronous-search by opensearch-project.

the class AsynchronousSearchContextPermitsTests method setupThreadPool.

@BeforeClass
public static void setupThreadPool() {
    int writeThreadPoolSize = randomIntBetween(1, 2);
    int writeThreadPoolQueueSize = randomIntBetween(1, 2);
    Settings settings = Settings.builder().put("thread_pool." + AsynchronousSearchPlugin.OPEN_DISTRO_ASYNC_SEARCH_GENERIC_THREAD_POOL_NAME + ".size", writeThreadPoolSize).put("thread_pool." + AsynchronousSearchPlugin.OPEN_DISTRO_ASYNC_SEARCH_GENERIC_THREAD_POOL_NAME + ".queue_size", writeThreadPoolQueueSize).build();
    final int availableProcessors = OpenSearchExecutors.allocatedProcessors(settings);
    ScalingExecutorBuilder scalingExecutorBuilder = new ScalingExecutorBuilder(AsynchronousSearchPlugin.OPEN_DISTRO_ASYNC_SEARCH_GENERIC_THREAD_POOL_NAME, 1, Math.min(2 * availableProcessors, Math.max(128, 512)), TimeValue.timeValueMinutes(30));
    threadPool = new TestThreadPool("PermitsTests", settings, scalingExecutorBuilder);
}
Also used : ScalingExecutorBuilder(org.opensearch.threadpool.ScalingExecutorBuilder) TestThreadPool(org.opensearch.threadpool.TestThreadPool) Settings(org.opensearch.common.settings.Settings) BeforeClass(org.junit.BeforeClass)

Example 3 with ScalingExecutorBuilder

use of org.opensearch.threadpool.ScalingExecutorBuilder in project asynchronous-search by opensearch-project.

the class AsynchronousSearchPlugin method getExecutorBuilders.

// TODO Revisit these once we performance test the feature
@Override
public List<ExecutorBuilder<?>> getExecutorBuilders(Settings settings) {
    final int availableProcessors = OpenSearchExecutors.allocatedProcessors(settings);
    List<ExecutorBuilder<?>> executorBuilders = new ArrayList<>();
    executorBuilders.add(new ScalingExecutorBuilder(OPEN_DISTRO_ASYNC_SEARCH_GENERIC_THREAD_POOL_NAME, 1, Math.min(2 * availableProcessors, Math.max(128, 512)), TimeValue.timeValueMinutes(30)));
    return executorBuilders;
}
Also used : ExecutorBuilder(org.opensearch.threadpool.ExecutorBuilder) ScalingExecutorBuilder(org.opensearch.threadpool.ScalingExecutorBuilder) ScalingExecutorBuilder(org.opensearch.threadpool.ScalingExecutorBuilder) ArrayList(java.util.ArrayList)

Example 4 with ScalingExecutorBuilder

use of org.opensearch.threadpool.ScalingExecutorBuilder in project asynchronous-search by opensearch-project.

the class AsynchronousSearchServiceTests method createObjects.

@Before
public void createObjects() {
    settings = Settings.builder().put("node.name", "test").put("cluster.name", "ClusterServiceTests").put(AsynchronousSearchActiveStore.NODE_CONCURRENT_RUNNING_SEARCHES_SETTING.getKey(), 10).put(AsynchronousSearchService.PERSIST_SEARCH_FAILURES_SETTING.getKey(), true).build();
    final Set<Setting<?>> settingsSet = Stream.concat(ClusterSettings.BUILT_IN_CLUSTER_SETTINGS.stream(), Stream.of(AsynchronousSearchActiveStore.NODE_CONCURRENT_RUNNING_SEARCHES_SETTING, AsynchronousSearchService.MAX_KEEP_ALIVE_SETTING, AsynchronousSearchService.PERSIST_SEARCH_FAILURES_SETTING, AsynchronousSearchService.MAX_SEARCH_RUNNING_TIME_SETTING, AsynchronousSearchService.MAX_WAIT_FOR_COMPLETION_TIMEOUT_SETTING)).collect(Collectors.toSet());
    final int availableProcessors = OpenSearchExecutors.allocatedProcessors(settings);
    List<ExecutorBuilder<?>> executorBuilders = new ArrayList<>();
    executorBuilders.add(new ScalingExecutorBuilder(AsynchronousSearchPlugin.OPEN_DISTRO_ASYNC_SEARCH_GENERIC_THREAD_POOL_NAME, 1, Math.min(2 * availableProcessors, Math.max(128, 512)), TimeValue.timeValueMinutes(30)));
    executorBuilder = executorBuilders.get(0);
    clusterSettings = new ClusterSettings(settings, settingsSet);
    blockPersistence = false;
}
Also used : ExecutorBuilder(org.opensearch.threadpool.ExecutorBuilder) ScalingExecutorBuilder(org.opensearch.threadpool.ScalingExecutorBuilder) ScalingExecutorBuilder(org.opensearch.threadpool.ScalingExecutorBuilder) ClusterSettings(org.opensearch.common.settings.ClusterSettings) Setting(org.opensearch.common.settings.Setting) ArrayList(java.util.ArrayList) Before(org.junit.Before)

Example 5 with ScalingExecutorBuilder

use of org.opensearch.threadpool.ScalingExecutorBuilder in project asynchronous-search by opensearch-project.

the class AsynchronousSearchPostProcessorTests method createObjects.

@Before
public void createObjects() {
    Settings settings = Settings.builder().put("node.name", "test").put("cluster.name", "ClusterServiceTests").put(AsynchronousSearchActiveStore.NODE_CONCURRENT_RUNNING_SEARCHES_SETTING.getKey(), 10).build();
    final Set<Setting<?>> settingsSet = Stream.concat(ClusterSettings.BUILT_IN_CLUSTER_SETTINGS.stream(), Stream.of(AsynchronousSearchActiveStore.NODE_CONCURRENT_RUNNING_SEARCHES_SETTING, AsynchronousSearchService.PERSIST_SEARCH_FAILURES_SETTING, AsynchronousSearchService.MAX_KEEP_ALIVE_SETTING, AsynchronousSearchService.MAX_SEARCH_RUNNING_TIME_SETTING, AsynchronousSearchService.MAX_WAIT_FOR_COMPLETION_TIMEOUT_SETTING)).collect(Collectors.toSet());
    final int availableProcessors = OpenSearchExecutors.allocatedProcessors(settings);
    List<ExecutorBuilder<?>> executorBuilders = new ArrayList<>();
    executorBuilders.add(new ScalingExecutorBuilder(AsynchronousSearchPlugin.OPEN_DISTRO_ASYNC_SEARCH_GENERIC_THREAD_POOL_NAME, 1, Math.min(2 * availableProcessors, Math.max(128, 512)), TimeValue.timeValueMinutes(30)));
    executorBuilder = executorBuilders.get(0);
    clusterSettings = new ClusterSettings(settings, settingsSet);
}
Also used : ExecutorBuilder(org.opensearch.threadpool.ExecutorBuilder) ScalingExecutorBuilder(org.opensearch.threadpool.ScalingExecutorBuilder) ScalingExecutorBuilder(org.opensearch.threadpool.ScalingExecutorBuilder) ClusterSettings(org.opensearch.common.settings.ClusterSettings) Setting(org.opensearch.common.settings.Setting) ArrayList(java.util.ArrayList) Settings(org.opensearch.common.settings.Settings) ClusterSettings(org.opensearch.common.settings.ClusterSettings) Before(org.junit.Before)

Aggregations

ScalingExecutorBuilder (org.opensearch.threadpool.ScalingExecutorBuilder)12 Settings (org.opensearch.common.settings.Settings)9 ArrayList (java.util.ArrayList)7 ExecutorBuilder (org.opensearch.threadpool.ExecutorBuilder)7 Before (org.junit.Before)6 ClusterSettings (org.opensearch.common.settings.ClusterSettings)6 Setting (org.opensearch.common.settings.Setting)6 TestThreadPool (org.opensearch.threadpool.TestThreadPool)5 Matchers.containsString (org.hamcrest.Matchers.containsString)4 TimeValue (org.opensearch.common.unit.TimeValue)4 AsynchronousSearchContextId (org.opensearch.search.asynchronous.context.AsynchronousSearchContextId)4 AsynchronousSearchProgressListener (org.opensearch.search.asynchronous.listener.AsynchronousSearchProgressListener)4 SetOnce (org.apache.lucene.util.SetOnce)2 User (org.opensearch.commons.authuser.User)2 AsynchronousSearchTask (org.opensearch.search.asynchronous.task.AsynchronousSearchTask)2 BeforeClass (org.junit.BeforeClass)1 SearchRequest (org.opensearch.action.search.SearchRequest)1 SearchResponse (org.opensearch.action.search.SearchResponse)1 SubmitAsynchronousSearchRequest (org.opensearch.search.asynchronous.request.SubmitAsynchronousSearchRequest)1 InternalSearchResponse (org.opensearch.search.internal.InternalSearchResponse)1