Search in sources :

Example 86 with ClusterService

use of org.opensearch.cluster.service.ClusterService in project asynchronous-search by opensearch-project.

the class AsynchronousSearchServiceFreeContextTests method getClusterService.

private ClusterService getClusterService(DiscoveryNode discoveryNode, ThreadPool testThreadPool) {
    ClusterService clusterService = ClusterServiceUtils.createClusterService(testThreadPool, discoveryNode, clusterSettings);
    ClusterServiceUtils.setState(clusterService, ClusterStateCreationUtils.stateWithActivePrimary(AsynchronousSearchPersistenceService.ASYNC_SEARCH_RESPONSE_INDEX, true, randomInt(5)));
    return clusterService;
}
Also used : ClusterService(org.opensearch.cluster.service.ClusterService)

Example 87 with ClusterService

use of org.opensearch.cluster.service.ClusterService in project asynchronous-search by opensearch-project.

the class AsynchronousSearchServiceFreeContextTests method testFreeContextRunningUserMatches.

public void testFreeContextRunningUserMatches() throws InterruptedException {
    DiscoveryNode discoveryNode = new DiscoveryNode("node", OpenSearchTestCase.buildNewFakeTransportAddress(), emptyMap(), DiscoveryNodeRole.BUILT_IN_ROLES, Version.CURRENT);
    ThreadPool testThreadPool = null;
    try {
        testThreadPool = new TestThreadPool(OPEN_DISTRO_ASYNC_SEARCH_GENERIC_THREAD_POOL_NAME, executorBuilder);
        ClusterService mockClusterService = getClusterService(discoveryNode, testThreadPool);
        MockClient mockClient = new MockClient(testThreadPool);
        AsynchronousSearchActiveStore mockStore = mock(AsynchronousSearchActiveStore.class);
        AsynchronousSearchPersistenceService persistenceService = new AsynchronousSearchPersistenceService(mockClient, mockClusterService, testThreadPool);
        AsynchronousSearchService asService = new AsynchronousSearchService(persistenceService, mockStore, mockClient, mockClusterService, testThreadPool, new InternalAsynchronousSearchStats(), new NamedWriteableRegistry(emptyList()));
        TimeValue keepAlive = timeValueDays(9);
        boolean keepOnCompletion = true;
        User user1 = randomBoolean() ? randomUser() : null;
        SearchRequest searchRequest = new SearchRequest();
        SubmitAsynchronousSearchRequest submitAsynchronousSearchRequest = new SubmitAsynchronousSearchRequest(searchRequest);
        submitAsynchronousSearchRequest.keepOnCompletion(keepOnCompletion);
        submitAsynchronousSearchRequest.keepAlive(keepAlive);
        AsynchronousSearchProgressListener asProgressListener = mockAsynchronousSearchProgressListener(testThreadPool);
        AsynchronousSearchContextId asContextId = new AsynchronousSearchContextId(UUID.randomUUID().toString(), randomNonNegativeLong());
        AsynchronousSearchActiveContext asActiveContext = new AsynchronousSearchActiveContext(asContextId, discoveryNode.getId(), keepAlive, true, testThreadPool, testThreadPool::absoluteTimeInMillis, asProgressListener, user1, () -> true);
        // bootstrap search
        AsynchronousSearchTask task = new AsynchronousSearchTask(randomNonNegativeLong(), "transport", SearchAction.NAME, TaskId.EMPTY_TASK_ID, emptyMap(), asActiveContext, null, (c) -> {
        }) {

            @Override
            public boolean isCancelled() {
                return true;
            }
        };
        asActiveContext.setTask(task);
        asActiveContext.setState(AsynchronousSearchState.RUNNING);
        when(mockStore.getContext(any())).thenReturn(Optional.of(asActiveContext));
        persisted = false;
        // user
        User user = randomBoolean() ? null : user1;
        // task cancellation fails
        CountDownLatch latch = new CountDownLatch(1);
        asService.freeContext(asActiveContext.getAsynchronousSearchId(), asActiveContext.getContextId(), user, new LatchedActionListener<>(ActionListener.wrap(Assert::assertTrue, e -> {
            fail("active context should have been deleted");
        }), latch));
        latch.await();
        mockClusterService.stop();
    } finally {
        ThreadPool.terminate(testThreadPool, 30, TimeUnit.SECONDS);
    }
}
Also used : NamedWriteableRegistry(org.opensearch.common.io.stream.NamedWriteableRegistry) SubmitAsynchronousSearchRequest(org.opensearch.search.asynchronous.request.SubmitAsynchronousSearchRequest) SearchRequest(org.opensearch.action.search.SearchRequest) DiscoveryNode(org.opensearch.cluster.node.DiscoveryNode) TestClientUtils.randomUser(org.opensearch.search.asynchronous.utils.TestClientUtils.randomUser) User(org.opensearch.commons.authuser.User) SubmitAsynchronousSearchRequest(org.opensearch.search.asynchronous.request.SubmitAsynchronousSearchRequest) AsynchronousSearchContextId(org.opensearch.search.asynchronous.context.AsynchronousSearchContextId) TestThreadPool(org.opensearch.threadpool.TestThreadPool) ThreadPool(org.opensearch.threadpool.ThreadPool) InternalAsynchronousSearchStats(org.opensearch.search.asynchronous.stats.InternalAsynchronousSearchStats) TestThreadPool(org.opensearch.threadpool.TestThreadPool) AsynchronousSearchActiveContext(org.opensearch.search.asynchronous.context.active.AsynchronousSearchActiveContext) CountDownLatch(java.util.concurrent.CountDownLatch) AsynchronousSearchActiveStore(org.opensearch.search.asynchronous.context.active.AsynchronousSearchActiveStore) AsynchronousSearchProgressListener(org.opensearch.search.asynchronous.listener.AsynchronousSearchProgressListener) AsynchronousSearchTestCase.mockAsynchronousSearchProgressListener(org.opensearch.search.asynchronous.commons.AsynchronousSearchTestCase.mockAsynchronousSearchProgressListener) AsynchronousSearchTask(org.opensearch.search.asynchronous.task.AsynchronousSearchTask) ClusterService(org.opensearch.cluster.service.ClusterService) Assert(org.junit.Assert) TimeValue(org.opensearch.common.unit.TimeValue)

Example 88 with ClusterService

use of org.opensearch.cluster.service.ClusterService in project asynchronous-search by opensearch-project.

the class AsynchronousSearchServiceFreeContextTests method testFreeContextUserNotMatches.

public void testFreeContextUserNotMatches() throws InterruptedException {
    DiscoveryNode discoveryNode = new DiscoveryNode("node", OpenSearchTestCase.buildNewFakeTransportAddress(), emptyMap(), DiscoveryNodeRole.BUILT_IN_ROLES, Version.CURRENT);
    ThreadPool testThreadPool = null;
    try {
        testThreadPool = new TestThreadPool(OPEN_DISTRO_ASYNC_SEARCH_GENERIC_THREAD_POOL_NAME, executorBuilder);
        ClusterService mockClusterService = getClusterService(discoveryNode, testThreadPool);
        MockClient mockClient = new MockClient(testThreadPool);
        AsynchronousSearchPersistenceService persistenceService = new AsynchronousSearchPersistenceService(mockClient, mockClusterService, testThreadPool);
        AsynchronousSearchService asService = new AsynchronousSearchService(persistenceService, new AsynchronousSearchActiveStore(mockClusterService), mockClient, mockClusterService, testThreadPool, new InternalAsynchronousSearchStats(), new NamedWriteableRegistry(emptyList()));
        TimeValue keepAlive = timeValueHours(9);
        boolean keepOnCompletion = true;
        User user1 = randomUser();
        SearchRequest searchRequest = new SearchRequest();
        SubmitAsynchronousSearchRequest submitAsynchronousSearchRequest = new SubmitAsynchronousSearchRequest(searchRequest);
        submitAsynchronousSearchRequest.keepOnCompletion(keepOnCompletion);
        submitAsynchronousSearchRequest.keepAlive(keepAlive);
        AsynchronousSearchActiveContext context = (AsynchronousSearchActiveContext) asService.createAndStoreContext(submitAsynchronousSearchRequest, System.currentTimeMillis(), () -> null, user1);
        // bootstrap search
        AsynchronousSearchTask task = new AsynchronousSearchTask(randomNonNegativeLong(), "transport", SearchAction.NAME, TaskId.EMPTY_TASK_ID, emptyMap(), context, null, (c) -> {
        });
        asService.bootstrapSearch(task, context.getContextId());
        persisted = false;
        CountDownLatch latch = new CountDownLatch(1);
        User user2 = randomUser();
        asService.freeContext(context.getAsynchronousSearchId(), context.getContextId(), user2, new LatchedActionListener<>(ActionListener.wrap(r -> {
            fail("expected security exception but got delete ack " + r + " search creator " + user1 + " " + "accessing user " + user2);
        }, e -> {
            assertTrue(e.getClass().getName(), e instanceof ResourceNotFoundException);
        }), latch));
        latch.await();
        mockClusterService.stop();
    } finally {
        ThreadPool.terminate(testThreadPool, 30, TimeUnit.SECONDS);
    }
}
Also used : NamedWriteableRegistry(org.opensearch.common.io.stream.NamedWriteableRegistry) SubmitAsynchronousSearchRequest(org.opensearch.search.asynchronous.request.SubmitAsynchronousSearchRequest) SearchRequest(org.opensearch.action.search.SearchRequest) DiscoveryNode(org.opensearch.cluster.node.DiscoveryNode) TestClientUtils.randomUser(org.opensearch.search.asynchronous.utils.TestClientUtils.randomUser) User(org.opensearch.commons.authuser.User) SubmitAsynchronousSearchRequest(org.opensearch.search.asynchronous.request.SubmitAsynchronousSearchRequest) TestThreadPool(org.opensearch.threadpool.TestThreadPool) ThreadPool(org.opensearch.threadpool.ThreadPool) InternalAsynchronousSearchStats(org.opensearch.search.asynchronous.stats.InternalAsynchronousSearchStats) TestThreadPool(org.opensearch.threadpool.TestThreadPool) AsynchronousSearchActiveContext(org.opensearch.search.asynchronous.context.active.AsynchronousSearchActiveContext) CountDownLatch(java.util.concurrent.CountDownLatch) AsynchronousSearchActiveStore(org.opensearch.search.asynchronous.context.active.AsynchronousSearchActiveStore) AsynchronousSearchTask(org.opensearch.search.asynchronous.task.AsynchronousSearchTask) ClusterService(org.opensearch.cluster.service.ClusterService) ResourceNotFoundException(org.opensearch.ResourceNotFoundException) TimeValue(org.opensearch.common.unit.TimeValue)

Example 89 with ClusterService

use of org.opensearch.cluster.service.ClusterService in project asynchronous-search by opensearch-project.

the class AsynchronousSearchServiceUpdateContextTests method testUpdateContextTimedOut.

public void testUpdateContextTimedOut() throws InterruptedException {
    DiscoveryNode discoveryNode = new DiscoveryNode("node", OpenSearchTestCase.buildNewFakeTransportAddress(), emptyMap(), DiscoveryNodeRole.BUILT_IN_ROLES, Version.CURRENT);
    ThreadPool testThreadPool = null;
    try {
        testThreadPool = new TestThreadPool(OPEN_DISTRO_ASYNC_SEARCH_GENERIC_THREAD_POOL_NAME, executorBuilder);
        ClusterService mockClusterService = getClusterService(discoveryNode, testThreadPool);
        MockClient mockClient = new MockClient(testThreadPool);
        AsynchronousSearchActiveStore mockStore = mock(AsynchronousSearchActiveStore.class);
        AsynchronousSearchPersistenceService persistenceService = new AsynchronousSearchPersistenceService(mockClient, mockClusterService, testThreadPool);
        AsynchronousSearchService asService = new AsynchronousSearchService(persistenceService, mockStore, mockClient, mockClusterService, testThreadPool, new InternalAsynchronousSearchStats(), new NamedWriteableRegistry(emptyList()));
        TimeValue keepAlive = timeValueHours(9);
        boolean keepOnCompletion = true;
        User user1 = randomBoolean() ? randomUser() : null;
        SearchRequest searchRequest = new SearchRequest();
        SubmitAsynchronousSearchRequest submitAsynchronousSearchRequest = new SubmitAsynchronousSearchRequest(searchRequest);
        submitAsynchronousSearchRequest.keepOnCompletion(keepOnCompletion);
        submitAsynchronousSearchRequest.keepAlive(keepAlive);
        AsynchronousSearchProgressListener asProgressListener = mockAsynchronousSearchProgressListener(testThreadPool);
        AsynchronousSearchContextId asContextId = new AsynchronousSearchContextId(UUID.randomUUID().toString(), randomNonNegativeLong());
        MockAsynchronousSearchActiveContext asActiveContext = new MockAsynchronousSearchActiveContext(asContextId, discoveryNode.getId(), keepAlive, true, testThreadPool, testThreadPool::absoluteTimeInMillis, asProgressListener, user1);
        // bootstrap search
        AsynchronousSearchTask task = new AsynchronousSearchTask(randomNonNegativeLong(), "transport", SearchAction.NAME, TaskId.EMPTY_TASK_ID, emptyMap(), asActiveContext, null, (c) -> {
        });
        asActiveContext.setTask(task);
        simulateIsAlive = true;
        simulateTimedOut = true;
        when(mockStore.getContext(asContextId)).thenReturn(Optional.of(asActiveContext));
        CountDownLatch latch = new CountDownLatch(1);
        asService.updateKeepAliveAndGetContext(asActiveContext.getAsynchronousSearchId(), timeValueHours(9), asActiveContext.getContextId(), randomUser(), new LatchedActionListener<>(wrap(r -> fail("expected timedout exception"), e -> assertTrue(e instanceof OpenSearchTimeoutException)), latch));
        latch.await();
        mockClusterService.stop();
    } finally {
        ThreadPool.terminate(testThreadPool, 30, TimeUnit.SECONDS);
    }
}
Also used : NamedWriteableRegistry(org.opensearch.common.io.stream.NamedWriteableRegistry) SubmitAsynchronousSearchRequest(org.opensearch.search.asynchronous.request.SubmitAsynchronousSearchRequest) SearchRequest(org.opensearch.action.search.SearchRequest) DiscoveryNode(org.opensearch.cluster.node.DiscoveryNode) TestClientUtils.randomUser(org.opensearch.search.asynchronous.utils.TestClientUtils.randomUser) User(org.opensearch.commons.authuser.User) OpenSearchTimeoutException(org.opensearch.OpenSearchTimeoutException) SubmitAsynchronousSearchRequest(org.opensearch.search.asynchronous.request.SubmitAsynchronousSearchRequest) AsynchronousSearchContextId(org.opensearch.search.asynchronous.context.AsynchronousSearchContextId) TestThreadPool(org.opensearch.threadpool.TestThreadPool) ThreadPool(org.opensearch.threadpool.ThreadPool) InternalAsynchronousSearchStats(org.opensearch.search.asynchronous.stats.InternalAsynchronousSearchStats) TestThreadPool(org.opensearch.threadpool.TestThreadPool) CountDownLatch(java.util.concurrent.CountDownLatch) AsynchronousSearchActiveStore(org.opensearch.search.asynchronous.context.active.AsynchronousSearchActiveStore) AsynchronousSearchProgressListener(org.opensearch.search.asynchronous.listener.AsynchronousSearchProgressListener) AsynchronousSearchTestCase.mockAsynchronousSearchProgressListener(org.opensearch.search.asynchronous.commons.AsynchronousSearchTestCase.mockAsynchronousSearchProgressListener) AsynchronousSearchTask(org.opensearch.search.asynchronous.task.AsynchronousSearchTask) ClusterService(org.opensearch.cluster.service.ClusterService) TimeValue(org.opensearch.common.unit.TimeValue)

Example 90 with ClusterService

use of org.opensearch.cluster.service.ClusterService in project asynchronous-search by opensearch-project.

the class AsynchronousSearchServiceUpdateContextTests method testUpdateContextPermitAcquisitionFailure.

public void testUpdateContextPermitAcquisitionFailure() throws InterruptedException {
    DiscoveryNode discoveryNode = new DiscoveryNode("node", OpenSearchTestCase.buildNewFakeTransportAddress(), emptyMap(), DiscoveryNodeRole.BUILT_IN_ROLES, Version.CURRENT);
    ThreadPool testThreadPool = null;
    try {
        testThreadPool = new TestThreadPool(OPEN_DISTRO_ASYNC_SEARCH_GENERIC_THREAD_POOL_NAME, executorBuilder);
        ClusterService mockClusterService = getClusterService(discoveryNode, testThreadPool);
        MockClient mockClient = new MockClient(testThreadPool);
        AsynchronousSearchActiveStore mockStore = mock(AsynchronousSearchActiveStore.class);
        AsynchronousSearchPersistenceService persistenceService = new AsynchronousSearchPersistenceService(mockClient, mockClusterService, testThreadPool);
        AsynchronousSearchService asService = new AsynchronousSearchService(persistenceService, mockStore, mockClient, mockClusterService, testThreadPool, new InternalAsynchronousSearchStats(), new NamedWriteableRegistry(emptyList()));
        TimeValue keepAlive = timeValueHours(9);
        boolean keepOnCompletion = true;
        User user1 = randomBoolean() ? randomUser() : null;
        SearchRequest searchRequest = new SearchRequest();
        SubmitAsynchronousSearchRequest submitAsynchronousSearchRequest = new SubmitAsynchronousSearchRequest(searchRequest);
        submitAsynchronousSearchRequest.keepOnCompletion(keepOnCompletion);
        submitAsynchronousSearchRequest.keepAlive(keepAlive);
        AsynchronousSearchProgressListener asProgressListener = mockAsynchronousSearchProgressListener(testThreadPool);
        AsynchronousSearchContextId asContextId = new AsynchronousSearchContextId(UUID.randomUUID().toString(), randomNonNegativeLong());
        MockAsynchronousSearchActiveContext asActiveContext = new MockAsynchronousSearchActiveContext(asContextId, discoveryNode.getId(), keepAlive, true, testThreadPool, testThreadPool::absoluteTimeInMillis, asProgressListener, user1);
        // bootstrap search
        AsynchronousSearchTask task = new AsynchronousSearchTask(randomNonNegativeLong(), "transport", SearchAction.NAME, TaskId.EMPTY_TASK_ID, emptyMap(), asActiveContext, null, (c) -> {
        });
        asActiveContext.setTask(task);
        simulateIsAlive = true;
        simulateUncheckedException = true;
        long oldExpiration = asActiveContext.getExpirationTimeMillis();
        when(mockStore.getContext(asContextId)).thenReturn(Optional.of(asActiveContext));
        CountDownLatch latch = new CountDownLatch(1);
        asService.updateKeepAliveAndGetContext(asActiveContext.getAsynchronousSearchId(), timeValueHours(9), asActiveContext.getContextId(), randomUser(), new LatchedActionListener<>(wrap(r -> assertEquals("active context should not have been updated on permit acquisition failure", asActiveContext.getExpirationTimeMillis(), oldExpiration), e -> fail("expected update to succeed but got " + e.getMessage())), latch));
        latch.await();
        assertEquals("update should have been attempted on index", mockClient.updateCount.intValue(), 1);
        mockClusterService.stop();
    } finally {
        ThreadPool.terminate(testThreadPool, 30, TimeUnit.SECONDS);
    }
}
Also used : NamedWriteableRegistry(org.opensearch.common.io.stream.NamedWriteableRegistry) SubmitAsynchronousSearchRequest(org.opensearch.search.asynchronous.request.SubmitAsynchronousSearchRequest) SearchRequest(org.opensearch.action.search.SearchRequest) DiscoveryNode(org.opensearch.cluster.node.DiscoveryNode) TestClientUtils.randomUser(org.opensearch.search.asynchronous.utils.TestClientUtils.randomUser) User(org.opensearch.commons.authuser.User) SubmitAsynchronousSearchRequest(org.opensearch.search.asynchronous.request.SubmitAsynchronousSearchRequest) AsynchronousSearchContextId(org.opensearch.search.asynchronous.context.AsynchronousSearchContextId) TestThreadPool(org.opensearch.threadpool.TestThreadPool) ThreadPool(org.opensearch.threadpool.ThreadPool) InternalAsynchronousSearchStats(org.opensearch.search.asynchronous.stats.InternalAsynchronousSearchStats) TestThreadPool(org.opensearch.threadpool.TestThreadPool) CountDownLatch(java.util.concurrent.CountDownLatch) AsynchronousSearchActiveStore(org.opensearch.search.asynchronous.context.active.AsynchronousSearchActiveStore) AsynchronousSearchProgressListener(org.opensearch.search.asynchronous.listener.AsynchronousSearchProgressListener) AsynchronousSearchTestCase.mockAsynchronousSearchProgressListener(org.opensearch.search.asynchronous.commons.AsynchronousSearchTestCase.mockAsynchronousSearchProgressListener) AsynchronousSearchTask(org.opensearch.search.asynchronous.task.AsynchronousSearchTask) ClusterService(org.opensearch.cluster.service.ClusterService) TimeValue(org.opensearch.common.unit.TimeValue)

Aggregations

ClusterService (org.opensearch.cluster.service.ClusterService)296 ThreadPool (org.opensearch.threadpool.ThreadPool)123 Settings (org.opensearch.common.settings.Settings)115 ClusterState (org.opensearch.cluster.ClusterState)106 DiscoveryNode (org.opensearch.cluster.node.DiscoveryNode)103 TestThreadPool (org.opensearch.threadpool.TestThreadPool)93 TransportService (org.opensearch.transport.TransportService)86 ClusterSettings (org.opensearch.common.settings.ClusterSettings)76 ActionListener (org.opensearch.action.ActionListener)75 Before (org.junit.Before)66 ActionFilters (org.opensearch.action.support.ActionFilters)66 CountDownLatch (java.util.concurrent.CountDownLatch)65 HashSet (java.util.HashSet)63 TimeValue (org.opensearch.common.unit.TimeValue)61 IOException (java.io.IOException)56 IndexMetadata (org.opensearch.cluster.metadata.IndexMetadata)53 OpenSearchTestCase (org.opensearch.test.OpenSearchTestCase)48 Collections (java.util.Collections)47 ClusterName (org.opensearch.cluster.ClusterName)47 Metadata (org.opensearch.cluster.metadata.Metadata)47