Search in sources :

Example 1 with SearchDeletedEvent

use of org.opensearch.search.asynchronous.context.state.event.SearchDeletedEvent in project asynchronous-search by opensearch-project.

the class AsynchronousSearchPostProcessorTests method testProcessSearchFailureOnDeletedContext.

public void testProcessSearchFailureOnDeletedContext() throws AsynchronousSearchStateMachineClosedException {
    DiscoveryNode discoveryNode = new DiscoveryNode("node", OpenSearchTestCase.buildNewFakeTransportAddress(), Collections.emptyMap(), DiscoveryNodeRole.BUILT_IN_ROLES, Version.CURRENT);
    AtomicBoolean activeContextCleanUpConsumerInvocation = new AtomicBoolean();
    ThreadPool testThreadPool = null;
    try {
        testThreadPool = new TestThreadPool(this.getClass().getName(), executorBuilder);
        ClusterService mockClusterService = ClusterServiceUtils.createClusterService(testThreadPool, discoveryNode, clusterSettings);
        FakeClient fakeClient = new FakeClient(testThreadPool);
        AsynchronousSearchActiveStore asActiveStore = new AsynchronousSearchActiveStore(mockClusterService);
        AsynchronousSearchPersistenceService persistenceService = new AsynchronousSearchPersistenceService(fakeClient, mockClusterService, testThreadPool);
        AsynchronousSearchService asService = new AsynchronousSearchService(persistenceService, asActiveStore, fakeClient, mockClusterService, testThreadPool, new InternalAsynchronousSearchStats(), new NamedWriteableRegistry(emptyList()));
        AsynchronousSearchStateMachine asStateMachine = asService.getStateMachine();
        ClusterService clusterService = ClusterServiceUtils.createClusterService(testThreadPool, discoveryNode, clusterSettings);
        AsynchronousSearchPostProcessor postProcessor = new AsynchronousSearchPostProcessor(persistenceService, asActiveStore, asStateMachine, (context) -> activeContextCleanUpConsumerInvocation.compareAndSet(false, true), testThreadPool, clusterService);
        SubmitAsynchronousSearchRequest submitAsynchronousSearchRequest = new SubmitAsynchronousSearchRequest(new SearchRequest());
        submitAsynchronousSearchRequest.keepOnCompletion(true);
        submitAsynchronousSearchRequest.keepAlive(TimeValue.timeValueHours(1));
        AsynchronousSearchActiveContext context = (AsynchronousSearchActiveContext) asService.createAndStoreContext(submitAsynchronousSearchRequest, System.currentTimeMillis(), () -> InternalAggregationTestCase.emptyReduceContextBuilder(), null);
        ShardSearchFailure shardSearchFailure = new ShardSearchFailure(new RuntimeException("runtime-exception"));
        SearchPhaseExecutionException exception = new SearchPhaseExecutionException("phase", "msg", new NullPointerException(), new ShardSearchFailure[] { shardSearchFailure });
        asStateMachine.trigger(new SearchStartedEvent(context, new SearchTask(0, "n/a", "n/a", () -> "test", null, Collections.emptyMap())));
        asStateMachine.trigger(new SearchDeletedEvent(context));
        AsynchronousSearchResponse asResponse = postProcessor.processSearchFailure(exception, context.getContextId());
        assertNull(asResponse.getId());
        assertNull(asResponse.getSearchResponse());
        assertEquals(-1L, asResponse.getExpirationTimeMillis());
        assertEquals(-1L, asResponse.getStartTimeMillis());
        assertEquals(AsynchronousSearchState.FAILED, asResponse.getState());
        assertThat(asResponse.getError(), instanceOf(SearchPhaseExecutionException.class));
        assertFalse(activeContextCleanUpConsumerInvocation.get());
        assertEquals(0, fakeClient.persistenceCount);
    } finally {
        ThreadPool.terminate(testThreadPool, 200, TimeUnit.MILLISECONDS);
    }
}
Also used : NamedWriteableRegistry(org.opensearch.common.io.stream.NamedWriteableRegistry) AsynchronousSearchResponse(org.opensearch.search.asynchronous.response.AsynchronousSearchResponse) SubmitAsynchronousSearchRequest(org.opensearch.search.asynchronous.request.SubmitAsynchronousSearchRequest) SearchRequest(org.opensearch.action.search.SearchRequest) DiscoveryNode(org.opensearch.cluster.node.DiscoveryNode) SubmitAsynchronousSearchRequest(org.opensearch.search.asynchronous.request.SubmitAsynchronousSearchRequest) SearchPhaseExecutionException(org.opensearch.action.search.SearchPhaseExecutionException) TestThreadPool(org.opensearch.threadpool.TestThreadPool) ThreadPool(org.opensearch.threadpool.ThreadPool) InternalAsynchronousSearchStats(org.opensearch.search.asynchronous.stats.InternalAsynchronousSearchStats) TestThreadPool(org.opensearch.threadpool.TestThreadPool) AsynchronousSearchActiveContext(org.opensearch.search.asynchronous.context.active.AsynchronousSearchActiveContext) AsynchronousSearchActiveStore(org.opensearch.search.asynchronous.context.active.AsynchronousSearchActiveStore) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) ClusterService(org.opensearch.cluster.service.ClusterService) SearchTask(org.opensearch.action.search.SearchTask) SearchStartedEvent(org.opensearch.search.asynchronous.context.state.event.SearchStartedEvent) AsynchronousSearchStateMachine(org.opensearch.search.asynchronous.context.state.AsynchronousSearchStateMachine) AsynchronousSearchPostProcessor(org.opensearch.search.asynchronous.processor.AsynchronousSearchPostProcessor) ShardSearchFailure(org.opensearch.action.search.ShardSearchFailure) SearchDeletedEvent(org.opensearch.search.asynchronous.context.state.event.SearchDeletedEvent)

Example 2 with SearchDeletedEvent

use of org.opensearch.search.asynchronous.context.state.event.SearchDeletedEvent in project asynchronous-search by opensearch-project.

the class AsynchronousSearchStateMachineTests method testStateMachine.

public void testStateMachine() throws InterruptedException, BrokenBarrierException {
    DiscoveryNode discoveryNode = new DiscoveryNode("node", OpenSearchTestCase.buildNewFakeTransportAddress(), Collections.emptyMap(), DiscoveryNodeRole.BUILT_IN_ROLES, Version.CURRENT);
    TestThreadPool threadPool = null;
    try {
        threadPool = new TestThreadPool("test", executorBuilder);
        ClusterService mockClusterService = createClusterService(settings, threadPool, discoveryNode, clusterSettings);
        FakeClient fakeClient = new FakeClient(threadPool);
        AsynchronousSearchActiveStore asActiveStore = new AsynchronousSearchActiveStore(mockClusterService);
        AsynchronousSearchPersistenceService persistenceService = new AsynchronousSearchPersistenceService(fakeClient, mockClusterService, threadPool);
        CustomContextListener customContextListener = new CustomContextListener();
        AsynchronousSearchService asService = new AsynchronousSearchService(persistenceService, asActiveStore, fakeClient, mockClusterService, threadPool, customContextListener, new NamedWriteableRegistry(emptyList()));
        AsynchronousSearchProgressListener asProgressListener = mockAsynchronousSearchProgressListener(threadPool);
        AsynchronousSearchContextId asContextId = new AsynchronousSearchContextId(UUID.randomUUID().toString(), randomNonNegativeLong());
        TimeValue keepAlive = TimeValue.timeValueDays(randomInt(100));
        AsynchronousSearchActiveContext context = new AsynchronousSearchActiveContext(asContextId, discoveryNode.getId(), keepAlive, true, threadPool, threadPool::absoluteTimeInMillis, asProgressListener, null, () -> true);
        assertNull(context.getTask());
        assertEquals(context.getAsynchronousSearchState(), INIT);
        AsynchronousSearchStateMachine stateMachine = asService.getStateMachine();
        AtomicInteger numCompleted = new AtomicInteger();
        AtomicInteger numFailure = new AtomicInteger();
        doConcurrentStateMachineTrigger(stateMachine, new SearchStartedEvent(context, new AsynchronousSearchTask(randomNonNegativeLong(), "transport", SearchAction.NAME, TaskId.EMPTY_TASK_ID, emptyMap(), context, null, (a) -> {
        })), RUNNING, IllegalStateException.class, Optional.empty());
        boolean success = randomBoolean();
        assertNotNull(context.getTask());
        if (randomBoolean()) {
            // delete running context
            doConcurrentStateMachineTrigger(stateMachine, new SearchDeletedEvent(context), CLOSED, AsynchronousSearchStateMachineClosedException.class, Optional.empty());
        } else {
            if (success) {
                doConcurrentStateMachineTrigger(stateMachine, new SearchFailureEvent(context, new RuntimeException("test")), FAILED, IllegalStateException.class, Optional.empty());
                numFailure.getAndIncrement();
            } else {
                // success or failure
                doConcurrentStateMachineTrigger(stateMachine, new SearchSuccessfulEvent(context, getMockSearchResponse()), SUCCEEDED, IllegalStateException.class, Optional.empty());
                numCompleted.getAndIncrement();
            }
            doConcurrentStateMachineTrigger(stateMachine, new BeginPersistEvent(context), PERSISTING, IllegalStateException.class, Optional.of(AsynchronousSearchStateMachineClosedException.class));
            waitUntil(() -> context.getAsynchronousSearchState().equals(CLOSED), 1, TimeUnit.MINUTES);
            assertTrue(context.getAsynchronousSearchState().toString() + " numFailure : " + numFailure.get() + " numSuccess : " + numCompleted.get(), context.getAsynchronousSearchState().equals(CLOSED));
            assertEquals(1, customContextListener.getPersistedCount() + customContextListener.getPersistFailedCount());
        }
        assertEquals(numCompleted.get(), customContextListener.getCompletedCount());
        assertEquals(numFailure.get(), customContextListener.getFailedCount());
        assertEquals("success:" + success, 0, customContextListener.getRunningCount());
        assertEquals(1, customContextListener.getDeletedCount());
    } finally {
        ThreadPool.terminate(threadPool, 100, TimeUnit.MILLISECONDS);
    }
}
Also used : NamedWriteableRegistry(org.opensearch.common.io.stream.NamedWriteableRegistry) DiscoveryNode(org.opensearch.cluster.node.DiscoveryNode) SearchSuccessfulEvent(org.opensearch.search.asynchronous.context.state.event.SearchSuccessfulEvent) SearchFailureEvent(org.opensearch.search.asynchronous.context.state.event.SearchFailureEvent) TestThreadPool(org.opensearch.threadpool.TestThreadPool) SearchStartedEvent(org.opensearch.search.asynchronous.context.state.event.SearchStartedEvent) AsynchronousSearchStateMachine(org.opensearch.search.asynchronous.context.state.AsynchronousSearchStateMachine) BeginPersistEvent(org.opensearch.search.asynchronous.context.state.event.BeginPersistEvent) AsynchronousSearchStateMachineClosedException(org.opensearch.search.asynchronous.context.state.AsynchronousSearchStateMachineClosedException) TimeValue(org.opensearch.common.unit.TimeValue) SearchDeletedEvent(org.opensearch.search.asynchronous.context.state.event.SearchDeletedEvent) AsynchronousSearchContextId(org.opensearch.search.asynchronous.context.AsynchronousSearchContextId) AsynchronousSearchActiveContext(org.opensearch.search.asynchronous.context.active.AsynchronousSearchActiveContext) AsynchronousSearchActiveStore(org.opensearch.search.asynchronous.context.active.AsynchronousSearchActiveStore) AsynchronousSearchProgressListener(org.opensearch.search.asynchronous.listener.AsynchronousSearchProgressListener) AsynchronousSearchTask(org.opensearch.search.asynchronous.task.AsynchronousSearchTask) TestUtils.createClusterService(org.opensearch.search.asynchronous.utils.TestUtils.createClusterService) ClusterService(org.opensearch.cluster.service.ClusterService) AtomicInteger(java.util.concurrent.atomic.AtomicInteger)

Example 3 with SearchDeletedEvent

use of org.opensearch.search.asynchronous.context.state.event.SearchDeletedEvent in project asynchronous-search by opensearch-project.

the class AsynchronousSearchPostProcessorTests method testProcessSearchResponseOnClosedContext.

public void testProcessSearchResponseOnClosedContext() throws AsynchronousSearchStateMachineClosedException {
    DiscoveryNode discoveryNode = new DiscoveryNode("node", OpenSearchTestCase.buildNewFakeTransportAddress(), Collections.emptyMap(), DiscoveryNodeRole.BUILT_IN_ROLES, Version.CURRENT);
    AtomicBoolean activeContextCleanUpConsumerInvocation = new AtomicBoolean();
    ThreadPool testThreadPool = null;
    try {
        testThreadPool = new TestThreadPool(this.getClass().getName(), executorBuilder);
        ClusterService mockClusterService = ClusterServiceUtils.createClusterService(testThreadPool, discoveryNode, clusterSettings);
        FakeClient fakeClient = new FakeClient(testThreadPool);
        AsynchronousSearchActiveStore asActiveStore = new AsynchronousSearchActiveStore(mockClusterService);
        AsynchronousSearchPersistenceService persistenceService = new AsynchronousSearchPersistenceService(fakeClient, mockClusterService, testThreadPool);
        AsynchronousSearchService asService = new AsynchronousSearchService(persistenceService, asActiveStore, fakeClient, mockClusterService, testThreadPool, new InternalAsynchronousSearchStats(), new NamedWriteableRegistry(emptyList()));
        AsynchronousSearchStateMachine asStateMachine = asService.getStateMachine();
        ClusterService clusterService = ClusterServiceUtils.createClusterService(testThreadPool, discoveryNode, clusterSettings);
        AsynchronousSearchPostProcessor postProcessor = new AsynchronousSearchPostProcessor(persistenceService, asActiveStore, asStateMachine, (context) -> activeContextCleanUpConsumerInvocation.compareAndSet(false, true), testThreadPool, clusterService);
        SubmitAsynchronousSearchRequest submitAsynchronousSearchRequest = new SubmitAsynchronousSearchRequest(new SearchRequest());
        submitAsynchronousSearchRequest.keepOnCompletion(true);
        submitAsynchronousSearchRequest.keepAlive(TimeValue.timeValueHours(1));
        AsynchronousSearchActiveContext context = (AsynchronousSearchActiveContext) asService.createAndStoreContext(submitAsynchronousSearchRequest, System.currentTimeMillis(), () -> InternalAggregationTestCase.emptyReduceContextBuilder(), null);
        asStateMachine.trigger(new SearchStartedEvent(context, new SearchTask(0, "n/a", "n/a", () -> "test", null, Collections.emptyMap())));
        asStateMachine.trigger(new SearchDeletedEvent(context));
        SearchResponse mockSearchResponse = getMockSearchResponse();
        AsynchronousSearchResponse asResponse = postProcessor.processSearchResponse(mockSearchResponse, context.getContextId());
        assertNull(asResponse.getId());
        assertNull(asResponse.getError());
        assertEquals(-1L, asResponse.getExpirationTimeMillis());
        assertEquals(AsynchronousSearchState.SUCCEEDED, asResponse.getState());
        assertEquals(-1L, asResponse.getStartTimeMillis());
        AsynchronousSearchAssertions.assertSearchResponses(mockSearchResponse, asResponse.getSearchResponse());
        assertFalse(activeContextCleanUpConsumerInvocation.get());
        assertEquals(0, fakeClient.persistenceCount);
    } finally {
        ThreadPool.terminate(testThreadPool, 200, TimeUnit.MILLISECONDS);
    }
}
Also used : NamedWriteableRegistry(org.opensearch.common.io.stream.NamedWriteableRegistry) AsynchronousSearchResponse(org.opensearch.search.asynchronous.response.AsynchronousSearchResponse) SubmitAsynchronousSearchRequest(org.opensearch.search.asynchronous.request.SubmitAsynchronousSearchRequest) SearchRequest(org.opensearch.action.search.SearchRequest) DiscoveryNode(org.opensearch.cluster.node.DiscoveryNode) SubmitAsynchronousSearchRequest(org.opensearch.search.asynchronous.request.SubmitAsynchronousSearchRequest) TestThreadPool(org.opensearch.threadpool.TestThreadPool) ThreadPool(org.opensearch.threadpool.ThreadPool) InternalAsynchronousSearchStats(org.opensearch.search.asynchronous.stats.InternalAsynchronousSearchStats) TestThreadPool(org.opensearch.threadpool.TestThreadPool) AsynchronousSearchActiveContext(org.opensearch.search.asynchronous.context.active.AsynchronousSearchActiveContext) AsynchronousSearchActiveStore(org.opensearch.search.asynchronous.context.active.AsynchronousSearchActiveStore) AsynchronousSearchResponse(org.opensearch.search.asynchronous.response.AsynchronousSearchResponse) SearchResponse(org.opensearch.action.search.SearchResponse) InternalSearchResponse(org.opensearch.search.internal.InternalSearchResponse) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) ClusterService(org.opensearch.cluster.service.ClusterService) SearchTask(org.opensearch.action.search.SearchTask) SearchStartedEvent(org.opensearch.search.asynchronous.context.state.event.SearchStartedEvent) AsynchronousSearchStateMachine(org.opensearch.search.asynchronous.context.state.AsynchronousSearchStateMachine) AsynchronousSearchPostProcessor(org.opensearch.search.asynchronous.processor.AsynchronousSearchPostProcessor) SearchDeletedEvent(org.opensearch.search.asynchronous.context.state.event.SearchDeletedEvent)

Aggregations

DiscoveryNode (org.opensearch.cluster.node.DiscoveryNode)3 ClusterService (org.opensearch.cluster.service.ClusterService)3 NamedWriteableRegistry (org.opensearch.common.io.stream.NamedWriteableRegistry)3 AsynchronousSearchActiveContext (org.opensearch.search.asynchronous.context.active.AsynchronousSearchActiveContext)3 AsynchronousSearchActiveStore (org.opensearch.search.asynchronous.context.active.AsynchronousSearchActiveStore)3 AsynchronousSearchStateMachine (org.opensearch.search.asynchronous.context.state.AsynchronousSearchStateMachine)3 SearchDeletedEvent (org.opensearch.search.asynchronous.context.state.event.SearchDeletedEvent)3 SearchStartedEvent (org.opensearch.search.asynchronous.context.state.event.SearchStartedEvent)3 TestThreadPool (org.opensearch.threadpool.TestThreadPool)3 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)2 SearchRequest (org.opensearch.action.search.SearchRequest)2 SearchTask (org.opensearch.action.search.SearchTask)2 AsynchronousSearchPostProcessor (org.opensearch.search.asynchronous.processor.AsynchronousSearchPostProcessor)2 SubmitAsynchronousSearchRequest (org.opensearch.search.asynchronous.request.SubmitAsynchronousSearchRequest)2 AsynchronousSearchResponse (org.opensearch.search.asynchronous.response.AsynchronousSearchResponse)2 InternalAsynchronousSearchStats (org.opensearch.search.asynchronous.stats.InternalAsynchronousSearchStats)2 ThreadPool (org.opensearch.threadpool.ThreadPool)2 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)1 SearchPhaseExecutionException (org.opensearch.action.search.SearchPhaseExecutionException)1 SearchResponse (org.opensearch.action.search.SearchResponse)1