Search in sources :

Example 71 with ActionListener

use of org.elasticsearch.action.ActionListener in project elasticsearch by elastic.

the class TransportReplicationAction method acquirePrimaryShardReference.

/**
     * Tries to acquire reference to {@link IndexShard} to perform a primary operation. Released after performing primary operation locally
     * and replication of the operation to all replica shards is completed / failed (see {@link ReplicationOperation}).
     */
private void acquirePrimaryShardReference(ShardId shardId, String allocationId, ActionListener<PrimaryShardReference> onReferenceAcquired) {
    IndexShard indexShard = getIndexShard(shardId);
    // the replica will take over and a replica will be assigned to the first node.
    if (indexShard.routingEntry().primary() == false) {
        throw new ReplicationOperation.RetryOnPrimaryException(indexShard.shardId(), "actual shard is not a primary " + indexShard.routingEntry());
    }
    final String actualAllocationId = indexShard.routingEntry().allocationId().getId();
    if (actualAllocationId.equals(allocationId) == false) {
        throw new ShardNotFoundException(shardId, "expected aID [{}] but found [{}]", allocationId, actualAllocationId);
    }
    ActionListener<Releasable> onAcquired = new ActionListener<Releasable>() {

        @Override
        public void onResponse(Releasable releasable) {
            onReferenceAcquired.onResponse(new PrimaryShardReference(indexShard, releasable));
        }

        @Override
        public void onFailure(Exception e) {
            onReferenceAcquired.onFailure(e);
        }
    };
    indexShard.acquirePrimaryOperationLock(onAcquired, executor);
}
Also used : ShardNotFoundException(org.elasticsearch.index.shard.ShardNotFoundException) ActionListener(org.elasticsearch.action.ActionListener) IndexShard(org.elasticsearch.index.shard.IndexShard) Releasable(org.elasticsearch.common.lease.Releasable) ElasticsearchException(org.elasticsearch.ElasticsearchException) IndexClosedException(org.elasticsearch.indices.IndexClosedException) NodeClosedException(org.elasticsearch.node.NodeClosedException) ShardNotFoundException(org.elasticsearch.index.shard.ShardNotFoundException) ConnectTransportException(org.elasticsearch.transport.ConnectTransportException) IndexNotFoundException(org.elasticsearch.index.IndexNotFoundException) NoNodeAvailableException(org.elasticsearch.client.transport.NoNodeAvailableException) TransportException(org.elasticsearch.transport.TransportException) ClusterBlockException(org.elasticsearch.cluster.block.ClusterBlockException) IOException(java.io.IOException) UnavailableShardsException(org.elasticsearch.action.UnavailableShardsException)

Example 72 with ActionListener

use of org.elasticsearch.action.ActionListener in project elasticsearch by elastic.

the class TransportTasksAction method nodeOperation.

private void nodeOperation(NodeTaskRequest nodeTaskRequest, ActionListener<NodeTasksResponse> listener) {
    TasksRequest request = nodeTaskRequest.tasksRequest;
    List<OperationTask> tasks = new ArrayList<>();
    processTasks(request, tasks::add);
    if (tasks.isEmpty()) {
        listener.onResponse(new NodeTasksResponse(clusterService.localNode().getId(), emptyList(), emptyList()));
        return;
    }
    AtomicArray<Tuple<TaskResponse, Exception>> responses = new AtomicArray<>(tasks.size());
    final AtomicInteger counter = new AtomicInteger(tasks.size());
    for (int i = 0; i < tasks.size(); i++) {
        final int taskIndex = i;
        ActionListener<TaskResponse> taskListener = new ActionListener<TaskResponse>() {

            @Override
            public void onResponse(TaskResponse response) {
                responses.setOnce(taskIndex, response == null ? null : new Tuple<>(response, null));
                respondIfFinished();
            }

            @Override
            public void onFailure(Exception e) {
                responses.setOnce(taskIndex, new Tuple<>(null, e));
                respondIfFinished();
            }

            private void respondIfFinished() {
                if (counter.decrementAndGet() != 0) {
                    return;
                }
                List<TaskResponse> results = new ArrayList<>();
                List<TaskOperationFailure> exceptions = new ArrayList<>();
                for (AtomicArray.Entry<Tuple<TaskResponse, Exception>> response : responses.asList()) {
                    if (response.value.v1() == null) {
                        assert response.value.v2() != null;
                        exceptions.add(new TaskOperationFailure(clusterService.localNode().getId(), tasks.get(taskIndex).getId(), response.value.v2()));
                    } else {
                        assert response.value.v2() == null;
                        results.add(response.value.v1());
                    }
                }
                listener.onResponse(new NodeTasksResponse(clusterService.localNode().getId(), results, exceptions));
            }
        };
        try {
            taskOperation(request, tasks.get(taskIndex), taskListener);
        } catch (Exception e) {
            taskListener.onFailure(e);
        }
    }
}
Also used : AtomicArray(org.elasticsearch.common.util.concurrent.AtomicArray) ArrayList(java.util.ArrayList) FailedNodeException(org.elasticsearch.action.FailedNodeException) NodeShouldNotConnectException(org.elasticsearch.transport.NodeShouldNotConnectException) ResourceNotFoundException(org.elasticsearch.ResourceNotFoundException) NoSuchNodeException(org.elasticsearch.action.NoSuchNodeException) IOException(java.io.IOException) TransportException(org.elasticsearch.transport.TransportException) ActionListener(org.elasticsearch.action.ActionListener) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) TaskOperationFailure(org.elasticsearch.action.TaskOperationFailure) Tuple(org.elasticsearch.common.collect.Tuple)

Example 73 with ActionListener

use of org.elasticsearch.action.ActionListener in project elasticsearch by elastic.

the class ReplicationOperation method performOnReplica.

private void performOnReplica(final ShardRouting shard, final ReplicaRequest replicaRequest) {
    if (logger.isTraceEnabled()) {
        logger.trace("[{}] sending op [{}] to replica {} for request [{}]", shard.shardId(), opType, shard, replicaRequest);
    }
    totalShards.incrementAndGet();
    pendingActions.incrementAndGet();
    replicasProxy.performOn(shard, replicaRequest, new ActionListener<ReplicaResponse>() {

        @Override
        public void onResponse(ReplicaResponse response) {
            successfulShards.incrementAndGet();
            primary.updateLocalCheckpointForShard(response.allocationId(), response.localCheckpoint());
            decPendingAndFinishIfNeeded();
        }

        @Override
        public void onFailure(Exception replicaException) {
            logger.trace((org.apache.logging.log4j.util.Supplier<?>) () -> new ParameterizedMessage("[{}] failure while performing [{}] on replica {}, request [{}]", shard.shardId(), opType, shard, replicaRequest), replicaException);
            if (TransportActions.isShardNotAvailableException(replicaException)) {
                decPendingAndFinishIfNeeded();
            } else {
                RestStatus restStatus = ExceptionsHelper.status(replicaException);
                shardReplicaFailures.add(new ReplicationResponse.ShardInfo.Failure(shard.shardId(), shard.currentNodeId(), replicaException, restStatus, false));
                String message = String.format(Locale.ROOT, "failed to perform %s on replica %s", opType, shard);
                replicasProxy.failShardIfNeeded(shard, replicaRequest.primaryTerm(), message, replicaException, ReplicationOperation.this::decPendingAndFinishIfNeeded, ReplicationOperation.this::onPrimaryDemoted, throwable -> decPendingAndFinishIfNeeded());
            }
        }
    });
}
Also used : ElasticsearchException(org.elasticsearch.ElasticsearchException) ShardRouting(org.elasticsearch.cluster.routing.ShardRouting) ShardId(org.elasticsearch.index.shard.ShardId) Nullable(org.elasticsearch.common.Nullable) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) ParameterizedMessage(org.apache.logging.log4j.message.ParameterizedMessage) Supplier(java.util.function.Supplier) TransportActions(org.elasticsearch.action.support.TransportActions) ArrayList(java.util.ArrayList) ClusterState(org.elasticsearch.cluster.ClusterState) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Locale(java.util.Locale) VersionConflictEngineException(org.elasticsearch.index.engine.VersionConflictEngineException) Set(java.util.Set) IndexShardRoutingTable(org.elasticsearch.cluster.routing.IndexShardRoutingTable) IOException(java.io.IOException) Collectors(java.util.stream.Collectors) Sets(org.elasticsearch.common.util.set.Sets) UnavailableShardsException(org.elasticsearch.action.UnavailableShardsException) ActiveShardCount(org.elasticsearch.action.support.ActiveShardCount) Objects(java.util.Objects) Consumer(java.util.function.Consumer) ExceptionsHelper(org.elasticsearch.ExceptionsHelper) List(java.util.List) Logger(org.apache.logging.log4j.Logger) IndexRoutingTable(org.elasticsearch.cluster.routing.IndexRoutingTable) IndexMetaData(org.elasticsearch.cluster.metadata.IndexMetaData) StreamInput(org.elasticsearch.common.io.stream.StreamInput) RestStatus(org.elasticsearch.rest.RestStatus) AllocationId(org.elasticsearch.cluster.routing.AllocationId) Collections(java.util.Collections) ActionListener(org.elasticsearch.action.ActionListener) ElasticsearchException(org.elasticsearch.ElasticsearchException) VersionConflictEngineException(org.elasticsearch.index.engine.VersionConflictEngineException) IOException(java.io.IOException) UnavailableShardsException(org.elasticsearch.action.UnavailableShardsException) RestStatus(org.elasticsearch.rest.RestStatus) Supplier(java.util.function.Supplier) ParameterizedMessage(org.apache.logging.log4j.message.ParameterizedMessage)

Example 74 with ActionListener

use of org.elasticsearch.action.ActionListener in project crate by crate.

the class TransportKillNodeAction method broadcast.

public void broadcast(Request request, ActionListener<KillResponse> listener, Collection<String> excludedNodeIds) {
    Stream<DiscoveryNode> nodes = StreamSupport.stream(clusterService.state().nodes().spliterator(), false);
    Collection<DiscoveryNode> filteredNodes = nodes.filter(node -> !excludedNodeIds.contains(node.getId())).collect(Collectors.toList());
    listener = new MultiActionListener<>(filteredNodes.size(), KillResponse.MERGE_FUNCTION, listener);
    DefaultTransportResponseHandler<KillResponse> responseHandler = new DefaultTransportResponseHandler<KillResponse>(listener) {

        @Override
        public KillResponse newInstance() {
            return new KillResponse(0);
        }
    };
    for (DiscoveryNode node : filteredNodes) {
        transportService.sendRequest(node, name, request, responseHandler);
    }
}
Also used : java.util(java.util) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) AbstractComponent(org.elasticsearch.common.component.AbstractComponent) TransportRequest(org.elasticsearch.transport.TransportRequest) DefaultTransportResponseHandler(io.crate.executor.transport.DefaultTransportResponseHandler) Callable(java.util.concurrent.Callable) MultiActionListener(io.crate.executor.MultiActionListener) Collectors(java.util.stream.Collectors) FutureCallback(com.google.common.util.concurrent.FutureCallback) NodeActionRequestHandler(io.crate.executor.transport.NodeActionRequestHandler) Futures(com.google.common.util.concurrent.Futures) DiscoveryNode(org.elasticsearch.cluster.node.DiscoveryNode) Settings(org.elasticsearch.common.settings.Settings) Stream(java.util.stream.Stream) NodeAction(io.crate.executor.transport.NodeAction) ClusterService(org.elasticsearch.cluster.ClusterService) JobContextService(io.crate.jobs.JobContextService) ThreadPool(org.elasticsearch.threadpool.ThreadPool) StreamSupport(java.util.stream.StreamSupport) TransportService(org.elasticsearch.transport.TransportService) Nonnull(javax.annotation.Nonnull) ActionListener(org.elasticsearch.action.ActionListener) Nullable(javax.annotation.Nullable) DiscoveryNode(org.elasticsearch.cluster.node.DiscoveryNode) DefaultTransportResponseHandler(io.crate.executor.transport.DefaultTransportResponseHandler)

Example 75 with ActionListener

use of org.elasticsearch.action.ActionListener in project crate by crate.

the class RepositoryServiceTest method testRepositoryIsDroppedOnFailure.

@Test
public void testRepositoryIsDroppedOnFailure() throws Throwable {
    expectedException.expect(RepositoryException.class);
    // add repo to cluster service so that it exists..
    RepositoriesMetaData repos = new RepositoriesMetaData(new RepositoryMetaData("repo1", "fs", Settings.EMPTY));
    ClusterState state = ClusterState.builder(new ClusterName("dummy")).metaData(MetaData.builder().putCustom(RepositoriesMetaData.TYPE, repos)).build();
    ClusterService clusterService = new NoopClusterService(state);
    final ActionFilters actionFilters = mock(ActionFilters.class, Answers.RETURNS_MOCKS.get());
    IndexNameExpressionResolver indexNameExpressionResolver = new IndexNameExpressionResolver(Settings.EMPTY);
    final AtomicBoolean deleteRepoCalled = new AtomicBoolean(false);
    TransportDeleteRepositoryAction deleteRepositoryAction = new TransportDeleteRepositoryAction(Settings.EMPTY, mock(TransportService.class, Answers.RETURNS_MOCKS.get()), clusterService, mock(RepositoriesService.class), threadPool, actionFilters, indexNameExpressionResolver) {

        @Override
        protected void doExecute(Task task, DeleteRepositoryRequest request, ActionListener<DeleteRepositoryResponse> listener) {
            deleteRepoCalled.set(true);
            listener.onResponse(mock(DeleteRepositoryResponse.class));
        }
    };
    TransportPutRepositoryAction putRepo = new TransportPutRepositoryAction(Settings.EMPTY, mock(TransportService.class, Answers.RETURNS_MOCKS.get()), clusterService, mock(RepositoriesService.class), threadPool, actionFilters, indexNameExpressionResolver) {

        @Override
        protected void doExecute(Task task, PutRepositoryRequest request, ActionListener<PutRepositoryResponse> listener) {
            listener.onFailure(new RepositoryException(request.name(), "failure"));
        }
    };
    RepositoryService repositoryService = new RepositoryService(clusterService, deleteRepositoryAction, putRepo);
    try {
        repositoryService.execute(new CreateRepositoryAnalyzedStatement("repo1", "fs", Settings.EMPTY)).get(10, TimeUnit.SECONDS);
    } catch (ExecutionException e) {
        assertThat(deleteRepoCalled.get(), is(true));
        throw e.getCause();
    }
}
Also used : RepositoriesMetaData(org.elasticsearch.cluster.metadata.RepositoriesMetaData) ClusterState(org.elasticsearch.cluster.ClusterState) Task(org.elasticsearch.tasks.Task) TransportDeleteRepositoryAction(org.elasticsearch.action.admin.cluster.repositories.delete.TransportDeleteRepositoryAction) DeleteRepositoryRequest(org.elasticsearch.action.admin.cluster.repositories.delete.DeleteRepositoryRequest) RepositoryException(org.elasticsearch.repositories.RepositoryException) ActionFilters(org.elasticsearch.action.support.ActionFilters) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) CreateRepositoryAnalyzedStatement(io.crate.analyze.CreateRepositoryAnalyzedStatement) NoopClusterService(org.elasticsearch.test.cluster.NoopClusterService) ClusterService(org.elasticsearch.cluster.ClusterService) DeleteRepositoryResponse(org.elasticsearch.action.admin.cluster.repositories.delete.DeleteRepositoryResponse) ActionListener(org.elasticsearch.action.ActionListener) TransportService(org.elasticsearch.transport.TransportService) RepositoriesService(org.elasticsearch.repositories.RepositoriesService) RepositoryMetaData(org.elasticsearch.cluster.metadata.RepositoryMetaData) ClusterName(org.elasticsearch.cluster.ClusterName) IndexNameExpressionResolver(org.elasticsearch.cluster.metadata.IndexNameExpressionResolver) TransportPutRepositoryAction(org.elasticsearch.action.admin.cluster.repositories.put.TransportPutRepositoryAction) ExecutionException(java.util.concurrent.ExecutionException) NoopClusterService(org.elasticsearch.test.cluster.NoopClusterService) PutRepositoryRequest(org.elasticsearch.action.admin.cluster.repositories.put.PutRepositoryRequest) Test(org.junit.Test) CrateUnitTest(io.crate.test.integration.CrateUnitTest)

Aggregations

ActionListener (org.elasticsearch.action.ActionListener)148 IOException (java.io.IOException)75 List (java.util.List)58 ThreadPool (org.elasticsearch.threadpool.ThreadPool)53 ClusterState (org.elasticsearch.cluster.ClusterState)50 ArrayList (java.util.ArrayList)46 Settings (org.elasticsearch.common.settings.Settings)45 Map (java.util.Map)42 DiscoveryNode (org.elasticsearch.cluster.node.DiscoveryNode)41 ShardId (org.elasticsearch.index.shard.ShardId)40 Collections (java.util.Collections)39 Set (java.util.Set)39 ClusterService (org.elasticsearch.cluster.service.ClusterService)35 Logger (org.apache.logging.log4j.Logger)34 HashMap (java.util.HashMap)32 TransportService (org.elasticsearch.transport.TransportService)32 ElasticsearchException (org.elasticsearch.ElasticsearchException)31 Collectors (java.util.stream.Collectors)30 Index (org.elasticsearch.index.Index)30 Test (org.junit.Test)30