Search in sources :

Example 1 with VotingConfigExclusion

use of org.elasticsearch.cluster.coordination.CoordinationMetadata.VotingConfigExclusion in project crate by crate.

the class AddVotingConfigExclusionsRequest method resolveVotingConfigExclusions.

Set<VotingConfigExclusion> resolveVotingConfigExclusions(ClusterState currentState) {
    final DiscoveryNodes allNodes = currentState.nodes();
    final Set<VotingConfigExclusion> resolvedNodes = Arrays.stream(allNodes.resolveNodes(nodeDescriptions)).map(allNodes::get).filter(DiscoveryNode::isMasterEligibleNode).map(VotingConfigExclusion::new).collect(Collectors.toSet());
    if (resolvedNodes.isEmpty()) {
        throw new IllegalArgumentException("add voting config exclusions request for " + Arrays.asList(nodeDescriptions) + " matched no master-eligible nodes");
    }
    resolvedNodes.removeIf(n -> currentState.getVotingConfigExclusions().contains(n));
    return resolvedNodes;
}
Also used : VotingConfigExclusion(org.elasticsearch.cluster.coordination.CoordinationMetadata.VotingConfigExclusion) DiscoveryNode(org.elasticsearch.cluster.node.DiscoveryNode) DiscoveryNodes(org.elasticsearch.cluster.node.DiscoveryNodes)

Example 2 with VotingConfigExclusion

use of org.elasticsearch.cluster.coordination.CoordinationMetadata.VotingConfigExclusion in project crate by crate.

the class TransportClearVotingConfigExclusionsAction method masterOperation.

@Override
protected void masterOperation(ClearVotingConfigExclusionsRequest request, ClusterState initialState, ActionListener<ClearVotingConfigExclusionsResponse> listener) throws Exception {
    final long startTimeMillis = threadPool.relativeTimeInMillis();
    final Predicate<ClusterState> allExclusionsRemoved = newState -> {
        for (VotingConfigExclusion tombstone : initialState.getVotingConfigExclusions()) {
            // NB checking for the existence of any node with this persistent ID, because persistent IDs are how votes are counted.
            if (newState.nodes().nodeExists(tombstone.getNodeId())) {
                return false;
            }
        }
        return true;
    };
    if (request.getWaitForRemoval() && allExclusionsRemoved.test(initialState) == false) {
        final ClusterStateObserver clusterStateObserver = new ClusterStateObserver(initialState, clusterService, request.getTimeout(), logger);
        clusterStateObserver.waitForNextChange(new Listener() {

            @Override
            public void onNewClusterState(ClusterState state) {
                submitClearVotingConfigExclusionsTask(request, startTimeMillis, listener);
            }

            @Override
            public void onClusterServiceClose() {
                listener.onFailure(new ElasticsearchException("cluster service closed while waiting for removal of nodes " + initialState.getVotingConfigExclusions()));
            }

            @Override
            public void onTimeout(TimeValue timeout) {
                listener.onFailure(new ElasticsearchTimeoutException("timed out waiting for removal of nodes; if nodes should not be removed, set waitForRemoval to false. " + initialState.getVotingConfigExclusions()));
            }
        }, allExclusionsRemoved);
    } else {
        submitClearVotingConfigExclusionsTask(request, startTimeMillis, listener);
    }
}
Also used : ElasticsearchException(org.elasticsearch.ElasticsearchException) Priority(org.elasticsearch.common.Priority) Listener(org.elasticsearch.cluster.ClusterStateObserver.Listener) Predicate(java.util.function.Predicate) ClusterService(org.elasticsearch.cluster.service.ClusterService) IOException(java.io.IOException) CoordinationMetadata(org.elasticsearch.cluster.coordination.CoordinationMetadata) Names(org.elasticsearch.threadpool.ThreadPool.Names) Inject(org.elasticsearch.common.inject.Inject) TransportMasterNodeAction(org.elasticsearch.action.support.master.TransportMasterNodeAction) ElasticsearchTimeoutException(org.elasticsearch.ElasticsearchTimeoutException) ClusterState(org.elasticsearch.cluster.ClusterState) Metadata(org.elasticsearch.cluster.metadata.Metadata) ClusterStateUpdateTask(org.elasticsearch.cluster.ClusterStateUpdateTask) VotingConfigExclusion(org.elasticsearch.cluster.coordination.CoordinationMetadata.VotingConfigExclusion) ClusterBlockException(org.elasticsearch.cluster.block.ClusterBlockException) StreamInput(org.elasticsearch.common.io.stream.StreamInput) TimeValue(io.crate.common.unit.TimeValue) ThreadPool(org.elasticsearch.threadpool.ThreadPool) TransportService(org.elasticsearch.transport.TransportService) ClusterStateObserver(org.elasticsearch.cluster.ClusterStateObserver) ClusterBlockLevel(org.elasticsearch.cluster.block.ClusterBlockLevel) IndexNameExpressionResolver(org.elasticsearch.cluster.metadata.IndexNameExpressionResolver) ActionListener(org.elasticsearch.action.ActionListener) VotingConfigExclusion(org.elasticsearch.cluster.coordination.CoordinationMetadata.VotingConfigExclusion) ClusterState(org.elasticsearch.cluster.ClusterState) ClusterStateObserver(org.elasticsearch.cluster.ClusterStateObserver) Listener(org.elasticsearch.cluster.ClusterStateObserver.Listener) ActionListener(org.elasticsearch.action.ActionListener) ElasticsearchTimeoutException(org.elasticsearch.ElasticsearchTimeoutException) ElasticsearchException(org.elasticsearch.ElasticsearchException) TimeValue(io.crate.common.unit.TimeValue)

Example 3 with VotingConfigExclusion

use of org.elasticsearch.cluster.coordination.CoordinationMetadata.VotingConfigExclusion in project crate by crate.

the class CoordinationMetadataTests method testVotingTombstoneXContent.

public void testVotingTombstoneXContent() throws IOException {
    VotingConfigExclusion originalTombstone = new VotingConfigExclusion(randomAlphaOfLength(10), randomAlphaOfLength(10));
    final XContentBuilder builder = JsonXContent.contentBuilder();
    originalTombstone.toXContent(builder, ToXContent.EMPTY_PARAMS);
    try (XContentParser parser = createParser(JsonXContent.JSON_XCONTENT, BytesReference.bytes(builder))) {
        final VotingConfigExclusion fromXContentTombstone = VotingConfigExclusion.fromXContent(parser);
        assertThat(originalTombstone, equalTo(fromXContentTombstone));
    }
}
Also used : VotingConfigExclusion(org.elasticsearch.cluster.coordination.CoordinationMetadata.VotingConfigExclusion) XContentBuilder(org.elasticsearch.common.xcontent.XContentBuilder) XContentParser(org.elasticsearch.common.xcontent.XContentParser)

Example 4 with VotingConfigExclusion

use of org.elasticsearch.cluster.coordination.CoordinationMetadata.VotingConfigExclusion in project crate by crate.

the class Coordinator method improveConfiguration.

// Package-private for testing
ClusterState improveConfiguration(ClusterState clusterState) {
    assert Thread.holdsLock(mutex) : "Coordinator mutex not held";
    // exclude any nodes whose ID is in the voting config exclusions list ...
    final Stream<String> excludedNodeIds = clusterState.getVotingConfigExclusions().stream().map(VotingConfigExclusion::getNodeId);
    // ... and also automatically exclude the node IDs of master-ineligible nodes that were previously master-eligible and are still in
    // the voting config. We could exclude all the master-ineligible nodes here, but there could be quite a few of them and that makes
    // the logging much harder to follow.
    final Stream<String> masterIneligibleNodeIdsInVotingConfig = StreamSupport.stream(clusterState.nodes().spliterator(), false).filter(n -> n.isMasterEligibleNode() == false && (clusterState.getLastAcceptedConfiguration().getNodeIds().contains(n.getId()) || clusterState.getLastCommittedConfiguration().getNodeIds().contains(n.getId()))).map(DiscoveryNode::getId);
    final Set<DiscoveryNode> liveNodes = StreamSupport.stream(clusterState.nodes().spliterator(), false).filter(DiscoveryNode::isMasterEligibleNode).filter(coordinationState.get()::containsJoinVoteFor).collect(Collectors.toSet());
    final VotingConfiguration newConfig = reconfigurator.reconfigure(liveNodes, Stream.concat(masterIneligibleNodeIdsInVotingConfig, excludedNodeIds).collect(Collectors.toSet()), getLocalNode(), clusterState.getLastAcceptedConfiguration());
    if (newConfig.equals(clusterState.getLastAcceptedConfiguration()) == false) {
        assert coordinationState.get().joinVotesHaveQuorumFor(newConfig);
        return ClusterState.builder(clusterState).metadata(Metadata.builder(clusterState.metadata()).coordinationMetadata(CoordinationMetadata.builder(clusterState.coordinationMetadata()).lastAcceptedConfiguration(newConfig).build())).build();
    }
    return clusterState;
}
Also used : VotingConfigExclusion(org.elasticsearch.cluster.coordination.CoordinationMetadata.VotingConfigExclusion) NO_MASTER_BLOCK_ID(org.elasticsearch.cluster.coordination.NoMasterBlockService.NO_MASTER_BLOCK_ID) RerouteService(org.elasticsearch.cluster.routing.RerouteService) Level(org.apache.logging.log4j.Level) AllocationService(org.elasticsearch.cluster.routing.allocation.AllocationService) Random(java.util.Random) ClusterBlocks(org.elasticsearch.cluster.block.ClusterBlocks) JsonXContent(org.elasticsearch.common.xcontent.json.JsonXContent) STATE_NOT_RECOVERED_BLOCK(org.elasticsearch.gateway.GatewayService.STATE_NOT_RECOVERED_BLOCK) PeerFinder(org.elasticsearch.discovery.PeerFinder) ClusterState(org.elasticsearch.cluster.ClusterState) ClusterStateUpdateTask(org.elasticsearch.cluster.ClusterStateUpdateTask) Settings(org.elasticsearch.common.settings.Settings) VotingConfigExclusion(org.elasticsearch.cluster.coordination.CoordinationMetadata.VotingConfigExclusion) FollowerCheckRequest(org.elasticsearch.cluster.coordination.FollowersChecker.FollowerCheckRequest) ClusterName(org.elasticsearch.cluster.ClusterName) Releasable(org.elasticsearch.common.lease.Releasable) DiscoveryStats(org.elasticsearch.discovery.DiscoveryStats) Priority(org.elasticsearch.common.Priority) InitialJoinAccumulator(org.elasticsearch.cluster.coordination.JoinHelper.InitialJoinAccumulator) Setting(org.elasticsearch.common.settings.Setting) ClusterApplier(org.elasticsearch.cluster.service.ClusterApplier) Collection(java.util.Collection) Set(java.util.Set) ClusterChangedEvent(org.elasticsearch.cluster.ClusterChangedEvent) Collectors(java.util.stream.Collectors) List(java.util.List) Logger(org.apache.logging.log4j.Logger) Stream(java.util.stream.Stream) TransportAddress(org.elasticsearch.common.transport.TransportAddress) ClusterStateUpdaters.hideStateIfNotRecovered(org.elasticsearch.gateway.ClusterStateUpdaters.hideStateIfNotRecovered) TimeValue(io.crate.common.unit.TimeValue) Optional(java.util.Optional) Empty(org.elasticsearch.transport.TransportResponse.Empty) SeedHostsProvider(org.elasticsearch.discovery.SeedHostsProvider) ListenableFuture(org.elasticsearch.common.util.concurrent.ListenableFuture) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) ParameterizedMessage(org.apache.logging.log4j.message.ParameterizedMessage) Names(org.elasticsearch.threadpool.ThreadPool.Names) Supplier(java.util.function.Supplier) Strings(org.elasticsearch.common.Strings) ArrayList(java.util.ArrayList) XContentHelper(org.elasticsearch.common.xcontent.XContentHelper) HashSet(java.util.HashSet) Metadata(org.elasticsearch.cluster.metadata.Metadata) DiscoveryNode(org.elasticsearch.cluster.node.DiscoveryNode) NamedWriteableRegistry(org.elasticsearch.common.io.stream.NamedWriteableRegistry) HandshakingTransportAddressConnector(org.elasticsearch.discovery.HandshakingTransportAddressConnector) DiscoveryModule(org.elasticsearch.discovery.DiscoveryModule) BiConsumer(java.util.function.BiConsumer) VotingConfiguration(org.elasticsearch.cluster.coordination.CoordinationMetadata.VotingConfiguration) StreamSupport(java.util.stream.StreamSupport) TransportService(org.elasticsearch.transport.TransportService) Nullable(javax.annotation.Nullable) DiscoveryNodes(org.elasticsearch.cluster.node.DiscoveryNodes) EsExecutors(org.elasticsearch.common.util.concurrent.EsExecutors) SetOnce(org.apache.lucene.util.SetOnce) MasterService(org.elasticsearch.cluster.service.MasterService) Discovery(org.elasticsearch.discovery.Discovery) ClusterStateTaskConfig(org.elasticsearch.cluster.ClusterStateTaskConfig) IOException(java.io.IOException) AbstractLifecycleComponent(org.elasticsearch.common.component.AbstractLifecycleComponent) Scheduler(org.elasticsearch.threadpool.Scheduler) ClusterSettings(org.elasticsearch.common.settings.ClusterSettings) SeedHostsResolver(org.elasticsearch.discovery.SeedHostsResolver) LocalClusterUpdateTask(org.elasticsearch.cluster.LocalClusterUpdateTask) ClusterApplyListener(org.elasticsearch.cluster.service.ClusterApplier.ClusterApplyListener) LogManager(org.apache.logging.log4j.LogManager) Collections(java.util.Collections) ActionListener(org.elasticsearch.action.ActionListener) ClusterFormationState(org.elasticsearch.cluster.coordination.ClusterFormationFailureHelper.ClusterFormationState) DiscoveryNode(org.elasticsearch.cluster.node.DiscoveryNode) VotingConfiguration(org.elasticsearch.cluster.coordination.CoordinationMetadata.VotingConfiguration)

Example 5 with VotingConfigExclusion

use of org.elasticsearch.cluster.coordination.CoordinationMetadata.VotingConfigExclusion in project crate by crate.

the class TransportAddVotingConfigExclusionsActionTests method createThreadPoolAndClusterService.

@BeforeClass
public static void createThreadPoolAndClusterService() {
    threadPool = new TestThreadPool("test", Settings.EMPTY);
    localNode = makeDiscoveryNode("local");
    localNodeExclusion = new VotingConfigExclusion(localNode);
    otherNode1 = makeDiscoveryNode("other1");
    otherNode1Exclusion = new VotingConfigExclusion(otherNode1);
    otherNode2 = makeDiscoveryNode("other2");
    otherNode2Exclusion = new VotingConfigExclusion(otherNode2);
    otherDataNode = new DiscoveryNode("data", "data", buildNewFakeTransportAddress(), emptyMap(), emptySet(), Version.CURRENT);
    clusterService = createClusterService(threadPool, localNode);
}
Also used : VotingConfigExclusion(org.elasticsearch.cluster.coordination.CoordinationMetadata.VotingConfigExclusion) DiscoveryNode(org.elasticsearch.cluster.node.DiscoveryNode) TestThreadPool(org.elasticsearch.threadpool.TestThreadPool) BeforeClass(org.junit.BeforeClass)

Aggregations

VotingConfigExclusion (org.elasticsearch.cluster.coordination.CoordinationMetadata.VotingConfigExclusion)11 DiscoveryNode (org.elasticsearch.cluster.node.DiscoveryNode)6 ClusterState (org.elasticsearch.cluster.ClusterState)5 IOException (java.io.IOException)4 TimeValue (io.crate.common.unit.TimeValue)3 Set (java.util.Set)3 ActionListener (org.elasticsearch.action.ActionListener)3 ClusterName (org.elasticsearch.cluster.ClusterName)3 ClusterStateUpdateTask (org.elasticsearch.cluster.ClusterStateUpdateTask)3 Metadata (org.elasticsearch.cluster.metadata.Metadata)3 Collections (java.util.Collections)2 HashSet (java.util.HashSet)2 Predicate (java.util.function.Predicate)2 ElasticsearchException (org.elasticsearch.ElasticsearchException)2 ElasticsearchTimeoutException (org.elasticsearch.ElasticsearchTimeoutException)2 ClusterStateObserver (org.elasticsearch.cluster.ClusterStateObserver)2 CoordinationMetadata (org.elasticsearch.cluster.coordination.CoordinationMetadata)2 DiscoveryNodes (org.elasticsearch.cluster.node.DiscoveryNodes)2 Builder (org.elasticsearch.cluster.node.DiscoveryNodes.Builder)2 Priority (org.elasticsearch.common.Priority)2