Search in sources :

Example 1 with ZenPing

use of org.elasticsearch.discovery.zen.ZenPing in project elasticsearch by elastic.

the class DiscoveryWithServiceDisruptionsIT method testUnicastSinglePingResponseContainsMaster.

/**
     * A 4 node cluster with m_m_n set to 3 and each node has one unicast endpoint. One node partitions from the master node.
     * The temporal unicast responses is empty. When partition is solved the one ping response contains a master node.
     * The rejoining node should take this master node and connect.
     */
public void testUnicastSinglePingResponseContainsMaster() throws Exception {
    List<String> nodes = startCluster(4, -1, new int[] { 0 });
    // Figure out what is the elected master node
    final String masterNode = internalCluster().getMasterName();
    logger.info("---> legit elected master node={}", masterNode);
    List<String> otherNodes = new ArrayList<>(nodes);
    otherNodes.remove(masterNode);
    // <-- Don't isolate the node that is in the unicast endpoint for all the other nodes.
    otherNodes.remove(nodes.get(0));
    final String isolatedNode = otherNodes.get(0);
    // Forcefully clean temporal response lists on all nodes. Otherwise the node in the unicast host list
    // includes all the other nodes that have pinged it and the issue doesn't manifest
    ZenPing zenPing = ((TestZenDiscovery) internalCluster().getInstance(Discovery.class)).getZenPing();
    if (zenPing instanceof UnicastZenPing) {
        ((UnicastZenPing) zenPing).clearTemporalResponses();
    }
    // Simulate a network issue between the unlucky node and elected master node in both directions.
    NetworkDisruption networkDisconnect = new NetworkDisruption(new TwoPartitions(masterNode, isolatedNode), new NetworkDisconnect());
    setDisruptionScheme(networkDisconnect);
    networkDisconnect.startDisrupting();
    // Wait until elected master has removed that the unlucky node...
    ensureStableCluster(3, masterNode);
    // The isolate master node must report no master, so it starts with pinging
    assertNoMaster(isolatedNode);
    networkDisconnect.stopDisrupting();
    // Wait until the master node sees all 4 nodes again.
    ensureStableCluster(4);
    // The elected master shouldn't have changed, since the isolated node never could have elected himself as
    // master since m_m_n of 3 could never be satisfied.
    assertMaster(masterNode, nodes);
}
Also used : UnicastZenPing(org.elasticsearch.discovery.zen.UnicastZenPing) TwoPartitions(org.elasticsearch.test.disruption.NetworkDisruption.TwoPartitions) ArrayList(java.util.ArrayList) ZenDiscovery(org.elasticsearch.discovery.zen.ZenDiscovery) TestZenDiscovery(org.elasticsearch.test.discovery.TestZenDiscovery) ZenPing(org.elasticsearch.discovery.zen.ZenPing) UnicastZenPing(org.elasticsearch.discovery.zen.UnicastZenPing) TestZenDiscovery(org.elasticsearch.test.discovery.TestZenDiscovery) NetworkDisruption(org.elasticsearch.test.disruption.NetworkDisruption) NetworkDisconnect(org.elasticsearch.test.disruption.NetworkDisruption.NetworkDisconnect)

Example 2 with ZenPing

use of org.elasticsearch.discovery.zen.ZenPing in project elasticsearch by elastic.

the class DiscoveryWithServiceDisruptionsIT method startCluster.

private List<String> startCluster(int numberOfNodes, int minimumMasterNode, @Nullable int[] unicastHostsOrdinals) throws ExecutionException, InterruptedException {
    configureCluster(numberOfNodes, unicastHostsOrdinals, minimumMasterNode);
    List<String> nodes = internalCluster().startNodes(numberOfNodes);
    ensureStableCluster(numberOfNodes);
    // TODO: this is a temporary solution so that nodes will not base their reaction to a partition based on previous successful results
    ZenPing zenPing = ((TestZenDiscovery) internalCluster().getInstance(Discovery.class)).getZenPing();
    if (zenPing instanceof UnicastZenPing) {
        ((UnicastZenPing) zenPing).clearTemporalResponses();
    }
    return nodes;
}
Also used : UnicastZenPing(org.elasticsearch.discovery.zen.UnicastZenPing) ZenDiscovery(org.elasticsearch.discovery.zen.ZenDiscovery) TestZenDiscovery(org.elasticsearch.test.discovery.TestZenDiscovery) ZenPing(org.elasticsearch.discovery.zen.ZenPing) UnicastZenPing(org.elasticsearch.discovery.zen.UnicastZenPing) TestZenDiscovery(org.elasticsearch.test.discovery.TestZenDiscovery)

Example 3 with ZenPing

use of org.elasticsearch.discovery.zen.ZenPing in project elasticsearch by elastic.

the class DiscoveryWithServiceDisruptionsIT method testIsolatedUnicastNodes.

public void testIsolatedUnicastNodes() throws Exception {
    List<String> nodes = startCluster(4, -1, new int[] { 0 });
    // Figure out what is the elected master node
    final String unicastTarget = nodes.get(0);
    Set<String> unicastTargetSide = new HashSet<>();
    unicastTargetSide.add(unicastTarget);
    Set<String> restOfClusterSide = new HashSet<>();
    restOfClusterSide.addAll(nodes);
    restOfClusterSide.remove(unicastTarget);
    // Forcefully clean temporal response lists on all nodes. Otherwise the node in the unicast host list
    // includes all the other nodes that have pinged it and the issue doesn't manifest
    ZenPing zenPing = ((TestZenDiscovery) internalCluster().getInstance(Discovery.class)).getZenPing();
    if (zenPing instanceof UnicastZenPing) {
        ((UnicastZenPing) zenPing).clearTemporalResponses();
    }
    // Simulate a network issue between the unicast target node and the rest of the cluster
    NetworkDisruption networkDisconnect = new NetworkDisruption(new TwoPartitions(unicastTargetSide, restOfClusterSide), new NetworkDisconnect());
    setDisruptionScheme(networkDisconnect);
    networkDisconnect.startDisrupting();
    // Wait until elected master has removed that the unlucky node...
    ensureStableCluster(3, nodes.get(1));
    // The isolate master node must report no master, so it starts with pinging
    assertNoMaster(unicastTarget);
    networkDisconnect.stopDisrupting();
    // Wait until the master node sees all 3 nodes again.
    ensureStableCluster(4);
}
Also used : UnicastZenPing(org.elasticsearch.discovery.zen.UnicastZenPing) TwoPartitions(org.elasticsearch.test.disruption.NetworkDisruption.TwoPartitions) ZenDiscovery(org.elasticsearch.discovery.zen.ZenDiscovery) TestZenDiscovery(org.elasticsearch.test.discovery.TestZenDiscovery) ZenPing(org.elasticsearch.discovery.zen.ZenPing) UnicastZenPing(org.elasticsearch.discovery.zen.UnicastZenPing) TestZenDiscovery(org.elasticsearch.test.discovery.TestZenDiscovery) NetworkDisruption(org.elasticsearch.test.disruption.NetworkDisruption) NetworkDisconnect(org.elasticsearch.test.disruption.NetworkDisruption.NetworkDisconnect) HashSet(java.util.HashSet)

Aggregations

UnicastZenPing (org.elasticsearch.discovery.zen.UnicastZenPing)3 ZenDiscovery (org.elasticsearch.discovery.zen.ZenDiscovery)3 ZenPing (org.elasticsearch.discovery.zen.ZenPing)3 TestZenDiscovery (org.elasticsearch.test.discovery.TestZenDiscovery)3 NetworkDisruption (org.elasticsearch.test.disruption.NetworkDisruption)2 NetworkDisconnect (org.elasticsearch.test.disruption.NetworkDisruption.NetworkDisconnect)2 TwoPartitions (org.elasticsearch.test.disruption.NetworkDisruption.TwoPartitions)2 ArrayList (java.util.ArrayList)1 HashSet (java.util.HashSet)1