Search in sources :

Example 21 with ConfiguredNode

use of com.yahoo.vdslib.distribution.ConfiguredNode in project vespa by vespa-engine.

the class StateChangeTest method testMaintenanceWithoutInitIfRetired.

@Test
public void testMaintenanceWithoutInitIfRetired() throws Exception {
    List<ConfiguredNode> nodes = new ArrayList<>();
    for (int i = 0; i < 10; i++) {
        boolean retired = (i == 6);
        nodes.add(new ConfiguredNode(i, retired));
    }
    FleetControllerOptions options = new FleetControllerOptions("mycluster", nodes);
    options.maxSlobrokDisconnectGracePeriod = 60 * 1000;
    initialize(options);
    communicator.setNodeState(new Node(NodeType.STORAGE, 6), State.DOWN, "Connection error: Closed at other end");
    ctrl.tick();
    assertEquals("version:4 distributor:10 storage:10 .6.s:m", ctrl.getSystemState().toString());
    NodeState ns = ctrl.getReportedNodeState(new Node(NodeType.STORAGE, 6));
    assertTrue(ns.toString(), ns.getDescription().indexOf("Connection error: Closed at other end") != -1);
    tick(1000);
    communicator.setNodeState(new Node(NodeType.STORAGE, 6), new NodeState(NodeType.STORAGE, State.INITIALIZING).setInitProgress(0.0), "");
    ctrl.tick();
    // Still maintenance since .i progress 0.0 is really down.
    assertEquals("version:4 distributor:10 storage:10 .6.s:m", ctrl.getSystemState().toString());
    communicator.setNodeState(new Node(NodeType.STORAGE, 6), new NodeState(NodeType.STORAGE, State.INITIALIZING).setInitProgress(0.6), "");
    ctrl.tick();
    // Still maintenance since configured.
    assertEquals("version:4 distributor:10 storage:10 .6.s:m", ctrl.getSystemState().toString());
    tick(1000);
    communicator.setNodeState(new Node(NodeType.STORAGE, 6), new NodeState(NodeType.STORAGE, State.UP), "");
    ctrl.tick();
    assertEquals("version:5 distributor:10 storage:10 .6.s:r", ctrl.getSystemState().toString());
    assert (!ctrl.getReportedNodeState(new Node(NodeType.STORAGE, 6)).hasDescription());
    verifyNodeEvents(new Node(NodeType.STORAGE, 6), "Event: storage.6: Now reporting state U\n" + "Event: storage.6: Altered node state in cluster state from 'D: Node not seen in slobrok.' to 'R'\n" + "Event: storage.6: Failed to get node state: D: Connection error: Closed at other end\n" + "Event: storage.6: Stopped or possibly crashed after 0 ms, which is before stable state time period. Premature crash count is now 1.\n" + "Event: storage.6: Altered node state in cluster state from 'R' to 'M: Connection error: Closed at other end'\n" + "Event: storage.6: Now reporting state I, i 0.00 (ls)\n" + "Event: storage.6: Now reporting state I, i 0.600 (read)\n" + "Event: storage.6: Now reporting state U\n" + "Event: storage.6: Altered node state in cluster state from 'M: Connection error: Closed at other end' to 'R'\n");
}
Also used : ConfiguredNode(com.yahoo.vdslib.distribution.ConfiguredNode) ConfiguredNode(com.yahoo.vdslib.distribution.ConfiguredNode) Test(org.junit.Test)

Example 22 with ConfiguredNode

use of com.yahoo.vdslib.distribution.ConfiguredNode in project vespa by vespa-engine.

the class StateChangeTest method testMaintenanceToDownIfPastTransitionTimeAndRetired.

@Test
public void testMaintenanceToDownIfPastTransitionTimeAndRetired() throws Exception {
    List<ConfiguredNode> nodes = new ArrayList<>();
    for (int i = 0; i < 10; i++) {
        boolean retired = (i == 6);
        nodes.add(new ConfiguredNode(i, retired));
    }
    FleetControllerOptions options = new FleetControllerOptions("mycluster", nodes);
    options.maxSlobrokDisconnectGracePeriod = 60 * 1000;
    initialize(options);
    communicator.setNodeState(new Node(NodeType.STORAGE, 6), State.DOWN, "Connection error: Closed at other end");
    ctrl.tick();
    assertEquals("version:4 distributor:10 storage:10 .6.s:m", ctrl.getSystemState().toString());
    timer.advanceTime(100000);
    ctrl.tick();
    assertEquals("version:5 distributor:10 storage:10 .6.s:d", ctrl.getSystemState().toString());
}
Also used : ConfiguredNode(com.yahoo.vdslib.distribution.ConfiguredNode) ConfiguredNode(com.yahoo.vdslib.distribution.ConfiguredNode) Test(org.junit.Test)

Example 23 with ConfiguredNode

use of com.yahoo.vdslib.distribution.ConfiguredNode in project vespa by vespa-engine.

the class StateChangeTest method testSetAllTimestampsAfterDowntime.

@Test
public void testSetAllTimestampsAfterDowntime() throws Exception {
    startingTest("StateChangeTest::testSetAllTimestampsAfterDowntime");
    FleetControllerOptions options = new FleetControllerOptions("mycluster", createNodes(10));
    setUpFleetController(true, options);
    setUpVdsNodes(true, new DummyVdsNodeOptions());
    waitForStableSystem();
    StateWaiter waiter = new StateWaiter(timer);
    fleetController.addSystemStateListener(waiter);
    // Simulate netsplit. Take node down without node booting
    assertEquals(true, nodes.get(0).isDistributor());
    nodes.get(0).disconnectImmediately();
    waiter.waitForState("version:\\d+ distributor:10 .0.s:d storage:10", timeoutMS);
    // Add node back.
    nodes.get(0).connect();
    waitForStableSystem();
    // At this time, node taken down should have cluster states with all starting timestamps set. Others node should not.
    for (DummyVdsNode node : nodes) {
        node.waitForSystemStateVersion(waiter.getCurrentSystemState().getVersion(), timeoutMS);
        List<ClusterState> states = node.getSystemStatesReceived();
        ClusterState lastState = states.get(0);
        StringBuilder stateHistory = new StringBuilder();
        for (ClusterState state : states) {
            stateHistory.append(state.toString()).append("\n");
        }
        if (node.getNode().equals(new Node(NodeType.DISTRIBUTOR, 0))) {
            for (ConfiguredNode i : options.nodes) {
                Node nodeId = new Node(NodeType.STORAGE, i.index());
                long ts = lastState.getNodeState(nodeId).getStartTimestamp();
                assertTrue(nodeId + "\n" + stateHistory + "\nWas " + ts + " should be " + fleetController.getCluster().getNodeInfo(nodeId).getStartTimestamp(), ts > 0);
            }
        } else {
            for (ConfiguredNode i : options.nodes) {
                Node nodeId = new Node(NodeType.STORAGE, i.index());
                assertTrue(nodeId.toString(), lastState.getNodeState(nodeId).getStartTimestamp() == 0);
            }
        }
        for (ConfiguredNode i : options.nodes) {
            Node nodeId = new Node(NodeType.DISTRIBUTOR, i.index());
            assertTrue(nodeId.toString(), lastState.getNodeState(nodeId).getStartTimestamp() == 0);
        }
    }
}
Also used : StateWaiter(com.yahoo.vespa.clustercontroller.core.testutils.StateWaiter) ConfiguredNode(com.yahoo.vdslib.distribution.ConfiguredNode) ConfiguredNode(com.yahoo.vdslib.distribution.ConfiguredNode) Test(org.junit.Test)

Example 24 with ConfiguredNode

use of com.yahoo.vdslib.distribution.ConfiguredNode in project vespa by vespa-engine.

the class SetNodeStatesForClusterRequest method calculateResult.

@Override
public SetResponse calculateResult(RemoteClusterControllerTask.Context context) throws StateRestApiException {
    if (condition != SetUnitStateRequest.Condition.FORCE) {
        // go down, etc. This is prohibited in Condition.SAFE.
        throw new InvalidContentException("Setting all nodes in a cluster to a state is only supported with FORCE");
    }
    for (ConfiguredNode configuredNode : context.cluster.getConfiguredNodes().values()) {
        Node node = new Node(NodeType.STORAGE, configuredNode.index());
        SetResponse setResponse = SetNodeStateRequest.setWantedState(context.cluster, condition, newStates, node, context.nodeStateOrHostInfoChangeHandler, context.currentConsolidatedState);
        if (!setResponse.getWasModified()) {
            throw new InternalFailure("We have not yet implemented the meaning of " + "failing to set the wanted state for a subset of nodes: " + "condition = " + condition + ", newStates = " + newStates + ", currentConsolidatedState = " + context.currentConsolidatedState);
        }
    }
    // 'true' here means the current state now equals the request's wanted state.
    return new SetResponse("ok", true);
}
Also used : SetResponse(com.yahoo.vespa.clustercontroller.utils.staterestapi.response.SetResponse) InvalidContentException(com.yahoo.vespa.clustercontroller.utils.staterestapi.errors.InvalidContentException) InternalFailure(com.yahoo.vespa.clustercontroller.utils.staterestapi.errors.InternalFailure) ConfiguredNode(com.yahoo.vdslib.distribution.ConfiguredNode) ConfiguredNode(com.yahoo.vdslib.distribution.ConfiguredNode)

Example 25 with ConfiguredNode

use of com.yahoo.vdslib.distribution.ConfiguredNode in project vespa by vespa-engine.

the class ClusterFixture method markNodeAsConfigRetired.

public ClusterFixture markNodeAsConfigRetired(int nodeIndex) {
    Set<ConfiguredNode> configuredNodes = new HashSet<>(cluster.getConfiguredNodes().values());
    configuredNodes.remove(new ConfiguredNode(nodeIndex, false));
    configuredNodes.add(new ConfiguredNode(nodeIndex, true));
    cluster.setNodes(configuredNodes);
    return this;
}
Also used : ConfiguredNode(com.yahoo.vdslib.distribution.ConfiguredNode) HashSet(java.util.HashSet)

Aggregations

ConfiguredNode (com.yahoo.vdslib.distribution.ConfiguredNode)26 Test (org.junit.Test)8 NodeState (com.yahoo.vdslib.state.NodeState)7 Distribution (com.yahoo.vdslib.distribution.Distribution)6 HashSet (java.util.HashSet)5 Node (com.yahoo.vdslib.state.Node)3 ArrayList (java.util.ArrayList)3 Group (com.yahoo.vdslib.distribution.Group)2 ClusterState (com.yahoo.vdslib.state.ClusterState)2 NodeType (com.yahoo.vdslib.state.NodeType)2 StorDistributionConfig (com.yahoo.vespa.config.content.StorDistributionConfig)2 Int32Value (com.yahoo.jrt.Int32Value)1 Request (com.yahoo.jrt.Request)1 Spec (com.yahoo.jrt.Spec)1 StringValue (com.yahoo.jrt.StringValue)1 Supervisor (com.yahoo.jrt.Supervisor)1 Target (com.yahoo.jrt.Target)1 Transport (com.yahoo.jrt.Transport)1 DatabaseHandler (com.yahoo.vespa.clustercontroller.core.database.DatabaseHandler)1 VdsClusterHtmlRendrer (com.yahoo.vespa.clustercontroller.core.status.statuspage.VdsClusterHtmlRendrer)1