Search in sources :

Example 81 with KeeperException

use of org.apache.zookeeper.KeeperException in project incubator-gobblin by apache.

the class ZookeeperLeaderElection method determineLeadership.

private void determineLeadership() {
    ReentrantReadWriteLock.WriteLock lock = this.readWriteLock.writeLock();
    lock.lock();
    try {
        List<String> children = this.zooKeeper.getChildren().forPath(this.leaderElectionNode);
        Collections.sort(children);
        int idx = children.indexOf(this.nodeId);
        if (idx == 0) {
            Stat stat = this.zooKeeper.checkExists().forPath(this.leaderNode);
            if (stat == null) {
                this.zooKeeper.create().forPath(this.leaderNode, serializeMetadata(this.localMetadata));
            } else {
                this.zooKeeper.setData().forPath(this.leaderNode, serializeMetadata(this.localMetadata));
            }
            this.isLeader = true;
        } else {
            this.isLeader = false;
            String watchedNode = this.leaderElectionNode + "/" + children.get(idx - 1);
            this.zooKeeper.checkExists().usingWatcher(new DetermineLeadershipWatcher()).forPath(watchedNode);
        }
        findLeader();
    } catch (KeeperException exc) {
        reset();
    } catch (Throwable exc) {
        log.error("Fatal failure.", exc);
        this.fatalFailure = true;
    } finally {
        lock.unlock();
    }
}
Also used : Stat(org.apache.zookeeper.data.Stat) ReentrantReadWriteLock(java.util.concurrent.locks.ReentrantReadWriteLock) KeeperException(org.apache.zookeeper.KeeperException)

Example 82 with KeeperException

use of org.apache.zookeeper.KeeperException in project herddb by diennea.

the class ZookeeperClientSideMetadataProvider method getServerHostData.

@Override
public ServerHostData getServerHostData(String nodeId) throws ClientSideMetadataProviderException {
    ServerHostData cached = servers.get(nodeId);
    if (cached != null) {
        return cached;
    }
    ZooKeeper zooKeeper = getZooKeeper();
    try {
        for (int i = 0; i < MAX_TRIALS; i++) {
            try {
                Stat stat = new Stat();
                byte[] node = zooKeeper.getData(basePath + "/nodes/" + nodeId, null, stat);
                NodeMetadata nodeMetadata = NodeMetadata.deserialize(node, stat.getVersion());
                ServerHostData result = new ServerHostData(nodeMetadata.host, nodeMetadata.port, "?", nodeMetadata.ssl, new HashMap<>());
                servers.put(nodeId, result);
                return result;
            } catch (KeeperException.NoNodeException ex) {
                return null;
            } catch (KeeperException.ConnectionLossException ex) {
                LOG.log(Level.SEVERE, "tmp error getServerHostData for " + nodeId + ": " + ex);
                try {
                    Thread.sleep(i * 500 + 1000);
                } catch (InterruptedException exit) {
                    throw new ClientSideMetadataProviderException(exit);
                }
            } catch (KeeperException | InterruptedException | IOException ex) {
                throw new ClientSideMetadataProviderException(ex);
            } finally {
                if (ownZooKeeper) {
                    try {
                        zooKeeper.close();
                    } catch (InterruptedException ex) {
                        throw new ClientSideMetadataProviderException(ex);
                    }
                }
            }
        }
    } finally {
        if (ownZooKeeper) {
            try {
                zooKeeper.close();
            } catch (InterruptedException ex) {
                throw new ClientSideMetadataProviderException(ex);
            }
        }
    }
    throw new ClientSideMetadataProviderException("Could not find a server info for node " + nodeId + " in time");
}
Also used : IOException(java.io.IOException) ServerHostData(herddb.network.ServerHostData) NodeMetadata(herddb.model.NodeMetadata) ZooKeeper(org.apache.zookeeper.ZooKeeper) Stat(org.apache.zookeeper.data.Stat) KeeperException(org.apache.zookeeper.KeeperException)

Example 83 with KeeperException

use of org.apache.zookeeper.KeeperException in project herddb by diennea.

the class ZookeeperMetadataStorageManager method listNodes.

@Override
public List<NodeMetadata> listNodes() throws MetadataStorageManagerException {
    try {
        List<String> children = ensureZooKeeper().getChildren(nodesPath, mainWatcher, null);
        LOGGER.severe("listNodes: for " + nodesPath + ": " + children);
        List<NodeMetadata> result = new ArrayList<>();
        for (String child : children) {
            NodeMetadata nodeMetadata = getNode(child);
            result.add(nodeMetadata);
        }
        return result;
    } catch (IOException | InterruptedException | KeeperException err) {
        handleSessionExpiredError(err);
        throw new MetadataStorageManagerException(err);
    }
}
Also used : NodeMetadata(herddb.model.NodeMetadata) MetadataStorageManagerException(herddb.metadata.MetadataStorageManagerException) ArrayList(java.util.ArrayList) IOException(java.io.IOException) KeeperException(org.apache.zookeeper.KeeperException)

Example 84 with KeeperException

use of org.apache.zookeeper.KeeperException in project herddb by diennea.

the class ZookeeperMetadataStorageManager method updateTableSpaceReplicaState.

@Override
public void updateTableSpaceReplicaState(TableSpaceReplicaState state) throws MetadataStorageManagerException {
    try {
        String tableSpacePath = tableSpacesReplicasPath + "/" + state.uuid;
        byte[] data = state.serialize();
        try {
            ensureZooKeeper().setData(tableSpacePath + "/" + state.nodeId, data, -1);
        } catch (KeeperException.NoNodeException notExists) {
            try {
                ensureZooKeeper().create(tableSpacePath, data, ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
            } catch (KeeperException.NodeExistsException existsRoot) {
            }
            ensureZooKeeper().create(tableSpacePath + "/" + state.nodeId, data, ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
        }
    } catch (InterruptedException | KeeperException | IOException err) {
        handleSessionExpiredError(err);
        throw new MetadataStorageManagerException(err);
    }
}
Also used : MetadataStorageManagerException(herddb.metadata.MetadataStorageManagerException) IOException(java.io.IOException) KeeperException(org.apache.zookeeper.KeeperException)

Example 85 with KeeperException

use of org.apache.zookeeper.KeeperException in project SilverKing by Morgan-Stanley.

the class DependencyWatcher method build.

private void build(Map<String, Long> curBuild) {
    try {
        boolean buildOK;
        long ringConfigVersion;
        Log.warning("New build triggered");
        buildOK = false;
        ringConfigVersion = -1;
        try {
            if (TopoRingConstants.verbose) {
                System.out.println("Building tree");
            }
            RingTreeRecipe recipe;
            Topology topology;
            WeightSpecifications weightSpecs;
            ExclusionSet exclusionSet;
            ExclusionSet instanceExclusionSet;
            StoragePolicyGroup storagePolicyGroup;
            long topologyVersion;
            long weightsVersion;
            long exclusionVersion;
            long instanceExclusionVersion;
            long storagePolicyGroupVersion;
            RingConfiguration ringConfig;
            ZooKeeperExtended zk;
            HostGroupTable hostGroupTable;
            long hostGroupTableVersion;
            ExclusionSet mergedExclusionSet;
            zk = mc.getZooKeeper();
            topologyVersion = curBuild.get(mp.getTopologyPath());
            weightsVersion = curBuild.get(mp.getWeightsPath());
            exclusionVersion = curBuild.get(mp.getExclusionsPath());
            instanceExclusionVersion = curBuild.get(dhtMC.getMetaPaths().getInstanceExclusionsPath());
            storagePolicyGroupVersion = curBuild.get(mp.getStoragePolicyGroupPath());
            ringConfigVersion = curBuild.get(mp.getConfigPath());
            topology = new TopologyZK(cloudMC).readFromZK(topologyVersion, null);
            weightSpecs = new WeightsZK(mc).readFromZK(weightsVersion, null);
            exclusionSet = new ExclusionSet(new ServerSetExtensionZK(mc, mc.getMetaPaths().getExclusionsPath()).readFromZK(exclusionVersion, null));
            if (options.ignoreInstanceExclusions) {
                instanceExclusionSet = ExclusionSet.emptyExclusionSet(0);
            } else {
                try {
                    instanceExclusionSet = new ExclusionSet(new ServerSetExtensionZK(mc, dhtMC.getMetaPaths().getInstanceExclusionsPath()).readFromZK(instanceExclusionVersion, null));
                } catch (Exception e) {
                    Log.warning("No instance ExclusionSet found");
                    instanceExclusionSet = ExclusionSet.emptyExclusionSet(0);
                }
            }
            mergedExclusionSet = ExclusionSet.union(exclusionSet, instanceExclusionSet);
            storagePolicyGroup = new StoragePolicyGroupZK(mc).readFromZK(storagePolicyGroupVersion, null);
            ringConfig = new RingConfigurationZK(mc).readFromZK(ringConfigVersion, null);
            Log.warningf("ringConfiguration %s", ringConfig);
            hostGroupTableVersion = zk.getLatestVersion(cloudMC.getMetaPaths().getHostGroupPath());
            hostGroupTable = new HostGroupTableZK(cloudMC).readFromZK(hostGroupTableVersion, null);
            try {
                recipe = new RingTreeRecipe(topology, ringConfig.getRingParentName(), weightSpecs, mergedExclusionSet, storagePolicyGroup, ringConfig.getStoragePolicyName(), hostGroupTable, ringConfig.getHostGroups(), ringConfigVersion, DHTUtil.currentTimeMillis());
                Log.warning("Recipe.ringParent: " + recipe.ringParent);
            } catch (RuntimeException re) {
                re.printStackTrace(System.out);
                Log.warning("ringConfig: ", ringConfig + " " + re);
                Log.logErrorWarning(re);
                throw re;
            }
            RingTree ringTree;
            RingTree prevRingTree;
            long configInstanceVersion;
            String newInstancePath;
            configInstanceVersion = mc.getLatestConfigInstanceVersion(ringConfigVersion);
            if (configInstanceVersion >= 0 && !ignoreSource) {
                prevRingTree = SingleRingZK.readTree(mc, ringConfigVersion, configInstanceVersion);
            } else {
                prevRingTree = null;
            }
            if (prevRingTree == null || ignoreFeasibility || RingTreeBuilder.convergenceFeasible(prevRingTree, storagePolicyGroup, ringConfig.getStoragePolicyName(), ringConfig.getRingParentName(), exclusionSet)) {
                ringTree = RingTreeBuilder.create(recipe, prevRingTree);
                // ringTree = RingTreeBuilder.create(recipe, null); // for testing without movement reduction
                newInstancePath = mc.createConfigInstancePath(ringConfigVersion);
                SingleRingZK.writeTree(mc, topologyVersion, newInstancePath, ringTree);
                if (TopoRingConstants.verbose) {
                    System.out.println(ringTree);
                    System.out.println(ringConfigVersion);
                    System.out.println(configInstanceVersion);
                    System.out.println(topologyVersion);
                    System.out.println(newInstancePath);
                    System.out.println("Building complete");
                }
                buildOK = true;
            } else {
                Log.warning("Convergence is infeasible. A region in prevTree does not have a viable server.");
            }
        } catch (IOException ioe) {
            Log.logErrorWarning(ioe);
        } catch (KeeperException ke) {
            Log.logErrorWarning(ke);
        }
        if (exitAfterBuild) {
            if (buildOK) {
                try {
                    if (ringConfigVersion < 0) {
                        throw new RuntimeException("ringConfigVersion < 0");
                    }
                    setRing(ringConfigVersion);
                } catch (KeeperException ke) {
                    Log.logErrorWarning(ke);
                    buildOK = false;
                }
            }
            System.exit(buildOK ? 0 : -1);
        }
    } catch (RuntimeException re) {
        re.printStackTrace();
    } finally {
        Log.warning("Leaving build");
    }
}
Also used : RingConfigurationZK(com.ms.silverking.cloud.toporing.meta.RingConfigurationZK) HostGroupTable(com.ms.silverking.cloud.config.HostGroupTable) ZooKeeperExtended(com.ms.silverking.cloud.zookeeper.ZooKeeperExtended) Topology(com.ms.silverking.cloud.topology.Topology) IOException(java.io.IOException) StoragePolicyGroupZK(com.ms.silverking.cloud.toporing.meta.StoragePolicyGroupZK) StoragePolicyGroup(com.ms.silverking.cloud.storagepolicy.StoragePolicyGroup) ServerSetExtensionZK(com.ms.silverking.cloud.meta.ServerSetExtensionZK) KeeperException(org.apache.zookeeper.KeeperException) IOException(java.io.IOException) CmdLineException(org.kohsuke.args4j.CmdLineException) WeightSpecifications(com.ms.silverking.cloud.toporing.meta.WeightSpecifications) TopologyZK(com.ms.silverking.cloud.topology.TopologyZK) RingConfiguration(com.ms.silverking.cloud.toporing.meta.RingConfiguration) NamedRingConfiguration(com.ms.silverking.cloud.toporing.meta.NamedRingConfiguration) WeightsZK(com.ms.silverking.cloud.toporing.meta.WeightsZK) ExclusionSet(com.ms.silverking.cloud.meta.ExclusionSet) HostGroupTableZK(com.ms.silverking.cloud.meta.HostGroupTableZK) KeeperException(org.apache.zookeeper.KeeperException)

Aggregations

KeeperException (org.apache.zookeeper.KeeperException)566 IOException (java.io.IOException)188 Stat (org.apache.zookeeper.data.Stat)127 ZooKeeper (org.apache.zookeeper.ZooKeeper)87 ArrayList (java.util.ArrayList)51 NoNodeException (org.apache.zookeeper.KeeperException.NoNodeException)45 Watcher (org.apache.zookeeper.Watcher)39 WatchedEvent (org.apache.zookeeper.WatchedEvent)38 Test (org.junit.jupiter.api.Test)38 CountDownLatch (java.util.concurrent.CountDownLatch)30 SolrException (org.apache.solr.common.SolrException)30 HashMap (java.util.HashMap)29 List (java.util.List)28 ACL (org.apache.zookeeper.data.ACL)27 Test (org.junit.Test)27 HeliosRuntimeException (com.spotify.helios.common.HeliosRuntimeException)25 ServerName (org.apache.hadoop.hbase.ServerName)24 Map (java.util.Map)23 IZooReaderWriter (org.apache.accumulo.fate.zookeeper.IZooReaderWriter)23 InterruptedIOException (java.io.InterruptedIOException)20