Search in sources :

Example 11 with ExclusionSet

use of com.ms.silverking.cloud.meta.ExclusionSet in project SilverKing by Morgan-Stanley.

the class DHTRingMaster method readExclusions.

private ExclusionSet readExclusions(String ringName, long ringVersion) throws KeeperException, IOException {
    ExclusionZK exclusionZK;
    ExclusionSet instanceExclusionSet;
    ExclusionSet exclusionSet;
    com.ms.silverking.cloud.meta.MetaClient cloudMC;
    com.ms.silverking.cloud.toporing.meta.MetaClient ringMC;
    ringMC = com.ms.silverking.cloud.toporing.meta.MetaClient.createMetaClient(ringName, ringVersion, zkConfig);
    cloudMC = ringMC.createCloudMC();
    exclusionZK = new ExclusionZK(cloudMC);
    try {
        exclusionSet = exclusionZK.readLatestFromZK();
    } catch (Exception e) {
        Log.logErrorWarning(e);
        Log.warningAsync("No ExclusionSet found. Using empty set.");
        exclusionSet = ExclusionSet.emptyExclusionSet(0);
    }
    try {
        instanceExclusionSet = new ExclusionSet(new ServerSetExtensionZK(mc, mc.getMetaPaths().getInstanceExclusionsPath()).readLatestFromZK());
    } catch (Exception e) {
        Log.logErrorWarning(e);
        Log.warningAsync("No instance ExclusionSet found. Using empty set.");
        instanceExclusionSet = ExclusionSet.emptyExclusionSet(0);
    }
    return ExclusionSet.union(exclusionSet, instanceExclusionSet);
}
Also used : ExclusionZK(com.ms.silverking.cloud.meta.ExclusionZK) ExclusionSet(com.ms.silverking.cloud.meta.ExclusionSet) CmdLineException(org.kohsuke.args4j.CmdLineException) KeeperException(org.apache.zookeeper.KeeperException) IOException(java.io.IOException) AlreadyBoundException(java.rmi.AlreadyBoundException) ServerSetExtensionZK(com.ms.silverking.cloud.meta.ServerSetExtensionZK)

Example 12 with ExclusionSet

use of com.ms.silverking.cloud.meta.ExclusionSet in project SilverKing by Morgan-Stanley.

the class NodeRingMaster2 method getAllCurrentAndTargetNonExcludedNonLocalReplicaServers.

public Set<IPAndPort> getAllCurrentAndTargetNonExcludedNonLocalReplicaServers() {
    Set<IPAndPort> allReplicas;
    ExclusionSet curExclusionSet;
    Set<IPAndPort> nonExcludedReplicas;
    allReplicas = getAllCurrentAndTargetReplicaServers();
    curExclusionSet = latestExclusionSet;
    nonExcludedReplicas = new HashSet<>(curExclusionSet.filterByIP(allReplicas));
    nonExcludedReplicas.remove(nodeID);
    return ImmutableSet.copyOf(nonExcludedReplicas);
}
Also used : IPAndPort(com.ms.silverking.net.IPAndPort) ExclusionSet(com.ms.silverking.cloud.meta.ExclusionSet)

Example 13 with ExclusionSet

use of com.ms.silverking.cloud.meta.ExclusionSet in project SilverKing by Morgan-Stanley.

the class RingMapState method readInitialExclusions.

/*
    private void waitForExclusionSet() {
    	Log.warningf("waitForExclusionSet: %s", ringIDAndVersionPair);
    	exclusionSetLock.lock();
    	try {
    		while (!exclusionSetInitialized) {
    			try {
					exclusionSetCV.await();
				} catch (InterruptedException e) {
				}
    		}
    	} finally {
        	exclusionSetLock.unlock();
    	}
    	Log.warningf("out waitForExclusionSet: %s", ringIDAndVersionPair);
    }
    */
private void readInitialExclusions(MetaClient mc) throws KeeperException {
    ExclusionZK exclusionZK;
    ExclusionSet instanceExclusionSet;
    ExclusionSet exclusionSet;
    RingTree newRingTree;
    ResolvedReplicaMap newResolvedReplicaMap;
    Log.warning("RingMapState reading initial exclusions");
    exclusionZK = new ExclusionZK(mc);
    try {
        exclusionSet = exclusionZK.readLatestFromZK();
    } catch (Exception e) {
        Log.logErrorWarning(e);
        Log.warning("No ExclusionSet found. Using empty set.");
        exclusionSet = ExclusionSet.emptyExclusionSet(0);
    }
    curExclusionSet = exclusionSet;
    Log.warning("curExclusionSet initialized:\n", curExclusionSet);
    try {
        instanceExclusionSet = new ExclusionSet(new ServerSetExtensionZK(dhtMC, dhtMC.getMetaPaths().getInstanceExclusionsPath()).readLatestFromZK());
    } catch (Exception e) {
        Log.logErrorWarning(e);
        Log.warning("No instance ExclusionSet found. Using empty set.");
        instanceExclusionSet = ExclusionSet.emptyExclusionSet(0);
    }
    curInstanceExclusionSet = instanceExclusionSet;
    Log.warning("curInstanceExclusionSet initialized:\n", curInstanceExclusionSet);
    try {
        newRingTree = RingTreeBuilder.removeExcludedNodes(rawRingTree, ExclusionSet.union(curExclusionSet, curInstanceExclusionSet));
    } catch (Exception e) {
        Log.logErrorWarning(e);
        throw new RuntimeException(e);
    }
    newResolvedReplicaMap = newRingTree.getResolvedMap(ringConfig.getRingParentName(), new ReplicaNaiveIPPrioritizer());
    ringTreeMinusExclusions = newRingTree;
    resolvedReplicaMapMinusExclusions = newResolvedReplicaMap;
    System.out.println("\tResolved Map");
    resolvedReplicaMapMinusExclusions.display();
    Log.warning("RingMapState done reading initial exclusions");
}
Also used : ExclusionZK(com.ms.silverking.cloud.meta.ExclusionZK) ExclusionSet(com.ms.silverking.cloud.meta.ExclusionSet) RingTree(com.ms.silverking.cloud.toporing.RingTree) InstantiatedRingTree(com.ms.silverking.cloud.toporing.InstantiatedRingTree) ResolvedReplicaMap(com.ms.silverking.cloud.toporing.ResolvedReplicaMap) InvalidRingException(com.ms.silverking.cloud.toporing.InvalidRingException) KeeperException(org.apache.zookeeper.KeeperException) IOException(java.io.IOException) InvalidTransitionException(com.ms.silverking.cloud.dht.daemon.storage.convergence.InvalidTransitionException) ServerSetExtensionZK(com.ms.silverking.cloud.meta.ServerSetExtensionZK)

Example 14 with ExclusionSet

use of com.ms.silverking.cloud.meta.ExclusionSet in project SilverKing by Morgan-Stanley.

the class RingMapState2 method readInitialExclusions.

private void readInitialExclusions(MetaClient mc) throws KeeperException {
    ExclusionZK exclusionZK;
    ExclusionSet instanceExclusionSet;
    ExclusionSet exclusionSet;
    RingTree newRingTree;
    ResolvedReplicaMap newResolvedReplicaMap;
    Log.warning("RingMapState reading initial exclusions");
    exclusionZK = new ExclusionZK(mc);
    try {
        exclusionSet = exclusionZK.readLatestFromZK();
    } catch (Exception e) {
        Log.logErrorWarning(e);
        Log.warning("No ExclusionSet found. Using empty set.");
        exclusionSet = ExclusionSet.emptyExclusionSet(0);
    }
    curExclusionSet = exclusionSet;
    Log.warning("curExclusionSet initialized:\n", curExclusionSet);
    try {
        instanceExclusionSet = new ExclusionSet(new ServerSetExtensionZK(dhtMC, dhtMC.getMetaPaths().getInstanceExclusionsPath()).readLatestFromZK());
    } catch (Exception e) {
        Log.logErrorWarning(e);
        Log.warning("No instance ExclusionSet found. Using empty set.");
        instanceExclusionSet = ExclusionSet.emptyExclusionSet(0);
    }
    curInstanceExclusionSet = instanceExclusionSet;
    Log.warning("curInstanceExclusionSet initialized:\n", curInstanceExclusionSet);
    curUnionExclusionSet = ExclusionSet.union(curExclusionSet, curInstanceExclusionSet);
    Log.warningf("curUnionExclusionSet initialized: %s", curUnionExclusionSet);
    try {
        newRingTree = RingTreeBuilder.removeExcludedNodes(rawRingTree, curUnionExclusionSet);
    } catch (Exception e) {
        Log.logErrorWarning(e);
        throw new RuntimeException(e);
    }
    newResolvedReplicaMap = newRingTree.getResolvedMap(ringConfig.getRingParentName(), new SimpleIPDistancePrioritizer(nodeID, defaultIPDistanceMask));
    ringTreeMinusExclusions = newRingTree;
    resolvedReplicaMapMinusExclusions = newResolvedReplicaMap;
    if (Log.levelMet(Level.INFO)) {
        System.out.println("\tResolved Map");
        resolvedReplicaMapMinusExclusions.display();
    }
    Log.warning("RingMapState done reading initial exclusions");
}
Also used : ExclusionZK(com.ms.silverking.cloud.meta.ExclusionZK) ExclusionSet(com.ms.silverking.cloud.meta.ExclusionSet) RingTree(com.ms.silverking.cloud.toporing.RingTree) InstantiatedRingTree(com.ms.silverking.cloud.toporing.InstantiatedRingTree) ResolvedReplicaMap(com.ms.silverking.cloud.toporing.ResolvedReplicaMap) InvalidRingException(com.ms.silverking.cloud.toporing.InvalidRingException) KeeperException(org.apache.zookeeper.KeeperException) IOException(java.io.IOException) InvalidTransitionException(com.ms.silverking.cloud.dht.daemon.storage.convergence.InvalidTransitionException) ServerSetExtensionZK(com.ms.silverking.cloud.meta.ServerSetExtensionZK)

Aggregations

ExclusionSet (com.ms.silverking.cloud.meta.ExclusionSet)14 IOException (java.io.IOException)8 KeeperException (org.apache.zookeeper.KeeperException)8 ServerSetExtensionZK (com.ms.silverking.cloud.meta.ServerSetExtensionZK)5 ExclusionZK (com.ms.silverking.cloud.meta.ExclusionZK)4 IPAndPort (com.ms.silverking.net.IPAndPort)4 CmdLineException (org.kohsuke.args4j.CmdLineException)4 HostGroupTable (com.ms.silverking.cloud.config.HostGroupTable)3 InstantiatedRingTree (com.ms.silverking.cloud.toporing.InstantiatedRingTree)3 ResolvedReplicaMap (com.ms.silverking.cloud.toporing.ResolvedReplicaMap)3 ImmutableSet (com.google.common.collect.ImmutableSet)2 ReplicaNaiveIPPrioritizer (com.ms.silverking.cloud.dht.daemon.ReplicaNaiveIPPrioritizer)2 InvalidTransitionException (com.ms.silverking.cloud.dht.daemon.storage.convergence.InvalidTransitionException)2 ClassVars (com.ms.silverking.cloud.dht.meta.ClassVars)2 HostGroupTableZK (com.ms.silverking.cloud.meta.HostGroupTableZK)2 StoragePolicyGroup (com.ms.silverking.cloud.storagepolicy.StoragePolicyGroup)2 Topology (com.ms.silverking.cloud.topology.Topology)2 TopologyZK (com.ms.silverking.cloud.topology.TopologyZK)2 InvalidRingException (com.ms.silverking.cloud.toporing.InvalidRingException)2 RingTree (com.ms.silverking.cloud.toporing.RingTree)2