Search in sources :

Example 31 with IStormClusterState

use of org.apache.storm.cluster.IStormClusterState in project storm by apache.

the class Nimbus method rmDependencyJarsInTopology.

@VisibleForTesting
public void rmDependencyJarsInTopology(String topoId) {
    try {
        BlobStore store = blobStore;
        IStormClusterState state = stormClusterState;
        StormTopology topo = readStormTopologyAsNimbus(topoId, store);
        List<String> dependencyJars = topo.get_dependency_jars();
        LOG.info("Removing dependency jars from blobs - {}", dependencyJars);
        if (dependencyJars != null && !dependencyJars.isEmpty()) {
            for (String key : dependencyJars) {
                rmBlobKey(store, key, state);
            }
        }
    } catch (Exception e) {
        //Yes eat the exception
        LOG.info("Exception {}", e);
    }
}
Also used : StormTopology(org.apache.storm.generated.StormTopology) IStormClusterState(org.apache.storm.cluster.IStormClusterState) BlobStore(org.apache.storm.blobstore.BlobStore) LocalFsBlobStore(org.apache.storm.blobstore.LocalFsBlobStore) AuthorizationException(org.apache.storm.generated.AuthorizationException) NotAliveException(org.apache.storm.generated.NotAliveException) InterruptedIOException(java.io.InterruptedIOException) TException(org.apache.thrift.TException) IOException(java.io.IOException) AlreadyAliveException(org.apache.storm.generated.AlreadyAliveException) KeyAlreadyExistsException(org.apache.storm.generated.KeyAlreadyExistsException) KeyNotFoundException(org.apache.storm.generated.KeyNotFoundException) InvalidTopologyException(org.apache.storm.generated.InvalidTopologyException) BindException(java.net.BindException) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 32 with IStormClusterState

use of org.apache.storm.cluster.IStormClusterState in project storm by apache.

the class Nimbus method renewCredentials.

private void renewCredentials() throws Exception {
    if (!isLeader()) {
        LOG.info("not a leader, skipping credential renewal.");
        return;
    }
    IStormClusterState state = stormClusterState;
    BlobStore store = blobStore;
    Collection<ICredentialsRenewer> renewers = credRenewers;
    Object lock = credUpdateLock;
    List<String> assignedIds = state.activeStorms();
    if (assignedIds != null) {
        for (String id : assignedIds) {
            Map<String, Object> topoConf = Collections.unmodifiableMap(tryReadTopoConf(id, store));
            synchronized (lock) {
                Credentials origCreds = state.credentials(id, null);
                if (origCreds != null) {
                    Map<String, String> orig = origCreds.get_creds();
                    Map<String, String> newCreds = new HashMap<>(orig);
                    for (ICredentialsRenewer renewer : renewers) {
                        LOG.info("Renewing Creds For {} with {}", id, renewer);
                        renewer.renew(newCreds, topoConf);
                    }
                    if (!newCreds.equals(origCreds)) {
                        state.setCredentials(id, new Credentials(newCreds), topoConf);
                    }
                }
            }
        }
    }
}
Also used : ICredentialsRenewer(org.apache.storm.security.auth.ICredentialsRenewer) HashMap(java.util.HashMap) IStormClusterState(org.apache.storm.cluster.IStormClusterState) BlobStore(org.apache.storm.blobstore.BlobStore) LocalFsBlobStore(org.apache.storm.blobstore.LocalFsBlobStore) Credentials(org.apache.storm.generated.Credentials)

Example 33 with IStormClusterState

use of org.apache.storm.cluster.IStormClusterState in project storm by apache.

the class Nimbus method updateHeartbeats.

private void updateHeartbeats(String topoId, Set<List<Integer>> allExecutors, Assignment existingAssignment) {
    LOG.debug("Updating heartbeats for {} {}", topoId, allExecutors);
    IStormClusterState state = stormClusterState;
    Map<List<Integer>, Map<String, Object>> executorBeats = StatsUtil.convertExecutorBeats(state.executorBeats(topoId, existingAssignment.get_executor_node_port()));
    Map<List<Integer>, Map<String, Object>> cache = StatsUtil.updateHeartbeatCache(heartbeatsCache.get().get(topoId), executorBeats, allExecutors, Utils.getInt(conf.get(Config.NIMBUS_TASK_TIMEOUT_SECS)));
    heartbeatsCache.getAndUpdate(new Assoc<>(topoId, cache));
}
Also used : ArrayList(java.util.ArrayList) List(java.util.List) IStormClusterState(org.apache.storm.cluster.IStormClusterState) Map(java.util.Map) TimeCacheMap(org.apache.storm.utils.TimeCacheMap) ImmutableMap(com.google.common.collect.ImmutableMap) HashMap(java.util.HashMap)

Aggregations

IStormClusterState (org.apache.storm.cluster.IStormClusterState)33 KeyNotFoundException (org.apache.storm.generated.KeyNotFoundException)17 IOException (java.io.IOException)16 NotAliveException (org.apache.storm.generated.NotAliveException)15 TException (org.apache.thrift.TException)15 InterruptedIOException (java.io.InterruptedIOException)14 BindException (java.net.BindException)14 AlreadyAliveException (org.apache.storm.generated.AlreadyAliveException)14 AuthorizationException (org.apache.storm.generated.AuthorizationException)14 InvalidTopologyException (org.apache.storm.generated.InvalidTopologyException)14 KeyAlreadyExistsException (org.apache.storm.generated.KeyAlreadyExistsException)14 HashMap (java.util.HashMap)13 ArrayList (java.util.ArrayList)11 BlobStore (org.apache.storm.blobstore.BlobStore)11 LocalFsBlobStore (org.apache.storm.blobstore.LocalFsBlobStore)11 List (java.util.List)9 Map (java.util.Map)9 ImmutableMap (com.google.common.collect.ImmutableMap)7 HashSet (java.util.HashSet)7 TimeCacheMap (org.apache.storm.utils.TimeCacheMap)7