Search in sources :

Example 11 with VisibleForTesting

use of org.apache.storm.shade.com.google.common.annotations.VisibleForTesting in project storm by apache.

the class Nimbus method checkAuthorization.

@VisibleForTesting
public void checkAuthorization(String topoName, Map<String, Object> topoConf, String operation, ReqContext context) throws AuthorizationException {
    IAuthorizer impersonationAuthorizer = impersonationAuthorizationHandler;
    if (context == null) {
        context = ReqContext.context();
    }
    Map<String, Object> checkConf = new HashMap<>();
    if (topoConf != null) {
        checkConf.putAll(topoConf);
    } else if (topoName != null) {
        checkConf.put(Config.TOPOLOGY_NAME, topoName);
    }
    if (context.isImpersonating()) {
        LOG.info("principal: {} is trying to impersonate principal: {}", context.realPrincipal(), context.principal());
        if (impersonationAuthorizer == null) {
            LOG.warn("impersonation attempt but {} has no authorizer configured. potential security risk, " + "please see SECURITY.MD to learn how to configure impersonation authorizer.", DaemonConfig.NIMBUS_IMPERSONATION_AUTHORIZER);
        } else {
            if (!impersonationAuthorizer.permit(context, operation, checkConf)) {
                ThriftAccessLogger.logAccess(context.requestID(), context.remoteAddress(), context.principal(), operation, topoName, "access-denied");
                throw new WrappedAuthorizationException("principal " + context.realPrincipal() + " is not authorized to impersonate principal " + context.principal() + " from host " + context.remoteAddress() + " Please see SECURITY.MD to learn how to configure impersonation acls.");
            }
        }
    }
    IAuthorizer aclHandler = authorizationHandler;
    if (aclHandler != null) {
        if (!aclHandler.permit(context, operation, checkConf)) {
            ThriftAccessLogger.logAccess(context.requestID(), context.remoteAddress(), context.principal(), operation, topoName, "access-denied");
            throw new WrappedAuthorizationException(operation + (topoName != null ? " on topology " + topoName : "") + " is not authorized");
        } else {
            ThriftAccessLogger.logAccess(context.requestID(), context.remoteAddress(), context.principal(), operation, topoName, "access-granted");
        }
    }
}
Also used : WrappedAuthorizationException(org.apache.storm.utils.WrappedAuthorizationException) HashMap(java.util.HashMap) IAuthorizer(org.apache.storm.security.auth.IAuthorizer) VisibleForTesting(org.apache.storm.shade.com.google.common.annotations.VisibleForTesting)

Example 12 with VisibleForTesting

use of org.apache.storm.shade.com.google.common.annotations.VisibleForTesting in project storm by apache.

the class Nimbus method rmDependencyJarsInTopology.

@VisibleForTesting
public void rmDependencyJarsInTopology(String topoId) {
    try {
        BlobStore store = blobStore;
        IStormClusterState state = stormClusterState;
        StormTopology topo = readStormTopologyAsNimbus(topoId, topoCache);
        List<String> dependencyJars = topo.get_dependency_jars();
        LOG.info("Removing dependency jars from blobs - {}", dependencyJars);
        if (dependencyJars != null && !dependencyJars.isEmpty()) {
            for (String key : dependencyJars) {
                rmBlobKey(store, key, state);
            }
        }
    } catch (Exception e) {
        // Yes eat the exception
        LOG.info("Exception {}", e);
    }
}
Also used : StormTopology(org.apache.storm.generated.StormTopology) IStormClusterState(org.apache.storm.cluster.IStormClusterState) BlobStore(org.apache.storm.blobstore.BlobStore) LocalFsBlobStore(org.apache.storm.blobstore.LocalFsBlobStore) WrappedAuthorizationException(org.apache.storm.utils.WrappedAuthorizationException) IOException(java.io.IOException) IllegalStateException(org.apache.storm.generated.IllegalStateException) AlreadyAliveException(org.apache.storm.generated.AlreadyAliveException) WrappedNotAliveException(org.apache.storm.utils.WrappedNotAliveException) WrappedInvalidTopologyException(org.apache.storm.utils.WrappedInvalidTopologyException) AuthorizationException(org.apache.storm.generated.AuthorizationException) NotAliveException(org.apache.storm.generated.NotAliveException) WrappedAlreadyAliveException(org.apache.storm.utils.WrappedAlreadyAliveException) InterruptedIOException(java.io.InterruptedIOException) KeyAlreadyExistsException(org.apache.storm.generated.KeyAlreadyExistsException) TException(org.apache.storm.thrift.TException) WrappedIllegalStateException(org.apache.storm.utils.WrappedIllegalStateException) KeyNotFoundException(org.apache.storm.generated.KeyNotFoundException) InvalidTopologyException(org.apache.storm.generated.InvalidTopologyException) BindException(java.net.BindException) VisibleForTesting(org.apache.storm.shade.com.google.common.annotations.VisibleForTesting)

Example 13 with VisibleForTesting

use of org.apache.storm.shade.com.google.common.annotations.VisibleForTesting in project storm by apache.

the class Nimbus method cleanInbox.

/**
 * Deletes jar files in dirLoc older than seconds.
 *
 * @param dirLoc  the location to look in for file
 * @param seconds how old is too old and should be deleted
 */
@VisibleForTesting
public static void cleanInbox(String dirLoc, int seconds) {
    final long now = Time.currentTimeMillis();
    final long ms = Time.secsToMillis(seconds);
    File dir = new File(dirLoc);
    for (File f : dir.listFiles((file) -> file.isFile() && ((file.lastModified() + ms) <= now))) {
        if (f.delete()) {
            LOG.info("Cleaning inbox ... deleted: {}", f.getName());
        } else {
            LOG.error("Cleaning inbox ... error deleting: {}", f.getName());
        }
    }
}
Also used : File(java.io.File) VisibleForTesting(org.apache.storm.shade.com.google.common.annotations.VisibleForTesting)

Example 14 with VisibleForTesting

use of org.apache.storm.shade.com.google.common.annotations.VisibleForTesting in project storm by apache.

the class Nimbus method setUpAckerExecutorConfigs.

@VisibleForTesting
public static void setUpAckerExecutorConfigs(String topoName, Map<String, Object> totalConfToSave, Map<String, Object> totalConf, int estimatedNumWorker) {
    int numAckerExecs;
    int numAckerExecsPerWorker;
    if (totalConf.get(Config.TOPOLOGY_ACKER_EXECUTORS) == null) {
        numAckerExecsPerWorker = ObjectReader.getInt(totalConf.get(Config.TOPOLOGY_RAS_ACKER_EXECUTORS_PER_WORKER));
        numAckerExecs = estimatedNumWorker * numAckerExecsPerWorker;
    } else {
        numAckerExecs = ObjectReader.getInt(totalConf.get(Config.TOPOLOGY_ACKER_EXECUTORS));
        if (estimatedNumWorker == 0) {
            numAckerExecsPerWorker = 0;
        } else {
            numAckerExecsPerWorker = (int) Math.ceil((double) numAckerExecs / (double) estimatedNumWorker);
        }
    }
    totalConfToSave.put(Config.TOPOLOGY_RAS_ACKER_EXECUTORS_PER_WORKER, numAckerExecsPerWorker);
    totalConfToSave.put(Config.TOPOLOGY_ACKER_EXECUTORS, numAckerExecs);
    LOG.info("Config {} set to: {} for topology: {}", Config.TOPOLOGY_RAS_ACKER_EXECUTORS_PER_WORKER, numAckerExecsPerWorker, topoName);
    LOG.info("Config {} set to: {} for topology: {}", Config.TOPOLOGY_ACKER_EXECUTORS, numAckerExecs, topoName);
}
Also used : WorkerMetricPoint(org.apache.storm.generated.WorkerMetricPoint) DataPoint(org.apache.storm.metric.api.DataPoint) VisibleForTesting(org.apache.storm.shade.com.google.common.annotations.VisibleForTesting)

Aggregations

VisibleForTesting (org.apache.storm.shade.com.google.common.annotations.VisibleForTesting)14 IOException (java.io.IOException)6 WrappedAuthorizationException (org.apache.storm.utils.WrappedAuthorizationException)5 HashMap (java.util.HashMap)4 IStormClusterState (org.apache.storm.cluster.IStormClusterState)4 AuthorizationException (org.apache.storm.generated.AuthorizationException)4 KeyNotFoundException (org.apache.storm.generated.KeyNotFoundException)4 File (java.io.File)3 InterruptedIOException (java.io.InterruptedIOException)3 BindException (java.net.BindException)3 Map (java.util.Map)3 AlreadyAliveException (org.apache.storm.generated.AlreadyAliveException)3 IllegalStateException (org.apache.storm.generated.IllegalStateException)3 InvalidTopologyException (org.apache.storm.generated.InvalidTopologyException)3 KeyAlreadyExistsException (org.apache.storm.generated.KeyAlreadyExistsException)3 NotAliveException (org.apache.storm.generated.NotAliveException)3 HashSet (java.util.HashSet)2 BlobStore (org.apache.storm.blobstore.BlobStore)2 LocalFsBlobStore (org.apache.storm.blobstore.LocalFsBlobStore)2 WorkerMetricPoint (org.apache.storm.generated.WorkerMetricPoint)2