Search in sources :

Example 61 with TException

use of org.apache.thrift.TException in project storm by apache.

the class Nimbus method uploadNewCredentials.

@Override
public void uploadNewCredentials(String topoName, Credentials credentials) throws NotAliveException, InvalidTopologyException, AuthorizationException, TException {
    try {
        uploadNewCredentialsCalls.mark();
        IStormClusterState state = stormClusterState;
        String topoId = toTopoId(topoName);
        if (topoId == null) {
            throw new NotAliveException(topoName + " is not alive");
        }
        Map<String, Object> topoConf = tryReadTopoConf(topoId, blobStore);
        if (credentials == null) {
            credentials = new Credentials(Collections.emptyMap());
        }
        checkAuthorization(topoName, topoConf, "uploadNewCredentials");
        synchronized (credUpdateLock) {
            state.setCredentials(topoId, credentials, topoConf);
        }
    } catch (Exception e) {
        LOG.warn("Upload Creds topology exception. (topology name='{}')", topoName, e);
        if (e instanceof TException) {
            throw (TException) e;
        }
        throw new RuntimeException(e);
    }
}
Also used : TException(org.apache.thrift.TException) NotAliveException(org.apache.storm.generated.NotAliveException) IStormClusterState(org.apache.storm.cluster.IStormClusterState) Credentials(org.apache.storm.generated.Credentials) AuthorizationException(org.apache.storm.generated.AuthorizationException) NotAliveException(org.apache.storm.generated.NotAliveException) InterruptedIOException(java.io.InterruptedIOException) TException(org.apache.thrift.TException) IOException(java.io.IOException) AlreadyAliveException(org.apache.storm.generated.AlreadyAliveException) KeyAlreadyExistsException(org.apache.storm.generated.KeyAlreadyExistsException) KeyNotFoundException(org.apache.storm.generated.KeyNotFoundException) InvalidTopologyException(org.apache.storm.generated.InvalidTopologyException) BindException(java.net.BindException)

Example 62 with TException

use of org.apache.thrift.TException in project storm by apache.

the class Nimbus method downloadBlobChunk.

@SuppressWarnings("deprecation")
@Override
public ByteBuffer downloadBlobChunk(String session) throws AuthorizationException, TException {
    try {
        BufferInputStream is = blobDownloaders.get(session);
        if (is == null) {
            throw new RuntimeException("Blob for session " + session + " does not exist (or timed out)");
        }
        byte[] ret = is.read();
        if (ret.length == 0) {
            is.close();
            blobDownloaders.remove(session);
        } else {
            blobDownloaders.put(session, is);
        }
        LOG.debug("Sending {} bytes", ret.length);
        return ByteBuffer.wrap(ret);
    } catch (Exception e) {
        LOG.warn("download blob chunk exception.", e);
        if (e instanceof TException) {
            throw (TException) e;
        }
        throw new RuntimeException(e);
    }
}
Also used : BufferInputStream(org.apache.storm.utils.BufferInputStream) TException(org.apache.thrift.TException) AuthorizationException(org.apache.storm.generated.AuthorizationException) NotAliveException(org.apache.storm.generated.NotAliveException) InterruptedIOException(java.io.InterruptedIOException) TException(org.apache.thrift.TException) IOException(java.io.IOException) AlreadyAliveException(org.apache.storm.generated.AlreadyAliveException) KeyAlreadyExistsException(org.apache.storm.generated.KeyAlreadyExistsException) KeyNotFoundException(org.apache.storm.generated.KeyNotFoundException) InvalidTopologyException(org.apache.storm.generated.InvalidTopologyException) BindException(java.net.BindException)

Example 63 with TException

use of org.apache.thrift.TException in project storm by apache.

the class Nimbus method finishBlobUpload.

@SuppressWarnings("deprecation")
@Override
public void finishBlobUpload(String session) throws AuthorizationException, TException {
    try {
        OutputStream os = blobUploaders.get(session);
        if (os == null) {
            throw new RuntimeException("Blob for session " + session + " does not exist (or timed out)");
        }
        os.close();
        LOG.info("Finished uploading blob for session {}. Closing session.", session);
        blobUploaders.remove(session);
    } catch (Exception e) {
        LOG.warn("finish blob upload exception.", e);
        if (e instanceof TException) {
            throw (TException) e;
        }
        throw new RuntimeException(e);
    }
}
Also used : TException(org.apache.thrift.TException) FileOutputStream(java.io.FileOutputStream) AtomicOutputStream(org.apache.storm.blobstore.AtomicOutputStream) OutputStream(java.io.OutputStream) AuthorizationException(org.apache.storm.generated.AuthorizationException) NotAliveException(org.apache.storm.generated.NotAliveException) InterruptedIOException(java.io.InterruptedIOException) TException(org.apache.thrift.TException) IOException(java.io.IOException) AlreadyAliveException(org.apache.storm.generated.AlreadyAliveException) KeyAlreadyExistsException(org.apache.storm.generated.KeyAlreadyExistsException) KeyNotFoundException(org.apache.storm.generated.KeyNotFoundException) InvalidTopologyException(org.apache.storm.generated.InvalidTopologyException) BindException(java.net.BindException)

Example 64 with TException

use of org.apache.thrift.TException in project storm by apache.

the class Nimbus method beginBlobDownload.

@SuppressWarnings("deprecation")
@Override
public BeginDownloadResult beginBlobDownload(String key) throws AuthorizationException, KeyNotFoundException, TException {
    try {
        InputStreamWithMeta is = blobStore.getBlob(key, getSubject());
        String sessionId = Utils.uuid();
        BeginDownloadResult ret = new BeginDownloadResult(is.getVersion(), sessionId);
        ret.set_data_size(is.getFileLength());
        blobDownloaders.put(sessionId, new BufferInputStream(is, (int) conf.getOrDefault(Config.STORM_BLOBSTORE_INPUTSTREAM_BUFFER_SIZE_BYTES, 65536)));
        LOG.info("Created download session for {}", key);
        return ret;
    } catch (Exception e) {
        LOG.warn("begin blob download exception.", e);
        if (e instanceof TException) {
            throw (TException) e;
        }
        throw new RuntimeException(e);
    }
}
Also used : BufferInputStream(org.apache.storm.utils.BufferInputStream) TException(org.apache.thrift.TException) InputStreamWithMeta(org.apache.storm.blobstore.InputStreamWithMeta) BeginDownloadResult(org.apache.storm.generated.BeginDownloadResult) AuthorizationException(org.apache.storm.generated.AuthorizationException) NotAliveException(org.apache.storm.generated.NotAliveException) InterruptedIOException(java.io.InterruptedIOException) TException(org.apache.thrift.TException) IOException(java.io.IOException) AlreadyAliveException(org.apache.storm.generated.AlreadyAliveException) KeyAlreadyExistsException(org.apache.storm.generated.KeyAlreadyExistsException) KeyNotFoundException(org.apache.storm.generated.KeyNotFoundException) InvalidTopologyException(org.apache.storm.generated.InvalidTopologyException) BindException(java.net.BindException)

Example 65 with TException

use of org.apache.thrift.TException in project storm by apache.

the class Nimbus method getComponentPageInfo.

@Override
public ComponentPageInfo getComponentPageInfo(String topoId, String componentId, String window, boolean includeSys) throws NotAliveException, AuthorizationException, TException {
    try {
        getComponentPageInfoCalls.mark();
        CommonTopoInfo info = getCommonTopoInfo(topoId, "getComponentPageInfo");
        if (info.base == null) {
            throw new NotAliveException(topoId);
        }
        StormTopology topology = info.topology;
        Map<String, Object> topoConf = info.topoConf;
        Assignment assignment = info.assignment;
        Map<List<Long>, List<Object>> exec2NodePort = new HashMap<>();
        Map<String, String> nodeToHost;
        Map<List<Long>, List<Object>> exec2HostPort = new HashMap<>();
        if (assignment != null) {
            Map<List<Long>, NodeInfo> execToNodeInfo = assignment.get_executor_node_port();
            nodeToHost = assignment.get_node_host();
            for (Entry<List<Long>, NodeInfo> entry : execToNodeInfo.entrySet()) {
                NodeInfo ni = entry.getValue();
                List<Object> nodePort = Arrays.asList(ni.get_node(), ni.get_port_iterator().next());
                List<Object> hostPort = Arrays.asList(nodeToHost.get(ni.get_node()), ni.get_port_iterator().next());
                exec2NodePort.put(entry.getKey(), nodePort);
                exec2HostPort.put(entry.getKey(), hostPort);
            }
        } else {
            nodeToHost = Collections.emptyMap();
        }
        ComponentPageInfo compPageInfo = StatsUtil.aggCompExecsStats(exec2HostPort, info.taskToComponent, info.beats, window, includeSys, topoId, topology, componentId);
        if (compPageInfo.get_component_type() == ComponentType.SPOUT) {
            compPageInfo.set_resources_map(setResourcesDefaultIfNotSet(ResourceUtils.getSpoutsResources(topology, topoConf), componentId, topoConf));
        } else {
            //bolt
            compPageInfo.set_resources_map(setResourcesDefaultIfNotSet(ResourceUtils.getBoltsResources(topology, topoConf), componentId, topoConf));
        }
        compPageInfo.set_topology_name(info.topoName);
        compPageInfo.set_errors(stormClusterState.errors(topoId, componentId));
        compPageInfo.set_topology_status(extractStatusStr(info.base));
        if (info.base.is_set_component_debug()) {
            DebugOptions debug = info.base.get_component_debug().get(componentId);
            if (debug != null) {
                compPageInfo.set_debug_options(debug);
            }
        }
        // Add the event logger details.
        Map<String, List<Integer>> compToTasks = Utils.reverseMap(info.taskToComponent);
        if (compToTasks.containsKey(StormCommon.EVENTLOGGER_COMPONENT_ID)) {
            List<Integer> tasks = compToTasks.get(StormCommon.EVENTLOGGER_COMPONENT_ID);
            tasks.sort(null);
            // Find the task the events from this component route to.
            int taskIndex = TupleUtils.chooseTaskIndex(Collections.singletonList(componentId), tasks.size());
            int taskId = tasks.get(taskIndex);
            String host = null;
            Integer port = null;
            for (Entry<List<Long>, List<Object>> entry : exec2HostPort.entrySet()) {
                int start = entry.getKey().get(0).intValue();
                int end = entry.getKey().get(1).intValue();
                if (taskId >= start && taskId <= end) {
                    host = (String) entry.getValue().get(0);
                    port = ((Number) entry.getValue().get(1)).intValue();
                    break;
                }
            }
            if (host != null && port != null) {
                compPageInfo.set_eventlog_host(host);
                compPageInfo.set_eventlog_port(port);
            }
        }
        return compPageInfo;
    } catch (Exception e) {
        LOG.warn("getComponentPageInfo exception. (topo id='{}')", topoId, e);
        if (e instanceof TException) {
            throw (TException) e;
        }
        throw new RuntimeException(e);
    }
}
Also used : TException(org.apache.thrift.TException) HashMap(java.util.HashMap) StormTopology(org.apache.storm.generated.StormTopology) DebugOptions(org.apache.storm.generated.DebugOptions) Assignment(org.apache.storm.generated.Assignment) SchedulerAssignment(org.apache.storm.scheduler.SchedulerAssignment) NotAliveException(org.apache.storm.generated.NotAliveException) ArrayList(java.util.ArrayList) List(java.util.List) DataPoint(org.apache.storm.metric.api.DataPoint) AuthorizationException(org.apache.storm.generated.AuthorizationException) NotAliveException(org.apache.storm.generated.NotAliveException) InterruptedIOException(java.io.InterruptedIOException) TException(org.apache.thrift.TException) IOException(java.io.IOException) AlreadyAliveException(org.apache.storm.generated.AlreadyAliveException) KeyAlreadyExistsException(org.apache.storm.generated.KeyAlreadyExistsException) KeyNotFoundException(org.apache.storm.generated.KeyNotFoundException) InvalidTopologyException(org.apache.storm.generated.InvalidTopologyException) BindException(java.net.BindException) ComponentPageInfo(org.apache.storm.generated.ComponentPageInfo) NodeInfo(org.apache.storm.generated.NodeInfo)

Aggregations

TException (org.apache.thrift.TException)381 IOException (java.io.IOException)164 MetaException (org.apache.hadoop.hive.metastore.api.MetaException)57 NoSuchObjectException (org.apache.hadoop.hive.metastore.api.NoSuchObjectException)48 ArrayList (java.util.ArrayList)42 HashMap (java.util.HashMap)40 Table (org.apache.hadoop.hive.metastore.api.Table)38 Map (java.util.Map)30 TBinaryProtocol (org.apache.thrift.protocol.TBinaryProtocol)29 AuthorizationException (org.apache.storm.generated.AuthorizationException)27 Test (org.junit.Test)26 List (java.util.List)25 InvalidObjectException (org.apache.hadoop.hive.metastore.api.InvalidObjectException)24 UnknownHostException (java.net.UnknownHostException)23 TProtocol (org.apache.thrift.protocol.TProtocol)23 FileNotFoundException (java.io.FileNotFoundException)22 HiveSQLException (org.apache.hive.service.cli.HiveSQLException)22 InvalidMetaException (com.netflix.metacat.common.server.connectors.exception.InvalidMetaException)21 LoginException (javax.security.auth.login.LoginException)21 ConnectorException (com.netflix.metacat.common.server.connectors.exception.ConnectorException)20