Search in sources :

Example 1 with TException

use of org.apache.thrift7.TException in project storm by nathanmarz.

the class ReturnResultsReducer method complete.

@Override
public void complete(ReturnResultsState state, TridentCollector collector) {
    // only one of the multireducers will receive the tuples
    if (state.returnInfo != null) {
        String result = JSONValue.toJSONString(state.results);
        Map retMap = (Map) JSONValue.parse(state.returnInfo);
        final String host = (String) retMap.get("host");
        final int port = Utils.getInt(retMap.get("port"));
        String id = (String) retMap.get("id");
        DistributedRPCInvocations.Iface client;
        if (local) {
            client = (DistributedRPCInvocations.Iface) ServiceRegistry.getService(host);
        } else {
            List server = new ArrayList() {

                {
                    add(host);
                    add(port);
                }
            };
            if (!_clients.containsKey(server)) {
                _clients.put(server, new DRPCInvocationsClient(host, port));
            }
            client = _clients.get(server);
        }
        try {
            client.result(id, result);
        } catch (TException e) {
            collector.reportError(e);
        }
    }
}
Also used : DRPCInvocationsClient(backtype.storm.drpc.DRPCInvocationsClient) TException(org.apache.thrift7.TException) ArrayList(java.util.ArrayList) DistributedRPCInvocations(backtype.storm.generated.DistributedRPCInvocations) ArrayList(java.util.ArrayList) List(java.util.List) HashMap(java.util.HashMap) Map(java.util.Map)

Example 2 with TException

use of org.apache.thrift7.TException in project storm by nathanmarz.

the class DRPCInvocationsClient method connect.

private void connect() throws TException {
    conn = new TFramedTransport(new TSocket(host, port));
    client = new DistributedRPCInvocations.Client(new TBinaryProtocol(conn));
    conn.open();
}
Also used : TBinaryProtocol(org.apache.thrift7.protocol.TBinaryProtocol) TFramedTransport(org.apache.thrift7.transport.TFramedTransport) DistributedRPCInvocations(backtype.storm.generated.DistributedRPCInvocations) TSocket(org.apache.thrift7.transport.TSocket)

Example 3 with TException

use of org.apache.thrift7.TException in project storm by nathanmarz.

the class DRPCSpout method nextTuple.

@Override
public void nextTuple() {
    boolean gotRequest = false;
    if (_local_drpc_id == null) {
        for (int i = 0; i < _clients.size(); i++) {
            DRPCInvocationsClient client = _clients.get(i);
            try {
                DRPCRequest req = client.fetchRequest(_function);
                if (req.get_request_id().length() > 0) {
                    Map returnInfo = new HashMap();
                    returnInfo.put("id", req.get_request_id());
                    returnInfo.put("host", client.getHost());
                    returnInfo.put("port", client.getPort());
                    gotRequest = true;
                    _collector.emit(new Values(req.get_func_args(), JSONValue.toJSONString(returnInfo)), new DRPCMessageId(req.get_request_id(), i));
                    break;
                }
            } catch (TException e) {
                LOG.error("Failed to fetch DRPC result from DRPC server", e);
            }
        }
    } else {
        DistributedRPCInvocations.Iface drpc = (DistributedRPCInvocations.Iface) ServiceRegistry.getService(_local_drpc_id);
        if (drpc != null) {
            // can happen during shutdown of drpc while topology is still up
            try {
                DRPCRequest req = drpc.fetchRequest(_function);
                if (req.get_request_id().length() > 0) {
                    Map returnInfo = new HashMap();
                    returnInfo.put("id", req.get_request_id());
                    returnInfo.put("host", _local_drpc_id);
                    returnInfo.put("port", 0);
                    gotRequest = true;
                    _collector.emit(new Values(req.get_func_args(), JSONValue.toJSONString(returnInfo)), new DRPCMessageId(req.get_request_id(), 0));
                }
            } catch (TException e) {
                throw new RuntimeException(e);
            }
        }
    }
    if (!gotRequest) {
        Utils.sleep(1);
    }
}
Also used : TException(org.apache.thrift7.TException) HashMap(java.util.HashMap) Values(backtype.storm.tuple.Values) DistributedRPCInvocations(backtype.storm.generated.DistributedRPCInvocations) DRPCRequest(backtype.storm.generated.DRPCRequest) HashMap(java.util.HashMap) Map(java.util.Map)

Example 4 with TException

use of org.apache.thrift7.TException in project jstorm by alibaba.

the class ClusterInfoBolt method getClusterInfo.

private void getClusterInfo(Client client) {
    try {
        ClusterSummary clusterSummary = client.getClusterInfo();
        List<SupervisorSummary> supervisorSummaryList = clusterSummary.get_supervisors();
        int totalWorkers = 0;
        int usedWorkers = 0;
        for (SupervisorSummary summary : supervisorSummaryList) {
            totalWorkers += summary.get_num_workers();
            usedWorkers += summary.get_num_used_workers();
        }
        int freeWorkers = totalWorkers - usedWorkers;
        LOGGER.info("cluster totalWorkers = " + totalWorkers + ", usedWorkers = " + usedWorkers + ", freeWorkers  = " + freeWorkers);
        HttpCatClient.sendMetric("ClusterMonitor", "freeSlots", "avg", String.valueOf(freeWorkers));
        HttpCatClient.sendMetric("ClusterMonitor", "totalSlots", "avg", String.valueOf(totalWorkers));
        List<TopologySummary> topologySummaryList = clusterSummary.get_topologies();
        long clusterTPS = 0l;
        for (TopologySummary topology : topologySummaryList) {
            long topologyTPS = getTopologyTPS(topology, client);
            clusterTPS += topologyTPS;
            if (topology.get_name().startsWith("ClusterMonitor")) {
                continue;
            }
            HttpCatClient.sendMetric(topology.get_name(), topology.get_name() + "-TPS", "avg", String.valueOf(topologyTPS));
        }
        HttpCatClient.sendMetric("ClusterMonitor", "ClusterEmitTPS", "avg", String.valueOf(clusterTPS));
    } catch (TException e) {
        initClient(configMap);
        LOGGER.error("get client info error.", e);
    } catch (NotAliveException nae) {
        LOGGER.warn("topology is dead.", nae);
    }
}
Also used : TException(org.apache.thrift7.TException) NotAliveException(backtype.storm.generated.NotAliveException) ClusterSummary(backtype.storm.generated.ClusterSummary) SupervisorSummary(backtype.storm.generated.SupervisorSummary) TopologySummary(backtype.storm.generated.TopologySummary)

Example 5 with TException

use of org.apache.thrift7.TException in project storm-kestrel by nathanmarz.

the class KestrelThriftSpout method fail.

public void fail(Object msgId) {
    KestrelSourceId sourceId = (KestrelSourceId) msgId;
    KestrelClientInfo info = _kestrels.get(sourceId.index);
    // see not above about why this works with blacklisting strategy
    try {
        if (info.client != null) {
            HashSet xids = new HashSet();
            xids.add(sourceId.id);
            info.client.abort(_queueName, xids);
        }
    } catch (TException e) {
        blacklist(info, e);
    }
}
Also used : TException(org.apache.thrift7.TException) HashSet(java.util.HashSet)

Aggregations

TException (org.apache.thrift7.TException)8 DistributedRPCInvocations (backtype.storm.generated.DistributedRPCInvocations)4 HashMap (java.util.HashMap)4 Map (java.util.Map)4 ArrayList (java.util.ArrayList)3 HashSet (java.util.HashSet)3 List (java.util.List)3 TBinaryProtocol (org.apache.thrift7.protocol.TBinaryProtocol)2 TFramedTransport (org.apache.thrift7.transport.TFramedTransport)2 TSocket (org.apache.thrift7.transport.TSocket)2 DRPCInvocationsClient (backtype.storm.drpc.DRPCInvocationsClient)1 ClusterSummary (backtype.storm.generated.ClusterSummary)1 DRPCRequest (backtype.storm.generated.DRPCRequest)1 DistributedRPC (backtype.storm.generated.DistributedRPC)1 NotAliveException (backtype.storm.generated.NotAliveException)1 SupervisorSummary (backtype.storm.generated.SupervisorSummary)1 TopologySummary (backtype.storm.generated.TopologySummary)1 Values (backtype.storm.tuple.Values)1 NimbusClient (backtype.storm.utils.NimbusClient)1 LinkedList (java.util.LinkedList)1