Search in sources :

Example 1 with Iface

use of backtype.storm.generated.Nimbus.Iface in project jstorm by alibaba.

the class JStormUtils method getMetrics.

/**
     * Get Topology Metrics
     *
     * @param topologyName topology name
     * @param metricType,  please refer to MetaType, default to MetaType.TASK.getT()
     * @param window,      please refer to AsmWindow, default to AsmWindow.M1_WINDOW
     * @return
     */
public static Map<String, Double> getMetrics(Map conf, String topologyName, MetaType metricType, Integer window) {
    NimbusClientWrapper nimbusClient = null;
    Iface client = null;
    Map<String, Double> summary = new HashMap<>();
    try {
        nimbusClient = new NimbusClientWrapper();
        nimbusClient.init(conf);
        client = nimbusClient.getClient();
        String topologyId = client.getTopologyId(topologyName);
        if (metricType == null) {
            metricType = MetaType.TASK;
        }
        List<MetricInfo> allTaskMetrics = client.getMetrics(topologyId, metricType.getT());
        if (allTaskMetrics == null) {
            throw new RuntimeException("Failed to get metrics");
        }
        if (window == null || !AsmWindow.TIME_WINDOWS.contains(window)) {
            window = AsmWindow.M1_WINDOW;
        }
        for (MetricInfo taskMetrics : allTaskMetrics) {
            Map<String, Map<Integer, MetricSnapshot>> metrics = taskMetrics.get_metrics();
            if (metrics == null) {
                System.out.println("Failed to get task metrics");
                continue;
            }
            for (Entry<String, Map<Integer, MetricSnapshot>> entry : metrics.entrySet()) {
                String key = entry.getKey();
                MetricSnapshot metric = entry.getValue().get(window);
                if (metric == null) {
                    throw new RuntimeException("Failed to get one minute metrics of " + key);
                }
                if (metric.get_metricType() == MetricType.COUNTER.getT()) {
                    summary.put(key, (double) metric.get_longValue());
                } else if (metric.get_metricType() == MetricType.GAUGE.getT()) {
                    summary.put(key, metric.get_doubleValue());
                } else {
                    summary.put(key, metric.get_mean());
                }
            }
        }
        return summary;
    } catch (Exception e) {
        throw new RuntimeException(e);
    } finally {
        if (client != null) {
            nimbusClient.cleanup();
        }
    }
}
Also used : NimbusClientWrapper(backtype.storm.utils.NimbusClientWrapper) HashMap(java.util.HashMap) ExecuteException(org.apache.commons.exec.ExecuteException) TException(org.apache.thrift.TException) IOException(java.io.IOException) Iface(backtype.storm.generated.Nimbus.Iface) MetricInfo(backtype.storm.generated.MetricInfo) Map(java.util.Map) HashMap(java.util.HashMap) TreeMap(java.util.TreeMap) MetricSnapshot(backtype.storm.generated.MetricSnapshot)

Example 2 with Iface

use of backtype.storm.generated.Nimbus.Iface in project jstorm by alibaba.

the class NimbusServer method initThrift.

@SuppressWarnings("rawtypes")
private void initThrift(Map conf) throws TTransportException {
    Integer thrift_port = JStormUtils.parseInt(conf.get(Config.NIMBUS_THRIFT_PORT));
    TNonblockingServerSocket socket = new TNonblockingServerSocket(thrift_port);
    Integer maxReadBufSize = JStormUtils.parseInt(conf.get(Config.NIMBUS_THRIFT_MAX_BUFFER_SIZE));
    THsHaServer.Args args = new THsHaServer.Args(socket);
    args.workerThreads(ServiceHandler.THREAD_NUM);
    args.protocolFactory(new TBinaryProtocol.Factory(false, true, maxReadBufSize, -1));
    args.processor(new Nimbus.Processor<Iface>(serviceHandler));
    args.maxReadBufferBytes = maxReadBufSize;
    thriftServer = new THsHaServer(args);
    LOG.info("Successfully started nimbus: started Thrift server...");
    thriftServer.serve();
}
Also used : Iface(backtype.storm.generated.Nimbus.Iface) THsHaServer(org.apache.thrift.server.THsHaServer) TBinaryProtocol(org.apache.thrift.protocol.TBinaryProtocol) TNonblockingServerSocket(org.apache.thrift.transport.TNonblockingServerSocket) Nimbus(backtype.storm.generated.Nimbus) INimbus(backtype.storm.scheduler.INimbus)

Aggregations

Iface (backtype.storm.generated.Nimbus.Iface)2 MetricInfo (backtype.storm.generated.MetricInfo)1 MetricSnapshot (backtype.storm.generated.MetricSnapshot)1 Nimbus (backtype.storm.generated.Nimbus)1 INimbus (backtype.storm.scheduler.INimbus)1 NimbusClientWrapper (backtype.storm.utils.NimbusClientWrapper)1 IOException (java.io.IOException)1 HashMap (java.util.HashMap)1 Map (java.util.Map)1 TreeMap (java.util.TreeMap)1 ExecuteException (org.apache.commons.exec.ExecuteException)1 TException (org.apache.thrift.TException)1 TBinaryProtocol (org.apache.thrift.protocol.TBinaryProtocol)1 THsHaServer (org.apache.thrift.server.THsHaServer)1 TNonblockingServerSocket (org.apache.thrift.transport.TNonblockingServerSocket)1