Search in sources :

Example 6 with TaskInfo

use of com.alibaba.jstorm.task.TaskInfo in project jstorm by alibaba.

the class Common method mkTaskMaker.

@SuppressWarnings({ "rawtypes", "unchecked" })
public static Integer mkTaskMaker(Map<Object, Object> stormConf, Map<String, ?> cidSpec, Map<Integer, TaskInfo> rtn, Integer cnt) {
    if (cidSpec == null) {
        LOG.warn("Component map is empty");
        return cnt;
    }
    Set<?> entrySet = cidSpec.entrySet();
    for (Iterator<?> it = entrySet.iterator(); it.hasNext(); ) {
        Entry entry = (Entry) it.next();
        Object obj = entry.getValue();
        ComponentCommon common = null;
        String componentType = "bolt";
        if (obj instanceof Bolt) {
            common = ((Bolt) obj).get_common();
            componentType = "bolt";
        } else if (obj instanceof SpoutSpec) {
            common = ((SpoutSpec) obj).get_common();
            componentType = "spout";
        } else if (obj instanceof StateSpoutSpec) {
            common = ((StateSpoutSpec) obj).get_common();
            componentType = "spout";
        }
        if (common == null) {
            throw new RuntimeException("No ComponentCommon of " + entry.getKey());
        }
        int declared = Thrift.parallelismHint(common);
        Integer parallelism = declared;
        // Map tmp = (Map) Utils_clj.from_json(common.get_json_conf());
        Map newStormConf = new HashMap(stormConf);
        // newStormConf.putAll(tmp);
        Integer maxParallelism = JStormUtils.parseInt(newStormConf.get(Config.TOPOLOGY_MAX_TASK_PARALLELISM));
        if (maxParallelism != null) {
            parallelism = Math.min(maxParallelism, declared);
        }
        for (int i = 0; i < parallelism; i++) {
            cnt++;
            TaskInfo taskInfo = new TaskInfo((String) entry.getKey(), componentType);
            rtn.put(cnt, taskInfo);
        }
    }
    return cnt;
}
Also used : ShellBolt(backtype.storm.task.ShellBolt) IBolt(backtype.storm.task.IBolt) SystemBolt(backtype.storm.metric.SystemBolt) TaskInfo(com.alibaba.jstorm.task.TaskInfo) Entry(java.util.Map.Entry)

Example 7 with TaskInfo

use of com.alibaba.jstorm.task.TaskInfo in project jstorm by alibaba.

the class TopologyAssign method pushTaskStartEvent.

protected void pushTaskStartEvent(Assignment oldAssignment, Assignment newAssignment, TopologyAssignEvent event) throws Exception {
    // notify jstorm monitor on task assign/reassign/rebalance
    TaskStartEvent taskEvent = new TaskStartEvent();
    taskEvent.setOldAssignment(oldAssignment);
    taskEvent.setNewAssignment(newAssignment);
    taskEvent.setTopologyId(event.getTopologyId());
    Map<Integer, String> task2Component;
    // get from nimbus cache first
    Map<Integer, TaskInfo> taskInfoMap = Cluster.get_all_taskInfo(nimbusData.getStormClusterState(), event.getTopologyId());
    if (taskInfoMap != null) {
        task2Component = Common.getTaskToComponent(taskInfoMap);
    } else {
        task2Component = Common.getTaskToComponent(Cluster.get_all_taskInfo(nimbusData.getStormClusterState(), event.getTopologyId()));
    }
    taskEvent.setTask2Component(task2Component);
    ClusterMetricsRunnable.pushEvent(taskEvent);
}
Also used : TaskInfo(com.alibaba.jstorm.task.TaskInfo) TaskStartEvent(com.alibaba.jstorm.daemon.nimbus.metric.assignment.TaskStartEvent)

Aggregations

TaskInfo (com.alibaba.jstorm.task.TaskInfo)7 InvalidTopologyException (backtype.storm.generated.InvalidTopologyException)2 TopologyTaskHbInfo (backtype.storm.generated.TopologyTaskHbInfo)2 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)2 AlreadyAliveException (backtype.storm.generated.AlreadyAliveException)1 Bolt (backtype.storm.generated.Bolt)1 ComponentSummary (backtype.storm.generated.ComponentSummary)1 ErrorInfo (backtype.storm.generated.ErrorInfo)1 KeyAlreadyExistsException (backtype.storm.generated.KeyAlreadyExistsException)1 KeyNotFoundException (backtype.storm.generated.KeyNotFoundException)1 MetricInfo (backtype.storm.generated.MetricInfo)1 NotAliveException (backtype.storm.generated.NotAliveException)1 SpoutSpec (backtype.storm.generated.SpoutSpec)1 TaskHeartbeat (backtype.storm.generated.TaskHeartbeat)1 TaskSummary (backtype.storm.generated.TaskSummary)1 TopologyAssignException (backtype.storm.generated.TopologyAssignException)1 TopologyInfo (backtype.storm.generated.TopologyInfo)1 TopologyMetric (backtype.storm.generated.TopologyMetric)1 TopologySummary (backtype.storm.generated.TopologySummary)1 SystemBolt (backtype.storm.metric.SystemBolt)1