Search in sources :

Example 1 with EmitInfo

use of org.apache.storm.hooks.info.EmitInfo in project storm by apache.

the class Task method getOutgoingTasks.

public List<Integer> getOutgoingTasks(String stream, List<Object> values) {
    if (debug) {
        LOG.info("Emitting Tuple: taskId={} componentId={} stream={} values={}", taskId, componentId, stream, values);
    }
    ArrayList<Integer> outTasks = new ArrayList<>();
    ArrayList<LoadAwareCustomStreamGrouping> groupers = streamToGroupers.get(stream);
    if (null != groupers) {
        for (int i = 0; i < groupers.size(); ++i) {
            LoadAwareCustomStreamGrouping grouper = groupers.get(i);
            if (grouper == GrouperFactory.DIRECT) {
                throw new IllegalArgumentException("Cannot do regular emit to direct stream");
            }
            List<Integer> compTasks = grouper.chooseTasks(taskId, values);
            outTasks.addAll(compTasks);
        }
    } else {
        throw new IllegalArgumentException("Unknown stream ID: " + stream);
    }
    if (!userTopologyContext.getHooks().isEmpty()) {
        new EmitInfo(values, stream, taskId, outTasks).applyOn(userTopologyContext);
    }
    try {
        if (emitSampler.getAsBoolean()) {
            executorStats.emittedTuple(stream);
            this.taskMetrics.emittedTuple(stream);
            executorStats.transferredTuples(stream, outTasks.size());
            this.taskMetrics.transferredTuples(stream, outTasks.size());
        }
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
    return outTasks;
}
Also used : ArrayList(java.util.ArrayList) LoadAwareCustomStreamGrouping(org.apache.storm.grouping.LoadAwareCustomStreamGrouping) EmitInfo(org.apache.storm.hooks.info.EmitInfo) IOException(java.io.IOException)

Example 2 with EmitInfo

use of org.apache.storm.hooks.info.EmitInfo in project storm by apache.

the class Task method getOutgoingTasks.

public List<Integer> getOutgoingTasks(Integer outTaskId, String stream, List<Object> values) {
    if (debug) {
        LOG.info("Emitting direct: {}; {} {} {} ", outTaskId, componentId, stream, values);
    }
    String targetComponent = workerTopologyContext.getComponentId(outTaskId);
    Map<String, LoadAwareCustomStreamGrouping> componentGrouping = streamComponentToGrouper.get(stream);
    LoadAwareCustomStreamGrouping grouping = componentGrouping.get(targetComponent);
    if (null == grouping) {
        outTaskId = null;
    }
    if (grouping != null && grouping != GrouperFactory.DIRECT) {
        throw new IllegalArgumentException("Cannot emitDirect to a task expecting a regular grouping");
    }
    if (!userTopologyContext.getHooks().isEmpty()) {
        new EmitInfo(values, stream, taskId, Collections.singletonList(outTaskId)).applyOn(userTopologyContext);
    }
    try {
        if (emitSampler.getAsBoolean()) {
            executorStats.emittedTuple(stream);
            this.taskMetrics.emittedTuple(stream);
            if (null != outTaskId) {
                executorStats.transferredTuples(stream, 1);
                this.taskMetrics.transferredTuples(stream, 1);
            }
        }
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
    if (null != outTaskId) {
        return Collections.singletonList(outTaskId);
    }
    return new ArrayList<>(0);
}
Also used : ArrayList(java.util.ArrayList) LoadAwareCustomStreamGrouping(org.apache.storm.grouping.LoadAwareCustomStreamGrouping) EmitInfo(org.apache.storm.hooks.info.EmitInfo) IOException(java.io.IOException)

Aggregations

IOException (java.io.IOException)2 ArrayList (java.util.ArrayList)2 LoadAwareCustomStreamGrouping (org.apache.storm.grouping.LoadAwareCustomStreamGrouping)2 EmitInfo (org.apache.storm.hooks.info.EmitInfo)2