Examples with Fields backtype.storm.tuple.Fields used on opensource projects

Example 61 with Fields

use of backtype.storm.tuple.Fields in project jstorm by alibaba.

the class GroupedStream method partitionAggregate.

@Override
public IAggregatableStream partitionAggregate(Fields inputFields, Aggregator agg, Fields functionFields) {
    Aggregator groupedAgg = new GroupedAggregator(agg, _groupFields, inputFields, functionFields.size());
    Fields allInFields = TridentUtils.fieldsUnion(_groupFields, inputFields);
    Fields allOutFields = TridentUtils.fieldsConcat(_groupFields, functionFields);
    Stream s = _stream.partitionAggregate(allInFields, groupedAgg, allOutFields);
    return new GroupedStream(s, _groupFields);
}

Example 62 with Fields

use of backtype.storm.tuple.Fields in project jstorm by alibaba.

the class DelimitedRecordFormat method format.

@Override
public byte[] format(Tuple tuple) {
    StringBuilder sb = new StringBuilder();
    Fields fields = this.fields == null ? tuple.getFields() : this.fields;
    int size = fields.size();
    for (int i = 0; i < size; i++) {
        sb.append(tuple.getValueByField(fields.get(i)));
        if (i != size - 1) {
            sb.append(this.fieldDelimiter);
        }
    }
    sb.append(this.recordDelimiter);
    return sb.toString().getBytes();
}

Example 63 with Fields

use of backtype.storm.tuple.Fields in project storm by nathanmarz.

the class TridentSpoutExecutor method declareOutputFields.

@Override
public void declareOutputFields(OutputFieldsDeclarer declarer) {
    List<String> fields = new ArrayList(_spout.getOutputFields().toList());
    fields.add(0, ID_FIELD);
    declarer.declareStream(_streamName, new Fields(fields));
}

Example 64 with Fields

use of backtype.storm.tuple.Fields in project storm by nathanmarz.

the class MasterBatchCoordinator method declareOutputFields.

@Override
public void declareOutputFields(OutputFieldsDeclarer declarer) {
    // in partitioned example, in case an emitter task receives a later transaction than it's emitted so far,
    // when it sees the earlier txid it should know to emit nothing
    declarer.declareStream(BATCH_STREAM_ID, new Fields("tx"));
    declarer.declareStream(COMMIT_STREAM_ID, new Fields("tx"));
    declarer.declareStream(SUCCESS_STREAM_ID, new Fields("tx"));
}

Example 65 with Fields

use of backtype.storm.tuple.Fields in project storm by nathanmarz.

the class TridentUtils method getSingleOutputStreamFields.

public static Fields getSingleOutputStreamFields(IComponent component) {
    OutputFieldsGetter getter = new OutputFieldsGetter();
    component.declareOutputFields(getter);
    Map<String, StreamInfo> declaration = getter.getFieldsDeclaration();
    if (declaration.size() != 1) {
        throw new RuntimeException("Trident only supports components that emit a single stream");
    }
    StreamInfo si = declaration.values().iterator().next();
    if (si.is_direct()) {
        throw new RuntimeException("Trident does not support direct streams");
    }
    return new Fields(si.get_output_fields());
}