Examples with Fields cascading.tuple.Fields used on opensource projects

Example 6 with Fields

use of cascading.tuple.Fields in project SpyGlass by ParallelAI.

the class HBaseScheme method columns.

private String[] columns(String[] familyNames, Fields[] fieldsArray) {
    if (columns != null) {
        return columns;
    }
    int size = 0;
    for (Fields fields : fieldsArray) {
        size += fields.size();
    }
    columns = new String[size];
    int count = 0;
    for (int i = 0; i < fieldsArray.length; i++) {
        Fields fields = fieldsArray[i];
        for (int j = 0; j < fields.size(); j++) {
            if (familyNames == null) {
                columns[count++] = hbaseColumn((String) fields.get(j));
            } else {
                columns[count++] = hbaseColumn(familyNames[i]) + (String) fields.get(j);
            }
        }
    }
    return columns;
}

Example 7 with Fields

use of cascading.tuple.Fields in project SpyGlass by ParallelAI.

the class HBaseScheme method setSourceSink.

private void setSourceSink(Fields keyFields, Fields[] columnFields) {
    Fields allFields = keyFields;
    if (columnFields.length != 0) {
        // prepend
        allFields = Fields.join(keyFields, Fields.join(columnFields));
    }
    setSourceFields(allFields);
    setSinkFields(allFields);
}

Example 8 with Fields

use of cascading.tuple.Fields in project SpyGlass by ParallelAI.

the class HBaseRawScheme method sink.

@SuppressWarnings("unchecked")
@Override
public void sink(FlowProcess<JobConf> flowProcess, SinkCall<Object[], OutputCollector> sinkCall) throws IOException {
    TupleEntry tupleEntry = sinkCall.getOutgoingEntry();
    OutputCollector outputCollector = sinkCall.getOutput();
    Tuple key = tupleEntry.selectTuple(RowKeyField);
    Object okey = key.getObject(0);
    ImmutableBytesWritable keyBytes = getBytes(okey);
    Put put = new Put(keyBytes.get());
    Fields outFields = tupleEntry.getFields().subtract(RowKeyField);
    if (null != outFields) {
        TupleEntry values = tupleEntry.selectEntry(outFields);
        for (int n = 0; n < values.getFields().size(); n++) {
            Object o = values.get(n);
            ImmutableBytesWritable valueBytes = getBytes(o);
            Comparable field = outFields.get(n);
            ColumnName cn = parseColumn((String) field);
            if (null == cn.family) {
                if (n >= familyNames.length)
                    cn.family = familyNames[familyNames.length - 1];
                else
                    cn.family = familyNames[n];
            }
            if (null != o || writeNulls)
                put.add(Bytes.toBytes(cn.family), Bytes.toBytes(cn.name), valueBytes.get());
        }
    }
    outputCollector.collect(null, put);
}

Example 9 with Fields

use of cascading.tuple.Fields in project SpyGlass by ParallelAI.

the class HBaseRawScheme method setSourceFields.

private void setSourceFields() {
    Fields sourceFields = Fields.join(RowKeyField, RowField);
    setSourceFields(sourceFields);
}