Search in sources :

Example 1 with InputStreamInputFormat

use of org.apache.sysml.runtime.util.InputStreamInputFormat in project incubator-systemml by apache.

the class FrameReaderTextCSV method readFrameFromInputStream.

@Override
public FrameBlock readFrameFromInputStream(InputStream is, ValueType[] schema, String[] names, long rlen, long clen) throws IOException, DMLRuntimeException {
    //allocate output frame block
    ValueType[] lschema = createOutputSchema(schema, clen);
    String[] lnames = createOutputNames(names, clen);
    FrameBlock ret = createOutputFrameBlock(lschema, lnames, rlen);
    //core read (sequential/parallel) 
    InputStreamInputFormat informat = new InputStreamInputFormat(is);
    InputSplit split = informat.getSplits(null, 1)[0];
    readCSVFrameFromInputSplit(split, informat, null, ret, schema, names, rlen, clen, 0, true);
    return ret;
}
Also used : InputStreamInputFormat(org.apache.sysml.runtime.util.InputStreamInputFormat) ValueType(org.apache.sysml.parser.Expression.ValueType) FrameBlock(org.apache.sysml.runtime.matrix.data.FrameBlock) InputSplit(org.apache.hadoop.mapred.InputSplit)

Aggregations

InputSplit (org.apache.hadoop.mapred.InputSplit)1 ValueType (org.apache.sysml.parser.Expression.ValueType)1 FrameBlock (org.apache.sysml.runtime.matrix.data.FrameBlock)1 InputStreamInputFormat (org.apache.sysml.runtime.util.InputStreamInputFormat)1