Search in sources :

Example 6 with Matrix

use of org.apache.sysml.udf.Matrix in project incubator-systemml by apache.

the class DynamicReadMatrixCP method execute.

@Override
public void execute() {
    try {
        String fname = ((Scalar) this.getFunctionInput(0)).getValue();
        Integer m = Integer.parseInt(((Scalar) this.getFunctionInput(1)).getValue());
        Integer n = Integer.parseInt(((Scalar) this.getFunctionInput(2)).getValue());
        String format = ((Scalar) this.getFunctionInput(3)).getValue();
        InputInfo ii = InputInfo.stringToInputInfo(format);
        OutputInfo oi = OutputInfo.BinaryBlockOutputInfo;
        MatrixBlock mbTmp = DataConverter.readMatrixFromHDFS(fname, ii, m, n, ConfigurationManager.getBlocksize(), ConfigurationManager.getBlocksize());
        String fnameTmp = createOutputFilePathAndName("TMP");
        _ret = new Matrix(fnameTmp, m, n, ValueType.Double);
        _ret.setMatrixDoubleArray(mbTmp, oi, ii);
    // NOTE: The packagesupport wrapper creates a new MatrixObjectNew with the given
    // matrix block. This leads to a dirty state of the new object. Hence, the resulting
    // intermediate plan variable will be exported in front of MR jobs and during this export
    // the format will be changed to binary block (the contract of external functions),
    // no matter in which format the original matrix was.
    } catch (Exception e) {
        throw new RuntimeException("Error executing dynamic read of matrix", e);
    }
}
Also used : OutputInfo(org.apache.sysml.runtime.matrix.data.OutputInfo) MatrixBlock(org.apache.sysml.runtime.matrix.data.MatrixBlock) InputInfo(org.apache.sysml.runtime.matrix.data.InputInfo) Matrix(org.apache.sysml.udf.Matrix) Scalar(org.apache.sysml.udf.Scalar)

Example 7 with Matrix

use of org.apache.sysml.udf.Matrix in project incubator-systemml by apache.

the class DynamicReadMatrixRcCP method execute.

@Override
public void execute() {
    try {
        String fname = ((Scalar) this.getFunctionInput(0)).getValue();
        Integer m = Integer.parseInt(((Scalar) this.getFunctionInput(1)).getValue());
        Integer n = Integer.parseInt(((Scalar) this.getFunctionInput(2)).getValue());
        String format = ((Scalar) this.getFunctionInput(3)).getValue();
        InputInfo ii = InputInfo.stringToInputInfo(format);
        OutputInfo oi = OutputInfo.BinaryBlockOutputInfo;
        String fnameTmp = createOutputFilePathAndName("TMP");
        _ret = new Matrix(fnameTmp, m, n, ValueType.Double);
        MatrixBlock mbTmp = DataConverter.readMatrixFromHDFS(fname, ii, m, n, ConfigurationManager.getBlocksize(), ConfigurationManager.getBlocksize());
        _ret.setMatrixDoubleArray(mbTmp, oi, ii);
        _rc = new Scalar(ScalarValueType.Integer, "0");
    // NOTE: The packagesupport wrapper creates a new MatrixObjectNew with the given
    // matrix block. This leads to a dirty state of the new object. Hence, the resulting
    // intermediate plan variable will be exported in front of MR jobs and during this export
    // the format will be changed to binary block (the contract of external functions),
    // no matter in which format the original matrix was.
    } catch (Exception e) {
        _rc = new Scalar(ScalarValueType.Integer, "1");
    // throw new PackageRuntimeException("Error executing dynamic read of matrix",e);
    }
}
Also used : OutputInfo(org.apache.sysml.runtime.matrix.data.OutputInfo) MatrixBlock(org.apache.sysml.runtime.matrix.data.MatrixBlock) InputInfo(org.apache.sysml.runtime.matrix.data.InputInfo) Matrix(org.apache.sysml.udf.Matrix) Scalar(org.apache.sysml.udf.Scalar)

Example 8 with Matrix

use of org.apache.sysml.udf.Matrix in project incubator-systemml by apache.

the class DynamicWriteMatrixCP method execute.

@Override
public void execute() {
    boolean success = false;
    try {
        Matrix mat = (Matrix) this.getFunctionInput(0);
        String fname = ((Scalar) this.getFunctionInput(1)).getValue();
        String format = ((Scalar) this.getFunctionInput(2)).getValue();
        MatrixObject mo = mat.getMatrixObject();
        MatrixCharacteristics mc = mo.getMatrixCharacteristics();
        OutputInfo oi = OutputInfo.stringToOutputInfo(format);
        MatrixBlock mb = mo.acquireRead();
        DataConverter.writeMatrixToHDFS(mb, fname, oi, mc);
        mo.release();
        success = true;
    } catch (Exception e) {
        throw new RuntimeException("Error executing dynamic write of matrix", e);
    }
    _success = new Scalar(ScalarValueType.Boolean, String.valueOf(success));
}
Also used : OutputInfo(org.apache.sysml.runtime.matrix.data.OutputInfo) MatrixBlock(org.apache.sysml.runtime.matrix.data.MatrixBlock) Matrix(org.apache.sysml.udf.Matrix) MatrixObject(org.apache.sysml.runtime.controlprogram.caching.MatrixObject) Scalar(org.apache.sysml.udf.Scalar) MatrixCharacteristics(org.apache.sysml.runtime.matrix.MatrixCharacteristics)

Example 9 with Matrix

use of org.apache.sysml.udf.Matrix in project incubator-systemml by apache.

the class MultiInputCbind method allocateOutput.

private void allocateOutput() {
    String dir = createOutputFilePathAndName("TMP");
    ret = new Matrix(dir, numRetRows, numRetCols, ValueType.Double);
    retMB = new MatrixBlock((int) numRetRows, (int) numRetCols, false);
    retMB.allocateDenseBlock();
}
Also used : MatrixBlock(org.apache.sysml.runtime.matrix.data.MatrixBlock) Matrix(org.apache.sysml.udf.Matrix)

Example 10 with Matrix

use of org.apache.sysml.udf.Matrix in project incubator-systemml by apache.

the class OrderWrapper method execute.

@Override
public void execute() {
    try {
        Matrix inM = (Matrix) getFunctionInput(0);
        double[][] inData = inM.getMatrixAsDoubleArray();
        int col = Integer.parseInt(((Scalar) getFunctionInput(1)).getValue());
        boolean desc = Boolean.parseBoolean(((Scalar) getFunctionInput(2)).getValue());
        // sort input matrix (in-place)
        if (// asc
        !desc)
            Arrays.sort(inData, new AscRowComparator(col - 1));
        else
            // desc
            Arrays.sort(inData, new DescRowComparator(col - 1));
        // create and copy output matrix
        String dir = createOutputFilePathAndName(OUTPUT_FILE);
        ret = new Matrix(dir, inM.getNumRows(), inM.getNumCols(), ValueType.Double);
        ret.setMatrixDoubleArray(inData);
    } catch (Exception e) {
        throw new RuntimeException("Error executing external order function", e);
    }
}
Also used : Matrix(org.apache.sysml.udf.Matrix)

Aggregations

Matrix (org.apache.sysml.udf.Matrix)33 MatrixBlock (org.apache.sysml.runtime.matrix.data.MatrixBlock)19 Scalar (org.apache.sysml.udf.Scalar)14 IOException (java.io.IOException)8 MatrixObject (org.apache.sysml.runtime.controlprogram.caching.MatrixObject)8 OutputInfo (org.apache.sysml.runtime.matrix.data.OutputInfo)6 DMLRuntimeException (org.apache.sysml.runtime.DMLRuntimeException)5 ArrayList (java.util.ArrayList)4 HashMap (java.util.HashMap)4 StringTokenizer (java.util.StringTokenizer)4 InputInfo (org.apache.sysml.runtime.matrix.data.InputInfo)4 MatrixCharacteristics (org.apache.sysml.runtime.matrix.MatrixCharacteristics)3 DataOutputStream (java.io.DataOutputStream)2 FileSystem (org.apache.hadoop.fs.FileSystem)2 Path (org.apache.hadoop.fs.Path)2 LongWritable (org.apache.hadoop.io.LongWritable)2 Text (org.apache.hadoop.io.Text)2 InputSplit (org.apache.hadoop.mapred.InputSplit)2 JobConf (org.apache.hadoop.mapred.JobConf)2 TextInputFormat (org.apache.hadoop.mapred.TextInputFormat)2