Search in sources :

Example 76 with MatrixObject

use of org.apache.sysml.runtime.controlprogram.caching.MatrixObject in project incubator-systemml by apache.

the class ProgramBlock method checkSparsity.

private static void checkSparsity(Instruction lastInst, LocalVariableMap vars) {
    for (String varname : vars.keySet()) {
        Data dat = vars.get(varname);
        if (dat instanceof MatrixObject) {
            MatrixObject mo = (MatrixObject) dat;
            if (mo.isDirty() && !mo.isPartitioned()) {
                MatrixBlock mb = mo.acquireRead();
                boolean sparse1 = mb.isInSparseFormat();
                long nnz1 = mb.getNonZeros();
                synchronized (mb) {
                    // potential state change
                    mb.recomputeNonZeros();
                    mb.examSparsity();
                }
                if (mb.isInSparseFormat() && mb.isAllocated()) {
                    mb.getSparseBlock().checkValidity(mb.getNumRows(), mb.getNumColumns(), mb.getNonZeros(), true);
                }
                boolean sparse2 = mb.isInSparseFormat();
                long nnz2 = mb.getNonZeros();
                mo.release();
                if (nnz1 != nnz2)
                    throw new DMLRuntimeException("Matrix nnz meta data was incorrect: (" + varname + ", actual=" + nnz1 + ", expected=" + nnz2 + ", inst=" + lastInst + ")");
                if (sparse1 != sparse2 && mb.isAllocated())
                    throw new DMLRuntimeException("Matrix was in wrong data representation: (" + varname + ", actual=" + sparse1 + ", expected=" + sparse2 + ", nrow=" + mb.getNumRows() + ", ncol=" + mb.getNumColumns() + ", nnz=" + nnz1 + ", inst=" + lastInst + ")");
            }
        }
    }
}
Also used : MatrixBlock(org.apache.sysml.runtime.matrix.data.MatrixBlock) MatrixObject(org.apache.sysml.runtime.controlprogram.caching.MatrixObject) Data(org.apache.sysml.runtime.instructions.cp.Data) DMLRuntimeException(org.apache.sysml.runtime.DMLRuntimeException)

Example 77 with MatrixObject

use of org.apache.sysml.runtime.controlprogram.caching.MatrixObject in project incubator-systemml by apache.

the class ExecutionContext method setMatrixOutput.

public void setMatrixOutput(String varName, MatrixBlock outputData, UpdateType flag, String opcode) {
    if (flag.isInPlace()) {
        // modify metadata to carry update status
        MatrixObject mo = getMatrixObject(varName);
        mo.setUpdateType(flag);
    }
    // default case
    setMatrixOutput(varName, outputData, opcode);
}
Also used : MatrixObject(org.apache.sysml.runtime.controlprogram.caching.MatrixObject)

Example 78 with MatrixObject

use of org.apache.sysml.runtime.controlprogram.caching.MatrixObject in project incubator-systemml by apache.

the class ExecutionContext method releaseMatrixOutputForGPUInstruction.

public void releaseMatrixOutputForGPUInstruction(String varName) {
    MatrixObject mo = getMatrixObject(varName);
    if (mo.getGPUObject(getGPUContext(0)) == null || !mo.getGPUObject(getGPUContext(0)).isAllocated()) {
        throw new DMLRuntimeException("No output is allocated on GPU");
    }
    mo.getGPUObject(getGPUContext(0)).releaseOutput();
}
Also used : MatrixObject(org.apache.sysml.runtime.controlprogram.caching.MatrixObject) DMLRuntimeException(org.apache.sysml.runtime.DMLRuntimeException)

Example 79 with MatrixObject

use of org.apache.sysml.runtime.controlprogram.caching.MatrixObject in project incubator-systemml by apache.

the class ExecutionContext method getSparseMatrixOutputForGPUInstruction.

/**
 * Allocates a sparse matrix in CSR format on the GPU.
 * Assumes that mat.getNumRows() returns a valid number
 *
 * @param varName variable name
 * @param numRows number of rows of matrix object
 * @param numCols number of columns of matrix object
 * @param nnz number of non zeroes
 * @return matrix object
 */
public Pair<MatrixObject, Boolean> getSparseMatrixOutputForGPUInstruction(String varName, long numRows, long numCols, long nnz) {
    MatrixObject mo = allocateGPUMatrixObject(varName, numRows, numCols);
    mo.getMatrixCharacteristics().setNonZeros(nnz);
    boolean allocated = mo.getGPUObject(getGPUContext(0)).acquireDeviceModifySparse();
    return new Pair<>(mo, allocated);
}
Also used : MatrixObject(org.apache.sysml.runtime.controlprogram.caching.MatrixObject) Pair(org.apache.sysml.runtime.matrix.data.Pair)

Example 80 with MatrixObject

use of org.apache.sysml.runtime.controlprogram.caching.MatrixObject in project incubator-systemml by apache.

the class ExecutionContext method setMetaData.

public void setMetaData(String varName, long nrows, long ncols) {
    MatrixObject mo = getMatrixObject(varName);
    if (mo.getNumRows() == nrows && mo.getNumColumns() == ncols)
        return;
    MetaData oldMetaData = mo.getMetaData();
    if (oldMetaData == null || !(oldMetaData instanceof MetaDataFormat))
        throw new DMLRuntimeException("Metadata not available");
    MatrixCharacteristics mc = new MatrixCharacteristics(nrows, ncols, (int) mo.getNumRowsPerBlock(), (int) mo.getNumColumnsPerBlock());
    mo.setMetaData(new MetaDataFormat(mc, ((MetaDataFormat) oldMetaData).getOutputInfo(), ((MetaDataFormat) oldMetaData).getInputInfo()));
}
Also used : MetaDataFormat(org.apache.sysml.runtime.matrix.MetaDataFormat) MatrixObject(org.apache.sysml.runtime.controlprogram.caching.MatrixObject) MetaData(org.apache.sysml.runtime.matrix.MetaData) DMLRuntimeException(org.apache.sysml.runtime.DMLRuntimeException) MatrixCharacteristics(org.apache.sysml.runtime.matrix.MatrixCharacteristics)

Aggregations

MatrixObject (org.apache.sysml.runtime.controlprogram.caching.MatrixObject)201 DMLRuntimeException (org.apache.sysml.runtime.DMLRuntimeException)74 MatrixCharacteristics (org.apache.sysml.runtime.matrix.MatrixCharacteristics)45 MatrixBlock (org.apache.sysml.runtime.matrix.data.MatrixBlock)39 Data (org.apache.sysml.runtime.instructions.cp.Data)37 MetaDataFormat (org.apache.sysml.runtime.matrix.MetaDataFormat)26 Pointer (jcuda.Pointer)20 CSRPointer (org.apache.sysml.runtime.instructions.gpu.context.CSRPointer)20 IOException (java.io.IOException)17 ArrayList (java.util.ArrayList)16 ScalarObject (org.apache.sysml.runtime.instructions.cp.ScalarObject)14 OutputInfo (org.apache.sysml.runtime.matrix.data.OutputInfo)13 CacheableData (org.apache.sysml.runtime.controlprogram.caching.CacheableData)12 RDDObject (org.apache.sysml.runtime.instructions.spark.data.RDDObject)12 Hop (org.apache.sysml.hops.Hop)11 MatrixFormatMetaData (org.apache.sysml.runtime.matrix.MatrixFormatMetaData)11 ParForProgramBlock (org.apache.sysml.runtime.controlprogram.ParForProgramBlock)10 MatrixIndexes (org.apache.sysml.runtime.matrix.data.MatrixIndexes)10 Path (org.apache.hadoop.fs.Path)9 LongWritable (org.apache.hadoop.io.LongWritable)9