Search in sources :

Example 56 with Instruction

use of org.apache.sysml.runtime.instructions.Instruction in project incubator-systemml by apache.

the class ProgramConverter method rParseWhileProgramBlock.

private static WhileProgramBlock rParseWhileProgramBlock(String in, Program prog, int id) {
    String lin = in.substring(PARFOR_PB_WHILE.length(), in.length() - PARFOR_PB_END.length());
    HierarchyAwareStringTokenizer st = new HierarchyAwareStringTokenizer(lin, COMPONENTS_DELIM);
    // predicate instructions
    ArrayList<Instruction> inst = parseInstructions(st.nextToken(), id);
    // exit instructions
    ArrayList<Instruction> exit = parseInstructions(st.nextToken(), id);
    // program blocks
    ArrayList<ProgramBlock> pbs = rParseProgramBlocks(st.nextToken(), prog, id);
    WhileProgramBlock wpb = new WhileProgramBlock(prog, inst);
    wpb.setExitInstructions2(exit);
    wpb.setChildBlocks(pbs);
    return wpb;
}
Also used : FunctionProgramBlock(org.apache.sysml.runtime.controlprogram.FunctionProgramBlock) WhileProgramBlock(org.apache.sysml.runtime.controlprogram.WhileProgramBlock) ExternalFunctionProgramBlock(org.apache.sysml.runtime.controlprogram.ExternalFunctionProgramBlock) ForProgramBlock(org.apache.sysml.runtime.controlprogram.ForProgramBlock) IfProgramBlock(org.apache.sysml.runtime.controlprogram.IfProgramBlock) ProgramBlock(org.apache.sysml.runtime.controlprogram.ProgramBlock) ParForProgramBlock(org.apache.sysml.runtime.controlprogram.ParForProgramBlock) WhileProgramBlock(org.apache.sysml.runtime.controlprogram.WhileProgramBlock) GPUInstruction(org.apache.sysml.runtime.instructions.gpu.GPUInstruction) MRJobInstruction(org.apache.sysml.runtime.instructions.MRJobInstruction) CPInstruction(org.apache.sysml.runtime.instructions.cp.CPInstruction) ExternalFunctionInvocationInstruction(org.apache.sysml.udf.ExternalFunctionInvocationInstruction) SpoofCPInstruction(org.apache.sysml.runtime.instructions.cp.SpoofCPInstruction) Instruction(org.apache.sysml.runtime.instructions.Instruction) SPInstruction(org.apache.sysml.runtime.instructions.spark.SPInstruction) VariableCPInstruction(org.apache.sysml.runtime.instructions.cp.VariableCPInstruction) FunctionCallCPInstruction(org.apache.sysml.runtime.instructions.cp.FunctionCallCPInstruction) MRInstruction(org.apache.sysml.runtime.instructions.mr.MRInstruction)

Example 57 with Instruction

use of org.apache.sysml.runtime.instructions.Instruction in project incubator-systemml by apache.

the class OptimizerRuleBased method recompileLIX.

protected void recompileLIX(OptNode n, LocalVariableMap vars) {
    Hop h = OptTreeConverter.getAbstractPlanMapping().getMappedHop(n.getID());
    // set forced exec type
    h.setForcedExecType(LopProperties.ExecType.CP);
    n.setExecType(ExecType.CP);
    // recompile parent pb
    long pid = OptTreeConverter.getAbstractPlanMapping().getMappedParentID(n.getID());
    OptNode nParent = OptTreeConverter.getAbstractPlanMapping().getOptNode(pid);
    Object[] o = OptTreeConverter.getAbstractPlanMapping().getMappedProg(pid);
    StatementBlock sb = (StatementBlock) o[0];
    ProgramBlock pb = (ProgramBlock) o[1];
    // keep modified estimated of partitioned rix (in same dag as lix)
    HashMap<Hop, Double> estRix = getPartitionedRIXEstimates(nParent);
    // construct new instructions
    ArrayList<Instruction> newInst = Recompiler.recompileHopsDag(sb, sb.getHops(), vars, null, false, false, 0);
    pb.setInstructions(newInst);
    // reset all rix estimated (modified by recompile)
    resetPartitionRIXEstimates(estRix);
    // set new mem estimate (last, otherwise overwritten from recompile)
    h.setMemEstimate(_rm - 1);
}
Also used : Hop(org.apache.sysml.hops.Hop) MultiThreadedHop(org.apache.sysml.hops.Hop.MultiThreadedHop) MatrixObject(org.apache.sysml.runtime.controlprogram.caching.MatrixObject) RDDObject(org.apache.sysml.runtime.instructions.spark.data.RDDObject) FunctionProgramBlock(org.apache.sysml.runtime.controlprogram.FunctionProgramBlock) ForProgramBlock(org.apache.sysml.runtime.controlprogram.ForProgramBlock) ProgramBlock(org.apache.sysml.runtime.controlprogram.ProgramBlock) ParForProgramBlock(org.apache.sysml.runtime.controlprogram.ParForProgramBlock) Instruction(org.apache.sysml.runtime.instructions.Instruction) FunctionCallCPInstruction(org.apache.sysml.runtime.instructions.cp.FunctionCallCPInstruction) FunctionStatementBlock(org.apache.sysml.parser.FunctionStatementBlock) ParForStatementBlock(org.apache.sysml.parser.ParForStatementBlock) StatementBlock(org.apache.sysml.parser.StatementBlock)

Example 58 with Instruction

use of org.apache.sysml.runtime.instructions.Instruction in project incubator-systemml by apache.

the class ProgramRecompiler method createNestedParallelismToInstructionSet.

// /////
// additional general-purpose functionalities
protected static ArrayList<Instruction> createNestedParallelismToInstructionSet(String iterVar, String offset) {
    // create instruction string
    StringBuilder sb = new StringBuilder("CP" + Lop.OPERAND_DELIMITOR + "+" + Lop.OPERAND_DELIMITOR);
    sb.append(iterVar);
    sb.append(Lop.DATATYPE_PREFIX + "SCALAR" + Lop.VALUETYPE_PREFIX + "INT" + Lop.OPERAND_DELIMITOR);
    sb.append(offset);
    sb.append(Lop.DATATYPE_PREFIX + "SCALAR" + Lop.VALUETYPE_PREFIX + "INT" + Lop.OPERAND_DELIMITOR);
    sb.append(iterVar);
    sb.append(Lop.DATATYPE_PREFIX + "SCALAR" + Lop.VALUETYPE_PREFIX + "INT");
    String str = sb.toString();
    // create instruction set
    ArrayList<Instruction> tmp = new ArrayList<>();
    Instruction inst = BinaryCPInstruction.parseInstruction(str);
    tmp.add(inst);
    return tmp;
}
Also used : ArrayList(java.util.ArrayList) BinaryCPInstruction(org.apache.sysml.runtime.instructions.cp.BinaryCPInstruction) FunctionCallCPInstruction(org.apache.sysml.runtime.instructions.cp.FunctionCallCPInstruction) Instruction(org.apache.sysml.runtime.instructions.Instruction)

Example 59 with Instruction

use of org.apache.sysml.runtime.instructions.Instruction in project incubator-systemml by apache.

the class ExternalFunctionProgramBlock method execute.

/**
 * Method to be invoked to execute instructions for the external function
 * invocation
 */
@Override
public void execute(ExecutionContext ec) {
    _runID = _idSeq.getNextID();
    changeTmpInput(_runID, ec);
    changeTmpOutput(_runID);
    // export input variables to HDFS (see RunMRJobs)
    ArrayList<DataIdentifier> inputParams = null;
    try {
        inputParams = getInputParams();
        for (DataIdentifier di : inputParams) {
            Data d = ec.getVariable(di.getName());
            if (d.getDataType().isMatrix())
                ((MatrixObject) d).exportData();
        }
    } catch (Exception e) {
        throw new DMLRuntimeException(this.printBlockErrorLocation() + "Error exporting input variables to HDFS", e);
    }
    // convert block to cell
    if (block2CellInst != null) {
        ArrayList<Instruction> tempInst = new ArrayList<>();
        tempInst.addAll(block2CellInst);
        try {
            this.executeInstructions(tempInst, ec);
        } catch (Exception e) {
            throw new DMLRuntimeException(this.printBlockErrorLocation() + "Error executing " + tempInst.toString(), e);
        }
    }
    // now execute package function
    for (int i = 0; i < _inst.size(); i++) {
        try {
            if (_inst.get(i) instanceof ExternalFunctionInvocationInstruction)
                ((ExternalFunctionInvocationInstruction) _inst.get(i)).processInstruction(ec);
        } catch (Exception e) {
            throw new DMLRuntimeException(this.printBlockErrorLocation() + "Failed to execute instruction " + _inst.get(i).toString(), e);
        }
    }
    // convert cell to block
    if (cell2BlockInst != null) {
        ArrayList<Instruction> tempInst = new ArrayList<>();
        try {
            tempInst.clear();
            tempInst.addAll(cell2BlockInst);
            this.executeInstructions(tempInst, ec);
        } catch (Exception e) {
            throw new DMLRuntimeException(this.printBlockErrorLocation() + "Failed to execute instruction " + cell2BlockInst.toString(), e);
        }
    }
    // check return values
    checkOutputParameters(ec.getVariables());
}
Also used : ExternalFunctionInvocationInstruction(org.apache.sysml.udf.ExternalFunctionInvocationInstruction) DataIdentifier(org.apache.sysml.parser.DataIdentifier) ArrayList(java.util.ArrayList) Data(org.apache.sysml.runtime.instructions.cp.Data) MRJobInstruction(org.apache.sysml.runtime.instructions.MRJobInstruction) VariableCPInstruction(org.apache.sysml.runtime.instructions.cp.VariableCPInstruction) ExternalFunctionInvocationInstruction(org.apache.sysml.udf.ExternalFunctionInvocationInstruction) Instruction(org.apache.sysml.runtime.instructions.Instruction) DMLRuntimeException(org.apache.sysml.runtime.DMLRuntimeException) DMLRuntimeException(org.apache.sysml.runtime.DMLRuntimeException)

Example 60 with Instruction

use of org.apache.sysml.runtime.instructions.Instruction in project incubator-systemml by apache.

the class ExternalFunctionProgramBlock method createFunctionObject.

@SuppressWarnings("unchecked")
protected PackageFunction createFunctionObject(String className, String configFile) {
    try {
        // create instance of package function
        Class<Instruction> cla = (Class<Instruction>) Class.forName(className);
        Object o = cla.newInstance();
        if (!(o instanceof PackageFunction))
            throw new DMLRuntimeException(this.printBlockErrorLocation() + "Class is not of type PackageFunction");
        PackageFunction fun = (PackageFunction) o;
        // configure package function
        fun.setConfiguration(configFile);
        fun.setBaseDir(_baseDir);
        return fun;
    } catch (Exception e) {
        throw new DMLRuntimeException(this.printBlockErrorLocation() + "Error instantiating package function ", e);
    }
}
Also used : MatrixObject(org.apache.sysml.runtime.controlprogram.caching.MatrixObject) PackageFunction(org.apache.sysml.udf.PackageFunction) MRJobInstruction(org.apache.sysml.runtime.instructions.MRJobInstruction) VariableCPInstruction(org.apache.sysml.runtime.instructions.cp.VariableCPInstruction) ExternalFunctionInvocationInstruction(org.apache.sysml.udf.ExternalFunctionInvocationInstruction) Instruction(org.apache.sysml.runtime.instructions.Instruction) DMLRuntimeException(org.apache.sysml.runtime.DMLRuntimeException) DMLRuntimeException(org.apache.sysml.runtime.DMLRuntimeException)

Aggregations

Instruction (org.apache.sysml.runtime.instructions.Instruction)73 MRJobInstruction (org.apache.sysml.runtime.instructions.MRJobInstruction)50 FunctionCallCPInstruction (org.apache.sysml.runtime.instructions.cp.FunctionCallCPInstruction)35 VariableCPInstruction (org.apache.sysml.runtime.instructions.cp.VariableCPInstruction)31 CPInstruction (org.apache.sysml.runtime.instructions.cp.CPInstruction)28 ArrayList (java.util.ArrayList)21 ForProgramBlock (org.apache.sysml.runtime.controlprogram.ForProgramBlock)19 FunctionProgramBlock (org.apache.sysml.runtime.controlprogram.FunctionProgramBlock)18 IfProgramBlock (org.apache.sysml.runtime.controlprogram.IfProgramBlock)18 ProgramBlock (org.apache.sysml.runtime.controlprogram.ProgramBlock)18 WhileProgramBlock (org.apache.sysml.runtime.controlprogram.WhileProgramBlock)18 ExternalFunctionInvocationInstruction (org.apache.sysml.udf.ExternalFunctionInvocationInstruction)18 DMLRuntimeException (org.apache.sysml.runtime.DMLRuntimeException)17 SPInstruction (org.apache.sysml.runtime.instructions.spark.SPInstruction)16 Lop (org.apache.sysml.lops.Lop)15 MRInstruction (org.apache.sysml.runtime.instructions.mr.MRInstruction)15 GPUInstruction (org.apache.sysml.runtime.instructions.gpu.GPUInstruction)14 RandInstruction (org.apache.sysml.runtime.instructions.mr.RandInstruction)14 SeqInstruction (org.apache.sysml.runtime.instructions.mr.SeqInstruction)14 ParForProgramBlock (org.apache.sysml.runtime.controlprogram.ParForProgramBlock)13