Search in sources :

Example 91 with Lop

use of org.apache.sysml.lops.Lop in project incubator-systemml by apache.

the class Recompiler method recompileHopsDagInstructions.

public static ArrayList<Instruction> recompileHopsDagInstructions(Hop hops) throws DMLRuntimeException, HopsException, LopsException, IOException {
    ArrayList<Instruction> newInst = null;
    //need for synchronization as we do temp changes in shared hops/lops
    synchronized (hops) {
        LOG.debug("\n**************** Optimizer (Recompile) *************\nMemory Budget = " + OptimizerUtils.toMB(OptimizerUtils.getLocalMemBudget()) + " MB");
        // clear existing lops
        hops.resetVisitStatus();
        rClearLops(hops);
        // construct lops			
        Dag<Lop> dag = new Dag<Lop>();
        Lop lops = hops.constructLops();
        lops.addToDag(dag);
        // generate runtime instructions (incl piggybacking)
        newInst = dag.getJobs(null, ConfigurationManager.getDMLConfig());
    }
    // explain recompiled instructions
    if (DMLScript.EXPLAIN == ExplainType.RECOMPILE_HOPS)
        LOG.info("EXPLAIN RECOMPILE \nPRED (line " + hops.getBeginLine() + "):\n" + Explain.explain(hops, 1));
    if (DMLScript.EXPLAIN == ExplainType.RECOMPILE_RUNTIME)
        LOG.info("EXPLAIN RECOMPILE \nPRED (line " + hops.getBeginLine() + "):\n" + Explain.explain(newInst, 1));
    return newInst;
}
Also used : Dag(org.apache.sysml.lops.compile.Dag) MRJobInstruction(org.apache.sysml.runtime.instructions.MRJobInstruction) Instruction(org.apache.sysml.runtime.instructions.Instruction) FunctionCallCPInstruction(org.apache.sysml.runtime.instructions.cp.FunctionCallCPInstruction) SeqInstruction(org.apache.sysml.runtime.instructions.mr.SeqInstruction) RandInstruction(org.apache.sysml.runtime.instructions.mr.RandInstruction) Lop(org.apache.sysml.lops.Lop)

Example 92 with Lop

use of org.apache.sysml.lops.Lop in project incubator-systemml by apache.

the class Recompiler method recompileHopsDagInstructions.

public static ArrayList<Instruction> recompileHopsDagInstructions(StatementBlock sb, ArrayList<Hop> hops) throws HopsException, LopsException, DMLRuntimeException, IOException {
    ArrayList<Instruction> newInst = null;
    //however, we create deep copies for most dags to allow for concurrent recompile
    synchronized (hops) {
        LOG.debug("\n**************** Optimizer (Recompile) *************\nMemory Budget = " + OptimizerUtils.toMB(OptimizerUtils.getLocalMemBudget()) + " MB");
        // clear existing lops
        Hop.resetVisitStatus(hops);
        for (Hop hopRoot : hops) rClearLops(hopRoot);
        // construct lops			
        Dag<Lop> dag = new Dag<Lop>();
        for (Hop hopRoot : hops) {
            Lop lops = hopRoot.constructLops();
            lops.addToDag(dag);
        }
        // generate runtime instructions (incl piggybacking)
        newInst = dag.getJobs(sb, ConfigurationManager.getDMLConfig());
    }
    // explain recompiled hops / instructions
    if (DMLScript.EXPLAIN == ExplainType.RECOMPILE_HOPS) {
        LOG.info("EXPLAIN RECOMPILE \nGENERIC (lines " + sb.getBeginLine() + "-" + sb.getEndLine() + "):\n" + Explain.explainHops(hops, 1));
    }
    if (DMLScript.EXPLAIN == ExplainType.RECOMPILE_RUNTIME) {
        LOG.info("EXPLAIN RECOMPILE \nGENERIC (lines " + sb.getBeginLine() + "-" + sb.getEndLine() + "):\n" + Explain.explain(newInst, 1));
    }
    return newInst;
}
Also used : Hop(org.apache.sysml.hops.Hop) Dag(org.apache.sysml.lops.compile.Dag) MRJobInstruction(org.apache.sysml.runtime.instructions.MRJobInstruction) Instruction(org.apache.sysml.runtime.instructions.Instruction) FunctionCallCPInstruction(org.apache.sysml.runtime.instructions.cp.FunctionCallCPInstruction) SeqInstruction(org.apache.sysml.runtime.instructions.mr.SeqInstruction) RandInstruction(org.apache.sysml.runtime.instructions.mr.RandInstruction) Lop(org.apache.sysml.lops.Lop)

Aggregations

Lop (org.apache.sysml.lops.Lop)92 MultiThreadedHop (org.apache.sysml.hops.Hop.MultiThreadedHop)34 ExecType (org.apache.sysml.lops.LopProperties.ExecType)27 Group (org.apache.sysml.lops.Group)23 ArrayList (java.util.ArrayList)18 Aggregate (org.apache.sysml.lops.Aggregate)16 LopsException (org.apache.sysml.lops.LopsException)16 DataPartition (org.apache.sysml.lops.DataPartition)15 Instruction (org.apache.sysml.runtime.instructions.Instruction)15 Data (org.apache.sysml.lops.Data)12 MRJobInstruction (org.apache.sysml.runtime.instructions.MRJobInstruction)12 Dag (org.apache.sysml.lops.compile.Dag)10 Transform (org.apache.sysml.lops.Transform)8 Unary (org.apache.sysml.lops.Unary)8 HashMap (java.util.HashMap)7 Hop (org.apache.sysml.hops.Hop)7 UnaryCP (org.apache.sysml.lops.UnaryCP)7 FunctionCallCPInstruction (org.apache.sysml.runtime.instructions.cp.FunctionCallCPInstruction)7 RandInstruction (org.apache.sysml.runtime.instructions.mr.RandInstruction)7 SeqInstruction (org.apache.sysml.runtime.instructions.mr.SeqInstruction)7