Search in sources :

Example 16 with CMOperator

use of org.apache.sysml.runtime.matrix.operators.CMOperator in project incubator-systemml by apache.

the class CMCOVMRMapper method configure.

@Override
public void configure(JobConf job) {
    super.configure(job);
    try {
        CM_N_COVInstruction[] cmIns = MRJobConfiguration.getCM_N_COVInstructions(job);
        for (CM_N_COVInstruction ins : cmIns) {
            if (ins.getOperator() instanceof COVOperator)
                covTags.add(ins.input);
            else // CMOperator
            {
                cmTags.add(ins.input);
                cmFn.put(ins.input, CM.getCMFnObject(((CMOperator) ins.getOperator()).getAggOpType()));
            }
        }
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
    for (byte tag : cmTags) {
        cmNcovCache.holdPlace(tag, CM_N_COVCell.class);
    }
    for (byte tag : covTags) {
        cmNcovCache.holdPlace(tag, CM_N_COVCell.class);
    }
}
Also used : CM_N_COVInstruction(org.apache.sysml.runtime.instructions.mr.CM_N_COVInstruction) DMLRuntimeException(org.apache.sysml.runtime.DMLRuntimeException) COVOperator(org.apache.sysml.runtime.matrix.operators.COVOperator) CMOperator(org.apache.sysml.runtime.matrix.operators.CMOperator) DMLRuntimeException(org.apache.sysml.runtime.DMLRuntimeException) IOException(java.io.IOException)

Example 17 with CMOperator

use of org.apache.sysml.runtime.matrix.operators.CMOperator in project incubator-systemml by apache.

the class GroupedAggMRReducer method configure.

@Override
public void configure(JobConf job) {
    super.configure(job);
    try {
        GroupedAggregateInstruction[] grpaggIns = MRJobConfiguration.getGroupedAggregateInstructions(job);
        if (grpaggIns == null)
            throw new RuntimeException("no GroupAggregate Instructions found!");
        for (GroupedAggregateInstruction ins : grpaggIns) {
            grpaggInstructions.put(ins.output, ins);
            if (ins.getOperator() instanceof CMOperator)
                cmFn.put(ins.output, CM.getCMFnObject(((CMOperator) ins.getOperator()).getAggOpType()));
            outputIndexesMapping.put(ins.output, getOutputIndexes(ins.output));
        }
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}
Also used : CMOperator(org.apache.sysml.runtime.matrix.operators.CMOperator) GroupedAggregateInstruction(org.apache.sysml.runtime.instructions.mr.GroupedAggregateInstruction) IOException(java.io.IOException)

Aggregations

CMOperator (org.apache.sysml.runtime.matrix.operators.CMOperator)17 DMLRuntimeException (org.apache.sysml.runtime.DMLRuntimeException)11 KahanObject (org.apache.sysml.runtime.instructions.cp.KahanObject)7 IOException (java.io.IOException)6 AggregateOperator (org.apache.sysml.runtime.matrix.operators.AggregateOperator)6 CM_COV_Object (org.apache.sysml.runtime.instructions.cp.CM_COV_Object)5 CM (org.apache.sysml.runtime.functionobjects.CM)4 MatrixBlock (org.apache.sysml.runtime.matrix.data.MatrixBlock)4 WeightedCell (org.apache.sysml.runtime.matrix.data.WeightedCell)4 AggregateOperationTypes (org.apache.sysml.runtime.matrix.operators.CMOperator.AggregateOperationTypes)4 CPOperand (org.apache.sysml.runtime.instructions.cp.CPOperand)3 GroupedAggregateInstruction (org.apache.sysml.runtime.instructions.mr.GroupedAggregateInstruction)3 SparkExecutionContext (org.apache.sysml.runtime.controlprogram.context.SparkExecutionContext)2 MatrixIndexes (org.apache.sysml.runtime.matrix.data.MatrixIndexes)2 COVOperator (org.apache.sysml.runtime.matrix.operators.COVOperator)2 Operator (org.apache.sysml.runtime.matrix.operators.Operator)2 ArrayList (java.util.ArrayList)1 HashMap (java.util.HashMap)1 ExecutorService (java.util.concurrent.ExecutorService)1 Future (java.util.concurrent.Future)1