Examples with RightIndex org.apache.sysml.lops.RightIndex used on opensource projects

Search in sources :

Example 1 with RightIndex

use of org.apache.sysml.lops.RightIndex in project incubator-systemml by apache.

the class IndexingOp method constructLops.

@Override
public Lop constructLops() {
    // return already created lops
    if (getLops() != null)
        return getLops();
    Hop input = getInput().get(0);
    // rewrite remove unnecessary right indexing
    if (HopRewriteUtils.isUnnecessaryRightIndexing(this)) {
        setLops(input.constructLops());
    } else // actual lop construction, incl operator selection
    {
        try {
            ExecType et = optFindExecType();
            if (et == ExecType.MR) {
                IndexingMethod method = optFindIndexingMethod(_rowLowerEqualsUpper, _colLowerEqualsUpper, input._dim1, input._dim2, _dim1, _dim2);
                Lop dummy = Data.createLiteralLop(ValueType.INT, Integer.toString(-1));
                RightIndex reindex = new RightIndex(input.constructLops(), getInput().get(1).constructLops(), getInput().get(2).constructLops(), getInput().get(3).constructLops(), getInput().get(4).constructLops(), dummy, dummy, getDataType(), getValueType(), et);
                setOutputDimensions(reindex);
                setLineNumbers(reindex);
                if (method == IndexingMethod.MR_RIX) {
                    Group group1 = new Group(reindex, Group.OperationTypes.Sort, DataType.MATRIX, getValueType());
                    setOutputDimensions(group1);
                    setLineNumbers(group1);
                    Aggregate agg1 = new Aggregate(group1, Aggregate.OperationTypes.Sum, DataType.MATRIX, getValueType(), et);
                    setOutputDimensions(agg1);
                    setLineNumbers(agg1);
                    setLops(agg1);
                } else // method == IndexingMethod.MR_VRIX
                {
                    setLops(reindex);
                }
            } else if (et == ExecType.SPARK) {
                IndexingMethod method = optFindIndexingMethod(_rowLowerEqualsUpper, _colLowerEqualsUpper, input._dim1, input._dim2, _dim1, _dim2);
                SparkAggType aggtype = (method == IndexingMethod.MR_VRIX || isBlockAligned()) ? SparkAggType.NONE : SparkAggType.MULTI_BLOCK;
                Lop dummy = Data.createLiteralLop(ValueType.INT, Integer.toString(-1));
                RightIndex reindex = new RightIndex(input.constructLops(), getInput().get(1).constructLops(), getInput().get(2).constructLops(), getInput().get(3).constructLops(), getInput().get(4).constructLops(), dummy, dummy, getDataType(), getValueType(), aggtype, et);
                setOutputDimensions(reindex);
                setLineNumbers(reindex);
                setLops(reindex);
            } else // CP or GPU
            {
                Lop dummy = Data.createLiteralLop(ValueType.INT, Integer.toString(-1));
                RightIndex reindex = new RightIndex(input.constructLops(), getInput().get(1).constructLops(), getInput().get(2).constructLops(), getInput().get(3).constructLops(), getInput().get(4).constructLops(), dummy, dummy, getDataType(), getValueType(), et);
                setOutputDimensions(reindex);
                setLineNumbers(reindex);
                setLops(reindex);
            }
        } catch (Exception e) {
            throw new HopsException(this.printErrorLocation() + "In IndexingOp Hop, error constructing Lops ", e);
        }
    }
    // add reblock/checkpoint lops if necessary
    constructAndSetLopsDataFlowProperties();
    return getLops();
}
Also used : Group(org.apache.sysml.lops.Group) SparkAggType(org.apache.sysml.hops.AggBinaryOp.SparkAggType) RightIndex(org.apache.sysml.lops.RightIndex) ExecType(org.apache.sysml.lops.LopProperties.ExecType) Lop(org.apache.sysml.lops.Lop) Aggregate(org.apache.sysml.lops.Aggregate)

Example 2 with RightIndex

use of org.apache.sysml.lops.RightIndex in project systemml by apache.

the class IndexingOp method constructLops.

@Override
public Lop constructLops() {
    // return already created lops
    if (getLops() != null)
        return getLops();
    Hop input = getInput().get(0);
    // rewrite remove unnecessary right indexing
    if (HopRewriteUtils.isUnnecessaryRightIndexing(this)) {
        setLops(input.constructLops());
    } else // actual lop construction, incl operator selection
    {
        try {
            ExecType et = optFindExecType();
            if (et == ExecType.MR) {
                IndexingMethod method = optFindIndexingMethod(_rowLowerEqualsUpper, _colLowerEqualsUpper, input._dim1, input._dim2, _dim1, _dim2);
                Lop dummy = Data.createLiteralLop(ValueType.INT, Integer.toString(-1));
                RightIndex reindex = new RightIndex(input.constructLops(), getInput().get(1).constructLops(), getInput().get(2).constructLops(), getInput().get(3).constructLops(), getInput().get(4).constructLops(), dummy, dummy, getDataType(), getValueType(), et);
                setOutputDimensions(reindex);
                setLineNumbers(reindex);
                if (method == IndexingMethod.MR_RIX) {
                    Group group1 = new Group(reindex, Group.OperationTypes.Sort, DataType.MATRIX, getValueType());
                    setOutputDimensions(group1);
                    setLineNumbers(group1);
                    Aggregate agg1 = new Aggregate(group1, Aggregate.OperationTypes.Sum, DataType.MATRIX, getValueType(), et);
                    setOutputDimensions(agg1);
                    setLineNumbers(agg1);
                    setLops(agg1);
                } else // method == IndexingMethod.MR_VRIX
                {
                    setLops(reindex);
                }
            } else if (et == ExecType.SPARK) {
                IndexingMethod method = optFindIndexingMethod(_rowLowerEqualsUpper, _colLowerEqualsUpper, input._dim1, input._dim2, _dim1, _dim2);
                SparkAggType aggtype = (method == IndexingMethod.MR_VRIX || isBlockAligned()) ? SparkAggType.NONE : SparkAggType.MULTI_BLOCK;
                Lop dummy = Data.createLiteralLop(ValueType.INT, Integer.toString(-1));
                RightIndex reindex = new RightIndex(input.constructLops(), getInput().get(1).constructLops(), getInput().get(2).constructLops(), getInput().get(3).constructLops(), getInput().get(4).constructLops(), dummy, dummy, getDataType(), getValueType(), aggtype, et);
                setOutputDimensions(reindex);
                setLineNumbers(reindex);
                setLops(reindex);
            } else // CP or GPU
            {
                Lop dummy = Data.createLiteralLop(ValueType.INT, Integer.toString(-1));
                RightIndex reindex = new RightIndex(input.constructLops(), getInput().get(1).constructLops(), getInput().get(2).constructLops(), getInput().get(3).constructLops(), getInput().get(4).constructLops(), dummy, dummy, getDataType(), getValueType(), et);
                setOutputDimensions(reindex);
                setLineNumbers(reindex);
                setLops(reindex);
            }
        } catch (Exception e) {
            throw new HopsException(this.printErrorLocation() + "In IndexingOp Hop, error constructing Lops ", e);
        }
    }
    // add reblock/checkpoint lops if necessary
    constructAndSetLopsDataFlowProperties();
    return getLops();
}
Also used : Group(org.apache.sysml.lops.Group) SparkAggType(org.apache.sysml.hops.AggBinaryOp.SparkAggType) RightIndex(org.apache.sysml.lops.RightIndex) ExecType(org.apache.sysml.lops.LopProperties.ExecType) Lop(org.apache.sysml.lops.Lop) Aggregate(org.apache.sysml.lops.Aggregate)

Example 3 with RightIndex

use of org.apache.sysml.lops.RightIndex in project systemml by apache.

the class LeftIndexingOp method constructLops.

@Override
public Lop constructLops() {
    // return already created lops
    if (getLops() != null)
        return getLops();
    try {
        ExecType et = optFindExecType();
        if (et == ExecType.MR) {
            // the right matrix is reindexed
            Lop top = getInput().get(2).constructLops();
            Lop bottom = getInput().get(3).constructLops();
            Lop left = getInput().get(4).constructLops();
            Lop right = getInput().get(5).constructLops();
            // right hand matrix
            Lop nrow = new UnaryCP(getInput().get(0).constructLops(), OperationTypes.NROW, DataType.SCALAR, ValueType.INT);
            Lop ncol = new UnaryCP(getInput().get(0).constructLops(), OperationTypes.NCOL, DataType.SCALAR, ValueType.INT);
            Lop rightInput = null;
            if (isRightHandSideScalar()) {
                // insert cast to matrix if necessary (for reuse MR runtime)
                rightInput = new UnaryCP(getInput().get(1).constructLops(), OperationTypes.CAST_AS_MATRIX, DataType.MATRIX, ValueType.DOUBLE);
                rightInput.getOutputParameters().setDimensions(1L, 1L, ConfigurationManager.getBlocksize(), ConfigurationManager.getBlocksize(), -1L);
            } else
                rightInput = getInput().get(1).constructLops();
            RightIndex reindex = new RightIndex(rightInput, top, bottom, left, right, nrow, ncol, getDataType(), getValueType(), et, true);
            reindex.getOutputParameters().setDimensions(getInput().get(0).getDim1(), getInput().get(0).getDim2(), getRowsInBlock(), getColsInBlock(), getNnz());
            setLineNumbers(reindex);
            Group group1 = new Group(reindex, Group.OperationTypes.Sort, DataType.MATRIX, getValueType());
            group1.getOutputParameters().setDimensions(getInput().get(0).getDim1(), getInput().get(0).getDim2(), getRowsInBlock(), getColsInBlock(), getNnz());
            setLineNumbers(group1);
            // the left matrix is zeroed out
            ZeroOut zeroout = new ZeroOut(getInput().get(0).constructLops(), top, bottom, left, right, getInput().get(0).getDim1(), getInput().get(0).getDim2(), getDataType(), getValueType(), et);
            zeroout.getOutputParameters().setDimensions(getInput().get(0).getDim1(), getInput().get(0).getDim2(), getRowsInBlock(), getColsInBlock(), getNnz());
            setLineNumbers(zeroout);
            Group group2 = new Group(zeroout, Group.OperationTypes.Sort, DataType.MATRIX, getValueType());
            group2.getOutputParameters().setDimensions(getInput().get(0).getDim1(), getInput().get(0).getDim2(), getRowsInBlock(), getColsInBlock(), getNnz());
            setLineNumbers(group2);
            Binary binary = new Binary(group1, group2, HopsOpOp2LopsB.get(Hop.OpOp2.PLUS), getDataType(), getValueType(), et);
            binary.getOutputParameters().setDimensions(getInput().get(0).getDim1(), getInput().get(0).getDim2(), getRowsInBlock(), getColsInBlock(), getNnz());
            setLineNumbers(binary);
            setLops(binary);
        } else if (et == ExecType.SPARK) {
            Hop left = getInput().get(0);
            Hop right = getInput().get(1);
            LeftIndexingMethod method = getOptMethodLeftIndexingMethod(left.getDim1(), left.getDim2(), left.getRowsInBlock(), left.getColsInBlock(), left.getNnz(), right.getDim1(), right.getDim2(), right.getNnz(), right.getDataType());
            // insert cast to matrix if necessary (for reuse broadcast runtime)
            Lop rightInput = right.constructLops();
            if (isRightHandSideScalar()) {
                rightInput = new UnaryCP(rightInput, (left.getDataType() == DataType.MATRIX ? OperationTypes.CAST_AS_MATRIX : OperationTypes.CAST_AS_FRAME), left.getDataType(), right.getValueType());
                long bsize = ConfigurationManager.getBlocksize();
                rightInput.getOutputParameters().setDimensions(1, 1, bsize, bsize, -1);
            }
            LeftIndex leftIndexLop = new LeftIndex(left.constructLops(), rightInput, getInput().get(2).constructLops(), getInput().get(3).constructLops(), getInput().get(4).constructLops(), getInput().get(5).constructLops(), getDataType(), getValueType(), et, getSpLixCacheType(method));
            setOutputDimensions(leftIndexLop);
            setLineNumbers(leftIndexLop);
            setLops(leftIndexLop);
        } else {
            LeftIndex left = new LeftIndex(getInput().get(0).constructLops(), getInput().get(1).constructLops(), getInput().get(2).constructLops(), getInput().get(3).constructLops(), getInput().get(4).constructLops(), getInput().get(5).constructLops(), getDataType(), getValueType(), et);
            setOutputDimensions(left);
            setLineNumbers(left);
            setLops(left);
        }
    } catch (Exception e) {
        throw new HopsException(this.printErrorLocation() + "In LeftIndexingOp Hop, error in constructing Lops ", e);
    }
    // add reblock/checkpoint lops if necessary
    constructAndSetLopsDataFlowProperties();
    return getLops();
}
Also used : Group(org.apache.sysml.lops.Group) ZeroOut(org.apache.sysml.lops.ZeroOut) RightIndex(org.apache.sysml.lops.RightIndex) LeftIndex(org.apache.sysml.lops.LeftIndex) ExecType(org.apache.sysml.lops.LopProperties.ExecType) Binary(org.apache.sysml.lops.Binary) Lop(org.apache.sysml.lops.Lop) UnaryCP(org.apache.sysml.lops.UnaryCP)

Example 4 with RightIndex

use of org.apache.sysml.lops.RightIndex in project incubator-systemml by apache.

the class LeftIndexingOp method constructLops.

@Override
public Lop constructLops() {
    // return already created lops
    if (getLops() != null)
        return getLops();
    try {
        ExecType et = optFindExecType();
        if (et == ExecType.MR) {
            // the right matrix is reindexed
            Lop top = getInput().get(2).constructLops();
            Lop bottom = getInput().get(3).constructLops();
            Lop left = getInput().get(4).constructLops();
            Lop right = getInput().get(5).constructLops();
            // right hand matrix
            Lop nrow = new UnaryCP(getInput().get(0).constructLops(), OperationTypes.NROW, DataType.SCALAR, ValueType.INT);
            Lop ncol = new UnaryCP(getInput().get(0).constructLops(), OperationTypes.NCOL, DataType.SCALAR, ValueType.INT);
            Lop rightInput = null;
            if (isRightHandSideScalar()) {
                // insert cast to matrix if necessary (for reuse MR runtime)
                rightInput = new UnaryCP(getInput().get(1).constructLops(), OperationTypes.CAST_AS_MATRIX, DataType.MATRIX, ValueType.DOUBLE);
                rightInput.getOutputParameters().setDimensions(1L, 1L, ConfigurationManager.getBlocksize(), ConfigurationManager.getBlocksize(), -1L);
            } else
                rightInput = getInput().get(1).constructLops();
            RightIndex reindex = new RightIndex(rightInput, top, bottom, left, right, nrow, ncol, getDataType(), getValueType(), et, true);
            reindex.getOutputParameters().setDimensions(getInput().get(0).getDim1(), getInput().get(0).getDim2(), getRowsInBlock(), getColsInBlock(), getNnz());
            setLineNumbers(reindex);
            Group group1 = new Group(reindex, Group.OperationTypes.Sort, DataType.MATRIX, getValueType());
            group1.getOutputParameters().setDimensions(getInput().get(0).getDim1(), getInput().get(0).getDim2(), getRowsInBlock(), getColsInBlock(), getNnz());
            setLineNumbers(group1);
            // the left matrix is zeroed out
            ZeroOut zeroout = new ZeroOut(getInput().get(0).constructLops(), top, bottom, left, right, getInput().get(0).getDim1(), getInput().get(0).getDim2(), getDataType(), getValueType(), et);
            zeroout.getOutputParameters().setDimensions(getInput().get(0).getDim1(), getInput().get(0).getDim2(), getRowsInBlock(), getColsInBlock(), getNnz());
            setLineNumbers(zeroout);
            Group group2 = new Group(zeroout, Group.OperationTypes.Sort, DataType.MATRIX, getValueType());
            group2.getOutputParameters().setDimensions(getInput().get(0).getDim1(), getInput().get(0).getDim2(), getRowsInBlock(), getColsInBlock(), getNnz());
            setLineNumbers(group2);
            Binary binary = new Binary(group1, group2, HopsOpOp2LopsB.get(Hop.OpOp2.PLUS), getDataType(), getValueType(), et);
            binary.getOutputParameters().setDimensions(getInput().get(0).getDim1(), getInput().get(0).getDim2(), getRowsInBlock(), getColsInBlock(), getNnz());
            setLineNumbers(binary);
            setLops(binary);
        } else if (et == ExecType.SPARK) {
            Hop left = getInput().get(0);
            Hop right = getInput().get(1);
            LeftIndexingMethod method = getOptMethodLeftIndexingMethod(left.getDim1(), left.getDim2(), left.getRowsInBlock(), left.getColsInBlock(), left.getNnz(), right.getDim1(), right.getDim2(), right.getNnz(), right.getDataType());
            // insert cast to matrix if necessary (for reuse broadcast runtime)
            Lop rightInput = right.constructLops();
            if (isRightHandSideScalar()) {
                rightInput = new UnaryCP(rightInput, (left.getDataType() == DataType.MATRIX ? OperationTypes.CAST_AS_MATRIX : OperationTypes.CAST_AS_FRAME), left.getDataType(), right.getValueType());
                long bsize = ConfigurationManager.getBlocksize();
                rightInput.getOutputParameters().setDimensions(1, 1, bsize, bsize, -1);
            }
            LeftIndex leftIndexLop = new LeftIndex(left.constructLops(), rightInput, getInput().get(2).constructLops(), getInput().get(3).constructLops(), getInput().get(4).constructLops(), getInput().get(5).constructLops(), getDataType(), getValueType(), et, getSpLixCacheType(method));
            setOutputDimensions(leftIndexLop);
            setLineNumbers(leftIndexLop);
            setLops(leftIndexLop);
        } else {
            LeftIndex left = new LeftIndex(getInput().get(0).constructLops(), getInput().get(1).constructLops(), getInput().get(2).constructLops(), getInput().get(3).constructLops(), getInput().get(4).constructLops(), getInput().get(5).constructLops(), getDataType(), getValueType(), et);
            setOutputDimensions(left);
            setLineNumbers(left);
            setLops(left);
        }
    } catch (Exception e) {
        throw new HopsException(this.printErrorLocation() + "In LeftIndexingOp Hop, error in constructing Lops ", e);
    }
    // add reblock/checkpoint lops if necessary
    constructAndSetLopsDataFlowProperties();
    return getLops();
}
Also used : Group(org.apache.sysml.lops.Group) ZeroOut(org.apache.sysml.lops.ZeroOut) RightIndex(org.apache.sysml.lops.RightIndex) LeftIndex(org.apache.sysml.lops.LeftIndex) ExecType(org.apache.sysml.lops.LopProperties.ExecType) Binary(org.apache.sysml.lops.Binary) Lop(org.apache.sysml.lops.Lop) UnaryCP(org.apache.sysml.lops.UnaryCP)

Aggregations

Group (org.apache.sysml.lops.Group)4 Lop (org.apache.sysml.lops.Lop)4 ExecType (org.apache.sysml.lops.LopProperties.ExecType)4 RightIndex (org.apache.sysml.lops.RightIndex)4 SparkAggType (org.apache.sysml.hops.AggBinaryOp.SparkAggType)2 Aggregate (org.apache.sysml.lops.Aggregate)2 Binary (org.apache.sysml.lops.Binary)2 LeftIndex (org.apache.sysml.lops.LeftIndex)2 UnaryCP (org.apache.sysml.lops.UnaryCP)2 ZeroOut (org.apache.sysml.lops.ZeroOut)2
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial