Search in sources :

Example 1 with WindowFunctionInfo

use of com.sap.hadoop.windowing.functions2.FunctionRegistry.WindowFunctionInfo in project SQLWindowing by hbutani.

the class WindowFunctionTranslation method translateWindowSpec.

public static WindowDef translateWindowSpec(QueryDef qDef, InputInfo iInfo, WindowFunctionSpec wFnSpec) throws WindowingException {
    WindowSpec wSpec = wFnSpec.getWindowSpec();
    if (wSpec == null)
        return null;
    WindowFunctionInfo wFnInfo = FunctionRegistry.getWindowFunctionInfo(wFnSpec.getName());
    String desc = wFnSpec.toString();
    if (wSpec != null && !wFnInfo.isSupportsWindow()) {
        throw new WindowingException(sprintf("Function %s doesn't support windowing", desc));
    }
    return WindowSpecTranslation.translateWindowSpecOnInput(qDef, wSpec, iInfo, desc);
}
Also used : WindowFunctionInfo(com.sap.hadoop.windowing.functions2.FunctionRegistry.WindowFunctionInfo) WindowingException(com.sap.hadoop.windowing.WindowingException) WindowSpec(com.sap.hadoop.windowing.query2.specification.WindowSpec)

Example 2 with WindowFunctionInfo

use of com.sap.hadoop.windowing.functions2.FunctionRegistry.WindowFunctionInfo in project SQLWindowing by hbutani.

the class WindowingTableFunction method execute.

@SuppressWarnings({ "unchecked", "rawtypes" })
@Override
public void execute(PartitionIterator<Object> pItr, Partition outP) throws WindowingException {
    ArrayList<List<?>> oColumns = new ArrayList<List<?>>();
    Partition iPart = pItr.getPartition();
    StructObjectInspector inputOI;
    try {
        inputOI = (StructObjectInspector) iPart.getSerDe().getObjectInspector();
    } catch (SerDeException se) {
        throw new WindowingException(se);
    }
    try {
        for (WindowFunctionDef wFn : wFnDefs) {
            boolean processWindow = wFn.getWindow() != null;
            pItr.reset();
            if (!processWindow) {
                GenericUDAFEvaluator fEval = wFn.getEvaluator();
                Object[] args = new Object[wFn.getArgs().size()];
                AggregationBuffer aggBuffer = fEval.getNewAggregationBuffer();
                while (pItr.hasNext()) {
                    Object row = pItr.next();
                    int i = 0;
                    for (ArgDef arg : wFn.getArgs()) {
                        args[i++] = arg.getExprEvaluator().evaluate(row);
                    }
                    fEval.aggregate(aggBuffer, args);
                }
                Object out = fEval.evaluate(aggBuffer);
                WindowFunctionInfo wFnInfo = FunctionRegistry.getWindowFunctionInfo(wFn.getSpec().getName());
                if (!wFnInfo.isPivotResult()) {
                    out = new SameList(iPart.size(), out);
                }
                oColumns.add((List<?>) out);
            } else {
                oColumns.add(executeFnwithWindow(getQueryDef(), wFn, iPart));
            }
        }
        for (int i = 0; i < iPart.size(); i++) {
            ArrayList oRow = new ArrayList();
            Object iRow = iPart.getAt(i);
            for (StructField f : inputOI.getAllStructFieldRefs()) {
                oRow.add(inputOI.getStructFieldData(iRow, f));
            }
            for (int j = 0; j < oColumns.size(); j++) {
                oRow.add(oColumns.get(j).get(i));
            }
            outP.append(oRow);
        }
    } catch (HiveException he) {
        throw new WindowingException(he);
    }
}
Also used : Partition(com.sap.hadoop.windowing.runtime2.Partition) HiveException(org.apache.hadoop.hive.ql.metadata.HiveException) GenericUDAFEvaluator(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator) ArrayList(java.util.ArrayList) ArgDef(com.sap.hadoop.windowing.query2.definition.ArgDef) WindowFunctionInfo(com.sap.hadoop.windowing.functions2.FunctionRegistry.WindowFunctionInfo) StructField(org.apache.hadoop.hive.serde2.objectinspector.StructField) SameList(com.sap.hadoop.ds.SameList) WindowingException(com.sap.hadoop.windowing.WindowingException) ArrayList(java.util.ArrayList) SameList(com.sap.hadoop.ds.SameList) List(java.util.List) AggregationBuffer(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationBuffer) WindowFunctionDef(com.sap.hadoop.windowing.query2.definition.WindowFunctionDef) SerDeException(org.apache.hadoop.hive.serde2.SerDeException) StructObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector)

Aggregations

WindowingException (com.sap.hadoop.windowing.WindowingException)2 WindowFunctionInfo (com.sap.hadoop.windowing.functions2.FunctionRegistry.WindowFunctionInfo)2 SameList (com.sap.hadoop.ds.SameList)1 ArgDef (com.sap.hadoop.windowing.query2.definition.ArgDef)1 WindowFunctionDef (com.sap.hadoop.windowing.query2.definition.WindowFunctionDef)1 WindowSpec (com.sap.hadoop.windowing.query2.specification.WindowSpec)1 Partition (com.sap.hadoop.windowing.runtime2.Partition)1 ArrayList (java.util.ArrayList)1 List (java.util.List)1 HiveException (org.apache.hadoop.hive.ql.metadata.HiveException)1 GenericUDAFEvaluator (org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator)1 AggregationBuffer (org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationBuffer)1 SerDeException (org.apache.hadoop.hive.serde2.SerDeException)1 StructField (org.apache.hadoop.hive.serde2.objectinspector.StructField)1 StructObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector)1