Search in sources :

Example 21 with WindowingException

use of com.sap.hadoop.windowing.WindowingException in project SQLWindowing by hbutani.

the class Translator method translate.

public QueryDef translate(QuerySpec qSpec, WindowingShell wShell) throws WindowingException {
    // clone the cfg
    HiveConf qCfg = new HiveConf(wShell.getCfg());
    QueryDef qry = new QueryDef();
    qry.setSpec(qSpec);
    QueryTranslationInfo transInfo = new QueryTranslationInfo();
    transInfo.setHiveCfg(qCfg);
    transInfo.setWshell(wShell);
    try {
        transInfo.setHive(Hive.get(qCfg));
        transInfo.setHiveMSClient(HiveUtils.getClient(qCfg));
    } catch (HiveException he) {
        throw new WindowingException(he);
    }
    qry.setTranslationInfo(transInfo);
    InputTranslation.translate(qry);
    WhereTranslation.translate(qry);
    OutputTranslation.translate(qry);
    return qry;
}
Also used : HiveException(org.apache.hadoop.hive.ql.metadata.HiveException) WindowingException(com.sap.hadoop.windowing.WindowingException) HiveConf(org.apache.hadoop.hive.conf.HiveConf) QueryDef(com.sap.hadoop.windowing.query2.definition.QueryDef)

Example 22 with WindowingException

use of com.sap.hadoop.windowing.WindowingException in project SQLWindowing by hbutani.

the class WindowFunctionTranslation method setupRankingArgs.

static void setupRankingArgs(QueryDef qDef, TableFuncDef windowTableFnDef, WindowFunctionDef wFnDef, WindowFunctionSpec wSpec) throws WindowingException {
    if (wSpec.getArgs().size() > 0) {
        throw new WindowingException("Ranking Functions can take no arguments");
    }
    QueryInputDef inpDef = windowTableFnDef.getInput();
    InputInfo inpInfo = qDef.getTranslationInfo().getInputInfo(inpDef);
    OrderDef oDef = getTableFuncOrderDef(windowTableFnDef);
    ArrayList<OrderColumnDef> oCols = oDef.getColumns();
    for (OrderColumnDef oCol : oCols) {
        wFnDef.addArg(TranslateUtils.buildArgDef(qDef, inpInfo, oCol.getExpression()));
    }
}
Also used : InputInfo(com.sap.hadoop.windowing.query2.translate.QueryTranslationInfo.InputInfo) OrderColumnDef(com.sap.hadoop.windowing.query2.definition.OrderColumnDef) QueryInputDef(com.sap.hadoop.windowing.query2.definition.QueryInputDef) WindowingException(com.sap.hadoop.windowing.WindowingException) OrderDef(com.sap.hadoop.windowing.query2.definition.OrderDef)

Example 23 with WindowingException

use of com.sap.hadoop.windowing.WindowingException in project SQLWindowing by hbutani.

the class WindowFunctionTranslation method validateWindowDefForWFn.

public static void validateWindowDefForWFn(TableFuncDef tFnDef, WindowFunctionDef wFnDef) throws WindowingException {
    WindowDef tWindow = tFnDef.getWindow();
    WindowDef fWindow = wFnDef.getWindow();
    PartitionDef tPart = tWindow == null ? null : tWindow.getPartDef();
    PartitionDef fPart = fWindow == null ? null : fWindow.getPartDef();
    if (!TranslateUtils.isCompatible(tPart, fPart)) {
        throw new WindowingException(sprintf("Window Function '%s' has an incompatible partition clause", wFnDef.getSpec()));
    }
    OrderDef tOrder = tWindow == null ? null : tWindow.getOrderDef();
    OrderDef fOrder = fWindow == null ? null : fWindow.getOrderDef();
    if (!TranslateUtils.isCompatible(tOrder, fOrder)) {
        throw new WindowingException(sprintf("Window Function '%s' has an incompatible order clause", wFnDef.getSpec()));
    }
}
Also used : WindowDef(com.sap.hadoop.windowing.query2.definition.WindowDef) PartitionDef(com.sap.hadoop.windowing.query2.definition.PartitionDef) WindowingException(com.sap.hadoop.windowing.WindowingException) OrderDef(com.sap.hadoop.windowing.query2.definition.OrderDef)

Example 24 with WindowingException

use of com.sap.hadoop.windowing.WindowingException in project SQLWindowing by hbutani.

the class WindowFunctionTranslation method setupEvaluator.

static void setupEvaluator(WindowFunctionDef wFnDef) throws WindowingException {
    try {
        WindowFunctionSpec wSpec = wFnDef.getSpec();
        ArrayList<ArgDef> args = wFnDef.getArgs();
        ArrayList<ObjectInspector> argOIs = getWritableObjectInspector(args);
        GenericUDAFEvaluator wFnEval = org.apache.hadoop.hive.ql.exec.FunctionRegistry.getGenericUDAFEvaluator(wSpec.getName(), argOIs, wSpec.isDistinct(), wSpec.isStar());
        ObjectInspector[] funcArgOIs = null;
        if (args != null) {
            funcArgOIs = new ObjectInspector[args.size()];
            int i = 0;
            for (ArgDef arg : args) {
                funcArgOIs[i++] = arg.getOI();
            }
        }
        ObjectInspector OI = wFnEval.init(GenericUDAFEvaluator.Mode.COMPLETE, funcArgOIs);
        wFnDef.setEvaluator(wFnEval);
        wFnDef.setOI(OI);
    } catch (HiveException he) {
        throw new WindowingException(he);
    }
}
Also used : StructObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector) ObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector) HiveException(org.apache.hadoop.hive.ql.metadata.HiveException) GenericUDAFEvaluator(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator) WindowingException(com.sap.hadoop.windowing.WindowingException) WindowFunctionSpec(com.sap.hadoop.windowing.query2.specification.WindowFunctionSpec) ArgDef(com.sap.hadoop.windowing.query2.definition.ArgDef)

Example 25 with WindowingException

use of com.sap.hadoop.windowing.WindowingException in project SQLWindowing by hbutani.

the class WindowFunctionTranslation method translateWindowSpec.

public static WindowDef translateWindowSpec(QueryDef qDef, InputInfo iInfo, WindowFunctionSpec wFnSpec) throws WindowingException {
    WindowSpec wSpec = wFnSpec.getWindowSpec();
    if (wSpec == null)
        return null;
    WindowFunctionInfo wFnInfo = FunctionRegistry.getWindowFunctionInfo(wFnSpec.getName());
    String desc = wFnSpec.toString();
    if (wSpec != null && !wFnInfo.isSupportsWindow()) {
        throw new WindowingException(sprintf("Function %s doesn't support windowing", desc));
    }
    return WindowSpecTranslation.translateWindowSpecOnInput(qDef, wSpec, iInfo, desc);
}
Also used : WindowFunctionInfo(com.sap.hadoop.windowing.functions2.FunctionRegistry.WindowFunctionInfo) WindowingException(com.sap.hadoop.windowing.WindowingException) WindowSpec(com.sap.hadoop.windowing.query2.specification.WindowSpec)

Aggregations

WindowingException (com.sap.hadoop.windowing.WindowingException)62 HiveException (org.apache.hadoop.hive.ql.metadata.HiveException)18 SerDeException (org.apache.hadoop.hive.serde2.SerDeException)11 ObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector)10 IOException (java.io.IOException)9 SerDe (org.apache.hadoop.hive.serde2.SerDe)9 ExprNodeEvaluator (org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator)8 ArrayList (java.util.ArrayList)7 StructObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector)7 HiveMetaStoreClient (org.apache.hadoop.hive.metastore.HiveMetaStoreClient)6 Properties (java.util.Properties)5 Path (org.apache.hadoop.fs.Path)5 MetaException (org.apache.hadoop.hive.metastore.api.MetaException)5 Table (org.apache.hadoop.hive.metastore.api.Table)5 ExprNodeDesc (org.apache.hadoop.hive.ql.plan.ExprNodeDesc)5 Writable (org.apache.hadoop.io.Writable)5 TableFuncDef (com.sap.hadoop.windowing.query2.definition.TableFuncDef)4 HiveConf (org.apache.hadoop.hive.conf.HiveConf)4 ASTNode (org.apache.hadoop.hive.ql.parse.ASTNode)4 ArgDef (com.sap.hadoop.windowing.query2.definition.ArgDef)3