Search in sources :

Example 11 with QueryDef

use of com.sap.hadoop.windowing.query2.definition.QueryDef in project SQLWindowing by hbutani.

the class WindowingShell method execute.

protected void execute(QueryDef q, QueryOutputPrinter outP) throws WindowingException {
    QuerySpec qSpec = q.getSpec();
    try {
        executor.execute(q, this);
    } finally {
        HiveTableDef hiveTable = q.getHiveTableDef();
        if (hiveTable instanceof HiveQueryDef) {
            String tableName = hiveTable.getHiveTableSpec().getTableName();
            hiveQryExec.dropTable(tableName);
        }
    }
    if (qSpec.getOutput().getHiveTable() != null) {
        loadToOutputTable(q);
    }
    if (outP != null) {
        outP.printQueryOutput(q, cfg);
    }
}
Also used : HiveQueryDef(com.sap.hadoop.windowing.query2.definition.HiveQueryDef) QuerySpec(com.sap.hadoop.windowing.query2.specification.QuerySpec) HiveTableDef(com.sap.hadoop.windowing.query2.definition.HiveTableDef)

Example 12 with QueryDef

use of com.sap.hadoop.windowing.query2.definition.QueryDef in project SQLWindowing by hbutani.

the class WindowingShell method execute.

public void execute(String query, QueryOutputPrinter outP) throws WindowingException {
    QuerySpec qSpec = parse(query);
    QueryDef q = translator.translate(qSpec, this);
    ArrayList<QueryDef> componentQueries;
    executor.beforeComponentization(q, this);
    QueryComponentizer qC = new QueryComponentizer(q, this);
    componentQueries = qC.componentize();
    executor.beforeExecute(q, componentQueries, this);
    try {
        for (QueryDef cqDef : componentQueries) {
            execute(cqDef, outP);
        }
    } finally {
        executor.afterExecute(q, componentQueries, this);
    }
}
Also used : QueryComponentizer(com.sap.hadoop.windowing.query2.translate.QueryComponentizer) QuerySpec(com.sap.hadoop.windowing.query2.specification.QuerySpec) QueryDef(com.sap.hadoop.windowing.query2.definition.QueryDef) HiveQueryDef(com.sap.hadoop.windowing.query2.definition.HiveQueryDef)

Example 13 with QueryDef

use of com.sap.hadoop.windowing.query2.definition.QueryDef in project SQLWindowing by hbutani.

the class Translator method translate.

public QueryDef translate(QuerySpec qSpec, WindowingShell wShell) throws WindowingException {
    // clone the cfg
    HiveConf qCfg = new HiveConf(wShell.getCfg());
    QueryDef qry = new QueryDef();
    qry.setSpec(qSpec);
    QueryTranslationInfo transInfo = new QueryTranslationInfo();
    transInfo.setHiveCfg(qCfg);
    transInfo.setWshell(wShell);
    try {
        transInfo.setHive(Hive.get(qCfg));
        transInfo.setHiveMSClient(HiveUtils.getClient(qCfg));
    } catch (HiveException he) {
        throw new WindowingException(he);
    }
    qry.setTranslationInfo(transInfo);
    InputTranslation.translate(qry);
    WhereTranslation.translate(qry);
    OutputTranslation.translate(qry);
    return qry;
}
Also used : HiveException(org.apache.hadoop.hive.ql.metadata.HiveException) WindowingException(com.sap.hadoop.windowing.WindowingException) HiveConf(org.apache.hadoop.hive.conf.HiveConf) QueryDef(com.sap.hadoop.windowing.query2.definition.QueryDef)

Example 14 with QueryDef

use of com.sap.hadoop.windowing.query2.definition.QueryDef in project SQLWindowing by hbutani.

the class WindowFunctionTranslation method setupRankingArgs.

static void setupRankingArgs(QueryDef qDef, TableFuncDef windowTableFnDef, WindowFunctionDef wFnDef, WindowFunctionSpec wSpec) throws WindowingException {
    if (wSpec.getArgs().size() > 0) {
        throw new WindowingException("Ranking Functions can take no arguments");
    }
    QueryInputDef inpDef = windowTableFnDef.getInput();
    InputInfo inpInfo = qDef.getTranslationInfo().getInputInfo(inpDef);
    OrderDef oDef = getTableFuncOrderDef(windowTableFnDef);
    ArrayList<OrderColumnDef> oCols = oDef.getColumns();
    for (OrderColumnDef oCol : oCols) {
        wFnDef.addArg(TranslateUtils.buildArgDef(qDef, inpInfo, oCol.getExpression()));
    }
}
Also used : InputInfo(com.sap.hadoop.windowing.query2.translate.QueryTranslationInfo.InputInfo) OrderColumnDef(com.sap.hadoop.windowing.query2.definition.OrderColumnDef) QueryInputDef(com.sap.hadoop.windowing.query2.definition.QueryInputDef) WindowingException(com.sap.hadoop.windowing.WindowingException) OrderDef(com.sap.hadoop.windowing.query2.definition.OrderDef)

Example 15 with QueryDef

use of com.sap.hadoop.windowing.query2.definition.QueryDef in project SQLWindowing by hbutani.

the class WindowFunctionTranslation method addInputColumnsToList.

public static void addInputColumnsToList(QueryDef qDef, TableFuncDef windowTableFnDef, ArrayList<String> fieldNames, ArrayList<ObjectInspector> fieldOIs) {
    QueryTranslationInfo tInfo = qDef.getTranslationInfo();
    InputInfo iInfo = tInfo.getInputInfo(windowTableFnDef.getInput());
    StructObjectInspector OI = (StructObjectInspector) iInfo.getOI();
    for (StructField f : OI.getAllStructFieldRefs()) {
        fieldNames.add(f.getFieldName());
        fieldOIs.add(f.getFieldObjectInspector());
    }
}
Also used : InputInfo(com.sap.hadoop.windowing.query2.translate.QueryTranslationInfo.InputInfo) StructField(org.apache.hadoop.hive.serde2.objectinspector.StructField) StructObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector)

Aggregations

WindowingException (com.sap.hadoop.windowing.WindowingException)15 QueryDef (com.sap.hadoop.windowing.query2.definition.QueryDef)7 TableFuncDef (com.sap.hadoop.windowing.query2.definition.TableFuncDef)7 InputInfo (com.sap.hadoop.windowing.query2.translate.QueryTranslationInfo.InputInfo)7 ColumnDef (com.sap.hadoop.windowing.query2.definition.ColumnDef)6 QueryInputDef (com.sap.hadoop.windowing.query2.definition.QueryInputDef)6 ExprNodeEvaluator (org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator)6 ObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector)6 OrderColumnDef (com.sap.hadoop.windowing.query2.definition.OrderColumnDef)5 QuerySpec (com.sap.hadoop.windowing.query2.specification.QuerySpec)5 ExprNodeDesc (org.apache.hadoop.hive.ql.plan.ExprNodeDesc)5 TableFunctionEvaluator (com.sap.hadoop.windowing.functions2.TableFunctionEvaluator)4 ArgDef (com.sap.hadoop.windowing.query2.definition.ArgDef)4 OrderDef (com.sap.hadoop.windowing.query2.definition.OrderDef)4 LocalExecutorTest (com.sap.hadoop.windowing.testutils.LocalExecutorTest)4 Test (org.junit.Test)4 HiveQueryDef (com.sap.hadoop.windowing.query2.definition.HiveQueryDef)3 HiveTableDef (com.sap.hadoop.windowing.query2.definition.HiveTableDef)3 HiveTableSpec (com.sap.hadoop.windowing.query2.specification.HiveTableSpec)3 ArrayList (java.util.ArrayList)3