Search in sources :

Example 1 with InputInfo

use of com.sap.hadoop.windowing.query2.translate.QueryTranslationInfo.InputInfo in project SQLWindowing by hbutani.

the class WindowSpecTranslation method translateWindow.

/*
	 * compute the Description to use for the Input.
	 * get the inputInfo for the input: if the function has a MapPhase use the Map Inputfo. 
	 * invoke translateWindowSpecOnInput on WdwSpec of TblFunc
	 * If TableFunc is the FunctionRegistry.WINDOWING_TABLE_FUNCTION:
	 * - 
	 */
static WindowDef translateWindow(QueryDef qDef, TableFuncDef tFnDef) throws WindowingException {
    QueryTranslationInfo tInfo = qDef.getTranslationInfo();
    TableFuncSpec tFnSpec = tFnDef.getTableFuncSpec();
    /*
		 * for now the Language only allows explicit specification of Partition & Order clauses.
		 * Easy to allow references to a Global Window Spec.
		 */
    WindowSpec wSpec = new WindowSpec();
    wSpec.setPartition(tFnSpec.getPartition());
    wSpec.setOrder(tFnSpec.getOrder());
    QueryInputDef iDef = tFnDef.getInput();
    if (wSpec.getPartition() == null) {
        return null;
    }
    String desc = getInputDescription(qDef, tFnDef);
    TableFunctionEvaluator tFn = tFnDef.getFunction();
    InputInfo iInfo = null;
    if (tFn.isTransformsRawInput()) {
        iInfo = tInfo.getMapInputInfo(tFnDef);
    } else {
        iInfo = tInfo.getInputInfo(iDef);
    }
    return translateWindowSpecOnInput(qDef, wSpec, iInfo, desc);
}
Also used : TableFunctionEvaluator(com.sap.hadoop.windowing.functions2.TableFunctionEvaluator) InputInfo(com.sap.hadoop.windowing.query2.translate.QueryTranslationInfo.InputInfo) QueryInputDef(com.sap.hadoop.windowing.query2.definition.QueryInputDef) TableFuncSpec(com.sap.hadoop.windowing.query2.specification.TableFuncSpec) WindowSpec(com.sap.hadoop.windowing.query2.specification.WindowSpec)

Example 2 with InputInfo

use of com.sap.hadoop.windowing.query2.translate.QueryTranslationInfo.InputInfo in project SQLWindowing by hbutani.

the class WindowSpecTranslation method translateOrderColumn.

static OrderColumnDef translateOrderColumn(QueryDef qDef, InputInfo iInfo, OrderColumnSpec oSpec) throws WindowingException {
    OrderColumnDef ocDef = new OrderColumnDef(oSpec);
    translateColumn(qDef, ocDef, iInfo, oSpec);
    TranslateUtils.validateComparable(ocDef.getOI(), sprintf("Partition Column %s is not comparable", oSpec));
    return ocDef;
}
Also used : OrderColumnDef(com.sap.hadoop.windowing.query2.definition.OrderColumnDef)

Example 3 with InputInfo

use of com.sap.hadoop.windowing.query2.translate.QueryTranslationInfo.InputInfo in project SQLWindowing by hbutani.

the class WindowSpecTranslation method translatePartitionColumn.

static ColumnDef translatePartitionColumn(QueryDef qDef, InputInfo iInfo, ColumnSpec cSpec) throws WindowingException {
    ColumnDef cDef = new ColumnDef(cSpec);
    translateColumn(qDef, cDef, iInfo, cSpec);
    TranslateUtils.validateComparable(cDef.getOI(), sprintf("Partition Column %s is not comparable", cSpec));
    return cDef;
}
Also used : OrderColumnDef(com.sap.hadoop.windowing.query2.definition.OrderColumnDef) ColumnDef(com.sap.hadoop.windowing.query2.definition.ColumnDef)

Example 4 with InputInfo

use of com.sap.hadoop.windowing.query2.translate.QueryTranslationInfo.InputInfo in project SQLWindowing by hbutani.

the class WindowSpecTranslation method translateWindowSpecOnInput.

/*
	 * <ol>
	 * <li> If wSpec points to a source WindowSpec. Validate that it is valid. If it hasn't been already translated then translate it.
	 * <li> Start with an empty WdwDef or a cloned WdwDef from the source WdwDef.
	 * <li> translate the PartitionSpec if it exists. Replace the existing PDef with this; also remove the OrderDef.
	 * <li> translate the OrderSpec if it exists. Replace existing OrderDef with this.
	 * <li> add in Partition Columns if not in OrderDef already.
	 * <li> translate the WindowSpec if it exists. Replace existing WdwDef with it.
	 * <li> If name is non-null add this def to TranslationInfo::nameToWdwDef map.
	 * </ol>
	 */
static WindowDef translateWindowSpecOnInput(QueryDef qDef, WindowSpec wSpec, InputInfo iInfo, String inputDesc) throws WindowingException {
    QuerySpec qSpec = qDef.getSpec();
    WindowDef wDef;
    fillInWindowSpec(qSpec, wSpec.getSourceId(), wSpec);
    wDef = new WindowDef(wSpec);
    PartitionSpec pSpec = wSpec.getPartition();
    OrderSpec oSpec = wSpec.getOrder();
    WindowFrameSpec wFrameSpec = wSpec.getWindow();
    PartitionDef pDef = translatePartition(qDef, iInfo, pSpec);
    OrderDef oDef = translateOrder(qDef, inputDesc, iInfo, oSpec, pDef);
    WindowFrameDef wdwDef = translateWindowFrame(qDef, wFrameSpec, iInfo);
    wDef.setPartDef(pDef);
    wDef.setOrderDef(oDef);
    wDef.setWindow(wdwDef);
    return wDef;
}
Also used : OrderSpec(com.sap.hadoop.windowing.query2.specification.OrderSpec) WindowFrameDef(com.sap.hadoop.windowing.query2.definition.WindowFrameDef) WindowDef(com.sap.hadoop.windowing.query2.definition.WindowDef) PartitionDef(com.sap.hadoop.windowing.query2.definition.PartitionDef) QuerySpec(com.sap.hadoop.windowing.query2.specification.QuerySpec) OrderDef(com.sap.hadoop.windowing.query2.definition.OrderDef) PartitionSpec(com.sap.hadoop.windowing.query2.specification.PartitionSpec) WindowFrameSpec(com.sap.hadoop.windowing.query2.specification.WindowFrameSpec)

Example 5 with InputInfo

use of com.sap.hadoop.windowing.query2.translate.QueryTranslationInfo.InputInfo in project SQLWindowing by hbutani.

the class WindowSpecTranslation method translatePartition.

static PartitionDef translatePartition(QueryDef qDef, InputInfo iInfo, PartitionSpec spec) throws WindowingException {
    if (spec == null || spec.getColumns() == null || spec.getColumns().size() == 0)
        return null;
    PartitionDef pDef = new PartitionDef(spec);
    for (ColumnSpec colSpec : spec.getColumns()) {
        ColumnDef cDef = translatePartitionColumn(qDef, iInfo, colSpec);
        pDef.addColumn(cDef);
    }
    return pDef;
}
Also used : OrderColumnSpec(com.sap.hadoop.windowing.query2.specification.OrderColumnSpec) ColumnSpec(com.sap.hadoop.windowing.query2.specification.ColumnSpec) PartitionDef(com.sap.hadoop.windowing.query2.definition.PartitionDef) OrderColumnDef(com.sap.hadoop.windowing.query2.definition.OrderColumnDef) ColumnDef(com.sap.hadoop.windowing.query2.definition.ColumnDef)

Aggregations

WindowingException (com.sap.hadoop.windowing.WindowingException)8 InputInfo (com.sap.hadoop.windowing.query2.translate.QueryTranslationInfo.InputInfo)8 ColumnDef (com.sap.hadoop.windowing.query2.definition.ColumnDef)6 OrderColumnDef (com.sap.hadoop.windowing.query2.definition.OrderColumnDef)6 ExprNodeEvaluator (org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator)5 ExprNodeDesc (org.apache.hadoop.hive.ql.plan.ExprNodeDesc)5 ObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector)5 QueryInputDef (com.sap.hadoop.windowing.query2.definition.QueryInputDef)4 ASTNode (org.apache.hadoop.hive.ql.parse.ASTNode)4 TableFunctionEvaluator (com.sap.hadoop.windowing.functions2.TableFunctionEvaluator)3 ArgDef (com.sap.hadoop.windowing.query2.definition.ArgDef)3 OrderDef (com.sap.hadoop.windowing.query2.definition.OrderDef)3 StructObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector)3 PartitionDef (com.sap.hadoop.windowing.query2.definition.PartitionDef)2 TableFuncDef (com.sap.hadoop.windowing.query2.definition.TableFuncDef)2 WindowDef (com.sap.hadoop.windowing.query2.definition.WindowDef)2 WindowFrameDef (com.sap.hadoop.windowing.query2.definition.WindowFrameDef)2 OrderColumnSpec (com.sap.hadoop.windowing.query2.specification.OrderColumnSpec)2 QuerySpec (com.sap.hadoop.windowing.query2.specification.QuerySpec)2 RangeBoundarySpec (com.sap.hadoop.windowing.query2.specification.WindowFrameSpec.RangeBoundarySpec)2