Search in sources :

Example 1 with IntervalJoinFunction

use of org.apache.flink.table.runtime.operators.join.interval.IntervalJoinFunction in project flink by apache.

the class StreamExecIntervalJoin method translateToPlanInternal.

@Override
@SuppressWarnings("unchecked")
protected Transformation<RowData> translateToPlanInternal(PlannerBase planner, ExecNodeConfig config) {
    ExecEdge leftInputEdge = getInputEdges().get(0);
    ExecEdge rightInputEdge = getInputEdges().get(1);
    RowType leftRowType = (RowType) leftInputEdge.getOutputType();
    RowType rightRowType = (RowType) rightInputEdge.getOutputType();
    Transformation<RowData> leftInputTransform = (Transformation<RowData>) leftInputEdge.translateToPlan(planner);
    Transformation<RowData> rightInputTransform = (Transformation<RowData>) rightInputEdge.translateToPlan(planner);
    RowType returnType = (RowType) getOutputType();
    InternalTypeInfo<RowData> returnTypeInfo = InternalTypeInfo.of(returnType);
    JoinSpec joinSpec = intervalJoinSpec.getJoinSpec();
    IntervalJoinSpec.WindowBounds windowBounds = intervalJoinSpec.getWindowBounds();
    switch(joinSpec.getJoinType()) {
        case INNER:
        case LEFT:
        case RIGHT:
        case FULL:
            long relativeWindowSize = windowBounds.getLeftUpperBound() - windowBounds.getLeftLowerBound();
            if (relativeWindowSize < 0) {
                LOGGER.warn("The relative time interval size " + relativeWindowSize + "is negative, please check the join conditions.");
                return createNegativeWindowSizeJoin(joinSpec, leftInputTransform, rightInputTransform, leftRowType.getFieldCount(), rightRowType.getFieldCount(), returnTypeInfo, config);
            } else {
                GeneratedJoinCondition joinCondition = JoinUtil.generateConditionFunction(config.getTableConfig(), joinSpec, leftRowType, rightRowType);
                IntervalJoinFunction joinFunction = new IntervalJoinFunction(joinCondition, returnTypeInfo, joinSpec.getFilterNulls());
                TwoInputTransformation<RowData, RowData, RowData> transform;
                if (windowBounds.isEventTime()) {
                    transform = createRowTimeJoin(leftInputTransform, rightInputTransform, returnTypeInfo, joinFunction, joinSpec, windowBounds, config);
                } else {
                    transform = createProcTimeJoin(leftInputTransform, rightInputTransform, returnTypeInfo, joinFunction, joinSpec, windowBounds, config);
                }
                if (inputsContainSingleton()) {
                    transform.setParallelism(1);
                    transform.setMaxParallelism(1);
                }
                // set KeyType and Selector for state
                RowDataKeySelector leftSelect = KeySelectorUtil.getRowDataSelector(joinSpec.getLeftKeys(), InternalTypeInfo.of(leftRowType));
                RowDataKeySelector rightSelect = KeySelectorUtil.getRowDataSelector(joinSpec.getRightKeys(), InternalTypeInfo.of(rightRowType));
                transform.setStateKeySelectors(leftSelect, rightSelect);
                transform.setStateKeyType(leftSelect.getProducedType());
                return transform;
            }
        default:
            throw new TableException("Interval Join: " + joinSpec.getJoinType() + " Join between stream " + "and stream is not supported yet.\nplease re-check " + "interval join statement according to description above.");
    }
}
Also used : TwoInputTransformation(org.apache.flink.streaming.api.transformations.TwoInputTransformation) UnionTransformation(org.apache.flink.streaming.api.transformations.UnionTransformation) OneInputTransformation(org.apache.flink.streaming.api.transformations.OneInputTransformation) Transformation(org.apache.flink.api.dag.Transformation) TableException(org.apache.flink.table.api.TableException) ExecEdge(org.apache.flink.table.planner.plan.nodes.exec.ExecEdge) JoinSpec(org.apache.flink.table.planner.plan.nodes.exec.spec.JoinSpec) IntervalJoinSpec(org.apache.flink.table.planner.plan.nodes.exec.spec.IntervalJoinSpec) RowType(org.apache.flink.table.types.logical.RowType) IntervalJoinFunction(org.apache.flink.table.runtime.operators.join.interval.IntervalJoinFunction) RowData(org.apache.flink.table.data.RowData) GeneratedJoinCondition(org.apache.flink.table.runtime.generated.GeneratedJoinCondition) RowDataKeySelector(org.apache.flink.table.runtime.keyselector.RowDataKeySelector) IntervalJoinSpec(org.apache.flink.table.planner.plan.nodes.exec.spec.IntervalJoinSpec)

Aggregations

Transformation (org.apache.flink.api.dag.Transformation)1 OneInputTransformation (org.apache.flink.streaming.api.transformations.OneInputTransformation)1 TwoInputTransformation (org.apache.flink.streaming.api.transformations.TwoInputTransformation)1 UnionTransformation (org.apache.flink.streaming.api.transformations.UnionTransformation)1 TableException (org.apache.flink.table.api.TableException)1 RowData (org.apache.flink.table.data.RowData)1 ExecEdge (org.apache.flink.table.planner.plan.nodes.exec.ExecEdge)1 IntervalJoinSpec (org.apache.flink.table.planner.plan.nodes.exec.spec.IntervalJoinSpec)1 JoinSpec (org.apache.flink.table.planner.plan.nodes.exec.spec.JoinSpec)1 GeneratedJoinCondition (org.apache.flink.table.runtime.generated.GeneratedJoinCondition)1 RowDataKeySelector (org.apache.flink.table.runtime.keyselector.RowDataKeySelector)1 IntervalJoinFunction (org.apache.flink.table.runtime.operators.join.interval.IntervalJoinFunction)1 RowType (org.apache.flink.table.types.logical.RowType)1