Search in sources :

Example 1 with StreamingSemiAntiJoinOperator

use of org.apache.flink.table.runtime.operators.join.stream.StreamingSemiAntiJoinOperator in project flink by apache.

the class StreamExecJoin method translateToPlanInternal.

@Override
@SuppressWarnings("unchecked")
protected Transformation<RowData> translateToPlanInternal(PlannerBase planner, ExecNodeConfig config) {
    final ExecEdge leftInputEdge = getInputEdges().get(0);
    final ExecEdge rightInputEdge = getInputEdges().get(1);
    final Transformation<RowData> leftTransform = (Transformation<RowData>) leftInputEdge.translateToPlan(planner);
    final Transformation<RowData> rightTransform = (Transformation<RowData>) rightInputEdge.translateToPlan(planner);
    final RowType leftType = (RowType) leftInputEdge.getOutputType();
    final RowType rightType = (RowType) rightInputEdge.getOutputType();
    JoinUtil.validateJoinSpec(joinSpec, leftType, rightType, true);
    final int[] leftJoinKey = joinSpec.getLeftKeys();
    final int[] rightJoinKey = joinSpec.getRightKeys();
    final InternalTypeInfo<RowData> leftTypeInfo = InternalTypeInfo.of(leftType);
    final JoinInputSideSpec leftInputSpec = JoinUtil.analyzeJoinInput(leftTypeInfo, leftJoinKey, leftUniqueKeys);
    final InternalTypeInfo<RowData> rightTypeInfo = InternalTypeInfo.of(rightType);
    final JoinInputSideSpec rightInputSpec = JoinUtil.analyzeJoinInput(rightTypeInfo, rightJoinKey, rightUniqueKeys);
    GeneratedJoinCondition generatedCondition = JoinUtil.generateConditionFunction(config.getTableConfig(), joinSpec, leftType, rightType);
    long minRetentionTime = config.getStateRetentionTime();
    AbstractStreamingJoinOperator operator;
    FlinkJoinType joinType = joinSpec.getJoinType();
    if (joinType == FlinkJoinType.ANTI || joinType == FlinkJoinType.SEMI) {
        operator = new StreamingSemiAntiJoinOperator(joinType == FlinkJoinType.ANTI, leftTypeInfo, rightTypeInfo, generatedCondition, leftInputSpec, rightInputSpec, joinSpec.getFilterNulls(), minRetentionTime);
    } else {
        boolean leftIsOuter = joinType == FlinkJoinType.LEFT || joinType == FlinkJoinType.FULL;
        boolean rightIsOuter = joinType == FlinkJoinType.RIGHT || joinType == FlinkJoinType.FULL;
        operator = new StreamingJoinOperator(leftTypeInfo, rightTypeInfo, generatedCondition, leftInputSpec, rightInputSpec, leftIsOuter, rightIsOuter, joinSpec.getFilterNulls(), minRetentionTime);
    }
    final RowType returnType = (RowType) getOutputType();
    final TwoInputTransformation<RowData, RowData, RowData> transform = ExecNodeUtil.createTwoInputTransformation(leftTransform, rightTransform, createTransformationMeta(JOIN_TRANSFORMATION, config), operator, InternalTypeInfo.of(returnType), leftTransform.getParallelism());
    // set KeyType and Selector for state
    RowDataKeySelector leftSelect = KeySelectorUtil.getRowDataSelector(leftJoinKey, leftTypeInfo);
    RowDataKeySelector rightSelect = KeySelectorUtil.getRowDataSelector(rightJoinKey, rightTypeInfo);
    transform.setStateKeySelectors(leftSelect, rightSelect);
    transform.setStateKeyType(leftSelect.getProducedType());
    return transform;
}
Also used : TwoInputTransformation(org.apache.flink.streaming.api.transformations.TwoInputTransformation) Transformation(org.apache.flink.api.dag.Transformation) ExecEdge(org.apache.flink.table.planner.plan.nodes.exec.ExecEdge) RowType(org.apache.flink.table.types.logical.RowType) FlinkJoinType(org.apache.flink.table.runtime.operators.join.FlinkJoinType) JoinInputSideSpec(org.apache.flink.table.runtime.operators.join.stream.state.JoinInputSideSpec) RowData(org.apache.flink.table.data.RowData) GeneratedJoinCondition(org.apache.flink.table.runtime.generated.GeneratedJoinCondition) AbstractStreamingJoinOperator(org.apache.flink.table.runtime.operators.join.stream.AbstractStreamingJoinOperator) StreamingJoinOperator(org.apache.flink.table.runtime.operators.join.stream.StreamingJoinOperator) RowDataKeySelector(org.apache.flink.table.runtime.keyselector.RowDataKeySelector) AbstractStreamingJoinOperator(org.apache.flink.table.runtime.operators.join.stream.AbstractStreamingJoinOperator) StreamingSemiAntiJoinOperator(org.apache.flink.table.runtime.operators.join.stream.StreamingSemiAntiJoinOperator)

Aggregations

Transformation (org.apache.flink.api.dag.Transformation)1 TwoInputTransformation (org.apache.flink.streaming.api.transformations.TwoInputTransformation)1 RowData (org.apache.flink.table.data.RowData)1 ExecEdge (org.apache.flink.table.planner.plan.nodes.exec.ExecEdge)1 GeneratedJoinCondition (org.apache.flink.table.runtime.generated.GeneratedJoinCondition)1 RowDataKeySelector (org.apache.flink.table.runtime.keyselector.RowDataKeySelector)1 FlinkJoinType (org.apache.flink.table.runtime.operators.join.FlinkJoinType)1 AbstractStreamingJoinOperator (org.apache.flink.table.runtime.operators.join.stream.AbstractStreamingJoinOperator)1 StreamingJoinOperator (org.apache.flink.table.runtime.operators.join.stream.StreamingJoinOperator)1 StreamingSemiAntiJoinOperator (org.apache.flink.table.runtime.operators.join.stream.StreamingSemiAntiJoinOperator)1 JoinInputSideSpec (org.apache.flink.table.runtime.operators.join.stream.state.JoinInputSideSpec)1 RowType (org.apache.flink.table.types.logical.RowType)1