Search in sources :

Example 6 with GroupCombineFunction

use of org.apache.flink.api.common.functions.GroupCombineFunction in project flink by apache.

the class AllGroupReduceDriver method prepare.

// --------------------------------------------------------------------------------------------
@Override
public void prepare() throws Exception {
    final TaskConfig config = this.taskContext.getTaskConfig();
    this.strategy = config.getDriverStrategy();
    switch(this.strategy) {
        case ALL_GROUP_REDUCE_COMBINE:
            if (!(this.taskContext.getStub() instanceof GroupCombineFunction)) {
                throw new Exception("Using combiner on a UDF that does not implement the combiner interface " + GroupCombineFunction.class.getName());
            }
        case ALL_GROUP_REDUCE:
        case ALL_GROUP_COMBINE:
            break;
        default:
            throw new Exception("Unrecognized driver strategy for AllGroupReduce driver: " + this.strategy.name());
    }
    this.serializer = this.taskContext.<IT>getInputSerializer(0).getSerializer();
    this.input = this.taskContext.getInput(0);
    ExecutionConfig executionConfig = taskContext.getExecutionConfig();
    this.objectReuseEnabled = executionConfig.isObjectReuseEnabled();
    if (LOG.isDebugEnabled()) {
        LOG.debug("AllGroupReduceDriver object reuse: " + (this.objectReuseEnabled ? "ENABLED" : "DISABLED") + ".");
    }
}
Also used : GroupCombineFunction(org.apache.flink.api.common.functions.GroupCombineFunction) TaskConfig(org.apache.flink.runtime.operators.util.TaskConfig) ExecutionConfig(org.apache.flink.api.common.ExecutionConfig)

Aggregations

GroupCombineFunction (org.apache.flink.api.common.functions.GroupCombineFunction)6 ExecutionEnvironment (org.apache.flink.api.java.ExecutionEnvironment)2 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)2 Tuple3 (org.apache.flink.api.java.tuple.Tuple3)2 Collector (org.apache.flink.util.Collector)2 Test (org.junit.Test)2 IOException (java.io.IOException)1 ParameterizedType (java.lang.reflect.ParameterizedType)1 Type (java.lang.reflect.Type)1 ExecutionConfig (org.apache.flink.api.common.ExecutionConfig)1 CombineFunction (org.apache.flink.api.common.functions.CombineFunction)1 Keys (org.apache.flink.api.common.operators.Keys)1 SelectorFunctionKeys (org.apache.flink.api.common.operators.Keys.SelectorFunctionKeys)1 Ordering (org.apache.flink.api.common.operators.Ordering)1 UnaryOperatorInformation (org.apache.flink.api.common.operators.UnaryOperatorInformation)1 GroupCombineOperatorBase (org.apache.flink.api.common.operators.base.GroupCombineOperatorBase)1 CancelTaskException (org.apache.flink.runtime.execution.CancelTaskException)1 ExceptionInChainedStubException (org.apache.flink.runtime.operators.chaining.ExceptionInChainedStubException)1 CombiningUnilateralSortMerger (org.apache.flink.runtime.operators.sort.CombiningUnilateralSortMerger)1 UnilateralSortMerger (org.apache.flink.runtime.operators.sort.UnilateralSortMerger)1