Examples with KeyDiscarder org.apache.flink.python.api.functions.util.KeyDiscarder used on opensource projects

Example 1 with KeyDiscarder

use of org.apache.flink.python.api.functions.util.KeyDiscarder in project flink by apache.

the class PythonPlanBinder method createHashPartitionOperation.

@SuppressWarnings("unchecked")
private void createHashPartitionOperation(PythonOperationInfo info) throws IOException {
    DataSet op1 = (DataSet) sets.get(info.parentID);
    sets.put(info.setID, op1.partitionByHash(info.keys).setParallelism(getParallelism(info)).map(new KeyDiscarder()).setParallelism(getParallelism(info)).name("HashPartitionPostStep"));
}

Example 2 with KeyDiscarder

use of org.apache.flink.python.api.functions.util.KeyDiscarder in project flink by apache.

the class PythonPlanBinder method createDistinctOperation.

@SuppressWarnings("unchecked")
private void createDistinctOperation(PythonOperationInfo info) throws IOException {
    DataSet op = (DataSet) sets.get(info.parentID);
    sets.put(info.setID, op.distinct(info.keys).setParallelism(getParallelism(info)).name("Distinct").map(new KeyDiscarder()).setParallelism(getParallelism(info)).name("DistinctPostStep"));
}