Search in sources :

Example 1 with RangeBoundaries

use of org.apache.flink.api.common.distributions.RangeBoundaries in project flink by apache.

the class AssignRangeIndex method mapPartition.

@Override
public void mapPartition(Iterable<IN> values, Collector<Tuple2<Integer, IN>> out) throws Exception {
    List<Object> broadcastVariable = getRuntimeContext().getBroadcastVariable("RangeBoundaries");
    if (broadcastVariable == null || broadcastVariable.size() != 1) {
        throw new RuntimeException("AssignRangePartition require a single RangeBoundaries as broadcast input.");
    }
    Object[][] boundaryObjects = (Object[][]) broadcastVariable.get(0);
    RangeBoundaries rangeBoundaries = new CommonRangeBoundaries(typeComparator.createComparator(), boundaryObjects);
    Tuple2<Integer, IN> tupleWithPartitionId = new Tuple2<>();
    for (IN record : values) {
        tupleWithPartitionId.f0 = rangeBoundaries.getRangeIndex(record);
        tupleWithPartitionId.f1 = record;
        out.collect(tupleWithPartitionId);
    }
}
Also used : CommonRangeBoundaries(org.apache.flink.api.common.distributions.CommonRangeBoundaries) RangeBoundaries(org.apache.flink.api.common.distributions.RangeBoundaries) Tuple2(org.apache.flink.api.java.tuple.Tuple2) CommonRangeBoundaries(org.apache.flink.api.common.distributions.CommonRangeBoundaries)

Aggregations

CommonRangeBoundaries (org.apache.flink.api.common.distributions.CommonRangeBoundaries)1 RangeBoundaries (org.apache.flink.api.common.distributions.RangeBoundaries)1 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)1