Search in sources :

Example 16 with LongSumAggregator

use of org.apache.flink.api.common.aggregators.LongSumAggregator in project flink by apache.

the class AggregatorsITCase method testAggregatorWithoutParameterForIterate.

@Test
public void testAggregatorWithoutParameterForIterate() throws Exception {
    /*
		 * Test aggregator without parameter for iterate
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    env.setParallelism(parallelism);
    DataSet<Integer> initialSolutionSet = CollectionDataSets.getIntegerDataSet(env);
    IterativeDataSet<Integer> iteration = initialSolutionSet.iterate(MAX_ITERATIONS);
    // register aggregator
    LongSumAggregator aggr = new LongSumAggregator();
    iteration.registerAggregator(NEGATIVE_ELEMENTS_AGGR, aggr);
    // register convergence criterion
    iteration.registerAggregationConvergenceCriterion(NEGATIVE_ELEMENTS_AGGR, aggr, new NegativeElementsConvergenceCriterion());
    DataSet<Integer> updatedDs = iteration.map(new SubtractOneMap());
    iteration.closeWith(updatedDs).writeAsText(resultPath);
    env.execute();
    expected = "-3\n" + "-2\n" + "-2\n" + "-1\n" + "-1\n" + "-1\n" + "0\n" + "0\n" + "0\n" + "0\n" + "1\n" + "1\n" + "1\n" + "1\n" + "1\n";
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) LongSumAggregator(org.apache.flink.api.common.aggregators.LongSumAggregator) Test(org.junit.Test)

Aggregations

LongSumAggregator (org.apache.flink.api.common.aggregators.LongSumAggregator)16 ExecutionEnvironment (org.apache.flink.api.java.ExecutionEnvironment)15 Test (org.junit.Test)15 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)10 Vertex (org.apache.flink.graph.Vertex)6 DataSet (org.apache.flink.api.java.DataSet)5 DiscardingOutputFormat (org.apache.flink.api.java.io.DiscardingOutputFormat)5 DeltaIterationResultSet (org.apache.flink.api.java.operators.DeltaIterationResultSet)4 TwoInputUdfOperator (org.apache.flink.api.java.operators.TwoInputUdfOperator)4 Tuple3 (org.apache.flink.api.java.tuple.Tuple3)4 Graph (org.apache.flink.graph.Graph)4 Plan (org.apache.flink.api.common.Plan)2 JobVertex (org.apache.flink.runtime.jobgraph.JobVertex)2 Method (java.lang.reflect.Method)1 Iterator (java.util.Iterator)1 InvalidProgramException (org.apache.flink.api.common.InvalidProgramException)1 AggregatorRegistry (org.apache.flink.api.common.aggregators.AggregatorRegistry)1 AggregatorWithName (org.apache.flink.api.common.aggregators.AggregatorWithName)1 FilterFunction (org.apache.flink.api.common.functions.FilterFunction)1 MapFunction (org.apache.flink.api.common.functions.MapFunction)1