Search in sources :

Example 1 with RichFilterFunction

use of org.apache.flink.api.common.functions.RichFilterFunction in project flink by apache.

the class AggregatorsITCase method testDistributedCacheWithIterations.

@Test
public void testDistributedCacheWithIterations() throws Exception {
    File tempFile = new File(testPath);
    try (FileWriter writer = new FileWriter(tempFile)) {
        writer.write(testString);
    }
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    env.registerCachedFile(resultPath, testName);
    IterativeDataSet<Long> solution = env.fromElements(1L).iterate(2);
    solution.closeWith(env.generateSequence(1, 2).filter(new RichFilterFunction<Long>() {

        @Override
        public void open(Configuration parameters) throws Exception {
            File file = getRuntimeContext().getDistributedCache().getFile(testName);
            BufferedReader reader = new BufferedReader(new FileReader(file));
            String output = reader.readLine();
            reader.close();
            assertEquals(output, testString);
        }

        @Override
        public boolean filter(Long value) throws Exception {
            return false;
        }
    }).withBroadcastSet(solution, "SOLUTION")).output(new DiscardingOutputFormat<Long>());
    env.execute();
    // this will be a useless verification now.
    expected = testString;
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Configuration(org.apache.flink.configuration.Configuration) FileWriter(java.io.FileWriter) RichFilterFunction(org.apache.flink.api.common.functions.RichFilterFunction) BufferedReader(java.io.BufferedReader) FileReader(java.io.FileReader) File(java.io.File) Test(org.junit.Test)

Aggregations

BufferedReader (java.io.BufferedReader)1 File (java.io.File)1 FileReader (java.io.FileReader)1 FileWriter (java.io.FileWriter)1 RichFilterFunction (org.apache.flink.api.common.functions.RichFilterFunction)1 ExecutionEnvironment (org.apache.flink.api.java.ExecutionEnvironment)1 Configuration (org.apache.flink.configuration.Configuration)1 Test (org.junit.Test)1