Search in sources :

Example 41 with PackagedProgram

use of org.apache.flink.client.program.PackagedProgram in project flink by apache.

the class DumpCompiledPlanTest method verifyOptimizedPlan.

private void verifyOptimizedPlan(Class<?> entrypoint, String... args) throws Exception {
    final PackagedProgram program = PackagedProgram.newBuilder().setEntryPointClassName(entrypoint.getName()).setArguments(args).build();
    final Pipeline pipeline = PackagedProgramUtils.getPipelineFromProgram(program, new Configuration(), 1, true);
    assertTrue(pipeline instanceof Plan);
    final Plan plan = (Plan) pipeline;
    final OptimizedPlan op = compileNoStats(plan);
    final PlanJSONDumpGenerator dumper = new PlanJSONDumpGenerator();
    final String json = dumper.getOptimizerPlanAsJSON(op);
    try (JsonParser parser = new JsonFactory().createParser(json)) {
        while (parser.nextToken() != null) {
        }
    }
}
Also used : PackagedProgram(org.apache.flink.client.program.PackagedProgram) PlanJSONDumpGenerator(org.apache.flink.optimizer.plandump.PlanJSONDumpGenerator) Configuration(org.apache.flink.configuration.Configuration) JsonFactory(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonFactory) Plan(org.apache.flink.api.common.Plan) OptimizedPlan(org.apache.flink.optimizer.plan.OptimizedPlan) Pipeline(org.apache.flink.api.dag.Pipeline) OptimizedPlan(org.apache.flink.optimizer.plan.OptimizedPlan) JsonParser(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParser)

Example 42 with PackagedProgram

use of org.apache.flink.client.program.PackagedProgram in project flink by apache.

the class PreviewPlanDumpTest method verifyPlanDump.

private static void verifyPlanDump(Class<?> entrypoint, String... args) throws Exception {
    final PackagedProgram program = PackagedProgram.newBuilder().setEntryPointClassName(entrypoint.getName()).setArguments(args).build();
    final Pipeline pipeline = PackagedProgramUtils.getPipelineFromProgram(program, new Configuration(), 1, true);
    assertTrue(pipeline instanceof Plan);
    final Plan plan = (Plan) pipeline;
    final List<DataSinkNode> sinks = Optimizer.createPreOptimizedPlan(plan);
    final PlanJSONDumpGenerator dumper = new PlanJSONDumpGenerator();
    final String json = dumper.getPactPlanAsJSON(sinks);
    try (JsonParser parser = new JsonFactory().createParser(json)) {
        while (parser.nextToken() != null) {
        }
    }
}
Also used : PackagedProgram(org.apache.flink.client.program.PackagedProgram) PlanJSONDumpGenerator(org.apache.flink.optimizer.plandump.PlanJSONDumpGenerator) Configuration(org.apache.flink.configuration.Configuration) DataSinkNode(org.apache.flink.optimizer.dag.DataSinkNode) JsonFactory(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonFactory) Plan(org.apache.flink.api.common.Plan) Pipeline(org.apache.flink.api.dag.Pipeline) JsonParser(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParser)

Aggregations

PackagedProgram (org.apache.flink.client.program.PackagedProgram)42 Test (org.junit.Test)25 File (java.io.File)20 Configuration (org.apache.flink.configuration.Configuration)19 URL (java.net.URL)13 ProgramInvocationException (org.apache.flink.client.program.ProgramInvocationException)12 Path (org.apache.flink.core.fs.Path)12 FileNotFoundException (java.io.FileNotFoundException)7 IOException (java.io.IOException)7 CommandLine (org.apache.commons.cli.CommandLine)6 RunOptions (org.apache.flink.client.cli.RunOptions)5 CompilerException (org.apache.flink.optimizer.CompilerException)5 FlinkException (org.apache.flink.util.FlinkException)5 Pipeline (org.apache.flink.api.dag.Pipeline)4 DataStatistics (org.apache.flink.optimizer.DataStatistics)4 Optimizer (org.apache.flink.optimizer.Optimizer)4 DefaultCostEstimator (org.apache.flink.optimizer.costs.DefaultCostEstimator)4 JobID (org.apache.flink.api.common.JobID)3 ApplicationConfiguration (org.apache.flink.client.deployment.application.ApplicationConfiguration)3 MiniClusterResourceConfiguration (org.apache.flink.runtime.testutils.MiniClusterResourceConfiguration)3