Search in sources :

Example 1 with Plan

use of org.apache.flink.api.common.Plan in project flink by apache.

the class RemoteEnvironment method execute.

// ------------------------------------------------------------------------
@Override
public JobExecutionResult execute(String jobName) throws Exception {
    PlanExecutor executor = getExecutor();
    Plan p = createProgramPlan(jobName);
    // Session management is disabled, revert this commit to enable
    //p.setJobId(jobID);
    //p.setSessionTimeout(sessionTimeout);
    JobExecutionResult result = executor.executePlan(p);
    this.lastJobExecutionResult = result;
    return result;
}
Also used : JobExecutionResult(org.apache.flink.api.common.JobExecutionResult) PlanExecutor(org.apache.flink.api.common.PlanExecutor) Plan(org.apache.flink.api.common.Plan)

Example 2 with Plan

use of org.apache.flink.api.common.Plan in project flink by apache.

the class SemanticPropertiesPrecedenceTest method testFunctionSkipCodeAnalysisAnnotationPrecedence.

@Test
public void testFunctionSkipCodeAnalysisAnnotationPrecedence() {
    ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    env.getConfig().setCodeAnalysisMode(CodeAnalysisMode.OPTIMIZE);
    @SuppressWarnings("unchecked") DataSet<Tuple3<Long, String, Integer>> input = env.fromElements(Tuple3.of(3l, "test", 42));
    input.map(new WildcardForwardedMapperWithSkipAnnotation<Tuple3<Long, String, Integer>>()).output(new DiscardingOutputFormat<Tuple3<Long, String, Integer>>());
    Plan plan = env.createProgramPlan();
    GenericDataSinkBase<?> sink = plan.getDataSinks().iterator().next();
    MapOperatorBase<?, ?, ?> mapper = (MapOperatorBase<?, ?, ?>) sink.getInput();
    SingleInputSemanticProperties semantics = mapper.getSemanticProperties();
    FieldSet fw1 = semantics.getForwardingTargetFields(0, 0);
    FieldSet fw2 = semantics.getForwardingTargetFields(0, 1);
    FieldSet fw3 = semantics.getForwardingTargetFields(0, 2);
    assertNotNull(fw1);
    assertNotNull(fw2);
    assertNotNull(fw3);
    assertFalse(fw1.contains(0));
    assertFalse(fw2.contains(1));
    assertFalse(fw3.contains(2));
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Plan(org.apache.flink.api.common.Plan) MapOperatorBase(org.apache.flink.api.common.operators.base.MapOperatorBase) FieldSet(org.apache.flink.api.common.operators.util.FieldSet) Tuple3(org.apache.flink.api.java.tuple.Tuple3) SingleInputSemanticProperties(org.apache.flink.api.common.operators.SingleInputSemanticProperties) Test(org.junit.Test)

Example 3 with Plan

use of org.apache.flink.api.common.Plan in project flink by apache.

the class SemanticPropertiesPrecedenceTest method testFunctionAnalyzerPrecedence.

@Test
public void testFunctionAnalyzerPrecedence() {
    ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    env.getConfig().setCodeAnalysisMode(CodeAnalysisMode.OPTIMIZE);
    @SuppressWarnings("unchecked") DataSet<Tuple3<Long, String, Integer>> input = env.fromElements(Tuple3.of(3l, "test", 42));
    input.map(new WildcardForwardedMapper<Tuple3<Long, String, Integer>>()).output(new DiscardingOutputFormat<Tuple3<Long, String, Integer>>());
    Plan plan = env.createProgramPlan();
    GenericDataSinkBase<?> sink = plan.getDataSinks().iterator().next();
    MapOperatorBase<?, ?, ?> mapper = (MapOperatorBase<?, ?, ?>) sink.getInput();
    SingleInputSemanticProperties semantics = mapper.getSemanticProperties();
    FieldSet fw1 = semantics.getForwardingTargetFields(0, 0);
    FieldSet fw2 = semantics.getForwardingTargetFields(0, 1);
    FieldSet fw3 = semantics.getForwardingTargetFields(0, 2);
    assertNotNull(fw1);
    assertNotNull(fw2);
    assertNotNull(fw3);
    assertTrue(fw1.contains(0));
    assertTrue(fw2.contains(1));
    assertTrue(fw3.contains(2));
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Plan(org.apache.flink.api.common.Plan) MapOperatorBase(org.apache.flink.api.common.operators.base.MapOperatorBase) FieldSet(org.apache.flink.api.common.operators.util.FieldSet) Tuple3(org.apache.flink.api.java.tuple.Tuple3) SingleInputSemanticProperties(org.apache.flink.api.common.operators.SingleInputSemanticProperties) Test(org.junit.Test)

Example 4 with Plan

use of org.apache.flink.api.common.Plan in project flink by apache.

the class ContextEnvironment method getExecutionPlan.

@Override
public String getExecutionPlan() throws Exception {
    Plan plan = createProgramPlan("unnamed job");
    OptimizedPlan op = ClusterClient.getOptimizedPlan(client.compiler, plan, getParallelism());
    PlanJSONDumpGenerator gen = new PlanJSONDumpGenerator();
    return gen.getOptimizerPlanAsJSON(op);
}
Also used : PlanJSONDumpGenerator(org.apache.flink.optimizer.plandump.PlanJSONDumpGenerator) Plan(org.apache.flink.api.common.Plan) OptimizedPlan(org.apache.flink.optimizer.plan.OptimizedPlan) OptimizedPlan(org.apache.flink.optimizer.plan.OptimizedPlan)

Example 5 with Plan

use of org.apache.flink.api.common.Plan in project flink by apache.

the class DetachedEnvironment method execute.

@Override
public JobExecutionResult execute(String jobName) throws Exception {
    Plan p = createProgramPlan(jobName);
    setDetachedPlan(ClusterClient.getOptimizedPlan(client.compiler, p, getParallelism()));
    LOG.warn("Job was executed in detached mode, the results will be available on completion.");
    this.lastJobExecutionResult = DetachedJobExecutionResult.INSTANCE;
    return this.lastJobExecutionResult;
}
Also used : Plan(org.apache.flink.api.common.Plan) FlinkPlan(org.apache.flink.optimizer.plan.FlinkPlan)

Aggregations

Plan (org.apache.flink.api.common.Plan)290 Test (org.junit.Test)265 ExecutionEnvironment (org.apache.flink.api.java.ExecutionEnvironment)244 OptimizedPlan (org.apache.flink.optimizer.plan.OptimizedPlan)215 SinkPlanNode (org.apache.flink.optimizer.plan.SinkPlanNode)140 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)103 Tuple3 (org.apache.flink.api.java.tuple.Tuple3)89 DualInputPlanNode (org.apache.flink.optimizer.plan.DualInputPlanNode)78 SingleInputPlanNode (org.apache.flink.optimizer.plan.SingleInputPlanNode)77 JobGraphGenerator (org.apache.flink.optimizer.plantranslate.JobGraphGenerator)48 SourcePlanNode (org.apache.flink.optimizer.plan.SourcePlanNode)46 FieldSet (org.apache.flink.api.common.operators.util.FieldSet)37 DiscardingOutputFormat (org.apache.flink.api.java.io.DiscardingOutputFormat)32 InvalidProgramException (org.apache.flink.api.common.InvalidProgramException)30 FieldList (org.apache.flink.api.common.operators.util.FieldList)27 IdentityMapper (org.apache.flink.optimizer.testfunctions.IdentityMapper)26 GlobalProperties (org.apache.flink.optimizer.dataproperties.GlobalProperties)25 LocalProperties (org.apache.flink.optimizer.dataproperties.LocalProperties)25 Channel (org.apache.flink.optimizer.plan.Channel)19 MapOperatorBase (org.apache.flink.api.common.operators.base.MapOperatorBase)18