Examples with ExecutionPlan org.apache.samza.execution.ExecutionPlan used on opensource projects

Example 6 with ExecutionPlan

use of org.apache.samza.execution.ExecutionPlan in project samza by apache.

the class LocalApplicationRunner method run.

@Override
public void run(StreamApplication app) {
    try {
        // 1. initialize and plan
        ExecutionPlan plan = getExecutionPlan(app);
        writePlanJsonFile(plan.getPlanAsJson());
        // 2. create the necessary streams
        createStreams(plan.getIntermediateStreams());
        // 3. create the StreamProcessors
        if (plan.getJobConfigs().isEmpty()) {
            throw new SamzaException("No jobs to run.");
        }
        plan.getJobConfigs().forEach(jobConfig -> {
            log.debug("Starting job {} StreamProcessor with config {}", jobConfig.getName(), jobConfig);
            LocalStreamProcessorLifeCycleListener listener = new LocalStreamProcessorLifeCycleListener();
            StreamProcessor processor = createStreamProcessor(jobConfig, app, listener);
            listener.setProcessor(processor);
            processors.add(processor);
        });
        numProcessorsToStart.set(processors.size());
        // 4. start the StreamProcessors
        processors.forEach(StreamProcessor::start);
    } catch (Exception e) {
        throw new SamzaException("Failed to start application", e);
    }
}

Example 7 with ExecutionPlan

use of org.apache.samza.execution.ExecutionPlan in project samza by apache.

the class RemoteApplicationRunner method run.

/**
   * Run the {@link StreamApplication} on the remote cluster
   * @param app a StreamApplication
   */
@Override
public void run(StreamApplication app) {
    try {
        // 1. initialize and plan
        ExecutionPlan plan = getExecutionPlan(app);
        writePlanJsonFile(plan.getPlanAsJson());
        // 2. create the necessary streams
        getStreamManager().createStreams(plan.getIntermediateStreams());
        // 3. submit jobs for remote execution
        plan.getJobConfigs().forEach(jobConfig -> {
            log.info("Starting job {} with config {}", jobConfig.getName(), jobConfig);
            JobRunner runner = new JobRunner(jobConfig);
            runner.run(true);
        });
    } catch (Throwable t) {
        throw new SamzaException("Failed to run application", t);
    }
}

Example 8 with ExecutionPlan

use of org.apache.samza.execution.ExecutionPlan in project samza by apache.

the class RemoteApplicationRunner method kill.

@Override
public void kill(StreamApplication app) {
    try {
        ExecutionPlan plan = getExecutionPlan(app);
        plan.getJobConfigs().forEach(jobConfig -> {
            log.info("Killing job {}", jobConfig.getName());
            JobRunner runner = new JobRunner(jobConfig);
            runner.kill();
        });
    } catch (Throwable t) {
        throw new SamzaException("Failed to kill application", t);
    }
}