Examples with Resources co.cask.cdap.api.Resources used on opensource projects

Example 11 with Resources

use of co.cask.cdap.api.Resources in project cdap by caskdata.

the class DistributedFlowProgramRunner method setupLaunchConfig.

@Override
protected void setupLaunchConfig(LaunchConfig launchConfig, Program program, ProgramOptions options, CConfiguration cConf, Configuration hConf, File tempDir) {
    // Add runnables
    Map<String, String> args = options.getUserArguments().asMap();
    FlowSpecification flowSpec = getFlowSpecification(program);
    for (Map.Entry<String, FlowletDefinition> entry : flowSpec.getFlowlets().entrySet()) {
        FlowletDefinition flowletDefinition = entry.getValue();
        FlowletSpecification flowletSpec = flowletDefinition.getFlowletSpec();
        String flowletName = entry.getKey();
        Map<String, String> flowletArgs = RuntimeArguments.extractScope(FlowUtils.FLOWLET_SCOPE, flowletName, args);
        Resources resources = SystemArguments.getResources(flowletArgs, flowletSpec.getResources());
        launchConfig.addRunnable(entry.getKey(), new FlowletTwillRunnable(flowletName), resources, flowletDefinition.getInstances());
    }
}

Example 12 with Resources

use of co.cask.cdap.api.Resources in project cdap by caskdata.

the class BatchPhaseSpecTest method testDescription.

@Test
public void testDescription() throws Exception {
    /*
     * source1 --|
     *           |--> sink.connector
     * source2 --|
     */
    PipelinePhase.Builder builder = PipelinePhase.builder(ImmutableSet.of(BatchSource.PLUGIN_TYPE, Constants.CONNECTOR_TYPE)).addStage(StageInfo.builder("source1", BatchSource.PLUGIN_TYPE).build()).addStage(StageInfo.builder("source2", BatchSource.PLUGIN_TYPE).addInputSchema("a", Schema.recordOf("stuff", Schema.Field.of("x", Schema.of(Schema.Type.INT)))).build()).addStage(StageInfo.builder("sink.connector", Constants.CONNECTOR_TYPE).build()).addConnection("source1", "sink.connector").addConnection("source2", "sink.connector");
    BatchPhaseSpec phaseSpec = new BatchPhaseSpec("phase-1", builder.build(), new Resources(), new Resources(), new Resources(), false, false, Collections.<String, String>emptyMap(), 0, Collections.<String, String>emptyMap());
    Assert.assertEquals("Sources 'source1', 'source2' to sinks 'sink.connector'.", phaseSpec.getDescription());
}

Example 13 with Resources

use of co.cask.cdap.api.Resources in project cdap by caskdata.

the class ScoreCounter method configure.

@Override
public void configure() {
    setDescription("reads game results and counts statistics per team");
    setMapperResources(new Resources(512));
}

Example 14 with Resources

use of co.cask.cdap.api.Resources in project cdap by caskdata.

the class WikipediaDataDownloader method configure.

@Override
protected void configure() {
    setName(NAME);
    setDescription("A MapReduce program that downloads Wikipedia data and stores it into a dataset.");
    setMapperResources(new Resources(512));
}

Example 15 with Resources

use of co.cask.cdap.api.Resources in project cdap by caskdata.

the class TopNMapReduce method configure.

@Override
protected void configure() {
    setName(NAME);
    setDescription("A MapReduce job that returns the top-n words in a dataset.");
    setMapperResources(new Resources(512));
    setReducerResources(new Resources(512));
}