Examples with TopologyBuilder org.apache.heron.api.topology.TopologyBuilder used on opensource projects

Example 36 with TopologyBuilder

use of org.apache.heron.api.topology.TopologyBuilder in project heron by twitter.

the class BaseWindowedBoltTest method testSettingWaterMarkInterval.

@Test
public void testSettingWaterMarkInterval() {
    final Object[] args = new Object[] { -1L, 0L, 1L, 2L, 5L, 10L, null };
    for (Object arg : args) {
        TopologyBuilder builder = new TopologyBuilder();
        Object arg0 = arg;
        try {
            Duration watermarkInterval = null;
            if (arg0 != null) {
                watermarkInterval = Duration.ofMillis((Long) arg0);
            }
            builder.setBolt("testBolt", new TestBolt().withWatermarkInterval(watermarkInterval));
            if (arg0 == null) {
                fail(String.format("Watermark interval cannot be null -- watermarkInterval: %s", arg0));
            }
            if ((Long) arg0 <= 0) {
                fail(String.format("Watermark interval cannot be zero or less -- watermarkInterval: " + "%s", arg0));
            }
        } catch (IllegalArgumentException e) {
            if (arg0 != null && (Long) arg0 > 0) {
                fail(String.format("Exception: %s thrown on valid input -- watermarkInterval: %s", e.getMessage(), arg0));
            }
        }
    }
}

Example 37 with TopologyBuilder

use of org.apache.heron.api.topology.TopologyBuilder in project heron by twitter.

the class HeronSubmitterTest method testTopologySubmissionWhenTmpDirectoryIsEmptyPath.

@Test(expected = TopologySubmissionException.class)
@PrepareForTest(HeronSubmitter.class)
public void testTopologySubmissionWhenTmpDirectoryIsEmptyPath() throws AlreadyAliveException, InvalidTopologyException {
    TopologyBuilder builder = createTopologyBuilderWithMinimumSetup();
    Config conf = new Config();
    Map<String, String> map = new HashMap();
    map.put("cmdline.topologydefn.tmpdirectory", "");
    PowerMockito.spy(HeronSubmitter.class);
    Mockito.when(HeronSubmitter.getHeronCmdOptions()).thenReturn(map);
    HeronSubmitter.submitTopology("test", conf, builder.createTopology());
}

Example 38 with TopologyBuilder

use of org.apache.heron.api.topology.TopologyBuilder in project heron by twitter.

the class HeronSubmitterTest method testInvalidTopologySubmission.

@Test(expected = InvalidTopologyException.class)
public void testInvalidTopologySubmission() throws AlreadyAliveException, InvalidTopologyException {
    TopologyBuilder builder = new TopologyBuilder();
    int spouts = 2;
    int bolts = 2;
    builder.setSpout("word", new TestSpout(), spouts);
    builder.setBolt("exclaim1", new TestBolt(), bolts).shuffleGrouping("word");
    Config conf = new Config();
    conf.setDebug(true);
    // Put an arbitrary large number here if you don't want to slow the topology down
    conf.setMaxSpoutPending(1000 * 1000 * 1000);
    // To enable acking
    conf.setTopologyReliabilityMode(Config.TopologyReliabilityMode.ATLEAST_ONCE);
    conf.put(Config.TOPOLOGY_WORKER_CHILDOPTS, "-XX:+HeapDumpOnOutOfMemoryError");
    // component resource configuration
    conf.setComponentCpu("word", 0.5);
    conf.setComponentRam("word", ByteAmount.fromMegabytes(10));
    conf.setComponentDisk("word", ByteAmount.fromMegabytes(10));
    conf.setComponentCpu("exclaim1", 0.5);
    conf.setComponentRam("exclaim1", ByteAmount.fromMegabytes(10));
    conf.setComponentDisk("exclaim1", ByteAmount.fromMegabytes(10));
    // container resource configuration
    conf.setContainerDiskRequested(ByteAmount.fromMegabytes(10));
    conf.setContainerRamRequested(ByteAmount.fromMegabytes(10));
    conf.setContainerCpuRequested(1);
    // Set the number of workers or stream managers
    conf.setNumStmgrs(2);
    HeronSubmitter.submitTopology("test", conf, builder.createTopology());
}

Example 39 with TopologyBuilder

use of org.apache.heron.api.topology.TopologyBuilder in project heron by twitter.

the class HeronSubmitterTest method testValidTopologySubmission.

@Test
@PrepareForTest(HeronSubmitter.class)
public void testValidTopologySubmission() throws AlreadyAliveException, InvalidTopologyException {
    TopologyBuilder builder = createTopologyBuilderWithMinimumSetup();
    Config conf = new Config();
    Map<String, String> map = new HashMap();
    map.put("cmdline.topologydefn.tmpdirectory", folder.getRoot().getPath());
    PowerMockito.spy(HeronSubmitter.class);
    Mockito.when(HeronSubmitter.getHeronCmdOptions()).thenReturn(map);
    HeronSubmitter.submitTopology("test", conf, builder.createTopology());
}

Example 40 with TopologyBuilder

use of org.apache.heron.api.topology.TopologyBuilder in project heron by twitter.

the class HeronExecutorTaskTest method createTestTopology.

Topology createTestTopology(String name) {
    TopologyBuilder builder = new TopologyBuilder();
    builder.setSpout("spout-1", new TestSpout(), 2);
    builder.setBolt("bolt-1", new TestBolt(), 1).shuffleGrouping("spout-1");
    HeronTopology topology = builder.createTopology();
    org.apache.heron.api.Config config = new org.apache.heron.api.Config();
    return topology.setName(name).setConfig(config).setState(TopologyState.RUNNING).getTopology();
}