Examples with EcoTopologyDefinition com.twitter.heron.eco.definition.EcoTopologyDefinition used on opensource projects

Example 21 with EcoTopologyDefinition

use of com.twitter.heron.eco.definition.EcoTopologyDefinition in project incubator-heron by apache.

the class ConfigBuilderTest method testBuildConfig_MBAllocationTooSmall_ExceptionThrown.

@Test(expected = IllegalArgumentException.class)
public void testBuildConfig_MBAllocationTooSmall_ExceptionThrown() throws Exception {
    Config config = null;
    try {
        EcoParser ecoParser = new EcoParser();
        InputStream inputStream = new ByteArrayInputStream(INCORRECT_RAM_MEGABYTES.getBytes());
        FileInputStream mockPropsStream = PowerMockito.mock(FileInputStream.class);
        EcoTopologyDefinition ecoTopologyDefinition = ecoParser.parseFromInputStream(inputStream, mockPropsStream, false);
        config = subject.buildConfig(ecoTopologyDefinition);
    } finally {
        assertNull(config);
    }
}

Example 22 with EcoTopologyDefinition

use of com.twitter.heron.eco.definition.EcoTopologyDefinition in project incubator-heron by apache.

the class EcoParserTest method testParseFromInputStream_PropertyFiltering_SubstitutesAsExpected.

@Test
public void testParseFromInputStream_PropertyFiltering_SubstitutesAsExpected() throws Exception {
    InputStream inputStream = new ByteArrayInputStream(PROPERTY_SUBSTITUION_YAML.getBytes());
    InputStream propsStream = new ByteArrayInputStream(SAMPLE_PROPERTIES.getBytes());
    EcoTopologyDefinition ecoTopologyDefinition = subject.parseFromInputStream(inputStream, propsStream, false);
    BoltDefinition bolt = ecoTopologyDefinition.getBolt("ibasic-print-bolt");
    List<Object> args = bolt.getConfigMethods().get(0).getArgs();
    assertThat(args.get(0), is(equalTo("ecoValueOne")));
}

Example 23 with EcoTopologyDefinition

use of com.twitter.heron.eco.definition.EcoTopologyDefinition in project incubator-heron by apache.

the class Eco method printTopologyInfo.

static void printTopologyInfo(EcoExecutionContext ctx) {
    EcoTopologyDefinition t = ctx.getTopologyDefinition();
    LOG.info("---------- TOPOLOGY DETAILS ----------");
    LOG.info(String.format("Topology Name: %s", t.getName()));
    LOG.info("--------------- SPOUTS ---------------");
    for (SpoutDefinition s : t.getSpouts()) {
        LOG.info(String.format("%s [%d] (%s)", s.getId(), s.getParallelism(), s.getClassName()));
    }
    LOG.info("---------------- BOLTS ---------------");
    for (BoltDefinition b : t.getBolts()) {
        LOG.info(String.format("%s [%d] (%s)", b.getId(), b.getParallelism(), b.getClassName()));
    }
    LOG.info("--------------- STREAMS ---------------");
    for (StreamDefinition sd : t.getStreams()) {
        LOG.info(String.format("%s --%s--> %s", sd.getFrom(), sd.getGrouping().getType(), sd.getTo()));
    }
    LOG.info("--------------------------------------");
}

Example 24 with EcoTopologyDefinition

use of com.twitter.heron.eco.definition.EcoTopologyDefinition in project incubator-heron by apache.

the class EcoParser method loadTopologyFromYaml.

private EcoTopologyDefinition loadTopologyFromYaml(Yaml yaml, InputStream inputStream, InputStream propsIn, boolean envFilter) throws IOException {
    LOG.info("Parsing eco config file");
    ByteArrayOutputStream bos = new ByteArrayOutputStream();
    int b;
    while ((b = inputStream.read()) != -1) {
        bos.write(b);
    }
    String yamlDefinitionStr = bos.toString();
    // properties file substitution
    if (propsIn != null) {
        LOG.info("Performing property substitution.");
        Properties props = new Properties();
        props.load(propsIn);
        for (Object key : props.keySet()) {
            yamlDefinitionStr = yamlDefinitionStr.replace("${" + key + "}", props.getProperty((String) key));
        }
    } else {
        LOG.info("Not performing property substitution.");
    }
    // environment variable substitution
    if (envFilter) {
        LOG.info("Performing environment variable substitution.");
        Map<String, String> envs = System.getenv();
        for (String key : envs.keySet()) {
            yamlDefinitionStr = yamlDefinitionStr.replace("${ENV-" + key + "}", envs.get(key));
        }
    } else {
        LOG.info("Not performing environment variable substitution.");
    }
    return (EcoTopologyDefinition) yaml.load(yamlDefinitionStr);
}

Example 25 with EcoTopologyDefinition

use of com.twitter.heron.eco.definition.EcoTopologyDefinition in project incubator-heron by apache.

the class SpoutBuilder method buildSpouts.

protected void buildSpouts(EcoExecutionContext executionContext, TopologyBuilder builder, ObjectBuilder objectBuilder) throws ClassNotFoundException, InstantiationException, IllegalAccessException, NoSuchFieldException, InvocationTargetException {
    EcoTopologyDefinition topologyDefinition = executionContext.getTopologyDefinition();
    for (ObjectDefinition def : topologyDefinition.getSpouts()) {
        Object obj = objectBuilder.buildObject(def, executionContext);
        builder.setSpout(def.getId(), (IRichSpout) obj, def.getParallelism());
        executionContext.addSpout(def.getId(), obj);
    }
}