Search in sources :

Example 1 with UnableToStartException

use of org.apache.metron.integration.UnableToStartException in project metron by apache.

the class ZKServerComponent method start.

@Override
public void start() throws UnableToStartException {
    try {
        testZkServer = new TestingServer(true);
        zookeeperUrl = testZkServer.getConnectString();
        if (postStartCallback.isPresent()) {
            postStartCallback.get().accept(this);
        }
    } catch (Exception e) {
        throw new UnableToStartException("Unable to start TestingServer", e);
    }
}
Also used : TestingServer(org.apache.curator.test.TestingServer) UnableToStartException(org.apache.metron.integration.UnableToStartException) UnableToStartException(org.apache.metron.integration.UnableToStartException) IOException(java.io.IOException)

Example 2 with UnableToStartException

use of org.apache.metron.integration.UnableToStartException in project metron by apache.

the class SolrComponent method start.

@Override
public void start() throws UnableToStartException {
    try {
        File baseDir = Files.createTempDirectory("solrcomponent").toFile();
        baseDir.deleteOnExit();
        miniSolrCloudCluster = new MiniSolrCloudCluster(1, baseDir.toPath(), JettyConfig.builder().setPort(port).build());
        for (String name : collections.keySet()) {
            String configPath = collections.get(name);
            miniSolrCloudCluster.uploadConfigSet(new File(configPath).toPath(), name);
            CollectionAdminRequest.createCollection(name, 1, 1).process(miniSolrCloudCluster.getSolrClient());
        }
        if (postStartCallback != null) {
            postStartCallback.apply(this);
        }
    } catch (Exception e) {
        throw new UnableToStartException(e.getMessage(), e);
    }
}
Also used : UnableToStartException(org.apache.metron.integration.UnableToStartException) MiniSolrCloudCluster(org.apache.solr.cloud.MiniSolrCloudCluster) File(java.io.File) UnableToStartException(org.apache.metron.integration.UnableToStartException) SolrServerException(org.apache.solr.client.solrj.SolrServerException) KeeperException(org.apache.zookeeper.KeeperException) IOException(java.io.IOException)

Example 3 with UnableToStartException

use of org.apache.metron.integration.UnableToStartException in project metron by apache.

the class ElasticSearchComponent method start.

@Override
public void start() throws UnableToStartException {
    File logDir = new File(indexDir, "/logs");
    File dataDir = new File(indexDir, "/data");
    try {
        cleanDir(logDir);
        cleanDir(dataDir);
    } catch (IOException e) {
        throw new UnableToStartException("Unable to clean log or data directories", e);
    }
    Settings.Builder settingsBuilder = Settings.builder().put("cluster.name", "metron").put("path.logs", logDir.getAbsolutePath()).put("path.data", dataDir.getAbsolutePath()).put("path.home", indexDir.getAbsoluteFile()).put("transport.type", "netty4").put("http.enabled", "true");
    if (extraElasticSearchSettings != null) {
        settingsBuilder = settingsBuilder.put(extraElasticSearchSettings);
    }
    node = new TestNode(settingsBuilder.build(), asList(Netty4Plugin.class));
    client = node.client();
    try {
        node.start();
    } catch (NodeValidationException e) {
        throw new UnableToStartException("Error starting ES node.", e);
    }
    waitForCluster(client, ClusterHealthStatus.YELLOW, STARTUP_TIMEOUT);
    for (Mapping m : Optional.ofNullable(mappings).orElse(new ArrayList<>())) {
        client.admin().indices().prepareCreate(m.index).addMapping(m.docType, m.mapping).get();
    }
    indexDao = new ElasticsearchDao().withRefreshPolicy(WriteRequest.RefreshPolicy.WAIT_UNTIL);
    indexDao.init(accessConfig);
}
Also used : NodeValidationException(org.elasticsearch.node.NodeValidationException) ElasticsearchDao(org.apache.metron.elasticsearch.dao.ElasticsearchDao) UnableToStartException(org.apache.metron.integration.UnableToStartException) IOException(java.io.IOException) File(java.io.File) Settings(org.elasticsearch.common.settings.Settings)

Example 4 with UnableToStartException

use of org.apache.metron.integration.UnableToStartException in project metron by apache.

the class TestConfig method componentRunner.

@Bean(destroyMethod = "stop")
public ComponentRunner componentRunner(ZKServerComponent zkServerComponent, KafkaComponent kafkaWithZKComponent) {
    ComponentRunner runner = new ComponentRunner.Builder().withComponent("zk", zkServerComponent).withCustomShutdownOrder(new String[] { "search", "zk" }).build();
    try {
        runner.start();
        File globalConfigFile = new File("src/test/resources/zookeeper/global.json");
        try (BufferedReader r = new BufferedReader(new InputStreamReader(new FileInputStream(globalConfigFile), StandardCharsets.UTF_8))) {
            String globalConfig = IOUtils.toString(r);
            ConfigurationsUtils.writeGlobalConfigToZookeeper(globalConfig.getBytes(StandardCharsets.UTF_8), zkServerComponent.getConnectionString());
        } catch (Exception e) {
            throw new IllegalStateException("Unable to upload global config", e);
        }
    } catch (UnableToStartException e) {
        e.printStackTrace();
    }
    return runner;
}
Also used : UnableToStartException(org.apache.metron.integration.UnableToStartException) ComponentRunner(org.apache.metron.integration.ComponentRunner) UnableToStartException(org.apache.metron.integration.UnableToStartException) RestException(org.apache.metron.rest.RestException) Bean(org.springframework.context.annotation.Bean)

Example 5 with UnableToStartException

use of org.apache.metron.integration.UnableToStartException in project metron by apache.

the class ParserTopologyComponent method start.

@Override
public void start() throws UnableToStartException {
    try {
        final Map<String, Object> stormConf = new HashMap<>();
        stormConf.put(Config.TOPOLOGY_DEBUG, true);
        ParserTopologyBuilder.ParserTopology topologyBuilder = ParserTopologyBuilder.build(topologyProperties.getProperty(ZKServerComponent.ZOOKEEPER_PROPERTY), Optional.ofNullable(brokerUrl), sensorTypes, (x, y) -> Collections.nCopies(sensorTypes.size(), 1), (x, y) -> Collections.nCopies(sensorTypes.size(), 1), (x, y) -> 1, (x, y) -> 1, (x, y) -> 1, (x, y) -> 1, (x, y) -> Collections.nCopies(sensorTypes.size(), new HashMap<>()), (x, y) -> null, (x, y) -> outputTopic, (x, y) -> errorTopic, (x, y) -> {
            Config c = new Config();
            c.putAll(stormConf);
            return c;
        });
        stormCluster = new LocalCluster();
        stormCluster.submitTopology(getTopologyName(), stormConf, topologyBuilder.getBuilder().createTopology());
    } catch (Exception e) {
        throw new UnableToStartException("Unable to start parser topology for sensorTypes: " + sensorTypes, e);
    }
}
Also used : LocalCluster(org.apache.storm.LocalCluster) HashMap(java.util.HashMap) Config(org.apache.storm.Config) ParserTopologyBuilder(org.apache.metron.parsers.topology.ParserTopologyBuilder) UnableToStartException(org.apache.metron.integration.UnableToStartException) UnableToStartException(org.apache.metron.integration.UnableToStartException)

Aggregations

UnableToStartException (org.apache.metron.integration.UnableToStartException)7 IOException (java.io.IOException)3 File (java.io.File)2 LocalCluster (org.apache.storm.LocalCluster)2 InvocationTargetException (java.lang.reflect.InvocationTargetException)1 URL (java.net.URL)1 HashMap (java.util.HashMap)1 RetryPolicy (org.apache.curator.RetryPolicy)1 CuratorFramework (org.apache.curator.framework.CuratorFramework)1 ExponentialBackoffRetry (org.apache.curator.retry.ExponentialBackoffRetry)1 TestingServer (org.apache.curator.test.TestingServer)1 Configuration (org.apache.hadoop.conf.Configuration)1 FileContext (org.apache.hadoop.fs.FileContext)1 Path (org.apache.hadoop.fs.Path)1 YarnConfiguration (org.apache.hadoop.yarn.conf.YarnConfiguration)1 MiniYARNCluster (org.apache.hadoop.yarn.server.MiniYARNCluster)1 CapacityScheduler (org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler)1 ElasticsearchDao (org.apache.metron.elasticsearch.dao.ElasticsearchDao)1 ComponentRunner (org.apache.metron.integration.ComponentRunner)1 ParserTopologyBuilder (org.apache.metron.parsers.topology.ParserTopologyBuilder)1