Examples with KafkaServer kafka.server.KafkaServer used on opensource projects

Example 6 with KafkaServer

use of kafka.server.KafkaServer in project opennms by OpenNMS.

the class JUnitKafkaServer method before.

@Override
public void before() throws Exception {
    zkServer = new TestingServer();
    // Start ZooKeeper, this method will block until the service has started
    zkServer.start();
    getAvailablePort(kafkaPort, 9192);
    // Delete any existing Kafka log directory
    FileUtils.deleteDirectory(new File("target/kafka-log"));
    localhost = getLocalhost();
    final Properties properties = new Properties();
    properties.put("broker.id", "1");
    properties.put("auto.create.topics.enable", "true");
    properties.put("num.partitions", "100");
    properties.put("enable.zookeeper", "true");
    properties.put("host.name", localhost);
    properties.put("log.dir", "target/kafka-log");
    properties.put("port", String.valueOf(kafkaPort.get()));
    properties.put("zookeeper.connect", zkServer.getConnectString());
    properties.put("listeners", "PLAINTEXT://" + "localhost:" + String.valueOf(kafkaPort.get()));
    System.err.println("Kafka server properties: " + properties);
    kafkaConfig = new KafkaConfig(properties);
    final List<KafkaMetricsReporter> kmrList = new ArrayList<>();
    final Buffer<KafkaMetricsReporter> metricsList = scala.collection.JavaConversions.asScalaBuffer(kmrList);
    kafkaServer = new KafkaServer(kafkaConfig, new SystemTime(), Option.<String>empty(), metricsList);
    kafkaServer.startup();
    await().atMost(1, MINUTES).until(this::getBrokerMetadatas, hasSize(greaterThanOrEqualTo(1)));
    System.err.println("Kafka Address: " + getKafkaConnectString());
    System.err.println("Zookeeper Address: " + getZookeeperConnectString());
}

Example 7 with KafkaServer

use of kafka.server.KafkaServer in project ignite by apache.

the class TestKafkaBroker method createTopic.

/**
 * Creates a topic.
 *
 * @param topic Topic name.
 * @param partitions Number of partitions for the topic.
 * @param replicationFactor Replication factor.
 * @throws TimeoutException If operation is timed out.
 * @throws InterruptedException If interrupted.
 */
public void createTopic(String topic, int partitions, int replicationFactor) throws TimeoutException, InterruptedException {
    List<KafkaServer> servers = new ArrayList<>();
    servers.add(kafkaSrv);
    TestUtils.createTopic(zkUtils, topic, partitions, replicationFactor, scala.collection.JavaConversions.asScalaBuffer(servers), new Properties());
}

Example 8 with KafkaServer

use of kafka.server.KafkaServer in project incubator-gobblin by apache.

the class KafkaClusterTestBase method startCluster.

public void startCluster() {
    // Start Zookeeper.
    _zkServer = new EmbeddedZookeeper();
    _zkConnectString = "127.0.0.1:" + _zkServer.port();
    _zkClient = new ZkClient(_zkConnectString, 30000, 30000, ZKStringSerializer$.MODULE$);
    // Start Kafka Cluster.
    for (int i = 0; i < clusterCount; i++) {
        KafkaServer _kafkaServer = createKafkaServer(i, _zkConnectString);
        kafkaBrokerList.add(_kafkaServer);
    }
}

Example 9 with KafkaServer

use of kafka.server.KafkaServer in project wildfly-camel by wildfly-extras.

the class EmbeddedKafkaBroker method startup.

public void startup() {
    for (int i = 0; i < ports.size(); i++) {
        Integer port = ports.get(i);
        File logDir = FileUtils.constructTempDir("kafka-local");
        Properties properties = new Properties();
        properties.putAll(baseProperties);
        properties.setProperty("zookeeper.connect", zkConnection);
        properties.setProperty("broker.id", String.valueOf(i + 1));
        properties.setProperty("host.name", "localhost");
        properties.setProperty("port", Integer.toString(port));
        properties.setProperty("log.dir", logDir.getAbsolutePath());
        properties.setProperty("num.partitions", String.valueOf(1));
        properties.setProperty("auto.create.topics.enable", String.valueOf(Boolean.TRUE));
        System.out.println("EmbeddedKafkaBroker: local directory: " + logDir.getAbsolutePath());
        properties.setProperty("log.flush.interval.messages", String.valueOf(1));
        properties.setProperty("offsets.topic.replication.factor", String.valueOf(1));
        KafkaServer broker = startBroker(properties);
        brokers.add(broker);
        logDirs.add(logDir);
    }
}

Example 10 with KafkaServer

use of kafka.server.KafkaServer in project nd4j by deeplearning4j.

the class EmbeddedKafkaCluster method startBroker.

private KafkaServer startBroker(Properties props) {
    KafkaServer server = new KafkaServer(new KafkaConfig(props), new SystemTime());
    server.startup();
    return server;
}