Examples with BatchSourceConfig org.apache.pulsar.common.io.BatchSourceConfig used on opensource projects

Example 11 with BatchSourceConfig

use of org.apache.pulsar.common.io.BatchSourceConfig in project incubator-pulsar by apache.

the class BatchDataGeneratorExec method main.

public static void main(final String[] args) throws Exception {
    final String cronString = "0 0/5 * * * ?";
    final Map<String, Object> discoveryConfig = new HashMap<>();
    discoveryConfig.put(CronTriggerer.CRON_KEY, cronString);
    final BatchSourceConfig batchSourceConfig = BatchSourceConfig.builder().discoveryTriggererClassName(CronTriggerer.class.getName()).discoveryTriggererConfig(discoveryConfig).build();
    final SourceConfig sourceConfig = SourceConfig.builder().batchSourceConfig(batchSourceConfig).className(BatchDataGeneratorSource.class.getName()).configs(new HashMap<>()).name("BatchDataGenerator").parallelism(1).topicName("persistent://public/default/batchdatagenerator").build();
    final LocalRunner localRunner = LocalRunner.builder().brokerServiceUrl("pulsar://localhost:6650").sourceConfig(sourceConfig).build();
    localRunner.start(false);
    TimeUnit.MINUTES.sleep(30);
    localRunner.stop();
    System.exit(0);
}

Example 12 with BatchSourceConfig

use of org.apache.pulsar.common.io.BatchSourceConfig in project incubator-pulsar by apache.

the class BatchSourceExecutorTest method createBatchSourceConfig.

private static BatchSourceConfig createBatchSourceConfig() {
    BatchSourceConfig testBatchConfig = new BatchSourceConfig();
    testBatchConfig.setDiscoveryTriggererClassName(TestDiscoveryTriggerer.class.getName());
    Map<String, Object> triggererConfig = new HashMap<>();
    triggererConfig.put("DELAY_MS", 500);
    testBatchConfig.setDiscoveryTriggererConfig(triggererConfig);
    return testBatchConfig;
}

Example 13 with BatchSourceConfig

use of org.apache.pulsar.common.io.BatchSourceConfig in project incubator-pulsar by apache.

the class SourceConfigUtilsTest method createBatchSourceConfig.

private BatchSourceConfig createBatchSourceConfig() {
    BatchSourceConfig batchSourceConfig = new BatchSourceConfig();
    batchSourceConfig.setDiscoveryTriggererClassName(TestTriggerer.class.getName());
    Map<String, Object> batchConfig = new HashMap<>();
    batchConfig.put("foo", "bar");
    batchSourceConfig.setDiscoveryTriggererConfig(batchConfig);
    return batchSourceConfig;
}

Example 14 with BatchSourceConfig

use of org.apache.pulsar.common.io.BatchSourceConfig in project incubator-pulsar by apache.

the class SourceConfigUtilsTest method createSourceConfigWithBatch.

private SourceConfig createSourceConfigWithBatch() {
    SourceConfig sourceConfig = createSourceConfig();
    BatchSourceConfig batchSourceConfig = createBatchSourceConfig();
    sourceConfig.setBatchSourceConfig(batchSourceConfig);
    return sourceConfig;
}

Example 15 with BatchSourceConfig

use of org.apache.pulsar.common.io.BatchSourceConfig in project incubator-pulsar by apache.

the class SourceConfigUtilsTest method testMergeDifferentBatchTriggerer.

@Test(expectedExceptions = IllegalArgumentException.class, expectedExceptionsMessageRegExp = "DiscoverTriggerer class cannot be updated for batchsources")
public void testMergeDifferentBatchTriggerer() {
    SourceConfig sourceConfig = createSourceConfigWithBatch();
    BatchSourceConfig batchSourceConfig = createBatchSourceConfig();
    batchSourceConfig.setDiscoveryTriggererClassName("SomeOtherClassName");
    SourceConfig newSourceConfig = createUpdatedSourceConfig("batchSourceConfig", batchSourceConfig);
    SourceConfigUtils.validateUpdate(sourceConfig, newSourceConfig);
}