Search in sources :

Example 1 with PulsarSubscriber

use of org.apache.flink.connector.pulsar.source.enumerator.subscriber.PulsarSubscriber in project flink by apache.

the class PulsarSourceEnumeratorTest method createEnumerator.

private PulsarSourceEnumerator createEnumerator(SubscriptionType subscriptionType, Set<String> topicsToSubscribe, MockSplitEnumeratorContext<PulsarPartitionSplit> enumContext, boolean enablePeriodicPartitionDiscovery, PulsarSourceEnumState sourceEnumState) {
    // Use a TopicPatternSubscriber so that no exception if a subscribed topic hasn't been
    // created yet.
    String topicRegex = String.join("|", topicsToSubscribe);
    Pattern topicPattern = Pattern.compile(topicRegex);
    PulsarSubscriber subscriber = getTopicPatternSubscriber(topicPattern, RegexSubscriptionMode.AllTopics);
    Configuration configuration = operator().config();
    configuration.set(PULSAR_SUBSCRIPTION_TYPE, subscriptionType);
    if (enablePeriodicPartitionDiscovery) {
        configuration.set(PULSAR_PARTITION_DISCOVERY_INTERVAL_MS, 60L);
    } else {
        configuration.set(PULSAR_PARTITION_DISCOVERY_INTERVAL_MS, -1L);
    }
    SourceConfiguration sourceConfiguration = new SourceConfiguration(configuration);
    SplitsAssignmentState assignmentState = new SplitsAssignmentState(latest(), sourceConfiguration, sourceEnumState);
    return new PulsarSourceEnumerator(subscriber, StartCursor.earliest(), new FullRangeGenerator(), sourceConfiguration, enumContext, assignmentState);
}
Also used : Pattern(java.util.regex.Pattern) SourceConfiguration(org.apache.flink.connector.pulsar.source.config.SourceConfiguration) Configuration(org.apache.flink.configuration.Configuration) FullRangeGenerator(org.apache.flink.connector.pulsar.source.enumerator.topic.range.FullRangeGenerator) SourceConfiguration(org.apache.flink.connector.pulsar.source.config.SourceConfiguration) PulsarSubscriber(org.apache.flink.connector.pulsar.source.enumerator.subscriber.PulsarSubscriber)

Aggregations

Pattern (java.util.regex.Pattern)1 Configuration (org.apache.flink.configuration.Configuration)1 SourceConfiguration (org.apache.flink.connector.pulsar.source.config.SourceConfiguration)1 PulsarSubscriber (org.apache.flink.connector.pulsar.source.enumerator.subscriber.PulsarSubscriber)1 FullRangeGenerator (org.apache.flink.connector.pulsar.source.enumerator.topic.range.FullRangeGenerator)1