Search in sources :

Example 1 with Hash

use of org.apache.pulsar.client.impl.Hash in project flink by apache.

the class KeyHashTopicRouter method route.

@Override
public String route(IN in, String key, List<String> partitions, PulsarSinkContext context) {
    checkArgument(!partitions.isEmpty(), "You should provide topics for routing topic by message key hash.");
    int topicIndex;
    if (Strings.isNullOrEmpty(key)) {
        // We would randomly pick one topic to write.
        topicIndex = ThreadLocalRandom.current().nextInt(partitions.size());
    } else {
        // Hash the message key and choose the topic to write.
        Hash hash = messageKeyHash.getHash();
        int code = hash.makeHash(key);
        topicIndex = signSafeMod(code, partitions.size());
    }
    return partitions.get(topicIndex);
}
Also used : Hash(org.apache.pulsar.client.impl.Hash)

Example 2 with Hash

use of org.apache.pulsar.client.impl.Hash in project flink by apache.

the class KeyHashTopicRouterTest method routeMessageByMessageKey.

@ParameterizedTest
@EnumSource(MessageKeyHash.class)
void routeMessageByMessageKey(MessageKeyHash keyHash) {
    SinkConfiguration configuration = sinkConfiguration(keyHash);
    String messageKey = randomAlphanumeric(10);
    KeyHashTopicRouter<String> router = new KeyHashTopicRouter<>(configuration);
    List<String> topics = Stream.generate(() -> randomAlphanumeric(10)).distinct().limit(10).collect(toList());
    Hash hash = keyHash.getHash();
    int index = signSafeMod(hash.makeHash(messageKey), topics.size());
    String desiredTopic = topics.get(index);
    String message = randomAlphanumeric(10);
    String topic = router.route(message, messageKey, topics, mock(PulsarSinkContext.class));
    assertEquals(topic, desiredTopic);
}
Also used : PulsarSinkContext(org.apache.flink.connector.pulsar.sink.writer.context.PulsarSinkContext) SinkConfiguration(org.apache.flink.connector.pulsar.sink.config.SinkConfiguration) Hash(org.apache.pulsar.client.impl.Hash) EnumSource(org.junit.jupiter.params.provider.EnumSource) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Aggregations

Hash (org.apache.pulsar.client.impl.Hash)2 SinkConfiguration (org.apache.flink.connector.pulsar.sink.config.SinkConfiguration)1 PulsarSinkContext (org.apache.flink.connector.pulsar.sink.writer.context.PulsarSinkContext)1 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)1 EnumSource (org.junit.jupiter.params.provider.EnumSource)1