Search in sources :

Example 6 with MessageReceiver

use of com.google.cloud.pubsub.v1.MessageReceiver in project beam by apache.

the class TestPubsub method waitForNMessages.

/**
 * Repeatedly pull messages from {@link #subscriptionPath()}, returns after receiving {@code n}
 * messages or after waiting for {@code timeoutDuration}.
 */
public List<PubsubMessage> waitForNMessages(int n, Duration timeoutDuration) throws IOException, InterruptedException {
    Preconditions.checkNotNull(subscriptionPath);
    BlockingQueue<com.google.pubsub.v1.PubsubMessage> receivedMessages = new LinkedBlockingDeque<>(n);
    MessageReceiver receiver = (com.google.pubsub.v1.PubsubMessage message, AckReplyConsumer replyConsumer) -> {
        if (receivedMessages.offer(message)) {
            replyConsumer.ack();
        } else {
            replyConsumer.nack();
        }
    };
    Subscriber subscriber = Subscriber.newBuilder(subscriptionPath.getPath(), receiver).setCredentialsProvider(pipelineOptions::getGcpCredential).setChannelProvider(channelProvider).setEndpoint(pubsubEndpoint).build();
    subscriber.startAsync();
    DateTime startTime = new DateTime();
    int timeoutSeconds = timeoutDuration.toStandardSeconds().getSeconds();
    while (receivedMessages.size() < n && Seconds.secondsBetween(startTime, new DateTime()).getSeconds() < timeoutSeconds) {
        try {
            Thread.sleep(1000);
        } catch (InterruptedException ignored) {
        }
    }
    subscriber.stopAsync();
    subscriber.awaitTerminated();
    return receivedMessages.stream().map((message) -> new PubsubMessage(message.getData().toByteArray(), message.getAttributesMap(), message.getMessageId())).collect(Collectors.toList());
}
Also used : Statement(org.junit.runners.model.Statement) TopicPath(org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.TopicPath) TestPipelineOptions(org.apache.beam.sdk.testing.TestPipelineOptions) ManagedChannel(io.grpc.ManagedChannel) TestRule(org.junit.rules.TestRule) Duration(org.joda.time.Duration) TimeoutException(java.util.concurrent.TimeoutException) Subscriber(com.google.cloud.pubsub.v1.Subscriber) Streams(org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.Streams) AckReplyConsumer(com.google.cloud.pubsub.v1.AckReplyConsumer) Publisher(com.google.cloud.pubsub.v1.Publisher) Seconds(org.joda.time.Seconds) FixedTransportChannelProvider(com.google.api.gax.rpc.FixedTransportChannelProvider) ThreadLocalRandom(java.util.concurrent.ThreadLocalRandom) TestPipeline(org.apache.beam.sdk.testing.TestPipeline) Iterables(org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.Iterables) SubscriptionAdminClient(com.google.cloud.pubsub.v1.SubscriptionAdminClient) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) PipelineOptions(org.apache.beam.sdk.options.PipelineOptions) Nullable(org.checkerframework.checker.nullness.qual.Nullable) DateTimeFormat(org.joda.time.format.DateTimeFormat) MessageReceiver(com.google.cloud.pubsub.v1.MessageReceiver) ApiFutures(com.google.api.core.ApiFutures) DateTimeFormatter(org.joda.time.format.DateTimeFormatter) SubscriptionPath(org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.SubscriptionPath) DateTime(org.joda.time.DateTime) IOException(java.io.IOException) BlockingQueue(java.util.concurrent.BlockingQueue) Description(org.junit.runner.Description) TopicAdminSettings(com.google.cloud.pubsub.v1.TopicAdminSettings) Collectors(java.util.stream.Collectors) ApiFuture(com.google.api.core.ApiFuture) ByteString(com.google.protobuf.ByteString) ExecutionException(java.util.concurrent.ExecutionException) GrpcTransportChannel(com.google.api.gax.grpc.GrpcTransportChannel) ManagedChannelBuilder(io.grpc.ManagedChannelBuilder) List(java.util.List) Matchers.containsInAnyOrder(org.hamcrest.Matchers.containsInAnyOrder) TransportChannelProvider(com.google.api.gax.rpc.TransportChannelProvider) TopicAdminClient(com.google.cloud.pubsub.v1.TopicAdminClient) Preconditions(org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions) Matcher(org.hamcrest.Matcher) Instant(org.joda.time.Instant) LinkedBlockingDeque(java.util.concurrent.LinkedBlockingDeque) SubscriptionAdminSettings(com.google.cloud.pubsub.v1.SubscriptionAdminSettings) PushConfig(com.google.pubsub.v1.PushConfig) LinkedBlockingDeque(java.util.concurrent.LinkedBlockingDeque) DateTime(org.joda.time.DateTime) MessageReceiver(com.google.cloud.pubsub.v1.MessageReceiver) Subscriber(com.google.cloud.pubsub.v1.Subscriber) AckReplyConsumer(com.google.cloud.pubsub.v1.AckReplyConsumer)

Example 7 with MessageReceiver

use of com.google.cloud.pubsub.v1.MessageReceiver in project flink by apache.

the class PubsubHelper method subscribeToSubscription.

public Subscriber subscribeToSubscription(String project, String subscription, MessageReceiver messageReceiver) {
    ProjectSubscriptionName subscriptionName = ProjectSubscriptionName.of(project, subscription);
    Subscriber subscriber = Subscriber.newBuilder(subscriptionName, messageReceiver).setChannelProvider(channelProvider).setCredentialsProvider(EmulatorCredentialsProvider.create()).build();
    subscriber.startAsync();
    return subscriber;
}
Also used : ProjectSubscriptionName(com.google.pubsub.v1.ProjectSubscriptionName) Subscriber(com.google.cloud.pubsub.v1.Subscriber)

Aggregations

Subscriber (com.google.cloud.pubsub.v1.Subscriber)4 AckReplyConsumer (com.google.cloud.pubsub.spi.v1.AckReplyConsumer)3 MessageReceiver (com.google.cloud.pubsub.spi.v1.MessageReceiver)3 Subscriber (com.google.cloud.pubsub.spi.v1.Subscriber)3 PubsubMessage (com.google.pubsub.v1.PubsubMessage)3 SubscriptionName (com.google.pubsub.v1.SubscriptionName)3 AckReplyConsumer (com.google.cloud.pubsub.v1.AckReplyConsumer)2 MessageReceiver (com.google.cloud.pubsub.v1.MessageReceiver)2 TopicName (com.google.pubsub.v1.TopicName)2 ApiFuture (com.google.api.core.ApiFuture)1 ApiFutures (com.google.api.core.ApiFutures)1 GrpcTransportChannel (com.google.api.gax.grpc.GrpcTransportChannel)1 FixedTransportChannelProvider (com.google.api.gax.rpc.FixedTransportChannelProvider)1 TransportChannelProvider (com.google.api.gax.rpc.TransportChannelProvider)1 Publisher (com.google.cloud.pubsub.spi.v1.Publisher)1 SubscriptionAdminClient (com.google.cloud.pubsub.spi.v1.SubscriptionAdminClient)1 Publisher (com.google.cloud.pubsub.v1.Publisher)1 SubscriptionAdminClient (com.google.cloud.pubsub.v1.SubscriptionAdminClient)1 SubscriptionAdminSettings (com.google.cloud.pubsub.v1.SubscriptionAdminSettings)1 TopicAdminClient (com.google.cloud.pubsub.v1.TopicAdminClient)1