Search in sources :

Example 1 with TopicPath

use of org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.TopicPath in project beam by apache.

the class PubsubJsonClientTest method listTopics.

@Test
public void listTopics() throws Exception {
    ListTopicsResponse expectedResponse1 = new ListTopicsResponse();
    expectedResponse1.setTopics(Collections.singletonList(buildTopic(1)));
    expectedResponse1.setNextPageToken("AVgJH3Z7aHxiDBs");
    ListTopicsResponse expectedResponse2 = new ListTopicsResponse();
    expectedResponse2.setTopics(Collections.singletonList(buildTopic(2)));
    Topics.List request = mockPubsub.projects().topics().list(PROJECT.getPath());
    when((Object) request.execute()).thenReturn(expectedResponse1, expectedResponse2);
    List<TopicPath> topicPaths = client.listTopics(PROJECT);
    assertEquals(2, topicPaths.size());
}
Also used : TopicPath(org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.TopicPath) Topics(com.google.api.services.pubsub.Pubsub.Projects.Topics) ListTopicsResponse(com.google.api.services.pubsub.model.ListTopicsResponse) Test(org.junit.Test)

Example 2 with TopicPath

use of org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.TopicPath in project beam by apache.

the class FhirIOReadIT method setup.

@Before
public void setup() throws Exception {
    healthcareDataset = String.format(HEALTHCARE_DATASET_TEMPLATE, project);
    if (client == null) {
        this.client = new HttpHealthcareApiClient();
    }
    pubsub = PubsubGrpcClient.FACTORY.newClient(null, null, pipelineOptions);
    TopicPath topicPath = PubsubClient.topicPathFromPath(pubsubTopic);
    pubsub.createTopic(topicPath);
    SubscriptionPath subscriptionPath = PubsubClient.subscriptionPathFromPath(pubsubSubscription);
    pubsub.createSubscription(topicPath, subscriptionPath, 60);
    client.createFhirStore(healthcareDataset, fhirStoreName, version, pubsubTopic);
    // Execute bundles to trigger FHIR notificiations to input topic
    FhirIOTestUtil.executeFhirBundles(client, healthcareDataset + "/fhirStores/" + fhirStoreName, FhirIOTestUtil.BUNDLES.get(version));
}
Also used : TopicPath(org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.TopicPath) SubscriptionPath(org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.SubscriptionPath) Before(org.junit.Before)

Example 3 with TopicPath

use of org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.TopicPath in project beam by apache.

the class PubsubClientTest method topicPathFromNameWellFormed.

@Test
public void topicPathFromNameWellFormed() {
    TopicPath path = PubsubClient.topicPathFromName("test", "something");
    assertEquals("projects/test/topics/something", path.getPath());
    assertEquals("/topics/test/something", path.getFullPath());
}
Also used : TopicPath(org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.TopicPath) Test(org.junit.Test)

Example 4 with TopicPath

use of org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.TopicPath in project beam by apache.

the class PubsubHelper method createOrReuseTopic.

/**
 * Create a topic from short name if it does not already exist. The topic will not be deleted on
 * cleanup. Return full topic name.
 */
public TopicPath createOrReuseTopic(String shortTopic) throws IOException {
    TopicPath topic = PubsubClient.topicPathFromName(project, shortTopic);
    while (true) {
        try {
            NexmarkUtils.console("create topic %s", topic);
            pubsubClient.createTopic(topic);
            return topic;
        } catch (GoogleJsonResponseException ex) {
            if (topicExists(shortTopic)) {
                NexmarkUtils.console("topic %s already exists", topic);
                return topic;
            }
            try {
                if (!BackOffUtils.next(sleeper, backOff)) {
                    NexmarkUtils.console("too many retries for creating/reusing topic %s", topic);
                    throw ex;
                }
            } catch (InterruptedException in) {
                throw new IOException(in);
            }
        }
    }
}
Also used : GoogleJsonResponseException(com.google.api.client.googleapis.json.GoogleJsonResponseException) TopicPath(org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.TopicPath) IOException(java.io.IOException)

Example 5 with TopicPath

use of org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.TopicPath in project beam by apache.

the class PubsubHelper method subscriptionExists.

/**
 * Does subscription corresponding to short name exist?
 */
public boolean subscriptionExists(String shortTopic, String shortSubscription) throws IOException {
    TopicPath topic = PubsubClient.topicPathFromName(project, shortTopic);
    SubscriptionPath subscription = PubsubClient.subscriptionPathFromName(project, shortSubscription);
    return pubsubClient.listSubscriptions(PubsubClient.projectPathFromId(project), topic).stream().anyMatch(subscription::equals);
}
Also used : TopicPath(org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.TopicPath) SubscriptionPath(org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.SubscriptionPath)

Aggregations

TopicPath (org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.TopicPath)17 SubscriptionPath (org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.SubscriptionPath)10 IOException (java.io.IOException)6 Test (org.junit.Test)6 GoogleJsonResponseException (com.google.api.client.googleapis.json.GoogleJsonResponseException)3 PipelineOptions (org.apache.beam.sdk.options.PipelineOptions)3 ByteString (com.google.protobuf.ByteString)2 PubsubSource (org.apache.beam.sdk.io.gcp.pubsub.PubsubUnboundedSource.PubsubSource)2 ApiFuture (com.google.api.core.ApiFuture)1 ApiFutures (com.google.api.core.ApiFutures)1 GrpcTransportChannel (com.google.api.gax.grpc.GrpcTransportChannel)1 FixedTransportChannelProvider (com.google.api.gax.rpc.FixedTransportChannelProvider)1 TransportChannelProvider (com.google.api.gax.rpc.TransportChannelProvider)1 Subscriptions (com.google.api.services.pubsub.Pubsub.Projects.Subscriptions)1 Topics (com.google.api.services.pubsub.Pubsub.Projects.Topics)1 ListSubscriptionsResponse (com.google.api.services.pubsub.model.ListSubscriptionsResponse)1 ListTopicsResponse (com.google.api.services.pubsub.model.ListTopicsResponse)1 AckReplyConsumer (com.google.cloud.pubsub.v1.AckReplyConsumer)1 MessageReceiver (com.google.cloud.pubsub.v1.MessageReceiver)1 Publisher (com.google.cloud.pubsub.v1.Publisher)1