Search in sources :

Example 16 with ApiException

use of com.google.api.gax.rpc.ApiException in project divolte-collector by divolte.

the class GoogleCloudPubSubFlusherTest method testMessagesAreAbandonedOnNonRetriableFailure.

@Test
public void testMessagesAreAbandonedOnNonRetriableFailure() throws IOException {
    // Simulate a failure on send that indicates a retry isn't allowed.
    final Publisher publisher = mockPublisher.orElseThrow(IllegalStateException::new);
    when(publisher.publish(any(PubsubMessage.class))).thenReturn(failedFuture(new ApiException("simulated permanent failure", new IOException(), GrpcStatusCode.of(Status.Code.NOT_FOUND), false)));
    // Here we send the message.
    processSingleMessage();
    // Now we check the invocations…
    verify(publisher).publish(any(PubsubMessage.class));
    verifyNoMoreInteractions(publisher);
}
Also used : Publisher(com.google.cloud.pubsub.v1.Publisher) IOException(java.io.IOException) PubsubMessage(com.google.pubsub.v1.PubsubMessage) ApiException(com.google.api.gax.rpc.ApiException) Test(org.junit.Test)

Example 17 with ApiException

use of com.google.api.gax.rpc.ApiException in project java-docs-samples by GoogleCloudPlatform.

the class CreatePullSubscriptionExample method main.

/**
 * Create a pull subscription.
 *
 * @param args topic subscriptionId
 * @throws Exception exception thrown if operation is unsuccessful
 */
public static void main(String... args) throws Exception {
    // Your Google Cloud Platform project ID
    String projectId = ServiceOptions.getDefaultProjectId();
    // Your topic ID, eg. "my-topic"
    String topicId = args[0];
    // Your subscription ID eg. "my-sub"
    String subscriptionId = args[1];
    ProjectTopicName topicName = ProjectTopicName.of(projectId, topicId);
    // Create a new subscription
    ProjectSubscriptionName subscriptionName = ProjectSubscriptionName.of(projectId, subscriptionId);
    try (SubscriptionAdminClient subscriptionAdminClient = SubscriptionAdminClient.create()) {
        // create a pull subscription with default acknowledgement deadline (= 10 seconds)
        Subscription subscription = subscriptionAdminClient.createSubscription(subscriptionName, topicName, PushConfig.getDefaultInstance(), 0);
    } catch (ApiException e) {
        // example : code = ALREADY_EXISTS(409) implies subscription already exists
        System.out.print(e.getStatusCode().getCode());
        System.out.print(e.isRetryable());
    }
    System.out.printf("Subscription %s:%s created.\n", subscriptionName.getProject(), subscriptionName.getSubscription());
}
Also used : ProjectSubscriptionName(com.google.pubsub.v1.ProjectSubscriptionName) SubscriptionAdminClient(com.google.cloud.pubsub.v1.SubscriptionAdminClient) ProjectTopicName(com.google.pubsub.v1.ProjectTopicName) Subscription(com.google.pubsub.v1.Subscription) ApiException(com.google.api.gax.rpc.ApiException)

Example 18 with ApiException

use of com.google.api.gax.rpc.ApiException in project java-docs-samples by GoogleCloudPlatform.

the class CreateTopicExample method main.

/**
 * Create a topic.
 *
 * @param args topicId
 * @throws Exception exception thrown if operation is unsuccessful
 */
public static void main(String... args) throws Exception {
    // Your Google Cloud Platform project ID
    String projectId = ServiceOptions.getDefaultProjectId();
    // Your topic ID, eg. "my-topic"
    String topicId = args[0];
    // Create a new topic
    ProjectTopicName topic = ProjectTopicName.of(projectId, topicId);
    try (TopicAdminClient topicAdminClient = TopicAdminClient.create()) {
        topicAdminClient.createTopic(topic);
    } catch (ApiException e) {
        // example : code = ALREADY_EXISTS(409) implies topic already exists
        System.out.print(e.getStatusCode().getCode());
        System.out.print(e.isRetryable());
    }
    System.out.printf("Topic %s:%s created.\n", topic.getProject(), topic.getTopic());
}
Also used : TopicAdminClient(com.google.cloud.pubsub.v1.TopicAdminClient) ProjectTopicName(com.google.pubsub.v1.ProjectTopicName) ApiException(com.google.api.gax.rpc.ApiException)

Example 19 with ApiException

use of com.google.api.gax.rpc.ApiException in project beam by apache.

the class TopicBacklogReaderImplTest method computeMessageStats_failure.

@SuppressWarnings("incompatible")
@Test
public void computeMessageStats_failure() {
    when(mockClient.computeMessageStats(example(TopicPath.class), example(Partition.class), example(Offset.class), Offset.of(Integer.MAX_VALUE))).thenReturn(ApiFutures.immediateFailedFuture(new CheckedApiException(Code.UNAVAILABLE).underlying));
    ApiException e = assertThrows(ApiException.class, () -> reader.computeMessageStats(example(Offset.class)));
    assertEquals(Code.UNAVAILABLE, e.getStatusCode().getCode());
}
Also used : Partition(com.google.cloud.pubsublite.Partition) TopicPath(com.google.cloud.pubsublite.TopicPath) CheckedApiException(com.google.cloud.pubsublite.internal.CheckedApiException) Offset(com.google.cloud.pubsublite.Offset) CheckedApiException(com.google.cloud.pubsublite.internal.CheckedApiException) ApiException(com.google.api.gax.rpc.ApiException) Test(org.junit.Test)

Example 20 with ApiException

use of com.google.api.gax.rpc.ApiException in project beam by apache.

the class StorageApiFlushAndFinalizeDoFn method process.

@SuppressWarnings({ "nullness" })
@ProcessElement
public void process(PipelineOptions pipelineOptions, @Element KV<String, Operation> element) throws Exception {
    final String streamId = element.getKey();
    final Operation operation = element.getValue();
    final DatasetService datasetService = getDatasetService(pipelineOptions);
    // Flush the stream. If the flush offset < 0, that means we only need to finalize.
    long offset = operation.flushOffset;
    if (offset >= 0) {
        Instant now = Instant.now();
        RetryManager<FlushRowsResponse, Context<FlushRowsResponse>> retryManager = new RetryManager<>(Duration.standardSeconds(1), Duration.standardMinutes(1), 3);
        retryManager.addOperation(// runOperation
        c -> {
            try {
                flushOperationsSent.inc();
                return datasetService.flush(streamId, offset);
            } catch (Exception e) {
                throw new RuntimeException(e);
            }
        }, // onError
        contexts -> {
            Throwable error = Iterables.getFirst(contexts, null).getError();
            LOG.warn("Flush of stream " + streamId + " to offset " + offset + " failed with " + error);
            flushOperationsFailed.inc();
            if (error instanceof ApiException) {
                Code statusCode = ((ApiException) error).getStatusCode().getCode();
                if (statusCode.equals(Code.ALREADY_EXISTS)) {
                    flushOperationsAlreadyExists.inc();
                    // Implies that we have already flushed up to this point, so don't retry.
                    return RetryType.DONT_RETRY;
                }
                if (statusCode.equals(Code.INVALID_ARGUMENT)) {
                    flushOperationsInvalidArgument.inc();
                    // TODO: Storage API should provide a more-specific way of identifying this failure.
                    return RetryType.DONT_RETRY;
                }
            }
            return RetryType.RETRY_ALL_OPERATIONS;
        }, // onSuccess
        c -> {
            flushOperationsSucceeded.inc();
        }, new Context<>());
        retryManager.run(true);
        java.time.Duration timeElapsed = java.time.Duration.between(now, Instant.now());
        flushLatencyDistribution.update(timeElapsed.toMillis());
    }
    // or we would end up with duplicates.
    if (operation.finalizeStream) {
        RetryManager<FinalizeWriteStreamResponse, Context<FinalizeWriteStreamResponse>> retryManager = new RetryManager<>(Duration.standardSeconds(1), Duration.standardMinutes(1), 3);
        retryManager.addOperation(c -> {
            finalizeOperationsSent.inc();
            return datasetService.finalizeWriteStream(streamId);
        }, contexts -> {
            LOG.warn("Finalize of stream " + streamId + " failed with " + Iterables.getFirst(contexts, null).getError());
            finalizeOperationsFailed.inc();
            return RetryType.RETRY_ALL_OPERATIONS;
        }, r -> {
            finalizeOperationsSucceeded.inc();
        }, new Context<>());
        retryManager.run(true);
    }
}
Also used : Context(org.apache.beam.sdk.io.gcp.bigquery.RetryManager.Operation.Context) FinalizeWriteStreamResponse(com.google.cloud.bigquery.storage.v1.FinalizeWriteStreamResponse) Instant(java.time.Instant) DatasetService(org.apache.beam.sdk.io.gcp.bigquery.BigQueryServices.DatasetService) Operation(org.apache.beam.sdk.io.gcp.bigquery.StorageApiFlushAndFinalizeDoFn.Operation) Code(com.google.api.gax.rpc.StatusCode.Code) IOException(java.io.IOException) ApiException(com.google.api.gax.rpc.ApiException) FlushRowsResponse(com.google.cloud.bigquery.storage.v1.FlushRowsResponse) ApiException(com.google.api.gax.rpc.ApiException)

Aggregations

ApiException (com.google.api.gax.rpc.ApiException)30 Test (org.junit.Test)21 IOException (java.io.IOException)5 PubsubMessage (com.google.pubsub.v1.PubsubMessage)4 StatusCode (com.google.api.gax.rpc.StatusCode)3 Publisher (com.google.cloud.pubsub.v1.Publisher)3 CheckedApiException (com.google.cloud.pubsublite.internal.CheckedApiException)3 ProjectSubscriptionName (com.google.pubsub.v1.ProjectSubscriptionName)3 ProjectTopicName (com.google.pubsub.v1.ProjectTopicName)3 RpcReadAttempt (org.apache.beam.sdk.io.gcp.firestore.RpcQos.RpcReadAttempt)3 Code (com.google.api.gax.rpc.StatusCode.Code)2 Offset (com.google.cloud.pubsublite.Offset)2 Write (com.google.firestore.v1.Write)2 ByteString (com.google.protobuf.ByteString)2 TopicName (com.google.pubsub.v1.TopicName)2 Span (io.opencensus.trace.Span)2 FirestoreProtoHelpers.newWrite (org.apache.beam.sdk.io.gcp.firestore.FirestoreProtoHelpers.newWrite)2 WriteElement (org.apache.beam.sdk.io.gcp.firestore.FirestoreV1WriteFn.WriteElement)2 SubscriptionPath (org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.SubscriptionPath)2 Instant (org.joda.time.Instant)2