Search in sources :

Example 1 with ApiException

use of com.google.api.gax.rpc.ApiException in project divolte-collector by divolte.

the class GoogleCloudPubSubFlusherTest method testMessagesAreRetriedOnRetriableFailure.

@Test
public void testMessagesAreRetriedOnRetriableFailure() throws IOException {
    // Simulate a failure on the first send that indicates a retry should succeed.
    final Publisher publisher = mockPublisher.orElseThrow(IllegalStateException::new);
    when(publisher.publish(any(PubsubMessage.class))).thenReturn(failedFuture(new ApiException("simulated transient failure", new IOException(), GrpcStatusCode.of(Status.Code.INTERNAL), true))).thenAnswer(invocationOnMock -> completedFuture(String.valueOf(messageIdCounter++)));
    // Here we send the message.
    processSingleMessage();
    // Now we check the invocations…
    verify(publisher, times(2)).publish(any(PubsubMessage.class));
    verifyNoMoreInteractions(publisher);
}
Also used : Publisher(com.google.cloud.pubsub.v1.Publisher) IOException(java.io.IOException) PubsubMessage(com.google.pubsub.v1.PubsubMessage) ApiException(com.google.api.gax.rpc.ApiException) Test(org.junit.Test)

Example 2 with ApiException

use of com.google.api.gax.rpc.ApiException in project java-docs-samples by GoogleCloudPlatform.

the class PublisherExample method main.

/**
 * Publish messages to a topic.
 * @param args topic name, number of messages
 */
public static void main(String... args) throws Exception {
    // topic id, eg. "my-topic"
    String topicId = args[0];
    int messageCount = Integer.parseInt(args[1]);
    ProjectTopicName topicName = ProjectTopicName.of(PROJECT_ID, topicId);
    Publisher publisher = null;
    try {
        // Create a publisher instance with default settings bound to the topic
        publisher = Publisher.newBuilder(topicName).build();
        for (int i = 0; i < messageCount; i++) {
            String message = "message-" + i;
            // convert message to bytes
            ByteString data = ByteString.copyFromUtf8(message);
            PubsubMessage pubsubMessage = PubsubMessage.newBuilder().setData(data).build();
            // schedule a message to be published, messages are automatically batched
            ApiFuture<String> future = publisher.publish(pubsubMessage);
            // add an asynchronous callback to handle success / failure
            ApiFutures.addCallback(future, new ApiFutureCallback<String>() {

                @Override
                public void onFailure(Throwable throwable) {
                    if (throwable instanceof ApiException) {
                        ApiException apiException = ((ApiException) throwable);
                        // details on the API exception
                        System.out.println(apiException.getStatusCode().getCode());
                        System.out.println(apiException.isRetryable());
                    }
                    System.out.println("Error publishing message : " + message);
                }

                @Override
                public void onSuccess(String messageId) {
                    // Once published, returns server-assigned message ids (unique within the topic)
                    System.out.println(messageId);
                }
            });
        }
    } finally {
        if (publisher != null) {
            // When finished with the publisher, shutdown to free up resources.
            publisher.shutdown();
        }
    }
}
Also used : ByteString(com.google.protobuf.ByteString) ByteString(com.google.protobuf.ByteString) ProjectTopicName(com.google.pubsub.v1.ProjectTopicName) Publisher(com.google.cloud.pubsub.v1.Publisher) PubsubMessage(com.google.pubsub.v1.PubsubMessage) ApiException(com.google.api.gax.rpc.ApiException)

Example 3 with ApiException

use of com.google.api.gax.rpc.ApiException in project beam by apache.

the class RpcQosTest method attemptEnforcesActiveStateToPerformOperations_maxAttemptsExhausted.

@Test
public void attemptEnforcesActiveStateToPerformOperations_maxAttemptsExhausted() throws InterruptedException {
    RpcQosOptions rpcQosOptions = options.toBuilder().withMaxAttempts(1).unsafeBuild();
    RpcQos qos = new RpcQosImpl(rpcQosOptions, random, sleeper, counterFactory, distributionFactory);
    RpcReadAttempt readAttempt = qos.newReadAttempt(RPC_ATTEMPT_CONTEXT);
    readAttempt.recordRequestStart(monotonicClock.instant());
    readAttempt.recordRequestFailed(monotonicClock.instant());
    try {
        readAttempt.checkCanRetry(monotonicClock.instant(), RETRYABLE_ERROR);
        fail("expected error to be re-thrown due to max attempts exhaustion");
    } catch (ApiException e) {
    // expected
    }
    try {
        readAttempt.recordStreamValue(monotonicClock.instant());
        fail("expected IllegalStateException due to attempt being in terminal state");
    } catch (IllegalStateException e) {
    // expected
    }
    verify(sleeper, times(0)).sleep(// happens in checkCanRetry when the backoff is checked
    anyLong());
    verify(counterThrottlingMs, times(0)).inc(anyLong());
    verify(counterRpcFailures, times(1)).inc();
    verify(counterRpcSuccesses, times(0)).inc();
    verify(counterRpcStreamValueReceived, times(0)).inc();
}
Also used : RpcReadAttempt(org.apache.beam.sdk.io.gcp.firestore.RpcQos.RpcReadAttempt) ApiException(com.google.api.gax.rpc.ApiException) Test(org.junit.Test)

Example 4 with ApiException

use of com.google.api.gax.rpc.ApiException in project beam by apache.

the class RpcQosTest method attemptsExhaustCorrectly.

@Test
public void attemptsExhaustCorrectly() throws InterruptedException {
    RpcQosOptions rpcQosOptions = options.toBuilder().withMaxAttempts(3).unsafeBuild();
    RpcQos qos = new RpcQosImpl(rpcQosOptions, random, sleeper, counterFactory, distributionFactory);
    RpcReadAttempt readAttempt = qos.newReadAttempt(RPC_ATTEMPT_CONTEXT);
    // try 1
    readAttempt.recordRequestStart(monotonicClock.instant());
    readAttempt.recordRequestFailed(monotonicClock.instant());
    readAttempt.checkCanRetry(monotonicClock.instant(), RETRYABLE_ERROR);
    // try 2
    readAttempt.recordRequestStart(monotonicClock.instant());
    readAttempt.recordRequestFailed(monotonicClock.instant());
    readAttempt.checkCanRetry(monotonicClock.instant(), RETRYABLE_ERROR);
    // try 3
    readAttempt.recordRequestStart(monotonicClock.instant());
    readAttempt.recordRequestFailed(monotonicClock.instant());
    try {
        readAttempt.checkCanRetry(monotonicClock.instant(), RETRYABLE_ERROR);
        fail("expected retry to be exhausted after third attempt");
    } catch (ApiException e) {
        assertSame(e, RETRYABLE_ERROR);
    }
    verify(counterThrottlingMs, times(0)).inc(anyLong());
    verify(counterRpcFailures, times(3)).inc();
    verify(counterRpcSuccesses, times(0)).inc();
    verify(counterRpcStreamValueReceived, times(0)).inc();
}
Also used : RpcReadAttempt(org.apache.beam.sdk.io.gcp.firestore.RpcQos.RpcReadAttempt) ApiException(com.google.api.gax.rpc.ApiException) Test(org.junit.Test)

Example 5 with ApiException

use of com.google.api.gax.rpc.ApiException in project beam by apache.

the class RpcQosTest method attemptThrowsOnNonRetryableError.

@Test
public void attemptThrowsOnNonRetryableError() throws InterruptedException {
    RpcQosOptions rpcQosOptions = options.toBuilder().withMaxAttempts(3).unsafeBuild();
    RpcQos qos = new RpcQosImpl(rpcQosOptions, random, sleeper, counterFactory, distributionFactory);
    RpcReadAttempt readAttempt = qos.newReadAttempt(RPC_ATTEMPT_CONTEXT);
    readAttempt.recordRequestStart(monotonicClock.instant());
    // try 1
    readAttempt.recordRequestFailed(monotonicClock.instant());
    try {
        readAttempt.checkCanRetry(monotonicClock.instant(), NON_RETRYABLE_ERROR);
        fail("expected non-retryable error to throw error on first occurrence");
    } catch (ApiException e) {
        assertSame(e, NON_RETRYABLE_ERROR);
    }
    verify(counterThrottlingMs, times(0)).inc(anyLong());
    verify(counterRpcFailures, times(1)).inc();
    verify(counterRpcSuccesses, times(0)).inc();
    verify(counterRpcStreamValueReceived, times(0)).inc();
}
Also used : RpcReadAttempt(org.apache.beam.sdk.io.gcp.firestore.RpcQos.RpcReadAttempt) ApiException(com.google.api.gax.rpc.ApiException) Test(org.junit.Test)

Aggregations

ApiException (com.google.api.gax.rpc.ApiException)23 Test (org.junit.Test)14 IOException (java.io.IOException)5 PubsubMessage (com.google.pubsub.v1.PubsubMessage)4 RpcReadAttempt (org.apache.beam.sdk.io.gcp.firestore.RpcQos.RpcReadAttempt)4 StatusCode (com.google.api.gax.rpc.StatusCode)3 Publisher (com.google.cloud.pubsub.v1.Publisher)3 CheckedApiException (com.google.cloud.pubsublite.internal.CheckedApiException)3 ProjectTopicName (com.google.pubsub.v1.ProjectTopicName)3 Code (com.google.api.gax.rpc.StatusCode.Code)2 Offset (com.google.cloud.pubsublite.Offset)2 Write (com.google.firestore.v1.Write)2 ByteString (com.google.protobuf.ByteString)2 Span (io.opencensus.trace.Span)2 FirestoreProtoHelpers.newWrite (org.apache.beam.sdk.io.gcp.firestore.FirestoreProtoHelpers.newWrite)2 WriteElement (org.apache.beam.sdk.io.gcp.firestore.FirestoreV1WriteFn.WriteElement)2 Element (org.apache.beam.sdk.io.gcp.firestore.RpcQos.RpcWriteAttempt.Element)2 SubscriptionPath (org.apache.beam.sdk.io.gcp.pubsub.PubsubClient.SubscriptionPath)2 Instant (org.joda.time.Instant)2 MetricDescriptor (com.google.api.MetricDescriptor)1