Search in sources :

Example 1 with ApiFutureCallback

use of com.google.api.core.ApiFutureCallback in project java-pubsublite-kafka by googleapis.

the class SingleSubscriptionConsumerImpl method poll.

@Override
public ConsumerRecords<byte[], byte[]> poll(Duration duration) {
    if (autocommit) {
        ApiFuture<?> future = commitAll();
        ApiFutures.addCallback(future, new ApiFutureCallback<Object>() {

            @Override
            public void onFailure(Throwable throwable) {
                logger.atWarning().withCause(throwable).log("Failed to commit offsets.");
            }

            @Override
            public void onSuccess(Object result) {
            }
        }, MoreExecutors.directExecutor());
    }
    Map<Partition, Queue<SequencedMessage>> partitionQueues = doPoll(duration);
    Map<TopicPartition, List<ConsumerRecord<byte[], byte[]>>> records = new HashMap<>();
    partitionQueues.forEach((partition, queue) -> {
        if (queue.isEmpty())
            return;
        List<ConsumerRecord<byte[], byte[]>> partitionRecords = queue.stream().map(message -> RecordTransforms.fromMessage(message, topic, partition)).collect(Collectors.toList());
        records.put(new TopicPartition(topic.toString(), (int) partition.value()), partitionRecords);
    });
    return new ConsumerRecords<>(records);
}
Also used : MoreExecutors(com.google.common.util.concurrent.MoreExecutors) TimeoutException(java.util.concurrent.TimeoutException) KafkaException(org.apache.kafka.common.KafkaException) HashMap(java.util.HashMap) SequencedMessage(com.google.cloud.pubsublite.SequencedMessage) ConsumerRecords(org.apache.kafka.clients.consumer.ConsumerRecords) TopicPath(com.google.cloud.pubsublite.TopicPath) ArrayList(java.util.ArrayList) ImmutableList(com.google.common.collect.ImmutableList) ExtractStatus(com.google.cloud.pubsublite.internal.ExtractStatus) Duration(java.time.Duration) Map(java.util.Map) SimpleEntry(java.util.AbstractMap.SimpleEntry) NamedTarget(com.google.cloud.pubsublite.proto.SeekRequest.NamedTarget) ApiServiceUtils.blockingShutdown(com.google.cloud.pubsublite.internal.wire.ApiServiceUtils.blockingShutdown) TopicPartition(org.apache.kafka.common.TopicPartition) ImmutableSet(com.google.common.collect.ImmutableSet) ApiFutures(com.google.api.core.ApiFutures) Partition(com.google.cloud.pubsublite.Partition) ImmutableMap(com.google.common.collect.ImmutableMap) Offset(com.google.cloud.pubsublite.Offset) SeekRequest(com.google.cloud.pubsublite.proto.SeekRequest) WakeupException(org.apache.kafka.common.errors.WakeupException) GuardedBy(com.google.errorprone.annotations.concurrent.GuardedBy) Set(java.util.Set) ApiFutureCallback(com.google.api.core.ApiFutureCallback) MILLISECONDS(java.util.concurrent.TimeUnit.MILLISECONDS) CloseableMonitor(com.google.cloud.pubsublite.internal.CloseableMonitor) KafkaExceptionUtils.toKafka(com.google.cloud.pubsublite.kafka.KafkaExceptionUtils.toKafka) GoogleLogger(com.google.common.flogger.GoogleLogger) Collectors(java.util.stream.Collectors) ApiFuture(com.google.api.core.ApiFuture) SettableApiFuture(com.google.api.core.SettableApiFuture) List(java.util.List) ConsumerRecord(org.apache.kafka.clients.consumer.ConsumerRecord) Optional(java.util.Optional) CommitFailedException(org.apache.kafka.clients.consumer.CommitFailedException) Queue(java.util.Queue) TopicPartition(org.apache.kafka.common.TopicPartition) Partition(com.google.cloud.pubsublite.Partition) HashMap(java.util.HashMap) ConsumerRecords(org.apache.kafka.clients.consumer.ConsumerRecords) ConsumerRecord(org.apache.kafka.clients.consumer.ConsumerRecord) TopicPartition(org.apache.kafka.common.TopicPartition) ArrayList(java.util.ArrayList) ImmutableList(com.google.common.collect.ImmutableList) List(java.util.List) Queue(java.util.Queue)

Example 2 with ApiFutureCallback

use of com.google.api.core.ApiFutureCallback in project java-bigtable by googleapis.

the class DynamicFlowControlCallable method futureCall.

@Override
public ApiFuture futureCall(Object request, ApiCallContext context) {
    final Runnable flowControllerRunnable = new DynamicFlowControlRunnable();
    ApiFuture future = innerCallable.futureCall(request, context);
    ApiFutures.addCallback(future, new ApiFutureCallback() {

        @Override
        public void onFailure(Throwable t) {
            // skipping recording the latencies for those.
            if (t instanceof DeadlineExceededException) {
                flowControllerRunnable.run();
            }
        }

        @Override
        public void onSuccess(Object result) {
            flowControllerRunnable.run();
        }
    }, MoreExecutors.directExecutor());
    return future;
}
Also used : ApiFuture(com.google.api.core.ApiFuture) ApiFutureCallback(com.google.api.core.ApiFutureCallback) DeadlineExceededException(com.google.api.gax.rpc.DeadlineExceededException)

Aggregations

ApiFuture (com.google.api.core.ApiFuture)2 ApiFutureCallback (com.google.api.core.ApiFutureCallback)2 ApiFutures (com.google.api.core.ApiFutures)1 SettableApiFuture (com.google.api.core.SettableApiFuture)1 DeadlineExceededException (com.google.api.gax.rpc.DeadlineExceededException)1 Offset (com.google.cloud.pubsublite.Offset)1 Partition (com.google.cloud.pubsublite.Partition)1 SequencedMessage (com.google.cloud.pubsublite.SequencedMessage)1 TopicPath (com.google.cloud.pubsublite.TopicPath)1 CloseableMonitor (com.google.cloud.pubsublite.internal.CloseableMonitor)1 ExtractStatus (com.google.cloud.pubsublite.internal.ExtractStatus)1 ApiServiceUtils.blockingShutdown (com.google.cloud.pubsublite.internal.wire.ApiServiceUtils.blockingShutdown)1 KafkaExceptionUtils.toKafka (com.google.cloud.pubsublite.kafka.KafkaExceptionUtils.toKafka)1 SeekRequest (com.google.cloud.pubsublite.proto.SeekRequest)1 NamedTarget (com.google.cloud.pubsublite.proto.SeekRequest.NamedTarget)1 ImmutableList (com.google.common.collect.ImmutableList)1 ImmutableMap (com.google.common.collect.ImmutableMap)1 ImmutableSet (com.google.common.collect.ImmutableSet)1 GoogleLogger (com.google.common.flogger.GoogleLogger)1 MoreExecutors (com.google.common.util.concurrent.MoreExecutors)1