Search in sources :

Example 1 with BatchingSettings

use of com.google.api.gax.batching.BatchingSettings in project google-cloud-java by GoogleCloudPlatform.

the class PublisherSnippets method getPublisherWithCustomBatchSettings.

public Publisher getPublisherWithCustomBatchSettings(TopicName topicName) throws Exception {
    // [START pubsub_publisher_batch_settings]
    // Batch settings control how the publisher batches messages
    // default : 1kb
    long requestBytesThreshold = 5000L;
    // default : 100
    long messageCountBatchSize = 10L;
    // default : 1 ms
    Duration publishDelayThreshold = Duration.ofMillis(100);
    // Publish request get triggered based on request size, messages count & time since last publish
    BatchingSettings batchingSettings = BatchingSettings.newBuilder().setElementCountThreshold(messageCountBatchSize).setRequestByteThreshold(requestBytesThreshold).setDelayThreshold(publishDelayThreshold).build();
    Publisher publisher = Publisher.defaultBuilder(topicName).setBatchingSettings(batchingSettings).build();
    // [END pubsub_publisher_batch_settings]
    return publisher;
}
Also used : Duration(org.threeten.bp.Duration) Publisher(com.google.cloud.pubsub.spi.v1.Publisher) BatchingSettings(com.google.api.gax.batching.BatchingSettings)

Example 2 with BatchingSettings

use of com.google.api.gax.batching.BatchingSettings in project divolte-collector by divolte.

the class GoogleCloudPubSubSinkConfiguration method getFactory.

@Override
public SinkFactory getFactory() {
    final RetrySettings retrySettings = this.retrySettings.createRetrySettings();
    final BatchingSettings batchingSettings = this.batchingSettings.createBatchingSettings();
    final Optional<String> emulator = Optional.ofNullable(System.getenv("PUBSUB_EMULATOR_HOST"));
    return emulator.map(hostport -> createFlushingPool(retrySettings, batchingSettings, hostport)).orElseGet(() -> createFlushingPool(retrySettings, batchingSettings));
}
Also used : JsonProperty(com.fasterxml.jackson.annotation.JsonProperty) Topic(com.google.pubsub.v1.Topic) GoogleCloudPubSubFlushingPool(io.divolte.server.topicsinks.pubsub.GoogleCloudPubSubFlushingPool) ParametersAreNullableByDefault(javax.annotation.ParametersAreNullableByDefault) ManagedChannel(io.grpc.ManagedChannel) LoggerFactory(org.slf4j.LoggerFactory) NoCredentialsProvider(com.google.api.gax.core.NoCredentialsProvider) ParametersAreNonnullByDefault(javax.annotation.ParametersAreNonnullByDefault) RetrySettings(com.google.api.gax.retrying.RetrySettings) Valid(javax.validation.Valid) FixedTransportChannelProvider(com.google.api.gax.rpc.FixedTransportChannelProvider) Duration(java.time.Duration) ProjectTopicName(com.google.pubsub.v1.ProjectTopicName) Logger(org.slf4j.Logger) BatchingSettings(com.google.api.gax.batching.BatchingSettings) MoreObjects(com.google.common.base.MoreObjects) IOException(java.io.IOException) Streams(com.google.common.collect.Streams) UncheckedIOException(java.io.UncheckedIOException) GrpcTransportChannel(com.google.api.gax.grpc.GrpcTransportChannel) ManagedChannelBuilder(io.grpc.ManagedChannelBuilder) TransportChannelProvider(com.google.api.gax.rpc.TransportChannelProvider) JsonCreator(com.fasterxml.jackson.annotation.JsonCreator) IOExceptions(io.divolte.server.IOExceptions) Optional(java.util.Optional) com.google.cloud.pubsub.v1(com.google.cloud.pubsub.v1) ProjectName(com.google.pubsub.v1.ProjectName) RetrySettings(com.google.api.gax.retrying.RetrySettings) BatchingSettings(com.google.api.gax.batching.BatchingSettings)

Example 3 with BatchingSettings

use of com.google.api.gax.batching.BatchingSettings in project divolte-collector by divolte.

the class GoogleCloudPubSubSinkConfigurationTest method testDefaultBatchingConfigurationValid.

@Test
public void testDefaultBatchingConfigurationValid() {
    // Check that we can generate settings from our defaults.
    final BatchingSettings batchingSettings = GoogleCloudPubSubSinkConfiguration.DEFAULT_BATCHING_SETTINGS.createBatchingSettings();
    assertNotNull(batchingSettings);
}
Also used : BatchingSettings(com.google.api.gax.batching.BatchingSettings) Test(org.junit.Test)

Aggregations

BatchingSettings (com.google.api.gax.batching.BatchingSettings)3 JsonCreator (com.fasterxml.jackson.annotation.JsonCreator)1 JsonProperty (com.fasterxml.jackson.annotation.JsonProperty)1 NoCredentialsProvider (com.google.api.gax.core.NoCredentialsProvider)1 GrpcTransportChannel (com.google.api.gax.grpc.GrpcTransportChannel)1 RetrySettings (com.google.api.gax.retrying.RetrySettings)1 FixedTransportChannelProvider (com.google.api.gax.rpc.FixedTransportChannelProvider)1 TransportChannelProvider (com.google.api.gax.rpc.TransportChannelProvider)1 Publisher (com.google.cloud.pubsub.spi.v1.Publisher)1 com.google.cloud.pubsub.v1 (com.google.cloud.pubsub.v1)1 MoreObjects (com.google.common.base.MoreObjects)1 Streams (com.google.common.collect.Streams)1 ProjectName (com.google.pubsub.v1.ProjectName)1 ProjectTopicName (com.google.pubsub.v1.ProjectTopicName)1 Topic (com.google.pubsub.v1.Topic)1 IOExceptions (io.divolte.server.IOExceptions)1 GoogleCloudPubSubFlushingPool (io.divolte.server.topicsinks.pubsub.GoogleCloudPubSubFlushingPool)1 ManagedChannel (io.grpc.ManagedChannel)1 ManagedChannelBuilder (io.grpc.ManagedChannelBuilder)1 IOException (java.io.IOException)1