Search in sources :

Example 1 with CloudWatchAsyncClient

use of software.amazon.awssdk.services.cloudwatch.CloudWatchAsyncClient in project aws-sdk-java-v2 by aws.

the class CloudWatchMetricPublisherTest method closeWaitsForUploadToComplete.

@Test(timeout = 10_000)
public void closeWaitsForUploadToComplete() throws InterruptedException {
    CountDownLatch cloudwatchPutCalledLatch = new CountDownLatch(1);
    CompletableFuture<PutMetricDataResponse> result = new CompletableFuture<>();
    CloudWatchAsyncClient cloudWatch = Mockito.mock(CloudWatchAsyncClient.class);
    try (CloudWatchMetricPublisher publisher = CloudWatchMetricPublisher.builder().cloudWatchClient(cloudWatch).uploadFrequency(Duration.ofMinutes(60)).build()) {
        MetricCollector collector = newCollector();
        collector.reportMetric(HttpMetric.AVAILABLE_CONCURRENCY, 5);
        publisher.publish(new FixedTimeMetricCollection(collector.collect()));
        Mockito.when(cloudWatch.putMetricData(any(PutMetricDataRequest.class))).thenAnswer(x -> {
            cloudwatchPutCalledLatch.countDown();
            return result;
        });
        publisher.publish(MetricCollector.create("test").collect());
        Thread closeThread = new Thread(publisher::close);
        assertThat(publisher.isShutdown()).isFalse();
        closeThread.start();
        // Wait until cloudwatch is called
        cloudwatchPutCalledLatch.await();
        // Wait to make sure the close thread seems to be waiting for the cloudwatch call to complete
        Thread.sleep(1_000);
        assertThat(closeThread.isAlive()).isTrue();
        // Complete the cloudwatch call
        result.complete(null);
        // Make sure the close thread finishes
        closeThread.join(5_000);
        assertThat(closeThread.isAlive()).isFalse();
    }
}
Also used : CompletableFuture(java.util.concurrent.CompletableFuture) CloudWatchAsyncClient(software.amazon.awssdk.services.cloudwatch.CloudWatchAsyncClient) PutMetricDataRequest(software.amazon.awssdk.services.cloudwatch.model.PutMetricDataRequest) PutMetricDataResponse(software.amazon.awssdk.services.cloudwatch.model.PutMetricDataResponse) MetricCollector(software.amazon.awssdk.metrics.MetricCollector) CountDownLatch(java.util.concurrent.CountDownLatch) Test(org.junit.Test)

Example 2 with CloudWatchAsyncClient

use of software.amazon.awssdk.services.cloudwatch.CloudWatchAsyncClient in project graylog-plugin-integrations by Graylog2.

the class KinesisConsumer method run.

public void run() {
    LOG.debug("Starting the Kinesis Consumer.");
    AwsCredentialsProvider credentialsProvider = AWSAuthFactory.create(request.region(), request.awsAccessKeyId(), request.awsSecretAccessKey(), request.assumeRoleArn());
    final Region region = Region.of(request.region());
    final DynamoDbAsyncClientBuilder dynamoDbClientBuilder = DynamoDbAsyncClient.builder();
    AWSClientBuilderUtil.initializeBuilder(dynamoDbClientBuilder, request.dynamodbEndpoint(), region, credentialsProvider);
    final DynamoDbAsyncClient dynamoClient = dynamoDbClientBuilder.build();
    final CloudWatchAsyncClientBuilder cloudwatchClientBuilder = CloudWatchAsyncClient.builder();
    AWSClientBuilderUtil.initializeBuilder(cloudwatchClientBuilder, request.cloudwatchEndpoint(), region, credentialsProvider);
    final CloudWatchAsyncClient cloudWatchClient = cloudwatchClientBuilder.build();
    final KinesisAsyncClientBuilder kinesisAsyncClientBuilder = KinesisAsyncClient.builder();
    AWSClientBuilderUtil.initializeBuilder(kinesisAsyncClientBuilder, request.kinesisEndpoint(), region, credentialsProvider);
    final KinesisAsyncClient kinesisAsyncClient = KinesisClientUtil.createKinesisAsyncClient(kinesisAsyncClientBuilder);
    final String workerId = String.format(Locale.ENGLISH, "graylog-node-%s", nodeId.anonymize());
    LOG.debug("Using workerId [{}].", workerId);
    // The application name needs to be unique per input/consumer.
    final String applicationName = String.format(Locale.ENGLISH, "graylog-aws-plugin-%s", kinesisStreamName);
    LOG.debug("Using Kinesis applicationName [{}].", applicationName);
    // The KinesisShardProcessorFactory contains the message processing logic.
    final KinesisShardProcessorFactory kinesisShardProcessorFactory = new KinesisShardProcessorFactory(objectMapper, transport, handleMessageCallback, kinesisStreamName, awsMessageType);
    ConfigsBuilder configsBuilder = new ConfigsBuilder(kinesisStreamName, applicationName, kinesisAsyncClient, dynamoClient, cloudWatchClient, workerId, kinesisShardProcessorFactory);
    final PollingConfig pollingConfig = new PollingConfig(kinesisStreamName, kinesisAsyncClient);
    // Default max records per request is 10k.
    if (recordBatchSize != null) {
        LOG.debug("Using explicit batch size [{}]", recordBatchSize);
        pollingConfig.maxRecords(recordBatchSize);
    }
    this.kinesisScheduler = new Scheduler(configsBuilder.checkpointConfig(), configsBuilder.coordinatorConfig(), configsBuilder.leaseManagementConfig(), configsBuilder.lifecycleConfig(), configsBuilder.metricsConfig(), configsBuilder.processorConfig(), configsBuilder.retrievalConfig().retrievalSpecificConfig(pollingConfig));
    LOG.debug("Starting Kinesis scheduler.");
    kinesisScheduler.run();
    LOG.debug("After Kinesis scheduler stopped.");
}
Also used : KinesisAsyncClient(software.amazon.awssdk.services.kinesis.KinesisAsyncClient) CloudWatchAsyncClient(software.amazon.awssdk.services.cloudwatch.CloudWatchAsyncClient) Scheduler(software.amazon.kinesis.coordinator.Scheduler) PollingConfig(software.amazon.kinesis.retrieval.polling.PollingConfig) ConfigsBuilder(software.amazon.kinesis.common.ConfigsBuilder) CloudWatchAsyncClientBuilder(software.amazon.awssdk.services.cloudwatch.CloudWatchAsyncClientBuilder) AwsCredentialsProvider(software.amazon.awssdk.auth.credentials.AwsCredentialsProvider) DynamoDbAsyncClient(software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient) Region(software.amazon.awssdk.regions.Region) DynamoDbAsyncClientBuilder(software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClientBuilder) KinesisAsyncClientBuilder(software.amazon.awssdk.services.kinesis.KinesisAsyncClientBuilder)

Aggregations

CloudWatchAsyncClient (software.amazon.awssdk.services.cloudwatch.CloudWatchAsyncClient)2 CompletableFuture (java.util.concurrent.CompletableFuture)1 CountDownLatch (java.util.concurrent.CountDownLatch)1 Test (org.junit.Test)1 AwsCredentialsProvider (software.amazon.awssdk.auth.credentials.AwsCredentialsProvider)1 MetricCollector (software.amazon.awssdk.metrics.MetricCollector)1 Region (software.amazon.awssdk.regions.Region)1 CloudWatchAsyncClientBuilder (software.amazon.awssdk.services.cloudwatch.CloudWatchAsyncClientBuilder)1 PutMetricDataRequest (software.amazon.awssdk.services.cloudwatch.model.PutMetricDataRequest)1 PutMetricDataResponse (software.amazon.awssdk.services.cloudwatch.model.PutMetricDataResponse)1 DynamoDbAsyncClient (software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient)1 DynamoDbAsyncClientBuilder (software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClientBuilder)1 KinesisAsyncClient (software.amazon.awssdk.services.kinesis.KinesisAsyncClient)1 KinesisAsyncClientBuilder (software.amazon.awssdk.services.kinesis.KinesisAsyncClientBuilder)1 ConfigsBuilder (software.amazon.kinesis.common.ConfigsBuilder)1 Scheduler (software.amazon.kinesis.coordinator.Scheduler)1 PollingConfig (software.amazon.kinesis.retrieval.polling.PollingConfig)1