Search in sources :

Example 11 with Shard

use of software.amazon.awssdk.services.kinesis.model.Shard in project beam by apache.

the class SimplifiedKinesisClientTest method shouldListAllShardsForTimestampWithRetriedDescribeStreamSummaryCallAfterStreamCreationTimestamp.

@Test
public void shouldListAllShardsForTimestampWithRetriedDescribeStreamSummaryCallAfterStreamCreationTimestamp() throws TransientKinesisException {
    Shard shard1 = Shard.builder().shardId(SHARD_1).build();
    Shard shard2 = Shard.builder().shardId(SHARD_2).build();
    Shard shard3 = Shard.builder().shardId(SHARD_3).build();
    int hoursDifference = 1;
    int retentionPeriodHours = hoursDifference * 3;
    Instant streamCreationTimestamp = CURRENT_TIMESTAMP.minus(Duration.standardHours(retentionPeriodHours));
    Instant startingPointTimestamp = streamCreationTimestamp.plus(Duration.standardHours(hoursDifference));
    when(currentInstantSupplier.get()).thenReturn(CURRENT_TIMESTAMP);
    when(kinesis.describeStreamSummary(DescribeStreamSummaryRequest.builder().streamName(STREAM).build())).thenThrow(LimitExceededException.builder().message("Fake Exception: Limit exceeded").build()).thenReturn(DescribeStreamSummaryResponse.builder().streamDescriptionSummary(StreamDescriptionSummary.builder().retentionPeriodHours(retentionPeriodHours).streamCreationTimestamp(TimeUtil.toJava(streamCreationTimestamp)).build()).build());
    ShardFilter shardFilter = ShardFilter.builder().type(ShardFilterType.AT_TIMESTAMP).timestamp(TimeUtil.toJava(startingPointTimestamp)).build();
    when(kinesis.listShards(ListShardsRequest.builder().streamName(STREAM).shardFilter(shardFilter).maxResults(1_000).build())).thenReturn(ListShardsResponse.builder().shards(shard1, shard2, shard3).nextToken(null).build());
    List<Shard> shards = underTest.listShardsAtPoint(STREAM, new StartingPoint(startingPointTimestamp));
    assertThat(shards).containsOnly(shard1, shard2, shard3);
}
Also used : Instant(org.joda.time.Instant) ShardFilter(software.amazon.awssdk.services.kinesis.model.ShardFilter) Shard(software.amazon.awssdk.services.kinesis.model.Shard) Datapoint(software.amazon.awssdk.services.cloudwatch.model.Datapoint) Test(org.junit.Test)

Example 12 with Shard

use of software.amazon.awssdk.services.kinesis.model.Shard in project beam by apache.

the class SimplifiedKinesisClientTest method shouldListAllShardsForTrimHorizonWithPagedResults.

@Test
public void shouldListAllShardsForTrimHorizonWithPagedResults() throws Exception {
    Shard shard1 = Shard.builder().shardId(SHARD_1).build();
    Shard shard2 = Shard.builder().shardId(SHARD_2).build();
    Shard shard3 = Shard.builder().shardId(SHARD_3).build();
    ShardFilter shardFilter = ShardFilter.builder().type(ShardFilterType.AT_TRIM_HORIZON).build();
    String nextListShardsToken = "testNextToken";
    when(kinesis.listShards(ListShardsRequest.builder().streamName(STREAM).shardFilter(shardFilter).maxResults(1_000).build())).thenReturn(ListShardsResponse.builder().shards(shard1, shard2).nextToken(nextListShardsToken).build());
    when(kinesis.listShards(ListShardsRequest.builder().maxResults(1_000).shardFilter(shardFilter).nextToken(nextListShardsToken).build())).thenReturn(ListShardsResponse.builder().shards(shard3).nextToken(null).build());
    List<Shard> shards = underTest.listShardsAtPoint(STREAM, new StartingPoint(InitialPositionInStream.TRIM_HORIZON));
    assertThat(shards).containsOnly(shard1, shard2, shard3);
}
Also used : ShardFilter(software.amazon.awssdk.services.kinesis.model.ShardFilter) Shard(software.amazon.awssdk.services.kinesis.model.Shard) Test(org.junit.Test)

Aggregations

Shard (software.amazon.awssdk.services.kinesis.model.Shard)12 Test (org.junit.Test)9 ShardFilter (software.amazon.awssdk.services.kinesis.model.ShardFilter)6 Instant (org.joda.time.Instant)5 ArrayList (java.util.ArrayList)2 ImmutableList (org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.ImmutableList)2 Datapoint (software.amazon.awssdk.services.cloudwatch.model.Datapoint)2 DescribeStreamSummaryRequest (software.amazon.awssdk.services.kinesis.model.DescribeStreamSummaryRequest)2 ListShardsRequest (software.amazon.awssdk.services.kinesis.model.ListShardsRequest)2 ListShardsResponse (software.amazon.awssdk.services.kinesis.model.ListShardsResponse)2 URI (java.net.URI)1 UTF_8 (java.nio.charset.StandardCharsets.UTF_8)1 List (java.util.List)1 BiFunction (java.util.function.BiFunction)1 Function (java.util.function.Function)1 Function.identity (java.util.function.Function.identity)1 IntFunction (java.util.function.IntFunction)1 Collectors.toList (java.util.stream.Collectors.toList)1 IntStream.range (java.util.stream.IntStream.range)1 PipelineExecutionException (org.apache.beam.sdk.Pipeline.PipelineExecutionException)1