Search in sources :

Example 11 with AmazonKinesis

use of com.amazonaws.services.kinesis.AmazonKinesis in project druid by druid-io.

the class KinesisRecordSupplierTest method testIsClosedShardEmpty.

@Test
public void testIsClosedShardEmpty() {
    AmazonKinesis mockKinesis = EasyMock.mock(AmazonKinesis.class);
    KinesisRecordSupplier target = new KinesisRecordSupplier(mockKinesis, recordsPerFetch, 0, 2, false, 100, 5000, 5000, 60000, 5, true);
    Record record = new Record();
    final String shardWithoutRecordsAndNullNextIterator = "0";
    setupMockKinesisForShardId(mockKinesis, shardWithoutRecordsAndNullNextIterator, new ArrayList<>(), null);
    final String shardWithRecordsAndNullNextIterator = "1";
    setupMockKinesisForShardId(mockKinesis, shardWithRecordsAndNullNextIterator, Collections.singletonList(record), null);
    final String shardWithoutRecordsAndNonNullNextIterator = "2";
    setupMockKinesisForShardId(mockKinesis, shardWithoutRecordsAndNonNullNextIterator, new ArrayList<>(), "nextIterator");
    final String shardWithRecordsAndNonNullNextIterator = "3";
    setupMockKinesisForShardId(mockKinesis, shardWithRecordsAndNonNullNextIterator, Collections.singletonList(record), "nextIterator");
    EasyMock.replay(mockKinesis);
    // A closed shard is empty only when the records are empty and the next iterator is null
    Assert.assertTrue(target.isClosedShardEmpty(STREAM, shardWithoutRecordsAndNullNextIterator));
    Assert.assertFalse(target.isClosedShardEmpty(STREAM, shardWithRecordsAndNullNextIterator));
    Assert.assertFalse(target.isClosedShardEmpty(STREAM, shardWithoutRecordsAndNonNullNextIterator));
    Assert.assertFalse(target.isClosedShardEmpty(STREAM, shardWithRecordsAndNonNullNextIterator));
}
Also used : Record(com.amazonaws.services.kinesis.model.Record) OrderedPartitionableRecord(org.apache.druid.indexing.seekablestream.common.OrderedPartitionableRecord) AmazonKinesis(com.amazonaws.services.kinesis.AmazonKinesis) Test(org.junit.Test)

Aggregations

AmazonKinesis (com.amazonaws.services.kinesis.AmazonKinesis)11 Properties (java.util.Properties)6 Test (org.junit.Test)6 ClientConfiguration (com.amazonaws.ClientConfiguration)4 HashSet (java.util.HashSet)4 ClientConfigurationFactory (com.amazonaws.ClientConfigurationFactory)3 AmazonKinesisClient (com.amazonaws.services.kinesis.AmazonKinesisClient)3 DescribeStreamResult (com.amazonaws.services.kinesis.model.DescribeStreamResult)3 ListShardsRequest (com.amazonaws.services.kinesis.model.ListShardsRequest)3 ListShardsResult (com.amazonaws.services.kinesis.model.ListShardsResult)3 HashMap (java.util.HashMap)3 AmazonServiceException (com.amazonaws.AmazonServiceException)2 ErrorType (com.amazonaws.AmazonServiceException.ErrorType)2 SdkClientException (com.amazonaws.SdkClientException)2 AmazonKinesisException (com.amazonaws.services.kinesis.model.AmazonKinesisException)2 ExpiredIteratorException (com.amazonaws.services.kinesis.model.ExpiredIteratorException)2 GetRecordsResult (com.amazonaws.services.kinesis.model.GetRecordsResult)2 ProvisionedThroughputExceededException (com.amazonaws.services.kinesis.model.ProvisionedThroughputExceededException)2 Shard (com.amazonaws.services.kinesis.model.Shard)2 InetAddress (java.net.InetAddress)2