Examples with TestConsumer org.apache.flink.streaming.connectors.kinesis.testutils.TestUtils.TestConsumer used on opensource projects

Example 6 with TestConsumer

use of org.apache.flink.streaming.connectors.kinesis.testutils.TestUtils.TestConsumer in project flink by apache.

the class FanOutRecordPublisherTest method testShardConsumerCompletesIfResourceNotFoundExceptionThrownFromSubscription.

@Test
public void testShardConsumerCompletesIfResourceNotFoundExceptionThrownFromSubscription() throws Exception {
    ResourceNotFoundException exception = ResourceNotFoundException.builder().build();
    SubscriptionErrorKinesisV2 kinesis = FakeKinesisFanOutBehavioursFactory.errorDuringSubscription(exception);
    RecordPublisher recordPublisher = createRecordPublisher(kinesis);
    TestConsumer consumer = new TestConsumer();
    assertEquals(COMPLETE, recordPublisher.run(consumer));
    // Will exit on the first subscription
    assertEquals(1, kinesis.getNumberOfSubscribeToShardInvocations());
}

Example 7 with TestConsumer

use of org.apache.flink.streaming.connectors.kinesis.testutils.TestUtils.TestConsumer in project flink by apache.

the class FanOutRecordPublisherTest method testResourceNotFoundWhenObtainingSubscriptionTerminatesApplication.

@Test
public void testResourceNotFoundWhenObtainingSubscriptionTerminatesApplication() throws Exception {
    thrown.expect(ResourceNotFoundException.class);
    KinesisProxyV2Interface kinesis = FakeKinesisFanOutBehavioursFactory.resourceNotFoundWhenObtainingSubscription();
    RecordPublisher recordPublisher = createRecordPublisher(kinesis);
    recordPublisher.run(new TestConsumer());
}

Example 8 with TestConsumer

use of org.apache.flink.streaming.connectors.kinesis.testutils.TestUtils.TestConsumer in project flink by apache.

the class PollingRecordPublisherTest method testRunGetShardIteratorReturnsNullIsComplete.

@Test
public void testRunGetShardIteratorReturnsNullIsComplete() throws Exception {
    KinesisProxyInterface fakeKinesis = FakeKinesisBehavioursFactory.noShardsFoundForRequestedStreamsBehaviour();
    PollingRecordPublisher recordPublisher = createPollingRecordPublisher(fakeKinesis);
    assertEquals(COMPLETE, recordPublisher.run(new TestConsumer()));
}

Example 9 with TestConsumer

use of org.apache.flink.streaming.connectors.kinesis.testutils.TestUtils.TestConsumer in project flink by apache.

the class PollingRecordPublisherTest method testRunPublishesRecordsToConsumer.

@Test
public void testRunPublishesRecordsToConsumer() throws Exception {
    KinesisProxyInterface fakeKinesis = totalNumOfRecordsAfterNumOfGetRecordsCalls(5, 1, 100);
    PollingRecordPublisher recordPublisher = createPollingRecordPublisher(fakeKinesis);
    TestConsumer consumer = new TestConsumer();
    recordPublisher.run(consumer);
    assertEquals(1, consumer.getRecordBatches().size());
    assertEquals(5, consumer.getRecordBatches().get(0).getDeaggregatedRecordSize());
    assertEquals(100L, consumer.getRecordBatches().get(0).getMillisBehindLatest(), 0);
}

Example 10 with TestConsumer

use of org.apache.flink.streaming.connectors.kinesis.testutils.TestUtils.TestConsumer in project flink by apache.

the class PollingRecordPublisherTest method testRunGetRecordsRecoversFromExpiredIteratorException.

@Test
public void testRunGetRecordsRecoversFromExpiredIteratorException() throws Exception {
    KinesisProxyInterface fakeKinesis = spy(FakeKinesisBehavioursFactory.totalNumOfRecordsAfterNumOfGetRecordsCallsWithUnexpectedExpiredIterator(2, 2, 1, 500));
    PollingRecordPublisher recordPublisher = createPollingRecordPublisher(fakeKinesis);
    recordPublisher.run(new TestConsumer());
    // Get shard iterator is called twice, once during first run, secondly to refresh expired
    // iterator
    verify(fakeKinesis, times(2)).getShardIterator(any(), any(), any());
}