Search in sources :

Example 1 with KinesisProxyV2Interface

use of org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface in project flink by apache.

the class KinesisDataFetcherTest method testRecordPublisherFactoryIsTornDown.

@Test(timeout = 1000L)
public void testRecordPublisherFactoryIsTornDown() throws InterruptedException {
    KinesisProxyV2Interface kinesisV2 = mock(KinesisProxyV2Interface.class);
    TestableKinesisDataFetcher<String> fetcher = new TestableKinesisDataFetcher<String>(singletonList("fakeStream1"), new TestSourceContext<>(), TestUtils.efoProperties(), new KinesisDeserializationSchemaWrapper<>(new SimpleStringSchema()), 10, 2, new AtomicReference<>(), new LinkedList<>(), new HashMap<>(), mock(KinesisProxyInterface.class), kinesisV2) {
    };
    fetcher.shutdownFetcher();
    fetcher.awaitTermination();
}
Also used : SimpleStringSchema(org.apache.flink.api.common.serialization.SimpleStringSchema) KinesisProxyV2Interface(org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface) KinesisProxyInterface(org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyInterface) TestableKinesisDataFetcher(org.apache.flink.streaming.connectors.kinesis.testutils.TestableKinesisDataFetcher) Test(org.junit.Test)

Example 2 with KinesisProxyV2Interface

use of org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface in project flink by apache.

the class KinesisDataFetcherTest method testRecordPublisherFactoryIsTornDownWhenDeregisterStreamConsumerThrowsException.

@Test(timeout = 10000)
public void testRecordPublisherFactoryIsTornDownWhenDeregisterStreamConsumerThrowsException() throws InterruptedException {
    KinesisProxyV2Interface kinesisV2 = mock(KinesisProxyV2Interface.class);
    TestableKinesisDataFetcher<String> fetcher = new TestableKinesisDataFetcher<String>(singletonList("fakeStream1"), new TestSourceContext<>(), TestUtils.efoProperties(), new KinesisDeserializationSchemaWrapper<>(new SimpleStringSchema()), 10, 2, new AtomicReference<>(), new LinkedList<>(), new HashMap<>(), mock(KinesisProxyInterface.class), kinesisV2) {

        @Override
        protected void deregisterStreamConsumer() {
            throw new RuntimeException();
        }
    };
    fetcher.shutdownFetcher();
    verify(kinesisV2).close();
    fetcher.awaitTermination();
}
Also used : SimpleStringSchema(org.apache.flink.api.common.serialization.SimpleStringSchema) KinesisProxyV2Interface(org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface) KinesisProxyInterface(org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyInterface) TestableKinesisDataFetcher(org.apache.flink.streaming.connectors.kinesis.testutils.TestableKinesisDataFetcher) Test(org.junit.Test)

Example 3 with KinesisProxyV2Interface

use of org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface in project flink by apache.

the class FanOutRecordPublisherTest method testInterruptedPublisherReturnsCancelled.

@Test
public void testInterruptedPublisherReturnsCancelled() throws Exception {
    KinesisProxyV2Interface kinesis = FakeKinesisFanOutBehavioursFactory.errorDuringSubscription(new SdkInterruptedException(null));
    RecordPublisher publisher = createRecordPublisher(kinesis, StartingPosition.continueFromSequenceNumber(SEQUENCE_NUMBER));
    RecordPublisherRunResult actual = publisher.run(new TestConsumer());
    assertEquals(CANCELLED, actual);
}
Also used : RecordPublisher(org.apache.flink.streaming.connectors.kinesis.internals.publisher.RecordPublisher) RecordPublisherRunResult(org.apache.flink.streaming.connectors.kinesis.internals.publisher.RecordPublisher.RecordPublisherRunResult) SdkInterruptedException(com.amazonaws.http.timers.client.SdkInterruptedException) KinesisProxyV2Interface(org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface) TestConsumer(org.apache.flink.streaming.connectors.kinesis.testutils.TestUtils.TestConsumer) Test(org.junit.Test)

Example 4 with KinesisProxyV2Interface

use of org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface in project flink by apache.

the class FanOutRecordPublisherTest method testResourceNotFoundWhenObtainingSubscriptionTerminatesApplication.

@Test
public void testResourceNotFoundWhenObtainingSubscriptionTerminatesApplication() throws Exception {
    thrown.expect(ResourceNotFoundException.class);
    KinesisProxyV2Interface kinesis = FakeKinesisFanOutBehavioursFactory.resourceNotFoundWhenObtainingSubscription();
    RecordPublisher recordPublisher = createRecordPublisher(kinesis);
    recordPublisher.run(new TestConsumer());
}
Also used : RecordPublisher(org.apache.flink.streaming.connectors.kinesis.internals.publisher.RecordPublisher) KinesisProxyV2Interface(org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface) TestConsumer(org.apache.flink.streaming.connectors.kinesis.testutils.TestUtils.TestConsumer) Test(org.junit.Test)

Example 5 with KinesisProxyV2Interface

use of org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface in project flink by apache.

the class StreamConsumerRegistrarTest method testRegisterStreamConsumerRegistersNewStreamConsumer.

@Test
public void testRegisterStreamConsumerRegistersNewStreamConsumer() throws Exception {
    FullJitterBackoff backoff = mock(FullJitterBackoff.class);
    KinesisProxyV2Interface kinesis = FakeKinesisFanOutBehavioursFactory.streamConsumerNotFound();
    StreamConsumerRegistrar registrar = createRegistrar(kinesis, backoff);
    String result = registrar.registerStreamConsumer(STREAM, "name");
    assertEquals(STREAM_CONSUMER_ARN_NEW, result);
}
Also used : KinesisProxyV2Interface(org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface) FullJitterBackoff(org.apache.flink.streaming.connectors.kinesis.proxy.FullJitterBackoff) Test(org.junit.Test)

Aggregations

KinesisProxyV2Interface (org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface)15 Test (org.junit.Test)14 FullJitterBackoff (org.apache.flink.streaming.connectors.kinesis.proxy.FullJitterBackoff)6 SdkInterruptedException (com.amazonaws.http.timers.client.SdkInterruptedException)3 RecordPublisher (org.apache.flink.streaming.connectors.kinesis.internals.publisher.RecordPublisher)3 TestConsumer (org.apache.flink.streaming.connectors.kinesis.testutils.TestUtils.TestConsumer)3 Properties (java.util.Properties)2 SimpleStringSchema (org.apache.flink.api.common.serialization.SimpleStringSchema)2 ShardConsumerTestUtils.fakeSequenceNumber (org.apache.flink.streaming.connectors.kinesis.internals.ShardConsumerTestUtils.fakeSequenceNumber)2 SequenceNumber (org.apache.flink.streaming.connectors.kinesis.model.SequenceNumber)2 KinesisProxyInterface (org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyInterface)2 TestableKinesisDataFetcher (org.apache.flink.streaming.connectors.kinesis.testutils.TestableKinesisDataFetcher)2 StartingPosition (software.amazon.awssdk.services.kinesis.model.StartingPosition)2 SdkClientException (com.amazonaws.SdkClientException)1 UserRecord (com.amazonaws.services.kinesis.clientlibrary.types.UserRecord)1 Date (java.util.Date)1 RecordPublisherRunResult (org.apache.flink.streaming.connectors.kinesis.internals.publisher.RecordPublisher.RecordPublisherRunResult)1 FanOutRecordPublisherConfiguration (org.apache.flink.streaming.connectors.kinesis.internals.publisher.fanout.FanOutRecordPublisherConfiguration)1 FanOutRecordPublisherFactory (org.apache.flink.streaming.connectors.kinesis.internals.publisher.fanout.FanOutRecordPublisherFactory)1 StreamConsumerRegistrar (org.apache.flink.streaming.connectors.kinesis.internals.publisher.fanout.StreamConsumerRegistrar)1