Examples with KinesisProxyV2Interface org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface used on opensource projects

Example 11 with KinesisProxyV2Interface

use of org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface in project flink by apache.

the class FanOutShardSubscriberTest method testTimeoutEnqueuingEvent.

@Test
public void testTimeoutEnqueuingEvent() throws Exception {
    thrown.expect(FanOutShardSubscriber.RecoverableFanOutSubscriberException.class);
    thrown.expectMessage("Timed out enqueuing event SubscriptionNextEvent");
    KinesisProxyV2Interface kinesis = FakeKinesisFanOutBehavioursFactory.shardThatCreatesBackpressureOnQueue();
    FanOutShardSubscriber subscriber = new FanOutShardSubscriber("consumerArn", "shardId", kinesis, DEFAULT_SUBSCRIBE_TO_SHARD_TIMEOUT, Duration.ofMillis(100));
    StartingPosition startingPosition = StartingPosition.builder().build();
    subscriber.subscribeToShardAndConsumeRecords(startingPosition, event -> {
        try {
            Thread.sleep(120);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    });
}

Example 12 with KinesisProxyV2Interface

use of org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface in project flink by apache.

the class StreamConsumerRegistrarTest method testCloseClosesProxy.

@Test
public void testCloseClosesProxy() {
    KinesisProxyV2Interface kinesis = mock(KinesisProxyV2Interface.class);
    StreamConsumerRegistrar registrar = createRegistrar(kinesis, mock(FullJitterBackoff.class));
    registrar.close();
    verify(kinesis).close();
}

Example 13 with KinesisProxyV2Interface

use of org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface in project flink by apache.

the class FanOutRecordPublisherTest method testToSdkV1Records.

@Test
public void testToSdkV1Records() throws Exception {
    Date now = new Date();
    byte[] data = new byte[] { 0, 1, 2, 3 };
    Record record = Record.builder().approximateArrivalTimestamp(now.toInstant()).partitionKey("pk").sequenceNumber("sn").data(SdkBytes.fromByteArray(data)).build();
    KinesisProxyV2Interface kinesis = singletonShard(createSubscribeToShardEvent(record));
    RecordPublisher publisher = createRecordPublisher(kinesis, latest());
    TestConsumer consumer = new TestConsumer();
    publisher.run(consumer);
    UserRecord actual = consumer.getRecordBatches().get(0).getDeaggregatedRecords().get(0);
    assertFalse(actual.isAggregated());
    assertEquals(now, actual.getApproximateArrivalTimestamp());
    assertEquals("sn", actual.getSequenceNumber());
    assertEquals("pk", actual.getPartitionKey());
    assertThat(toByteArray(actual.getData()), Matchers.equalTo(data));
}

Example 14 with KinesisProxyV2Interface

use of org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface in project flink by apache.

the class StreamConsumerRegistrarTest method testRegisterStreamConsumerThatAlreadyExistsAndActive.

@Test
public void testRegisterStreamConsumerThatAlreadyExistsAndActive() throws Exception {
    FullJitterBackoff backoff = mock(FullJitterBackoff.class);
    KinesisProxyV2Interface kinesis = FakeKinesisFanOutBehavioursFactory.existingActiveConsumer();
    StreamConsumerRegistrar registrar = createRegistrar(kinesis, backoff);
    String result = registrar.registerStreamConsumer(STREAM, "name");
    verify(backoff, never()).sleep(anyLong());
    assertEquals(STREAM_CONSUMER_ARN_EXISTING, result);
}

Example 15 with KinesisProxyV2Interface

use of org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxyV2Interface in project flink by apache.

the class StreamConsumerRegistrarUtil method createStreamConsumerRegistrar.

private static StreamConsumerRegistrar createStreamConsumerRegistrar(final Properties configProps, final List<String> streams) {
    FullJitterBackoff backoff = new FullJitterBackoff();
    FanOutRecordPublisherConfiguration configuration = new FanOutRecordPublisherConfiguration(configProps, streams);
    KinesisProxyV2Interface kinesis = KinesisProxyV2Factory.createKinesisProxyV2(configProps);
    return new StreamConsumerRegistrar(kinesis, configuration, backoff);
}