Search in sources :

Example 6 with GetRecordsResult

use of com.amazonaws.services.kinesis.model.GetRecordsResult in project storm by apache.

the class KinesisConnection method fetchRecords.

GetRecordsResult fetchRecords(String shardIterator) {
    GetRecordsRequest getRecordsRequest = new GetRecordsRequest();
    getRecordsRequest.setShardIterator(shardIterator);
    getRecordsRequest.setLimit(kinesisConnectionInfo.getRecordsLimit());
    GetRecordsResult getRecordsResult = kinesisClient.getRecords(getRecordsRequest);
    return getRecordsResult;
}
Also used : GetRecordsResult(com.amazonaws.services.kinesis.model.GetRecordsResult) GetRecordsRequest(com.amazonaws.services.kinesis.model.GetRecordsRequest)

Example 7 with GetRecordsResult

use of com.amazonaws.services.kinesis.model.GetRecordsResult in project camel by apache.

the class KinesisConsumerTest method exchangePropertiesAreSet.

@Test
public void exchangePropertiesAreSet() throws Exception {
    String partitionKey = "partitionKey";
    String sequenceNumber = "1";
    when(kinesisClient.getRecords(any(GetRecordsRequest.class))).thenReturn(new GetRecordsResult().withNextShardIterator("nextShardIterator").withRecords(new Record().withSequenceNumber(sequenceNumber).withApproximateArrivalTimestamp(new Date(42)).withPartitionKey(partitionKey)));
    undertest.poll();
    final ArgumentCaptor<Exchange> exchangeCaptor = ArgumentCaptor.forClass(Exchange.class);
    verify(processor).process(exchangeCaptor.capture(), any(AsyncCallback.class));
    assertThat(exchangeCaptor.getValue().getIn().getHeader(KinesisConstants.APPROX_ARRIVAL_TIME, long.class), is(42L));
    assertThat(exchangeCaptor.getValue().getIn().getHeader(KinesisConstants.PARTITION_KEY, String.class), is(partitionKey));
    assertThat(exchangeCaptor.getValue().getIn().getHeader(KinesisConstants.SEQUENCE_NUMBER, String.class), is(sequenceNumber));
}
Also used : Exchange(org.apache.camel.Exchange) GetRecordsResult(com.amazonaws.services.kinesis.model.GetRecordsResult) AsyncCallback(org.apache.camel.AsyncCallback) Record(com.amazonaws.services.kinesis.model.Record) Date(java.util.Date) GetRecordsRequest(com.amazonaws.services.kinesis.model.GetRecordsRequest) Test(org.junit.Test)

Example 8 with GetRecordsResult

use of com.amazonaws.services.kinesis.model.GetRecordsResult in project camel by apache.

the class KinesisConsumerTest method setup.

@Before
public void setup() throws Exception {
    KinesisEndpoint endpoint = new KinesisEndpoint(null, "streamName", component);
    endpoint.setAmazonKinesisClient(kinesisClient);
    endpoint.setIteratorType(ShardIteratorType.LATEST);
    undertest = new KinesisConsumer(endpoint, processor);
    when(kinesisClient.getRecords(any(GetRecordsRequest.class))).thenReturn(new GetRecordsResult().withNextShardIterator("nextShardIterator"));
    when(kinesisClient.describeStream(any(DescribeStreamRequest.class))).thenReturn(new DescribeStreamResult().withStreamDescription(new StreamDescription().withShards(new Shard().withShardId("shardId"))));
    when(kinesisClient.getShardIterator(any(GetShardIteratorRequest.class))).thenReturn(new GetShardIteratorResult().withShardIterator("shardIterator"));
}
Also used : GetRecordsResult(com.amazonaws.services.kinesis.model.GetRecordsResult) StreamDescription(com.amazonaws.services.kinesis.model.StreamDescription) GetShardIteratorRequest(com.amazonaws.services.kinesis.model.GetShardIteratorRequest) GetShardIteratorResult(com.amazonaws.services.kinesis.model.GetShardIteratorResult) DescribeStreamRequest(com.amazonaws.services.kinesis.model.DescribeStreamRequest) DescribeStreamResult(com.amazonaws.services.kinesis.model.DescribeStreamResult) Shard(com.amazonaws.services.kinesis.model.Shard) GetRecordsRequest(com.amazonaws.services.kinesis.model.GetRecordsRequest) Before(org.junit.Before)

Example 9 with GetRecordsResult

use of com.amazonaws.services.kinesis.model.GetRecordsResult in project camel by apache.

the class KinesisConsumerTest method recordsAreSentToTheProcessor.

@Test
public void recordsAreSentToTheProcessor() throws Exception {
    when(kinesisClient.getRecords(any(GetRecordsRequest.class))).thenReturn(new GetRecordsResult().withNextShardIterator("nextShardIterator").withRecords(new Record().withSequenceNumber("1"), new Record().withSequenceNumber("2")));
    int messageCount = undertest.poll();
    assertThat(messageCount, is(2));
    final ArgumentCaptor<Exchange> exchangeCaptor = ArgumentCaptor.forClass(Exchange.class);
    verify(processor, times(2)).process(exchangeCaptor.capture(), any(AsyncCallback.class));
    assertThat(exchangeCaptor.getAllValues().get(0).getIn().getBody(Record.class).getSequenceNumber(), is("1"));
    assertThat(exchangeCaptor.getAllValues().get(1).getIn().getBody(Record.class).getSequenceNumber(), is("2"));
}
Also used : Exchange(org.apache.camel.Exchange) GetRecordsResult(com.amazonaws.services.kinesis.model.GetRecordsResult) AsyncCallback(org.apache.camel.AsyncCallback) Record(com.amazonaws.services.kinesis.model.Record) GetRecordsRequest(com.amazonaws.services.kinesis.model.GetRecordsRequest) Test(org.junit.Test)

Aggregations

GetRecordsResult (com.amazonaws.services.kinesis.model.GetRecordsResult)9 GetRecordsRequest (com.amazonaws.services.kinesis.model.GetRecordsRequest)6 Record (com.amazonaws.services.kinesis.model.Record)4 Exchange (org.apache.camel.Exchange)3 ExpiredIteratorException (com.amazonaws.services.kinesis.model.ExpiredIteratorException)2 ProvisionedThroughputExceededException (com.amazonaws.services.kinesis.model.ProvisionedThroughputExceededException)2 AsyncCallback (org.apache.camel.AsyncCallback)2 Test (org.junit.Test)2 AmazonServiceException (com.amazonaws.AmazonServiceException)1 UserRecord (com.amazonaws.services.kinesis.clientlibrary.types.UserRecord)1 DescribeStreamRequest (com.amazonaws.services.kinesis.model.DescribeStreamRequest)1 DescribeStreamResult (com.amazonaws.services.kinesis.model.DescribeStreamResult)1 GetShardIteratorRequest (com.amazonaws.services.kinesis.model.GetShardIteratorRequest)1 GetShardIteratorResult (com.amazonaws.services.kinesis.model.GetShardIteratorResult)1 Shard (com.amazonaws.services.kinesis.model.Shard)1 StreamDescription (com.amazonaws.services.kinesis.model.StreamDescription)1 BigInteger (java.math.BigInteger)1 Date (java.util.Date)1 HashMap (java.util.HashMap)1 LinkedList (java.util.LinkedList)1