Search in sources :

Example 1 with GetRecordsResponse

use of software.amazon.awssdk.services.kinesis.model.GetRecordsResponse in project beam by apache.

the class SimplifiedKinesisClient method getRecords.

/**
 * Gets records from Kinesis and deaggregates them if needed.
 *
 * @return list of deaggregated records
 * @throws TransientKinesisException - in case of recoverable situation
 */
public GetKinesisRecordsResult getRecords(final String shardIterator, final String streamName, final String shardId, final Integer limit) throws TransientKinesisException {
    return wrapExceptions(() -> {
        GetRecordsResponse response = kinesis.getRecords(GetRecordsRequest.builder().shardIterator(shardIterator).limit(limit).build());
        List<Record> records = response.records();
        return new GetKinesisRecordsResult(deaggregate(records), response.nextShardIterator(), response.millisBehindLatest(), streamName, shardId);
    });
}
Also used : GetRecordsResponse(software.amazon.awssdk.services.kinesis.model.GetRecordsResponse) KinesisClientRecord(software.amazon.kinesis.retrieval.KinesisClientRecord) Record(software.amazon.awssdk.services.kinesis.model.Record)

Example 2 with GetRecordsResponse

use of software.amazon.awssdk.services.kinesis.model.GetRecordsResponse in project aws-doc-sdk-examples by awsdocs.

the class GetRecords method getStockTrades.

// snippet-start:[kinesis.java2.getrecord.main]
public static void getStockTrades(KinesisClient kinesisClient, String streamName) {
    String shardIterator;
    String lastShardId = null;
    // Retrieve the Shards from a Stream
    DescribeStreamRequest describeStreamRequest = DescribeStreamRequest.builder().streamName(streamName).build();
    List<Shard> shards = new ArrayList<>();
    DescribeStreamResponse streamRes;
    do {
        streamRes = kinesisClient.describeStream(describeStreamRequest);
        shards.addAll(streamRes.streamDescription().shards());
        if (shards.size() > 0) {
            lastShardId = shards.get(shards.size() - 1).shardId();
        }
    } while (streamRes.streamDescription().hasMoreShards());
    GetShardIteratorRequest itReq = GetShardIteratorRequest.builder().streamName(streamName).shardIteratorType("TRIM_HORIZON").shardId(lastShardId).build();
    GetShardIteratorResponse shardIteratorResult = kinesisClient.getShardIterator(itReq);
    shardIterator = shardIteratorResult.shardIterator();
    // Continuously read data records from shard.
    List<Record> records;
    // Create new GetRecordsRequest with existing shardIterator.
    // Set maximum records to return to 1000.
    GetRecordsRequest recordsRequest = GetRecordsRequest.builder().shardIterator(shardIterator).limit(1000).build();
    GetRecordsResponse result = kinesisClient.getRecords(recordsRequest);
    // Put result into record list. Result may be empty.
    records = result.records();
    // Print records
    for (Record record : records) {
        SdkBytes byteBuffer = record.data();
        System.out.println(String.format("Seq No: %s - %s", record.sequenceNumber(), new String(byteBuffer.asByteArray())));
    }
}
Also used : ArrayList(java.util.ArrayList) GetShardIteratorRequest(software.amazon.awssdk.services.kinesis.model.GetShardIteratorRequest) GetRecordsResponse(software.amazon.awssdk.services.kinesis.model.GetRecordsResponse) DescribeStreamResponse(software.amazon.awssdk.services.kinesis.model.DescribeStreamResponse) DescribeStreamRequest(software.amazon.awssdk.services.kinesis.model.DescribeStreamRequest) GetRecordsRequest(software.amazon.awssdk.services.kinesis.model.GetRecordsRequest) SdkBytes(software.amazon.awssdk.core.SdkBytes) GetShardIteratorResponse(software.amazon.awssdk.services.kinesis.model.GetShardIteratorResponse) Record(software.amazon.awssdk.services.kinesis.model.Record) Shard(software.amazon.awssdk.services.kinesis.model.Shard)

Aggregations

GetRecordsResponse (software.amazon.awssdk.services.kinesis.model.GetRecordsResponse)2 Record (software.amazon.awssdk.services.kinesis.model.Record)2 ArrayList (java.util.ArrayList)1 SdkBytes (software.amazon.awssdk.core.SdkBytes)1 DescribeStreamRequest (software.amazon.awssdk.services.kinesis.model.DescribeStreamRequest)1 DescribeStreamResponse (software.amazon.awssdk.services.kinesis.model.DescribeStreamResponse)1 GetRecordsRequest (software.amazon.awssdk.services.kinesis.model.GetRecordsRequest)1 GetShardIteratorRequest (software.amazon.awssdk.services.kinesis.model.GetShardIteratorRequest)1 GetShardIteratorResponse (software.amazon.awssdk.services.kinesis.model.GetShardIteratorResponse)1 Shard (software.amazon.awssdk.services.kinesis.model.Shard)1 KinesisClientRecord (software.amazon.kinesis.retrieval.KinesisClientRecord)1