Search in sources :

Example 1 with Record

use of software.amazon.awssdk.services.firehose.model.Record in project flink by apache.

the class KinesisFirehoseSinkWriterTest method getSizeInBytesReturnsSizeOfBlobBeforeBase64Encoding.

@Test
public void getSizeInBytesReturnsSizeOfBlobBeforeBase64Encoding() {
    String testString = "{many hands make light work;";
    Record record = Record.builder().data(SdkBytes.fromUtf8String(testString)).build();
    assertThat(sinkWriter.getSizeInBytes(record)).isEqualTo(testString.getBytes(StandardCharsets.US_ASCII).length);
}
Also used : Record(software.amazon.awssdk.services.firehose.model.Record) Test(org.junit.Test)

Example 2 with Record

use of software.amazon.awssdk.services.firehose.model.Record in project aws-doc-sdk-examples by awsdocs.

the class PutRecord method putSingleRecord.

// snippet-start:[firehose.java2.put_record.main]
public static void putSingleRecord(FirehoseClient firehoseClient, String textValue, String streamName) {
    try {
        SdkBytes sdkBytes = SdkBytes.fromByteArray(textValue.getBytes());
        Record record = Record.builder().data(sdkBytes).build();
        PutRecordRequest recordRequest = PutRecordRequest.builder().deliveryStreamName(streamName).record(record).build();
        PutRecordResponse recordResponse = firehoseClient.putRecord(recordRequest);
        System.out.println("The record ID is " + recordResponse.recordId());
    } catch (FirehoseException e) {
        System.out.println(e.getLocalizedMessage());
        System.exit(1);
    }
}
Also used : SdkBytes(software.amazon.awssdk.core.SdkBytes) PutRecordResponse(software.amazon.awssdk.services.firehose.model.PutRecordResponse) PutRecordRequest(software.amazon.awssdk.services.firehose.model.PutRecordRequest) FirehoseException(software.amazon.awssdk.services.firehose.model.FirehoseException) Record(software.amazon.awssdk.services.firehose.model.Record)

Example 3 with Record

use of software.amazon.awssdk.services.firehose.model.Record in project aws-doc-sdk-examples by awsdocs.

the class PutBatchRecords method addStockTradeData.

// snippet-start:[firehose.java2.put_batch_records.main]
public static void addStockTradeData(FirehoseClient firehoseClient, String streamName) {
    List<Record> recordList = new ArrayList<>();
    try {
        // Repeatedly send stock trades with a 100 milliseconds wait in between
        StockTradeGenerator stockTradeGenerator = new StockTradeGenerator();
        int index = 100;
        // Populate the list with StockTrade data
        for (int x = 0; x < index; x++) {
            StockTrade trade = stockTradeGenerator.getRandomTrade();
            byte[] bytes = trade.toJsonAsBytes();
            Record myRecord = Record.builder().data(SdkBytes.fromByteArray(bytes)).build();
            System.out.println("Adding trade: " + trade.toString());
            recordList.add(myRecord);
            Thread.sleep(100);
        }
        PutRecordBatchRequest recordBatchRequest = PutRecordBatchRequest.builder().deliveryStreamName(streamName).records(recordList).build();
        PutRecordBatchResponse recordResponse = firehoseClient.putRecordBatch(recordBatchRequest);
        System.out.println("The number of records added is: " + recordResponse.requestResponses().size());
        // Check the details of all records in this batch operation.
        String errorMsg = "";
        String errorCode = "";
        List<PutRecordBatchResponseEntry> results = recordResponse.requestResponses();
        for (PutRecordBatchResponseEntry result : results) {
            // Returns null if there is no error.
            errorCode = result.errorCode();
            if (errorCode == null) {
                System.out.println("Record " + result.recordId() + " was successfully added!");
            } else {
                errorMsg = result.errorMessage();
                System.out.println("Error code for record ID : " + result.recordId() + " is " + errorMsg);
            }
        }
    } catch (FirehoseException | InterruptedException e) {
        System.out.println(e.getLocalizedMessage());
        System.exit(1);
    }
}
Also used : ArrayList(java.util.ArrayList) PutRecordBatchResponse(software.amazon.awssdk.services.firehose.model.PutRecordBatchResponse) FirehoseException(software.amazon.awssdk.services.firehose.model.FirehoseException) Record(software.amazon.awssdk.services.firehose.model.Record) PutRecordBatchRequest(software.amazon.awssdk.services.firehose.model.PutRecordBatchRequest) PutRecordBatchResponseEntry(software.amazon.awssdk.services.firehose.model.PutRecordBatchResponseEntry)

Example 4 with Record

use of software.amazon.awssdk.services.firehose.model.Record in project flink by apache.

the class KinesisFirehoseSinkElementConverterTest method elementConverterUsesProvidedSchemaToSerializeRecord.

@Test
public void elementConverterUsesProvidedSchemaToSerializeRecord() {
    ElementConverter<String, Record> elementConverter = KinesisFirehoseSinkElementConverter.<String>builder().setSerializationSchema(new SimpleStringSchema()).build();
    String testString = "{many hands make light work;";
    Record serializedRecord = elementConverter.apply(testString, null);
    byte[] serializedString = (new SimpleStringSchema()).serialize(testString);
    assertThat(serializedRecord.data()).isEqualTo(SdkBytes.fromByteArray(serializedString));
}
Also used : SimpleStringSchema(org.apache.flink.api.common.serialization.SimpleStringSchema) Record(software.amazon.awssdk.services.firehose.model.Record) Test(org.junit.Test)

Aggregations

Record (software.amazon.awssdk.services.firehose.model.Record)4 Test (org.junit.Test)2 FirehoseException (software.amazon.awssdk.services.firehose.model.FirehoseException)2 ArrayList (java.util.ArrayList)1 SimpleStringSchema (org.apache.flink.api.common.serialization.SimpleStringSchema)1 SdkBytes (software.amazon.awssdk.core.SdkBytes)1 PutRecordBatchRequest (software.amazon.awssdk.services.firehose.model.PutRecordBatchRequest)1 PutRecordBatchResponse (software.amazon.awssdk.services.firehose.model.PutRecordBatchResponse)1 PutRecordBatchResponseEntry (software.amazon.awssdk.services.firehose.model.PutRecordBatchResponseEntry)1 PutRecordRequest (software.amazon.awssdk.services.firehose.model.PutRecordRequest)1 PutRecordResponse (software.amazon.awssdk.services.firehose.model.PutRecordResponse)1