Search in sources :

Example 1 with StreamRecord

use of software.amazon.qldb.tutorial.model.streams.StreamRecord in project amazon-qldb-dmv-sample-java by aws-samples.

the class StreamJournal method streamRecordsToJournalBlocks.

private static List<JournalBlock> streamRecordsToJournalBlocks() {
    Map<ByteBuffer, QldbRevision> revisionsByHash = new HashMap<>();
    recordBuffer.stream().filter(record -> record.getRecordType().equals("REVISION_DETAILS")).forEach(record -> {
        try {
            Revision revision = ((RevisionDetailsRecord) record.getPayload()).getRevision();
            byte[] revisionHash = revision.getHash();
            revisionsByHash.put(wrap(revisionHash).asReadOnlyBuffer(), QldbRevision.fromIon((IonStruct) Constants.MAPPER.writeValueAsIonValue(revision)));
        } catch (IOException e) {
            throw new IllegalArgumentException("Could not map RevisionDetailsRecord to QldbRevision.", e);
        }
    });
    return recordBuffer.stream().filter(streamRecord -> streamRecord.getRecordType().equals("BLOCK_SUMMARY")).map(streamRecord -> (BlockSummaryRecord) streamRecord.getPayload()).distinct().map(blockSummaryRecord -> blockSummaryRecordToJournalBlock(blockSummaryRecord, revisionsByHash)).sorted(Comparator.comparingLong(o -> o.getBlockAddress().getSequenceNo())).collect(Collectors.toList());
}
Also used : DefaultAWSCredentialsProviderChain(com.amazonaws.auth.DefaultAWSCredentialsProviderChain) Date(java.util.Date) JournalKinesisStreamDescription(com.amazonaws.services.qldb.model.JournalKinesisStreamDescription) LoggerFactory(org.slf4j.LoggerFactory) ByteBuffer(java.nio.ByteBuffer) CreateTable(software.amazon.qldb.tutorial.CreateTable) ObjectReader(com.fasterxml.jackson.databind.ObjectReader) AmazonIdentityManagementClientBuilder(com.amazonaws.services.identitymanagement.AmazonIdentityManagementClientBuilder) DescribeStreamResult(com.amazonaws.services.kinesis.model.DescribeStreamResult) RevisionDetailsRecord(software.amazon.qldb.tutorial.model.streams.RevisionDetailsRecord) StreamRecord(software.amazon.qldb.tutorial.model.streams.StreamRecord) StreamJournalToKinesisRequest(com.amazonaws.services.qldb.model.StreamJournalToKinesisRequest) Duration(java.time.Duration) Map(java.util.Map) ListJournalKinesisStreamsForLedgerResult(com.amazonaws.services.qldb.model.ListJournalKinesisStreamsForLedgerResult) AmazonIdentityManagement(com.amazonaws.services.identitymanagement.AmazonIdentityManagement) LEDGER_NAME(software.amazon.qldb.tutorial.Constants.LEDGER_NAME) CreateRoleRequest(com.amazonaws.services.identitymanagement.model.CreateRoleRequest) AmazonKinesis(com.amazonaws.services.kinesis.AmazonKinesis) ListJournalKinesisStreamsForLedgerRequest(com.amazonaws.services.qldb.model.ListJournalKinesisStreamsForLedgerRequest) AmazonQLDBClientBuilder(com.amazonaws.services.qldb.AmazonQLDBClientBuilder) Instant(java.time.Instant) Collectors(java.util.stream.Collectors) PutRolePolicyRequest(com.amazonaws.services.identitymanagement.model.PutRolePolicyRequest) Executors(java.util.concurrent.Executors) IonTextWriterBuilder(com.amazon.ion.system.IonTextWriterBuilder) List(java.util.List) IonReader(com.amazon.ion.IonReader) NoSuchEntityException(com.amazonaws.services.identitymanagement.model.NoSuchEntityException) GetRoleRequest(com.amazonaws.services.identitymanagement.model.GetRoleRequest) JournalBlock(software.amazon.qldb.tutorial.qldb.JournalBlock) Record(com.amazonaws.services.kinesis.model.Record) IonWriter(com.amazon.ion.IonWriter) ByteBuffer.wrap(java.nio.ByteBuffer.wrap) InsertDocument(software.amazon.qldb.tutorial.InsertDocument) IRecordProcessorCheckpointer(com.amazonaws.services.kinesis.clientlibrary.interfaces.IRecordProcessorCheckpointer) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) Function(java.util.function.Function) QldbRevision(software.amazon.qldb.tutorial.qldb.QldbRevision) STREAM_NAME(software.amazon.qldb.tutorial.Constants.STREAM_NAME) ArrayList(java.util.ArrayList) IonStruct(com.amazon.ion.IonStruct) Worker(com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker) ValidateQldbHashChain(software.amazon.qldb.tutorial.ValidateQldbHashChain) DescribeStreamRequest(com.amazonaws.services.kinesis.model.DescribeStreamRequest) IRecordProcessorFactory(com.amazonaws.services.kinesis.clientlibrary.interfaces.IRecordProcessorFactory) DescribeJournalKinesisStreamRequest(com.amazonaws.services.qldb.model.DescribeJournalKinesisStreamRequest) AWSCredentialsProvider(com.amazonaws.auth.AWSCredentialsProvider) ResourceNotFoundException(com.amazonaws.services.kinesis.model.ResourceNotFoundException) CreateLedger(software.amazon.qldb.tutorial.CreateLedger) BlockSummaryRecord(software.amazon.qldb.tutorial.model.streams.BlockSummaryRecord) IRecordProcessor(com.amazonaws.services.kinesis.clientlibrary.interfaces.IRecordProcessor) Logger(org.slf4j.Logger) StreamJournalToKinesisResult(com.amazonaws.services.qldb.model.StreamJournalToKinesisResult) AmazonQLDB(com.amazonaws.services.qldb.AmazonQLDB) SampleData(software.amazon.qldb.tutorial.model.SampleData) IonReaderBuilder(com.amazon.ion.system.IonReaderBuilder) CancelJournalKinesisStreamRequest(com.amazonaws.services.qldb.model.CancelJournalKinesisStreamRequest) GetRolePolicyRequest(com.amazonaws.services.identitymanagement.model.GetRolePolicyRequest) KinesisConfiguration(com.amazonaws.services.qldb.model.KinesisConfiguration) IOException(java.io.IOException) DescribeJournalKinesisStreamResult(com.amazonaws.services.qldb.model.DescribeJournalKinesisStreamResult) TRIM_HORIZON(com.amazonaws.services.kinesis.clientlibrary.lib.worker.InitialPositionInStream.TRIM_HORIZON) TimeUnit(java.util.concurrent.TimeUnit) Revision(software.amazon.qldb.tutorial.model.streams.Revision) DeleteLedger(software.amazon.qldb.tutorial.DeleteLedger) AmazonKinesisClientBuilder(com.amazonaws.services.kinesis.AmazonKinesisClientBuilder) ShutdownReason(com.amazonaws.services.kinesis.clientlibrary.lib.worker.ShutdownReason) Constants(software.amazon.qldb.tutorial.Constants) DeletionProtection(software.amazon.qldb.tutorial.DeletionProtection) Comparator(java.util.Comparator) KinesisClientLibConfiguration(com.amazonaws.services.kinesis.clientlibrary.lib.worker.KinesisClientLibConfiguration) RevisionDetailsRecord(software.amazon.qldb.tutorial.model.streams.RevisionDetailsRecord) IonStruct(com.amazon.ion.IonStruct) QldbRevision(software.amazon.qldb.tutorial.qldb.QldbRevision) Revision(software.amazon.qldb.tutorial.model.streams.Revision) HashMap(java.util.HashMap) QldbRevision(software.amazon.qldb.tutorial.qldb.QldbRevision) IOException(java.io.IOException) ByteBuffer(java.nio.ByteBuffer)

Aggregations

IonReader (com.amazon.ion.IonReader)1 IonStruct (com.amazon.ion.IonStruct)1 IonWriter (com.amazon.ion.IonWriter)1 IonReaderBuilder (com.amazon.ion.system.IonReaderBuilder)1 IonTextWriterBuilder (com.amazon.ion.system.IonTextWriterBuilder)1 AWSCredentialsProvider (com.amazonaws.auth.AWSCredentialsProvider)1 DefaultAWSCredentialsProviderChain (com.amazonaws.auth.DefaultAWSCredentialsProviderChain)1 AmazonIdentityManagement (com.amazonaws.services.identitymanagement.AmazonIdentityManagement)1 AmazonIdentityManagementClientBuilder (com.amazonaws.services.identitymanagement.AmazonIdentityManagementClientBuilder)1 CreateRoleRequest (com.amazonaws.services.identitymanagement.model.CreateRoleRequest)1 GetRolePolicyRequest (com.amazonaws.services.identitymanagement.model.GetRolePolicyRequest)1 GetRoleRequest (com.amazonaws.services.identitymanagement.model.GetRoleRequest)1 NoSuchEntityException (com.amazonaws.services.identitymanagement.model.NoSuchEntityException)1 PutRolePolicyRequest (com.amazonaws.services.identitymanagement.model.PutRolePolicyRequest)1 AmazonKinesis (com.amazonaws.services.kinesis.AmazonKinesis)1 AmazonKinesisClientBuilder (com.amazonaws.services.kinesis.AmazonKinesisClientBuilder)1 IRecordProcessor (com.amazonaws.services.kinesis.clientlibrary.interfaces.IRecordProcessor)1 IRecordProcessorCheckpointer (com.amazonaws.services.kinesis.clientlibrary.interfaces.IRecordProcessorCheckpointer)1 IRecordProcessorFactory (com.amazonaws.services.kinesis.clientlibrary.interfaces.IRecordProcessorFactory)1 TRIM_HORIZON (com.amazonaws.services.kinesis.clientlibrary.lib.worker.InitialPositionInStream.TRIM_HORIZON)1