Search in sources :

Example 6 with SeekableByteArrayInput

use of org.apache.avro.file.SeekableByteArrayInput in project nifi by apache.

the class TestMergeContent method getGenericRecordMap.

private Map<String, GenericRecord> getGenericRecordMap(byte[] data, Schema schema, String key) throws IOException {
    // create a reader for the merged contet
    DatumReader<GenericRecord> datumReader = new GenericDatumReader<>(schema);
    SeekableByteArrayInput input = new SeekableByteArrayInput(data);
    DataFileReader<GenericRecord> dataFileReader = new DataFileReader<>(input, datumReader);
    // read all the records into a map to verify all the records are there
    Map<String, GenericRecord> records = new HashMap<>();
    while (dataFileReader.hasNext()) {
        GenericRecord user = dataFileReader.next();
        records.put(user.get(key).toString(), user);
    }
    return records;
}
Also used : DataFileReader(org.apache.avro.file.DataFileReader) HashMap(java.util.HashMap) GenericDatumReader(org.apache.avro.generic.GenericDatumReader) GenericRecord(org.apache.avro.generic.GenericRecord) SeekableByteArrayInput(org.apache.avro.file.SeekableByteArrayInput)

Aggregations

SeekableByteArrayInput (org.apache.avro.file.SeekableByteArrayInput)6 GenericDatumReader (org.apache.avro.generic.GenericDatumReader)5 GenericRecord (org.apache.avro.generic.GenericRecord)5 DataFileReader (org.apache.avro.file.DataFileReader)4 Schema (org.apache.avro.Schema)3 SeekableInput (org.apache.avro.file.SeekableInput)2 Utf8 (org.apache.avro.util.Utf8)2 Test (org.junit.Test)2 RowVisitor (com.thinkbiganalytics.nifi.thrift.api.RowVisitor)1 ByteArrayOutputStream (java.io.ByteArrayOutputStream)1 IOException (java.io.IOException)1 BigDecimal (java.math.BigDecimal)1 Date (java.sql.Date)1 ResultSet (java.sql.ResultSet)1 ResultSetMetaData (java.sql.ResultSetMetaData)1 Time (java.sql.Time)1 Timestamp (java.sql.Timestamp)1 HashMap (java.util.HashMap)1 LinkedList (java.util.LinkedList)1 DataFileStream (org.apache.avro.file.DataFileStream)1