Search in sources :

Example 96 with Record

use of org.apache.avro.generic.GenericData.Record in project incubator-gobblin by apache.

the class AvroUtils method decorateRecord.

/**
 * Decorate a {@link GenericRecord} with additional fields and make it conform to an extended Schema
 * It is the caller's responsibility to ensure that the outputSchema is the merge of the inputRecord's schema
 * and the additional fields. The method does not check this for performance reasons, because it is expected to be called in the
 * critical path of processing a record.
 * Use {@link AvroUtils#decorateRecordSchema(Schema, List)} to generate such a Schema before calling this method.
 * @param inputRecord: record with data to be copied into the output record
 * @param fieldMap: values can be primitive types or GenericRecords if nested
 * @param outputSchema: the schema that the decoratedRecord will conform to
 * @return an outputRecord that contains a union of the fields in the inputRecord and the field-values in the fieldMap
 */
public static GenericRecord decorateRecord(GenericRecord inputRecord, @Nonnull Map<String, Object> fieldMap, Schema outputSchema) {
    GenericRecord outputRecord = new GenericData.Record(outputSchema);
    inputRecord.getSchema().getFields().forEach(f -> outputRecord.put(f.name(), inputRecord.get(f.name())));
    fieldMap.forEach((key, value) -> outputRecord.put(key, value));
    return outputRecord;
}
Also used : Record(org.apache.avro.generic.GenericData.Record) GenericRecord(org.apache.avro.generic.GenericRecord) GenericRecord(org.apache.avro.generic.GenericRecord)

Aggregations

Record (org.apache.avro.generic.GenericData.Record)96 Test (org.junit.Test)44 IndexedRecord (org.apache.avro.generic.IndexedRecord)43 Schema (org.apache.avro.Schema)33 ArrayList (java.util.ArrayList)24 GenericRecord (org.apache.avro.generic.GenericRecord)14 Field (org.apache.avro.Schema.Field)11 List (java.util.List)10 GenericData (org.apache.avro.generic.GenericData)10 TestRunner (org.apache.nifi.util.TestRunner)8 GenericRecordBuilder (org.apache.avro.generic.GenericRecordBuilder)7 JsonObject (com.google.gson.JsonObject)6 DataFileStream (org.apache.avro.file.DataFileStream)6 DataFileWriter (org.apache.avro.file.DataFileWriter)6 GenericDatumReader (org.apache.avro.generic.GenericDatumReader)6 Utf8 (org.apache.avro.util.Utf8)6 TMarketoOutputProperties (org.talend.components.marketo.tmarketooutput.TMarketoOutputProperties)6 ActivityRecord (com.marketo.mktows.ActivityRecord)5 ArrayOfLeadRecord (com.marketo.mktows.ArrayOfLeadRecord)5 LeadChangeRecord (com.marketo.mktows.LeadChangeRecord)5