Search in sources :

Example 11 with Record

use of org.apache.avro.generic.GenericData.Record in project voldemort by voldemort.

the class AvroGenericSerializerTest method testRoundtripAvroWithGenericRecord.

public void testRoundtripAvroWithGenericRecord() throws Exception {
    String jsonSchema = "{\"name\": \"Compact Disk\", \"type\": \"record\", " + "\"fields\": [" + "{\"name\": \"name\", \"type\": \"string\", \"order\": \"ascending\"}" + "]}";
    AvroGenericSerializer serializer = new AvroGenericSerializer(jsonSchema);
    Record record = new Record(Schema.parse(jsonSchema));
    // we need to use a Utf8 instance to map to a String.
    record.put("name", new Utf8("Hello"));
    byte[] bytes = serializer.toBytes(record);
    assertTrue(serializer.toObject(bytes).equals(record));
}
Also used : Utf8(org.apache.avro.util.Utf8) Record(org.apache.avro.generic.GenericData.Record)

Example 12 with Record

use of org.apache.avro.generic.GenericData.Record in project voldemort by voldemort.

the class AvroBackwardsCompatibilityTest method testAvroSchemaEvolution.

/*
     * This tests if a client tries to deserialize an object created using an
     * old schema is successful or not
     */
@Test
public void testAvroSchemaEvolution() throws IOException {
    String versionZero = "{\"type\": \"record\", \"name\": \"myrec\",\"fields\": [{ \"name\": \"original\", \"type\": \"string\" }]}";
    String versionOne = "{\"type\": \"record\", \"name\": \"myrec\",\"fields\": [{ \"name\": \"original\", \"type\": \"string\" } ," + "{ \"name\": \"new-field\", \"type\": \"string\", \"default\":\"\" }]}";
    Schema s0 = Schema.parse(versionZero);
    Schema s1 = Schema.parse(versionOne);
    Map<Integer, String> versions = new HashMap<Integer, String>();
    versions.put(0, versionZero);
    versions.put(1, versionOne);
    byte[] versionZeroBytes = writeVersion0(s0);
    GenericData.Record record = (Record) readVersion0(versions, versionZeroBytes);
}
Also used : Record(org.apache.avro.generic.GenericData.Record) HashMap(java.util.HashMap) Schema(org.apache.avro.Schema) Record(org.apache.avro.generic.GenericData.Record) GenericData(org.apache.avro.generic.GenericData) Test(org.junit.Test)

Example 13 with Record

use of org.apache.avro.generic.GenericData.Record in project voldemort by voldemort.

the class AvroBackwardsCompatibilityTest method writeVersion0.

private static byte[] writeVersion0(Schema s0) {
    GenericData.Record record = new GenericData.Record(s0);
    record.put("original", new Utf8("Abhinay"));
    AvroVersionedGenericSerializer serializer = new AvroVersionedGenericSerializer(s0.toString());
    return serializer.toBytes(record);
}
Also used : Record(org.apache.avro.generic.GenericData.Record) Utf8(org.apache.avro.util.Utf8) Record(org.apache.avro.generic.GenericData.Record) GenericData(org.apache.avro.generic.GenericData)

Aggregations

Record (org.apache.avro.generic.GenericData.Record)13 Utf8 (org.apache.avro.util.Utf8)6 GenericData (org.apache.avro.generic.GenericData)5 Test (org.junit.Test)5 GenericRecord (org.apache.avro.generic.GenericRecord)4 Iterator (java.util.Iterator)3 Schema (org.apache.avro.Schema)3 GenericRow (com.linkedin.pinot.core.data.GenericRow)1 PinotSegmentRecordReader (com.linkedin.pinot.core.data.readers.PinotSegmentRecordReader)1 File (java.io.File)1 IOException (java.io.IOException)1 URL (java.net.URL)1 HashMap (java.util.HashMap)1 DataFileWriter (org.apache.avro.file.DataFileWriter)1 GenericDatumReader (org.apache.avro.generic.GenericDatumReader)1 Pipeline (org.apache.crunch.Pipeline)1 MRPipeline (org.apache.crunch.impl.mr.MRPipeline)1 Path (org.apache.hadoop.fs.Path)1