Examples with GenericDatumReader org.apache.avro.generic.GenericDatumReader used on opensource projects

Example 1 with GenericDatumReader

use of org.apache.avro.generic.GenericDatumReader in project storm by apache.

the class AbstractAvroSerializer method read.

@Override
public GenericContainer read(Kryo kryo, Input input, Class<GenericContainer> aClass) {
    Schema theSchema = this.getSchema(input.readString());
    GenericDatumReader<GenericContainer> reader = new GenericDatumReader<>(theSchema);
    Decoder decoder = DecoderFactory.get().directBinaryDecoder(input, null);
    GenericContainer foo;
    try {
        foo = reader.read(null, decoder);
    } catch (IOException e) {
        throw new RuntimeException(e);
    }
    return foo;
}

Example 2 with GenericDatumReader

use of org.apache.avro.generic.GenericDatumReader in project hive by apache.

the class AvroGenericRecordWritable method readFields.

public void readFields(byte[] bytes, Schema writerSchema, Schema readerSchema) throws IOException {
    fileSchema = writerSchema;
    record = new GenericData.Record(writerSchema);
    GenericDatumReader<GenericRecord> gdr = new GenericDatumReader<GenericRecord>();
    gdr.setExpected(readerSchema);
    ByteArrayInputStream is = new ByteArrayInputStream(bytes);
    DataFileStream<GenericRecord> dfr = new DataFileStream<GenericRecord>(is, gdr);
    record = dfr.next(record);
    dfr.close();
}

Example 3 with GenericDatumReader

use of org.apache.avro.generic.GenericDatumReader in project trevni by cutting.

the class TestCases method fromJson.

private List<Object> fromJson(Schema schema, File file) throws Exception {
    InputStream in = new FileInputStream(file);
    List<Object> data = new ArrayList<Object>();
    try {
        DatumReader reader = new GenericDatumReader(schema);
        Decoder decoder = DecoderFactory.get().jsonDecoder(schema, in);
        while (true) data.add(reader.read(null, decoder));
    } catch (EOFException e) {
    } finally {
        in.close();
    }
    return data;
}

Example 4 with GenericDatumReader

use of org.apache.avro.generic.GenericDatumReader in project druid by druid-io.

the class InlineSchemaAvroBytesDecoder method parse.

@Override
public GenericRecord parse(ByteBuffer bytes) {
    DatumReader<GenericRecord> reader = new GenericDatumReader<GenericRecord>(schemaObj);
    ByteBufferInputStream inputStream = new ByteBufferInputStream(Collections.singletonList(bytes));
    try {
        return reader.read(null, DecoderFactory.get().binaryDecoder(inputStream, null));
    } catch (Exception e) {
        throw new ParseException(e, "Fail to decode avro message!");
    }
}

Example 5 with GenericDatumReader

use of org.apache.avro.generic.GenericDatumReader in project druid by druid-io.

the class InlineSchemasAvroBytesDecoder method parse.

// It is assumed that record has following format.
// byte 1 : version, static 0x1
// byte 2-5 : int schemaId
// remaining bytes would have avro data
@Override
public GenericRecord parse(ByteBuffer bytes) {
    if (bytes.remaining() < 5) {
        throw new ParseException("record must have at least 5 bytes carrying version and schemaId");
    }
    byte version = bytes.get();
    if (version != V1) {
        throw new ParseException("found record of arbitrary version [%s]", version);
    }
    int schemaId = bytes.getInt();
    Schema schemaObj = schemaObjs.get(schemaId);
    if (schemaObj == null) {
        throw new ParseException("Failed to find schema for id [%s]", schemaId);
    }
    try {
        DatumReader<GenericRecord> reader = new GenericDatumReader<GenericRecord>(schemaObj);
        ByteBufferInputStream inputStream = new ByteBufferInputStream(Collections.singletonList(bytes));
        return reader.read(null, DecoderFactory.get().binaryDecoder(inputStream, null));
    } catch (Exception e) {
        throw new ParseException(e, "Fail to decode avro message with schemaId [%s].", schemaId);
    }
}