Search in sources :

Example 16 with SpecificDatumWriter

use of org.apache.avro.specific.SpecificDatumWriter in project gora by apache.

the class AvroSerializerUtil method serializer.

@SuppressWarnings({ "unchecked", "rawtypes" })
public static <T> byte[] serializer(T value, Schema schema) throws IOException {
    SpecificDatumWriter writer = writerMap.get(schema.getFullName());
    if (writer == null) {
        // ignore dirty bits
        writer = new SpecificDatumWriter(schema);
        writerMap.put(schema.getFullName(), writer);
    }
    BinaryEncoder encoderFromCache = encoders.get();
    ByteArrayOutputStream bos = new ByteArrayOutputStream();
    outputStream.set(bos);
    BinaryEncoder encoder = EncoderFactory.get().directBinaryEncoder(bos, null);
    if (encoderFromCache == null) {
        encoders.set(encoder);
    }
    //reset the buffers
    ByteArrayOutputStream os = outputStream.get();
    os.reset();
    writer.write(value, encoder);
    encoder.flush();
    byte[] byteValue = os.toByteArray();
    return byteValue;
}
Also used : BinaryEncoder(org.apache.avro.io.BinaryEncoder) ByteArrayOutputStream(java.io.ByteArrayOutputStream) SpecificDatumWriter(org.apache.avro.specific.SpecificDatumWriter)

Example 17 with SpecificDatumWriter

use of org.apache.avro.specific.SpecificDatumWriter in project sling by apache.

the class AvroContentSerializer method exportToStream.

@Override
public void exportToStream(ResourceResolver resourceResolver, DistributionExportOptions options, OutputStream outputStream) throws DistributionException {
    DatumWriter<AvroShallowResource> datumWriter = new SpecificDatumWriter<AvroShallowResource>(AvroShallowResource.class);
    DataFileWriter<AvroShallowResource> writer = new DataFileWriter<AvroShallowResource>(datumWriter);
    try {
        writer.create(schema, outputStream);
    } catch (IOException e) {
        throw new DistributionException(e);
    }
    try {
        DistributionExportFilter filter = options.getFilter();
        for (DistributionExportFilter.TreeFilter treeFilter : filter.getNodeFilters()) {
            String path = treeFilter.getPath();
            Resource resource = resourceResolver.getResource(path);
            AvroShallowResource avroShallowResource = getAvroShallowResource(treeFilter, filter.getPropertyFilter(), resource);
            writer.append(avroShallowResource);
        }
        outputStream.flush();
    } catch (Exception e) {
        throw new DistributionException(e);
    } finally {
        try {
            writer.close();
        } catch (IOException e) {
        // do nothing
        }
    }
}
Also used : DataFileWriter(org.apache.avro.file.DataFileWriter) Resource(org.apache.sling.api.resource.Resource) DistributionException(org.apache.sling.distribution.common.DistributionException) DistributionExportFilter(org.apache.sling.distribution.serialization.DistributionExportFilter) IOException(java.io.IOException) DistributionException(org.apache.sling.distribution.common.DistributionException) PersistenceException(org.apache.sling.api.resource.PersistenceException) IOException(java.io.IOException) SpecificDatumWriter(org.apache.avro.specific.SpecificDatumWriter)

Aggregations

SpecificDatumWriter (org.apache.avro.specific.SpecificDatumWriter)17 DataFileWriter (org.apache.avro.file.DataFileWriter)7 ByteArrayOutputStream (java.io.ByteArrayOutputStream)5 Schema (org.apache.avro.Schema)4 BinaryEncoder (org.apache.avro.io.BinaryEncoder)4 IOException (java.io.IOException)3 ArrayList (java.util.ArrayList)3 HashMap (java.util.HashMap)3 Encoder (org.apache.avro.io.Encoder)3 Person (org.apache.crunch.test.Person)3 FileOutputStream (java.io.FileOutputStream)2 OutputBuffer (org.apache.cassandra.io.util.OutputBuffer)2 Address (org.apache.flink.api.io.avro.generated.Address)2 User (org.apache.flink.api.io.avro.generated.User)2 Address (org.apache.hadoop.hive.hbase.avro.Address)2 ContactInfo (org.apache.hadoop.hive.hbase.avro.ContactInfo)2 Employee (org.apache.hadoop.hive.hbase.avro.Employee)2 HomePhone (org.apache.hadoop.hive.hbase.avro.HomePhone)2 OfficePhone (org.apache.hadoop.hive.hbase.avro.OfficePhone)2 Before (org.junit.Before)2