Search in sources :

Example 16 with VersionedSchema

use of com.linkedin.databus2.schemas.VersionedSchema in project databus by linkedin.

the class DbusEventAvroDecoder method dumpEventValueInJSON.

public void dumpEventValueInJSON(DbusEvent e, OutputStream out) {
    byte[] md5 = new byte[16];
    e.schemaId(md5);
    SchemaId schemaId = new SchemaId(md5);
    VersionedSchema sourceSchema = _schemaSet.getById(schemaId);
    ByteBuffer valueBuffer = e.value();
    byte[] valueBytes = new byte[valueBuffer.remaining()];
    valueBuffer.get(valueBytes);
    try {
        Schema schema = sourceSchema.getSchema();
        DatumReader<Object> reader = new GenericDatumReader<Object>(schema);
        binDecoder.set(DecoderFactory.defaultFactory().createBinaryDecoder(valueBytes, binDecoder.get()));
        Object datum = reader.read(null, binDecoder.get());
        DatumWriter<Object> writer = new GenericDatumWriter<Object>(schema);
        JsonGenerator g = new JsonFactory().createJsonGenerator(out, JsonEncoding.UTF8);
        // write the src ID
        g.writeStartObject();
        g.writeFieldName(SRC_ID_FIELD_NAME);
        g.writeNumber(e.getSourceId());
        g.writeFieldName(OPCODE_FIELD_NAME);
        g.writeString(e.getOpcode().toString());
        g.writeFieldName("partId");
        g.writeNumber(Integer.valueOf(e.getPartitionId()));
        g.writeFieldName(VALUE_FIELD_NAME);
        writer.write(datum, new JsonEncoder(schema, g));
        g.writeEndObject();
        g.writeEndObject();
        try {
            g.writeEndObject();
        } catch (JsonGenerationException e_json) {
        // ignore the error as some how avro JsonEncoder may some times missing two }
        }
        g.flush();
    } catch (IOException e1) {
        LOG.error("event value serialization error; event = " + e, e1);
    }
}
Also used : JsonEncoder(org.apache.avro.io.JsonEncoder) GenericDatumReader(org.apache.avro.generic.GenericDatumReader) Schema(org.apache.avro.Schema) VersionedSchema(com.linkedin.databus2.schemas.VersionedSchema) JsonFactory(org.codehaus.jackson.JsonFactory) GenericDatumWriter(org.apache.avro.generic.GenericDatumWriter) IOException(java.io.IOException) VersionedSchema(com.linkedin.databus2.schemas.VersionedSchema) ByteBuffer(java.nio.ByteBuffer) SchemaId(com.linkedin.databus2.schemas.SchemaId) JsonGenerator(org.codehaus.jackson.JsonGenerator) JsonGenerationException(org.codehaus.jackson.JsonGenerationException)

Example 17 with VersionedSchema

use of com.linkedin.databus2.schemas.VersionedSchema in project databus by linkedin.

the class DbusEventAvroDecoder method getMetadataSchema.

/**
   * Returns the single version of the metadata schema specified in the given event's header.
   * For INTERNAL USE ONLY (by Espresso and Databus).  This is not a stable API and may change
   * without warning!
   *
   * @param e DbusEvent
   * @return {AvroSchema, "metadata-source", version} tuple for given event 'e' with
   *         metadata-schema-id; null if event contains no metadata
   * @throws DatabusRuntimeException if event contains metadata but schema to decode it is missing
   */
public VersionedSchema getMetadataSchema(DbusEvent e) {
    DbusEventPart metadataPart = e.getPayloadMetadataPart();
    if (null == metadataPart) {
        LOG.debug("No metadata for event " + e);
        return null;
    }
    VersionedSchema schema = getMetadataSchema(metadataPart);
    if (null == schema) {
        throw new DatabusRuntimeException("No schema available to decode metadata for event " + e);
    }
    return schema;
}
Also used : DbusEventPart(com.linkedin.databus.core.DbusEventPart) VersionedSchema(com.linkedin.databus2.schemas.VersionedSchema) DatabusRuntimeException(com.linkedin.databus.core.DatabusRuntimeException)

Aggregations

VersionedSchema (com.linkedin.databus2.schemas.VersionedSchema)17 Schema (org.apache.avro.Schema)6 DatabusRuntimeException (com.linkedin.databus.core.DatabusRuntimeException)5 SchemaId (com.linkedin.databus2.schemas.SchemaId)4 DbusEventAvroDecoder (com.linkedin.databus.client.DbusEventAvroDecoder)3 DbusEventPart (com.linkedin.databus.core.DbusEventPart)3 DatabusException (com.linkedin.databus2.core.DatabusException)3 SchemaRegistryService (com.linkedin.databus2.schemas.SchemaRegistryService)3 IOException (java.io.IOException)3 ByteBuffer (java.nio.ByteBuffer)3 IdNamePair (com.linkedin.databus.core.util.IdNamePair)2 RegisterResponseMetadataEntry (com.linkedin.databus2.core.container.request.RegisterResponseMetadataEntry)2 FileSystemSchemaRegistryService (com.linkedin.databus2.schemas.FileSystemSchemaRegistryService)2 VersionedSchemaSet (com.linkedin.databus2.schemas.VersionedSchemaSet)2 TableMapEvent (com.google.code.or.binlog.impl.event.TableMapEvent)1 Column (com.google.code.or.common.glossary.Column)1 Row (com.google.code.or.common.glossary.Row)1 BitColumn (com.google.code.or.common.glossary.column.BitColumn)1 BlobColumn (com.google.code.or.common.glossary.column.BlobColumn)1 DateColumn (com.google.code.or.common.glossary.column.DateColumn)1