Search in sources :

Example 11 with DatabusRuntimeException

use of com.linkedin.databus.core.DatabusRuntimeException in project databus by linkedin.

the class ORListener method frameAvroRecord.

private void frameAvroRecord(long tableId, BinlogEventV4Header bh, List<Row> rl, final DbusOpcode doc) {
    try {
        final long timestampInNanos = bh.getTimestamp() * 1000000L;
        final long scn = scn(_currFileNum, (int) bh.getPosition());
        final boolean isReplicated = false;
        final TableMapEvent tme = _tableMapEvents.get(tableId);
        String tableName = tme.getDatabaseName().toString().toLowerCase() + "." + tme.getTableName().toString().toLowerCase();
        VersionedSchema vs = _schemaRegistryService.fetchLatestVersionedSchemaBySourceName(_tableUriToSrcNameMap.get(tableName));
        Schema schema = vs.getSchema();
        if (_log.isDebugEnabled())
            _log.debug("File Number :" + _currFileNum + ", Position :" + (int) bh.getPosition() + ", SCN =" + scn);
        for (Row r : rl) {
            List<Column> cl = r.getColumns();
            GenericRecord gr = new GenericData.Record(schema);
            generateAvroEvent(schema, cl, gr);
            List<KeyPair> kps = generateKeyPair(cl, schema);
            DbChangeEntry db = new DbChangeEntry(scn, timestampInNanos, gr, doc, isReplicated, schema, kps);
            _transaction.getPerSourceTransaction(_tableUriToSrcIdMap.get(tableName)).mergeDbChangeEntrySet(db);
        }
    } catch (NoSuchSchemaException ne) {
        throw new DatabusRuntimeException(ne);
    } catch (DatabusException de) {
        throw new DatabusRuntimeException(de);
    }
}
Also used : TableMapEvent(com.google.code.or.binlog.impl.event.TableMapEvent) KeyPair(com.linkedin.databus2.producers.ds.KeyPair) Schema(org.apache.avro.Schema) PrimaryKeySchema(com.linkedin.databus2.producers.ds.PrimaryKeySchema) VersionedSchema(com.linkedin.databus2.schemas.VersionedSchema) VersionedSchema(com.linkedin.databus2.schemas.VersionedSchema) DbChangeEntry(com.linkedin.databus2.producers.ds.DbChangeEntry) NoSuchSchemaException(com.linkedin.databus2.schemas.NoSuchSchemaException) DatabusException(com.linkedin.databus2.core.DatabusException) FloatColumn(com.google.code.or.common.glossary.column.FloatColumn) SetColumn(com.google.code.or.common.glossary.column.SetColumn) DatetimeColumn(com.google.code.or.common.glossary.column.DatetimeColumn) EnumColumn(com.google.code.or.common.glossary.column.EnumColumn) TimestampColumn(com.google.code.or.common.glossary.column.TimestampColumn) TimeColumn(com.google.code.or.common.glossary.column.TimeColumn) BitColumn(com.google.code.or.common.glossary.column.BitColumn) LongColumn(com.google.code.or.common.glossary.column.LongColumn) YearColumn(com.google.code.or.common.glossary.column.YearColumn) Datetime2Column(com.google.code.or.common.glossary.column.Datetime2Column) DecimalColumn(com.google.code.or.common.glossary.column.DecimalColumn) DoubleColumn(com.google.code.or.common.glossary.column.DoubleColumn) LongLongColumn(com.google.code.or.common.glossary.column.LongLongColumn) TinyColumn(com.google.code.or.common.glossary.column.TinyColumn) Column(com.google.code.or.common.glossary.Column) StringColumn(com.google.code.or.common.glossary.column.StringColumn) Int24Column(com.google.code.or.common.glossary.column.Int24Column) BlobColumn(com.google.code.or.common.glossary.column.BlobColumn) NullColumn(com.google.code.or.common.glossary.column.NullColumn) ShortColumn(com.google.code.or.common.glossary.column.ShortColumn) DateColumn(com.google.code.or.common.glossary.column.DateColumn) GenericRecord(org.apache.avro.generic.GenericRecord) Row(com.google.code.or.common.glossary.Row) GenericRecord(org.apache.avro.generic.GenericRecord) DatabusRuntimeException(com.linkedin.databus.core.DatabusRuntimeException)

Example 12 with DatabusRuntimeException

use of com.linkedin.databus.core.DatabusRuntimeException in project databus by linkedin.

the class DbusEventAvroDecoder method getMetadataSchema.

/**
   * Returns the single version of the metadata schema specified in the given event's header.
   * For INTERNAL USE ONLY (by Espresso and Databus).  This is not a stable API and may change
   * without warning!
   *
   * @param e DbusEvent
   * @return {AvroSchema, "metadata-source", version} tuple for given event 'e' with
   *         metadata-schema-id; null if event contains no metadata
   * @throws DatabusRuntimeException if event contains metadata but schema to decode it is missing
   */
public VersionedSchema getMetadataSchema(DbusEvent e) {
    DbusEventPart metadataPart = e.getPayloadMetadataPart();
    if (null == metadataPart) {
        LOG.debug("No metadata for event " + e);
        return null;
    }
    VersionedSchema schema = getMetadataSchema(metadataPart);
    if (null == schema) {
        throw new DatabusRuntimeException("No schema available to decode metadata for event " + e);
    }
    return schema;
}
Also used : DbusEventPart(com.linkedin.databus.core.DbusEventPart) VersionedSchema(com.linkedin.databus2.schemas.VersionedSchema) DatabusRuntimeException(com.linkedin.databus.core.DatabusRuntimeException)

Aggregations

DatabusRuntimeException (com.linkedin.databus.core.DatabusRuntimeException)12 VersionedSchema (com.linkedin.databus2.schemas.VersionedSchema)5 DatabusException (com.linkedin.databus2.core.DatabusException)4 ByteBuffer (java.nio.ByteBuffer)4 BitColumn (com.google.code.or.common.glossary.column.BitColumn)2 BlobColumn (com.google.code.or.common.glossary.column.BlobColumn)2 DateColumn (com.google.code.or.common.glossary.column.DateColumn)2 Datetime2Column (com.google.code.or.common.glossary.column.Datetime2Column)2 DatetimeColumn (com.google.code.or.common.glossary.column.DatetimeColumn)2 DecimalColumn (com.google.code.or.common.glossary.column.DecimalColumn)2 DoubleColumn (com.google.code.or.common.glossary.column.DoubleColumn)2 EnumColumn (com.google.code.or.common.glossary.column.EnumColumn)2 FloatColumn (com.google.code.or.common.glossary.column.FloatColumn)2 Int24Column (com.google.code.or.common.glossary.column.Int24Column)2 LongColumn (com.google.code.or.common.glossary.column.LongColumn)2 LongLongColumn (com.google.code.or.common.glossary.column.LongLongColumn)2 NullColumn (com.google.code.or.common.glossary.column.NullColumn)2 SetColumn (com.google.code.or.common.glossary.column.SetColumn)2 ShortColumn (com.google.code.or.common.glossary.column.ShortColumn)2 StringColumn (com.google.code.or.common.glossary.column.StringColumn)2