Search in sources :

Example 6 with KeyPair

use of com.linkedin.databus2.ggParser.XmlStateMachine.ColumnsState.KeyPair in project databus by linkedin.

the class ORListener method frameAvroRecord.

private void frameAvroRecord(long tableId, BinlogEventV4Header bh, List<Row> rl, final DbusOpcode doc) {
    try {
        final long timestampInNanos = bh.getTimestamp() * 1000000L;
        final long scn = scn(_currFileNum, (int) bh.getPosition());
        final boolean isReplicated = false;
        final TableMapEvent tme = _tableMapEvents.get(tableId);
        String tableName = tme.getDatabaseName().toString().toLowerCase() + "." + tme.getTableName().toString().toLowerCase();
        VersionedSchema vs = _schemaRegistryService.fetchLatestVersionedSchemaBySourceName(_tableUriToSrcNameMap.get(tableName));
        Schema schema = vs.getSchema();
        if (_log.isDebugEnabled())
            _log.debug("File Number :" + _currFileNum + ", Position :" + (int) bh.getPosition() + ", SCN =" + scn);
        for (Row r : rl) {
            List<Column> cl = r.getColumns();
            GenericRecord gr = new GenericData.Record(schema);
            generateAvroEvent(vs, cl, gr);
            List<KeyPair> kps = generateKeyPair(gr, vs);
            DbChangeEntry db = new DbChangeEntry(scn, timestampInNanos, gr, doc, isReplicated, schema, kps);
            _transaction.getPerSourceTransaction(_tableUriToSrcIdMap.get(tableName)).mergeDbChangeEntrySet(db);
        }
    } catch (NoSuchSchemaException ne) {
        throw new DatabusRuntimeException(ne);
    } catch (DatabusException de) {
        throw new DatabusRuntimeException(de);
    }
}
Also used : TableMapEvent(com.google.code.or.binlog.impl.event.TableMapEvent) KeyPair(com.linkedin.databus2.producers.ds.KeyPair) Schema(org.apache.avro.Schema) PrimaryKeySchema(com.linkedin.databus2.producers.ds.PrimaryKeySchema) VersionedSchema(com.linkedin.databus2.schemas.VersionedSchema) VersionedSchema(com.linkedin.databus2.schemas.VersionedSchema) DbChangeEntry(com.linkedin.databus2.producers.ds.DbChangeEntry) NoSuchSchemaException(com.linkedin.databus2.schemas.NoSuchSchemaException) DatabusException(com.linkedin.databus2.core.DatabusException) LongColumn(com.google.code.or.common.glossary.column.LongColumn) LongLongColumn(com.google.code.or.common.glossary.column.LongLongColumn) TinyColumn(com.google.code.or.common.glossary.column.TinyColumn) Column(com.google.code.or.common.glossary.Column) Int24Column(com.google.code.or.common.glossary.column.Int24Column) NullColumn(com.google.code.or.common.glossary.column.NullColumn) ShortColumn(com.google.code.or.common.glossary.column.ShortColumn) GenericRecord(org.apache.avro.generic.GenericRecord) Row(com.google.code.or.common.glossary.Row) GenericRecord(org.apache.avro.generic.GenericRecord) DatabusRuntimeException(com.linkedin.databus.core.DatabusRuntimeException)

Aggregations

DatabusException (com.linkedin.databus2.core.DatabusException)6 Schema (org.apache.avro.Schema)6 KeyPair (com.linkedin.databus2.producers.ds.KeyPair)4 PrimaryKeySchema (com.linkedin.databus2.producers.ds.PrimaryKeySchema)4 VersionedSchema (com.linkedin.databus2.schemas.VersionedSchema)3 ArrayList (java.util.ArrayList)2 Field (org.apache.avro.Schema.Field)2 TableMapEvent (com.google.code.or.binlog.impl.event.TableMapEvent)1 Column (com.google.code.or.common.glossary.Column)1 Row (com.google.code.or.common.glossary.Row)1 Int24Column (com.google.code.or.common.glossary.column.Int24Column)1 LongColumn (com.google.code.or.common.glossary.column.LongColumn)1 LongLongColumn (com.google.code.or.common.glossary.column.LongLongColumn)1 NullColumn (com.google.code.or.common.glossary.column.NullColumn)1 ShortColumn (com.google.code.or.common.glossary.column.ShortColumn)1 TinyColumn (com.google.code.or.common.glossary.column.TinyColumn)1 DatabusRuntimeException (com.linkedin.databus.core.DatabusRuntimeException)1 KeyPair (com.linkedin.databus2.ggParser.XmlStateMachine.ColumnsState.KeyPair)1 DbChangeEntry (com.linkedin.databus2.producers.ds.DbChangeEntry)1 NoSuchSchemaException (com.linkedin.databus2.schemas.NoSuchSchemaException)1