Search in sources :

Example 1 with Value

use of io.debezium.document.Value in project debezium by debezium.

the class Wal2JsonReplicationMessage method transform.

private List<ReplicationMessage.Column> transform(final Document data, final String nameField, final String typeField, final String valueField, final String optionalsField) {
    final Array columnNames = data.getArray(nameField);
    final Array columnTypes = data.getArray(typeField);
    final Array columnValues = data.getArray(valueField);
    final Array columnOptionals = data.getArray(optionalsField);
    if (columnNames.size() != columnTypes.size() || columnNames.size() != columnValues.size()) {
        throw new ConnectException("Column related arrays do not have the same size");
    }
    final List<ReplicationMessage.Column> columns = new ArrayList<>(columnNames.size());
    for (int i = 0; i < columnNames.size(); i++) {
        final String columnName = columnNames.get(i).asString();
        final String columnTypeName = columnTypes.get(i).asString();
        final boolean columnOptional = columnOptionals != null ? columnOptionals.get(i).asBoolean() : false;
        final Value rawValue = columnValues.get(i);
        final PostgresType columnType = typeRegistry.get(parseType(columnName, columnTypeName));
        columns.add(new AbstractReplicationMessageColumn(columnName, columnType, columnTypeName, columnOptional, true) {

            @Override
            public Object getValue(PgConnectionSupplier connection, boolean includeUnknownDatatypes) {
                return Wal2JsonReplicationMessage.this.getValue(columnName, columnType, columnTypeName, rawValue, connection, includeUnknownDatatypes);
            }
        });
    }
    return columns;
}
Also used : AbstractReplicationMessageColumn(io.debezium.connector.postgresql.connection.AbstractReplicationMessageColumn) ArrayList(java.util.ArrayList) PGpoint(org.postgresql.geometric.PGpoint) Array(io.debezium.document.Array) PgArray(org.postgresql.jdbc.PgArray) PostgresType(io.debezium.connector.postgresql.PostgresType) AbstractReplicationMessageColumn(io.debezium.connector.postgresql.connection.AbstractReplicationMessageColumn) Value(io.debezium.document.Value) PgConnectionSupplier(io.debezium.connector.postgresql.RecordsStreamProducer.PgConnectionSupplier) ConnectException(org.apache.kafka.connect.errors.ConnectException)

Example 2 with Value

use of io.debezium.document.Value in project debezium by debezium.

the class NonStreamingWal2JsonMessageDecoder method processMessage.

@Override
public void processMessage(ByteBuffer buffer, ReplicationMessageProcessor processor, TypeRegistry typeRegistry) throws SQLException, InterruptedException {
    try {
        if (!buffer.hasArray()) {
            throw new IllegalStateException("Invalid buffer received from PG server during streaming replication");
        }
        final byte[] source = buffer.array();
        final byte[] content = Arrays.copyOfRange(source, buffer.arrayOffset(), source.length);
        final Document message = DocumentReader.floatNumbersAsTextReader().read(content);
        LOGGER.debug("Message arrived for decoding {}", message);
        final long txId = message.getLong("xid");
        final String timestamp = message.getString("timestamp");
        final long commitTime = dateTime.systemTimestamp(timestamp);
        final Array changes = message.getArray("change");
        Iterator<Entry> it = changes.iterator();
        while (it.hasNext()) {
            Value value = it.next().getValue();
            processor.process(new Wal2JsonReplicationMessage(txId, commitTime, value.asDocument(), containsMetadata, !it.hasNext(), typeRegistry));
        }
    } catch (final IOException e) {
        throw new ConnectException(e);
    }
}
Also used : Array(io.debezium.document.Array) Entry(io.debezium.document.Array.Entry) Value(io.debezium.document.Value) IOException(java.io.IOException) Document(io.debezium.document.Document) ConnectException(org.apache.kafka.connect.errors.ConnectException)

Aggregations

Array (io.debezium.document.Array)2 Value (io.debezium.document.Value)2 ConnectException (org.apache.kafka.connect.errors.ConnectException)2 PostgresType (io.debezium.connector.postgresql.PostgresType)1 PgConnectionSupplier (io.debezium.connector.postgresql.RecordsStreamProducer.PgConnectionSupplier)1 AbstractReplicationMessageColumn (io.debezium.connector.postgresql.connection.AbstractReplicationMessageColumn)1 Entry (io.debezium.document.Array.Entry)1 Document (io.debezium.document.Document)1 IOException (java.io.IOException)1 ArrayList (java.util.ArrayList)1 PGpoint (org.postgresql.geometric.PGpoint)1 PgArray (org.postgresql.jdbc.PgArray)1