Search in sources :

Example 66 with DecoderColumnHandle

use of io.trino.decoder.DecoderColumnHandle in project trino by trinodb.

the class TestAvroConfluentRowDecoder method testDecodingRows.

@Test
public void testDecodingRows() throws Exception {
    MockSchemaRegistryClient mockSchemaRegistryClient = new MockSchemaRegistryClient();
    Schema initialSchema = SchemaBuilder.record(TOPIC).fields().name("col1").type().intType().noDefault().name("col2").type().stringType().noDefault().name("col3").type().intType().intDefault(42).name("col4").type().nullable().intType().noDefault().name("col5").type().nullable().bytesType().noDefault().endRecord();
    Schema evolvedSchema = SchemaBuilder.record(TOPIC).fields().name("col1").type().intType().noDefault().name("col2").type().stringType().noDefault().name("col3").type().intType().intDefault(3).name("col4").type().nullable().intType().noDefault().name("col5").type().nullable().bytesType().noDefault().name("col6").type().optional().longType().endRecord();
    mockSchemaRegistryClient.register(TOPIC + "-value", initialSchema);
    mockSchemaRegistryClient.register(TOPIC + "-value", evolvedSchema);
    Set<DecoderColumnHandle> columnHandles = ImmutableSet.<DecoderColumnHandle>builder().add(new KafkaColumnHandle("col1", INTEGER, "col1", null, null, false, false, false)).add(new KafkaColumnHandle("col2", VARCHAR, "col2", null, null, false, false, false)).add(new KafkaColumnHandle("col3", INTEGER, "col3", null, null, false, false, false)).add(new KafkaColumnHandle("col4", INTEGER, "col4", null, null, false, false, false)).add(new KafkaColumnHandle("col5", VARBINARY, "col5", null, null, false, false, false)).add(new KafkaColumnHandle("col6", BIGINT, "col6", null, null, false, false, false)).build();
    RowDecoder rowDecoder = getRowDecoder(mockSchemaRegistryClient, columnHandles, evolvedSchema);
    testRow(rowDecoder, generateRecord(initialSchema, Arrays.asList(3, "string-3", 30, 300, ByteBuffer.wrap(new byte[] { 1, 2, 3 }))), 1);
    testRow(rowDecoder, generateRecord(initialSchema, Arrays.asList(3, "", 30, null, null)), 1);
    testRow(rowDecoder, generateRecord(initialSchema, Arrays.asList(3, "\u0394\u66f4\u6539", 30, null, ByteBuffer.wrap(new byte[] { 1, 2, 3 }))), 1);
    testRow(rowDecoder, generateRecord(evolvedSchema, Arrays.asList(4, "string-4", 40, 400, null, 4L)), 2);
    testRow(rowDecoder, generateRecord(evolvedSchema, Arrays.asList(5, "string-5", 50, 500, ByteBuffer.wrap(new byte[] { 1, 2, 3 }), null)), 2);
}
Also used : MockSchemaRegistryClient(io.confluent.kafka.schemaregistry.client.MockSchemaRegistryClient) Schema(org.apache.avro.Schema) KafkaColumnHandle(io.trino.plugin.kafka.KafkaColumnHandle) DecoderColumnHandle(io.trino.decoder.DecoderColumnHandle) RowDecoder(io.trino.decoder.RowDecoder) Test(org.testng.annotations.Test)

Aggregations

DecoderColumnHandle (io.trino.decoder.DecoderColumnHandle)66 FieldValueProvider (io.trino.decoder.FieldValueProvider)63 DecoderTestColumnHandle (io.trino.decoder.DecoderTestColumnHandle)61 Test (org.testng.annotations.Test)61 Schema (org.apache.avro.Schema)24 RowDecoder (io.trino.decoder.RowDecoder)17 ArrayType (io.trino.spi.type.ArrayType)15 ImmutableList (com.google.common.collect.ImmutableList)13 ArrayList (java.util.ArrayList)13 List (java.util.List)12 GenericArray (org.apache.avro.generic.GenericArray)11 Map (java.util.Map)9 HashMap (java.util.HashMap)8 ImmutableMap (com.google.common.collect.ImmutableMap)6 ByteBuffer (java.nio.ByteBuffer)4 GenericData (org.apache.avro.generic.GenericData)4 Collections.emptyMap (java.util.Collections.emptyMap)3 GenericRecord (org.apache.avro.generic.GenericRecord)3 GenericRecordBuilder (org.apache.avro.generic.GenericRecordBuilder)3 MockSchemaRegistryClient (io.confluent.kafka.schemaregistry.client.MockSchemaRegistryClient)2