Search in sources :

Example 1 with PbRowDataSerializationSchema

use of org.apache.flink.formats.protobuf.serialize.PbRowDataSerializationSchema in project pulsar-flink by streamnative.

the class RowDataDerSerializationSchemaTest method testProtoBufSerializeDeserialize.

@Test
public void testProtoBufSerializeDeserialize() throws Exception {
    String topicName = newTopic();
    RowType rowType = PbRowTypeInformation.generateRowType(SimpleTest.getDescriptor());
    PbRowDataSerializationSchema serializationSchema = new PbRowDataSerializationSchema(rowType, SimpleTest.class.getName());
    PbRowDataDeserializationSchema deserializationSchema = new PbRowDataDeserializationSchema(rowType, InternalTypeInfo.of(rowType), SimpleTest.class.getName(), false, false);
    SimpleTest simple = SimpleTest.newBuilder().setA(1).setB(2L).setC(false).setD(0.1f).setE(0.01).setF("haha").setG(ByteString.copyFrom(new byte[] { 1 })).setH(SimpleTest.Corpus.IMAGES).setFAbc7D(// test fieldNameToJsonName
    1).build();
    RowData rowData = deserializationSchema.deserialize(simple.toByteArray());
    org.apache.pulsar.client.api.Schema<SimpleTest> schema = org.apache.pulsar.client.api.Schema.PROTOBUF_NATIVE(SimpleTest.class);
    final FlinkSchema<RowData> rowDataFlinkSchema = new FlinkSchema<>(schema.getSchemaInfo(), serializationSchema, deserializationSchema);
    sendMessage(topicName, rowDataFlinkSchema, rowData);
    final CompletableFuture<byte[]> consumer = autoConsumer(topicName);
    RowData newRowData = deserializationSchema.deserialize(consumer.get(10000, TimeUnit.MILLISECONDS));
    newRowData = validatePbRow(newRowData, PbRowTypeInformation.generateRowType(SimpleTest.getDescriptor()));
    assertEquals(9, newRowData.getArity());
    assertEquals(1, newRowData.getInt(0));
    assertEquals(2L, newRowData.getLong(1));
    assertFalse((boolean) newRowData.getBoolean(2));
    assertEquals(Float.valueOf(0.1f), Float.valueOf(newRowData.getFloat(3)));
    assertEquals(Double.valueOf(0.01d), Double.valueOf(newRowData.getDouble(4)));
    assertEquals("haha", newRowData.getString(5).toString());
    assertEquals(1, (newRowData.getBinary(6))[0]);
    assertEquals("IMAGES", newRowData.getString(7).toString());
    assertEquals(1, newRowData.getInt(8));
}
Also used : PbRowDataDeserializationSchema(org.apache.flink.formats.protobuf.deserialize.PbRowDataDeserializationSchema) RowType(org.apache.flink.table.types.logical.RowType) ByteString(com.google.protobuf.ByteString) PbRowDataSerializationSchema(org.apache.flink.formats.protobuf.serialize.PbRowDataSerializationSchema) RowData(org.apache.flink.table.data.RowData) SimpleTest(org.apache.flink.formats.protobuf.testproto.SimpleTest) FlinkSchema(org.apache.flink.streaming.connectors.pulsar.serialization.FlinkSchema) SimpleTest(org.apache.flink.formats.protobuf.testproto.SimpleTest) Test(org.junit.Test)

Aggregations

ByteString (com.google.protobuf.ByteString)1 PbRowDataDeserializationSchema (org.apache.flink.formats.protobuf.deserialize.PbRowDataDeserializationSchema)1 PbRowDataSerializationSchema (org.apache.flink.formats.protobuf.serialize.PbRowDataSerializationSchema)1 SimpleTest (org.apache.flink.formats.protobuf.testproto.SimpleTest)1 FlinkSchema (org.apache.flink.streaming.connectors.pulsar.serialization.FlinkSchema)1 RowData (org.apache.flink.table.data.RowData)1 RowType (org.apache.flink.table.types.logical.RowType)1 Test (org.junit.Test)1