Search in sources :

Example 1 with SchemaSerializationException

use of org.apache.pulsar.client.api.SchemaSerializationException in project pulsar by apache.

the class ProtobufNativeSchemaUtils method serialize.

public static byte[] serialize(Descriptors.Descriptor descriptor) {
    byte[] schemaDataBytes;
    try {
        Map<String, FileDescriptorProto> fileDescriptorProtoCache = new HashMap<>();
        // recursively cache all FileDescriptorProto
        serializeFileDescriptor(descriptor.getFile(), fileDescriptorProtoCache);
        // extract root message path
        String rootMessageTypeName = descriptor.getFullName();
        String rootFileDescriptorName = descriptor.getFile().getFullName();
        // build FileDescriptorSet, this is equal to < protoc --include_imports --descriptor_set_out >
        byte[] fileDescriptorSet = FileDescriptorSet.newBuilder().addAllFile(fileDescriptorProtoCache.values()).build().toByteArray();
        // serialize to bytes
        ProtobufNativeSchemaData schemaData = ProtobufNativeSchemaData.builder().fileDescriptorSet(fileDescriptorSet).rootFileDescriptorName(rootFileDescriptorName).rootMessageTypeName(rootMessageTypeName).build();
        schemaDataBytes = new ObjectMapper().writeValueAsBytes(schemaData);
        logger.debug("descriptor '{}' serialized to '{}'.", descriptor.getFullName(), schemaDataBytes);
    } catch (Exception e) {
        e.printStackTrace();
        throw new SchemaSerializationException(e);
    }
    return schemaDataBytes;
}
Also used : HashMap(java.util.HashMap) ProtobufNativeSchemaData(org.apache.pulsar.common.protocol.schema.ProtobufNativeSchemaData) SchemaSerializationException(org.apache.pulsar.client.api.SchemaSerializationException) FileDescriptorProto(com.google.protobuf.DescriptorProtos.FileDescriptorProto) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) SchemaSerializationException(org.apache.pulsar.client.api.SchemaSerializationException)

Example 2 with SchemaSerializationException

use of org.apache.pulsar.client.api.SchemaSerializationException in project pulsar by apache.

the class GenericAvroWriter method write.

@Override
public synchronized byte[] write(GenericRecord message) {
    try {
        writer.write(((GenericAvroRecord) message).getAvroRecord(), this.encoder);
        this.encoder.flush();
        return this.byteArrayOutputStream.toByteArray();
    } catch (Exception e) {
        throw new SchemaSerializationException(e);
    } finally {
        this.byteArrayOutputStream.reset();
    }
}
Also used : SchemaSerializationException(org.apache.pulsar.client.api.SchemaSerializationException) SchemaSerializationException(org.apache.pulsar.client.api.SchemaSerializationException)

Example 3 with SchemaSerializationException

use of org.apache.pulsar.client.api.SchemaSerializationException in project pulsar by apache.

the class AvroReader method read.

@Override
public T read(InputStream inputStream) {
    try {
        BinaryDecoder decoderFromCache = decoders.get();
        BinaryDecoder decoder = DecoderFactory.get().binaryDecoder(inputStream, decoderFromCache);
        if (decoderFromCache == null) {
            decoders.set(decoder);
        }
        return reader.read(null, DecoderFactory.get().binaryDecoder(inputStream, decoder));
    } catch (Exception e) {
        throw new SchemaSerializationException(e);
    } finally {
        try {
            inputStream.close();
        } catch (IOException e) {
            log.error("AvroReader close inputStream close error", e);
        }
    }
}
Also used : SchemaSerializationException(org.apache.pulsar.client.api.SchemaSerializationException) IOException(java.io.IOException) BinaryDecoder(org.apache.avro.io.BinaryDecoder) SchemaSerializationException(org.apache.pulsar.client.api.SchemaSerializationException) IOException(java.io.IOException)

Example 4 with SchemaSerializationException

use of org.apache.pulsar.client.api.SchemaSerializationException in project pulsar by apache.

the class AvroReader method read.

@Override
public T read(byte[] bytes, int offset, int length) {
    try {
        BinaryDecoder decoderFromCache = decoders.get();
        BinaryDecoder decoder = DecoderFactory.get().binaryDecoder(bytes, offset, length, decoderFromCache);
        if (decoderFromCache == null) {
            decoders.set(decoder);
        }
        return reader.read(null, DecoderFactory.get().binaryDecoder(bytes, offset, length, decoder));
    } catch (IOException e) {
        throw new SchemaSerializationException(e);
    }
}
Also used : SchemaSerializationException(org.apache.pulsar.client.api.SchemaSerializationException) IOException(java.io.IOException) BinaryDecoder(org.apache.avro.io.BinaryDecoder)

Example 5 with SchemaSerializationException

use of org.apache.pulsar.client.api.SchemaSerializationException in project pulsar by apache.

the class SchemaCompatibilityCheckTest method testConsumerCompatibilityCheckCanReadLastTest.

@Test(dataProvider = "CanReadLastSchemaCompatibilityStrategy")
public void testConsumerCompatibilityCheckCanReadLastTest(SchemaCompatibilityStrategy schemaCompatibilityStrategy) throws Exception {
    final String tenant = PUBLIC_TENANT;
    final String topic = "test-consumer-compatibility";
    String namespace = "test-namespace-" + randomName(16);
    String fqtn = TopicName.get(TopicDomain.persistent.value(), tenant, namespace, topic).toString();
    NamespaceName namespaceName = NamespaceName.get(tenant, namespace);
    admin.namespaces().createNamespace(tenant + "/" + namespace, Sets.newHashSet(CLUSTER_NAME));
    admin.namespaces().setSchemaCompatibilityStrategy(namespaceName.toString(), schemaCompatibilityStrategy);
    admin.schemas().createSchema(fqtn, Schema.AVRO(Schemas.PersonOne.class).getSchemaInfo());
    admin.schemas().createSchema(fqtn, Schema.AVRO(SchemaDefinition.builder().withAlwaysAllowNull(false).withPojo(Schemas.PersonTwo.class).build()).getSchemaInfo());
    Consumer<Schemas.PersonThree> consumerThree = pulsarClient.newConsumer(Schema.AVRO(SchemaDefinition.<Schemas.PersonThree>builder().withAlwaysAllowNull(false).withSupportSchemaVersioning(true).withPojo(Schemas.PersonThree.class).build())).subscriptionName("test").topic(fqtn).subscribe();
    Producer<Schemas.PersonOne> producerOne = pulsarClient.newProducer(Schema.AVRO(Schemas.PersonOne.class)).topic(fqtn).create();
    Schemas.PersonOne personOne = new Schemas.PersonOne();
    personOne.setId(1);
    producerOne.send(personOne);
    Message<Schemas.PersonThree> message = null;
    try {
        message = consumerThree.receive();
        message.getValue();
    } catch (Exception e) {
        Assert.assertTrue(e instanceof SchemaSerializationException);
        consumerThree.acknowledge(message);
    }
    Producer<Schemas.PersonTwo> producerTwo = pulsarClient.newProducer(Schema.AVRO(SchemaDefinition.<Schemas.PersonTwo>builder().withAlwaysAllowNull(false).withSupportSchemaVersioning(true).withPojo(Schemas.PersonTwo.class).build())).topic(fqtn).create();
    Schemas.PersonTwo personTwo = new Schemas.PersonTwo();
    personTwo.setId(1);
    personTwo.setName("Jerry");
    producerTwo.send(personTwo);
    message = consumerThree.receive();
    Schemas.PersonThree personThree = message.getValue();
    consumerThree.acknowledge(message);
    assertEquals(personThree.getId(), 1);
    assertEquals(personThree.getName(), "Jerry");
    consumerThree.close();
    producerOne.close();
    producerTwo.close();
}
Also used : SchemaSerializationException(org.apache.pulsar.client.api.SchemaSerializationException) Schemas(org.apache.pulsar.schema.Schemas) SchemaSerializationException(org.apache.pulsar.client.api.SchemaSerializationException) NamespaceName(org.apache.pulsar.common.naming.NamespaceName) Test(org.testng.annotations.Test) MockedPulsarServiceBaseTest(org.apache.pulsar.broker.auth.MockedPulsarServiceBaseTest)

Aggregations

SchemaSerializationException (org.apache.pulsar.client.api.SchemaSerializationException)22 IOException (java.io.IOException)7 List (java.util.List)4 GenericRecord (org.apache.pulsar.client.api.schema.GenericRecord)4 SchemaInfo (org.apache.pulsar.common.schema.SchemaInfo)4 Test (org.testng.annotations.Test)4 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)3 FileDescriptorProto (com.google.protobuf.DescriptorProtos.FileDescriptorProto)3 Arrays (java.util.Arrays)3 HashMap (java.util.HashMap)3 Map (java.util.Map)3 SchemaValidationException (org.apache.avro.SchemaValidationException)3 Bar (org.apache.pulsar.client.impl.schema.SchemaTestUtils.Bar)3 Foo (org.apache.pulsar.client.impl.schema.SchemaTestUtils.Foo)3 NamespaceName (org.apache.pulsar.common.naming.NamespaceName)3 ProtobufNativeSchemaData (org.apache.pulsar.common.protocol.schema.ProtobufNativeSchemaData)3 Logger (org.slf4j.Logger)3 LoggerFactory (org.slf4j.LoggerFactory)3 JsonProcessingException (com.fasterxml.jackson.core.JsonProcessingException)2 FileDescriptorSet (com.google.protobuf.DescriptorProtos.FileDescriptorSet)2