Search in sources :

Example 26 with SchemaRegistryClient

use of io.confluent.kafka.schemaregistry.client.SchemaRegistryClient in project incubator-gobblin by apache.

the class ConfluentKafkaSchemaRegistryTest method testRegisterAndGetByKey.

@Test
public void testRegisterAndGetByKey() throws SchemaRegistryException {
    Properties properties = new Properties();
    properties.setProperty(KafkaSchemaRegistry.KAFKA_SCHEMA_REGISTRY_URL, TEST_URL);
    SchemaRegistryClient schemaRegistryClient = new MockSchemaRegistryClient();
    KafkaSchemaRegistry<Integer, Schema> kafkaSchemaRegistry = new ConfluentKafkaSchemaRegistry(properties, schemaRegistryClient);
    Schema schema = SchemaBuilder.record(TEST_RECORD_NAME).namespace(TEST_NAMESPACE).fields().name(TEST_FIELD_NAME).type().stringType().noDefault().endRecord();
    Integer id = kafkaSchemaRegistry.register(schema);
    Assert.assertEquals(schema, kafkaSchemaRegistry.getSchemaByKey(id));
}
Also used : MockSchemaRegistryClient(io.confluent.kafka.schemaregistry.client.MockSchemaRegistryClient) Schema(org.apache.avro.Schema) Properties(java.util.Properties) MockSchemaRegistryClient(io.confluent.kafka.schemaregistry.client.MockSchemaRegistryClient) SchemaRegistryClient(io.confluent.kafka.schemaregistry.client.SchemaRegistryClient) Test(org.testng.annotations.Test)

Example 27 with SchemaRegistryClient

use of io.confluent.kafka.schemaregistry.client.SchemaRegistryClient in project incubator-gobblin by apache.

the class KafkaDeserializerExtractorTest method testConfluentAvroDeserializer.

@Test
public void testConfluentAvroDeserializer() throws IOException, RestClientException {
    WorkUnitState mockWorkUnitState = getMockWorkUnitState();
    mockWorkUnitState.setProp("schema.registry.url", TEST_URL);
    Schema schema = SchemaBuilder.record(TEST_RECORD_NAME).namespace(TEST_NAMESPACE).fields().name(TEST_FIELD_NAME).type().stringType().noDefault().endRecord();
    GenericRecord testGenericRecord = new GenericRecordBuilder(schema).set(TEST_FIELD_NAME, "testValue").build();
    SchemaRegistryClient mockSchemaRegistryClient = mock(SchemaRegistryClient.class);
    when(mockSchemaRegistryClient.getByID(any(Integer.class))).thenReturn(schema);
    Serializer<Object> kafkaEncoder = new KafkaAvroSerializer(mockSchemaRegistryClient);
    Deserializer<Object> kafkaDecoder = new KafkaAvroDeserializer(mockSchemaRegistryClient);
    ByteBuffer testGenericRecordByteBuffer = ByteBuffer.wrap(kafkaEncoder.serialize(TEST_TOPIC_NAME, testGenericRecord));
    KafkaSchemaRegistry<Integer, Schema> mockKafkaSchemaRegistry = mock(KafkaSchemaRegistry.class);
    KafkaDeserializerExtractor kafkaDecoderExtractor = new KafkaDeserializerExtractor(mockWorkUnitState, Optional.fromNullable(Deserializers.CONFLUENT_AVRO), kafkaDecoder, mockKafkaSchemaRegistry);
    ByteArrayBasedKafkaRecord mockMessageAndOffset = getMockMessageAndOffset(testGenericRecordByteBuffer);
    Assert.assertEquals(kafkaDecoderExtractor.decodeRecord(mockMessageAndOffset), testGenericRecord);
}
Also used : WorkUnitState(org.apache.gobblin.configuration.WorkUnitState) Schema(org.apache.avro.Schema) KafkaAvroSerializer(io.confluent.kafka.serializers.KafkaAvroSerializer) KafkaAvroDeserializer(io.confluent.kafka.serializers.KafkaAvroDeserializer) ByteBuffer(java.nio.ByteBuffer) ByteArrayBasedKafkaRecord(org.apache.gobblin.kafka.client.ByteArrayBasedKafkaRecord) GenericRecordBuilder(org.apache.avro.generic.GenericRecordBuilder) GenericRecord(org.apache.avro.generic.GenericRecord) SchemaRegistryClient(io.confluent.kafka.schemaregistry.client.SchemaRegistryClient) Test(org.testng.annotations.Test)

Example 28 with SchemaRegistryClient

use of io.confluent.kafka.schemaregistry.client.SchemaRegistryClient in project beam by apache.

the class ConfluentSchemaRegistryDeserializerProviderTest method mockSchemaRegistryClient.

private static SchemaRegistryClient mockSchemaRegistryClient(String schemaRegistryUrl, String subject) {
    SchemaRegistryClient mockRegistryClient = MockSchemaRegistry.getClientForScope(schemaRegistryUrl);
    try {
        mockRegistryClient.register(subject, AVRO_SCHEMA_V1);
        mockRegistryClient.register(subject, AVRO_SCHEMA);
    } catch (IOException | RestClientException e) {
        throw new RuntimeException("Unable to register schema for subject: " + subject, e);
    }
    return mockRegistryClient;
}
Also used : RestClientException(io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException) IOException(java.io.IOException) SchemaRegistryClient(io.confluent.kafka.schemaregistry.client.SchemaRegistryClient)

Aggregations

SchemaRegistryClient (io.confluent.kafka.schemaregistry.client.SchemaRegistryClient)28 Test (org.junit.Test)21 MockSchemaRegistryClient (io.confluent.kafka.schemaregistry.client.MockSchemaRegistryClient)11 KsqlConfig (io.confluent.ksql.util.KsqlConfig)10 GenericRow (io.confluent.ksql.GenericRow)8 List (java.util.List)8 HashMap (java.util.HashMap)7 Schema (org.apache.avro.Schema)6 GenericRecord (org.apache.avro.generic.GenericRecord)5 RestClientException (io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException)4 KafkaAvroDeserializer (io.confluent.kafka.serializers.KafkaAvroDeserializer)4 KafkaAvroSerializer (io.confluent.kafka.serializers.KafkaAvroSerializer)4 IOException (java.io.IOException)4 ConsumerRecord (org.apache.kafka.clients.consumer.ConsumerRecord)4 Bytes (org.apache.kafka.common.utils.Bytes)4 Schema (org.apache.kafka.connect.data.Schema)4 Map (java.util.Map)3 GenericData (org.apache.avro.generic.GenericData)3 Test (org.testng.annotations.Test)3 SchemaMetadata (io.confluent.kafka.schemaregistry.client.SchemaMetadata)2