Examples with TopicSchemaTypes com.bakdata.quick.common.api.model.TopicSchemaTypes used on opensource projects

Example 1 with TopicSchemaTypes

use of com.bakdata.quick.common.api.model.TopicSchemaTypes in project quick by bakdata.

the class InferTopicTypeService method getTopicData.

@Override
public <K, V> Single<QuickTopicData<K, V>> getTopicData(final String topic) {
    TopicSchemaTypes types = this.localCache.getIfPresent(topic);
    if (types == null) {
        final Optional<TopicSchemaTypes> optionalTypes = this.inferSerDes(topic);
        if (optionalTypes.isEmpty()) {
            return Single.error(new IllegalStateException("Topic is empty. Cannot infer types."));
        }
        types = optionalTypes.get();
        this.localCache.put(topic, types);
    }
    final QuickData<K> keyInfo = this.fromType(types.getKeyType(), topic, true);
    final QuickData<V> valueInfo = this.fromType(types.getValueType(), topic, false);
    return Single.just(new QuickTopicData<>(topic, TopicWriteType.MUTABLE, keyInfo, valueInfo));
}

Example 2 with TopicSchemaTypes

use of com.bakdata.quick.common.api.model.TopicSchemaTypes in project quick by bakdata.

the class InferTopicTypeService method inferSerDes.

private Optional<TopicSchemaTypes> inferSerDes(final String topic) {
    final InferConsumer inferConsumer = new InferConsumer(this.bootstrapServer, topic, this.appId);
    final ConsumerRecords<byte[], byte[]> records = inferConsumer.fetchRecords();
    if (records.isEmpty()) {
        return Optional.empty();
    }
    final List<byte[]> keys = new ArrayList<>();
    final List<byte[]> values = new ArrayList<>();
    for (final ConsumerRecord<byte[], byte[]> record : records) {
        keys.add(record.key());
        values.add(record.value());
    }
    final QuickTopicType keyType = inferSerDe(keys);
    final QuickTopicType valueType = inferSerDe(values);
    return Optional.of(new TopicSchemaTypes(keyType, valueType));
}