Examples with KafkaRelation org.apache.spark.sql.kafka010.KafkaRelation used on opensource projects

Example 1 with KafkaRelation

use of org.apache.spark.sql.kafka010.KafkaRelation in project OpenLineage by OpenLineage.

the class KafkaRelationVisitor method apply.

@Override
public List<D> apply(LogicalPlan x) {
    KafkaRelation relation = (KafkaRelation) ((LogicalRelation) x).relation();
    scala.collection.immutable.Map<String, String> sourceOptions;
    try {
        Field sourceOptionsField = relation.getClass().getDeclaredField("sourceOptions");
        sourceOptionsField.setAccessible(true);
        sourceOptions = (scala.collection.immutable.Map<String, String>) sourceOptionsField.get(relation);
    } catch (Exception e) {
        log.error("Can't extract kafka server options", e);
        sourceOptions = Map$.MODULE$.empty();
    }
    return createDatasetsFromOptions(datasetFactory, sourceOptions, relation.schema());
}