Search in sources :

Example 1 with PreProcessedSpan

use of org.hypertrace.core.spannormalizer.jaeger.PreProcessedSpan in project hypertrace-ingester by hypertrace.

the class SpanNormalizer method buildTopology.

@Override
public StreamsBuilder buildTopology(Map<String, Object> streamsProperties, StreamsBuilder streamsBuilder, Map<String, KStream<?, ?>> inputStreams) {
    Config jobConfig = getJobConfig(streamsProperties);
    String inputTopic = jobConfig.getString(INPUT_TOPIC_CONFIG_KEY);
    String outputTopic = jobConfig.getString(OUTPUT_TOPIC_CONFIG_KEY);
    String bypassOutputTopic = jobConfig.getString(BYPASS_OUTPUT_TOPIC_CONFIG_KEY);
    String outputTopicRawLogs = jobConfig.getString(OUTPUT_TOPIC_RAW_LOGS_CONFIG_KEY);
    KStream<byte[], Span> inputStream = (KStream<byte[], Span>) inputStreams.get(inputTopic);
    if (inputStream == null) {
        inputStream = streamsBuilder.stream(inputTopic, Consumed.with(Serdes.ByteArray(), new JaegerSpanSerde()));
        inputStreams.put(inputTopic, inputStream);
    }
    KStream<byte[], PreProcessedSpan> preProcessedStream = inputStream.transform(JaegerSpanPreProcessor::new);
    KStream<TraceIdentity, RawSpan>[] branches = preProcessedStream.transform(JaegerSpanToAvroRawSpanTransformer::new).branch(new ByPassPredicate(jobConfig), (key, value) -> true);
    branches[0].transform(RawSpanToStructuredTraceTransformer::new).to(bypassOutputTopic);
    branches[1].to(outputTopic);
    preProcessedStream.transform(JaegerSpanToLogRecordsTransformer::new).to(outputTopicRawLogs);
    return streamsBuilder;
}
Also used : JaegerSpanPreProcessor(org.hypertrace.core.spannormalizer.jaeger.JaegerSpanPreProcessor) ByPassPredicate(org.hypertrace.core.spannormalizer.rawspan.ByPassPredicate) Config(com.typesafe.config.Config) KStream(org.apache.kafka.streams.kstream.KStream) JaegerSpanSerde(org.hypertrace.core.spannormalizer.jaeger.JaegerSpanSerde) PreProcessedSpan(org.hypertrace.core.spannormalizer.jaeger.PreProcessedSpan) Span(io.jaegertracing.api_v2.JaegerSpanInternalModel.Span) PreProcessedSpan(org.hypertrace.core.spannormalizer.jaeger.PreProcessedSpan) RawSpan(org.hypertrace.core.datamodel.RawSpan)

Aggregations

Config (com.typesafe.config.Config)1 Span (io.jaegertracing.api_v2.JaegerSpanInternalModel.Span)1 KStream (org.apache.kafka.streams.kstream.KStream)1 RawSpan (org.hypertrace.core.datamodel.RawSpan)1 JaegerSpanPreProcessor (org.hypertrace.core.spannormalizer.jaeger.JaegerSpanPreProcessor)1 JaegerSpanSerde (org.hypertrace.core.spannormalizer.jaeger.JaegerSpanSerde)1 PreProcessedSpan (org.hypertrace.core.spannormalizer.jaeger.PreProcessedSpan)1 ByPassPredicate (org.hypertrace.core.spannormalizer.rawspan.ByPassPredicate)1