Search in sources :

Example 41 with RawSpan

use of org.hypertrace.core.datamodel.RawSpan in project hypertrace-ingester by hypertrace.

the class ByPassPredicateTest method testByPassPredicateWhenKeyIsNotDefined.

@Test
void testByPassPredicateWhenKeyIsNotDefined() {
    Config jobConfig = ConfigFactory.parseMap(Map.of());
    ByPassPredicate byPassPredicateUnderTest = new ByPassPredicate(jobConfig);
    TraceIdentity traceIdentity = TraceIdentity.newBuilder().setTenantId(CUSTOMER_ID).setTraceId(ByteBuffer.wrap("trace-2".getBytes())).build();
    ByteBuffer span1 = ByteBuffer.wrap("span-1".getBytes());
    ByteBuffer span2 = ByteBuffer.wrap("span-2".getBytes());
    // tags won't matter as bypass.key is not defined
    RawSpan rawSpan = RawSpan.newBuilder().setCustomerId(CUSTOMER_ID).setTraceId(traceIdentity.getTraceId()).setEvent(Event.newBuilder().setCustomerId(CUSTOMER_ID).setEventId(span1).setAttributes(Attributes.newBuilder().setAttributeMap(Map.of("test.bypass", AttributeValue.newBuilder().setValue("true").build())).build()).build()).build();
    Assertions.assertFalse(byPassPredicateUnderTest.test(traceIdentity, rawSpan));
    // tags won't matter as bypass.key is not defined
    rawSpan = RawSpan.newBuilder().setCustomerId(CUSTOMER_ID).setTraceId(traceIdentity.getTraceId()).setEvent(Event.newBuilder().setCustomerId(CUSTOMER_ID).setEventId(span2).setAttributes(Attributes.newBuilder().setAttributeMap(Map.of("test.bypass", AttributeValue.newBuilder().setValue("false").build())).build()).build()).build();
    Assertions.assertFalse(byPassPredicateUnderTest.test(traceIdentity, rawSpan));
}
Also used : Config(com.typesafe.config.Config) TraceIdentity(org.hypertrace.core.spannormalizer.TraceIdentity) RawSpan(org.hypertrace.core.datamodel.RawSpan) ByteBuffer(java.nio.ByteBuffer) Test(org.junit.jupiter.api.Test)

Example 42 with RawSpan

use of org.hypertrace.core.datamodel.RawSpan in project hypertrace-ingester by hypertrace.

the class JaegerSpanToAvroRawSpanTransformer method transform.

@Override
public KeyValue<TraceIdentity, RawSpan> transform(byte[] key, PreProcessedSpan preProcessedSpan) {
    Span value = preProcessedSpan.getSpan();
    String tenantId = preProcessedSpan.getTenantId();
    try {
        RawSpan rawSpan = converter.convert(tenantId, value, preProcessedSpan.getEvent());
        if (null != rawSpan) {
            // these are spans per tenant that we were able to parse / convert, and had tenantId.
            tenantToSpanReceivedCount.computeIfAbsent(tenantId, tenant -> PlatformMetricsRegistry.registerCounter(VALID_SPAN_RECEIVED_COUNT, Map.of("tenantId", tenantId))).increment();
            // we use the (tenant_id, trace_id) as the key so that raw_span_grouper
            // job can do a groupByKey without having to create a repartition topic
            TraceIdentity traceIdentity = fastNewBuilder(TraceIdentity.Builder.class).setTenantId(tenantId).setTraceId(rawSpan.getTraceId()).build();
            return new KeyValue<>(traceIdentity, rawSpan);
        }
        statusToSpansCounter.computeIfAbsent("dropped", k -> PlatformMetricsRegistry.registerCounter(SPANS_COUNTER, Map.of("result", k))).increment();
        return null;
    } catch (Exception e) {
        LOGGER.error("Error converting spans - ", e);
        statusToSpansCounter.computeIfAbsent("error", k -> PlatformMetricsRegistry.registerCounter(SPANS_COUNTER, Map.of("result", k))).increment();
        return null;
    }
}
Also used : Counter(io.micrometer.core.instrument.Counter) RawSpan(org.hypertrace.core.datamodel.RawSpan) SPAN_NORMALIZER_JOB_CONFIG(org.hypertrace.core.spannormalizer.constants.SpanNormalizerConstants.SPAN_NORMALIZER_JOB_CONFIG) Logger(org.slf4j.Logger) Span(io.jaegertracing.api_v2.JaegerSpanInternalModel.Span) Config(com.typesafe.config.Config) SPANS_COUNTER(org.hypertrace.core.spannormalizer.jaeger.JaegerSpanPreProcessor.SPANS_COUNTER) Transformer(org.apache.kafka.streams.kstream.Transformer) AvroBuilderCache.fastNewBuilder(org.hypertrace.core.datamodel.shared.AvroBuilderCache.fastNewBuilder) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) KeyValue(org.apache.kafka.streams.KeyValue) LoggerFactory(org.slf4j.LoggerFactory) ConcurrentMap(java.util.concurrent.ConcurrentMap) PlatformMetricsRegistry(org.hypertrace.core.serviceframework.metrics.PlatformMetricsRegistry) ProcessorContext(org.apache.kafka.streams.processor.ProcessorContext) Map(java.util.Map) TraceIdentity(org.hypertrace.core.spannormalizer.TraceIdentity) KeyValue(org.apache.kafka.streams.KeyValue) AvroBuilderCache.fastNewBuilder(org.hypertrace.core.datamodel.shared.AvroBuilderCache.fastNewBuilder) TraceIdentity(org.hypertrace.core.spannormalizer.TraceIdentity) RawSpan(org.hypertrace.core.datamodel.RawSpan) RawSpan(org.hypertrace.core.datamodel.RawSpan) Span(io.jaegertracing.api_v2.JaegerSpanInternalModel.Span)

Example 43 with RawSpan

use of org.hypertrace.core.datamodel.RawSpan in project hypertrace-ingester by hypertrace.

the class JaegerSpanNormalizerTest method testServiceNameAddedToEvent.

@Test
public void testServiceNameAddedToEvent() throws Exception {
    String tenantId = "tenant-" + random.nextLong();
    Map<String, Object> configs = new HashMap<>(getCommonConfig());
    configs.putAll(Map.of("processor", Map.of("defaultTenantId", tenantId)));
    JaegerSpanNormalizer normalizer = JaegerSpanNormalizer.get(ConfigFactory.parseMap(configs));
    Process process = Process.newBuilder().setServiceName("testService").build();
    Span span = Span.newBuilder().setProcess(process).build();
    RawSpan rawSpan = normalizer.convert(tenantId, span, buildEvent(tenantId, span, Optional.empty()));
    Assertions.assertEquals("testService", rawSpan.getEvent().getServiceName());
    Assertions.assertEquals("testService", rawSpan.getEvent().getAttributes().getAttributeMap().get(RawSpanConstants.getValue(JaegerAttribute.JAEGER_ATTRIBUTE_SERVICE_NAME)).getValue());
    /**
     * the case when `jaegerSpan.getProcess().getServiceName()` is not populated but serviceName is
     * sent for key `jaeger.serviceName`
     */
    span = Span.newBuilder().addTags(KeyValue.newBuilder().setKey(JaegerSpanNormalizer.OLD_JAEGER_SERVICENAME_KEY).setVStr("testService")).addTags(KeyValue.newBuilder().setKey(RawSpanConstants.getValue(HTTP_REQUEST_METHOD)).setVStr("GET")).build();
    rawSpan = normalizer.convert("tenant-key", span, buildEvent(tenantId, span, Optional.empty()));
    Assertions.assertEquals("testService", rawSpan.getEvent().getServiceName());
    Assertions.assertEquals("testService", rawSpan.getEvent().getAttributes().getAttributeMap().get(RawSpanConstants.getValue(JaegerAttribute.JAEGER_ATTRIBUTE_SERVICE_NAME)).getValue());
    /**
     * the case when neither `jaegerSpan.getProcess().getServiceName()` nor `jaeger.serviceName` is
     * present in tag map
     */
    span = Span.newBuilder().addTags(KeyValue.newBuilder().setKey("someKey").setVStr("someValue")).build();
    rawSpan = normalizer.convert("tenant-key", span, buildEvent(tenantId, span, Optional.empty()));
    Assertions.assertNull(rawSpan.getEvent().getServiceName());
    Assertions.assertNotNull(rawSpan.getEvent().getAttributes().getAttributeMap());
    Assertions.assertNull(rawSpan.getEvent().getAttributes().getAttributeMap().get(RawSpanConstants.getValue(JaegerAttribute.JAEGER_ATTRIBUTE_SERVICE_NAME)));
}
Also used : HashMap(java.util.HashMap) Process(io.jaegertracing.api_v2.JaegerSpanInternalModel.Process) RawSpan(org.hypertrace.core.datamodel.RawSpan) Span(io.jaegertracing.api_v2.JaegerSpanInternalModel.Span) RawSpan(org.hypertrace.core.datamodel.RawSpan) Test(org.junit.jupiter.api.Test)

Aggregations

RawSpan (org.hypertrace.core.datamodel.RawSpan)43 Span (io.jaegertracing.api_v2.JaegerSpanInternalModel.Span)33 Test (org.junit.jupiter.api.Test)25 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)25 HashMap (java.util.HashMap)14 Config (com.typesafe.config.Config)12 TraceIdentity (org.hypertrace.core.spannormalizer.TraceIdentity)9 MethodSource (org.junit.jupiter.params.provider.MethodSource)9 StructuredTrace (org.hypertrace.core.datamodel.StructuredTrace)6 ByteBuffer (java.nio.ByteBuffer)5 Map (java.util.Map)5 Properties (java.util.Properties)5 StreamsBuilder (org.apache.kafka.streams.StreamsBuilder)5 TestOutputTopic (org.apache.kafka.streams.TestOutputTopic)5 TopologyTestDriver (org.apache.kafka.streams.TopologyTestDriver)5 JaegerSpanSerde (org.hypertrace.core.spannormalizer.jaeger.JaegerSpanSerde)5 SetEnvironmentVariable (org.junitpioneer.jupiter.SetEnvironmentVariable)5 ByteString (com.google.protobuf.ByteString)4 Counter (io.micrometer.core.instrument.Counter)4 Instant (java.time.Instant)4