Search in sources :

Example 11 with EventData

use of com.microsoft.azure.eventhubs.EventData in project zipkin-azure by openzipkin.

the class ZipkinEventProcessor method onEvents.

@Override
public void onEvents(PartitionContext context, Iterable<EventData> messages) throws ExecutionException, InterruptedException {
    // don't issue a write until we checkpoint or exit this callback
    List<Span> buffer = new ArrayList<>();
    for (EventData data : messages) {
        byte[] bytes = data.getBytes();
        List<Span> nextSpans = SpanDecoder.DETECTING_DECODER.readSpans(bytes);
        buffer.addAll(nextSpans);
        if (maybeCheckpoint(context, data, nextSpans.size())) {
            collector.accept(buffer, NOOP);
            buffer.clear();
        }
    }
    if (!buffer.isEmpty()) {
        collector.accept(buffer, NOOP);
    }
}
Also used : ArrayList(java.util.ArrayList) Span(zipkin.Span) EventData(com.microsoft.azure.eventhubs.EventData)

Example 12 with EventData

use of com.microsoft.azure.eventhubs.EventData in project zipkin-azure by openzipkin.

the class ZipkinEventProcessorTest method message.

static EventData message(String offset, long sequenceNumber, byte[] message) throws IllegalAccessException, NoSuchFieldException {
    EventData data = new EventData(message);
    LinkedHashMap<String, Object> sysProps = new LinkedHashMap<>();
    sysProps.put("x-opt-offset", offset);
    sysProps.put("x-opt-sequence-number", sequenceNumber);
    addSystemProperties(data, sysProps);
    return data;
}
Also used : EventData(com.microsoft.azure.eventhubs.EventData) LinkedHashMap(java.util.LinkedHashMap)

Example 13 with EventData

use of com.microsoft.azure.eventhubs.EventData in project zipkin-azure by openzipkin.

the class ZipkinEventProcessorTest method checkpointsOnBatchSize.

@Test
public void checkpointsOnBatchSize() throws Exception {
    EventData event1 = jsonMessageWithThreeSpans("a", 1);
    EventData event2 = json2MessageWithThreeSpans("b", 2);
    EventData event3 = thriftMessageWithThreeSpans("c", 3);
    EventData event4 = json2MessageWithThreeSpans("d", 4);
    // We don't expect a checkpoint, yet
    processor.onEvents(context, asList(event1, event2, event3));
    assertThat(checkpointEvents).isEmpty();
    assertThat(logger.messages.poll()).isNull();
    // We expect a checkpoint as we completed a batch
    processor.onEvents(context, asList(event4));
    assertThat(checkpointEvents).containsExactly(event4);
    assertThat(logger.messages.poll()).isEqualTo("FINE: Partition 1 checkpointing at d,4");
}
Also used : EventData(com.microsoft.azure.eventhubs.EventData) Test(org.junit.Test)

Example 14 with EventData

use of com.microsoft.azure.eventhubs.EventData in project zipkin-azure by openzipkin.

the class ZipkinEventProcessorTest method onEvents_singleDatum.

void onEvents_singleDatum(Codec codec) throws ExecutionException, InterruptedException {
    List<EventData> messages = asList(new EventData(codec.writeSpans(TestObjects.TRACE)));
    processor.onEvents(context, messages);
    assertThat(storage.spanStore().getRawTraces()).hasSize(1);
    assertThat(storage.spanStore().getRawTraces().get(0)).isEqualTo(TestObjects.TRACE);
}
Also used : EventData(com.microsoft.azure.eventhubs.EventData)

Example 15 with EventData

use of com.microsoft.azure.eventhubs.EventData in project zipkin-azure by openzipkin.

the class ZipkinEventProcessorTest method addSystemProperties.

static void addSystemProperties(EventData data, LinkedHashMap<String, Object> sysProps) throws IllegalAccessException, NoSuchFieldException {
    Field field = EventData.class.getDeclaredField("systemProperties");
    field.setAccessible(true);
    field.set(data, new EventData.SystemProperties(sysProps));
}
Also used : Field(java.lang.reflect.Field) EventData(com.microsoft.azure.eventhubs.EventData)

Aggregations

EventData (com.microsoft.azure.eventhubs.EventData)15 Test (org.junit.Test)10 ProvenanceEventRecord (org.apache.nifi.provenance.ProvenanceEventRecord)8 MockFlowFile (org.apache.nifi.util.MockFlowFile)8 IOException (java.io.IOException)2 ArrayList (java.util.ArrayList)2 Collections (java.util.Collections)2 HashMap (java.util.HashMap)2 List (java.util.List)2 Map (java.util.Map)2 ExecutionException (java.util.concurrent.ExecutionException)2 EventHubClient (com.microsoft.azure.eventhubs.EventHubClient)1 PartitionReceiver (com.microsoft.azure.eventhubs.PartitionReceiver)1 PartitionContext (com.microsoft.azure.eventprocessorhost.PartitionContext)1 ConnectionStringBuilder (com.microsoft.azure.servicebus.ConnectionStringBuilder)1 ServiceBusException (com.microsoft.azure.servicebus.ServiceBusException)1 OutputStream (java.io.OutputStream)1 Field (java.lang.reflect.Field)1 URI (java.net.URI)1 URISyntaxException (java.net.URISyntaxException)1