Examples with StreamKpiData org.zalando.nakadi.metrics.StreamKpiData used on opensource projects

Example 1 with StreamKpiData

use of org.zalando.nakadi.metrics.StreamKpiData in project nakadi by zalando.

the class StreamingState method onEnter.

@Override
public void onEnter() {
    final String kafkaFlushedBytesMetricName = MetricUtils.metricNameForHiLAStream(this.getContext().getParameters().getConsumingClient().getClientId(), this.getContext().getSubscription().getId());
    bytesSentMeterPerSubscription = this.getContext().getMetricRegistry().meter(kafkaFlushedBytesMetricName);
    lastKpiEventSent = System.currentTimeMillis();
    kpiDataPerEventType = this.getContext().getSubscription().getEventTypes().stream().collect(Collectors.toMap(et -> et, et -> new StreamKpiData()));
    idleStreamWatcher = new IdleStreamWatcher(getParameters().commitTimeoutMillis * 2);
    this.eventConsumer = getContext().getTimelineService().createEventConsumer(null);
    recreateTopologySubscription();
    addTask(this::recheckTopology);
    addTask(this::pollDataFromKafka);
    scheduleTask(this::checkBatchTimeouts, getParameters().batchTimeoutMillis, TimeUnit.MILLISECONDS);
    scheduleTask(() -> {
        final String debugMessage = "Stream timeout reached";
        this.sendMetadata(debugMessage);
        this.shutdownGracefully(debugMessage);
    }, getParameters().streamTimeoutMillis, TimeUnit.MILLISECONDS);
    this.lastCommitMillis = System.currentTimeMillis();
    scheduleTask(this::checkCommitTimeout, getParameters().commitTimeoutMillis, TimeUnit.MILLISECONDS);
    cursorResetSubscription = getZk().subscribeForCursorsReset(() -> addTask(this::resetSubscriptionCursorsCallback));
}

Example 2 with StreamKpiData

use of org.zalando.nakadi.metrics.StreamKpiData in project nakadi by zalando.

the class StreamingState method flushData.

private void flushData(final EventTypePartition pk, final List<ConsumedEvent> data, final Optional<String> metadata) {
    try {
        final NakadiCursor sentOffset = offsets.get(pk).getSentOffset();
        final SubscriptionCursor cursor = getContext().getCursorConverter().convert(sentOffset, getContext().getCursorTokenService().generateToken());
        final int batchSize = getContext().getWriter().writeSubscriptionBatch(getOut().getOutputStream(), cursor, data, metadata);
        bytesSentMeterPerSubscription.mark(batchSize);
        final StreamKpiData kpiData = kpiDataPerEventType.get(pk.getEventType());
        kpiData.addBytesSent(batchSize);
        kpiData.addNumberOfEventsSent(data.size());
        batchesSent++;
    } catch (final IOException e) {
        getLog().error("Failed to write data to output.", e);
        shutdownGracefully("Failed to write data to output");
    }
}