Search in sources :

Example 56 with NakadiCursor

use of org.zalando.nakadi.domain.NakadiCursor in project nakadi by zalando.

the class MultiTimelineEventConsumer method selectCorrectTopicRepo.

private TopicRepository selectCorrectTopicRepo(final NakadiCursor cursor, final Consumer<NakadiCursor> cursorReplacer, final Consumer<NakadiCursor> lastTimelinePosition) throws ServiceUnavailableException {
    final List<Timeline> eventTimelines = eventTypeTimelines.get(cursor.getEventType());
    final ListIterator<Timeline> itTimeline = eventTimelines.listIterator(eventTimelines.size());
    // select last timeline, and then move back until position was found.
    Timeline electedTimeline = itTimeline.previous();
    while (itTimeline.hasPrevious()) {
        final Timeline toCheck = itTimeline.previous();
        final NakadiCursor latest = toCheck.calculateNakadiLatestPosition(cursor.getPartition());
        if (latest == null) {
            electedTimeline = toCheck;
        } else if (comparator.compare(latest, cursor) > 0) {
            // There is a border case - latest is equal to begin (that means that there are no available events
            // there), and one should position on timeline that have something inside.
            final NakadiCursor firstItem = timelineService.getTopicRepository(toCheck).loadPartitionStatistics(toCheck, cursor.getPartition()).get().getFirst();
            if (comparator.compare(latest, firstItem) >= 0) {
                electedTimeline = toCheck;
            } else {
                LOG.info("Timeline {} is empty, skipping", toCheck);
            }
        } else {
            break;
        }
    }
    final TopicRepository result = timelineService.getTopicRepository(electedTimeline);
    if (electedTimeline.getOrder() != cursor.getTimeline().getOrder()) {
        // It seems that cursor jumped to different timeline. One need to fetch very first cursor in timeline.
        final NakadiCursor replacement = getBeforeFirstCursor(result, electedTimeline, cursor.getPartition());
        LOG.info("Replacing cursor because of jumping between timelines from {} to {}", cursor, replacement);
        cursorReplacer.accept(replacement);
    }
    lastTimelinePosition.accept(electedTimeline.calculateNakadiLatestPosition(cursor.getPartition()));
    return result;
}
Also used : Timeline(org.zalando.nakadi.domain.Timeline) NakadiCursor(org.zalando.nakadi.domain.NakadiCursor)

Aggregations

NakadiCursor (org.zalando.nakadi.domain.NakadiCursor)56 Timeline (org.zalando.nakadi.domain.Timeline)31 Test (org.junit.Test)27 ShiftedNakadiCursor (org.zalando.nakadi.domain.ShiftedNakadiCursor)21 InvalidCursorException (org.zalando.nakadi.exceptions.InvalidCursorException)14 ServiceUnavailableException (org.zalando.nakadi.exceptions.ServiceUnavailableException)12 SubscriptionCursorWithoutToken (org.zalando.nakadi.view.SubscriptionCursorWithoutToken)12 PartitionStatistics (org.zalando.nakadi.domain.PartitionStatistics)11 List (java.util.List)10 Map (java.util.Map)10 Collectors (java.util.stream.Collectors)10 Storage (org.zalando.nakadi.domain.Storage)10 TopicRepository (org.zalando.nakadi.repository.TopicRepository)10 Optional (java.util.Optional)8 LoggerFactory (org.slf4j.LoggerFactory)8 EventTypePartition (org.zalando.nakadi.domain.EventTypePartition)8 TimelineService (org.zalando.nakadi.service.timeline.TimelineService)8 Collections (java.util.Collections)7 Logger (org.slf4j.Logger)7 InternalNakadiException (org.zalando.nakadi.exceptions.InternalNakadiException)7