Examples with BatchItem org.zalando.nakadi.domain.BatchItem used on opensource projects

Example 6 with BatchItem

use of org.zalando.nakadi.domain.BatchItem in project nakadi by zalando.

the class MetadataEnrichmentStrategyTest method setEventTypeSchemaVersion.

@Test
public void setEventTypeSchemaVersion() throws Exception {
    final EventType eventType = buildDefaultEventType();
    final JSONObject event = buildBusinessEvent();
    final BatchItem batchItem = createBatchItem(event);
    assertThat(batchItem.getEvent().getJSONObject("metadata").optString("version"), isEmptyString());
    strategy.enrich(batchItem, eventType);
    assertThat(batchItem.getEvent().getJSONObject("metadata").getString("version"), equalTo("1.0.0"));
}

Example 7 with BatchItem

use of org.zalando.nakadi.domain.BatchItem in project nakadi by zalando.

the class KafkaTopicRepository method publishItem.

private static CompletableFuture<Exception> publishItem(final Producer<String, String> producer, final String topicId, final BatchItem item, final HystrixKafkaCircuitBreaker circuitBreaker) throws EventPublishingException {
    try {
        final CompletableFuture<Exception> result = new CompletableFuture<>();
        final ProducerRecord<String, String> kafkaRecord = new ProducerRecord<>(topicId, KafkaCursor.toKafkaPartition(item.getPartition()), item.getPartition(), item.dumpEventToString());
        circuitBreaker.markStart();
        producer.send(kafkaRecord, ((metadata, exception) -> {
            if (null != exception) {
                LOG.warn("Failed to publish to kafka topic {}", topicId, exception);
                item.updateStatusAndDetail(EventPublishingStatus.FAILED, "internal error");
                if (hasKafkaConnectionException(exception)) {
                    circuitBreaker.markFailure();
                } else {
                    circuitBreaker.markSuccessfully();
                }
                result.complete(exception);
            } else {
                item.updateStatusAndDetail(EventPublishingStatus.SUBMITTED, "");
                circuitBreaker.markSuccessfully();
                result.complete(null);
            }
        }));
        return result;
    } catch (final InterruptException e) {
        Thread.currentThread().interrupt();
        circuitBreaker.markSuccessfully();
        item.updateStatusAndDetail(EventPublishingStatus.FAILED, "internal error");
        throw new EventPublishingException("Error publishing message to kafka", e);
    } catch (final RuntimeException e) {
        circuitBreaker.markSuccessfully();
        item.updateStatusAndDetail(EventPublishingStatus.FAILED, "internal error");
        throw new EventPublishingException("Error publishing message to kafka", e);
    }
}

Example 8 with BatchItem

use of org.zalando.nakadi.domain.BatchItem in project nakadi by zalando.

the class EventPublisher method validate.

private void validate(final List<BatchItem> batch, final EventType eventType) throws EventValidationException, InternalNakadiException, NoSuchEventTypeException {
    for (final BatchItem item : batch) {
        item.setStep(EventPublishingStep.VALIDATING);
        try {
            validateSchema(item.getEvent(), eventType);
            validateEventSize(item);
        } catch (final EventValidationException e) {
            item.updateStatusAndDetail(EventPublishingStatus.FAILED, e.getMessage());
            throw e;
        }
    }
}

Example 9 with BatchItem

use of org.zalando.nakadi.domain.BatchItem in project nakadi by zalando.

the class KafkaRepositoryAT method whenBulkSendSuccessfullyThenUpdateBatchItemStatus.

@Test(timeout = 10000)
public void whenBulkSendSuccessfullyThenUpdateBatchItemStatus() throws Exception {
    final List<BatchItem> items = new ArrayList<>();
    final String topicId = TestUtils.randomValidEventTypeName();
    kafkaHelper.createTopic(topicId, ZOOKEEPER_URL);
    for (int i = 0; i < 10; i++) {
        final BatchItem item = BatchFactory.from("[{}]").get(0);
        item.setPartition("0");
        items.add(item);
    }
    kafkaTopicRepository.syncPostBatch(topicId, items);
    for (int i = 0; i < 10; i++) {
        assertThat(items.get(i).getResponse().getPublishingStatus(), equalTo(EventPublishingStatus.SUBMITTED));
    }
}

Example 10 with BatchItem

use of org.zalando.nakadi.domain.BatchItem in project nakadi by zalando.

the class MetadataEnrichmentStrategyTest method setReceivedAtWithSystemTimeInUTC.

@Test
public void setReceivedAtWithSystemTimeInUTC() throws Exception {
    final EventType eventType = buildDefaultEventType();
    final JSONObject event = buildBusinessEvent();
    final BatchItem batch = TestUtils.createBatchItem(event);
    assertThat(event.getJSONObject("metadata").optString("received_at"), isEmptyString());
    try {
        DateTimeUtils.setCurrentMillisFixed(0);
        strategy.enrich(batch, eventType);
    } finally {
        DateTimeUtils.setCurrentMillisSystem();
    }
    assertThat(batch.getEvent().getJSONObject("metadata").getString("received_at"), equalTo("1970-01-01T00:00:00.000Z"));
}