Examples with EventTypeStatistics org.zalando.nakadi.domain.EventTypeStatistics used on opensource projects

Example 1 with EventTypeStatistics

use of org.zalando.nakadi.domain.EventTypeStatistics in project nakadi by zalando.

the class EventStreamReadingAT method setupClass.

@BeforeClass
public static void setupClass() throws JsonProcessingException {
    eventType = EventTypeTestBuilder.builder().defaultStatistic(new EventTypeStatistics(PARTITIONS_NUM, PARTITIONS_NUM)).build();
    NakadiTestUtils.createEventTypeInNakadi(eventType);
    streamEndpoint = createStreamEndpointUrl(eventType.getName());
    // expect only one timeline, because we just created event type
    topicName = BaseAT.TIMELINE_REPOSITORY.listTimelinesOrdered(eventType.getName()).get(0).getTopic();
}

Example 2 with EventTypeStatistics

use of org.zalando.nakadi.domain.EventTypeStatistics in project nakadi by zalando.

the class EventStreamReadingAT method whenMemoryOverflowEventsDumped.

@Test(timeout = 10000)
public void whenMemoryOverflowEventsDumped() throws IOException {
    // Create event type
    final EventType loadEt = EventTypeTestBuilder.builder().defaultStatistic(new EventTypeStatistics(PARTITIONS_NUM, PARTITIONS_NUM)).build();
    NakadiTestUtils.createEventTypeInNakadi(loadEt);
    // Publish events to event type, that are not fitting memory
    final String evt = "{\"foo\":\"barbarbar\"}";
    final int eventCount = 2 * (10000 / evt.length());
    NakadiTestUtils.publishEvents(loadEt.getName(), eventCount, i -> evt);
    // Configure streaming so it will:(more than 10s and batch_limit
    // - definitely wait for more than test timeout (10s)
    // - collect batch, which size is greater than events published to this event type
    final String url = RestAssured.baseURI + ":" + RestAssured.port + createStreamEndpointUrl(loadEt.getName()) + "?batch_limit=" + (10 * eventCount) + "&stream_limit=" + (10 * eventCount) + "&batch_flush_timeout=11" + "&stream_timeout=11";
    final HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection();
    // Start from the begin.
    connection.setRequestProperty("X-Nakadi-Cursors", "[" + IntStream.range(0, PARTITIONS_NUM).mapToObj(i -> "{\"partition\": \"" + i + "\",\"offset\":\"begin\"}").collect(Collectors.joining(",")) + "]");
    Assert.assertEquals(HttpServletResponse.SC_OK, connection.getResponseCode());
    final InputStream inputStream = connection.getInputStream();
    final BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
    final String line = reader.readLine();
    Assert.assertNotNull(line);
// If we read at least one line, than it means, that we were able to read data before test timeout reached.
}

Example 3 with EventTypeStatistics

use of org.zalando.nakadi.domain.EventTypeStatistics in project nakadi by zalando.

the class NakadiTestUtils method createBusinessEventTypeWithPartitions.

public static EventType createBusinessEventTypeWithPartitions(final int partitionNum) throws JsonProcessingException {
    final EventTypeStatistics statistics = new EventTypeStatistics();
    statistics.setMessageSize(1);
    statistics.setMessagesPerMinute(1);
    statistics.setReadParallelism(partitionNum);
    statistics.setWriteParallelism(1);
    final EventType eventType = buildSimpleEventType();
    eventType.setCategory(EventCategory.BUSINESS);
    eventType.setEnrichmentStrategies(ImmutableList.of(EnrichmentStrategyDescriptor.METADATA_ENRICHMENT));
    eventType.setPartitionStrategy(PartitionStrategy.USER_DEFINED_STRATEGY);
    eventType.setDefaultStatistic(statistics);
    createEventTypeInNakadi(eventType);
    return eventType;
}

Example 4 with EventTypeStatistics

use of org.zalando.nakadi.domain.EventTypeStatistics in project nakadi by zalando.

the class PartitionsCalculatorTest method testIntegerOverflowOnStatisticsCalculation.

@Test
public void testIntegerOverflowOnStatisticsCalculation() throws IOException {
    final PartitionsCalculator calculator = buildTest();
    final EventTypeStatistics statistics = new EventTypeStatistics();
    statistics.setReadParallelism(1);
    statistics.setWriteParallelism(1);
    statistics.setMessagesPerMinute(1000000000);
    statistics.setMessageSize(1000000000);
    Assert.assertThat(calculator.getBestPartitionsCount(statistics), equalTo(6));
}