Search in sources :

Example 1 with ItemsWrapper

use of org.zalando.nakadi.domain.ItemsWrapper in project nakadi by zalando.

the class SubscriptionService method getSubscriptionStat.

public ItemsWrapper<SubscriptionEventTypeStats> getSubscriptionStat(final String subscriptionId) throws InconsistentStateException, NoSuchSubscriptionException, ServiceTemporarilyUnavailableException {
    final Subscription subscription;
    try {
        subscription = subscriptionRepository.getSubscription(subscriptionId);
    } catch (final ServiceUnavailableException ex) {
        throw new InconsistentStateException(ex.getMessage());
    }
    final List<SubscriptionEventTypeStats> subscriptionStat = createSubscriptionStat(subscription);
    return new ItemsWrapper<>(subscriptionStat);
}
Also used : ServiceUnavailableException(org.zalando.nakadi.exceptions.ServiceUnavailableException) SubscriptionEventTypeStats(org.zalando.nakadi.domain.SubscriptionEventTypeStats) Subscription(org.zalando.nakadi.domain.Subscription) ItemsWrapper(org.zalando.nakadi.domain.ItemsWrapper) InconsistentStateException(org.zalando.nakadi.exceptions.runtime.InconsistentStateException)

Example 2 with ItemsWrapper

use of org.zalando.nakadi.domain.ItemsWrapper in project nakadi by zalando.

the class NakadiTestUtils method getNumberOfAssignedStreams.

public static int getNumberOfAssignedStreams(final String sid) {
    final Response response = when().get("/subscriptions/{sid}/stats", sid).thenReturn();
    final ItemsWrapper<SubscriptionEventTypeStats> statsItems;
    try {
        statsItems = MAPPER.readValue(response.print(), new TypeReference<ItemsWrapper<SubscriptionEventTypeStats>>() {
        });
    } catch (final IOException e) {
        throw new AssertionError("Failed to get stats", e);
    }
    final long assignedUniqueStreamsCount = statsItems.getItems().stream().flatMap(stat -> stat.getPartitions().stream()).filter(p -> "assigned".equals(p.getState())).map(SubscriptionEventTypeStats.Partition::getStreamId).distinct().count();
    return (int) assignedUniqueStreamsCount;
}
Also used : Response(com.jayway.restassured.response.Response) IntStream(java.util.stream.IntStream) EventCategory(org.zalando.nakadi.domain.EventCategory) DateTimeZone(org.joda.time.DateTimeZone) RequestSpecification(com.jayway.restassured.specification.RequestSpecification) RandomSubscriptionBuilder(org.zalando.nakadi.utils.RandomSubscriptionBuilder) SubscriptionEventTypeStats(org.zalando.nakadi.domain.SubscriptionEventTypeStats) HttpStatus(org.apache.http.HttpStatus) RestAssured.given(com.jayway.restassured.RestAssured.given) Subscription(org.zalando.nakadi.domain.Subscription) EnrichmentStrategyDescriptor(org.zalando.nakadi.domain.EnrichmentStrategyDescriptor) Response(com.jayway.restassured.response.Response) MessageFormat.format(java.text.MessageFormat.format) JSONObject(org.json.JSONObject) JsonConfig(org.zalando.nakadi.config.JsonConfig) ImmutableList(com.google.common.collect.ImmutableList) EventTypeStatistics(org.zalando.nakadi.domain.EventTypeStatistics) Map(java.util.Map) TypeReference(com.fasterxml.jackson.core.type.TypeReference) ItemsWrapper(org.zalando.nakadi.domain.ItemsWrapper) IntFunction(java.util.function.IntFunction) EventType(org.zalando.nakadi.domain.EventType) RestAssured.when(com.jayway.restassured.RestAssured.when) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) DateTime(org.joda.time.DateTime) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException) IOException(java.io.IOException) JSON(com.jayway.restassured.http.ContentType.JSON) UUID(java.util.UUID) EventTypeTestBuilder(org.zalando.nakadi.utils.EventTypeTestBuilder) Collectors(java.util.stream.Collectors) List(java.util.List) SubscriptionBase(org.zalando.nakadi.domain.SubscriptionBase) PartitionStrategy(org.zalando.nakadi.partitioning.PartitionStrategy) TimelineView(org.zalando.nakadi.view.TimelineView) SubscriptionCursor(org.zalando.nakadi.view.SubscriptionCursor) RestAssured(com.jayway.restassured.RestAssured) Assert(org.junit.Assert) OK(org.springframework.http.HttpStatus.OK) ContentType(com.jayway.restassured.http.ContentType) JSONArray(org.json.JSONArray) SubscriptionEventTypeStats(org.zalando.nakadi.domain.SubscriptionEventTypeStats) TypeReference(com.fasterxml.jackson.core.type.TypeReference) IOException(java.io.IOException)

Example 3 with ItemsWrapper

use of org.zalando.nakadi.domain.ItemsWrapper in project nakadi by zalando.

the class HilaAT method whenPatchThenCursorsAreInitializedToDefault.

@Test(timeout = 15000)
public void whenPatchThenCursorsAreInitializedToDefault() throws Exception {
    final EventType et = createEventType();
    publishEvents(et.getName(), 10, i -> "{\"foo\": \"bar\"}");
    Thread.sleep(1000L);
    final Subscription s = createSubscription(RandomSubscriptionBuilder.builder().withEventType(et.getName()).withStartFrom(END).buildSubscriptionBase());
    given().body(MAPPER.writeValueAsString(new ItemsWrapper<>(Collections.emptyList()))).contentType(JSON).patch("/subscriptions/{id}/cursors", s.getId()).then().statusCode(SC_NO_CONTENT);
    final ItemsWrapper<SubscriptionCursor> subscriptionCursors = MAPPER.readValue(given().get("/subscriptions/{id}/cursors", s.getId()).getBody().asString(), new TypeReference<ItemsWrapper<SubscriptionCursor>>() {
    });
    final List<EventTypePartitionView> etStats = MAPPER.readValue(given().get("/event-types/{et}/partitions", et.getName()).getBody().asString(), new TypeReference<List<EventTypePartitionView>>() {
    });
    Assert.assertEquals(subscriptionCursors.getItems().size(), etStats.size());
    subscriptionCursors.getItems().forEach(sCursor -> {
        final boolean offsetSame = etStats.stream().anyMatch(ss -> ss.getPartitionId().equals(sCursor.getPartition()) && ss.getNewestAvailableOffset().equals(sCursor.getOffset()));
        // Check that after patch cursors are the same as END
        Assert.assertTrue(offsetSame);
    });
}
Also used : SubscriptionCursor(org.zalando.nakadi.view.SubscriptionCursor) NakadiTestUtils.createEventType(org.zalando.nakadi.webservice.utils.NakadiTestUtils.createEventType) EventType(org.zalando.nakadi.domain.EventType) EventTypePartitionView(org.zalando.nakadi.view.EventTypePartitionView) List(java.util.List) ImmutableList(com.google.common.collect.ImmutableList) Subscription(org.zalando.nakadi.domain.Subscription) NakadiTestUtils.createSubscription(org.zalando.nakadi.webservice.utils.NakadiTestUtils.createSubscription) ItemsWrapper(org.zalando.nakadi.domain.ItemsWrapper) Test(org.junit.Test)

Example 4 with ItemsWrapper

use of org.zalando.nakadi.domain.ItemsWrapper in project nakadi by zalando.

the class HilaAT method whenResetCursorsThenStreamFromResetCursorOffset.

@Test(timeout = 15000)
public void whenResetCursorsThenStreamFromResetCursorOffset() throws Exception {
    publishEvents(eventType.getName(), 20, i -> "{\"foo\":\"bar\"}");
    final TestStreamingClient client1 = TestStreamingClient.create(subscription.getId()).start();
    waitFor(() -> assertThat(client1.getBatches(), hasSize(10)));
    int statusCode = commitCursors(subscription.getId(), Collections.singletonList(client1.getBatches().get(9).getCursor()), client1.getSessionId());
    Assert.assertEquals(SC_NO_CONTENT, statusCode);
    final List<SubscriptionCursor> resetCursors = Collections.singletonList(client1.getBatches().get(4).getCursor());
    statusCode = given().body(MAPPER.writeValueAsString(new ItemsWrapper<>(resetCursors))).contentType(JSON).patch("/subscriptions/{id}/cursors", subscription.getId()).getStatusCode();
    Assert.assertEquals(SC_NO_CONTENT, statusCode);
    Assert.assertFalse(client1.isRunning());
    Assert.assertTrue(client1.getBatches().stream().anyMatch(streamBatch -> streamBatch.getMetadata() != null && streamBatch.getMetadata().getDebug().equals("Resetting subscription cursors")));
    final TestStreamingClient client2 = TestStreamingClient.create(subscription.getId()).start();
    waitFor(() -> assertThat(client2.getBatches(), hasSize(10)));
    Assert.assertEquals("001-0001-000000000000000005", client2.getBatches().get(0).getCursor().getOffset());
}
Also used : SubscriptionCursor(org.zalando.nakadi.view.SubscriptionCursor) CoreMatchers.is(org.hamcrest.CoreMatchers.is) NakadiTestUtils.createEventType(org.zalando.nakadi.webservice.utils.NakadiTestUtils.createEventType) NakadiTestUtils.getNumberOfAssignedStreams(org.zalando.nakadi.webservice.utils.NakadiTestUtils.getNumberOfAssignedStreams) NakadiTestUtils.publishEvent(org.zalando.nakadi.webservice.utils.NakadiTestUtils.publishEvent) RandomSubscriptionBuilder(org.zalando.nakadi.utils.RandomSubscriptionBuilder) Matchers.not(org.hamcrest.Matchers.not) HttpStatus(org.apache.http.HttpStatus) RestAssured.given(com.jayway.restassured.RestAssured.given) Subscription(org.zalando.nakadi.domain.Subscription) BaseAT(org.zalando.nakadi.webservice.BaseAT) AUTO(org.zalando.nakadi.domain.SubscriptionEventTypeStats.Partition.AssignmentType.AUTO) StringContains(org.hamcrest.core.StringContains) StreamBatch.singleEventBatch(org.zalando.nakadi.webservice.hila.StreamBatch.singleEventBatch) SubscriptionCursorWithoutToken(org.zalando.nakadi.view.SubscriptionCursorWithoutToken) TypeReference(com.fasterxml.jackson.core.type.TypeReference) ImmutableMap(com.google.common.collect.ImmutableMap) JSON(com.jayway.restassured.http.ContentType.JSON) BlacklistService(org.zalando.nakadi.service.BlacklistService) Collectors(java.util.stream.Collectors) NakadiTestUtils.commitCursors(org.zalando.nakadi.webservice.utils.NakadiTestUtils.commitCursors) List(java.util.List) EventTypePartitionView(org.zalando.nakadi.view.EventTypePartitionView) SubscriptionBase(org.zalando.nakadi.domain.SubscriptionBase) Optional(java.util.Optional) SubscriptionCursor(org.zalando.nakadi.view.SubscriptionCursor) SettingsControllerAT(org.zalando.nakadi.webservice.SettingsControllerAT) SC_NO_CONTENT(org.apache.http.HttpStatus.SC_NO_CONTENT) BEGIN(org.zalando.nakadi.domain.SubscriptionBase.InitialPosition.BEGIN) SubscriptionEventTypeStats(org.zalando.nakadi.domain.SubscriptionEventTypeStats) CoreMatchers.equalTo(org.hamcrest.CoreMatchers.equalTo) JsonTestHelper(org.zalando.nakadi.utils.JsonTestHelper) Cursor(org.zalando.nakadi.view.Cursor) MessageFormat.format(java.text.MessageFormat.format) Lists(com.google.common.collect.Lists) JsonConfig(org.zalando.nakadi.config.JsonConfig) ImmutableList(com.google.common.collect.ImmutableList) NakadiTestUtils(org.zalando.nakadi.webservice.utils.NakadiTestUtils) Matchers.hasSize(org.hamcrest.Matchers.hasSize) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) END(org.zalando.nakadi.domain.SubscriptionBase.InitialPosition.END) NakadiTestUtils.createSubscription(org.zalando.nakadi.webservice.utils.NakadiTestUtils.createSubscription) Before(org.junit.Before) ItemsWrapper(org.zalando.nakadi.domain.ItemsWrapper) SESSION_ID_UNKNOWN(org.zalando.nakadi.webservice.utils.TestStreamingClient.SESSION_ID_UNKNOWN) EventType(org.zalando.nakadi.domain.EventType) Matchers.empty(org.hamcrest.Matchers.empty) RestAssured.when(com.jayway.restassured.RestAssured.when) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Matchers(org.hamcrest.Matchers) TestUtils.waitFor(org.zalando.nakadi.utils.TestUtils.waitFor) Test(org.junit.Test) IOException(java.io.IOException) DIRECT(org.zalando.nakadi.domain.SubscriptionEventTypeStats.Partition.AssignmentType.DIRECT) TimeUnit(java.util.concurrent.TimeUnit) SC_OK(org.apache.http.HttpStatus.SC_OK) MatcherIgnoringToken.equalToBatchIgnoringToken(org.zalando.nakadi.webservice.hila.StreamBatch.MatcherIgnoringToken.equalToBatchIgnoringToken) NakadiTestUtils.publishEvents(org.zalando.nakadi.webservice.utils.NakadiTestUtils.publishEvents) TestStreamingClient(org.zalando.nakadi.webservice.utils.TestStreamingClient) SC_CONFLICT(org.apache.http.HttpStatus.SC_CONFLICT) Assert(org.junit.Assert) Collections(java.util.Collections) TestStreamingClient(org.zalando.nakadi.webservice.utils.TestStreamingClient) ItemsWrapper(org.zalando.nakadi.domain.ItemsWrapper) Test(org.junit.Test)

Example 5 with ItemsWrapper

use of org.zalando.nakadi.domain.ItemsWrapper in project nakadi by zalando.

the class HilaAT method whenPatchThenCursorsAreInitializedAndPatched.

@Test(timeout = 15000)
public void whenPatchThenCursorsAreInitializedAndPatched() throws Exception {
    final EventType et = createEventType();
    publishEvents(et.getName(), 10, i -> "{\"foo\": \"bar\"}");
    final List<EventTypePartitionView> etStats = MAPPER.readValue(given().get("/event-types/{et}/partitions", et.getName()).getBody().asString(), new TypeReference<List<EventTypePartitionView>>() {
    });
    final EventTypePartitionView begin = etStats.get(0);
    final Subscription s = createSubscription(RandomSubscriptionBuilder.builder().withEventType(et.getName()).withStartFrom(END).buildSubscriptionBase());
    given().body(MAPPER.writeValueAsString(new ItemsWrapper<>(Collections.singletonList(new SubscriptionCursorWithoutToken(et.getName(), begin.getPartitionId(), begin.getOldestAvailableOffset()))))).contentType(JSON).patch("/subscriptions/{id}/cursors", s.getId()).then().statusCode(SC_NO_CONTENT);
    final ItemsWrapper<SubscriptionCursor> subscriptionCursors = MAPPER.readValue(given().get("/subscriptions/{id}/cursors", s.getId()).getBody().asString(), new TypeReference<ItemsWrapper<SubscriptionCursor>>() {
    });
    Assert.assertEquals(subscriptionCursors.getItems().size(), etStats.size());
    subscriptionCursors.getItems().forEach(item -> {
        if (item.getPartition().equals(begin.getPartitionId())) {
            Assert.assertEquals(begin.getOldestAvailableOffset(), item.getOffset());
        } else {
            Assert.assertEquals(begin.getNewestAvailableOffset(), item.getOffset());
        }
    });
}
Also used : SubscriptionCursor(org.zalando.nakadi.view.SubscriptionCursor) SubscriptionCursorWithoutToken(org.zalando.nakadi.view.SubscriptionCursorWithoutToken) NakadiTestUtils.createEventType(org.zalando.nakadi.webservice.utils.NakadiTestUtils.createEventType) EventType(org.zalando.nakadi.domain.EventType) EventTypePartitionView(org.zalando.nakadi.view.EventTypePartitionView) List(java.util.List) ImmutableList(com.google.common.collect.ImmutableList) Subscription(org.zalando.nakadi.domain.Subscription) NakadiTestUtils.createSubscription(org.zalando.nakadi.webservice.utils.NakadiTestUtils.createSubscription) ItemsWrapper(org.zalando.nakadi.domain.ItemsWrapper) Test(org.junit.Test)

Aggregations

ItemsWrapper (org.zalando.nakadi.domain.ItemsWrapper)7 Subscription (org.zalando.nakadi.domain.Subscription)6 Test (org.junit.Test)5 ImmutableList (com.google.common.collect.ImmutableList)4 List (java.util.List)4 EventType (org.zalando.nakadi.domain.EventType)4 SubscriptionEventTypeStats (org.zalando.nakadi.domain.SubscriptionEventTypeStats)4 SubscriptionCursor (org.zalando.nakadi.view.SubscriptionCursor)4 NakadiTestUtils.createSubscription (org.zalando.nakadi.webservice.utils.NakadiTestUtils.createSubscription)3 TypeReference (com.fasterxml.jackson.core.type.TypeReference)2 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)2 RestAssured.given (com.jayway.restassured.RestAssured.given)2 RestAssured.when (com.jayway.restassured.RestAssured.when)2 JSON (com.jayway.restassured.http.ContentType.JSON)2 Response (com.jayway.restassured.response.Response)2 IOException (java.io.IOException)2 MessageFormat.format (java.text.MessageFormat.format)2 Collectors (java.util.stream.Collectors)2 HttpStatus (org.apache.http.HttpStatus)2 Assert (org.junit.Assert)2