Search in sources :

Example 1 with Response

use of com.rabbitmq.stream.impl.Client.Response in project rabbitmq-stream-java-client by rabbitmq.

the class SubEntryBatchingTest method publishConsumeCompressedMessages.

@ParameterizedTest
@MethodSource("compressionCodecFactories")
void publishConsumeCompressedMessages(CompressionCodecFactory compressionCodecFactory, TestInfo info) {
    Map<Compression, Integer> compressionToReadBytes = new HashMap<>();
    for (Compression compression : Compression.values()) {
        int batchCount = 100;
        int messagesInBatch = 30;
        int messageCount = batchCount * messagesInBatch;
        CountDownLatch publishLatch = new CountDownLatch(batchCount);
        Client publisher = cf.get(new ClientParameters().compressionCodecFactory(compressionCodecFactory).publishConfirmListener((publisherId, publishingId) -> publishLatch.countDown()));
        String s = TestUtils.streamName(info) + "_" + compression.name();
        try {
            Response response = publisher.create(s);
            assertThat(response.isOk()).isTrue();
            response = publisher.declarePublisher(b(0), null, s);
            assertThat(response.isOk()).isTrue();
            Set<String> publishedBodies = ConcurrentHashMap.newKeySet(messageCount);
            IntStream.range(0, batchCount).forEach(batchIndex -> {
                MessageBatch messageBatch = new MessageBatch(compression);
                IntStream.range(0, messagesInBatch).forEach(messageIndex -> {
                    String body = "batch " + batchIndex + " message " + messageIndex;
                    messageBatch.add(publisher.messageBuilder().addData(body.getBytes(UTF8)).build());
                    publishedBodies.add(body);
                });
                publisher.publishBatches(b(0), Collections.singletonList(messageBatch));
            });
            assertThat(latchAssert(publishLatch)).completes();
            Set<String> consumedBodies = ConcurrentHashMap.newKeySet(batchCount * messagesInBatch);
            CountDownLatch consumeLatch = new CountDownLatch(batchCount * messagesInBatch);
            CountMetricsCollector metricsCollector = new CountMetricsCollector();
            Client consumer = cf.get(new ClientParameters().compressionCodecFactory(compressionCodecFactory).chunkListener((client, subscriptionId, offset, messageCount1, dataSize) -> client.credit(subscriptionId, 1)).messageListener((subscriptionId, offset, chunkTimestamp, message) -> {
                consumedBodies.add(new String(message.getBodyAsBinary(), UTF8));
                consumeLatch.countDown();
            }).metricsCollector(metricsCollector));
            response = consumer.subscribe(b(1), s, OffsetSpecification.first(), 2);
            assertThat(response.isOk()).isTrue();
            assertThat(latchAssert(consumeLatch)).completes();
            assertThat(consumedBodies).hasSize(messageCount).hasSameSizeAs(publishedBodies);
            publishedBodies.forEach(publishedBody -> assertThat(consumedBodies.contains(publishedBody)).isTrue());
            compressionToReadBytes.put(compression, metricsCollector.readBytes.get());
        } finally {
            Response response = publisher.delete(s);
            assertThat(response.isOk()).isTrue();
        }
    }
    int plainReadBytes = compressionToReadBytes.get(Compression.NONE);
    Arrays.stream(Compression.values()).filter(comp -> comp != Compression.NONE).forEach(compression -> {
        assertThat(compressionToReadBytes.get(compression)).isLessThan(plainReadBytes);
    });
}
Also used : IntStream(java.util.stream.IntStream) Arrays(java.util.Arrays) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) TestUtils.latchAssert(com.rabbitmq.stream.impl.TestUtils.latchAssert) CommonsCompressCompressionCodecFactory(com.rabbitmq.stream.compression.CommonsCompressCompressionCodecFactory) HashMap(java.util.HashMap) Response(com.rabbitmq.stream.impl.Client.Response) TestUtils.b(com.rabbitmq.stream.impl.TestUtils.b) ClientParameters(com.rabbitmq.stream.impl.Client.ClientParameters) Charset(java.nio.charset.Charset) ExtendWith(org.junit.jupiter.api.extension.ExtendWith) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Map(java.util.Map) DefaultCompressionCodecFactory(com.rabbitmq.stream.compression.DefaultCompressionCodecFactory) MethodSource(org.junit.jupiter.params.provider.MethodSource) MetricsCollector(com.rabbitmq.stream.metrics.MetricsCollector) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Set(java.util.Set) Compression(com.rabbitmq.stream.compression.Compression) Collectors(java.util.stream.Collectors) StandardCharsets(java.nio.charset.StandardCharsets) TestInfo(org.junit.jupiter.api.TestInfo) Test(org.junit.jupiter.api.Test) CountDownLatch(java.util.concurrent.CountDownLatch) List(java.util.List) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) CompressionCodecFactory(com.rabbitmq.stream.compression.CompressionCodecFactory) Stream(java.util.stream.Stream) Collections(java.util.Collections) OffsetSpecification(com.rabbitmq.stream.OffsetSpecification) Compression(com.rabbitmq.stream.compression.Compression) HashMap(java.util.HashMap) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) CountDownLatch(java.util.concurrent.CountDownLatch) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Response(com.rabbitmq.stream.impl.Client.Response) ClientParameters(com.rabbitmq.stream.impl.Client.ClientParameters) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) MethodSource(org.junit.jupiter.params.provider.MethodSource)

Example 2 with Response

use of com.rabbitmq.stream.impl.Client.Response in project rabbitmq-stream-java-client by rabbitmq.

the class OffsetTrackingTest method consumeAndStore.

@ParameterizedTest
@MethodSource
void consumeAndStore(BiConsumer<String, Client> streamCreator, TestInfo info) throws Exception {
    String s = streamName(info);
    int batchSize = 100;
    int batchCount = 1_000;
    int messageCount = batchSize * batchCount;
    CountDownLatch publishLatch = new CountDownLatch(messageCount);
    Client publisher = cf.get(new ClientParameters().publishConfirmListener((publisherId, publishingId) -> publishLatch.countDown()));
    ExecutorService executorService = Executors.newCachedThreadPool();
    try {
        streamCreator.accept(s, publisher);
        byte[] body = new byte[100];
        AtomicInteger messageIdSequence = new AtomicInteger();
        // publishing a bunch of messages
        AtomicLong lastMessageId = new AtomicLong();
        publisher.declarePublisher(b(0), null, s);
        IntStream.range(0, batchCount).forEach(batchIndex -> publisher.publish(b(0), IntStream.range(0, batchSize).map(i -> messageIdSequence.incrementAndGet()).mapToObj(messageId -> {
            lastMessageId.set(messageId);
            return publisher.messageBuilder().addData(body).properties().messageId(messageId).messageBuilder().build();
        }).collect(Collectors.toList())));
        boolean done = publishLatch.await(2, TimeUnit.SECONDS);
        assertThat(done).isTrue();
        Stream<Tuple3<Integer, Integer, String>> testConfigurations = Stream.of(// { storeEvery, consumeCountFirst, reference }
        Tuple.of(100, messageCount / 10, "ref-1"), Tuple.of(50, messageCount / 20, "ref-2"), Tuple.of(10, messageCount / 100, "ref-3"));
        Function<Tuple3<Integer, Integer, String>, Callable<Void>> testConfigurationToTask = testConfiguration -> () -> {
            int storeEvery = testConfiguration._1;
            int consumeCountFirst = testConfiguration._2;
            String reference = testConfiguration._3;
            AtomicInteger consumeCount = new AtomicInteger();
            AtomicLong lastStoredOffset = new AtomicLong();
            AtomicLong lastConsumedMessageId = new AtomicLong();
            AtomicReference<Client> consumerReference = new AtomicReference<>();
            Set<Long> messageIdsSet = ConcurrentHashMap.newKeySet(messageCount);
            Collection<Long> messageIdsCollection = new ConcurrentLinkedQueue<>();
            CountDownLatch consumeLatch = new CountDownLatch(1);
            MessageListener messageListener = (subscriptionId, offset, chunkTimestamp, message) -> {
                if (consumeCount.get() <= consumeCountFirst) {
                    consumeCount.incrementAndGet();
                    long messageId = message.getProperties().getMessageIdAsLong();
                    messageIdsSet.add(messageId);
                    messageIdsCollection.add(messageId);
                    lastConsumedMessageId.set(messageId);
                    if (consumeCount.get() % storeEvery == 0) {
                        consumerReference.get().storeOffset(reference, s, offset);
                        lastStoredOffset.set(offset);
                    }
                } else {
                    consumeLatch.countDown();
                }
            };
            Client consumer = cf.get(new ClientParameters().creditNotification((subscriptionId, responseCode) -> LOGGER.debug("Received notification for subscription {}: {}", subscriptionId, responseCode)).chunkListener((client, subscriptionId, offset, messageCount1, dataSize) -> client.credit(subscriptionId, 1)).messageListener(messageListener));
            consumerReference.set(consumer);
            consumer.subscribe(b(0), s, OffsetSpecification.offset(0), 1);
            assertThat(consumeLatch.await(10, TimeUnit.SECONDS)).isTrue();
            Response response = consumer.unsubscribe(b(0));
            assertThat(response.isOk()).isTrue();
            assertThat(lastStoredOffset.get()).isPositive();
            waitAtMost(5, () -> lastStoredOffset.get() == consumerReference.get().queryOffset(reference, s).getOffset(), () -> "expecting last stored offset to be " + lastStoredOffset + ", but got " + consumerReference.get().queryOffset(reference, s));
            consumer.close();
            CountDownLatch consumeLatchSecondWave = new CountDownLatch(1);
            AtomicLong firstOffset = new AtomicLong(-1);
            messageListener = (subscriptionId, offset, chunkTimestamp, message) -> {
                firstOffset.compareAndSet(-1, offset);
                long messageId = message.getProperties().getMessageIdAsLong();
                if (lastConsumedMessageId.get() < messageId) {
                    messageIdsSet.add(messageId);
                    messageIdsCollection.add(messageId);
                    consumeCount.incrementAndGet();
                    if (message.getProperties().getMessageIdAsLong() == lastMessageId.get()) {
                        consumeLatchSecondWave.countDown();
                    }
                }
            };
            consumer = cf.get(new ClientParameters().chunkListener((client, subscriptionId, offset, messageCount1, dataSize) -> client.credit(subscriptionId, 1)).messageListener(messageListener));
            long offsetToStartFrom = consumer.queryOffset(reference, s).getOffset() + 1;
            consumer.subscribe(b(0), s, OffsetSpecification.offset(offsetToStartFrom), 1);
            assertThat(consumeLatchSecondWave.await(10, TimeUnit.SECONDS)).isTrue();
            // there can be a non-message entry that is skipped and makes
            // the first received message offset higher
            assertThat(firstOffset.get()).isGreaterThanOrEqualTo(offsetToStartFrom);
            response = consumer.unsubscribe(b(0));
            assertThat(response.isOk()).isTrue();
            assertThat(consumeCount.get()).as("check received all messages").isEqualTo(messageCount);
            assertThat(messageIdsCollection).as("check there are no duplicates").hasSameSizeAs(messageIdsSet);
            return null;
        };
        List<Future<Void>> futures = testConfigurations.map(testConfigurationToTask).map(task -> executorService.submit(task)).collect(Collectors.toList());
        forEach(futures, (i, task) -> {
            assertThatNoException().as("task " + i + " failed").isThrownBy(() -> task.get(10, TimeUnit.SECONDS));
        });
    } finally {
        publisher.delete(s);
        executorService.shutdownNow();
    }
}
Also used : IntStream(java.util.stream.IntStream) CsvSource(org.junit.jupiter.params.provider.CsvSource) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) LoggerFactory(org.slf4j.LoggerFactory) TestUtils.latchAssert(com.rabbitmq.stream.impl.TestUtils.latchAssert) Callable(java.util.concurrent.Callable) Response(com.rabbitmq.stream.impl.Client.Response) AtomicReference(java.util.concurrent.atomic.AtomicReference) Function(java.util.function.Function) TestUtils.b(com.rabbitmq.stream.impl.TestUtils.b) TestUtils.streamName(com.rabbitmq.stream.impl.TestUtils.streamName) ClientParameters(com.rabbitmq.stream.impl.Client.ClientParameters) Future(java.util.concurrent.Future) Assertions.assertThatNoException(org.assertj.core.api.Assertions.assertThatNoException) ExtendWith(org.junit.jupiter.api.extension.ExtendWith) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) TestUtils.forEach(com.rabbitmq.stream.impl.TestUtils.forEach) BiConsumer(java.util.function.BiConsumer) QueryOffsetResponse(com.rabbitmq.stream.impl.Client.QueryOffsetResponse) ExecutorService(java.util.concurrent.ExecutorService) MethodSource(org.junit.jupiter.params.provider.MethodSource) TestUtils.waitAtMost(com.rabbitmq.stream.impl.TestUtils.waitAtMost) Tuple(io.vavr.Tuple) Logger(org.slf4j.Logger) Collection(java.util.Collection) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Set(java.util.Set) MessageListener(com.rabbitmq.stream.impl.Client.MessageListener) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) StandardCharsets(java.nio.charset.StandardCharsets) Executors(java.util.concurrent.Executors) TestInfo(org.junit.jupiter.api.TestInfo) TimeUnit(java.util.concurrent.TimeUnit) Test(org.junit.jupiter.api.Test) CountDownLatch(java.util.concurrent.CountDownLatch) AtomicLong(java.util.concurrent.atomic.AtomicLong) List(java.util.List) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) Stream(java.util.stream.Stream) Tuple3(io.vavr.Tuple3) TestUtils.responseCode(com.rabbitmq.stream.impl.TestUtils.responseCode) StreamParametersBuilder(com.rabbitmq.stream.impl.Client.StreamParametersBuilder) Constants(com.rabbitmq.stream.Constants) Collections(java.util.Collections) OffsetSpecification(com.rabbitmq.stream.OffsetSpecification) ConcurrentLinkedQueue(java.util.concurrent.ConcurrentLinkedQueue) Set(java.util.Set) MessageListener(com.rabbitmq.stream.impl.Client.MessageListener) Callable(java.util.concurrent.Callable) AtomicReference(java.util.concurrent.atomic.AtomicReference) CountDownLatch(java.util.concurrent.CountDownLatch) Response(com.rabbitmq.stream.impl.Client.Response) QueryOffsetResponse(com.rabbitmq.stream.impl.Client.QueryOffsetResponse) AtomicLong(java.util.concurrent.atomic.AtomicLong) ClientParameters(com.rabbitmq.stream.impl.Client.ClientParameters) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Tuple3(io.vavr.Tuple3) ExecutorService(java.util.concurrent.ExecutorService) Collection(java.util.Collection) Future(java.util.concurrent.Future) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) MethodSource(org.junit.jupiter.params.provider.MethodSource)

Example 3 with Response

use of com.rabbitmq.stream.impl.Client.Response in project rabbitmq-stream-java-client by rabbitmq.

the class ClientTest method deleteNonStreamQueueShouldReturnError.

@Test
void deleteNonStreamQueueShouldReturnError() throws Exception {
    String nonStreamQueue = UUID.randomUUID().toString();
    ConnectionFactory connectionFactory = new ConnectionFactory();
    try (Connection amqpConnection = connectionFactory.newConnection()) {
        Channel c = amqpConnection.createChannel();
        c.queueDeclare(nonStreamQueue, false, true, false, null);
        Client.Response response = cf.get().delete(nonStreamQueue);
        assertThat(response.isOk()).isFalse();
        assertThat(response.getResponseCode()).isEqualTo(Constants.RESPONSE_CODE_STREAM_DOES_NOT_EXIST);
    }
}
Also used : ConnectionFactory(com.rabbitmq.client.ConnectionFactory) Channel(com.rabbitmq.client.Channel) Connection(com.rabbitmq.client.Connection) Response(com.rabbitmq.stream.impl.Client.Response) Test(org.junit.jupiter.api.Test) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 4 with Response

use of com.rabbitmq.stream.impl.Client.Response in project rabbitmq-stream-java-client by rabbitmq.

the class ClientTest method createWithInvalidNameShouldReturnError.

@ParameterizedTest
@ValueSource(strings = { "", "amq.somename" })
void createWithInvalidNameShouldReturnError(String name) {
    Client.Response response = cf.get().create(name);
    assertThat(response.isOk()).isFalse();
    assertThat(response.getResponseCode()).isEqualTo(Constants.RESPONSE_CODE_PRECONDITION_FAILED);
}
Also used : Response(com.rabbitmq.stream.impl.Client.Response) ValueSource(org.junit.jupiter.params.provider.ValueSource) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 5 with Response

use of com.rabbitmq.stream.impl.Client.Response in project rabbitmq-stream-java-client by rabbitmq.

the class ClientTest method createAlreadyExistingStreamShouldReturnError.

@Test
void createAlreadyExistingStreamShouldReturnError() {
    Client.Response response = cf.get().create(stream);
    assertThat(response.isOk()).isFalse();
    assertThat(response.getResponseCode()).isEqualTo(Constants.RESPONSE_CODE_STREAM_ALREADY_EXISTS);
}
Also used : Response(com.rabbitmq.stream.impl.Client.Response) Test(org.junit.jupiter.api.Test) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Aggregations

Response (com.rabbitmq.stream.impl.Client.Response)21 Test (org.junit.jupiter.api.Test)18 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)18 CountDownLatch (java.util.concurrent.CountDownLatch)11 IntStream (java.util.stream.IntStream)11 Assertions.assertThat (org.assertj.core.api.Assertions.assertThat)11 ClientParameters (com.rabbitmq.stream.impl.Client.ClientParameters)10 TestUtils.b (com.rabbitmq.stream.impl.TestUtils.b)10 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)10 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)10 TestInfo (org.junit.jupiter.api.TestInfo)10 ExtendWith (org.junit.jupiter.api.extension.ExtendWith)10 AtomicReference (java.util.concurrent.atomic.AtomicReference)9 ValueSource (org.junit.jupiter.params.provider.ValueSource)9 AtomicLong (java.util.concurrent.atomic.AtomicLong)8 StreamParametersBuilder (com.rabbitmq.stream.impl.Client.StreamParametersBuilder)7 TestUtils.latchAssert (com.rabbitmq.stream.impl.TestUtils.latchAssert)7 StandardCharsets (java.nio.charset.StandardCharsets)7 SECONDS (java.util.concurrent.TimeUnit.SECONDS)7 Collectors (java.util.stream.Collectors)7