Search in sources :

Example 41 with DestinationName

use of com.yahoo.pulsar.common.naming.DestinationName in project pulsar by yahoo.

the class PartitionedProducerConsumerTest method testAsyncPartitionedProducerConsumerQueueSizeOne.

@Test(timeOut = 4000)
public void testAsyncPartitionedProducerConsumerQueueSizeOne() throws Exception {
    log.info("-- Starting {} test --", methodName);
    final int totalMsg = 100;
    final Set<String> produceMsgs = Sets.newHashSet();
    final Set<String> consumeMsgs = Sets.newHashSet();
    int numPartitions = 4;
    DestinationName dn = DestinationName.get("persistent://my-property/use/my-ns/my-partitionedtopic1");
    ConsumerConfiguration conf = new ConsumerConfiguration();
    conf.setReceiverQueueSize(1);
    admin.persistentTopics().createPartitionedTopic(dn.toString(), numPartitions);
    ProducerConfiguration producerConf = new ProducerConfiguration();
    producerConf.setMessageRoutingMode(MessageRoutingMode.RoundRobinPartition);
    Producer producer = pulsarClient.createProducer(dn.toString(), producerConf);
    Consumer consumer = pulsarClient.subscribe(dn.toString(), "my-partitioned-subscriber", conf);
    // produce messages
    for (int i = 0; i < totalMsg; i++) {
        String message = "my-message-" + i;
        produceMsgs.add(message);
        producer.send(message.getBytes());
    }
    log.info(" start receiving messages :");
    // receive messages
    CountDownLatch latch = new CountDownLatch(totalMsg);
    receiveAsync(consumer, totalMsg, 0, latch, consumeMsgs, Executors.newFixedThreadPool(1));
    latch.await();
    // verify message produced correctly
    assertEquals(produceMsgs.size(), totalMsg);
    // verify produced and consumed messages must be exactly same
    produceMsgs.removeAll(consumeMsgs);
    assertTrue(produceMsgs.isEmpty());
    producer.close();
    consumer.unsubscribe();
    consumer.close();
    admin.persistentTopics().deletePartitionedTopic(dn.toString());
    log.info("-- Exiting {} test --", methodName);
}
Also used : DestinationName(com.yahoo.pulsar.common.naming.DestinationName) CountDownLatch(java.util.concurrent.CountDownLatch) Test(org.testng.annotations.Test)

Example 42 with DestinationName

use of com.yahoo.pulsar.common.naming.DestinationName in project pulsar by yahoo.

the class DiscoveryServiceTest method testGetPartitionsMetadata.

@Test
public void testGetPartitionsMetadata() throws Exception {
    DestinationName topic1 = DestinationName.get("persistent://test/local/ns/my-topic-1");
    PartitionedTopicMetadata m = service.getDiscoveryProvider().getPartitionedTopicMetadata(service, topic1, "role").get();
    assertEquals(m.partitions, 0);
    // Simulate ZK error
    mockZookKeeper.failNow(Code.SESSIONEXPIRED);
    DestinationName topic2 = DestinationName.get("persistent://test/local/ns/my-topic-2");
    CompletableFuture<PartitionedTopicMetadata> future = service.getDiscoveryProvider().getPartitionedTopicMetadata(service, topic2, "role");
    try {
        future.get();
        fail("Partition metadata lookup should have failed");
    } catch (ExecutionException e) {
        assertEquals(e.getCause().getClass(), SessionExpiredException.class);
    }
}
Also used : DestinationName(com.yahoo.pulsar.common.naming.DestinationName) SessionExpiredException(org.apache.zookeeper.KeeperException.SessionExpiredException) ExecutionException(java.util.concurrent.ExecutionException) PartitionedTopicMetadata(com.yahoo.pulsar.common.partition.PartitionedTopicMetadata) Test(org.testng.annotations.Test)

Example 43 with DestinationName

use of com.yahoo.pulsar.common.naming.DestinationName in project pulsar by yahoo.

the class PersistentTopicsImpl method peekNthMessage.

private CompletableFuture<Message> peekNthMessage(String destination, String subName, int messagePosition) {
    DestinationName ds = validateTopic(destination);
    String encodedSubName = Codec.encode(subName);
    final CompletableFuture<Message> future = new CompletableFuture<Message>();
    asyncGetRequest(persistentTopics.path(ds.getNamespace()).path(ds.getEncodedLocalName()).path("subscription").path(encodedSubName).path("position").path(String.valueOf(messagePosition)), new InvocationCallback<Response>() {

        @Override
        public void completed(Response response) {
            try {
                Message msg = getMessageFromHttpResponse(response);
                future.complete(msg);
            } catch (Exception e) {
                future.completeExceptionally(getApiException(e));
            }
        }

        @Override
        public void failed(Throwable throwable) {
            future.completeExceptionally(getApiException(throwable.getCause()));
        }
    });
    return future;
}
Also used : Response(javax.ws.rs.core.Response) CompletableFuture(java.util.concurrent.CompletableFuture) Message(com.yahoo.pulsar.client.api.Message) DestinationName(com.yahoo.pulsar.common.naming.DestinationName) ClientErrorException(javax.ws.rs.ClientErrorException) NotFoundException(com.yahoo.pulsar.client.admin.PulsarAdminException.NotFoundException) PulsarAdminException(com.yahoo.pulsar.client.admin.PulsarAdminException) ExecutionException(java.util.concurrent.ExecutionException) ServerErrorException(javax.ws.rs.ServerErrorException) WebApplicationException(javax.ws.rs.WebApplicationException)

Example 44 with DestinationName

use of com.yahoo.pulsar.common.naming.DestinationName in project pulsar by yahoo.

the class CliCommand method validatePersistentTopic.

String validatePersistentTopic(List<String> params) {
    String destination = checkArgument(params);
    DestinationName ds = DestinationName.get(destination);
    if (ds.getDomain() != DestinationDomain.persistent) {
        throw new ParameterException("Need to provide a persistent topic name");
    }
    return ds.toString();
}
Also used : DestinationName(com.yahoo.pulsar.common.naming.DestinationName) ParameterException(com.beust.jcommander.ParameterException)

Example 45 with DestinationName

use of com.yahoo.pulsar.common.naming.DestinationName in project pulsar by yahoo.

the class PersistentTopicsImpl method getPartitionedStatsAsync.

@Override
public CompletableFuture<PartitionedTopicStats> getPartitionedStatsAsync(String destination, boolean perPartition) {
    DestinationName ds = validateTopic(destination);
    final CompletableFuture<PartitionedTopicStats> future = new CompletableFuture<>();
    asyncGetRequest(persistentTopics.path(ds.getNamespace()).path(ds.getEncodedLocalName()).path("partitioned-stats"), new InvocationCallback<PartitionedTopicStats>() {

        @Override
        public void completed(PartitionedTopicStats response) {
            if (!perPartition) {
                response.partitions.clear();
            }
            future.complete(response);
        }

        @Override
        public void failed(Throwable throwable) {
            future.completeExceptionally(getApiException(throwable.getCause()));
        }
    });
    return future;
}
Also used : CompletableFuture(java.util.concurrent.CompletableFuture) PartitionedTopicStats(com.yahoo.pulsar.common.policies.data.PartitionedTopicStats) DestinationName(com.yahoo.pulsar.common.naming.DestinationName)

Aggregations

DestinationName (com.yahoo.pulsar.common.naming.DestinationName)91 Test (org.testng.annotations.Test)36 PulsarClientException (com.yahoo.pulsar.client.api.PulsarClientException)33 PersistentTopic (com.yahoo.pulsar.broker.service.persistent.PersistentTopic)26 CompletableFuture (java.util.concurrent.CompletableFuture)24 KeeperException (org.apache.zookeeper.KeeperException)23 PreconditionFailedException (com.yahoo.pulsar.client.admin.PulsarAdminException.PreconditionFailedException)22 WebApplicationException (javax.ws.rs.WebApplicationException)21 IOException (java.io.IOException)20 ManagedLedgerException (org.apache.bookkeeper.mledger.ManagedLedgerException)19 RestException (com.yahoo.pulsar.broker.web.RestException)18 NotFoundException (com.yahoo.pulsar.client.admin.PulsarAdminException.NotFoundException)18 PartitionedTopicMetadata (com.yahoo.pulsar.common.partition.PartitionedTopicMetadata)17 SubscriptionBusyException (com.yahoo.pulsar.broker.service.BrokerServiceException.SubscriptionBusyException)16 TopicBusyException (com.yahoo.pulsar.broker.service.BrokerServiceException.TopicBusyException)16 Path (javax.ws.rs.Path)16 NotAllowedException (com.yahoo.pulsar.broker.service.BrokerServiceException.NotAllowedException)14 ApiOperation (io.swagger.annotations.ApiOperation)14 ApiResponses (io.swagger.annotations.ApiResponses)14 Field (java.lang.reflect.Field)14