Search in sources :

Example 1 with MessageMetadata

use of com.google.cloud.pubsublite.MessageMetadata in project beam by apache.

the class PublisherCache method get.

synchronized Publisher<MessageMetadata> get(PublisherOptions options) throws ApiException {
    Publisher<MessageMetadata> publisher = livePublishers.get(options);
    if (publisher != null) {
        return publisher;
    }
    publisher = new PublisherAssembler(options).newPublisher();
    livePublishers.put(options, publisher);
    publisher.addListener(new Listener() {

        @Override
        public void failed(State s, Throwable t) {
            logger.warn("Publisher failed.", t);
            evict(options);
        }
    }, SystemExecutors.getFuturesExecutor());
    publisher.startAsync().awaitRunning();
    return publisher;
}
Also used : MessageMetadata(com.google.cloud.pubsublite.MessageMetadata) Listener(com.google.api.core.ApiService.Listener) State(com.google.api.core.ApiService.State)

Example 2 with MessageMetadata

use of com.google.cloud.pubsublite.MessageMetadata in project beam by apache.

the class PubsubLiteSinkTest method exceptionMixedWithOK.

@Test
public void exceptionMixedWithOK() throws Exception {
    Message message1 = Message.builder().build();
    Message message2 = Message.builder().setKey(ByteString.copyFromUtf8("abc")).build();
    Message message3 = Message.builder().setKey(ByteString.copyFromUtf8("def")).build();
    SettableApiFuture<MessageMetadata> future1 = SettableApiFuture.create();
    SettableApiFuture<MessageMetadata> future2 = SettableApiFuture.create();
    SettableApiFuture<MessageMetadata> future3 = SettableApiFuture.create();
    CountDownLatch startedLatch = new CountDownLatch(3);
    when(publisher.publish(message1)).then(invocation -> {
        startedLatch.countDown();
        return future1;
    });
    when(publisher.publish(message2)).then(invocation -> {
        startedLatch.countDown();
        return future2;
    });
    when(publisher.publish(message3)).then(invocation -> {
        startedLatch.countDown();
        return future3;
    });
    ExecutorService exec = Executors.newCachedThreadPool();
    exec.execute(() -> {
        try {
            startedLatch.await();
            future1.set(MessageMetadata.of(Partition.of(1), Offset.of(2)));
            future2.setException(new CheckedApiException(Code.INTERNAL).underlying);
            future3.set(MessageMetadata.of(Partition.of(1), Offset.of(3)));
        } catch (InterruptedException e) {
            fail();
            throw new RuntimeException(e);
        }
    });
    PipelineExecutionException e = assertThrows(PipelineExecutionException.class, () -> runWith(message1, message2, message3));
    verify(publisher, times(3)).publish(publishedMessageCaptor.capture());
    assertThat(publishedMessageCaptor.getAllValues(), containsInAnyOrder(message1, message2, message3));
    Optional<CheckedApiException> statusOr = ExtractStatus.extract(e.getCause());
    assertTrue(statusOr.isPresent());
    assertThat(statusOr.get().code(), equalTo(Code.INTERNAL));
    exec.shutdownNow();
}
Also used : MessageMetadata(com.google.cloud.pubsublite.MessageMetadata) Message(com.google.cloud.pubsublite.Message) PipelineExecutionException(org.apache.beam.sdk.Pipeline.PipelineExecutionException) ExecutorService(java.util.concurrent.ExecutorService) CheckedApiException(com.google.cloud.pubsublite.internal.CheckedApiException) CountDownLatch(java.util.concurrent.CountDownLatch) Test(org.junit.Test)

Aggregations

MessageMetadata (com.google.cloud.pubsublite.MessageMetadata)2 Listener (com.google.api.core.ApiService.Listener)1 State (com.google.api.core.ApiService.State)1 Message (com.google.cloud.pubsublite.Message)1 CheckedApiException (com.google.cloud.pubsublite.internal.CheckedApiException)1 CountDownLatch (java.util.concurrent.CountDownLatch)1 ExecutorService (java.util.concurrent.ExecutorService)1 PipelineExecutionException (org.apache.beam.sdk.Pipeline.PipelineExecutionException)1 Test (org.junit.Test)1