Examples with BufferedPublisher io.confluent.ksql.reactive.BufferedPublisher used on opensource projects

Example 1 with BufferedPublisher

use of io.confluent.ksql.reactive.BufferedPublisher in project ksql by confluentinc.

the class PushRoutingTest method shouldFail_hitRequestLimitLocal.

@Test
public void shouldFail_hitRequestLimitLocal() throws ExecutionException, InterruptedException {
    // Given:
    transientQueryQueue = new TransientQueryQueue(OptionalInt.empty(), 1, 100);
    when(pushRoutingOptions.getHasBeenForwarded()).thenReturn(true);
    final PushRouting routing = new PushRouting();
    BufferedPublisher<QueryRow> localPublisher = new BufferedPublisher<>(context);
    when(pushPhysicalPlanManager.execute()).thenReturn(localPublisher);
    // When:
    final PushConnectionsHandle handle = handlePushRouting(routing);
    context.runOnContext(v -> {
        localPublisher.accept(LOCAL_ROW1);
        localPublisher.accept(LOCAL_ROW2);
    });
    // Then:
    Set<List<?>> rows = waitOnRows(1);
    handle.close();
    assertThat(rows.contains(LOCAL_ROW1.value().values()), is(true));
    assertThat(handle.getError().getMessage(), containsString("Hit limit of request queue"));
}

Example 2 with BufferedPublisher

use of io.confluent.ksql.reactive.BufferedPublisher in project ksql by confluentinc.

the class PushPhysicalPlan method subscribeAndExecute.

// for testing only
BufferedPublisher<QueryRow> subscribeAndExecute(final Optional<Subscriber<QueryRow>> subscriber) {
    final Publisher publisher = new Publisher(context);
    subscriber.ifPresent(publisher::subscribe);
    context.runOnContext(v -> open(publisher));
    return publisher;
}

Example 3 with BufferedPublisher

use of io.confluent.ksql.reactive.BufferedPublisher in project ksql by confluentinc.

the class InsertsSubscriber method createInsertsSubscriber.

public static InsertsSubscriber createInsertsSubscriber(final ServiceContext serviceContext, final JsonObject properties, final DataSource dataSource, final KsqlConfig ksqlConfig, final Context context, final Subscriber<InsertResult> acksSubscriber, final WorkerExecutor workerExecutor) {
    final KsqlConfig configCopy = ksqlConfig.cloneWithPropertyOverwrite(properties.getMap());
    final Producer<byte[], byte[]> producer = serviceContext.getKafkaClientSupplier().getProducer(configCopy.originals());
    final PhysicalSchema physicalSchema = PhysicalSchema.from(dataSource.getSchema(), dataSource.getKsqlTopic().getKeyFormat().getFeatures(), dataSource.getKsqlTopic().getValueFormat().getFeatures());
    final KeySerdeFactory keySerdeFactory = new GenericKeySerDe();
    final Serde<GenericKey> keySerde = keySerdeFactory.create(dataSource.getKsqlTopic().getKeyFormat().getFormatInfo(), physicalSchema.keySchema(), ksqlConfig, serviceContext.getSchemaRegistryClientFactory(), "", NoopProcessingLogContext.INSTANCE, Optional.empty());
    final ValueSerdeFactory valueSerdeFactory = new GenericRowSerDe();
    final Serde<GenericRow> valueSerde = valueSerdeFactory.create(dataSource.getKsqlTopic().getValueFormat().getFormatInfo(), physicalSchema.valueSchema(), ksqlConfig, serviceContext.getSchemaRegistryClientFactory(), "", NoopProcessingLogContext.INSTANCE, Optional.empty());
    final BufferedPublisher<InsertResult> acksPublisher = new BufferedPublisher<>(context);
    acksPublisher.subscribe(acksSubscriber);
    return new InsertsSubscriber(context, producer, dataSource, keySerde.serializer(), valueSerde.serializer(), acksPublisher, workerExecutor);
}

Example 4 with BufferedPublisher

use of io.confluent.ksql.reactive.BufferedPublisher in project ksql by confluentinc.

the class BufferedPublisherVerificationTest method createPublisher.

@Override
public Publisher<JsonObject> createPublisher(long elements) {
    final Context context = vertx.getOrCreateContext();
    List<JsonObject> initialElements = new ArrayList<>();
    if (elements < Integer.MAX_VALUE) {
        for (long l = 0; l < elements; l++) {
            initialElements.add(generateRow(l));
        }
    }
    BufferedPublisher<JsonObject> bufferedPublisher = new BufferedPublisher<>(context, initialElements);
    // infinity
    if (elements != Long.MAX_VALUE) {
        context.runOnContext(v -> bufferedPublisher.complete());
    }
    return bufferedPublisher;
}