Search in sources :

Example 1 with ObjectMapperSerde

use of io.quarkus.kafka.client.serialization.ObjectMapperSerde in project AD482-apps by RedHatTraining.

the class AmountWasDepositedPipeline method onStart.

void onStart(@Observes StartupEvent startupEvent) {
    StreamsBuilder builder = new StreamsBuilder();
    ObjectMapperSerde<AmountWasDeposited> depositEventSerde = new ObjectMapperSerde<>(AmountWasDeposited.class);
    ObjectMapperSerde<HighValueDepositWasDetected> highValueEventSerde = new ObjectMapperSerde<>(HighValueDepositWasDetected.class);
    // TODO: Build the stream topology
    builder.stream(AMOUNT_WAS_DEPOSITED_TOPIC, Consumed.with(Serdes.Long(), depositEventSerde)).filter((key, deposit) -> deposit.amount > 1000).map((key, deposit) -> {
        logHighValueDepositAlert(deposit.bankAccountId, deposit.amount);
        return new KeyValue<>(deposit.bankAccountId, new HighValueDepositWasDetected(deposit.bankAccountId, deposit.amount));
    }).to(HIGH_VALUE_DEPOSIT_TOPIC, Produced.with(Serdes.Long(), highValueEventSerde));
    // TODO: Create a Kafka streams and start it
    streams = new KafkaStreams(builder.build(), generateStreamConfig());
    streams.start();
}
Also used : StreamsBuilder(org.apache.kafka.streams.StreamsBuilder) StreamsBuilder(org.apache.kafka.streams.StreamsBuilder) Produced(org.apache.kafka.streams.kstream.Produced) HighValueDepositWasDetected(com.redhat.training.bank.event.HighValueDepositWasDetected) Consumed(org.apache.kafka.streams.kstream.Consumed) Logger(org.jboss.logging.Logger) KeyValue(org.apache.kafka.streams.KeyValue) ShutdownEvent(io.quarkus.runtime.ShutdownEvent) ObjectMapperSerde(io.quarkus.kafka.client.serialization.ObjectMapperSerde) Observes(javax.enterprise.event.Observes) Serdes(org.apache.kafka.common.serialization.Serdes) ApplicationScoped(javax.enterprise.context.ApplicationScoped) StartupEvent(io.quarkus.runtime.StartupEvent) KafkaStreams(org.apache.kafka.streams.KafkaStreams) AmountWasDeposited(com.redhat.training.bank.event.AmountWasDeposited) KafkaStreams(org.apache.kafka.streams.KafkaStreams) HighValueDepositWasDetected(com.redhat.training.bank.event.HighValueDepositWasDetected) ObjectMapperSerde(io.quarkus.kafka.client.serialization.ObjectMapperSerde) AmountWasDeposited(com.redhat.training.bank.event.AmountWasDeposited)

Example 2 with ObjectMapperSerde

use of io.quarkus.kafka.client.serialization.ObjectMapperSerde in project AD482-apps by RedHatTraining.

the class BankAccountWasCreatedPipeline method onStart.

void onStart(@Observes StartupEvent startupEvent) {
    StreamsBuilder builder = new StreamsBuilder();
    ObjectMapperSerde<BankAccountWasCreated> eventSerde = new ObjectMapperSerde<>(BankAccountWasCreated.class);
    // TODO: Update the account type on each event
    builder.stream(BANK_ACCOUNT_WAS_CREATED_TOPIC, Consumed.with(Serdes.Long(), eventSerde)).foreach((key, creation) -> {
        updateAccountTypeFromEvent(creation);
    });
    // TODO: Create a Kafka streams and start it
    streams = new KafkaStreams(builder.build(), generateStreamConfig());
    streams.start();
}
Also used : StreamsBuilder(org.apache.kafka.streams.StreamsBuilder) KafkaStreams(org.apache.kafka.streams.KafkaStreams) BankAccountWasCreated(com.redhat.training.bank.event.BankAccountWasCreated) ObjectMapperSerde(io.quarkus.kafka.client.serialization.ObjectMapperSerde)

Example 3 with ObjectMapperSerde

use of io.quarkus.kafka.client.serialization.ObjectMapperSerde in project AD482-apps by RedHatTraining.

the class TumblingWindows method onStart.

void onStart(@Observes StartupEvent startupEvent) {
    StreamsBuilder builder = new StreamsBuilder();
    ObjectMapperSerde<PotentialCustomersWereDetected> customersEventSerde = new ObjectMapperSerde<>(PotentialCustomersWereDetected.class);
    // TODO: Build the stream topology
    builder.stream(POTENTIAL_CUSTOMERS_TOPIC, Consumed.with(Serdes.String(), customersEventSerde)).groupByKey().windowedBy(TimeWindows.of(Duration.ofSeconds(WINDOW_SIZE)).grace(Duration.ofSeconds(12))).count().toStream().print(Printed.toSysOut());
    streams = new KafkaStreams(builder.build(), generateStreamConfig());
    streams.start();
}
Also used : StreamsBuilder(org.apache.kafka.streams.StreamsBuilder) KafkaStreams(org.apache.kafka.streams.KafkaStreams) ObjectMapperSerde(io.quarkus.kafka.client.serialization.ObjectMapperSerde) PotentialCustomersWereDetected(com.redhat.customers.event.PotentialCustomersWereDetected)

Example 4 with ObjectMapperSerde

use of io.quarkus.kafka.client.serialization.ObjectMapperSerde in project AD482-apps by RedHatTraining.

the class StreamTopologyBuilder method buildTopology.

@Produces
public Topology buildTopology() {
    StreamsBuilder builder = new StreamsBuilder();
    ObjectMapperSerde<WindTurbine> turbineSerde = new ObjectMapperSerde<>(WindTurbine.class);
    builder.table("turbines", Consumed.with(Serdes.Integer(), turbineSerde), Materialized.<Integer, WindTurbine, KeyValueStore<Bytes, byte[]>>as("turbines-store").withKeySerde(Serdes.Integer()).withValueSerde(turbineSerde));
    KStream<Integer, Integer> wattsValuesStream = builder.stream("turbine-generated-watts", Consumed.with(Serdes.Integer(), Serdes.Integer()));
    ObjectMapperSerde<MWattsMeasurement> mwattsMeasurementSerde = new ObjectMapperSerde<>(MWattsMeasurement.class);
    wattsValuesStream.map((turbineId, watts) -> {
        Double megawatts = (double) watts / 1000000;
        MWattsMeasurement measurement = new MWattsMeasurement(turbineId, megawatts);
        System.out.println("MAP - Turbine: " + turbineId + " | " + watts + " Watts -> " + megawatts + " MWatts");
        return KeyValue.pair(turbineId, measurement);
    }).to("turbine-generated-mwatts", Produced.with(Serdes.Integer(), mwattsMeasurementSerde));
    ObjectMapperSerde<WindTurbineStats> statsSerde = new ObjectMapperSerde<>(WindTurbineStats.class);
    wattsValuesStream.groupByKey().windowedBy(TimeWindows.of(Duration.ofSeconds(10)).advanceBy(Duration.ofSeconds(10)).grace(Duration.ofSeconds(12))).count().suppress(Suppressed.untilWindowCloses(unbounded())).toStream().map((windowedTurbineId, count) -> {
        Integer turbineId = windowedTurbineId.key();
        WindTurbineStats stats = new WindTurbineStats(turbineId, count);
        System.out.println("COUNT - Turbine: " + turbineId + " | Count:" + count);
        return KeyValue.pair(turbineId, stats);
    }).to("turbine-stats", Produced.with(Serdes.Integer(), statsSerde));
    return builder.build();
}
Also used : StreamsBuilder(org.apache.kafka.streams.StreamsBuilder) Produces(javax.enterprise.inject.Produces) Produced(org.apache.kafka.streams.kstream.Produced) Consumed(org.apache.kafka.streams.kstream.Consumed) BufferConfig.unbounded(org.apache.kafka.streams.kstream.Suppressed.BufferConfig.unbounded) KeyValue(org.apache.kafka.streams.KeyValue) WindTurbine(com.redhat.energy.records.WindTurbine) Suppressed(org.apache.kafka.streams.kstream.Suppressed) KStream(org.apache.kafka.streams.kstream.KStream) Bytes(org.apache.kafka.common.utils.Bytes) ObjectMapperSerde(io.quarkus.kafka.client.serialization.ObjectMapperSerde) WindTurbineStats(com.redhat.energy.records.WindTurbineStats) TimeWindows(org.apache.kafka.streams.kstream.TimeWindows) Duration(java.time.Duration) KeyValueStore(org.apache.kafka.streams.state.KeyValueStore) Materialized(org.apache.kafka.streams.kstream.Materialized) Serdes(org.apache.kafka.common.serialization.Serdes) ApplicationScoped(javax.enterprise.context.ApplicationScoped) MWattsMeasurement(com.redhat.energy.records.MWattsMeasurement) Topology(org.apache.kafka.streams.Topology) StreamsBuilder(org.apache.kafka.streams.StreamsBuilder) Bytes(org.apache.kafka.common.utils.Bytes) MWattsMeasurement(com.redhat.energy.records.MWattsMeasurement) WindTurbineStats(com.redhat.energy.records.WindTurbineStats) ObjectMapperSerde(io.quarkus.kafka.client.serialization.ObjectMapperSerde) WindTurbine(com.redhat.energy.records.WindTurbine) Produces(javax.enterprise.inject.Produces)

Example 5 with ObjectMapperSerde

use of io.quarkus.kafka.client.serialization.ObjectMapperSerde in project AD482-apps by RedHatTraining.

the class StreamTopologyBuilderTest method setup.

@BeforeEach
public void setup() {
    // TODO: create the test driver
    StreamTopologyBuilder builder = new StreamTopologyBuilder();
    testDriver = new TopologyTestDriver(builder.buildTopology());
    // TODO: Create test topics
    wattsStream = testDriver.createInputTopic("turbine-generated-watts", new IntegerSerializer(), new IntegerSerializer());
    ObjectMapperSerde<MWattsMeasurement> measurementSerde = new ObjectMapperSerde<>(MWattsMeasurement.class);
    measurementsStream = testDriver.createOutputTopic("turbine-generated-mwatts", new IntegerDeserializer(), measurementSerde.deserializer());
}
Also used : IntegerDeserializer(org.apache.kafka.common.serialization.IntegerDeserializer) MWattsMeasurement(com.redhat.energy.records.MWattsMeasurement) TopologyTestDriver(org.apache.kafka.streams.TopologyTestDriver) ObjectMapperSerde(io.quarkus.kafka.client.serialization.ObjectMapperSerde) IntegerSerializer(org.apache.kafka.common.serialization.IntegerSerializer) BeforeEach(org.junit.jupiter.api.BeforeEach)

Aggregations

ObjectMapperSerde (io.quarkus.kafka.client.serialization.ObjectMapperSerde)15 StreamsBuilder (org.apache.kafka.streams.StreamsBuilder)13 ApplicationScoped (javax.enterprise.context.ApplicationScoped)8 Serdes (org.apache.kafka.common.serialization.Serdes)8 KafkaStreams (org.apache.kafka.streams.KafkaStreams)8 Produces (javax.enterprise.inject.Produces)7 KeyValue (org.apache.kafka.streams.KeyValue)7 Consumed (org.apache.kafka.streams.kstream.Consumed)7 Produced (org.apache.kafka.streams.kstream.Produced)7 ShutdownEvent (io.quarkus.runtime.ShutdownEvent)5 StartupEvent (io.quarkus.runtime.StartupEvent)5 Observes (javax.enterprise.event.Observes)5 KStream (org.apache.kafka.streams.kstream.KStream)5 Logger (org.jboss.logging.Logger)5 Bytes (org.apache.kafka.common.utils.Bytes)3 Topology (org.apache.kafka.streams.Topology)3 WindTurbineProfitMarginWasCalculated (com.redhat.energy.profit.event.WindTurbineProfitMarginWasCalculated)2 MWattsMeasurement (com.redhat.energy.records.MWattsMeasurement)2 TemperatureWasMeasuredInCelsius (com.redhat.monitor.event.TemperatureWasMeasuredInCelsius)2 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1