Search in sources :

Example 1 with TweetWord

use of datamodel.TweetWord in project hazelcast-jet-reference-manual by hazelcast.

the class BuildComputation method s14.

static void s14() {
    // tag::s14[]
    Pipeline p = Pipeline.create();
    p.<Tweet>drawFrom(Sources.mapJournal("tweets", mapPutEvents(), mapEventNewValue(), START_FROM_CURRENT)).flatMap(tweet -> traverseArray(tweet.text().toLowerCase().split("\\W+")).map(word -> new TweetWord(tweet.timestamp(), word))).filter(tweetWord -> !tweetWord.word().isEmpty()).addTimestamps(TweetWord::timestamp, TimeUnit.SECONDS.toMillis(5)).window(sliding(MINUTES.toMillis(1), SECONDS.toMillis(1))).groupingKey(TweetWord::word).aggregate(counting()).drainTo(Sinks.list("result"));
// end::s14[]
}
Also used : AggregateOperations.counting(com.hazelcast.jet.aggregate.AggregateOperations.counting) Delivery(datamodel.Delivery) MINUTES(java.util.concurrent.TimeUnit.MINUTES) Tweet(datamodel.Tweet) JoinClause.joinMapEntries(com.hazelcast.jet.pipeline.JoinClause.joinMapEntries) DistributedFunctions.wholeItem(com.hazelcast.jet.function.DistributedFunctions.wholeItem) BatchStage(com.hazelcast.jet.pipeline.BatchStage) Util.mapEventNewValue(com.hazelcast.jet.Util.mapEventNewValue) PageVisit(datamodel.PageVisit) AggregateOperation(com.hazelcast.jet.aggregate.AggregateOperation) START_FROM_CURRENT(com.hazelcast.jet.pipeline.JournalInitialPosition.START_FROM_CURRENT) DAG(com.hazelcast.jet.core.DAG) StreamHashJoinBuilder(com.hazelcast.jet.pipeline.StreamHashJoinBuilder) Tuple2(com.hazelcast.jet.datamodel.Tuple2) AddToCart(datamodel.AddToCart) Tuple3(com.hazelcast.jet.datamodel.Tuple3) StreamStage(com.hazelcast.jet.pipeline.StreamStage) StageWithGrouping(com.hazelcast.jet.pipeline.StageWithGrouping) Pipeline(com.hazelcast.jet.pipeline.Pipeline) Market(datamodel.Market) AggregateOperation2(com.hazelcast.jet.aggregate.AggregateOperation2) Tag(com.hazelcast.jet.datamodel.Tag) Sinks(com.hazelcast.jet.pipeline.Sinks) Payment(datamodel.Payment) Sources(com.hazelcast.jet.pipeline.Sources) Product(datamodel.Product) TimeUnit(java.util.concurrent.TimeUnit) BagsByTag(com.hazelcast.jet.datamodel.BagsByTag) WindowDefinition.sliding(com.hazelcast.jet.pipeline.WindowDefinition.sliding) LongAccumulator(com.hazelcast.jet.accumulator.LongAccumulator) Traversers.traverseArray(com.hazelcast.jet.Traversers.traverseArray) ItemsByTag(com.hazelcast.jet.datamodel.ItemsByTag) Entry(java.util.Map.Entry) Trade(datamodel.Trade) TweetWord(datamodel.TweetWord) Util.mapPutEvents(com.hazelcast.jet.Util.mapPutEvents) AggregateOperations.toBagsByTag(com.hazelcast.jet.aggregate.AggregateOperations.toBagsByTag) GroupAggregateBuilder(com.hazelcast.jet.pipeline.GroupAggregateBuilder) SECONDS(java.util.concurrent.TimeUnit.SECONDS) Broker(datamodel.Broker) TweetWord(datamodel.TweetWord) Pipeline(com.hazelcast.jet.pipeline.Pipeline)

Aggregations

Traversers.traverseArray (com.hazelcast.jet.Traversers.traverseArray)1 Util.mapEventNewValue (com.hazelcast.jet.Util.mapEventNewValue)1 Util.mapPutEvents (com.hazelcast.jet.Util.mapPutEvents)1 LongAccumulator (com.hazelcast.jet.accumulator.LongAccumulator)1 AggregateOperation (com.hazelcast.jet.aggregate.AggregateOperation)1 AggregateOperation2 (com.hazelcast.jet.aggregate.AggregateOperation2)1 AggregateOperations.counting (com.hazelcast.jet.aggregate.AggregateOperations.counting)1 AggregateOperations.toBagsByTag (com.hazelcast.jet.aggregate.AggregateOperations.toBagsByTag)1 DAG (com.hazelcast.jet.core.DAG)1 BagsByTag (com.hazelcast.jet.datamodel.BagsByTag)1 ItemsByTag (com.hazelcast.jet.datamodel.ItemsByTag)1 Tag (com.hazelcast.jet.datamodel.Tag)1 Tuple2 (com.hazelcast.jet.datamodel.Tuple2)1 Tuple3 (com.hazelcast.jet.datamodel.Tuple3)1 DistributedFunctions.wholeItem (com.hazelcast.jet.function.DistributedFunctions.wholeItem)1 BatchStage (com.hazelcast.jet.pipeline.BatchStage)1 GroupAggregateBuilder (com.hazelcast.jet.pipeline.GroupAggregateBuilder)1 JoinClause.joinMapEntries (com.hazelcast.jet.pipeline.JoinClause.joinMapEntries)1 START_FROM_CURRENT (com.hazelcast.jet.pipeline.JournalInitialPosition.START_FROM_CURRENT)1 Pipeline (com.hazelcast.jet.pipeline.Pipeline)1