Examples with Pipeline com.hazelcast.jet.pipeline.Pipeline used on opensource projects

Example 21 with Pipeline

use of com.hazelcast.jet.pipeline.Pipeline in project hazelcast-jet by hazelcast.

the class WriteFilePTest method smokeTest_bigFile.

@Test
public void smokeTest_bigFile() throws Exception {
    // Given
    Pipeline p = buildPipeline(null, null, false);
    addItemsToList(0, 100_000);
    // When
    instance.newJob(p).join();
    // Then
    checkFileContents(StandardCharsets.UTF_8, 100_000);
}

Example 22 with Pipeline

use of com.hazelcast.jet.pipeline.Pipeline in project hazelcast-jet-reference-manual by hazelcast.

the class BuildComputation method s14.

static void s14() {
    // tag::s14[]
    Pipeline p = Pipeline.create();
    p.<Tweet>drawFrom(Sources.mapJournal("tweets", mapPutEvents(), mapEventNewValue(), START_FROM_CURRENT)).flatMap(tweet -> traverseArray(tweet.text().toLowerCase().split("\\W+")).map(word -> new TweetWord(tweet.timestamp(), word))).filter(tweetWord -> !tweetWord.word().isEmpty()).addTimestamps(TweetWord::timestamp, TimeUnit.SECONDS.toMillis(5)).window(sliding(MINUTES.toMillis(1), SECONDS.toMillis(1))).groupingKey(TweetWord::word).aggregate(counting()).drainTo(Sinks.list("result"));
// end::s14[]
}

Example 23 with Pipeline

use of com.hazelcast.jet.pipeline.Pipeline in project hazelcast-jet-reference-manual by hazelcast.

the class BuildComputation method s5.

static void s5() {
    // tag::s5[]
    Pipeline p = Pipeline.create();
    p.drawFrom(Sources.<String>list("text")).flatMap(line -> traverseArray(line.toLowerCase().split("\\W+"))).filter(word -> !word.isEmpty()).aggregate(counting()).drainTo(Sinks.list("result"));
// end::s5[]
}

Example 24 with Pipeline

use of com.hazelcast.jet.pipeline.Pipeline in project hazelcast-jet-reference-manual by hazelcast.

the class BuildComputation method wordCountTwoSources.

// tag::s7[]
static void wordCountTwoSources() {
    Pipeline p = Pipeline.create();
    BatchStage<String> src1 = p.drawFrom(Sources.list("src1"));
    BatchStage<String> src2 = p.drawFrom(Sources.list("src2"));
    StageWithGrouping<String, String> grouped1 = groupByWord(src1);
    StageWithGrouping<String, String> grouped2 = groupByWord(src2);
    grouped1.aggregate2(grouped2, counting2()).drainTo(Sinks.map("result"));
}

Example 25 with Pipeline

use of com.hazelcast.jet.pipeline.Pipeline in project hazelcast-jet-reference-manual by hazelcast.

the class BuildComputation method s4.

static void s4() {
    // tag::s4[]
    Pipeline p = Pipeline.create();
    p.drawFrom(Sources.list("text")).aggregate(counting()).drainTo(Sinks.list("result"));
// end::s4[]
}