Examples with Pipeline com.hazelcast.jet.pipeline.Pipeline used on opensource projects

Example 36 with Pipeline

use of com.hazelcast.jet.pipeline.Pipeline in project hazelcast-jet-reference-manual by hazelcast.

the class FileAndSocket method s4.

static void s4() {
    // tag::s4[]
    Pipeline p = Pipeline.create();
    p.drawFrom(Sources.socket("localhost", 8080, StandardCharsets.UTF_8)).drainTo(Sinks.logger());
// end::s4[]
}

Example 37 with Pipeline

use of com.hazelcast.jet.pipeline.Pipeline in project hazelcast-jet-reference-manual by hazelcast.

the class FileAndSocket method s3.

static void s3() {
    // tag::s3[]
    Pipeline p = Pipeline.create();
    p.drawFrom(Sources.list("inputList")).drainTo(Sinks.files("/home/jet/output"));
// end::s3[]
}

Example 38 with Pipeline

use of com.hazelcast.jet.pipeline.Pipeline in project hazelcast-jet-reference-manual by hazelcast.

the class HdfsAndKafka method s3.

static void s3() {
    // tag::s3[]
    Properties props = new Properties();
    props.setProperty("bootstrap.servers", "localhost:9092");
    props.setProperty("key.serializer", StringSerializer.class.getCanonicalName());
    props.setProperty("key.deserializer", StringDeserializer.class.getCanonicalName());
    props.setProperty("value.serializer", IntegerSerializer.class.getCanonicalName());
    props.setProperty("value.deserializer", IntegerDeserializer.class.getCanonicalName());
    props.setProperty("auto.offset.reset", "earliest");
    Pipeline p = Pipeline.create();
    p.drawFrom(KafkaSources.kafka(props, "t1", "t2")).drainTo(KafkaSinks.kafka(props, "t3"));
// end::s3[]
}

Example 39 with Pipeline

use of com.hazelcast.jet.pipeline.Pipeline in project hazelcast-jet-reference-manual by hazelcast.

the class HdfsAndKafka method s2.

static void s2() {
    JobConf jobConfig = new JobConf();
    // tag::s2[]
    Pipeline p = Pipeline.create();
    p.drawFrom(HdfsSources.hdfs(jobConfig, (k, v) -> v.toString())).flatMap(line -> traverseArray(line.toLowerCase().split("\\W+")).filter(w -> !w.isEmpty())).groupingKey(wholeItem()).aggregate(counting()).drainTo(HdfsSinks.hdfs(jobConfig));
// end::s2[]
}

Example 40 with Pipeline

use of com.hazelcast.jet.pipeline.Pipeline in project hazelcast-jet-reference-manual by hazelcast.

the class ImdgConnectors method s13.

static void s13() {
    ClientConfig someClientConfig = new ClientConfig();
    // tag::s13[]
    Pipeline p = Pipeline.create();
    StreamStage<Entry<String, Long>> fromRemoteMap = p.drawFrom(Sources.<String, Long>remoteMapJournal("inputMap", someClientConfig, START_FROM_CURRENT));
    StreamStage<Entry<String, Long>> fromRemoteCache = p.drawFrom(Sources.<String, Long>remoteCacheJournal("inputCache", someClientConfig, START_FROM_CURRENT));
// end::s13[]
}