Search in sources :

Example 31 with JetInstance

use of com.hazelcast.jet.JetInstance in project hazelcast-jet-reference-manual by hazelcast.

the class JUS method s3.

static void s3() {
    JetInstance jet = Jet.newJetInstance();
    // tag::s3[]
    // <1>
    IMapJet<String, String> map = jet.getMap("large_map");
    Map<String, String> result = DistributedStream.fromMap(map).map(e -> e.getKey() + e.getValue()).collect(// <2>
    DistributedCollectors.toMap(v -> v, v -> v));
// end::s3[]
}
Also used : Arrays(java.util.Arrays) JetInstance(com.hazelcast.jet.JetInstance) IMapJet(com.hazelcast.jet.IMapJet) Collectors(java.util.stream.Collectors) DistributedFunctions.wholeItem(com.hazelcast.jet.function.DistributedFunctions.wholeItem) Sources(com.hazelcast.jet.pipeline.Sources) IMap(com.hazelcast.core.IMap) DistributedCollectors(com.hazelcast.jet.stream.DistributedCollectors) Stream(java.util.stream.Stream) Map(java.util.Map) IList(com.hazelcast.core.IList) DistributedStream(com.hazelcast.jet.stream.DistributedStream) Jet(com.hazelcast.jet.Jet) DistributedCollectors.toMap(com.hazelcast.jet.stream.DistributedCollectors.toMap) JetInstance(com.hazelcast.jet.JetInstance)

Example 32 with JetInstance

use of com.hazelcast.jet.JetInstance in project hazelcast-jet-reference-manual by hazelcast.

the class JUS method s4.

static void s4() {
    JetInstance jet = Jet.newJetInstance();
    // tag::s4[]
    IMap<String, Long> counts = DistributedStream.<String>fromList(jet.getList("text")).flatMap(line -> Stream.of(line.toLowerCase().split("\\W+"))).filter(word -> !word.isEmpty()).collect(DistributedCollectors.groupingByToIMap("counts", wholeItem(), DistributedCollectors.counting()));
// end::s4[]
}
Also used : Arrays(java.util.Arrays) JetInstance(com.hazelcast.jet.JetInstance) IMapJet(com.hazelcast.jet.IMapJet) Collectors(java.util.stream.Collectors) DistributedFunctions.wholeItem(com.hazelcast.jet.function.DistributedFunctions.wholeItem) Sources(com.hazelcast.jet.pipeline.Sources) IMap(com.hazelcast.core.IMap) DistributedCollectors(com.hazelcast.jet.stream.DistributedCollectors) Stream(java.util.stream.Stream) Map(java.util.Map) IList(com.hazelcast.core.IList) DistributedStream(com.hazelcast.jet.stream.DistributedStream) Jet(com.hazelcast.jet.Jet) DistributedCollectors.toMap(com.hazelcast.jet.stream.DistributedCollectors.toMap) JetInstance(com.hazelcast.jet.JetInstance)

Example 33 with JetInstance

use of com.hazelcast.jet.JetInstance in project hazelcast-jet-reference-manual by hazelcast.

the class JUS method s1.

static void s1() {
    JetInstance jet = Jet.newJetInstance();
    try {
        // tag::s1[]
        IMapJet<String, Integer> map = jet.getMap("latitudes");
        map.put("London", 51);
        map.put("Paris", 48);
        map.put("NYC", 40);
        map.put("Sydney", -34);
        map.put("Sao Paulo", -23);
        map.put("Jakarta", -6);
        DistributedStream.fromMap(map).filter(e -> e.getValue() < 0).forEach(System.out::println);
    // end::s1[]
    } finally {
        Jet.shutdownAll();
    }
}
Also used : Arrays(java.util.Arrays) JetInstance(com.hazelcast.jet.JetInstance) IMapJet(com.hazelcast.jet.IMapJet) Collectors(java.util.stream.Collectors) DistributedFunctions.wholeItem(com.hazelcast.jet.function.DistributedFunctions.wholeItem) Sources(com.hazelcast.jet.pipeline.Sources) IMap(com.hazelcast.core.IMap) DistributedCollectors(com.hazelcast.jet.stream.DistributedCollectors) Stream(java.util.stream.Stream) Map(java.util.Map) IList(com.hazelcast.core.IList) DistributedStream(com.hazelcast.jet.stream.DistributedStream) Jet(com.hazelcast.jet.Jet) DistributedCollectors.toMap(com.hazelcast.jet.stream.DistributedCollectors.toMap) JetInstance(com.hazelcast.jet.JetInstance)

Example 34 with JetInstance

use of com.hazelcast.jet.JetInstance in project hazelcast-jet-reference-manual by hazelcast.

the class StartTwoInstances method s1.

static void s1() {
    // tag::s1[]
    JetInstance jet = Jet.newJetInstance();
    Jet.newJetInstance();
// end::s1[]
}
Also used : JetInstance(com.hazelcast.jet.JetInstance)

Example 35 with JetInstance

use of com.hazelcast.jet.JetInstance in project hazelcast-jet-reference-manual by hazelcast.

the class S8 method main.

// end::s1[]
static void main() {
    // tag::s2[]
    JetInstance jet = Jet.newJetInstance();
    int upperBound = 10;
    DAG dag = new DAG();
    Vertex generateNumbers = dag.newVertex("generate-numbers", () -> new GenerateNumbersP(upperBound));
    Vertex logInput = dag.newVertex("log-input", DiagnosticProcessors.writeLoggerP(i -> "Received number: " + i));
    dag.edge(Edge.between(generateNumbers, logInput));
    try {
        jet.newJob(dag).join();
    } finally {
        Jet.shutdownAll();
    }
// end::s2[]
}
Also used : IntStream(java.util.stream.IntStream) AbstractProcessor(com.hazelcast.jet.core.AbstractProcessor) Traverser(com.hazelcast.jet.Traverser) DiagnosticProcessors(com.hazelcast.jet.core.processor.DiagnosticProcessors) JetInstance(com.hazelcast.jet.JetInstance) IntStream.range(java.util.stream.IntStream.range) ProcessorMetaSupplier(com.hazelcast.jet.core.ProcessorMetaSupplier) Processor(com.hazelcast.jet.core.Processor) Address(com.hazelcast.nio.Address) HashMap(java.util.HashMap) Traversers(com.hazelcast.jet.Traversers) Function(java.util.function.Function) Collectors(java.util.stream.Collectors) Vertex(com.hazelcast.jet.core.Vertex) List(java.util.List) Collectors.toList(java.util.stream.Collectors.toList) Map(java.util.Map) Context(com.hazelcast.jet.core.Processor.Context) Jet(com.hazelcast.jet.Jet) DAG(com.hazelcast.jet.core.DAG) Edge(com.hazelcast.jet.core.Edge) Nonnull(javax.annotation.Nonnull) ProcessorSupplier(com.hazelcast.jet.core.ProcessorSupplier) Vertex(com.hazelcast.jet.core.Vertex) JetInstance(com.hazelcast.jet.JetInstance) DAG(com.hazelcast.jet.core.DAG)

Aggregations

JetInstance (com.hazelcast.jet.JetInstance)84 Test (org.junit.Test)45 Job (com.hazelcast.jet.Job)32 JetConfig (com.hazelcast.jet.config.JetConfig)22 MockPS (com.hazelcast.jet.core.TestProcessors.MockPS)14 Pipeline (com.hazelcast.jet.pipeline.Pipeline)14 JobConfig (com.hazelcast.jet.config.JobConfig)13 StuckProcessor (com.hazelcast.jet.core.TestProcessors.StuckProcessor)12 DAG (com.hazelcast.jet.core.DAG)10 ExpectedException (org.junit.rules.ExpectedException)10 Jet (com.hazelcast.jet.Jet)9 Assert.assertEquals (org.junit.Assert.assertEquals)9 JetService (com.hazelcast.jet.impl.JetService)8 Map (java.util.Map)8 CountDownLatch (java.util.concurrent.CountDownLatch)8 JobRepository (com.hazelcast.jet.impl.JobRepository)7 Sources (com.hazelcast.jet.pipeline.Sources)7 CancellationException (java.util.concurrent.CancellationException)7 Assert.assertNotNull (org.junit.Assert.assertNotNull)7 Before (org.junit.Before)7