Search in sources :

Example 6 with JetEvent

use of com.hazelcast.jet.impl.JetEvent in project hazelcast by hazelcast.

the class WindowGroupTransform method addSlidingWindowTwoStage.

// ---------       ---------
// | source0 | ... | sourceN |
// ---------       ---------
// |               |
// local           local
// partitioned     partitioned
// v               v
// --------------------
// | accumulateByFrameP |
// --------------------
// |
// distributed
// partitioned
// v
// -------------------------
// | combineToSlidingWindowP |
// -------------------------
private void addSlidingWindowTwoStage(Planner p, SlidingWindowDefinition wDef) {
    SlidingWindowPolicy winPolicy = slidingWinPolicy(wDef.windowSize(), wDef.slideBy());
    Vertex v1 = p.dag.newVertex(name() + FIRST_STAGE_VERTEX_NAME_SUFFIX, accumulateByFrameP(keyFns, nCopies(keyFns.size(), (ToLongFunctionEx<JetEvent<?>>) JetEvent::timestamp), TimestampKind.EVENT, winPolicy, aggrOp));
    v1.localParallelism(determinedLocalParallelism());
    PlannerVertex pv2 = p.addVertex(this, name(), determinedLocalParallelism(), combineToSlidingWindowP(winPolicy, aggrOp, jetEventOfKeyedWindowResultFn()));
    p.addEdges(this, v1, (e, ord) -> e.partitioned(keyFns.get(ord), HASH_CODE));
    p.dag.edge(between(v1, pv2.v).distributed().partitioned(entryKey()));
}
Also used : PlannerVertex(com.hazelcast.jet.impl.pipeline.Planner.PlannerVertex) Vertex(com.hazelcast.jet.core.Vertex) PlannerVertex(com.hazelcast.jet.impl.pipeline.Planner.PlannerVertex) SlidingWindowPolicy(com.hazelcast.jet.core.SlidingWindowPolicy) JetEvent(com.hazelcast.jet.impl.JetEvent)

Example 7 with JetEvent

use of com.hazelcast.jet.impl.JetEvent in project hazelcast by hazelcast.

the class Util method logLateEvent.

/**
 * Logs a late event that was dropped.
 */
public static void logLateEvent(ILogger logger, long currentWm, @Nonnull Object item) {
    if (!logger.isInfoEnabled()) {
        return;
    }
    if (item instanceof JetEvent) {
        JetEvent event = (JetEvent) item;
        logger.info(format("Event dropped, late by %d ms. currentWatermark=%s, eventTime=%s, event=%s", currentWm - event.timestamp(), toLocalTime(currentWm), toLocalTime(event.timestamp()), event.payload()));
    } else {
        logger.info(format("Late event dropped. currentWatermark=%s, event=%s", new Watermark(currentWm), item));
    }
}
Also used : JetEvent(com.hazelcast.jet.impl.JetEvent) Watermark(com.hazelcast.jet.core.Watermark)

Aggregations

JetEvent (com.hazelcast.jet.impl.JetEvent)7 PlannerVertex (com.hazelcast.jet.impl.pipeline.Planner.PlannerVertex)6 SlidingWindowPolicy (com.hazelcast.jet.core.SlidingWindowPolicy)2 Vertex (com.hazelcast.jet.core.Vertex)2 Edge (com.hazelcast.jet.core.Edge)1 Watermark (com.hazelcast.jet.core.Watermark)1