Examples with LongLongAccumulator com.hazelcast.jet.accumulator.LongLongAccumulator used on opensource projects

Example 1 with LongLongAccumulator

use of com.hazelcast.jet.accumulator.LongLongAccumulator in project hazelcast by hazelcast.

the class StoreSnapshotTasklet method stateMachineStep.

private void stateMachineStep() {
    switch(state) {
        case DRAIN:
            progTracker.notDone();
            if (pendingEntry != null) {
                if (!ssWriter.offer(pendingEntry)) {
                    return;
                }
                progTracker.madeProgress();
            }
            pendingEntry = null;
            ProgressState result = inboundEdgeStream.drainTo(addToInboxFunction);
            if (result.isDone()) {
                assert ssWriter.isEmpty() : "input is done, but we had some entries and not the barrier";
                snapshotContext.storeSnapshotTaskletDone(pendingSnapshotId - 1, isHigherPrioritySource);
                state = DONE;
                progTracker.reset();
            }
            progTracker.madeProgress(result.isMadeProgress());
            if (hasReachedBarrier) {
                state = FLUSH;
                stateMachineStep();
            }
            break;
        case FLUSH:
            progTracker.notDone();
            if (ssWriter.flushAndResetMap()) {
                progTracker.madeProgress();
                state = REACHED_BARRIER;
            }
            break;
        case REACHED_BARRIER:
            if (ssWriter.hasPendingAsyncOps()) {
                progTracker.notDone();
                return;
            }
            // check for writing error
            Throwable error = ssWriter.getError();
            if (error != null) {
                logger.severe("Error writing to snapshot map", error);
                snapshotContext.reportError(error);
            }
            progTracker.madeProgress();
            long bytes = ssWriter.getTotalPayloadBytes();
            long keys = ssWriter.getTotalKeys();
            long chunks = ssWriter.getTotalChunks();
            snapshotContext.phase1DoneForTasklet(bytes, keys, chunks);
            metrics.set(new LongLongAccumulator(bytes, keys));
            ssWriter.resetStats();
            pendingSnapshotId++;
            hasReachedBarrier = false;
            state = DRAIN;
            progTracker.notDone();
            break;
        default:
            // note State.DONE also goes here
            throw new JetException("Unexpected state: " + state);
    }
}

Example 2 with LongLongAccumulator

use of com.hazelcast.jet.accumulator.LongLongAccumulator in project hazelcast by hazelcast.

the class AggregateOperationsTest method when_averagingLong_noInput_then_NaN.

@Test
public void when_averagingLong_noInput_then_NaN() {
    // Given
    AggregateOperation1<Long, LongLongAccumulator, Double> aggrOp = averagingLong(Long::longValue);
    LongLongAccumulator acc = aggrOp.createFn().get();
    // When
    double result = aggrOp.finishFn().apply(acc);
    // Then
    assertEquals(Double.NaN, result, 0.0);
}

Example 3 with LongLongAccumulator

use of com.hazelcast.jet.accumulator.LongLongAccumulator in project hazelcast-jet by hazelcast.

the class AggregateOperationsTest method when_averagingLongOverflow_thenException.

@Test
public void when_averagingLongOverflow_thenException() {
    // Given
    AggregateOperation1<Long, LongLongAccumulator, Double> aggrOp = averagingLong(Long::longValue);
    LongLongAccumulator acc = new LongLongAccumulator(Long.MAX_VALUE, 0L);
    // When and Then
    exception.expect(ArithmeticException.class);
    aggrOp.accumulateFn().accept(acc, 0L);
}

Example 4 with LongLongAccumulator

use of com.hazelcast.jet.accumulator.LongLongAccumulator in project hazelcast by hazelcast.

the class PipelineStreamTestSupport method streamStageFromList.

/**
 * The returned stream stage emits the items from the supplied list of
 * integers. It uses the integer as both timestamp and data.
 * <p>
 * The stage emits (1e9 / {@value SOURCE_EVENT_PERIOD_NANOS}) items per
 * second. After it exhausts the input, it remains idle forever.
 */
StreamStage<Integer> streamStageFromList(List<Integer> input, long earlyResultsPeriod) {
    Preconditions.checkPositive(earlyResultsPeriod, "earlyResultsPeriod must greater than zero");
    StreamSource<Integer> source = SourceBuilder.timestampedStream("sequence", x -> new LongLongAccumulator(System.nanoTime(), 0)).<Integer>fillBufferFn((deadline_emittedCount, buf) -> {
        int emittedCount = (int) deadline_emittedCount.get2();
        if (emittedCount == input.size() || System.nanoTime() < deadline_emittedCount.get1()) {
            return;
        }
        int item = input.get(emittedCount);
        buf.add(item, (long) item);
        deadline_emittedCount.add1(SOURCE_EVENT_PERIOD_NANOS);
        deadline_emittedCount.add2(1);
    }).build();
    return p.readFrom(source).withNativeTimestamps(2 * itemCount);
}

Example 5 with LongLongAccumulator

use of com.hazelcast.jet.accumulator.LongLongAccumulator in project hazelcast by hazelcast.

the class AggregateOperationsTest method when_averagingLong_sumTooLarge_then_exception.

@Test
public void when_averagingLong_sumTooLarge_then_exception() {
    // Given
    AggregateOperation1<Long, LongLongAccumulator, Double> aggrOp = averagingLong(Long::longValue);
    LongLongAccumulator acc = aggrOp.createFn().get();
    // Then
    exception.expect(ArithmeticException.class);
    // When
    aggrOp.accumulateFn().accept(acc, Long.MAX_VALUE);
    aggrOp.accumulateFn().accept(acc, 1L);
}