Search in sources :

Example 1 with SnapshotValidationRecord

use of com.hazelcast.jet.impl.SnapshotValidationRecord in project hazelcast by hazelcast.

the class SnapshotLargeChunk_IntegrationTest method test_snapshotRestoreLargeChunk.

@Test
public void test_snapshotRestoreLargeChunk() {
    HazelcastInstance instance = createHazelcastInstance();
    DAG dag = new DAG();
    dag.newVertex("src", LargeStateP::new).localParallelism(1);
    Job job = instance.getJet().newJob(dag, new JobConfig().setProcessingGuarantee(ProcessingGuarantee.EXACTLY_ONCE).setSnapshotIntervalMillis(DAYS.toMillis(1)));
    assertJobStatusEventually(job, RUNNING);
    job.restart();
    assertJobStatusEventually(job, RUNNING);
    // assert that the snapshot exists and that the chunk was large enough
    IMap<Object, Object> map = instance.getMap(JobRepository.snapshotDataMapName(job.getId(), 0));
    SnapshotValidationRecord validationRec = (SnapshotValidationRecord) map.get(SnapshotValidationRecord.KEY);
    assertEquals(1, validationRec.numChunks());
    IntSummaryStatistics stats = map.values().stream().filter(v -> v instanceof byte[]).collect(summarizingInt(v -> ((byte[]) v).length));
    assertTrue("min=" + stats.getMin(), stats.getMin() > AsyncSnapshotWriterImpl.DEFAULT_CHUNK_SIZE);
}
Also used : IntStream(java.util.stream.IntStream) Bits(com.hazelcast.internal.nio.Bits) AbstractProcessor(com.hazelcast.jet.core.AbstractProcessor) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) RunWith(org.junit.runner.RunWith) HazelcastSerialClassRunner(com.hazelcast.test.HazelcastSerialClassRunner) DAYS(java.util.concurrent.TimeUnit.DAYS) Assert.assertArrayEquals(org.junit.Assert.assertArrayEquals) Collectors.summarizingInt(java.util.stream.Collectors.summarizingInt) DAG(com.hazelcast.jet.core.DAG) Nonnull(javax.annotation.Nonnull) Job(com.hazelcast.jet.Job) JobRepository(com.hazelcast.jet.impl.JobRepository) HazelcastInstance(com.hazelcast.core.HazelcastInstance) JetTestSupport(com.hazelcast.jet.core.JetTestSupport) JobConfig(com.hazelcast.jet.config.JobConfig) AsyncSnapshotWriterImpl(com.hazelcast.jet.impl.util.AsyncSnapshotWriterImpl) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) Category(org.junit.experimental.categories.Category) IntSummaryStatistics(java.util.IntSummaryStatistics) RUNNING(com.hazelcast.jet.core.JobStatus.RUNNING) SnapshotValidationRecord(com.hazelcast.jet.impl.SnapshotValidationRecord) ProcessingGuarantee(com.hazelcast.jet.config.ProcessingGuarantee) Assert.assertEquals(org.junit.Assert.assertEquals) IMap(com.hazelcast.map.IMap) SnapshotValidationRecord(com.hazelcast.jet.impl.SnapshotValidationRecord) HazelcastInstance(com.hazelcast.core.HazelcastInstance) DAG(com.hazelcast.jet.core.DAG) Job(com.hazelcast.jet.Job) IntSummaryStatistics(java.util.IntSummaryStatistics) JobConfig(com.hazelcast.jet.config.JobConfig) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Aggregations

HazelcastInstance (com.hazelcast.core.HazelcastInstance)1 Bits (com.hazelcast.internal.nio.Bits)1 Job (com.hazelcast.jet.Job)1 JobConfig (com.hazelcast.jet.config.JobConfig)1 ProcessingGuarantee (com.hazelcast.jet.config.ProcessingGuarantee)1 AbstractProcessor (com.hazelcast.jet.core.AbstractProcessor)1 DAG (com.hazelcast.jet.core.DAG)1 JetTestSupport (com.hazelcast.jet.core.JetTestSupport)1 RUNNING (com.hazelcast.jet.core.JobStatus.RUNNING)1 JobRepository (com.hazelcast.jet.impl.JobRepository)1 SnapshotValidationRecord (com.hazelcast.jet.impl.SnapshotValidationRecord)1 AsyncSnapshotWriterImpl (com.hazelcast.jet.impl.util.AsyncSnapshotWriterImpl)1 IMap (com.hazelcast.map.IMap)1 HazelcastSerialClassRunner (com.hazelcast.test.HazelcastSerialClassRunner)1 ParallelJVMTest (com.hazelcast.test.annotation.ParallelJVMTest)1 QuickTest (com.hazelcast.test.annotation.QuickTest)1 IntSummaryStatistics (java.util.IntSummaryStatistics)1 DAYS (java.util.concurrent.TimeUnit.DAYS)1 Collectors.summarizingInt (java.util.stream.Collectors.summarizingInt)1 IntStream (java.util.stream.IntStream)1