Search in sources :

Example 1 with TimedDecimal

use of de.invesdwin.util.math.decimal.TimedDecimal in project invesdwin-context-persistence by subes.

the class RecordLogDirectoryPerformanceTest method testRecordLogDirectoryPerformance.

@Test
public void testRecordLogDirectoryPerformance() throws IOException, InterruptedException {
    final File directory = new File(ContextProperties.getCacheDirectory(), RecordLogDirectoryPerformanceTest.class.getSimpleName());
    Files.deleteNative(directory);
    Files.forceMkdir(directory);
    final File file = new File(directory, "testRecordLogDirectoryPerformance");
    final LoopInterruptedCheck loopCheck = new LoopInterruptedCheck(Duration.ONE_SECOND);
    final Instant writesStart = new Instant();
    try (RecordLogDirectory.Writer<TimedDecimal> writer = RecordLogDirectory.Writer.create(file, IndeedSerializer.valueOf(TimedDecimalSerde.GET), new SnappyCodec(), Integer.MAX_VALUE)) {
        int i = 0;
        for (final FDate date : newValues()) {
            writer.append(new TimedDecimal(date, date.millisValue()));
            i++;
            if (i % FLUSH_INTERVAL == 0) {
                if (loopCheck.check()) {
                    printProgress("Writes", writesStart, i, VALUES);
                }
            // writer.roll();
            }
        }
        writer.roll();
        printProgress("WritesFinished", writesStart, VALUES, VALUES);
    }
    readIterator(file);
}
Also used : Instant(de.invesdwin.util.time.Instant) RecordLogDirectory(com.indeed.lsmtree.recordlog.RecordLogDirectory) LoopInterruptedCheck(de.invesdwin.util.concurrent.loop.LoopInterruptedCheck) File(java.io.File) RecordFile(com.indeed.lsmtree.recordlog.RecordFile) TimedDecimal(de.invesdwin.util.math.decimal.TimedDecimal) SnappyCodec(com.indeed.util.compress.SnappyCodec) FDate(de.invesdwin.util.time.date.FDate) Test(org.junit.jupiter.api.Test)

Example 2 with TimedDecimal

use of de.invesdwin.util.math.decimal.TimedDecimal in project invesdwin-context-persistence by subes.

the class RecordLogDirectoryPerformanceTest method readIterator.

private void readIterator(final File file) throws IOException, InterruptedException {
    final LoopInterruptedCheck loopCheck = new LoopInterruptedCheck();
    final Instant readsStart = new Instant();
    for (int reads = 1; reads <= READS; reads++) {
        FDate prevValue = null;
        int count = 0;
        try (RecordLogDirectory<TimedDecimal> reader = new RecordLogDirectory.Builder<TimedDecimal>(file, IndeedSerializer.valueOf(TimedDecimalSerde.GET), new SnappyCodec()).build()) {
            final RecordFile.Reader<TimedDecimal> iterator = reader.reader();
            while (iterator.next()) {
                final FDate value = iterator.get().getTime();
                if (prevValue != null) {
                    Assertions.checkTrue(prevValue.isBefore(value));
                }
                prevValue = value;
                count++;
            }
            Assertions.checkEquals(count, VALUES);
            if (loopCheck.check()) {
                printProgress("Reads", readsStart, VALUES * reads, VALUES * READS);
            }
        }
    }
    printProgress("ReadsFinished", readsStart, VALUES * READS, VALUES * READS);
}
Also used : RecordFile(com.indeed.lsmtree.recordlog.RecordFile) Instant(de.invesdwin.util.time.Instant) RecordLogDirectory(com.indeed.lsmtree.recordlog.RecordLogDirectory) LoopInterruptedCheck(de.invesdwin.util.concurrent.loop.LoopInterruptedCheck) TimedDecimal(de.invesdwin.util.math.decimal.TimedDecimal) SnappyCodec(com.indeed.util.compress.SnappyCodec) FDate(de.invesdwin.util.time.date.FDate)

Example 3 with TimedDecimal

use of de.invesdwin.util.math.decimal.TimedDecimal in project invesdwin-context-persistence by subes.

the class BasicRecordFilePerformanceTest method readIterator.

private void readIterator(final File file) throws IOException, InterruptedException {
    final LoopInterruptedCheck loopCheck = new LoopInterruptedCheck();
    final Instant readsStart = new Instant();
    for (int reads = 1; reads <= READS; reads++) {
        FDate prevValue = null;
        int count = 0;
        try (BasicRecordFile<TimedDecimal> reader = new BasicRecordFile<>(file, IndeedSerializer.valueOf(TimedDecimalSerde.GET))) {
            final RecordFile.Reader<TimedDecimal> iterator = reader.reader();
            while (iterator.next()) {
                final FDate value = iterator.get().getTime();
                if (prevValue != null) {
                    Assertions.checkTrue(prevValue.isBefore(value));
                }
                prevValue = value;
                count++;
            }
            Assertions.checkEquals(count, VALUES);
            if (loopCheck.check()) {
                printProgress("Reads", readsStart, VALUES * reads, VALUES * READS);
            }
        }
    }
    printProgress("ReadsFinished", readsStart, VALUES * READS, VALUES * READS);
}
Also used : BasicRecordFile(com.indeed.lsmtree.recordlog.BasicRecordFile) RecordFile(com.indeed.lsmtree.recordlog.RecordFile) BasicRecordFile(com.indeed.lsmtree.recordlog.BasicRecordFile) Instant(de.invesdwin.util.time.Instant) LoopInterruptedCheck(de.invesdwin.util.concurrent.loop.LoopInterruptedCheck) TimedDecimal(de.invesdwin.util.math.decimal.TimedDecimal) FDate(de.invesdwin.util.time.date.FDate)

Example 4 with TimedDecimal

use of de.invesdwin.util.math.decimal.TimedDecimal in project invesdwin-context-persistence by subes.

the class BlockCompressedRecordFilePerformanceTest method testBlockCompressedRecordFilePerformance.

@Test
public void testBlockCompressedRecordFilePerformance() throws IOException, InterruptedException {
    final File directory = new File(ContextProperties.getCacheDirectory(), BlockCompressedRecordFilePerformanceTest.class.getSimpleName());
    Files.deleteNative(directory);
    Files.forceMkdir(directory);
    final File file = new File(directory, "testBlockCompressedRecordFilePerformance");
    final LoopInterruptedCheck loopCheck = new LoopInterruptedCheck(Duration.ONE_SECOND);
    final Instant writesStart = new Instant();
    try (BlockCompressedRecordFile.Writer<TimedDecimal> writer = new BlockCompressedRecordFile.Writer.Builder<TimedDecimal>(file, IndeedSerializer.valueOf(TimedDecimalSerde.GET)).build()) {
        int i = 0;
        for (final FDate date : newValues()) {
            writer.append(new TimedDecimal(date, date.millisValue()));
            i++;
            if (i % FLUSH_INTERVAL == 0) {
                if (loopCheck.check()) {
                    printProgress("Writes", writesStart, i, VALUES);
                }
            }
        }
        printProgress("WritesFinished", writesStart, VALUES, VALUES);
    }
    readIterator(file);
}
Also used : Instant(de.invesdwin.util.time.Instant) LoopInterruptedCheck(de.invesdwin.util.concurrent.loop.LoopInterruptedCheck) File(java.io.File) BlockCompressedRecordFile(com.indeed.lsmtree.recordlog.BlockCompressedRecordFile) RecordFile(com.indeed.lsmtree.recordlog.RecordFile) TimedDecimal(de.invesdwin.util.math.decimal.TimedDecimal) BlockCompressedRecordFile(com.indeed.lsmtree.recordlog.BlockCompressedRecordFile) FDate(de.invesdwin.util.time.date.FDate) Test(org.junit.jupiter.api.Test)

Example 5 with TimedDecimal

use of de.invesdwin.util.math.decimal.TimedDecimal in project invesdwin-context-persistence by subes.

the class BlockCompressedRecordFilePerformanceTest method readIterator.

private void readIterator(final File file) throws IOException, InterruptedException {
    final LoopInterruptedCheck loopCheck = new LoopInterruptedCheck();
    final Instant readsStart = new Instant();
    for (int reads = 1; reads <= READS; reads++) {
        FDate prevValue = null;
        int count = 0;
        try (BlockCompressedRecordFile<TimedDecimal> reader = new BlockCompressedRecordFile.Builder<TimedDecimal>(file, IndeedSerializer.valueOf(TimedDecimalSerde.GET), new SnappyCodec()).build()) {
            final RecordFile.Reader<TimedDecimal> iterator = reader.reader();
            while (iterator.next()) {
                final FDate value = iterator.get().getTime();
                if (prevValue != null) {
                    Assertions.checkTrue(prevValue.isBefore(value));
                }
                prevValue = value;
                count++;
            }
            Assertions.checkEquals(count, VALUES);
            if (loopCheck.check()) {
                printProgress("Reads", readsStart, VALUES * reads, VALUES * READS);
            }
        }
    }
    printProgress("ReadsFinished", readsStart, VALUES * READS, VALUES * READS);
}
Also used : BlockCompressedRecordFile(com.indeed.lsmtree.recordlog.BlockCompressedRecordFile) RecordFile(com.indeed.lsmtree.recordlog.RecordFile) Instant(de.invesdwin.util.time.Instant) LoopInterruptedCheck(de.invesdwin.util.concurrent.loop.LoopInterruptedCheck) TimedDecimal(de.invesdwin.util.math.decimal.TimedDecimal) SnappyCodec(com.indeed.util.compress.SnappyCodec) FDate(de.invesdwin.util.time.date.FDate) BlockCompressedRecordFile(com.indeed.lsmtree.recordlog.BlockCompressedRecordFile)

Aggregations

TimedDecimal (de.invesdwin.util.math.decimal.TimedDecimal)7 RecordFile (com.indeed.lsmtree.recordlog.RecordFile)6 LoopInterruptedCheck (de.invesdwin.util.concurrent.loop.LoopInterruptedCheck)6 Instant (de.invesdwin.util.time.Instant)6 FDate (de.invesdwin.util.time.date.FDate)6 SnappyCodec (com.indeed.util.compress.SnappyCodec)3 File (java.io.File)3 Test (org.junit.jupiter.api.Test)3 BasicRecordFile (com.indeed.lsmtree.recordlog.BasicRecordFile)2 BlockCompressedRecordFile (com.indeed.lsmtree.recordlog.BlockCompressedRecordFile)2 RecordLogDirectory (com.indeed.lsmtree.recordlog.RecordLogDirectory)2 Decimal (de.invesdwin.util.math.decimal.Decimal)1 FDate (de.invesdwin.util.time.fdate.FDate)1 ByteBuffer (java.nio.ByteBuffer)1