Search in sources :

Example 91 with Instant

use of de.invesdwin.util.time.Instant in project invesdwin-context-persistence by subes.

the class PersistentLiveSegment method putNextLiveValues.

public void putNextLiveValues(final ICloseableIterable<V> memoryValues) {
    final ADelegateRangeTable<String, TimeRange, SegmentStatus> segmentStatusTable = historicalSegmentTable.getStorage().getSegmentStatusTable();
    final SegmentStatus existingStatus = segmentStatusTable.get(hashKey, segmentedKey.getSegment());
    if (existingStatus == null) {
        segmentStatusTable.put(hashKey, segmentedKey.getSegment(), SegmentStatus.INITIALIZING);
    } else if (existingStatus != SegmentStatus.INITIALIZING) {
        throw UnknownArgumentException.newInstance(SegmentStatus.class, existingStatus);
    }
    final ATimeSeriesUpdater<SegmentedKey<K>, V> updater = new ATimeSeriesUpdater<SegmentedKey<K>, V>(segmentedKey, table) {

        @Override
        protected ICloseableIterable<? extends V> getSource(final FDate updateFrom) {
            return memoryValues;
        }

        @Override
        protected void onUpdateFinished(final Instant updateStart) {
        }

        @Override
        protected void onUpdateStart() {
        }

        @Override
        protected FDate extractEndTime(final V element) {
            return historicalSegmentTable.extractEndTime(element);
        }

        @Override
        protected void onFlush(final int flushIndex, final ATimeSeriesUpdater<SegmentedKey<K>, V>.UpdateProgress updateProgress) {
        }

        @Override
        protected boolean shouldRedoLastFile() {
            return false;
        }

        @Override
        public Percent getProgress() {
            return null;
        }
    };
    try {
        Assertions.checkTrue(updater.update());
    } catch (final IncompleteUpdateFoundException e) {
        throw new RuntimeException(e);
    }
    empty = false;
}
Also used : SegmentStatus(de.invesdwin.context.persistence.timeseriesdb.segmented.SegmentStatus) Instant(de.invesdwin.util.time.Instant) FDate(de.invesdwin.util.time.date.FDate) TimeRange(de.invesdwin.util.time.range.TimeRange) ATimeSeriesUpdater(de.invesdwin.context.persistence.timeseriesdb.updater.ATimeSeriesUpdater) SegmentedKey(de.invesdwin.context.persistence.timeseriesdb.segmented.SegmentedKey) IncompleteUpdateFoundException(de.invesdwin.context.persistence.timeseriesdb.IncompleteUpdateFoundException)

Example 92 with Instant

use of de.invesdwin.util.time.Instant in project invesdwin-context-persistence by subes.

the class PersistentTkrzwMapPerformanceTest method readIterator.

private void readIterator(final APersistentMap<FDate, FDate> table) {
    final Instant readsStart = new Instant();
    for (int reads = 1; reads <= READS; reads++) {
        // FDate prevValue = null;
        final Iterator<FDate> range = table.values().iterator();
        int count = 0;
        while (true) {
            try {
                final FDate value = range.next();
                // if (prevValue != null) {
                // Assertions.checkTrue(prevValue.isBefore(value));
                // }
                // prevValue = value;
                count++;
            } catch (final NoSuchElementException e) {
                break;
            }
        }
        Closeables.close(range);
        Assertions.checkEquals(count, VALUES);
        printProgress("Reads", readsStart, VALUES * reads, VALUES * READS);
    }
    printProgress("ReadsFinished", readsStart, VALUES * READS, VALUES * READS);
}
Also used : Instant(de.invesdwin.util.time.Instant) FDate(de.invesdwin.util.time.date.FDate) NoSuchElementException(java.util.NoSuchElementException)

Example 93 with Instant

use of de.invesdwin.util.time.Instant in project invesdwin-context-persistence by subes.

the class PersistentTkrzwMapPerformanceTest method testTkrzwMapPerformance.

@Test
public void testTkrzwMapPerformance() throws InterruptedException {
    // tkh = HashDBM
    // tkt = TreeDBM
    // tks = SkipDBM (make sure to call synchronize)
    @SuppressWarnings("resource") final APersistentMap<FDate, FDate> table = new APersistentMap<FDate, FDate>("testTkrzwMapPerformance.tkh") {

        @Override
        public File getBaseDirectory() {
            return ContextProperties.TEMP_DIRECTORY;
        }

        @Override
        public ISerde<FDate> newKeySerde() {
            return FDateSerde.GET;
        }

        @Override
        public ISerde<FDate> newValueSerde() {
            return FDateSerde.GET;
        }

        @Override
        protected IPersistentMapFactory<FDate, FDate> newFactory() {
            return new PersistentTkrzwMapFactory<FDate, FDate>();
        }
    };
    final LoopInterruptedCheck loopCheck = new LoopInterruptedCheck(Duration.ONE_SECOND);
    final Instant writesStart = new Instant();
    int i = 0;
    final TkrzwMap<FDate, FDate> delegate = (TkrzwMap<FDate, FDate>) table.getPreLockedDelegate();
    table.getReadLock().unlock();
    for (final FDate date : newValues()) {
        table.put(date, date);
        i++;
        if (i % FLUSH_INTERVAL == 0) {
            if (loopCheck.check()) {
                printProgress("Writes", writesStart, i, VALUES);
                delegate.getDbm().synchronize(true);
            }
        }
    }
    printProgress("WritesFinished", writesStart, VALUES, VALUES);
    delegate.getDbm().synchronize(true);
    readIterator(table);
    readGet(table);
    table.deleteTable();
}
Also used : APersistentMap(de.invesdwin.context.integration.persistentmap.APersistentMap) Instant(de.invesdwin.util.time.Instant) LoopInterruptedCheck(de.invesdwin.util.concurrent.loop.LoopInterruptedCheck) FDate(de.invesdwin.util.time.date.FDate) Test(org.junit.jupiter.api.Test)

Aggregations

Instant (de.invesdwin.util.time.Instant)93 FDate (de.invesdwin.util.time.date.FDate)78 LoopInterruptedCheck (de.invesdwin.util.concurrent.loop.LoopInterruptedCheck)65 NoSuchElementException (java.util.NoSuchElementException)30 Test (org.junit.jupiter.api.Test)25 File (java.io.File)16 PreparedStatement (java.sql.PreparedStatement)16 ResultSet (java.sql.ResultSet)15 Statement (java.sql.Statement)12 ATest (de.invesdwin.context.test.ATest)8 RecordFile (com.indeed.lsmtree.recordlog.RecordFile)6 ProcessedEventsRateString (de.invesdwin.util.lang.ProcessedEventsRateString)6 TimedDecimal (de.invesdwin.util.math.decimal.TimedDecimal)5 FDate (de.invesdwin.util.time.fdate.FDate)5 IOException (java.io.IOException)5 Connection (java.sql.Connection)5 APersistentMap (de.invesdwin.context.integration.persistentmap.APersistentMap)4 ATimeSeriesUpdater (de.invesdwin.context.persistence.timeseriesdb.updater.ATimeSeriesUpdater)4 Test (org.junit.Test)4 SnappyCodec (com.indeed.util.compress.SnappyCodec)3