Search in sources :

Example 1 with SerializingCollection

use of de.invesdwin.context.persistence.timeseriesdb.SerializingCollection in project invesdwin-context-persistence by subes.

the class FileLiveSegment method newSerializingCollection.

private SerializingCollection<V> newSerializingCollection() {
    final File file = getFile();
    Files.deleteQuietly(file);
    try {
        Files.forceMkdirParent(file);
    } catch (final IOException e) {
        throw new RuntimeException(e);
    }
    final TextDescription name = new TextDescription("%s[%s]: newSerializingCollection()", FileLiveSegment.class.getSimpleName(), segmentedKey);
    return new SerializingCollection<V>(name, file, false) {

        @Override
        protected ISerde<V> newSerde() {
            return historicalSegmentTable.newValueSerde();
        }

        @Override
        protected Integer getFixedLength() {
            return historicalSegmentTable.newValueFixedLength();
        }

        @Override
        protected OutputStream newCompressor(final OutputStream out) {
            return compressionFactory.newCompressor(out, LARGE_COMPRESSOR);
        }

        @Override
        protected InputStream newDecompressor(final InputStream inputStream) {
            return compressionFactory.newDecompressor(inputStream);
        }

        @Override
        protected InputStream newFileInputStream(final File file) throws IOException {
            throw new UnsupportedOperationException("use getFlushedValues() instead");
        }
    };
}
Also used : RetryLaterRuntimeException(de.invesdwin.context.integration.retry.RetryLaterRuntimeException) InputStream(java.io.InputStream) TextDescription(de.invesdwin.util.lang.description.TextDescription) SerializingCollection(de.invesdwin.context.persistence.timeseriesdb.SerializingCollection) OutputStream(java.io.OutputStream) PooledFastByteArrayOutputStream(de.invesdwin.util.streams.pool.PooledFastByteArrayOutputStream) IOException(java.io.IOException) File(java.io.File)

Example 2 with SerializingCollection

use of de.invesdwin.context.persistence.timeseriesdb.SerializingCollection in project invesdwin-context-persistence by subes.

the class FileLiveSegment method getFlushedValues.

private SerializingCollection<V> getFlushedValues() {
    synchronized (this) {
        if (needsFlush) {
            values.flush();
            needsFlush = false;
        }
    }
    final TextDescription name = new TextDescription("%s[%s]: getFlushedValues()", FileLiveSegment.class.getSimpleName(), segmentedKey);
    return new SerializingCollection<V>(name, values.getFile(), true) {

        @Override
        protected ISerde<V> newSerde() {
            return historicalSegmentTable.newValueSerde();
        }

        @Override
        protected Integer getFixedLength() {
            return historicalSegmentTable.newValueFixedLength();
        }

        @Override
        protected OutputStream newCompressor(final OutputStream out) {
            return compressionFactory.newCompressor(out, LARGE_COMPRESSOR);
        }

        @Override
        protected InputStream newDecompressor(final InputStream inputStream) {
            return compressionFactory.newDecompressor(inputStream);
        }

        @Override
        protected InputStream newFileInputStream(final File file) throws IOException {
            // keep file input stream open as shorty as possible to prevent too many open files error
            try (InputStream fis = super.newFileInputStream(file)) {
                final PooledFastByteArrayOutputStream bos = PooledFastByteArrayOutputStream.newInstance();
                IOUtils.copy(fis, bos.asNonClosing());
                return bos.asInputStream();
            } catch (final FileNotFoundException e) {
                // maybe retry because of this in the outer iterator?
                throw new RetryLaterRuntimeException("File might have been deleted in the mean time between read locks: " + file.getAbsolutePath(), e);
            }
        }
    };
}
Also used : InputStream(java.io.InputStream) TextDescription(de.invesdwin.util.lang.description.TextDescription) SerializingCollection(de.invesdwin.context.persistence.timeseriesdb.SerializingCollection) OutputStream(java.io.OutputStream) PooledFastByteArrayOutputStream(de.invesdwin.util.streams.pool.PooledFastByteArrayOutputStream) FileNotFoundException(java.io.FileNotFoundException) RetryLaterRuntimeException(de.invesdwin.context.integration.retry.RetryLaterRuntimeException) File(java.io.File) PooledFastByteArrayOutputStream(de.invesdwin.util.streams.pool.PooledFastByteArrayOutputStream)

Aggregations

RetryLaterRuntimeException (de.invesdwin.context.integration.retry.RetryLaterRuntimeException)2 SerializingCollection (de.invesdwin.context.persistence.timeseriesdb.SerializingCollection)2 TextDescription (de.invesdwin.util.lang.description.TextDescription)2 PooledFastByteArrayOutputStream (de.invesdwin.util.streams.pool.PooledFastByteArrayOutputStream)2 File (java.io.File)2 InputStream (java.io.InputStream)2 OutputStream (java.io.OutputStream)2 FileNotFoundException (java.io.FileNotFoundException)1 IOException (java.io.IOException)1