Search in sources :

Example 1 with SortedIteratorMerger

use of org.bboxdb.commons.SortedIteratorMerger in project bboxdb by jnidzwetzki.

the class SSTableCompactor method executeCompactation.

/**
 * Execute the compactation of the input sstables
 *
 * @return success or failure
 */
public void executeCompactation() throws StorageManagerException {
    try {
        // The iterators
        final List<Iterator<Tuple>> iterators = sstableIndexReader.stream().map(r -> r.iterator()).collect(Collectors.toList());
        final DuplicateResolver<Tuple> newestKeyResolver = TupleDuplicateResolverFactory.build(tupleStoreManager.getTupleStoreConfiguration());
        final SortedIteratorMerger<Tuple> sortedIteratorMerger = new SortedIteratorMerger<>(iterators, TupleHelper.TUPLE_KEY_COMPARATOR, newestKeyResolver);
        for (final Tuple tuple : sortedIteratorMerger) {
            checkForThreadTermination();
            addTupleToWriter(tuple);
        }
        readTuples = sortedIteratorMerger.getReadElements();
    } catch (StorageManagerException e) {
        handleErrorDuringCompact();
        throw e;
    } finally {
        closeSSTableWriter();
    }
}
Also used : SSTableWriter(org.bboxdb.storage.sstable.SSTableWriter) Logger(org.slf4j.Logger) Iterator(java.util.Iterator) Tuple(org.bboxdb.storage.entity.Tuple) TupleHelper(org.bboxdb.storage.util.TupleHelper) LoggerFactory(org.slf4j.LoggerFactory) SSTableConst(org.bboxdb.storage.sstable.SSTableConst) SortedIteratorMerger(org.bboxdb.commons.SortedIteratorMerger) Collectors(java.util.stream.Collectors) TupleDuplicateResolverFactory(org.bboxdb.storage.sstable.duplicateresolver.TupleDuplicateResolverFactory) ArrayList(java.util.ArrayList) DuplicateResolver(org.bboxdb.commons.DuplicateResolver) List(java.util.List) TupleStoreManager(org.bboxdb.storage.tuplestore.manager.TupleStoreManager) SSTableKeyIndexReader(org.bboxdb.storage.sstable.reader.SSTableKeyIndexReader) DeletedTuple(org.bboxdb.storage.entity.DeletedTuple) StorageManagerException(org.bboxdb.storage.StorageManagerException) Iterator(java.util.Iterator) SortedIteratorMerger(org.bboxdb.commons.SortedIteratorMerger) StorageManagerException(org.bboxdb.storage.StorageManagerException) Tuple(org.bboxdb.storage.entity.Tuple) DeletedTuple(org.bboxdb.storage.entity.DeletedTuple)

Aggregations

ArrayList (java.util.ArrayList)1 Iterator (java.util.Iterator)1 List (java.util.List)1 Collectors (java.util.stream.Collectors)1 DuplicateResolver (org.bboxdb.commons.DuplicateResolver)1 SortedIteratorMerger (org.bboxdb.commons.SortedIteratorMerger)1 StorageManagerException (org.bboxdb.storage.StorageManagerException)1 DeletedTuple (org.bboxdb.storage.entity.DeletedTuple)1 Tuple (org.bboxdb.storage.entity.Tuple)1 SSTableConst (org.bboxdb.storage.sstable.SSTableConst)1 SSTableWriter (org.bboxdb.storage.sstable.SSTableWriter)1 TupleDuplicateResolverFactory (org.bboxdb.storage.sstable.duplicateresolver.TupleDuplicateResolverFactory)1 SSTableKeyIndexReader (org.bboxdb.storage.sstable.reader.SSTableKeyIndexReader)1 TupleStoreManager (org.bboxdb.storage.tuplestore.manager.TupleStoreManager)1 TupleHelper (org.bboxdb.storage.util.TupleHelper)1 Logger (org.slf4j.Logger)1 LoggerFactory (org.slf4j.LoggerFactory)1