Search in sources :

Example 1 with SSTableServiceRunnable

use of org.bboxdb.storage.sstable.compact.SSTableServiceRunnable in project bboxdb by jnidzwetzki.

the class DiskStorage method startCompactThread.

/**
 * Start the compact thread if needed
 */
private void startCompactThread() {
    final SSTableServiceRunnable sstableCompactor = new SSTableServiceRunnable(this);
    final Thread compactThread = new Thread(sstableCompactor);
    compactThread.setName("Compact thread for: " + basedir);
    compactThread.start();
    runningThreads.add(compactThread);
}
Also used : SSTableServiceRunnable(org.bboxdb.storage.sstable.compact.SSTableServiceRunnable)

Example 2 with SSTableServiceRunnable

use of org.bboxdb.storage.sstable.compact.SSTableServiceRunnable in project bboxdb by jnidzwetzki.

the class TestTableCompactor method testCompactorRunnable.

/**
 * Test the compactor runnable
 * @throws RejectedException
 * @throws StorageManagerException
 * @throws InterruptedException
 * @throws BBoxDBException
 */
@Test(timeout = 60000)
public void testCompactorRunnable() throws StorageManagerException, RejectedException, BBoxDBException, InterruptedException {
    storageRegistry.createTable(TEST_RELATION, new TupleStoreConfiguration());
    final TupleStoreManager storageManager = storageRegistry.getTupleStoreManager(TEST_RELATION);
    Assert.assertTrue(storageManager.getServiceState().isInRunningState());
    // Create file 1
    for (int i = 0; i < 1000; i++) {
        storageManager.put(new Tuple(Integer.toString(i), BoundingBox.FULL_SPACE, "abc".getBytes()));
    }
    storageManager.flush();
    // Create file 2
    for (int i = 0; i < 1000; i++) {
        storageManager.put(new Tuple(Integer.toString(i), BoundingBox.FULL_SPACE, "abc".getBytes()));
    }
    storageManager.flush();
    final List<DiskStorage> storages = storageRegistry.getAllStorages();
    Assert.assertEquals(1, storages.size());
    final SSTableServiceRunnable ssTableCompactorRunnable = new SSTableServiceRunnable(storages.get(0));
    ssTableCompactorRunnable.forceMajorCompact(storageManager);
    // Test exception handler
    final List<ReadOnlyTupleStore> writtenStorages = storageManager.aquireStorage();
    storageManager.releaseStorage(writtenStorages);
    storageManager.shutdown();
    final List<SSTableFacade> tupleStorages = writtenStorages.stream().filter(r -> r instanceof SSTableFacade).map(r -> (SSTableFacade) r).collect(Collectors.toList());
    Assert.assertFalse(tupleStorages.isEmpty());
    ssTableCompactorRunnable.handleCompactException(tupleStorages);
}
Also used : Arrays(java.util.Arrays) SSTableWriter(org.bboxdb.storage.sstable.SSTableWriter) SSTableReader(org.bboxdb.storage.sstable.reader.SSTableReader) BeforeClass(org.junit.BeforeClass) RejectedException(org.bboxdb.commons.RejectedException) Tuple(org.bboxdb.storage.entity.Tuple) SSTableServiceRunnable(org.bboxdb.storage.sstable.compact.SSTableServiceRunnable) SSTableHelper(org.bboxdb.storage.sstable.SSTableHelper) SSTableFacade(org.bboxdb.storage.sstable.reader.SSTableFacade) BoundingBox(org.bboxdb.commons.math.BoundingBox) ArrayList(java.util.ArrayList) ReadOnlyTupleStore(org.bboxdb.storage.tuplestore.ReadOnlyTupleStore) TupleStoreManager(org.bboxdb.storage.tuplestore.manager.TupleStoreManager) Lists(com.google.common.collect.Lists) SSTableCompactor(org.bboxdb.storage.sstable.compact.SSTableCompactor) SSTableKeyIndexReader(org.bboxdb.storage.sstable.reader.SSTableKeyIndexReader) DeletedTuple(org.bboxdb.storage.entity.DeletedTuple) BBoxDBConfigurationManager(org.bboxdb.misc.BBoxDBConfigurationManager) DiskStorage(org.bboxdb.storage.tuplestore.DiskStorage) Before(org.junit.Before) AfterClass(org.junit.AfterClass) Iterator(java.util.Iterator) Test(org.junit.Test) Collectors(java.util.stream.Collectors) File(java.io.File) TupleStoreName(org.bboxdb.storage.entity.TupleStoreName) TupleStoreManagerRegistry(org.bboxdb.storage.tuplestore.manager.TupleStoreManagerRegistry) List(java.util.List) BBoxDBException(org.bboxdb.misc.BBoxDBException) TupleStoreConfiguration(org.bboxdb.storage.entity.TupleStoreConfiguration) Assert(org.junit.Assert) Collections(java.util.Collections) TupleStoreConfiguration(org.bboxdb.storage.entity.TupleStoreConfiguration) SSTableFacade(org.bboxdb.storage.sstable.reader.SSTableFacade) SSTableServiceRunnable(org.bboxdb.storage.sstable.compact.SSTableServiceRunnable) ReadOnlyTupleStore(org.bboxdb.storage.tuplestore.ReadOnlyTupleStore) DiskStorage(org.bboxdb.storage.tuplestore.DiskStorage) Tuple(org.bboxdb.storage.entity.Tuple) DeletedTuple(org.bboxdb.storage.entity.DeletedTuple) TupleStoreManager(org.bboxdb.storage.tuplestore.manager.TupleStoreManager) Test(org.junit.Test)

Aggregations

SSTableServiceRunnable (org.bboxdb.storage.sstable.compact.SSTableServiceRunnable)2 Lists (com.google.common.collect.Lists)1 File (java.io.File)1 ArrayList (java.util.ArrayList)1 Arrays (java.util.Arrays)1 Collections (java.util.Collections)1 Iterator (java.util.Iterator)1 List (java.util.List)1 Collectors (java.util.stream.Collectors)1 RejectedException (org.bboxdb.commons.RejectedException)1 BoundingBox (org.bboxdb.commons.math.BoundingBox)1 BBoxDBConfigurationManager (org.bboxdb.misc.BBoxDBConfigurationManager)1 BBoxDBException (org.bboxdb.misc.BBoxDBException)1 DeletedTuple (org.bboxdb.storage.entity.DeletedTuple)1 Tuple (org.bboxdb.storage.entity.Tuple)1 TupleStoreConfiguration (org.bboxdb.storage.entity.TupleStoreConfiguration)1 TupleStoreName (org.bboxdb.storage.entity.TupleStoreName)1 SSTableHelper (org.bboxdb.storage.sstable.SSTableHelper)1 SSTableWriter (org.bboxdb.storage.sstable.SSTableWriter)1 SSTableCompactor (org.bboxdb.storage.sstable.compact.SSTableCompactor)1