Examples with ScanAndCompareGarbageCollector org.apache.bookkeeper.bookie.ScanAndCompareGarbageCollector used on opensource projects

Example 6 with ScanAndCompareGarbageCollector

use of org.apache.bookkeeper.bookie.ScanAndCompareGarbageCollector in project bookkeeper by apache.

the class GcLedgersTest method testGcLedgersWithNoLedgers.

/*
     * in this scenario no ledger is created, so ledgeriterator's hasNext call would return false and next would be
     * null. GarbageCollector.gc is expected to behave normally
     */
@Test
public void testGcLedgersWithNoLedgers() throws Exception {
    final SortedSet<Long> createdLedgers = Collections.synchronizedSortedSet(new TreeSet<Long>());
    final List<Long> cleaned = new ArrayList<Long>();
    // no ledger created
    final GarbageCollector garbageCollector = new ScanAndCompareGarbageCollector(getLedgerManager(), new MockLedgerStorage(), baseConf, NullStatsLogger.INSTANCE);
    AtomicBoolean cleanerCalled = new AtomicBoolean(false);
    GarbageCollector.GarbageCleaner cleaner = new GarbageCollector.GarbageCleaner() {

        @Override
        public void clean(long ledgerId) {
            LOG.info("Cleaned {}", ledgerId);
            cleanerCalled.set(true);
        }
    };
    validateLedgerRangeIterator(createdLedgers);
    garbageCollector.gc(cleaner);
    assertFalse("Should have cleaned nothing, since no ledger is created", cleanerCalled.get());
}

Example 7 with ScanAndCompareGarbageCollector

use of org.apache.bookkeeper.bookie.ScanAndCompareGarbageCollector in project bookkeeper by apache.

the class GcLedgersTest method testGcLedgersWithLedgersInSameLedgerRange.

// in this scenario all the created ledgers are in one single ledger range.
@Test
public void testGcLedgersWithLedgersInSameLedgerRange() throws Exception {
    baseConf.setVerifyMetadataOnGc(true);
    final SortedSet<Long> createdLedgers = Collections.synchronizedSortedSet(new TreeSet<Long>());
    final SortedSet<Long> cleaned = Collections.synchronizedSortedSet(new TreeSet<Long>());
    // Create few ledgers which span over just one ledger range in the hierarchical ledger manager implementation
    final int numLedgers = 5;
    createLedgers(numLedgers, createdLedgers);
    final GarbageCollector garbageCollector = new ScanAndCompareGarbageCollector(getLedgerManager(), new MockLedgerStorage(), baseConf, NullStatsLogger.INSTANCE);
    GarbageCollector.GarbageCleaner cleaner = new GarbageCollector.GarbageCleaner() {

        @Override
        public void clean(long ledgerId) {
            LOG.info("Cleaned {}", ledgerId);
            cleaned.add(ledgerId);
        }
    };
    validateLedgerRangeIterator(createdLedgers);
    garbageCollector.gc(cleaner);
    assertTrue("Should have cleaned nothing", cleaned.isEmpty());
    for (long ledgerId : createdLedgers) {
        removeLedger(ledgerId);
    }
    garbageCollector.gc(cleaner);
    assertEquals("Should have cleaned all the created ledgers", createdLedgers, cleaned);
}

Example 8 with ScanAndCompareGarbageCollector

use of org.apache.bookkeeper.bookie.ScanAndCompareGarbageCollector in project bookkeeper by apache.

the class GcLedgersTest method testGcLedgersIfReadLedgerMetadataSaysNoSuchLedger.

/*
     * In this test scenario no ledger is deleted, but LedgerManager.readLedgerMetadata says there is NoSuchLedger. So
     * even in that case, GarbageCollector.gc shouldn't delete ledgers data.
     *
     * Consider the possible scenario - when the LedgerIterator is created that ledger is not deleted, so as per
     * LedgerIterator that is live ledger. But right after the LedgerIterator creation that ledger is deleted, so
     * readLedgerMetadata call would return NoSuchLedger. In this testscenario we are validating that as per Iterator if
     * that ledger is alive though currently that ledger is deleted, we should not clean data of that ledger.
     *
     * ScanAndCompareGarbageCollector/GC should clean data of ledger only if both the LedgerManager.getLedgerRanges says
     * that ledger is not existing and also ledgerManager.readLedgerMetadata fails with error
     * NoSuchLedgerExistsException.
     *
     */
@Test
public void testGcLedgersIfReadLedgerMetadataSaysNoSuchLedger() throws Exception {
    final SortedSet<Long> createdLedgers = Collections.synchronizedSortedSet(new TreeSet<Long>());
    final SortedSet<Long> cleaned = Collections.synchronizedSortedSet(new TreeSet<Long>());
    // Create few ledgers
    final int numLedgers = 5;
    createLedgers(numLedgers, createdLedgers);
    LedgerManager mockLedgerManager = new CleanupLedgerManager(getLedgerManager()) {

        @Override
        public void readLedgerMetadata(long ledgerId, GenericCallback<LedgerMetadata> readCb) {
            readCb.operationComplete(BKException.Code.NoSuchLedgerExistsException, null);
        }
    };
    final GarbageCollector garbageCollector = new ScanAndCompareGarbageCollector(mockLedgerManager, new MockLedgerStorage(), baseConf, NullStatsLogger.INSTANCE);
    GarbageCollector.GarbageCleaner cleaner = new GarbageCollector.GarbageCleaner() {

        @Override
        public void clean(long ledgerId) {
            LOG.info("Cleaned {}", ledgerId);
            cleaned.add(ledgerId);
        }
    };
    validateLedgerRangeIterator(createdLedgers);
    garbageCollector.gc(cleaner);
    assertTrue("Should have cleaned nothing", cleaned.isEmpty());
}