Examples with SweepResults com.palantir.atlasdb.keyvalue.api.SweepResults used on opensource projects

Example 6 with SweepResults

use of com.palantir.atlasdb.keyvalue.api.SweepResults in project atlasdb by palantir.

the class BackgroundSweeperFastTest method testMetricsRecordedAfterIncompleteRunForOneIterationOnly.

@Test
public void testMetricsRecordedAfterIncompleteRunForOneIterationOnly() {
    setNoProgress();
    setNextTableToSweep(TABLE_REF);
    SweepResults intermediateResults = ImmutableSweepResults.builder().staleValuesDeleted(2).cellTsPairsExamined(10).minSweptTimestamp(12345L).nextStartRow(Optional.of(new byte[] { 1, 2, 3 })).timeInMillis(10L).timeSweepStarted(20L).build();
    setupTaskRunner(intermediateResults);
    backgroundSweeper.runOnce();
    Mockito.verify(sweepMetricsManager).updateMetrics(intermediateResults, TABLE_REF);
}

Example 7 with SweepResults

use of com.palantir.atlasdb.keyvalue.api.SweepResults in project atlasdb by palantir.

the class SweepBenchmarks method runSingleSweep.

private Object runSingleSweep(RegeneratingTable table, int uniqueCellsToSweep) {
    SweepTaskRunner sweepTaskRunner = table.getSweepTaskRunner();
    SweepBatchConfig batchConfig = ImmutableSweepBatchConfig.builder().deleteBatchSize(DELETED_COUNT * uniqueCellsToSweep).candidateBatchSize(RegeneratingTable.SWEEP_DUPLICATES * uniqueCellsToSweep + 1).maxCellTsPairsToExamine(RegeneratingTable.SWEEP_DUPLICATES * uniqueCellsToSweep).build();
    SweepResults sweepResults = sweepTaskRunner.run(table.getTableRef(), batchConfig, PtBytes.EMPTY_BYTE_ARRAY);
    assertThat(sweepResults.getStaleValuesDeleted(), is((long) DELETED_COUNT * uniqueCellsToSweep));
    return sweepResults;
}

Example 8 with SweepResults

use of com.palantir.atlasdb.keyvalue.api.SweepResults in project atlasdb by palantir.

the class SweepBenchmarks method runMultiSweep.

private Object runMultiSweep(RegeneratingTable table) {
    SweepTaskRunner sweepTaskRunner = table.getSweepTaskRunner();
    SweepResults sweepResults = null;
    byte[] nextStartRow = PtBytes.EMPTY_BYTE_ARRAY;
    for (int i = 0; i < BATCH_SIZE; i++) {
        SweepBatchConfig batchConfig = ImmutableSweepBatchConfig.builder().deleteBatchSize(DELETED_COUNT).candidateBatchSize(1).maxCellTsPairsToExamine(RegeneratingTable.SWEEP_DUPLICATES).build();
        sweepResults = sweepTaskRunner.run(table.getTableRef(), batchConfig, nextStartRow);
        nextStartRow = sweepResults.getNextStartRow().get();
        assertThat(sweepResults.getStaleValuesDeleted(), is((long) DELETED_COUNT));
    }
    return sweepResults;
}

Example 9 with SweepResults

use of com.palantir.atlasdb.keyvalue.api.SweepResults in project atlasdb by palantir.

the class SpecificTableSweeper method runOnceAndSaveResults.

void runOnceAndSaveResults(TableToSweep tableToSweep, SweepBatchConfig batchConfig) {
    TableReference tableRef = tableToSweep.getTableRef();
    byte[] startRow = tableToSweep.getStartRow();
    SweepResults results = runOneIteration(tableRef, startRow, batchConfig);
    processSweepResults(tableToSweep, results);
}

Example 10 with SweepResults

use of com.palantir.atlasdb.keyvalue.api.SweepResults in project atlasdb by palantir.

the class SweepTaskRunner method runInternal.

private SweepResults runInternal(TableReference tableRef, SweepBatchConfig batchConfig, byte[] startRow, RunType runType) {
    Preconditions.checkNotNull(tableRef, "tableRef cannot be null");
    Preconditions.checkState(!AtlasDbConstants.hiddenTables.contains(tableRef));
    if (tableRef.getQualifiedName().startsWith(AtlasDbConstants.NAMESPACE_PREFIX)) {
        // this happens sometimes; I think it's because some places in the code can
        // start this sweeper without doing the full normally ordered KVSModule startup.
        // I did check and sweep.stats did contain the FQ table name for all of the tables,
        // so it is at least broken in some way that still allows namespaced tables to eventually be swept.
        log.warn("The sweeper should not be run on tables passed through namespace mapping.");
        return SweepResults.createEmptySweepResultWithNoMoreToSweep();
    }
    if (keyValueService.getMetadataForTable(tableRef).length == 0) {
        log.warn("The sweeper tried to sweep table '{}', but the table does not exist. Skipping table.", LoggingArgs.tableRef("tableRef", tableRef));
        return SweepResults.createEmptySweepResultWithNoMoreToSweep();
    }
    SweepStrategy sweepStrategy = sweepStrategyManager.get().getOrDefault(tableRef, SweepStrategy.CONSERVATIVE);
    Optional<Sweeper> maybeSweeper = Sweeper.of(sweepStrategy);
    return maybeSweeper.map(sweeper -> doRun(tableRef, batchConfig, startRow, runType, sweeper)).orElseGet(SweepResults::createEmptySweepResultWithNoMoreToSweep);
}