Examples with IndexSample org.neo4j.kernel.api.index.IndexSample used on opensource projects

Example 51 with IndexSample

use of org.neo4j.kernel.api.index.IndexSample in project neo4j by neo4j.

the class FusionIndexSampler method combineSamples.

public static IndexSample combineSamples(Iterable<IndexSample> samples) {
    long indexSize = 0;
    long uniqueValues = 0;
    long sampleSize = 0;
    for (IndexSample sample : samples) {
        indexSize += sample.indexSize();
        uniqueValues += sample.uniqueValues();
        sampleSize += sample.sampleSize();
    }
    return new IndexSample(indexSize, uniqueValues, sampleSize);
}

Example 52 with IndexSample

use of org.neo4j.kernel.api.index.IndexSample in project neo4j by neo4j.

the class BlockBasedIndexPopulatorTest method shouldBuildNonUniqueSampleAsPartOfScanCompleted.

@Test
void shouldBuildNonUniqueSampleAsPartOfScanCompleted() throws IndexEntryConflictException, IOException {
    // given
    ThreadSafePeakMemoryTracker memoryTracker = new ThreadSafePeakMemoryTracker();
    ByteBufferFactory bufferFactory = new ByteBufferFactory(UnsafeDirectByteBufferAllocator::new, 100);
    BlockBasedIndexPopulator<GenericKey, NativeIndexValue> populator = instantiatePopulator(NO_MONITOR, bufferFactory, memoryTracker);
    Collection<IndexEntryUpdate<?>> populationUpdates = batchOfUpdates();
    populator.add(populationUpdates, NULL);
    // when
    populator.scanCompleted(nullInstance, populationWorkScheduler, NULL);
    // Also a couple of updates afterwards
    int numberOfUpdatesAfterCompleted = 4;
    try (IndexUpdater updater = populator.newPopulatingUpdater(NULL)) {
        for (int i = 0; i < numberOfUpdatesAfterCompleted; i++) {
            updater.process(IndexEntryUpdate.add(10_000 + i, SCHEMA_DESCRIPTOR, intValue(i)));
        }
    }
    populator.close(true, NULL);
    // then
    IndexSample sample = populator.sample(NULL);
    assertEquals(populationUpdates.size(), sample.indexSize());
    assertEquals(populationUpdates.size(), sample.sampleSize());
    assertEquals(populationUpdates.size(), sample.uniqueValues());
    assertEquals(numberOfUpdatesAfterCompleted, sample.updates());
}

Example 53 with IndexSample

use of org.neo4j.kernel.api.index.IndexSample in project neo4j by neo4j.

the class BlockBasedIndexPopulatorTest method shouldCountExternalUpdatesAsSampleUpdates.

@Test
void shouldCountExternalUpdatesAsSampleUpdates() throws IOException, IndexEntryConflictException {
    // given
    BlockBasedIndexPopulator<GenericKey, NativeIndexValue> populator = instantiatePopulator(NO_MONITOR);
    try {
        populator.add(List.of(add(0), add(1)), NULL);
        try (IndexUpdater updater = populator.newPopulatingUpdater(NULL)) {
            updater.process(add(10));
            updater.process(add(11));
            updater.process(add(12));
        }
        // when
        populator.scanCompleted(nullInstance, populationWorkScheduler, NULL);
        try (IndexUpdater updater = populator.newPopulatingUpdater(NULL)) {
            updater.process(remove(10));
        }
        // then
        IndexSample sample = populator.sample(NULL);
        assertThat(sample.updates()).isEqualTo(4);
    } finally {
        populator.close(true, NULL);
    }
}

Example 54 with IndexSample

use of org.neo4j.kernel.api.index.IndexSample in project neo4j by neo4j.

the class FullScanNonUniqueIndexSamplerTest method shouldIncludeAllValuesInTree.

@Test
void shouldIncludeAllValuesInTree() throws Exception {
    // GIVEN
    Value[] values = generateNumberValues();
    buildTree(values);
    // WHEN
    IndexSample sample;
    try (GBPTree<GenericKey, NativeIndexValue> gbpTree = getTree()) {
        FullScanNonUniqueIndexSampler<GenericKey, NativeIndexValue> sampler = new FullScanNonUniqueIndexSampler<>(gbpTree, layout);
        sample = sampler.sample(NULL);
    }
    // THEN
    assertEquals(values.length, sample.sampleSize());
    assertEquals(countUniqueValues(values), sample.uniqueValues());
    assertEquals(values.length, sample.indexSize());
}

Example 55 with IndexSample

use of org.neo4j.kernel.api.index.IndexSample in project neo4j by neo4j.

the class OnlineIndexSamplingJob method run.

@Override
public void run() {
    try (DurationLogger durationLogger = new DurationLogger(log, "Sampling index " + indexUserDescription)) {
        try {
            try (var reader = indexProxy.newValueReader();
                var cursorContext = new CursorContext(pageCacheTracer.createPageCursorTracer(INDEX_SAMPLER_TAG));
                IndexSampler sampler = reader.createSampler()) {
                IndexSample sample = sampler.sampleIndex(cursorContext);
                // check again if the index is online before saving the counts in the store
                if (indexProxy.getState() == ONLINE) {
                    indexStatisticsStore.replaceStats(indexId, sample);
                    durationLogger.markAsFinished();
                    log.debug(format("Sampled index %s with %d unique values in sample of avg size %d taken from " + "index containing %d entries", indexUserDescription, sample.uniqueValues(), sample.sampleSize(), sample.indexSize()));
                } else {
                    durationLogger.markAsAborted("Index no longer ONLINE");
                }
            }
        } catch (IndexNotFoundKernelException e) {
            durationLogger.markAsAborted("Attempted to sample missing/already deleted index " + indexUserDescription);
        }
    }
}