Examples with IndexSampler org.neo4j.kernel.api.index.IndexSampler used on opensource projects

Example 6 with IndexSampler

use of org.neo4j.kernel.api.index.IndexSampler in project neo4j by neo4j.

the class LuceneSchemaIndexPopulationIT method partitionedIndexPopulation.

@ParameterizedTest
@ValueSource(ints = { 7, 11, 14, 20, 35, 58 })
void partitionedIndexPopulation(int affectedNodes) throws Exception {
    Path rootFolder = testDir.directory("partitionIndex" + affectedNodes).resolve("uniqueIndex" + affectedNodes);
    try (SchemaIndex uniqueIndex = LuceneSchemaIndexBuilder.create(descriptor, writable(), Config.defaults()).withFileSystem(fileSystem).withIndexRootFolder(rootFolder).build()) {
        uniqueIndex.open();
        // index is empty and not yet exist
        assertEquals(0, uniqueIndex.allDocumentsReader().maxCount());
        assertFalse(uniqueIndex.exists());
        try (LuceneIndexAccessor indexAccessor = new LuceneIndexAccessor(uniqueIndex, descriptor, SIMPLE_TOKEN_LOOKUP)) {
            generateUpdates(indexAccessor, affectedNodes);
            indexAccessor.force(NULL);
            // now index is online and should contain updates data
            assertTrue(uniqueIndex.isOnline());
            try (var indexReader = indexAccessor.newValueReader();
                NodeValueIterator results = new NodeValueIterator();
                IndexSampler indexSampler = indexReader.createSampler()) {
                indexReader.query(NULL_CONTEXT, results, unconstrained(), PropertyIndexQuery.exists(1));
                long[] nodes = PrimitiveLongCollections.asArray(results);
                assertEquals(affectedNodes, nodes.length);
                IndexSample sample = indexSampler.sampleIndex(NULL);
                assertEquals(affectedNodes, sample.indexSize());
                assertEquals(affectedNodes, sample.uniqueValues());
                assertEquals(affectedNodes, sample.sampleSize());
            }
        }
    }
}

Example 7 with IndexSampler

use of org.neo4j.kernel.api.index.IndexSampler in project neo4j by neo4j.

the class FusionIndexSampler method sampleIndex.

@Override
public IndexSample sampleIndex(CursorContext cursorContext) throws IndexNotFoundKernelException {
    List<IndexSample> samples = new ArrayList<>();
    Exception exception = null;
    for (IndexSampler sampler : samplers) {
        try {
            samples.add(sampler.sampleIndex(cursorContext));
        } catch (IndexNotFoundKernelException | RuntimeException e) {
            exception = Exceptions.chain(exception, e);
        }
    }
    if (exception != null) {
        throwIfUnchecked(exception);
        throwIfInstanceOf(exception, IndexNotFoundKernelException.class);
        throw new RuntimeException(exception);
    }
    return combineSamples(samples);
}

Example 8 with IndexSampler

use of org.neo4j.kernel.api.index.IndexSampler in project neo4j by neo4j.

the class NativeIndexReader method createSampler.

@Override
public IndexSampler createSampler() {
    // For a unique index there's an optimization, knowing that all values in it are unique, to simply count
    // the number of indexed values and create a sample for that count. The GBPTree doesn't have an O(1)
    // count mechanism, it will have to manually count the indexed values in it to get it.
    // For that reason this implementation opts for keeping complexity down by just using the existing
    // non-unique sampler which scans the index and counts (potentially duplicates, of which there will
    // be none in a unique index).
    FullScanNonUniqueIndexSampler<KEY, VALUE> sampler = new FullScanNonUniqueIndexSampler<>(tree, layout);
    return tracer -> {
        try {
            return sampler.sample(tracer);
        } catch (UncheckedIOException e) {
            if (getRootCause(e) instanceof FileIsNotMappedException) {
                IndexNotFoundKernelException exception = new IndexNotFoundKernelException("Index dropped while sampling.");
                exception.addSuppressed(e);
                throw exception;
            }
            throw e;
        }
    };
}

Example 9 with IndexSampler

use of org.neo4j.kernel.api.index.IndexSampler in project neo4j by neo4j.

the class OnlineIndexSamplingJob method run.

@Override
public void run() {
    try (DurationLogger durationLogger = new DurationLogger(log, "Sampling index " + indexUserDescription)) {
        try {
            try (var reader = indexProxy.newValueReader();
                var cursorContext = new CursorContext(pageCacheTracer.createPageCursorTracer(INDEX_SAMPLER_TAG));
                IndexSampler sampler = reader.createSampler()) {
                IndexSample sample = sampler.sampleIndex(cursorContext);
                // check again if the index is online before saving the counts in the store
                if (indexProxy.getState() == ONLINE) {
                    indexStatisticsStore.replaceStats(indexId, sample);
                    durationLogger.markAsFinished();
                    log.debug(format("Sampled index %s with %d unique values in sample of avg size %d taken from " + "index containing %d entries", indexUserDescription, sample.uniqueValues(), sample.sampleSize(), sample.indexSize()));
                } else {
                    durationLogger.markAsAborted("Index no longer ONLINE");
                }
            }
        } catch (IndexNotFoundKernelException e) {
            durationLogger.markAsAborted("Attempted to sample missing/already deleted index " + indexUserDescription);
        }
    }
}