Search in sources :

Example 1 with IndexEntryUpdate

use of org.neo4j.storageengine.api.IndexEntryUpdate in project neo4j by neo4j.

the class IndexingServiceIntegrationTest method tracePageCacheAccessOnIndexUpdatesApply.

@ParameterizedTest
@MethodSource("parameters")
void tracePageCacheAccessOnIndexUpdatesApply(GraphDatabaseSettings.SchemaIndex schemaIndex) throws KernelException {
    setUp(schemaIndex);
    var marker = Label.label("marker");
    var propertyName = "property";
    var testConstraint = "testConstraint";
    try (Transaction transaction = database.beginTx()) {
        transaction.schema().constraintFor(marker).withName(testConstraint).assertPropertyIsUnique(propertyName).create();
        transaction.commit();
    }
    var dependencyResolver = ((GraphDatabaseAPI) database).getDependencyResolver();
    var indexingService = dependencyResolver.resolveDependency(IndexingService.class);
    var pageCacheTracer = dependencyResolver.resolveDependency(PageCacheTracer.class);
    try (Transaction transaction = database.beginTx()) {
        var kernelTransaction = ((InternalTransaction) transaction).kernelTransaction();
        var indexDescriptor = kernelTransaction.schemaRead().indexGetForName(testConstraint);
        try (var cursorContext = new CursorContext(pageCacheTracer.createPageCursorTracer("tracePageCacheAccessOnIndexUpdatesApply"))) {
            Iterable<IndexEntryUpdate<IndexDescriptor>> updates = List.of(add(1, indexDescriptor, longValue(4)));
            indexingService.applyUpdates(updates, cursorContext);
            PageCursorTracer cursorTracer = cursorContext.getCursorTracer();
            assertEquals(5L, cursorTracer.pins());
            assertEquals(5L, cursorTracer.unpins());
            assertEquals(2L, cursorTracer.hits());
            assertEquals(3L, cursorTracer.faults());
        }
    }
}
Also used : IndexEntryUpdate(org.neo4j.storageengine.api.IndexEntryUpdate) PageCursorTracer(org.neo4j.io.pagecache.tracing.cursor.PageCursorTracer) InternalTransaction(org.neo4j.kernel.impl.coreapi.InternalTransaction) Transaction(org.neo4j.graphdb.Transaction) KernelTransaction(org.neo4j.kernel.api.KernelTransaction) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) CursorContext(org.neo4j.io.pagecache.context.CursorContext) InternalTransaction(org.neo4j.kernel.impl.coreapi.InternalTransaction) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) MethodSource(org.junit.jupiter.params.provider.MethodSource)

Example 2 with IndexEntryUpdate

use of org.neo4j.storageengine.api.IndexEntryUpdate in project neo4j by neo4j.

the class BlockBasedIndexPopulatorTest method shouldDeallocateAllAllocatedMemoryOnClose.

@Test
void shouldDeallocateAllAllocatedMemoryOnClose() throws IndexEntryConflictException, IOException {
    // given
    ThreadSafePeakMemoryTracker memoryTracker = new ThreadSafePeakMemoryTracker();
    ByteBufferFactory bufferFactory = new ByteBufferFactory(UnsafeDirectByteBufferAllocator::new, 100);
    BlockBasedIndexPopulator<GenericKey, NativeIndexValue> populator = instantiatePopulator(NO_MONITOR, bufferFactory, memoryTracker);
    boolean closed = false;
    try {
        // when
        Collection<IndexEntryUpdate<?>> updates = batchOfUpdates();
        populator.add(updates, NULL);
        int nextId = updates.size();
        externalUpdates(populator, nextId, nextId + 10);
        nextId = nextId + 10;
        long memoryBeforeScanCompleted = memoryTracker.usedNativeMemory();
        populator.scanCompleted(nullInstance, populationWorkScheduler, NULL);
        externalUpdates(populator, nextId, nextId + 10);
        // then
        assertTrue(memoryTracker.peakMemoryUsage() > memoryBeforeScanCompleted, "expected some memory to have been temporarily allocated in scanCompleted");
        populator.close(true, NULL);
        closed = true;
        bufferFactory.close();
        assertEquals(0, memoryTracker.usedNativeMemory());
    } finally {
        if (!closed) {
            populator.close(true, NULL);
        }
    }
}
Also used : IndexEntryUpdate(org.neo4j.storageengine.api.IndexEntryUpdate) ThreadSafePeakMemoryTracker(org.neo4j.memory.ThreadSafePeakMemoryTracker) ByteBufferFactory(org.neo4j.io.memory.ByteBufferFactory) Test(org.junit.jupiter.api.Test) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 3 with IndexEntryUpdate

use of org.neo4j.storageengine.api.IndexEntryUpdate in project neo4j by neo4j.

the class FusionIndexPopulatorTest method verifyAddWithCorrectPopulator.

private void verifyAddWithCorrectPopulator(IndexPopulator correctPopulator, Value... numberValues) throws IndexEntryConflictException {
    Collection<IndexEntryUpdate<LabelSchemaDescriptor>> update = Collections.singletonList(add(numberValues));
    fusionIndexPopulator.add(update, NULL);
    verify(correctPopulator).add(update, NULL);
    for (IndexPopulator alivePopulator : alivePopulators) {
        if (alivePopulator != correctPopulator) {
            verify(alivePopulator, never()).add(update, NULL);
        }
    }
}
Also used : IndexPopulator(org.neo4j.kernel.api.index.IndexPopulator) IndexEntryUpdate(org.neo4j.storageengine.api.IndexEntryUpdate)

Example 4 with IndexEntryUpdate

use of org.neo4j.storageengine.api.IndexEntryUpdate in project neo4j by neo4j.

the class NativeIndexPopulatorTests method applyInterleaved.

void applyInterleaved(IndexEntryUpdate<IndexDescriptor>[] updates, IndexPopulator populator) throws IndexEntryConflictException {
    boolean useUpdater = true;
    Collection<IndexEntryUpdate<IndexDescriptor>> populatorBatch = new ArrayList<>();
    IndexUpdater updater = populator.newPopulatingUpdater(null_property_accessor, NULL);
    for (IndexEntryUpdate<IndexDescriptor> update : updates) {
        if (random.nextInt(100) < 20) {
            if (useUpdater) {
                updater.close();
                populatorBatch = new ArrayList<>();
            } else {
                populator.add(populatorBatch, NULL);
                updater = populator.newPopulatingUpdater(null_property_accessor, NULL);
            }
            useUpdater = !useUpdater;
        }
        if (useUpdater) {
            updater.process(update);
        } else {
            populatorBatch.add(update);
        }
    }
    if (useUpdater) {
        updater.close();
    } else {
        populator.add(populatorBatch, NULL);
    }
}
Also used : IndexEntryUpdate(org.neo4j.storageengine.api.IndexEntryUpdate) ValueIndexEntryUpdate(org.neo4j.storageengine.api.ValueIndexEntryUpdate) ArrayList(java.util.ArrayList) IndexDescriptor(org.neo4j.internal.schema.IndexDescriptor) IndexUpdater(org.neo4j.kernel.api.index.IndexUpdater)

Example 5 with IndexEntryUpdate

use of org.neo4j.storageengine.api.IndexEntryUpdate in project neo4j by neo4j.

the class UniqueDatabaseIndexPopulatorTest method sampleIncludedUpdates.

@Test
void sampleIncludedUpdates() {
    LabelSchemaDescriptor schemaDescriptor = forLabel(1, 1);
    populator = newPopulator();
    List<IndexEntryUpdate<?>> updates = Arrays.asList(add(1, schemaDescriptor, "foo"), add(2, schemaDescriptor, "bar"), add(3, schemaDescriptor, "baz"), add(4, schemaDescriptor, "qux"));
    updates.forEach(populator::includeSample);
    IndexSample sample = populator.sample(NULL);
    assertEquals(new IndexSample(4, 4, 4), sample);
}
Also used : IndexEntryUpdate(org.neo4j.storageengine.api.IndexEntryUpdate) IndexSample(org.neo4j.kernel.api.index.IndexSample) LabelSchemaDescriptor(org.neo4j.internal.schema.LabelSchemaDescriptor) Test(org.junit.jupiter.api.Test)

Aggregations

IndexEntryUpdate (org.neo4j.storageengine.api.IndexEntryUpdate)24 Test (org.junit.jupiter.api.Test)18 IndexDescriptor (org.neo4j.internal.schema.IndexDescriptor)6 IndexSample (org.neo4j.kernel.api.index.IndexSample)6 IndexUpdater (org.neo4j.kernel.api.index.IndexUpdater)6 Value (org.neo4j.values.storable.Value)6 ArrayList (java.util.ArrayList)5 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)4 IndexPopulator (org.neo4j.kernel.api.index.IndexPopulator)4 ByteBufferFactory (org.neo4j.io.memory.ByteBufferFactory)3 ThreadSafePeakMemoryTracker (org.neo4j.memory.ThreadSafePeakMemoryTracker)3 ValueIndexEntryUpdate (org.neo4j.storageengine.api.ValueIndexEntryUpdate)3 PointValue (org.neo4j.values.storable.PointValue)3 IOException (java.io.IOException)2 HashSet (java.util.HashSet)2 ExecutorService (java.util.concurrent.ExecutorService)2 TokenNameLookup (org.neo4j.common.TokenNameLookup)2 Transaction (org.neo4j.graphdb.Transaction)2 LabelSchemaDescriptor (org.neo4j.internal.schema.LabelSchemaDescriptor)2 KernelTransaction (org.neo4j.kernel.api.KernelTransaction)2