Search in sources :

Example 6 with DynamicStringStore

use of org.neo4j.kernel.impl.store.DynamicStringStore in project neo4j by neo4j.

the class RecordLoading method safeLoadTokens.

static <RECORD extends TokenRecord> List<NamedToken> safeLoadTokens(TokenStore<RECORD> tokenStore, CursorContext cursorContext) {
    long highId = tokenStore.getHighId();
    List<NamedToken> tokens = new ArrayList<>();
    DynamicStringStore nameStore = tokenStore.getNameStore();
    List<DynamicRecord> nameRecords = new ArrayList<>();
    MutableLongSet seenRecordIds = new LongHashSet();
    int nameBlockSize = nameStore.getRecordDataSize();
    try (RecordReader<RECORD> tokenReader = new RecordReader<>(tokenStore, true, cursorContext);
        RecordReader<DynamicRecord> nameReader = new RecordReader<>(nameStore, false, cursorContext)) {
        for (long id = 0; id < highId; id++) {
            RECORD record = tokenReader.read(id);
            nameRecords.clear();
            if (record.inUse()) {
                String name;
                if (!NULL_REFERENCE.is(record.getNameId()) && safeLoadDynamicRecordChain(r -> nameRecords.add(r.copy()), nameReader, seenRecordIds, record.getNameId(), nameBlockSize)) {
                    record.addNameRecords(nameRecords);
                    name = tokenStore.getStringFor(record, StoreCursors.NULL);
                } else {
                    name = format("<name not loaded due to token(%d) referencing unused name record>", id);
                }
                tokens.add(new NamedToken(name, toIntExact(id), record.isInternal()));
            }
        }
    }
    return tokens;
}
Also used : DynamicRecord(org.neo4j.kernel.impl.store.record.DynamicRecord) PageCursor(org.neo4j.io.pagecache.PageCursor) ThrowingIntFunction(org.neo4j.function.ThrowingIntFunction) CursorContext(org.neo4j.io.pagecache.context.CursorContext) IndexType(org.neo4j.internal.schema.IndexType) Value(org.neo4j.values.storable.Value) ArrayList(java.util.ArrayList) IntObjectMap(org.eclipse.collections.api.map.primitive.IntObjectMap) InlineNodeLabels(org.neo4j.kernel.impl.store.InlineNodeLabels) NodeLabelsField(org.neo4j.kernel.impl.store.NodeLabelsField) TokenHolder(org.neo4j.token.api.TokenHolder) MutableLongCollection(org.eclipse.collections.api.collection.primitive.MutableLongCollection) NeoStores(org.neo4j.kernel.impl.store.NeoStores) LongHashSet(org.eclipse.collections.impl.set.mutable.primitive.LongHashSet) PROPERTY_CURSOR(org.neo4j.internal.recordstorage.RecordCursorTypes.PROPERTY_CURSOR) BiConsumer(java.util.function.BiConsumer) AbstractBaseRecord(org.neo4j.kernel.impl.store.record.AbstractBaseRecord) ConsistencyReport(org.neo4j.consistency.report.ConsistencyReport) TokenRecord(org.neo4j.kernel.impl.store.record.TokenRecord) Math.toIntExact(java.lang.Math.toIntExact) NODE_CURSOR(org.neo4j.internal.recordstorage.RecordCursorTypes.NODE_CURSOR) NO_VALUE(org.neo4j.values.storable.Values.NO_VALUE) MutableIntCollection(org.eclipse.collections.api.collection.primitive.MutableIntCollection) MutablePrimitiveObjectMap(org.eclipse.collections.api.map.primitive.MutablePrimitiveObjectMap) NULL_REFERENCE(org.neo4j.kernel.impl.store.record.Record.NULL_REFERENCE) PropertyRecord(org.neo4j.kernel.impl.store.record.PropertyRecord) RecordType(org.neo4j.consistency.RecordType) StoreCursors(org.neo4j.storageengine.api.cursor.StoreCursors) SchemaComplianceChecker.isValueSupportedByIndex(org.neo4j.consistency.checker.SchemaComplianceChecker.isValueSupportedByIndex) RELATIONSHIP_CURSOR(org.neo4j.internal.recordstorage.RecordCursorTypes.RELATIONSHIP_CURSOR) TokenStore(org.neo4j.kernel.impl.store.TokenStore) String.format(java.lang.String.format) GROUP_CURSOR(org.neo4j.internal.recordstorage.RecordCursorTypes.GROUP_CURSOR) DynamicRecord(org.neo4j.kernel.impl.store.record.DynamicRecord) MutableLongSet(org.eclipse.collections.api.set.primitive.MutableLongSet) Consumer(java.util.function.Consumer) DynamicNodeLabels(org.neo4j.kernel.impl.store.DynamicNodeLabels) RecordStore(org.neo4j.kernel.impl.store.RecordStore) List(java.util.List) PrimitiveRecord(org.neo4j.kernel.impl.store.record.PrimitiveRecord) RelationshipGroupRecord(org.neo4j.kernel.impl.store.record.RelationshipGroupRecord) TokenNotFoundException(org.neo4j.token.api.TokenNotFoundException) SchemaDescriptor(org.neo4j.internal.schema.SchemaDescriptor) NamedToken(org.neo4j.token.api.NamedToken) DynamicStringStore(org.neo4j.kernel.impl.store.DynamicStringStore) NodeRecord(org.neo4j.kernel.impl.store.record.NodeRecord) PropertySchemaType(org.neo4j.internal.schema.PropertySchemaType) RecordLoad(org.neo4j.kernel.impl.store.record.RecordLoad) RelationshipRecord(org.neo4j.kernel.impl.store.record.RelationshipRecord) DynamicStringStore(org.neo4j.kernel.impl.store.DynamicStringStore) ArrayList(java.util.ArrayList) LongHashSet(org.eclipse.collections.impl.set.mutable.primitive.LongHashSet) MutableLongSet(org.eclipse.collections.api.set.primitive.MutableLongSet) NamedToken(org.neo4j.token.api.NamedToken)

Example 7 with DynamicStringStore

use of org.neo4j.kernel.impl.store.DynamicStringStore in project neo4j by neo4j.

the class SchemaChecker method checkTokens.

private static <R extends TokenRecord> void checkTokens(TokenStore<R> store, Function<R, ConsistencyReport.NameConsistencyReport> report, Function<DynamicRecord, ConsistencyReport.DynamicConsistencyReport> dynamicRecordReport, PageCacheTracer pageCacheTracer) {
    DynamicStringStore nameStore = store.getNameStore();
    DynamicRecord nameRecord = nameStore.newRecord();
    long highId = store.getHighId();
    MutableLongSet seenNameRecordIds = LongSets.mutable.empty();
    int blockSize = store.getNameStore().getRecordDataSize();
    try (var cursorContext = new CursorContext(pageCacheTracer.createPageCursorTracer(CONSISTENCY_TOKEN_CHECKER_TAG));
        RecordReader<R> tokenReader = new RecordReader<>(store, true, cursorContext);
        RecordReader<DynamicRecord> nameReader = new RecordReader<>(store.getNameStore(), false, cursorContext)) {
        for (long id = 0; id < highId; id++) {
            R record = tokenReader.read(id);
            if (record.inUse() && !NULL_REFERENCE.is(record.getNameId())) {
                safeLoadDynamicRecordChain(r -> {
                }, nameReader, seenNameRecordIds, record.getNameId(), blockSize, (i, r) -> dynamicRecordReport.apply(nameRecord).circularReferenceNext(r), (i, r) -> report.apply(record).nameBlockNotInUse(nameRecord), (i, r) -> dynamicRecordReport.apply(nameRecord).nextNotInUse(r), (i, r) -> dynamicRecordReport.apply(r).emptyBlock(), r -> dynamicRecordReport.apply(r).recordNotFullReferencesNext(), r -> dynamicRecordReport.apply(r).invalidLength());
            }
        }
    }
}
Also used : DynamicRecord(org.neo4j.kernel.impl.store.record.DynamicRecord) MutableLongSet(org.eclipse.collections.api.set.primitive.MutableLongSet) DynamicStringStore(org.neo4j.kernel.impl.store.DynamicStringStore) CursorContext(org.neo4j.io.pagecache.context.CursorContext)

Example 8 with DynamicStringStore

use of org.neo4j.kernel.impl.store.DynamicStringStore in project neo4j by neo4j.

the class FullCheckIntegrationTest method shouldReportRelationshipLabelNameInconsistencies.

@Test
void shouldReportRelationshipLabelNameInconsistencies() throws Exception {
    // given
    final Reference<Integer> inconsistentName = new Reference<>();
    fixture.apply(new GraphStoreFixture.Transaction() {

        @Override
        protected void transactionData(GraphStoreFixture.TransactionDataBuilder tx, GraphStoreFixture.IdGenerator next) {
            inconsistentName.set(next.relationshipType());
            tx.relationshipType(inconsistentName.get(), "FOO", false);
        }
    });
    NeoStores neoStores = fixture.directStoreAccess().nativeStores();
    StoreCursors storeCursors = fixture.getStoreCursors();
    DynamicStringStore nameStore = neoStores.getRelationshipTypeTokenStore().getNameStore();
    DynamicRecord record = nameStore.newRecord();
    nameStore.getRecordByCursor(inconsistentName.get(), record, FORCE, storeCursors.readCursor(DYNAMIC_REL_TYPE_TOKEN_CURSOR));
    record.setNextBlock(record.getId());
    try (var storeCursor = storeCursors.writeCursor(DYNAMIC_REL_TYPE_TOKEN_CURSOR)) {
        nameStore.updateRecord(record, storeCursor, NULL, storeCursors);
    }
    // when
    ConsistencySummaryStatistics stats = check();
    // then
    on(stats).verify(RecordType.RELATIONSHIP_TYPE_NAME, 1).andThatsAllFolks();
}
Also used : DynamicRecord(org.neo4j.kernel.impl.store.record.DynamicRecord) DynamicStringStore(org.neo4j.kernel.impl.store.DynamicStringStore) AtomicReference(java.util.concurrent.atomic.AtomicReference) TransactionDataBuilder(org.neo4j.consistency.checking.GraphStoreFixture.TransactionDataBuilder) IdGenerator(org.neo4j.consistency.checking.GraphStoreFixture.IdGenerator) NeoStores(org.neo4j.kernel.impl.store.NeoStores) StoreCursors(org.neo4j.storageengine.api.cursor.StoreCursors) GraphStoreFixture(org.neo4j.consistency.checking.GraphStoreFixture) ConsistencySummaryStatistics(org.neo4j.consistency.report.ConsistencySummaryStatistics) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) Test(org.junit.jupiter.api.Test)

Example 9 with DynamicStringStore

use of org.neo4j.kernel.impl.store.DynamicStringStore in project neo4j by neo4j.

the class FullCheckIntegrationTest method shouldReportPropertyKeyInconsistencies.

@Test
void shouldReportPropertyKeyInconsistencies() throws Exception {
    // given
    final Reference<int[]> propertyKeyNameIds = new Reference<>();
    fixture.apply(new GraphStoreFixture.Transaction() {

        @Override
        protected void transactionData(GraphStoreFixture.TransactionDataBuilder tx, GraphStoreFixture.IdGenerator next) {
            int[] nameIds = tx.propertyKey(next.propertyKey(), "FOO", false);
            propertyKeyNameIds.set(nameIds);
        }
    });
    NeoStores neoStores = fixture.directStoreAccess().nativeStores();
    StoreCursors storeCursors = fixture.getStoreCursors();
    DynamicStringStore nameStore = neoStores.getPropertyKeyTokenStore().getNameStore();
    DynamicRecord record = nameStore.newRecord();
    nameStore.getRecordByCursor(propertyKeyNameIds.get()[0], record, FORCE, storeCursors.readCursor(DYNAMIC_PROPERTY_KEY_TOKEN_CURSOR));
    record.setInUse(false);
    try (var storeCursor = storeCursors.writeCursor(DYNAMIC_PROPERTY_KEY_TOKEN_CURSOR)) {
        nameStore.updateRecord(record, storeCursor, NULL, storeCursors);
    }
    // when
    ConsistencySummaryStatistics stats = check();
    // then
    on(stats).verify(RecordType.PROPERTY_KEY, 1).andThatsAllFolks();
}
Also used : DynamicRecord(org.neo4j.kernel.impl.store.record.DynamicRecord) DynamicStringStore(org.neo4j.kernel.impl.store.DynamicStringStore) AtomicReference(java.util.concurrent.atomic.AtomicReference) TransactionDataBuilder(org.neo4j.consistency.checking.GraphStoreFixture.TransactionDataBuilder) IdGenerator(org.neo4j.consistency.checking.GraphStoreFixture.IdGenerator) NeoStores(org.neo4j.kernel.impl.store.NeoStores) StoreCursors(org.neo4j.storageengine.api.cursor.StoreCursors) GraphStoreFixture(org.neo4j.consistency.checking.GraphStoreFixture) ConsistencySummaryStatistics(org.neo4j.consistency.report.ConsistencySummaryStatistics) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) Test(org.junit.jupiter.api.Test)

Example 10 with DynamicStringStore

use of org.neo4j.kernel.impl.store.DynamicStringStore in project neo4j by neo4j.

the class FullCheckIntegrationTest method shouldReportPropertyKeyNameInconsistencies.

@Test
void shouldReportPropertyKeyNameInconsistencies() throws Exception {
    // given
    final Reference<int[]> propertyKeyNameIds = new Reference<>();
    fixture.apply(new GraphStoreFixture.Transaction() {

        @Override
        protected void transactionData(GraphStoreFixture.TransactionDataBuilder tx, GraphStoreFixture.IdGenerator next) {
            int[] dynamicIds = tx.propertyKey(next.propertyKey(), "FOO", false);
            propertyKeyNameIds.set(dynamicIds);
        }
    });
    NeoStores neoStores = fixture.directStoreAccess().nativeStores();
    StoreCursors storeCursors = fixture.getStoreCursors();
    DynamicStringStore nameStore = neoStores.getPropertyKeyTokenStore().getNameStore();
    DynamicRecord record = nameStore.newRecord();
    nameStore.getRecordByCursor(propertyKeyNameIds.get()[0], record, FORCE, storeCursors.readCursor(DYNAMIC_PROPERTY_KEY_TOKEN_CURSOR));
    record.setNextBlock(record.getId());
    try (var storeCursor = storeCursors.writeCursor(DYNAMIC_PROPERTY_KEY_TOKEN_CURSOR)) {
        nameStore.updateRecord(record, storeCursor, NULL, storeCursors);
    }
    // when
    ConsistencySummaryStatistics stats = check();
    // then
    on(stats).verify(RecordType.PROPERTY_KEY_NAME, 1).andThatsAllFolks();
}
Also used : DynamicRecord(org.neo4j.kernel.impl.store.record.DynamicRecord) DynamicStringStore(org.neo4j.kernel.impl.store.DynamicStringStore) AtomicReference(java.util.concurrent.atomic.AtomicReference) TransactionDataBuilder(org.neo4j.consistency.checking.GraphStoreFixture.TransactionDataBuilder) IdGenerator(org.neo4j.consistency.checking.GraphStoreFixture.IdGenerator) NeoStores(org.neo4j.kernel.impl.store.NeoStores) StoreCursors(org.neo4j.storageengine.api.cursor.StoreCursors) GraphStoreFixture(org.neo4j.consistency.checking.GraphStoreFixture) ConsistencySummaryStatistics(org.neo4j.consistency.report.ConsistencySummaryStatistics) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) Test(org.junit.jupiter.api.Test)

Aggregations

DynamicStringStore (org.neo4j.kernel.impl.store.DynamicStringStore)13 DynamicRecord (org.neo4j.kernel.impl.store.record.DynamicRecord)10 NeoStores (org.neo4j.kernel.impl.store.NeoStores)7 StoreCursors (org.neo4j.storageengine.api.cursor.StoreCursors)5 DynamicArrayStore (org.neo4j.kernel.impl.store.DynamicArrayStore)4 AtomicReference (java.util.concurrent.atomic.AtomicReference)3 Test (org.junit.jupiter.api.Test)3 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)3 GraphStoreFixture (org.neo4j.consistency.checking.GraphStoreFixture)3 IdGenerator (org.neo4j.consistency.checking.GraphStoreFixture.IdGenerator)3 TransactionDataBuilder (org.neo4j.consistency.checking.GraphStoreFixture.TransactionDataBuilder)3 ConsistencySummaryStatistics (org.neo4j.consistency.report.ConsistencySummaryStatistics)3 ArrayList (java.util.ArrayList)2 List (java.util.List)2 MutableLongSet (org.eclipse.collections.api.set.primitive.MutableLongSet)2 CursorContext (org.neo4j.io.pagecache.context.CursorContext)2 NodeStore (org.neo4j.kernel.impl.store.NodeStore)2 PropertyStore (org.neo4j.kernel.impl.store.PropertyStore)2 RecordCursors (org.neo4j.kernel.impl.store.RecordCursors)2 RelationshipGroupStore (org.neo4j.kernel.impl.store.RelationshipGroupStore)2