Search in sources :

Example 11 with NodeStore

use of org.neo4j.kernel.impl.store.NodeStore in project neo4j by neo4j.

the class StoreMigrator method legacyNodesAsInput.

private InputIterable<InputNode> legacyNodesAsInput(NeoStores legacyStore, boolean requiresPropertyMigration, RecordCursors cursors) {
    NodeStore store = legacyStore.getNodeStore();
    final BiConsumer<InputNode, NodeRecord> propertyDecorator = propertyDecorator(requiresPropertyMigration, cursors);
    return new StoreScanAsInputIterable<InputNode, NodeRecord>(store) {

        @Override
        protected InputNode inputEntityOf(NodeRecord record) {
            InputNode node = new InputNode("legacy store", record.getId(), record.getId() * NodeRecordFormat.RECORD_SIZE, record.getId(), InputEntity.NO_PROPERTIES, record.getNextProp(), InputNode.NO_LABELS, record.getLabelField());
            propertyDecorator.accept(node, record);
            return node;
        }
    };
}
Also used : InputNode(org.neo4j.unsafe.impl.batchimport.input.InputNode) NodeRecord(org.neo4j.kernel.impl.store.record.NodeRecord) NodeStore(org.neo4j.kernel.impl.store.NodeStore)

Example 12 with NodeStore

use of org.neo4j.kernel.impl.store.NodeStore in project neo4j by neo4j.

the class PropertyDeduplicator method deduplicateProperties.

public void deduplicateProperties() throws IOException {
    StoreFactory factory = new StoreFactory(workingDir, pageCache, fileSystem, NullLogProvider.getInstance());
    try (NeoStores neoStores = factory.openNeoStores(StoreType.PROPERTY, StoreType.NODE, StoreType.SCHEMA)) {
        PropertyStore propertyStore = neoStores.getPropertyStore();
        NodeStore nodeStore = neoStores.getNodeStore();
        SchemaStore schemaStore = neoStores.getSchemaStore();
        PrimitiveLongObjectMap<List<DuplicateCluster>> duplicateClusters = collectConflictingProperties(propertyStore);
        resolveConflicts(duplicateClusters, propertyStore, nodeStore, schemaStore, neoStores.getStoreDir());
    }
}
Also used : NodeStore(org.neo4j.kernel.impl.store.NodeStore) SchemaStore(org.neo4j.kernel.impl.store.SchemaStore) NeoStores(org.neo4j.kernel.impl.store.NeoStores) ArrayList(java.util.ArrayList) List(java.util.List) StoreFactory(org.neo4j.kernel.impl.store.StoreFactory) PropertyStore(org.neo4j.kernel.impl.store.PropertyStore)

Example 13 with NodeStore

use of org.neo4j.kernel.impl.store.NodeStore in project neo4j by neo4j.

the class BatchingMultipleIndexPopulatorTest method populateFromQueuePopulatesWhenThresholdReached.

@Test
public void populateFromQueuePopulatesWhenThresholdReached() throws Exception {
    setProperty(QUEUE_THRESHOLD_NAME, 2);
    NeoStores neoStores = mock(NeoStores.class);
    NodeStore nodeStore = mock(NodeStore.class);
    when(neoStores.getNodeStore()).thenReturn(nodeStore);
    NeoStoreIndexStoreView storeView = new NeoStoreIndexStoreView(LockService.NO_LOCK_SERVICE, neoStores);
    BatchingMultipleIndexPopulator batchingPopulator = new BatchingMultipleIndexPopulator(storeView, mock(ExecutorService.class), NullLogProvider.getInstance());
    IndexPopulator populator1 = addPopulator(batchingPopulator, index1);
    IndexUpdater updater1 = mock(IndexUpdater.class);
    when(populator1.newPopulatingUpdater(any())).thenReturn(updater1);
    IndexPopulator populator2 = addPopulator(batchingPopulator, index42);
    IndexUpdater updater2 = mock(IndexUpdater.class);
    when(populator2.newPopulatingUpdater(any())).thenReturn(updater2);
    batchingPopulator.indexAllNodes();
    IndexEntryUpdate update1 = IndexEntryUpdate.add(1, index1.schema(), "foo");
    IndexEntryUpdate update2 = IndexEntryUpdate.add(2, index42.schema(), "bar");
    IndexEntryUpdate update3 = IndexEntryUpdate.add(3, index1.schema(), "baz");
    batchingPopulator.queue(update1);
    batchingPopulator.queue(update2);
    batchingPopulator.queue(update3);
    batchingPopulator.populateFromQueue(42);
    verify(updater1).process(update1);
    verify(updater1).process(update3);
    verify(updater2).process(update2);
}
Also used : NeoStoreIndexStoreView(org.neo4j.kernel.impl.transaction.state.storeview.NeoStoreIndexStoreView) IndexPopulator(org.neo4j.kernel.api.index.IndexPopulator) IndexEntryUpdate(org.neo4j.kernel.api.index.IndexEntryUpdate) NodeStore(org.neo4j.kernel.impl.store.NodeStore) NeoStores(org.neo4j.kernel.impl.store.NeoStores) ExecutorService(java.util.concurrent.ExecutorService) IndexUpdater(org.neo4j.kernel.api.index.IndexUpdater) Test(org.junit.Test)

Example 14 with NodeStore

use of org.neo4j.kernel.impl.store.NodeStore in project neo4j by neo4j.

the class MultipleIndexPopulatorUpdatesTest method updateForHigherNodeIgnoredWhenUsingFullNodeStoreScan.

@Test
public void updateForHigherNodeIgnoredWhenUsingFullNodeStoreScan() throws IndexPopulationFailedKernelException, IOException, IndexEntryConflictException {
    NeoStores neoStores = Mockito.mock(NeoStores.class);
    CountsTracker countsTracker = mock(CountsTracker.class);
    NodeStore nodeStore = mock(NodeStore.class);
    PropertyStore propertyStore = mock(PropertyStore.class);
    NodeRecord nodeRecord = getNodeRecord();
    PropertyRecord propertyRecord = getPropertyRecord();
    when(neoStores.getCounts()).thenReturn(countsTracker);
    when(neoStores.getNodeStore()).thenReturn(nodeStore);
    when(neoStores.getPropertyStore()).thenReturn(propertyStore);
    when(propertyStore.getPropertyRecordChain(anyInt())).thenReturn(Collections.singletonList(propertyRecord));
    when(countsTracker.nodeCount(anyInt(), any(Register.DoubleLongRegister.class))).thenReturn(Registers.newDoubleLongRegister(3, 3));
    when(nodeStore.getHighestPossibleIdInUse()).thenReturn(20L);
    when(nodeStore.newRecord()).thenReturn(nodeRecord);
    when(nodeStore.getRecord(anyInt(), eq(nodeRecord), any(RecordLoad.class))).thenAnswer(new SetNodeIdRecordAnswer(nodeRecord, 1));
    when(nodeStore.getRecord(eq(7L), eq(nodeRecord), any(RecordLoad.class))).thenAnswer(new SetNodeIdRecordAnswer(nodeRecord, 7));
    ProcessListenableNeoStoreIndexView storeView = new ProcessListenableNeoStoreIndexView(LockService.NO_LOCK_SERVICE, neoStores);
    MultipleIndexPopulator indexPopulator = new MultipleIndexPopulator(storeView, logProvider);
    storeView.setProcessListener(new NodeUpdateProcessListener(indexPopulator));
    IndexPopulator populator = createIndexPopulator();
    IndexUpdater indexUpdater = mock(IndexUpdater.class);
    when(populator.newPopulatingUpdater(storeView)).thenReturn(indexUpdater);
    addPopulator(indexPopulator, populator, 1, NewIndexDescriptorFactory.forLabel(1, 1));
    indexPopulator.create();
    StoreScan<IndexPopulationFailedKernelException> storeScan = indexPopulator.indexAllNodes();
    storeScan.run();
    Mockito.verify(indexUpdater, times(0)).process(any(IndexEntryUpdate.class));
}
Also used : IndexEntryUpdate(org.neo4j.kernel.api.index.IndexEntryUpdate) IndexPopulationFailedKernelException(org.neo4j.kernel.api.exceptions.index.IndexPopulationFailedKernelException) RecordLoad(org.neo4j.kernel.impl.store.record.RecordLoad) NodeRecord(org.neo4j.kernel.impl.store.record.NodeRecord) IndexPopulator(org.neo4j.kernel.api.index.IndexPopulator) NodeStore(org.neo4j.kernel.impl.store.NodeStore) PropertyRecord(org.neo4j.kernel.impl.store.record.PropertyRecord) NeoStores(org.neo4j.kernel.impl.store.NeoStores) CountsTracker(org.neo4j.kernel.impl.store.counts.CountsTracker) IndexUpdater(org.neo4j.kernel.api.index.IndexUpdater) PropertyStore(org.neo4j.kernel.impl.store.PropertyStore) Test(org.junit.Test)

Example 15 with NodeStore

use of org.neo4j.kernel.impl.store.NodeStore in project neo4j by neo4j.

the class CountsComputerTest method rebuildCounts.

private void rebuildCounts(long lastCommittedTransactionId) throws IOException {
    cleanupCountsForRebuilding();
    StoreFactory storeFactory = new StoreFactory(dir, pageCache, fs, NullLogProvider.getInstance());
    try (Lifespan life = new Lifespan();
        NeoStores neoStores = storeFactory.openAllNeoStores()) {
        NodeStore nodeStore = neoStores.getNodeStore();
        RelationshipStore relationshipStore = neoStores.getRelationshipStore();
        int highLabelId = (int) neoStores.getLabelTokenStore().getHighId();
        int highRelationshipTypeId = (int) neoStores.getRelationshipTypeTokenStore().getHighId();
        CountsComputer countsComputer = new CountsComputer(lastCommittedTransactionId, nodeStore, relationshipStore, highLabelId, highRelationshipTypeId);
        CountsTracker countsTracker = createCountsTracker();
        life.add(countsTracker.setInitializer(countsComputer));
    }
}
Also used : NodeStore(org.neo4j.kernel.impl.store.NodeStore) CountsComputer(org.neo4j.kernel.impl.store.CountsComputer) NeoStores(org.neo4j.kernel.impl.store.NeoStores) RelationshipStore(org.neo4j.kernel.impl.store.RelationshipStore) StoreFactory(org.neo4j.kernel.impl.store.StoreFactory) Lifespan(org.neo4j.kernel.lifecycle.Lifespan)

Aggregations

NodeStore (org.neo4j.kernel.impl.store.NodeStore)19 Test (org.junit.Test)10 NodeRecord (org.neo4j.kernel.impl.store.record.NodeRecord)10 NeoStores (org.neo4j.kernel.impl.store.NeoStores)9 PropertyStore (org.neo4j.kernel.impl.store.PropertyStore)4 RelationshipStore (org.neo4j.kernel.impl.store.RelationshipStore)3 StoreFactory (org.neo4j.kernel.impl.store.StoreFactory)3 File (java.io.File)2 ArrayList (java.util.ArrayList)2 Collection (java.util.Collection)2 ConsistencyReport (org.neo4j.consistency.report.ConsistencyReport)2 IndexEntryUpdate (org.neo4j.kernel.api.index.IndexEntryUpdate)2 IndexPopulator (org.neo4j.kernel.api.index.IndexPopulator)2 IndexUpdater (org.neo4j.kernel.api.index.IndexUpdater)2 CountsComputer (org.neo4j.kernel.impl.store.CountsComputer)2 DynamicArrayStore (org.neo4j.kernel.impl.store.DynamicArrayStore)2 DynamicRecordAllocator (org.neo4j.kernel.impl.store.DynamicRecordAllocator)2 DynamicStringStore (org.neo4j.kernel.impl.store.DynamicStringStore)2 InlineNodeLabels (org.neo4j.kernel.impl.store.InlineNodeLabels)2 NodeLabels (org.neo4j.kernel.impl.store.NodeLabels)2