Search in sources :

Example 6 with BatchingNeoStores

use of org.neo4j.internal.batchimport.store.BatchingNeoStores in project neo4j by neo4j.

the class HumanUnderstandableExecutionMonitorIT method shouldStartFromNonFirstStage.

@Test
void shouldStartFromNonFirstStage() {
    // given
    HumanUnderstandableExecutionMonitor monitor = new HumanUnderstandableExecutionMonitor(HumanUnderstandableExecutionMonitor.NO_MONITOR);
    Dependencies dependencies = new Dependencies();
    dependencies.satisfyDependency(Input.knownEstimates(10, 10, 10, 10, 10, 10, 10));
    BatchingNeoStores neoStores = mock(BatchingNeoStores.class);
    NodeStore nodeStore = mock(NodeStore.class);
    RelationshipStore relationshipStore = mock(RelationshipStore.class);
    when(neoStores.getNodeStore()).thenReturn(nodeStore);
    when(neoStores.getRelationshipStore()).thenReturn(relationshipStore);
    dependencies.satisfyDependency(neoStores);
    dependencies.satisfyDependency(IdMappers.actual());
    dependencies.satisfyDependency(mock(PageCacheArrayFactoryMonitor.class));
    dependencies.satisfyDependency(new DataStatistics(10, 10, new DataStatistics.RelationshipTypeCount[0]));
    monitor.initialize(dependencies);
    // when/then
    StageExecution execution = mock(StageExecution.class);
    when(execution.getStageName()).thenReturn(NodeDegreeCountStage.NAME);
    assertThatCode(() -> monitor.start(execution)).doesNotThrowAnyException();
}
Also used : NodeStore(org.neo4j.kernel.impl.store.NodeStore) BatchingNeoStores(org.neo4j.internal.batchimport.store.BatchingNeoStores) PageCacheArrayFactoryMonitor(org.neo4j.internal.batchimport.cache.PageCacheArrayFactoryMonitor) RelationshipStore(org.neo4j.kernel.impl.store.RelationshipStore) DataStatistics(org.neo4j.internal.batchimport.DataStatistics) Dependencies(org.neo4j.collection.Dependencies) Test(org.junit.jupiter.api.Test)

Example 7 with BatchingNeoStores

use of org.neo4j.internal.batchimport.store.BatchingNeoStores in project neo4j by neo4j.

the class ParallelBatchImporter method doImport.

@Override
public void doImport(Input input) throws IOException {
    try (BatchingNeoStores store = ImportLogic.instantiateNeoStores(fileSystem, databaseLayout, pageCacheTracer, recordFormats, config, logService, additionalInitialIds, dbConfig, jobScheduler, memoryTracker);
        ImportLogic logic = new ImportLogic(databaseLayout, store, config, dbConfig, logService, executionMonitor, recordFormats, badCollector, monitor, pageCacheTracer, indexImporterFactory, memoryTracker)) {
        store.createNew();
        logic.initialize(input);
        logic.importNodes();
        logic.prepareIdMapper();
        logic.importRelationships();
        logic.calculateNodeDegrees();
        logic.linkRelationshipsOfAllTypes();
        logic.defragmentRelationshipGroups();
        logic.buildCountsStore();
        logFilesInitializer.initializeLogFiles(databaseLayout, store.getNeoStores().getMetaDataStore(), fileSystem, BATCH_IMPORTER_CHECKPOINT);
        logic.success();
    }
}
Also used : BatchingNeoStores(org.neo4j.internal.batchimport.store.BatchingNeoStores)

Aggregations

BatchingNeoStores (org.neo4j.internal.batchimport.store.BatchingNeoStores)7 Test (org.junit.jupiter.api.Test)5 NodeStore (org.neo4j.kernel.impl.store.NodeStore)3 PageCacheArrayFactoryMonitor (org.neo4j.internal.batchimport.cache.PageCacheArrayFactoryMonitor)2 IdMapper (org.neo4j.internal.batchimport.cache.idmapping.IdMapper)2 ExecutionMonitor (org.neo4j.internal.batchimport.staging.ExecutionMonitor)2 NodeRecord (org.neo4j.kernel.impl.store.record.NodeRecord)2 Lifespan (org.neo4j.kernel.lifecycle.Lifespan)2 JobScheduler (org.neo4j.scheduler.JobScheduler)2 ThreadPoolJobScheduler (org.neo4j.test.scheduler.ThreadPoolJobScheduler)2 Dependencies (org.neo4j.collection.Dependencies)1 DataStatistics (org.neo4j.internal.batchimport.DataStatistics)1 Input (org.neo4j.internal.batchimport.input.Input)1 DefaultPageCacheTracer (org.neo4j.io.pagecache.tracing.DefaultPageCacheTracer)1 RelationshipStore (org.neo4j.kernel.impl.store.RelationshipStore)1