Search in sources :

Example 1 with IndexStoreView

use of org.neo4j.kernel.impl.api.index.IndexStoreView in project neo4j by neo4j.

the class GraphStoreFixture method directStoreAccess.

public DirectStoreAccess directStoreAccess() {
    if (directStoreAccess == null) {
        fileSystem = new DefaultFileSystemAbstraction();
        PageCache pageCache = getPageCache(fileSystem);
        LogProvider logProvider = NullLogProvider.getInstance();
        StoreFactory storeFactory = new StoreFactory(directory, pageCache, fileSystem, logProvider);
        neoStore = storeFactory.openAllNeoStores();
        StoreAccess nativeStores;
        if (keepStatistics) {
            AccessStatistics accessStatistics = new AccessStatistics();
            statistics = new VerboseStatistics(accessStatistics, new DefaultCounts(defaultConsistencyCheckThreadsNumber()), NullLog.getInstance());
            nativeStores = new AccessStatsKeepingStoreAccess(neoStore, accessStatistics);
        } else {
            statistics = Statistics.NONE;
            nativeStores = new StoreAccess(neoStore);
        }
        nativeStores.initialize();
        Config config = Config.empty();
        OperationalMode operationalMode = OperationalMode.single;
        IndexStoreView indexStoreView = new NeoStoreIndexStoreView(LockService.NO_LOCK_SERVICE, nativeStores.getRawNeoStores());
        Dependencies dependencies = new Dependencies();
        dependencies.satisfyDependencies(Config.defaults(), fileSystem, new SimpleLogService(logProvider, logProvider), indexStoreView, pageCache);
        KernelContext kernelContext = new SimpleKernelContext(directory, UNKNOWN, dependencies);
        LabelScanStore labelScanStore = startLabelScanStore(config, dependencies, kernelContext);
        directStoreAccess = new DirectStoreAccess(nativeStores, labelScanStore, createIndexes(fileSystem, config, operationalMode));
    }
    return directStoreAccess;
}
Also used : LabelScanStore(org.neo4j.kernel.api.labelscan.LabelScanStore) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) AccessStatsKeepingStoreAccess(org.neo4j.consistency.statistics.AccessStatsKeepingStoreAccess) DirectStoreAccess(org.neo4j.kernel.api.direct.DirectStoreAccess) StoreAccess(org.neo4j.kernel.impl.store.StoreAccess) SimpleLogService(org.neo4j.kernel.impl.logging.SimpleLogService) Config(org.neo4j.kernel.configuration.Config) DirectStoreAccess(org.neo4j.kernel.api.direct.DirectStoreAccess) AccessStatsKeepingStoreAccess(org.neo4j.consistency.statistics.AccessStatsKeepingStoreAccess) StoreFactory(org.neo4j.kernel.impl.store.StoreFactory) OperationalMode(org.neo4j.kernel.impl.factory.OperationalMode) NullLogProvider(org.neo4j.logging.NullLogProvider) FormattedLogProvider(org.neo4j.logging.FormattedLogProvider) LogProvider(org.neo4j.logging.LogProvider) NeoStoreIndexStoreView(org.neo4j.kernel.impl.transaction.state.storeview.NeoStoreIndexStoreView) DefaultCounts(org.neo4j.consistency.statistics.DefaultCounts) AccessStatistics(org.neo4j.consistency.statistics.AccessStatistics) IndexStoreView(org.neo4j.kernel.impl.api.index.IndexStoreView) NeoStoreIndexStoreView(org.neo4j.kernel.impl.transaction.state.storeview.NeoStoreIndexStoreView) SimpleKernelContext(org.neo4j.kernel.impl.spi.SimpleKernelContext) VerboseStatistics(org.neo4j.consistency.statistics.VerboseStatistics) Dependencies(org.neo4j.kernel.impl.util.Dependencies) PageCache(org.neo4j.io.pagecache.PageCache) SimpleKernelContext(org.neo4j.kernel.impl.spi.SimpleKernelContext) KernelContext(org.neo4j.kernel.impl.spi.KernelContext)

Example 2 with IndexStoreView

use of org.neo4j.kernel.impl.api.index.IndexStoreView in project neo4j by neo4j.

the class ConsistencyCheckService method runFullConsistencyCheck.

public Result runFullConsistencyCheck(final File storeDir, Config config, ProgressMonitorFactory progressFactory, final LogProvider logProvider, final FileSystemAbstraction fileSystem, final PageCache pageCache, final boolean verbose, File reportDir, CheckConsistencyConfig checkConsistencyConfig) throws ConsistencyCheckIncompleteException {
    Log log = logProvider.getLog(getClass());
    config = config.with(stringMap(GraphDatabaseSettings.read_only.name(), TRUE, GraphDatabaseSettings.label_index.name(), LabelIndex.AUTO.name()));
    StoreFactory factory = new StoreFactory(storeDir, config, new DefaultIdGeneratorFactory(fileSystem), pageCache, fileSystem, logProvider);
    ConsistencySummaryStatistics summary;
    final File reportFile = chooseReportPath(reportDir);
    Log reportLog = new ConsistencyReportLog(Suppliers.lazySingleton(() -> {
        try {
            return new PrintWriter(createOrOpenAsOuputStream(fileSystem, reportFile, true));
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }));
    // Bootstrap kernel extensions
    LifeSupport life = new LifeSupport();
    try (NeoStores neoStores = factory.openAllNeoStores()) {
        IndexStoreView indexStoreView = new NeoStoreIndexStoreView(LockService.NO_LOCK_SERVICE, neoStores);
        Dependencies dependencies = new Dependencies();
        dependencies.satisfyDependencies(config, fileSystem, new SimpleLogService(logProvider, logProvider), indexStoreView, pageCache);
        KernelContext kernelContext = new SimpleKernelContext(storeDir, UNKNOWN, dependencies);
        KernelExtensions extensions = life.add(new KernelExtensions(kernelContext, (Iterable) load(KernelExtensionFactory.class), dependencies, ignore()));
        life.start();
        LabelScanStore labelScanStore = life.add(extensions.resolveDependency(LabelScanStoreProvider.class, new NamedLabelScanStoreSelectionStrategy(config)).getLabelScanStore());
        SchemaIndexProvider indexes = life.add(extensions.resolveDependency(SchemaIndexProvider.class, HighestSelectionStrategy.getInstance()));
        int numberOfThreads = defaultConsistencyCheckThreadsNumber();
        Statistics statistics;
        StoreAccess storeAccess;
        AccessStatistics stats = new AccessStatistics();
        if (verbose) {
            statistics = new VerboseStatistics(stats, new DefaultCounts(numberOfThreads), log);
            storeAccess = new AccessStatsKeepingStoreAccess(neoStores, stats);
        } else {
            statistics = Statistics.NONE;
            storeAccess = new StoreAccess(neoStores);
        }
        storeAccess.initialize();
        DirectStoreAccess stores = new DirectStoreAccess(storeAccess, labelScanStore, indexes);
        FullCheck check = new FullCheck(progressFactory, statistics, numberOfThreads, checkConsistencyConfig);
        summary = check.execute(stores, new DuplicatingLog(log, reportLog));
    } finally {
        life.shutdown();
    }
    if (!summary.isConsistent()) {
        log.warn("See '%s' for a detailed consistency report.", reportFile.getPath());
        return Result.failure(reportFile);
    }
    return Result.success(reportFile);
}
Also used : LabelScanStore(org.neo4j.kernel.api.labelscan.LabelScanStore) AccessStatsKeepingStoreAccess(org.neo4j.consistency.statistics.AccessStatsKeepingStoreAccess) DirectStoreAccess(org.neo4j.kernel.api.direct.DirectStoreAccess) StoreAccess(org.neo4j.kernel.impl.store.StoreAccess) SimpleLogService(org.neo4j.kernel.impl.logging.SimpleLogService) DirectStoreAccess(org.neo4j.kernel.api.direct.DirectStoreAccess) StoreFactory(org.neo4j.kernel.impl.store.StoreFactory) DefaultCounts(org.neo4j.consistency.statistics.DefaultCounts) AccessStatistics(org.neo4j.consistency.statistics.AccessStatistics) SimpleKernelContext(org.neo4j.kernel.impl.spi.SimpleKernelContext) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) DuplicatingLog(org.neo4j.logging.DuplicatingLog) VerboseStatistics(org.neo4j.consistency.statistics.VerboseStatistics) Dependencies(org.neo4j.kernel.impl.util.Dependencies) NamedLabelScanStoreSelectionStrategy(org.neo4j.kernel.extension.dependency.NamedLabelScanStoreSelectionStrategy) ConsistencySummaryStatistics(org.neo4j.consistency.report.ConsistencySummaryStatistics) PrintWriter(java.io.PrintWriter) SimpleKernelContext(org.neo4j.kernel.impl.spi.SimpleKernelContext) KernelContext(org.neo4j.kernel.impl.spi.KernelContext) SchemaIndexProvider(org.neo4j.kernel.api.index.SchemaIndexProvider) Log(org.neo4j.logging.Log) DuplicatingLog(org.neo4j.logging.DuplicatingLog) DefaultIdGeneratorFactory(org.neo4j.kernel.impl.store.id.DefaultIdGeneratorFactory) AccessStatsKeepingStoreAccess(org.neo4j.consistency.statistics.AccessStatsKeepingStoreAccess) IOException(java.io.IOException) KernelExtensionFactory(org.neo4j.kernel.extension.KernelExtensionFactory) Statistics(org.neo4j.consistency.statistics.Statistics) VerboseStatistics(org.neo4j.consistency.statistics.VerboseStatistics) ConsistencySummaryStatistics(org.neo4j.consistency.report.ConsistencySummaryStatistics) AccessStatistics(org.neo4j.consistency.statistics.AccessStatistics) NeoStoreIndexStoreView(org.neo4j.kernel.impl.transaction.state.storeview.NeoStoreIndexStoreView) FullCheck(org.neo4j.consistency.checking.full.FullCheck) KernelExtensions(org.neo4j.kernel.extension.KernelExtensions) NeoStores(org.neo4j.kernel.impl.store.NeoStores) IndexStoreView(org.neo4j.kernel.impl.api.index.IndexStoreView) NeoStoreIndexStoreView(org.neo4j.kernel.impl.transaction.state.storeview.NeoStoreIndexStoreView) File(java.io.File)

Example 3 with IndexStoreView

use of org.neo4j.kernel.impl.api.index.IndexStoreView in project neo4j by neo4j.

the class FullLabelStream method applyTo.

@Override
public long applyTo(LabelScanWriter writer) throws IOException {
    // Keep the write for using it in visit
    this.writer = writer;
    IndexStoreView view = lazyIndexStoreView.get();
    StoreScan<IOException> scan = view.visitNodes(ArrayUtils.EMPTY_INT_ARRAY, ALWAYS_TRUE_INT, null, this, true);
    scan.run();
    return count;
}
Also used : IndexStoreView(org.neo4j.kernel.impl.api.index.IndexStoreView) IOException(java.io.IOException)

Example 4 with IndexStoreView

use of org.neo4j.kernel.impl.api.index.IndexStoreView in project neo4j by neo4j.

the class LabelScanViewNodeStoreScanTest method configureSamplersToNotUseOnlineSampling.

@Test
public void configureSamplersToNotUseOnlineSampling() throws Exception {
    LabelScanViewNodeStoreScan<Exception> scanViewStoreScan = getLabelScanViewStoreScan(new int[] { 1, 2 });
    IndexStoreView storeView = mock(IndexStoreView.class);
    LabelScanTestMultipleIndexPopulator indexPopulator = new LabelScanTestMultipleIndexPopulator(storeView, NullLogProvider.getInstance());
    List<MultipleIndexPopulator.IndexPopulation> populations = Arrays.asList(getPopulation(indexPopulator), getPopulation(indexPopulator));
    scanViewStoreScan.configure(populations);
    for (MultipleIndexPopulator.IndexPopulation population : populations) {
        verify(population.populator).configureSampling(false);
    }
}
Also used : MultipleIndexPopulator(org.neo4j.kernel.impl.api.index.MultipleIndexPopulator) IndexStoreView(org.neo4j.kernel.impl.api.index.IndexStoreView) Test(org.junit.Test)

Aggregations

IndexStoreView (org.neo4j.kernel.impl.api.index.IndexStoreView)4 IOException (java.io.IOException)2 AccessStatistics (org.neo4j.consistency.statistics.AccessStatistics)2 AccessStatsKeepingStoreAccess (org.neo4j.consistency.statistics.AccessStatsKeepingStoreAccess)2 DefaultCounts (org.neo4j.consistency.statistics.DefaultCounts)2 VerboseStatistics (org.neo4j.consistency.statistics.VerboseStatistics)2 DirectStoreAccess (org.neo4j.kernel.api.direct.DirectStoreAccess)2 LabelScanStore (org.neo4j.kernel.api.labelscan.LabelScanStore)2 SimpleLogService (org.neo4j.kernel.impl.logging.SimpleLogService)2 KernelContext (org.neo4j.kernel.impl.spi.KernelContext)2 SimpleKernelContext (org.neo4j.kernel.impl.spi.SimpleKernelContext)2 StoreAccess (org.neo4j.kernel.impl.store.StoreAccess)2 StoreFactory (org.neo4j.kernel.impl.store.StoreFactory)2 NeoStoreIndexStoreView (org.neo4j.kernel.impl.transaction.state.storeview.NeoStoreIndexStoreView)2 Dependencies (org.neo4j.kernel.impl.util.Dependencies)2 File (java.io.File)1 PrintWriter (java.io.PrintWriter)1 Test (org.junit.Test)1 FullCheck (org.neo4j.consistency.checking.full.FullCheck)1 ConsistencySummaryStatistics (org.neo4j.consistency.report.ConsistencySummaryStatistics)1