Examples with PopulationProgress org.neo4j.internal.kernel.api.PopulationProgress used on opensource projects

Example 11 with PopulationProgress

use of org.neo4j.internal.kernel.api.PopulationProgress in project neo4j by neo4j.

the class BlockBasedIndexPopulator method progress.

@Override
public PopulationProgress progress(PopulationProgress scanProgress) {
    // A general note on scanProgress.getTotal(). Before the scan is completed most progress parts will base their estimates on that value.
    // It is known that it may be slightly higher since it'll be based on store high-id, not the actual count.
    // This is fine, but it creates this small "jump" in the progress in the middle somewhere when it switches from scan to merge.
    // This also exists in the most basic population progress reports, but there it will be less visible since it will jump from
    // some close-to-100 percentage to 100% and ONLINE.
    // This progress report will consist of a couple of smaller parts, weighted differently based on empirically collected values.
    // The weights will not be absolutely correct in all environments, but they don't have to be either, it will just result in some
    // slices of the percentage progression range progressing at slightly different paces. However, progression of progress reporting
    // naturally fluctuates anyway due to data set and I/O etc. so this is not an actual problem.
    PopulationProgress.MultiBuilder builder = PopulationProgress.multiple();
    // Add scan progress (this one weights a bit heavier than the others)
    builder.add(scanProgress, 4);
    // Add merge progress
    if (!allScanUpdates.isEmpty()) {
        // The parts are merged in parallel so just take the first one and it will represent the whole merge progress.
        // It will be fairly accurate, but slightly off sometimes if other threads gets scheduling problems, i.e. if this part
        // finish far apart from others.
        long completed = 0;
        long total = 0;
        if (scanCompleted) {
            // We know the actual entry count to write during merge since we have been monitoring those values
            ThreadLocalBlockStorage part = first(allScanUpdates);
            completed = part.entriesMerged.get();
            total = part.totalEntriesToMerge;
        }
        builder.add(PopulationProgress.single(completed, total), 1);
    }
    // Add tree building incl. external updates
    PopulationProgress treeBuildProgress;
    if (allScanUpdates.stream().allMatch(part -> part.mergeStarted)) {
        long entryCount = allScanUpdates.stream().mapToLong(part -> part.count).sum() + externalUpdates.count();
        treeBuildProgress = PopulationProgress.single(numberOfAppliedScanUpdates.get() + numberOfAppliedExternalUpdates.get(), entryCount);
    } else {
        treeBuildProgress = PopulationProgress.NONE;
    }
    builder.add(treeBuildProgress, 2);
    return builder.build();
}

Example 12 with PopulationProgress

use of org.neo4j.internal.kernel.api.PopulationProgress in project neo4j by neo4j.

the class BuiltInProcedures method getIndexStatus.

private static IndexStatus getIndexStatus(SchemaReadCore schemaRead, IndexDescriptor index) {
    IndexStatus status = new IndexStatus();
    try {
        InternalIndexState internalIndexState = schemaRead.indexGetState(index);
        status.state = internalIndexState.toString();
        PopulationProgress progress = schemaRead.indexGetPopulationProgress(index);
        status.populationProgress = progress.toIndexPopulationProgress().getCompletedPercentage();
        status.failureMessage = internalIndexState == InternalIndexState.FAILED ? schemaRead.indexGetFailure(index) : "";
    } catch (IndexNotFoundKernelException e) {
        status.state = "NOT FOUND";
        status.populationProgress = 0D;
        status.failureMessage = "Index not found. It might have been concurrently dropped.";
    }
    return status;
}