Search in sources :

Example 16 with IndexNotFoundKernelException

use of org.neo4j.internal.kernel.api.exceptions.schema.IndexNotFoundKernelException in project neo4j by neo4j.

the class NativeIndexReader method createSampler.

@Override
public IndexSampler createSampler() {
    // For a unique index there's an optimization, knowing that all values in it are unique, to simply count
    // the number of indexed values and create a sample for that count. The GBPTree doesn't have an O(1)
    // count mechanism, it will have to manually count the indexed values in it to get it.
    // For that reason this implementation opts for keeping complexity down by just using the existing
    // non-unique sampler which scans the index and counts (potentially duplicates, of which there will
    // be none in a unique index).
    FullScanNonUniqueIndexSampler<KEY, VALUE> sampler = new FullScanNonUniqueIndexSampler<>(tree, layout);
    return tracer -> {
        try {
            return sampler.sample(tracer);
        } catch (UncheckedIOException e) {
            if (getRootCause(e) instanceof FileIsNotMappedException) {
                IndexNotFoundKernelException exception = new IndexNotFoundKernelException("Index dropped while sampling.");
                exception.addSuppressed(e);
                throw exception;
            }
            throw e;
        }
    };
}
Also used : IndexOrder(org.neo4j.internal.schema.IndexOrder) ValueIndexReader(org.neo4j.kernel.api.index.ValueIndexReader) Seeker(org.neo4j.index.internal.gbptree.Seeker) CursorContext(org.neo4j.io.pagecache.context.CursorContext) FileIsNotMappedException(org.neo4j.io.pagecache.impl.FileIsNotMappedException) ExceptionUtils.getRootCause(org.apache.commons.lang3.exception.ExceptionUtils.getRootCause) IndexQueryConstraints(org.neo4j.internal.kernel.api.IndexQueryConstraints) IOException(java.io.IOException) Value(org.neo4j.values.storable.Value) UncheckedIOException(java.io.UncheckedIOException) GBPTree(org.neo4j.index.internal.gbptree.GBPTree) IndexProgressor(org.neo4j.kernel.api.index.IndexProgressor) IndexNotFoundKernelException(org.neo4j.internal.kernel.api.exceptions.schema.IndexNotFoundKernelException) IndexSampler(org.neo4j.kernel.api.index.IndexSampler) PropertyIndexQuery(org.neo4j.internal.kernel.api.PropertyIndexQuery) QueryContext(org.neo4j.internal.kernel.api.QueryContext) IndexDescriptor(org.neo4j.internal.schema.IndexDescriptor) NEUTRAL(org.neo4j.kernel.impl.index.schema.NativeIndexKey.Inclusion.NEUTRAL) FileIsNotMappedException(org.neo4j.io.pagecache.impl.FileIsNotMappedException) UncheckedIOException(java.io.UncheckedIOException) IndexNotFoundKernelException(org.neo4j.internal.kernel.api.exceptions.schema.IndexNotFoundKernelException)

Example 17 with IndexNotFoundKernelException

use of org.neo4j.internal.kernel.api.exceptions.schema.IndexNotFoundKernelException in project neo4j by neo4j.

the class BuiltInProcedures method getIndexStatus.

private static IndexStatus getIndexStatus(SchemaReadCore schemaRead, IndexDescriptor index) {
    IndexStatus status = new IndexStatus();
    try {
        InternalIndexState internalIndexState = schemaRead.indexGetState(index);
        status.state = internalIndexState.toString();
        PopulationProgress progress = schemaRead.indexGetPopulationProgress(index);
        status.populationProgress = progress.toIndexPopulationProgress().getCompletedPercentage();
        status.failureMessage = internalIndexState == InternalIndexState.FAILED ? schemaRead.indexGetFailure(index) : "";
    } catch (IndexNotFoundKernelException e) {
        status.state = "NOT FOUND";
        status.populationProgress = 0D;
        status.failureMessage = "Index not found. It might have been concurrently dropped.";
    }
    return status;
}
Also used : InternalIndexState(org.neo4j.internal.kernel.api.InternalIndexState) PopulationProgress(org.neo4j.internal.kernel.api.PopulationProgress) IndexNotFoundKernelException(org.neo4j.internal.kernel.api.exceptions.schema.IndexNotFoundKernelException)

Example 18 with IndexNotFoundKernelException

use of org.neo4j.internal.kernel.api.exceptions.schema.IndexNotFoundKernelException in project neo4j by neo4j.

the class SchemaStatementProcedure method includeConstraint.

private static boolean includeConstraint(SchemaReadCore schemaRead, ConstraintDescriptor constraint) {
    // - Owned index must have this constraint as owning constraint
    if (constraint.isIndexBackedConstraint()) {
        IndexBackedConstraintDescriptor indexBackedConstraint = constraint.asIndexBackedConstraint();
        if (indexBackedConstraint.hasOwnedIndexId()) {
            IndexDescriptor backingIndex = schemaRead.indexGetForName(constraint.getName());
            if (backingIndex.getId() == indexBackedConstraint.ownedIndexId()) {
                try {
                    InternalIndexState internalIndexState = schemaRead.indexGetState(backingIndex);
                    OptionalLong owningConstraintId = backingIndex.getOwningConstraintId();
                    return internalIndexState == InternalIndexState.ONLINE && owningConstraintId.orElse(-1) == constraint.getId();
                } catch (IndexNotFoundKernelException e) {
                    return false;
                }
            }
        }
        return false;
    }
    return true;
}
Also used : InternalIndexState(org.neo4j.internal.kernel.api.InternalIndexState) IndexBackedConstraintDescriptor(org.neo4j.internal.schema.constraints.IndexBackedConstraintDescriptor) OptionalLong(java.util.OptionalLong) IndexNotFoundKernelException(org.neo4j.internal.kernel.api.exceptions.schema.IndexNotFoundKernelException) IndexDescriptor(org.neo4j.internal.schema.IndexDescriptor)

Example 19 with IndexNotFoundKernelException

use of org.neo4j.internal.kernel.api.exceptions.schema.IndexNotFoundKernelException in project neo4j by neo4j.

the class GraphCountsSection method indexes.

private static List<Map<String, Object>> indexes(TokenRead tokens, SchemaRead schemaRead, Anonymizer anonymizer) throws IndexNotFoundKernelException {
    List<Map<String, Object>> indexes = new ArrayList<>();
    Iterator<IndexDescriptor> iterator = schemaRead.indexesGetAll();
    while (iterator.hasNext()) {
        IndexDescriptor index = iterator.next();
        IndexType indexType = index.getIndexType();
        if (indexType == IndexType.FULLTEXT) {
            /* For full text indexes, we currently do not return its options, which makes returning information on
                 * this index not useful and if the index type is ignored, this would even be misleading.
                 */
            continue;
        }
        EntityType entityType = index.schema().entityType();
        Map<String, Object> data = new HashMap<>();
        switch(entityType) {
            case NODE:
                data.put("labels", map(index.schema().getEntityTokenIds(), id -> anonymizer.label(tokens.labelGetName(id), id)));
                break;
            case RELATIONSHIP:
                data.put("relationshipTypes", map(index.schema().getEntityTokenIds(), id -> anonymizer.relationshipType(tokens.relationshipTypeGetName(id), id)));
                break;
            default:
        }
        data.put("properties", map(index.schema().getPropertyIds(), id -> anonymizer.propertyKey(tokens.propertyKeyGetName(id), id)));
        var indexSample = schemaRead.indexSample(index);
        data.put("totalSize", indexSample.indexSize());
        data.put("updatesSinceEstimation", indexSample.updates());
        data.put("estimatedUniqueSize", indexSample.uniqueValues());
        data.put("indexType", indexType.name());
        indexes.add(data);
    }
    return indexes;
}
Also used : Arrays(java.util.Arrays) Iterator(java.util.Iterator) Read(org.neo4j.internal.kernel.api.Read) Iterators(org.neo4j.internal.helpers.collection.Iterators) IndexType(org.neo4j.internal.schema.IndexType) TokenRead(org.neo4j.internal.kernel.api.TokenRead) HashMap(java.util.HashMap) LoginContext(org.neo4j.internal.kernel.api.security.LoginContext) Kernel(org.neo4j.kernel.api.Kernel) ConstraintDescriptor(org.neo4j.internal.schema.ConstraintDescriptor) Collectors(java.util.stream.Collectors) ArrayList(java.util.ArrayList) List(java.util.List) SchemaRead(org.neo4j.internal.kernel.api.SchemaRead) Stream(java.util.stream.Stream) IndexNotFoundKernelException(org.neo4j.internal.kernel.api.exceptions.schema.IndexNotFoundKernelException) EntityType(org.neo4j.common.EntityType) Map(java.util.Map) KernelTransaction(org.neo4j.kernel.api.KernelTransaction) NamedToken(org.neo4j.token.api.NamedToken) IndexDescriptor(org.neo4j.internal.schema.IndexDescriptor) TransactionFailureException(org.neo4j.internal.kernel.api.exceptions.TransactionFailureException) IntFunction(java.util.function.IntFunction) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) IndexDescriptor(org.neo4j.internal.schema.IndexDescriptor) EntityType(org.neo4j.common.EntityType) IndexType(org.neo4j.internal.schema.IndexType) HashMap(java.util.HashMap) Map(java.util.Map)

Example 20 with IndexNotFoundKernelException

use of org.neo4j.internal.kernel.api.exceptions.schema.IndexNotFoundKernelException in project neo4j by neo4j.

the class IndexingService method start.

// Recovery semantics: This is to be called after init, and after the database has run recovery.
@Override
public void start() throws Exception {
    state = State.STARTING;
    // Recovery will not do refresh (update read views) while applying recovered transactions and instead
    // do it at one point after recovery... i.e. here
    indexMapRef.indexMapSnapshot().forEachIndexProxy(indexProxyOperation("refresh", IndexProxy::refresh));
    final MutableLongObjectMap<IndexDescriptor> rebuildingDescriptors = new LongObjectHashMap<>();
    indexMapRef.modify(indexMap -> {
        Map<InternalIndexState, List<IndexLogRecord>> indexStates = new EnumMap<>(InternalIndexState.class);
        Map<IndexProviderDescriptor, List<IndexLogRecord>> indexProviders = new HashMap<>();
        // Find all indexes that are not already online, do not require rebuilding, and create them
        indexMap.forEachIndexProxy((indexId, proxy) -> {
            InternalIndexState state = proxy.getState();
            IndexDescriptor descriptor = proxy.getDescriptor();
            IndexProviderDescriptor providerDescriptor = descriptor.getIndexProvider();
            IndexLogRecord indexLogRecord = new IndexLogRecord(descriptor);
            indexStates.computeIfAbsent(state, internalIndexState -> new ArrayList<>()).add(indexLogRecord);
            indexProviders.computeIfAbsent(providerDescriptor, indexProviderDescriptor -> new ArrayList<>()).add(indexLogRecord);
            internalLog.debug(indexStateInfo("start", state, descriptor));
            switch(state) {
                case ONLINE:
                case FAILED:
                    proxy.start();
                    break;
                case POPULATING:
                    // Remember for rebuilding right below in this method
                    rebuildingDescriptors.put(indexId, descriptor);
                    break;
                default:
                    throw new IllegalStateException("Unknown state: " + state);
            }
        });
        logIndexStateSummary("start", indexStates);
        logIndexProviderSummary(indexProviders);
        dontRebuildIndexesInReadOnlyMode(rebuildingDescriptors);
        // Drop placeholder proxies for indexes that need to be rebuilt
        dropRecoveringIndexes(indexMap, rebuildingDescriptors.keySet());
        // Rebuild indexes by recreating and repopulating them
        populateIndexesOfAllTypes(rebuildingDescriptors, indexMap);
        return indexMap;
    });
    indexStatisticsStore.start();
    samplingController.recoverIndexSamples();
    samplingController.start();
    // So at this point we've started population of indexes that needs to be rebuilt in the background.
    // Indexes backing uniqueness constraints are normally built within the transaction creating the constraint
    // and so we shouldn't leave such indexes in a populating state after recovery.
    // This is why we now go and wait for those indexes to be fully populated.
    rebuildingDescriptors.forEachKeyValue((indexId, index) -> {
        if (!index.isUnique()) {
            // It's not a uniqueness constraint, so don't wait for it to be rebuilt
            return;
        }
        IndexProxy proxy;
        try {
            proxy = getIndexProxy(index);
        } catch (IndexNotFoundKernelException e) {
            throw new IllegalStateException("What? This index was seen during recovery just now, why isn't it available now?", e);
        }
        if (proxy.getDescriptor().getOwningConstraintId().isEmpty()) {
            // so there's no gain in waiting for this index.
            return;
        }
        monitor.awaitingPopulationOfRecoveredIndex(index);
        awaitOnlineAfterRecovery(proxy);
    });
    state = State.RUNNING;
}
Also used : LifecycleAdapter(org.neo4j.kernel.lifecycle.LifecycleAdapter) Arrays(java.util.Arrays) ResourceIterator(org.neo4j.graphdb.ResourceIterator) Log(org.neo4j.logging.Log) CursorContext(org.neo4j.io.pagecache.context.CursorContext) TokenNameLookup(org.neo4j.common.TokenNameLookup) IndexPopulator(org.neo4j.kernel.api.index.IndexPopulator) LongObjectProcedure(org.eclipse.collections.api.block.procedure.primitive.LongObjectProcedure) UnaryOperator(java.util.function.UnaryOperator) Config(org.neo4j.configuration.Config) Value(org.neo4j.values.storable.Value) Preconditions(org.neo4j.util.Preconditions) IndexStatisticsStore(org.neo4j.kernel.impl.api.index.stats.IndexStatisticsStore) UnderlyingStorageException(org.neo4j.exceptions.UnderlyingStorageException) IndexUpdater(org.neo4j.kernel.api.index.IndexUpdater) IndexNotFoundKernelException(org.neo4j.internal.kernel.api.exceptions.schema.IndexNotFoundKernelException) IndexPopulationFailedKernelException(org.neo4j.kernel.api.exceptions.index.IndexPopulationFailedKernelException) LongHashSet(org.eclipse.collections.impl.set.mutable.primitive.LongHashSet) Map(java.util.Map) LongIterable(org.eclipse.collections.api.LongIterable) PageCacheTracer(org.neo4j.io.pagecache.tracing.PageCacheTracer) IndexProviderDescriptor(org.neo4j.internal.schema.IndexProviderDescriptor) Path(java.nio.file.Path) EnumMap(java.util.EnumMap) SYSTEM(org.neo4j.common.Subject.SYSTEM) Collection(java.util.Collection) Set(java.util.Set) IndexEntryUpdate(org.neo4j.storageengine.api.IndexEntryUpdate) Collectors(java.util.stream.Collectors) String.format(java.lang.String.format) UncheckedIOException(java.io.UncheckedIOException) List(java.util.List) EntityType(org.neo4j.common.EntityType) IndexPrototype(org.neo4j.internal.schema.IndexPrototype) FAILED(org.neo4j.internal.kernel.api.InternalIndexState.FAILED) IndexDescriptor(org.neo4j.internal.schema.IndexDescriptor) MutableBoolean(org.apache.commons.lang3.mutable.MutableBoolean) DatabaseReadOnlyChecker(org.neo4j.configuration.helpers.DatabaseReadOnlyChecker) NODE(org.neo4j.common.EntityType.NODE) Iterators.asResourceIterator(org.neo4j.internal.helpers.collection.Iterators.asResourceIterator) GraphDatabaseSettings(org.neo4j.configuration.GraphDatabaseSettings) InternalIndexState(org.neo4j.internal.kernel.api.InternalIndexState) IndexSamplingController(org.neo4j.kernel.impl.api.index.sampling.IndexSamplingController) LogProvider(org.neo4j.logging.LogProvider) HashMap(java.util.HashMap) MutableLongObjectMap(org.eclipse.collections.api.map.primitive.MutableLongObjectMap) Iterators.iterator(org.neo4j.internal.helpers.collection.Iterators.iterator) IndexProvider(org.neo4j.kernel.api.index.IndexProvider) ArrayList(java.util.ArrayList) IndexEntryConflictException(org.neo4j.kernel.api.exceptions.index.IndexEntryConflictException) LongObjectHashMap(org.eclipse.collections.impl.map.mutable.primitive.LongObjectHashMap) ThrowingConsumer(org.neo4j.function.ThrowingConsumer) POPULATING(org.neo4j.internal.kernel.api.InternalIndexState.POPULATING) IndexPopulationFailure.failure(org.neo4j.kernel.impl.api.index.IndexPopulationFailure.failure) ONLINE(org.neo4j.internal.kernel.api.InternalIndexState.ONLINE) IndexStoreViewFactory(org.neo4j.kernel.impl.transaction.state.storeview.IndexStoreViewFactory) JobScheduler(org.neo4j.scheduler.JobScheduler) MemoryTracker(org.neo4j.memory.MemoryTracker) NodePropertyAccessor(org.neo4j.storageengine.api.NodePropertyAccessor) IndexUpdateListener(org.neo4j.storageengine.api.IndexUpdateListener) Subject(org.neo4j.common.Subject) Iterators(org.neo4j.internal.helpers.collection.Iterators) IndexSamplingMode(org.neo4j.kernel.impl.api.index.sampling.IndexSamplingMode) Iterables.asList(org.neo4j.internal.helpers.collection.Iterables.asList) IOException(java.io.IOException) IndexActivationFailedKernelException(org.neo4j.kernel.api.exceptions.index.IndexActivationFailedKernelException) RELATIONSHIP(org.neo4j.common.EntityType.RELATIONSHIP) MutableLongSet(org.eclipse.collections.api.set.primitive.MutableLongSet) TimeUnit(java.util.concurrent.TimeUnit) KernelException(org.neo4j.exceptions.KernelException) StringJoiner(java.util.StringJoiner) UniquePropertyValueValidationException(org.neo4j.kernel.api.exceptions.schema.UniquePropertyValueValidationException) SchemaState(org.neo4j.internal.schema.SchemaState) LongSet(org.eclipse.collections.api.set.primitive.LongSet) LongObjectHashMap(org.eclipse.collections.impl.map.mutable.primitive.LongObjectHashMap) HashMap(java.util.HashMap) LongObjectHashMap(org.eclipse.collections.impl.map.mutable.primitive.LongObjectHashMap) IndexProviderDescriptor(org.neo4j.internal.schema.IndexProviderDescriptor) ArrayList(java.util.ArrayList) IndexNotFoundKernelException(org.neo4j.internal.kernel.api.exceptions.schema.IndexNotFoundKernelException) IndexDescriptor(org.neo4j.internal.schema.IndexDescriptor) InternalIndexState(org.neo4j.internal.kernel.api.InternalIndexState) List(java.util.List) ArrayList(java.util.ArrayList) Iterables.asList(org.neo4j.internal.helpers.collection.Iterables.asList) EnumMap(java.util.EnumMap)

Aggregations

IndexNotFoundKernelException (org.neo4j.internal.kernel.api.exceptions.schema.IndexNotFoundKernelException)21 IndexDescriptor (org.neo4j.internal.schema.IndexDescriptor)13 InternalIndexState (org.neo4j.internal.kernel.api.InternalIndexState)5 ArrayList (java.util.ArrayList)4 Test (org.junit.jupiter.api.Test)4 CursorContext (org.neo4j.io.pagecache.context.CursorContext)4 IndexSampler (org.neo4j.kernel.api.index.IndexSampler)4 IOException (java.io.IOException)3 SchemaRead (org.neo4j.internal.kernel.api.SchemaRead)3 IndexProxy (org.neo4j.kernel.impl.api.index.IndexProxy)3 UncheckedIOException (java.io.UncheckedIOException)2 Arrays (java.util.Arrays)2 HashMap (java.util.HashMap)2 List (java.util.List)2 Map (java.util.Map)2 Collectors (java.util.stream.Collectors)2 Test (org.junit.Test)2 EntityType (org.neo4j.common.EntityType)2 Iterators (org.neo4j.internal.helpers.collection.Iterators)2 IndexEntryConflictException (org.neo4j.kernel.api.exceptions.index.IndexEntryConflictException)2