Search in sources :

Example 16 with IndexEntryConflictException

use of org.neo4j.kernel.api.exceptions.index.IndexEntryConflictException in project neo4j by neo4j.

the class IndexUpdaterMap method close.

@Override
public void close() throws UnderlyingStorageException {
    Set<Pair<IndexDescriptor, UnderlyingStorageException>> exceptions = null;
    for (Map.Entry<IndexDescriptor, IndexUpdater> updaterEntry : updaterMap.entrySet()) {
        IndexUpdater updater = updaterEntry.getValue();
        try {
            updater.close();
        } catch (UncheckedIOException | IndexEntryConflictException e) {
            if (null == exceptions) {
                exceptions = new HashSet<>();
            }
            exceptions.add(Pair.of(updaterEntry.getKey(), new UnderlyingStorageException(e)));
        }
    }
    clear();
    if (null != exceptions) {
        throw new MultipleUnderlyingStorageExceptions(exceptions);
    }
}
Also used : UncheckedIOException(java.io.UncheckedIOException) IndexDescriptor(org.neo4j.internal.schema.IndexDescriptor) UnderlyingStorageException(org.neo4j.exceptions.UnderlyingStorageException) Map(java.util.Map) HashMap(java.util.HashMap) IndexEntryConflictException(org.neo4j.kernel.api.exceptions.index.IndexEntryConflictException) IndexUpdater(org.neo4j.kernel.api.index.IndexUpdater) Pair(org.neo4j.internal.helpers.collection.Pair) HashSet(java.util.HashSet) MultipleUnderlyingStorageExceptions(org.neo4j.kernel.impl.store.MultipleUnderlyingStorageExceptions)

Example 17 with IndexEntryConflictException

use of org.neo4j.kernel.api.exceptions.index.IndexEntryConflictException in project neo4j by neo4j.

the class ContractCheckingIndexProxyTest method closeWaitForUpdateToFinish.

@Test
void closeWaitForUpdateToFinish() throws InterruptedException {
    // GIVEN
    CountDownLatch latch = new CountDownLatch(1);
    final IndexProxy inner = new IndexProxyAdapter() {

        @Override
        public IndexUpdater newUpdater(IndexUpdateMode mode, CursorContext cursorContext) {
            return super.newUpdater(mode, cursorContext);
        }
    };
    final IndexProxy outer = newContractCheckingIndexProxy(inner);
    Thread actionThread = createActionThread(() -> outer.close(NULL));
    outer.start();
    // WHEN
    Thread updaterThread = runInSeparateThread(() -> {
        try (IndexUpdater updater = outer.newUpdater(IndexUpdateMode.ONLINE, NULL)) {
            updater.process((ValueIndexEntryUpdate<?>) null);
            try {
                actionThread.start();
                latch.await();
            } catch (InterruptedException e) {
                throw new RuntimeException(e);
            }
        } catch (IndexEntryConflictException e) {
            throw new RuntimeException(e);
        }
    });
    ThreadTestUtils.awaitThreadState(actionThread, TEST_TIMEOUT, Thread.State.TIMED_WAITING);
    latch.countDown();
    updaterThread.join();
    actionThread.join();
}
Also used : SchemaIndexTestHelper.mockIndexProxy(org.neo4j.kernel.impl.api.index.SchemaIndexTestHelper.mockIndexProxy) CursorContext(org.neo4j.io.pagecache.context.CursorContext) CountDownLatch(java.util.concurrent.CountDownLatch) IndexEntryConflictException(org.neo4j.kernel.api.exceptions.index.IndexEntryConflictException) IndexUpdater(org.neo4j.kernel.api.index.IndexUpdater) Test(org.junit.jupiter.api.Test)

Example 18 with IndexEntryConflictException

use of org.neo4j.kernel.api.exceptions.index.IndexEntryConflictException in project neo4j by neo4j.

the class SimpleUniquenessVerifier method verify.

@Override
public void verify(NodePropertyAccessor accessor, int[] propKeyIds, List<Value[]> updatedValueTuples) throws IndexEntryConflictException, IOException {
    try {
        DuplicateCheckingCollector collector = DuplicateCheckingCollector.forProperties(accessor, propKeyIds);
        for (Value[] valueTuple : updatedValueTuples) {
            collector.init();
            Query query = LuceneDocumentStructure.newSeekQuery(valueTuple);
            indexSearcher().search(query, collector);
        }
    } catch (IOException e) {
        Throwable cause = e.getCause();
        if (cause instanceof IndexEntryConflictException) {
            throw (IndexEntryConflictException) cause;
        }
        throw e;
    }
}
Also used : Query(org.apache.lucene.search.Query) TermQuery(org.apache.lucene.search.TermQuery) Value(org.neo4j.values.storable.Value) IOException(java.io.IOException) IndexEntryConflictException(org.neo4j.kernel.api.exceptions.index.IndexEntryConflictException)

Example 19 with IndexEntryConflictException

use of org.neo4j.kernel.api.exceptions.index.IndexEntryConflictException in project neo4j by neo4j.

the class SimpleUniquenessVerifier method verify.

@Override
public void verify(NodePropertyAccessor accessor, int[] propKeyIds) throws IndexEntryConflictException, IOException {
    try {
        DuplicateCheckingCollector collector = DuplicateCheckingCollector.forProperties(accessor, propKeyIds);
        IndexSearcher searcher = indexSearcher();
        for (LeafReaderContext leafReaderContext : searcher.getIndexReader().leaves()) {
            LeafReader leafReader = leafReaderContext.reader();
            for (FieldInfo fieldInfo : leafReader.getFieldInfos()) {
                String field = fieldInfo.name;
                if (LuceneDocumentStructure.useFieldForUniquenessVerification(field)) {
                    TermsEnum terms = leafReader.terms(field).iterator();
                    BytesRef termsRef;
                    while ((termsRef = terms.next()) != null) {
                        if (terms.docFreq() > 1) {
                            collector.init(terms.docFreq());
                            searcher.search(new TermQuery(new Term(field, termsRef)), collector);
                        }
                    }
                }
            }
        }
    } catch (IOException e) {
        Throwable cause = e.getCause();
        if (cause instanceof IndexEntryConflictException) {
            throw (IndexEntryConflictException) cause;
        }
        throw e;
    }
}
Also used : IndexSearcher(org.apache.lucene.search.IndexSearcher) TermQuery(org.apache.lucene.search.TermQuery) LeafReader(org.apache.lucene.index.LeafReader) Term(org.apache.lucene.index.Term) IOException(java.io.IOException) TermsEnum(org.apache.lucene.index.TermsEnum) LeafReaderContext(org.apache.lucene.index.LeafReaderContext) IndexEntryConflictException(org.neo4j.kernel.api.exceptions.index.IndexEntryConflictException) FieldInfo(org.apache.lucene.index.FieldInfo) BytesRef(org.apache.lucene.util.BytesRef)

Example 20 with IndexEntryConflictException

use of org.neo4j.kernel.api.exceptions.index.IndexEntryConflictException in project neo4j by neo4j.

the class NativeIndexAccessorTests method getValues.

@Test
void getValues() throws IndexEntryConflictException, IndexNotApplicableKernelException {
    // given
    int nUpdates = 10000;
    Iterator<ValueIndexEntryUpdate<IndexDescriptor>> randomUpdateGenerator = valueCreatorUtil.randomUpdateGenerator(random);
    // noinspection unchecked
    ValueIndexEntryUpdate<IndexDescriptor>[] someUpdates = new ValueIndexEntryUpdate[nUpdates];
    for (int i = 0; i < nUpdates; i++) {
        someUpdates[i] = randomUpdateGenerator.next();
    }
    processAll(someUpdates);
    Value[] allValues = ValueCreatorUtil.extractValuesFromUpdates(someUpdates);
    // Pick one out of all added values and do a range query for the value group of that value
    Value value = random.among(allValues);
    ValueGroup valueGroup = value.valueGroup();
    IndexValueCapability valueCapability = indexCapability().valueCapability(valueGroup.category());
    if (!valueCapability.equals(IndexValueCapability.YES)) {
        // We don't need to do this test
        return;
    }
    PropertyIndexQuery.RangePredicate<?> supportedQuery;
    List<Value> expectedValues;
    if (Values.isGeometryValue(value)) {
        // Unless it's a point value in which case we query for the specific coordinate reference system instead
        CoordinateReferenceSystem crs = ((PointValue) value).getCoordinateReferenceSystem();
        supportedQuery = PropertyIndexQuery.range(0, crs);
        expectedValues = Arrays.stream(allValues).filter(v -> v.valueGroup() == ValueGroup.GEOMETRY).filter(v -> ((PointValue) v).getCoordinateReferenceSystem() == crs).collect(Collectors.toList());
    } else {
        supportedQuery = PropertyIndexQuery.range(0, valueGroup);
        expectedValues = Arrays.stream(allValues).filter(v -> v.valueGroup() == valueGroup).collect(Collectors.toList());
    }
    // when
    try (var reader = accessor.newValueReader()) {
        SimpleEntityValueClient client = new SimpleEntityValueClient();
        reader.query(NULL_CONTEXT, client, unorderedValues(), supportedQuery);
        // then
        while (client.next()) {
            Value foundValue = client.values[0];
            assertTrue(expectedValues.remove(foundValue), "found value that was not expected " + foundValue);
        }
        assertThat(expectedValues.size()).as("did not find all expected values").isEqualTo(0);
    }
}
Also used : BeforeEach(org.junit.jupiter.api.BeforeEach) IndexOrder(org.neo4j.internal.schema.IndexOrder) Arrays(java.util.Arrays) ValueIndexReader(org.neo4j.kernel.api.index.ValueIndexReader) SimpleEntityValueClient(org.neo4j.storageengine.api.schema.SimpleEntityValueClient) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) Iterators.filter(org.neo4j.internal.helpers.collection.Iterators.filter) NULL_CONTEXT(org.neo4j.internal.kernel.api.QueryContext.NULL_CONTEXT) Value(org.neo4j.values.storable.Value) RandomValues(org.neo4j.values.storable.RandomValues) IndexUpdater(org.neo4j.kernel.api.index.IndexUpdater) Assertions.assertFalse(org.junit.jupiter.api.Assertions.assertFalse) NULL(org.neo4j.io.pagecache.context.CursorContext.NULL) IndexEntryUpdate.remove(org.neo4j.storageengine.api.IndexEntryUpdate.remove) Predicates.alwaysTrue(org.neo4j.function.Predicates.alwaysTrue) EMPTY_LONG_ARRAY(org.neo4j.collection.PrimitiveLongCollections.EMPTY_LONG_ARRAY) Predicate(java.util.function.Predicate) Collection(java.util.Collection) Set(java.util.Set) TestDirectory(org.neo4j.test.rule.TestDirectory) Collectors(java.util.stream.Collectors) PointValue(org.neo4j.values.storable.PointValue) String.format(java.lang.String.format) ValueIndexEntryUpdate(org.neo4j.storageengine.api.ValueIndexEntryUpdate) Test(org.junit.jupiter.api.Test) IndexValueCapability(org.neo4j.internal.schema.IndexValueCapability) IndexQueryConstraints.unconstrained(org.neo4j.internal.kernel.api.IndexQueryConstraints.unconstrained) List(java.util.List) Stream(java.util.stream.Stream) Assertions.assertTrue(org.junit.jupiter.api.Assertions.assertTrue) IndexNotApplicableKernelException(org.neo4j.internal.kernel.api.exceptions.schema.IndexNotApplicableKernelException) PropertyIndexQuery(org.neo4j.internal.kernel.api.PropertyIndexQuery) IndexDescriptor(org.neo4j.internal.schema.IndexDescriptor) ValueCreatorUtil.countUniqueValues(org.neo4j.kernel.impl.index.schema.ValueCreatorUtil.countUniqueValues) Assertions.assertThrows(org.junit.jupiter.api.Assertions.assertThrows) IndexCapability(org.neo4j.internal.schema.IndexCapability) IndexDirectoryStructure.directoriesByProvider(org.neo4j.kernel.api.index.IndexDirectoryStructure.directoriesByProvider) Iterables.asUniqueSet(org.neo4j.internal.helpers.collection.Iterables.asUniqueSet) IndexQueryConstraints.unorderedValues(org.neo4j.internal.kernel.api.IndexQueryConstraints.unorderedValues) IndexEntryUpdate.change(org.neo4j.storageengine.api.IndexEntryUpdate.change) Predicates.in(org.neo4j.function.Predicates.in) ArrayList(java.util.ArrayList) Values(org.neo4j.values.storable.Values) HashSet(java.util.HashSet) IndexEntryConflictException(org.neo4j.kernel.api.exceptions.index.IndexEntryConflictException) Assertions.assertThatThrownBy(org.assertj.core.api.Assertions.assertThatThrownBy) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) IndexSample(org.neo4j.kernel.api.index.IndexSample) Iterator(java.util.Iterator) ValueType(org.neo4j.values.storable.ValueType) IndexOrderCapability(org.neo4j.internal.schema.IndexOrderCapability) IndexQueryConstraints(org.neo4j.internal.kernel.api.IndexQueryConstraints) IndexDirectoryStructure(org.neo4j.kernel.api.index.IndexDirectoryStructure) IndexQueryConstraints.constrained(org.neo4j.internal.kernel.api.IndexQueryConstraints.constrained) PrimitiveLongCollections(org.neo4j.collection.PrimitiveLongCollections) Assertions.assertArrayEquals(org.junit.jupiter.api.Assertions.assertArrayEquals) LongIterator(org.eclipse.collections.api.iterator.LongIterator) IndexProgressor(org.neo4j.kernel.api.index.IndexProgressor) Values.of(org.neo4j.values.storable.Values.of) CoordinateReferenceSystem(org.neo4j.values.storable.CoordinateReferenceSystem) IndexSampler(org.neo4j.kernel.api.index.IndexSampler) ValueGroup(org.neo4j.values.storable.ValueGroup) Collections(java.util.Collections) ONLINE(org.neo4j.kernel.impl.api.index.IndexUpdateMode.ONLINE) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) PointValue(org.neo4j.values.storable.PointValue) ValueGroup(org.neo4j.values.storable.ValueGroup) ValueIndexEntryUpdate(org.neo4j.storageengine.api.ValueIndexEntryUpdate) PropertyIndexQuery(org.neo4j.internal.kernel.api.PropertyIndexQuery) Value(org.neo4j.values.storable.Value) PointValue(org.neo4j.values.storable.PointValue) CoordinateReferenceSystem(org.neo4j.values.storable.CoordinateReferenceSystem) IndexValueCapability(org.neo4j.internal.schema.IndexValueCapability) SimpleEntityValueClient(org.neo4j.storageengine.api.schema.SimpleEntityValueClient) Test(org.junit.jupiter.api.Test)

Aggregations

IndexEntryConflictException (org.neo4j.kernel.api.exceptions.index.IndexEntryConflictException)21 IOException (java.io.IOException)9 Test (org.junit.jupiter.api.Test)5 IndexDescriptor (org.neo4j.internal.schema.IndexDescriptor)5 UniquePropertyValueValidationException (org.neo4j.kernel.api.exceptions.schema.UniquePropertyValueValidationException)5 TermQuery (org.apache.lucene.search.TermQuery)4 IndexUpdater (org.neo4j.kernel.api.index.IndexUpdater)4 NewIndexDescriptor (org.neo4j.kernel.api.schema_new.index.NewIndexDescriptor)4 ArrayList (java.util.ArrayList)3 Arrays (java.util.Arrays)3 Collection (java.util.Collection)3 List (java.util.List)3 Test (org.junit.Test)3 Collections (java.util.Collections)2 HashMap (java.util.HashMap)2 HashSet (java.util.HashSet)2 Iterator (java.util.Iterator)2 Map (java.util.Map)2 Set (java.util.Set)2 Query (org.apache.lucene.search.Query)2