Search in sources :

Example 1 with PropertyAccessor

use of org.neo4j.kernel.api.index.PropertyAccessor in project neo4j by neo4j.

the class SimpleUniquenessVerifierTest method numericIndexVerificationSomeWithDuplicates.

@Test
public void numericIndexVerificationSomeWithDuplicates() throws Exception {
    List<Object> data = asList(Integer.MAX_VALUE, Long.MAX_VALUE, 42, Long.MAX_VALUE);
    PropertyAccessor propertyAccessor = newPropertyAccessor(data);
    insert(data);
    IndexSearcher indexSearcher = spy(searcherManager.acquire());
    try {
        runUniquenessVerification(propertyAccessor, indexSearcher);
        fail("Exception expected");
    } catch (Throwable t) {
        assertThat(t, instanceOf(IndexEntryConflictException.class));
    }
    verify(indexSearcher).search(any(Query.class), any(Collector.class));
}
Also used : IndexSearcher(org.apache.lucene.search.IndexSearcher) TestPropertyAccessor(org.neo4j.kernel.api.impl.index.TestPropertyAccessor) PropertyAccessor(org.neo4j.kernel.api.index.PropertyAccessor) Query(org.apache.lucene.search.Query) Collector(org.apache.lucene.search.Collector) Test(org.junit.Test)

Example 2 with PropertyAccessor

use of org.neo4j.kernel.api.index.PropertyAccessor in project neo4j by neo4j.

the class SimpleUniquenessVerifierTest method numericIndexVerificationSomePossibleDuplicates.

@Test
public void numericIndexVerificationSomePossibleDuplicates() throws Exception {
    List<Object> data = asList(42, Long.MAX_VALUE - 1, Long.MAX_VALUE);
    PropertyAccessor propertyAccessor = newPropertyAccessor(data);
    insert(data);
    IndexSearcher indexSearcher = spy(searcherManager.acquire());
    runUniquenessVerification(propertyAccessor, indexSearcher);
    verify(indexSearcher).search(any(Query.class), any(Collector.class));
}
Also used : IndexSearcher(org.apache.lucene.search.IndexSearcher) TestPropertyAccessor(org.neo4j.kernel.api.impl.index.TestPropertyAccessor) PropertyAccessor(org.neo4j.kernel.api.index.PropertyAccessor) Query(org.apache.lucene.search.Query) Collector(org.apache.lucene.search.Collector) Test(org.junit.Test)

Example 3 with PropertyAccessor

use of org.neo4j.kernel.api.index.PropertyAccessor in project neo4j by neo4j.

the class LuceneSchemaIndexUniquenessVerificationIT method verifyUniqueness.

private void verifyUniqueness(Collection<PropertyValue> data) throws IOException, IndexEntryConflictException {
    Object[] propertyValues = data.stream().map(property -> property.value).toArray();
    PropertyAccessor propertyAccessor = new TestPropertyAccessor(propertyValues);
    index.verifyUniqueness(propertyAccessor, new int[] { PROPERTY_KEY_ID });
}
Also used : IntStream(java.util.stream.IntStream) Arrays(java.util.Arrays) SchemaIndex(org.neo4j.kernel.api.impl.schema.SchemaIndex) IOUtils(org.neo4j.io.IOUtils) Exceptions(org.neo4j.helpers.Exceptions) RunWith(org.junit.runner.RunWith) Parameters(org.junit.runners.Parameterized.Parameters) Property(org.neo4j.kernel.api.properties.Property) ArrayList(java.util.ArrayList) Assert.assertThat(org.junit.Assert.assertThat) LuceneDocumentStructure(org.neo4j.kernel.api.impl.schema.LuceneDocumentStructure) Document(org.apache.lucene.document.Document) IndexEntryConflictException(org.neo4j.kernel.api.exceptions.index.IndexEntryConflictException) Randoms(org.neo4j.test.Randoms) LuceneSchemaIndexBuilder(org.neo4j.kernel.api.impl.schema.LuceneSchemaIndexBuilder) After(org.junit.After) ThreadLocalRandom(java.util.concurrent.ThreadLocalRandom) Strings(org.neo4j.helpers.Strings) Assert.fail(org.junit.Assert.fail) ArrayUtil(org.neo4j.helpers.ArrayUtil) Parameterized(org.junit.runners.Parameterized) Collectors.toSet(java.util.stream.Collectors.toSet) Before(org.junit.Before) Parameter(org.junit.runners.Parameterized.Parameter) Collection(java.util.Collection) Set(java.util.Set) NewIndexDescriptorFactory(org.neo4j.kernel.api.schema_new.index.NewIndexDescriptorFactory) TestDirectory(org.neo4j.test.rule.TestDirectory) Test(org.junit.Test) IOException(java.io.IOException) NewIndexDescriptor(org.neo4j.kernel.api.schema_new.index.NewIndexDescriptor) PropertyAccessor(org.neo4j.kernel.api.index.PropertyAccessor) Objects(java.util.Objects) Matchers.instanceOf(org.hamcrest.Matchers.instanceOf) List(java.util.List) Rule(org.junit.Rule) DefaultFileSystemRule(org.neo4j.test.rule.fs.DefaultFileSystemRule) RandomStringUtils(org.apache.commons.lang3.RandomStringUtils) PropertyAccessor(org.neo4j.kernel.api.index.PropertyAccessor)

Example 4 with PropertyAccessor

use of org.neo4j.kernel.api.index.PropertyAccessor in project neo4j by neo4j.

the class IndexIT method shouldRemoveAConstraintIndexWithoutOwnerInRecovery.

@Test
public void shouldRemoveAConstraintIndexWithoutOwnerInRecovery() throws Exception {
    // given
    PropertyAccessor propertyAccessor = mock(PropertyAccessor.class);
    ConstraintIndexCreator creator = new ConstraintIndexCreator(() -> kernel, indexingService, propertyAccessor, false);
    creator.createConstraintIndex(SchemaBoundary.map(descriptor));
    // when
    restartDb();
    // then
    ReadOperations readOperations = readOperationsInNewTransaction();
    assertEquals(emptySetOf(NewIndexDescriptor.class), asSet(readOperations.indexesGetForLabel(labelId)));
    commit();
}
Also used : ReadOperations(org.neo4j.kernel.api.ReadOperations) ConstraintIndexCreator(org.neo4j.kernel.impl.api.state.ConstraintIndexCreator) PropertyAccessor(org.neo4j.kernel.api.index.PropertyAccessor) NewIndexDescriptor(org.neo4j.kernel.api.schema_new.index.NewIndexDescriptor) Test(org.junit.Test) KernelIntegrationTest(org.neo4j.kernel.impl.api.integrationtest.KernelIntegrationTest)

Example 5 with PropertyAccessor

use of org.neo4j.kernel.api.index.PropertyAccessor in project neo4j by neo4j.

the class NeoStoreDataSource method start.

@Override
public void start() throws IOException {
    dependencies = new Dependencies();
    life = new LifeSupport();
    schemaIndexProvider = dependencyResolver.resolveDependency(SchemaIndexProvider.class, HighestSelectionStrategy.getInstance());
    labelScanStoreProvider = dependencyResolver.resolveDependency(LabelScanStoreProvider.class, new NamedLabelScanStoreSelectionStrategy(config));
    dependencyResolver.resolveDependency(LabelScanStoreProvider.class, new DeleteStoresFromOtherLabelScanStoreProviders(labelScanStoreProvider));
    IndexConfigStore indexConfigStore = new IndexConfigStore(storeDir, fs);
    dependencies.satisfyDependency(lockService);
    dependencies.satisfyDependency(indexConfigStore);
    life.add(indexConfigStore);
    // Monitor listeners
    LoggingLogFileMonitor loggingLogMonitor = new LoggingLogFileMonitor(msgLog);
    monitors.addMonitorListener(loggingLogMonitor);
    life.add(new Delegate(Lifecycles.multiple(indexProviders.values())));
    // Upgrade the store before we begin
    RecordFormats formats = selectStoreFormats(config, storeDir, fs, pageCache, logService);
    upgradeStore(formats);
    // Build all modules and their services
    StorageEngine storageEngine = null;
    try {
        UpdateableSchemaState updateableSchemaState = new KernelSchemaStateStore(logProvider);
        SynchronizedArrayIdOrderingQueue legacyIndexTransactionOrdering = new SynchronizedArrayIdOrderingQueue(20);
        storageEngine = buildStorageEngine(propertyKeyTokenHolder, labelTokens, relationshipTypeTokens, legacyIndexProviderLookup, indexConfigStore, updateableSchemaState::clear, legacyIndexTransactionOrdering);
        LogEntryReader<ReadableClosablePositionAwareChannel> logEntryReader = new VersionAwareLogEntryReader<>(storageEngine.commandReaderFactory());
        TransactionIdStore transactionIdStore = dependencies.resolveDependency(TransactionIdStore.class);
        LogVersionRepository logVersionRepository = dependencies.resolveDependency(LogVersionRepository.class);
        NeoStoreTransactionLogModule transactionLogModule = buildTransactionLogs(storeDir, config, logProvider, scheduler, fs, storageEngine, logEntryReader, legacyIndexTransactionOrdering, transactionIdStore, logVersionRepository);
        transactionLogModule.satisfyDependencies(dependencies);
        buildRecovery(fs, transactionIdStore, logVersionRepository, monitors.newMonitor(Recovery.Monitor.class), monitors.newMonitor(PositionToRecoverFrom.Monitor.class), transactionLogModule.logFiles(), startupStatistics, storageEngine, logEntryReader, transactionLogModule.logicalTransactionStore());
        // At the time of writing this comes from the storage engine (IndexStoreView)
        PropertyAccessor propertyAccessor = dependencies.resolveDependency(PropertyAccessor.class);
        final NeoStoreKernelModule kernelModule = buildKernel(transactionLogModule.transactionAppender(), dependencies.resolveDependency(IndexingService.class), storageEngine.storeReadLayer(), updateableSchemaState, dependencies.resolveDependency(LabelScanStore.class), storageEngine, indexConfigStore, transactionIdStore, availabilityGuard, clock, propertyAccessor);
        kernelModule.satisfyDependencies(dependencies);
        // Do these assignments last so that we can ensure no cyclical dependencies exist
        this.storageEngine = storageEngine;
        this.transactionLogModule = transactionLogModule;
        this.kernelModule = kernelModule;
        dependencies.satisfyDependency(this);
        dependencies.satisfyDependency(updateableSchemaState);
        dependencies.satisfyDependency(storageEngine.storeReadLayer());
        dependencies.satisfyDependency(logEntryReader);
        dependencies.satisfyDependency(storageEngine);
    } catch (Throwable e) {
        // Something unexpected happened during startup
        msgLog.warn("Exception occurred while setting up store modules. Attempting to close things down.", e);
        try {
            // Close the neostore, so that locks are released properly
            if (storageEngine != null) {
                storageEngine.forceClose();
            }
        } catch (Exception closeException) {
            msgLog.error("Couldn't close neostore after startup failure", closeException);
        }
        throw Exceptions.launderedException(e);
    }
    // NOTE: please make sure this is performed after having added everything to the life, in fact we would like
    // to perform the checkpointing as first step when the life is shutdown.
    life.add(lifecycleToTriggerCheckPointOnShutdown());
    try {
        life.start();
    } catch (Throwable e) {
        // Something unexpected happened during startup
        msgLog.warn("Exception occurred while starting the datasource. Attempting to close things down.", e);
        try {
            life.shutdown();
            // Close the neostore, so that locks are released properly
            storageEngine.forceClose();
        } catch (Exception closeException) {
            msgLog.error("Couldn't close neostore after startup failure", closeException);
        }
        throw Exceptions.launderedException(e);
    }
    /*
         * At this point recovery has completed and the datasource is ready for use. Whatever panic might have
         * happened before has been healed. So we can safely set the kernel health to ok.
         * This right now has any real effect only in the case of internal restarts (for example, after a store copy
         * in the case of HA). Standalone instances will have to be restarted by the user, as is proper for all
         * kernel panics.
         */
    databaseHealth.healed();
}
Also used : LabelScanStore(org.neo4j.kernel.api.labelscan.LabelScanStore) RecordStorageEngine(org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine) StorageEngine(org.neo4j.storageengine.api.StorageEngine) LogVersionRepository(org.neo4j.kernel.impl.transaction.log.LogVersionRepository) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) TransactionMonitor(org.neo4j.kernel.impl.transaction.TransactionMonitor) LoggingLogFileMonitor(org.neo4j.kernel.impl.transaction.log.LoggingLogFileMonitor) VisibleMigrationProgressMonitor(org.neo4j.kernel.impl.storemigration.monitoring.VisibleMigrationProgressMonitor) IndexingService(org.neo4j.kernel.impl.api.index.IndexingService) KernelSchemaStateStore(org.neo4j.kernel.impl.api.KernelSchemaStateStore) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) Dependencies(org.neo4j.kernel.impl.util.Dependencies) DeleteStoresFromOtherLabelScanStoreProviders(org.neo4j.kernel.extension.dependency.DeleteStoresFromOtherLabelScanStoreProviders) NamedLabelScanStoreSelectionStrategy(org.neo4j.kernel.extension.dependency.NamedLabelScanStoreSelectionStrategy) SchemaIndexProvider(org.neo4j.kernel.api.index.SchemaIndexProvider) LabelScanStoreProvider(org.neo4j.kernel.impl.api.scan.LabelScanStoreProvider) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) PropertyAccessor(org.neo4j.kernel.api.index.PropertyAccessor) LoggingLogFileMonitor(org.neo4j.kernel.impl.transaction.log.LoggingLogFileMonitor) IndexConfigStore(org.neo4j.kernel.impl.index.IndexConfigStore) IOException(java.io.IOException) KernelException(org.neo4j.kernel.api.exceptions.KernelException) UpdateableSchemaState(org.neo4j.kernel.impl.api.UpdateableSchemaState) RecordFormats(org.neo4j.kernel.impl.store.format.RecordFormats) SynchronizedArrayIdOrderingQueue(org.neo4j.kernel.impl.util.SynchronizedArrayIdOrderingQueue)

Aggregations

PropertyAccessor (org.neo4j.kernel.api.index.PropertyAccessor)7 Test (org.junit.Test)6 Collector (org.apache.lucene.search.Collector)3 IndexSearcher (org.apache.lucene.search.IndexSearcher)3 Query (org.apache.lucene.search.Query)3 TestPropertyAccessor (org.neo4j.kernel.api.impl.index.TestPropertyAccessor)3 IOException (java.io.IOException)2 NewIndexDescriptor (org.neo4j.kernel.api.schema_new.index.NewIndexDescriptor)2 IndexingService (org.neo4j.kernel.impl.api.index.IndexingService)2 ArrayList (java.util.ArrayList)1 Arrays (java.util.Arrays)1 Collection (java.util.Collection)1 List (java.util.List)1 Objects (java.util.Objects)1 Set (java.util.Set)1 ThreadLocalRandom (java.util.concurrent.ThreadLocalRandom)1 Collectors.toSet (java.util.stream.Collectors.toSet)1 IntStream (java.util.stream.IntStream)1 RandomStringUtils (org.apache.commons.lang3.RandomStringUtils)1 Document (org.apache.lucene.document.Document)1