Search in sources :

Example 21 with PropertyRecord

use of org.neo4j.kernel.impl.store.record.PropertyRecord in project neo4j by neo4j.

the class PropertyRecordCheckTest method shouldNotReportAnythingForPropertyRecordNotInUse.

@Test
public void shouldNotReportAnythingForPropertyRecordNotInUse() throws Exception {
    // given
    PropertyRecord property = notInUse(new PropertyRecord(42));
    // when
    ConsistencyReport.PropertyConsistencyReport report = check(property);
    // then
    verifyNoMoreInteractions(report);
}
Also used : PropertyRecord(org.neo4j.kernel.impl.store.record.PropertyRecord) ConsistencyReport(org.neo4j.consistency.report.ConsistencyReport) Test(org.junit.Test)

Example 22 with PropertyRecord

use of org.neo4j.kernel.impl.store.record.PropertyRecord in project neo4j by neo4j.

the class TransactionRecordState method extractCommands.

@Override
public void extractCommands(Collection<StorageCommand> commands) throws TransactionFailureException {
    assert !prepared : "Transaction has already been prepared";
    integrityValidator.validateTransactionStartKnowledge(lastCommittedTxWhenTransactionStarted);
    int noOfCommands = recordChangeSet.changeSize() + (neoStoreRecord != null ? neoStoreRecord.changeSize() : 0);
    for (RecordProxy<Integer, LabelTokenRecord, Void> record : recordChangeSet.getLabelTokenChanges().changes()) {
        commands.add(new Command.LabelTokenCommand(record.getBefore(), record.forReadingLinkage()));
    }
    for (RecordProxy<Integer, RelationshipTypeTokenRecord, Void> record : recordChangeSet.getRelationshipTypeTokenChanges().changes()) {
        commands.add(new Command.RelationshipTypeTokenCommand(record.getBefore(), record.forReadingLinkage()));
    }
    for (RecordProxy<Integer, PropertyKeyTokenRecord, Void> record : recordChangeSet.getPropertyKeyTokenChanges().changes()) {
        commands.add(new Command.PropertyKeyTokenCommand(record.getBefore(), record.forReadingLinkage()));
    }
    // Collect nodes, relationships, properties
    Command[] nodeCommands = EMPTY_COMMANDS;
    int skippedCommands = 0;
    if (recordChangeSet.getNodeRecords().changeSize() > 0) {
        nodeCommands = new Command[recordChangeSet.getNodeRecords().changeSize()];
        int i = 0;
        for (RecordProxy<Long, NodeRecord, Void> change : recordChangeSet.getNodeRecords().changes()) {
            NodeRecord record = prepared(change, nodeStore);
            integrityValidator.validateNodeRecord(record);
            nodeCommands[i++] = new Command.NodeCommand(change.getBefore(), record);
        }
        Arrays.sort(nodeCommands, COMMAND_SORTER);
    }
    Command[] relCommands = EMPTY_COMMANDS;
    if (recordChangeSet.getRelRecords().changeSize() > 0) {
        relCommands = new Command[recordChangeSet.getRelRecords().changeSize()];
        int i = 0;
        for (RecordProxy<Long, RelationshipRecord, Void> change : recordChangeSet.getRelRecords().changes()) {
            relCommands[i++] = new Command.RelationshipCommand(change.getBefore(), prepared(change, relationshipStore));
        }
        Arrays.sort(relCommands, COMMAND_SORTER);
    }
    Command[] propCommands = EMPTY_COMMANDS;
    if (recordChangeSet.getPropertyRecords().changeSize() > 0) {
        propCommands = new Command[recordChangeSet.getPropertyRecords().changeSize()];
        int i = 0;
        for (RecordProxy<Long, PropertyRecord, PrimitiveRecord> change : recordChangeSet.getPropertyRecords().changes()) {
            propCommands[i++] = new Command.PropertyCommand(change.getBefore(), prepared(change, propertyStore));
        }
        Arrays.sort(propCommands, COMMAND_SORTER);
    }
    Command[] relGroupCommands = EMPTY_COMMANDS;
    if (recordChangeSet.getRelGroupRecords().changeSize() > 0) {
        relGroupCommands = new Command[recordChangeSet.getRelGroupRecords().changeSize()];
        int i = 0;
        for (RecordProxy<Long, RelationshipGroupRecord, Integer> change : recordChangeSet.getRelGroupRecords().changes()) {
            if (change.isCreated() && !change.forReadingLinkage().inUse()) {
                /*
                     * This is an edge case that may come up and which we must handle properly. Relationship groups are
                     * not managed by the tx state, since they are created as side effects rather than through
                     * direct calls. However, they differ from say, dynamic records, in that their management can happen
                     * through separate code paths. What we are interested in here is the following scenario.
                     * 0. A node has one less relationship that is required to transition to dense node. The relationships
                     *    it has belong to at least two different types
                     * 1. In the same tx, a relationship is added making the node dense and all the relationships of a type
                     *    are removed from that node. Regardless of the order these operations happen, the creation of the
                     *    relationship (and the transition of the node to dense) will happen first.
                     * 2. A relationship group will be created because of the transition to dense and then deleted because
                     *    all the relationships it would hold are no longer there. This results in a relationship group
                     *    command that appears in the tx as not in use. Depending on the final order of operations, this
                     *    can end up using an id that is higher than the highest id seen so far. This may not be a problem
                     *    for a single instance, but it can result in errors in cases where transactions are applied
                     *    externally, such as backup or HA.
                     *
                     * The way we deal with this issue here is by not issuing a command for that offending record. This is
                     * safe, since the record is not in use and never was, so the high id is not necessary to change and
                     * the store remains consistent.
                     */
                skippedCommands++;
                continue;
            }
            relGroupCommands[i++] = new Command.RelationshipGroupCommand(change.getBefore(), prepared(change, relationshipGroupStore));
        }
        relGroupCommands = i < relGroupCommands.length ? Arrays.copyOf(relGroupCommands, i) : relGroupCommands;
        Arrays.sort(relGroupCommands, COMMAND_SORTER);
    }
    addFiltered(commands, Mode.CREATE, propCommands, relCommands, relGroupCommands, nodeCommands);
    addFiltered(commands, Mode.UPDATE, propCommands, relCommands, relGroupCommands, nodeCommands);
    addFiltered(commands, Mode.DELETE, propCommands, relCommands, relGroupCommands, nodeCommands);
    if (neoStoreRecord != null) {
        for (RecordProxy<Long, NeoStoreRecord, Void> change : neoStoreRecord.changes()) {
            commands.add(new Command.NeoStoreCommand(change.getBefore(), change.forReadingData()));
        }
    }
    for (RecordProxy<Long, SchemaRecord, SchemaRule> change : recordChangeSet.getSchemaRuleChanges().changes()) {
        integrityValidator.validateSchemaRule(change.getAdditionalData());
        commands.add(new Command.SchemaRuleCommand(change.getBefore(), change.forChangingData(), change.getAdditionalData()));
    }
    assert commands.size() == noOfCommands - skippedCommands : format("Expected %d final commands, got %d " + "instead, with %d skipped", noOfCommands, commands.size(), skippedCommands);
    prepared = true;
}
Also used : RelationshipTypeTokenRecord(org.neo4j.kernel.impl.store.record.RelationshipTypeTokenRecord) RelationshipRecord(org.neo4j.kernel.impl.store.record.RelationshipRecord) SchemaRule(org.neo4j.storageengine.api.schema.SchemaRule) PropertyKeyTokenRecord(org.neo4j.kernel.impl.store.record.PropertyKeyTokenRecord) NodeRecord(org.neo4j.kernel.impl.store.record.NodeRecord) NeoStoreRecord(org.neo4j.kernel.impl.store.record.NeoStoreRecord) PropertyRecord(org.neo4j.kernel.impl.store.record.PropertyRecord) SchemaRecord(org.neo4j.kernel.impl.store.record.SchemaRecord) LabelTokenRecord(org.neo4j.kernel.impl.store.record.LabelTokenRecord) PrimitiveRecord(org.neo4j.kernel.impl.store.record.PrimitiveRecord) RelationshipGroupRecord(org.neo4j.kernel.impl.store.record.RelationshipGroupRecord) StorageCommand(org.neo4j.storageengine.api.StorageCommand) Command(org.neo4j.kernel.impl.transaction.command.Command)

Example 23 with PropertyRecord

use of org.neo4j.kernel.impl.store.record.PropertyRecord in project neo4j by neo4j.

the class OnlineIndexUpdates method nodeFullyLoadProperties.

private Iterator<DefinedProperty> nodeFullyLoadProperties(long nodeId, NodeCommand nodeCommand, List<PropertyCommand> propertyCommands) {
    NodeRecord nodeRecord = (nodeCommand == null) ? loadNode(nodeId) : nodeCommand.getAfter();
    IteratingPropertyReceiver receiver = new IteratingPropertyReceiver();
    PrimitiveLongObjectMap<PropertyRecord> propertiesById = propertiesFromCommandsForNode(propertyCommands);
    propertyLoader.nodeLoadProperties(nodeRecord, propertiesById, receiver);
    return receiver;
}
Also used : NodeRecord(org.neo4j.kernel.impl.store.record.NodeRecord) IteratingPropertyReceiver(org.neo4j.kernel.impl.core.IteratingPropertyReceiver) PropertyRecord(org.neo4j.kernel.impl.store.record.PropertyRecord)

Example 24 with PropertyRecord

use of org.neo4j.kernel.impl.store.record.PropertyRecord in project neo4j by neo4j.

the class PhysicalLogCommandReaderV3_0_2 method readPropertyRecord.

private PropertyRecord readPropertyRecord(long id, ReadableChannel channel) throws IOException {
    // in_use(byte)+type(int)+key_indexId(int)+prop_blockId(long)+
    // prev_prop_id(long)+next_prop_id(long)
    PropertyRecord record = new PropertyRecord(id);
    // 1
    byte flags = channel.get();
    boolean inUse = bitFlag(flags, Record.IN_USE.byteValue());
    boolean nodeProperty = !bitFlag(flags, Record.REL_PROPERTY.byteValue());
    boolean requireSecondaryUnit = bitFlag(flags, Record.REQUIRE_SECONDARY_UNIT);
    boolean hasSecondaryUnit = bitFlag(flags, Record.HAS_SECONDARY_UNIT);
    record.setRequiresSecondaryUnit(requireSecondaryUnit);
    // 8
    long nextProp = channel.getLong();
    // 8
    long prevProp = channel.getLong();
    record.setNextProp(nextProp);
    record.setPrevProp(prevProp);
    // 8
    long primitiveId = channel.getLong();
    if (primitiveId != -1 && nodeProperty) {
        record.setNodeId(primitiveId);
    } else if (primitiveId != -1) {
        record.setRelId(primitiveId);
    }
    if (hasSecondaryUnit) {
        record.setSecondaryUnitId(channel.getLong());
    }
    int nrPropBlocks = channel.get();
    assert nrPropBlocks >= 0;
    if (nrPropBlocks > 0) {
        record.setInUse(true);
    }
    while (nrPropBlocks-- > 0) {
        PropertyBlock block = readPropertyBlock(channel);
        if (block == null) {
            return null;
        }
        record.addPropertyBlock(block);
    }
    int deletedRecords = readDynamicRecords(channel, record, PROPERTY_DELETED_DYNAMIC_RECORD_ADDER);
    if (deletedRecords == -1) {
        return null;
    }
    assert deletedRecords >= 0;
    while (deletedRecords-- > 0) {
        DynamicRecord read = readDynamicRecord(channel);
        if (read == null) {
            return null;
        }
        record.addDeletedRecord(read);
    }
    if ((inUse && !record.inUse()) || (!inUse && record.inUse())) {
        throw new IllegalStateException("Weird, inUse was read in as " + inUse + " but the record is " + record);
    }
    return record;
}
Also used : DynamicRecord(org.neo4j.kernel.impl.store.record.DynamicRecord) PropertyRecord(org.neo4j.kernel.impl.store.record.PropertyRecord) PropertyBlock(org.neo4j.kernel.impl.store.record.PropertyBlock)

Example 25 with PropertyRecord

use of org.neo4j.kernel.impl.store.record.PropertyRecord in project neo4j by neo4j.

the class PropertyCreator method createPropertyChain.

public long createPropertyChain(PrimitiveRecord owner, Iterator<PropertyBlock> properties, RecordAccess<Long, PropertyRecord, PrimitiveRecord> propertyRecords) {
    if (properties == null || !properties.hasNext()) {
        return Record.NO_NEXT_PROPERTY.intValue();
    }
    PropertyRecord currentRecord = propertyRecords.create(propertyRecordIdGenerator.nextId(), owner).forChangingData();
    currentRecord.setInUse(true);
    currentRecord.setCreated();
    PropertyRecord firstRecord = currentRecord;
    while (properties.hasNext()) {
        PropertyBlock block = properties.next();
        if (currentRecord.size() + block.getSize() > PropertyType.getPayloadSize()) {
            // Here it means the current block is done for
            PropertyRecord prevRecord = currentRecord;
            // Create new record
            long propertyId = propertyRecordIdGenerator.nextId();
            currentRecord = propertyRecords.create(propertyId, owner).forChangingData();
            currentRecord.setInUse(true);
            currentRecord.setCreated();
            // Set up links
            prevRecord.setNextProp(propertyId);
            currentRecord.setPrevProp(prevRecord.getId());
        // Now current is ready to start picking up blocks
        }
        currentRecord.addPropertyBlock(block);
    }
    return firstRecord.getId();
}
Also used : PropertyRecord(org.neo4j.kernel.impl.store.record.PropertyRecord) PropertyBlock(org.neo4j.kernel.impl.store.record.PropertyBlock)

Aggregations

PropertyRecord (org.neo4j.kernel.impl.store.record.PropertyRecord)230 PropertyBlock (org.neo4j.kernel.impl.store.record.PropertyBlock)86 Test (org.junit.Test)75 NodeRecord (org.neo4j.kernel.impl.store.record.NodeRecord)37 Test (org.junit.jupiter.api.Test)36 ConsistencyReport (org.neo4j.consistency.report.ConsistencyReport)35 DynamicRecord (org.neo4j.kernel.impl.store.record.DynamicRecord)28 RelationshipRecord (org.neo4j.kernel.impl.store.record.RelationshipRecord)19 ArrayList (java.util.ArrayList)17 Value (org.neo4j.values.storable.Value)17 PrimitiveRecord (org.neo4j.kernel.impl.store.record.PrimitiveRecord)14 PropertyKeyTokenRecord (org.neo4j.kernel.impl.store.record.PropertyKeyTokenRecord)14 RecordAccessStub (org.neo4j.consistency.store.RecordAccessStub)12 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)10 PropertyStore (org.neo4j.kernel.impl.store.PropertyStore)9 IOException (java.io.IOException)8 InterruptedIOException (java.io.InterruptedIOException)8 Pair (org.neo4j.helpers.collection.Pair)8 DefinedProperty (org.neo4j.kernel.api.properties.DefinedProperty)8 NodeStore (org.neo4j.kernel.impl.store.NodeStore)8