Search in sources :

Example 6 with EntityType

use of org.neo4j.common.EntityType in project neo4j by neo4j.

the class TokenScanWriteMonitor method main.

/**
 * Dumps a token scan write log as plain text. Arguments:
 * <ul>
 *     <li>{@value #ARG_TOFILE}: dumps to a .txt file next to the writelog</li>
 *     <li>{@value #ARG_TXFILTER}: filter for which tx ids to include in the dump.
 *     <p>
 *     Consists of one or more groups separated by comma.
 *     <p>
 *     Each group is either a txId, or a txId range, e.g. 123-456
 *     </li>
 * </ul>
 * <p>
 * How to interpret the dump, e.g:
 * <pre>
 * === ..../neostore.labelscanstore.db.writelog ===
 * [1,1]+tx:6,entity:0,token:0
 * [1,1]+tx:3,entity:20,token:0
 * [1,1]+tx:4,entity:40,token:0
 * [1,1]+tx:5,entity:60,token:0
 * [2,1]+tx:8,entity:80,token:1
 * [3,1]+tx:10,entity:41,token:1
 * [4,1]+tx:9,entity:21,token:1
 * [4,1]+tx:11,entity:61,token:1
 * [4,1]+range:0,tokenId:1
 *  [00000000 00000000 00000010 00000000 00000000 00000000 00000000 00000000]
 *  [00100000 00000000 00000000 00000000 00000000 00100000 00000000 00000000]
 * [5,1]+tx:12,entity:81,token:1
 * [5,1]+range:1,tokenId:1
 *  [00000000 00000000 00000000 00000000 00000000 00000001 00000000 00000000]
 *  [00000000 00000000 00000000 00000000 00000000 00000010 00000000 00000000]
 * [6,1]+tx:13,entity:1,token:1
 * [6,1]+range:0,tokenId:1
 *  [00100000 00000000 00000010 00000000 00000000 00100000 00000000 00000000]
 *  [00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000010]
 * [7,1]+tx:14,entity:62,token:1
 * [7,1]+range:0,tokenId:1
 * </pre>
 * How to interpret a message like:
 * <pre>
 * [1,1]+tx:6,entity:0,token:0
 *  ▲ ▲ ▲   ▲        ▲       ▲
 *  │ │ │   │        │       └── token id of the change
 *  │ │ │   │        └────────── entity id of the change
 *  │ │ │   └─────────────────── id of transaction making this particular change
 *  │ │ └─────────────────────── addition, a minus means removal
 *  │ └───────────────────────── flush, local to each write session, incremented when a batch of changes is flushed internally in a writer session
 *  └─────────────────────────── write session, incremented for each token index writer
 * </pre>
 * How to interpret a message like:
 * <pre>
 * [4,1]+range:0,tokenId:1
 *  [00000000 00000000 00000010 00000000 00000000 00000000 00000000 00000000]
 *  [00100000 00000000 00000000 00000000 00000000 00100000 00000000 00000000]
 * </pre>
 * First the first line (parts within bracket same as above):
 * <pre>
 * [4,1]+range:0,tokenId:1
 *             ▲         ▲
 *             │         └── token id of the changed bitset to apply
 *             └──────────── range, i.e. which bitset to apply this change for
 * </pre>
 * Then the bitsets are printed
 * <pre>
 *  [00000000 00000000 00000010 00000000 00000000 00000000 00000000 00000000] : state of the bitset for this token id before the change
 *  [00100000 00000000 00000000 00000000 00000000 00100000 00000000 00000000] : bits that applied to this bitset
 *                                                                              for addition the 1-bits denotes bits to be added
 *                                                                              for removal the 1-bits denotes bits to be removed
 * </pre>
 */
public static void main(String[] args) throws IOException {
    Args arguments = Args.withFlags(ARG_TOFILE).parse(args);
    if (arguments.orphans().size() == 0) {
        System.err.println("Please supply database directory");
        return;
    }
    DatabaseLayout databaseLayout = DatabaseLayout.ofFlat(Path.of(arguments.orphans().get(0)));
    FileSystemAbstraction fs = new DefaultFileSystemAbstraction();
    TxFilter txFilter = parseTxFilter(arguments.get(ARG_TXFILTER, null));
    PrintStream out = System.out;
    boolean redirectsToFile = arguments.getBoolean(ARG_TOFILE);
    for (EntityType entityType : EntityType.values()) {
        if (redirectsToFile) {
            Path outFile = Path.of(writeLogBaseFile(databaseLayout, entityType).toAbsolutePath() + ".txt");
            System.out.println("Redirecting output to " + outFile);
            out = new PrintStream(new BufferedOutputStream(Files.newOutputStream(outFile)));
        }
        Dumper dumper = new PrintStreamDumper(out);
        dump(fs, databaseLayout, dumper, txFilter, entityType);
        if (redirectsToFile) {
            out.close();
        }
    }
}
Also used : EntityType(org.neo4j.common.EntityType) Path(java.nio.file.Path) PrintStream(java.io.PrintStream) Args(org.neo4j.internal.helpers.Args) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) DatabaseLayout(org.neo4j.io.layout.DatabaseLayout) BufferedOutputStream(java.io.BufferedOutputStream)

Example 7 with EntityType

use of org.neo4j.common.EntityType in project neo4j by neo4j.

the class SchemaStore method buildSchemaDescriptor.

private static SchemaDescriptor buildSchemaDescriptor(Map<String, Value> props) throws MalformedSchemaRuleException {
    EntityType entityType = getEntityType(getString(PROP_SCHEMA_DESCRIPTOR_ENTITY_TYPE, props));
    PropertySchemaType propertySchemaType = getPropertySchemaType(getString(PROP_SCHEMA_DESCRIPTOR_PROPERTY_SCHEMA_TYPE, props));
    int[] entityIds = getIntArray(PROP_SCHEMA_DESCRIPTOR_ENTITY_IDS, props);
    int[] propertyIds = getIntArray(PROP_SCHEMA_DESCRIPTOR_PROPERTY_IDS, props);
    return new SchemaDescriptorImplementation(entityType, propertySchemaType, entityIds, propertyIds);
}
Also used : EntityType(org.neo4j.common.EntityType) SchemaDescriptorImplementation(org.neo4j.internal.schema.SchemaDescriptorImplementation) PropertySchemaType(org.neo4j.internal.schema.PropertySchemaType)

Example 8 with EntityType

use of org.neo4j.common.EntityType in project neo4j by neo4j.

the class SchemaStore method schemaDescriptorToMap.

private static void schemaDescriptorToMap(SchemaDescriptor schemaDescriptor, Map<String, Value> map) {
    EntityType entityType = schemaDescriptor.entityType();
    PropertySchemaType propertySchemaType = schemaDescriptor.propertySchemaType();
    int[] entityTokenIds = schemaDescriptor.getEntityTokenIds();
    int[] propertyIds = schemaDescriptor.getPropertyIds();
    putStringProperty(map, PROP_SCHEMA_DESCRIPTOR_ENTITY_TYPE, entityType.name());
    putStringProperty(map, PROP_SCHEMA_DESCRIPTOR_PROPERTY_SCHEMA_TYPE, propertySchemaType.name());
    putIntArrayProperty(map, PROP_SCHEMA_DESCRIPTOR_ENTITY_IDS, entityTokenIds);
    putIntArrayProperty(map, PROP_SCHEMA_DESCRIPTOR_PROPERTY_IDS, propertyIds);
}
Also used : EntityType(org.neo4j.common.EntityType) PropertySchemaType(org.neo4j.internal.schema.PropertySchemaType)

Example 9 with EntityType

use of org.neo4j.common.EntityType in project neo4j by neo4j.

the class LabelScanNodeViewTracingIT method findTokenIndex.

private IndexDescriptor findTokenIndex() {
    try (Transaction tx = database.beginTx()) {
        var nodeIndex = stream(tx.schema().getIndexes().spliterator(), false).map(indexDef -> ((IndexDefinitionImpl) indexDef).getIndexReference()).filter(index -> index.isTokenIndex() && index.schema().entityType() == EntityType.NODE).findFirst();
        assertTrue(nodeIndex.isPresent());
        return nodeIndex.get();
    }
}
Also used : IndexingService(org.neo4j.kernel.impl.api.index.IndexingService) Label(org.neo4j.graphdb.Label) IndexProxy(org.neo4j.kernel.impl.api.index.IndexProxy) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) Config(org.neo4j.configuration.Config) StoreScan(org.neo4j.kernel.impl.api.index.StoreScan) RecordStorageEngine(org.neo4j.internal.recordstorage.RecordStorageEngine) DefaultPageCacheTracer(org.neo4j.io.pagecache.tracing.DefaultPageCacheTracer) LockService(org.neo4j.lock.LockService) DbmsExtension(org.neo4j.test.extension.DbmsExtension) InternalTransaction(org.neo4j.kernel.impl.coreapi.InternalTransaction) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) Test(org.junit.jupiter.api.Test) INSTANCE(org.neo4j.memory.EmptyMemoryTracker.INSTANCE) Inject(org.neo4j.test.extension.Inject) StreamSupport.stream(java.util.stream.StreamSupport.stream) EntityType(org.neo4j.common.EntityType) Assertions.assertTrue(org.junit.jupiter.api.Assertions.assertTrue) IndexDescriptor(org.neo4j.internal.schema.IndexDescriptor) IndexDefinitionImpl(org.neo4j.kernel.impl.coreapi.schema.IndexDefinitionImpl) Transaction(org.neo4j.graphdb.Transaction) JobScheduler(org.neo4j.scheduler.JobScheduler) RandomStringUtils.randomAscii(org.apache.commons.lang3.RandomStringUtils.randomAscii) InternalTransaction(org.neo4j.kernel.impl.coreapi.InternalTransaction) Transaction(org.neo4j.graphdb.Transaction)

Example 10 with EntityType

use of org.neo4j.common.EntityType in project neo4j by neo4j.

the class IndexSizes method getEstimatedIndexSize.

long getEstimatedIndexSize(IndexDescriptor index) {
    EntityType entityType = index.schema().entityType();
    ConcurrentMap<IndexDescriptor, Long> map = entityType == EntityType.NODE ? nodeIndexSizes : relationshipIndexSizes;
    return map.get(index);
}
Also used : EntityType(org.neo4j.common.EntityType) IndexDescriptor(org.neo4j.internal.schema.IndexDescriptor)

Aggregations

EntityType (org.neo4j.common.EntityType)15 IndexDescriptor (org.neo4j.internal.schema.IndexDescriptor)9 Arrays (java.util.Arrays)6 Path (java.nio.file.Path)5 ArrayList (java.util.ArrayList)5 HashMap (java.util.HashMap)5 List (java.util.List)5 Map (java.util.Map)5 Config (org.neo4j.configuration.Config)5 Iterators (org.neo4j.internal.helpers.collection.Iterators)5 IndexNotFoundKernelException (org.neo4j.internal.kernel.api.exceptions.schema.IndexNotFoundKernelException)5 Collectors (java.util.stream.Collectors)4 IOException (java.io.IOException)3 UncheckedIOException (java.io.UncheckedIOException)3 String.format (java.lang.String.format)3 Iterator (java.util.Iterator)3 TimeUnit (java.util.concurrent.TimeUnit)3 InternalIndexState (org.neo4j.internal.kernel.api.InternalIndexState)3 IndexPrototype (org.neo4j.internal.schema.IndexPrototype)3 Log (org.neo4j.logging.Log)3