Search in sources :

Example 6 with InternalVertex

use of org.janusgraph.graphdb.internal.InternalVertex in project janusgraph by JanusGraph.

the class LRUVertexCache method get.

@Override
public InternalVertex get(long id, final Retriever<Long, InternalVertex> retriever) {
    final Long vertexId = id;
    InternalVertex vertex = cache.get(vertexId);
    if (vertex == null) {
        InternalVertex newVertex = volatileVertices.get(vertexId);
        if (newVertex == null) {
            newVertex = retriever.get(vertexId);
        }
        vertex = cache.putIfAbsent(vertexId, newVertex);
        if (vertex == null)
            vertex = newVertex;
    }
    return vertex;
}
Also used : NonBlockingHashMapLong(org.cliffc.high_scale_lib.NonBlockingHashMapLong) InternalVertex(org.janusgraph.graphdb.internal.InternalVertex)

Example 7 with InternalVertex

use of org.janusgraph.graphdb.internal.InternalVertex in project janusgraph by JanusGraph.

the class ModificationDeserializer method parseRelation.

public static InternalRelation parseRelation(TransactionLogHeader.Modification modification, StandardJanusGraphTx tx) {
    Change state = modification.state;
    assert state.isProper();
    long outVertexId = modification.outVertexId;
    Entry relEntry = modification.relationEntry;
    InternalVertex outVertex = tx.getInternalVertex(outVertexId);
    // Special relation parsing, compare to {@link RelationConstructor}
    RelationCache relCache = tx.getEdgeSerializer().readRelation(relEntry, false, tx);
    assert relCache.direction == Direction.OUT;
    InternalRelationType type = (InternalRelationType) tx.getExistingRelationType(relCache.typeId);
    assert type.getBaseType() == null;
    InternalRelation rel;
    if (type.isPropertyKey()) {
        if (state == Change.REMOVED) {
            rel = new StandardVertexProperty(relCache.relationId, (PropertyKey) type, outVertex, relCache.getValue(), ElementLifeCycle.Removed);
        } else {
            rel = new CacheVertexProperty(relCache.relationId, (PropertyKey) type, outVertex, relCache.getValue(), relEntry);
        }
    } else {
        assert type.isEdgeLabel();
        InternalVertex otherVertex = tx.getInternalVertex(relCache.getOtherVertexId());
        if (state == Change.REMOVED) {
            rel = new StandardEdge(relCache.relationId, (EdgeLabel) type, outVertex, otherVertex, ElementLifeCycle.Removed);
        } else {
            rel = new CacheEdge(relCache.relationId, (EdgeLabel) type, outVertex, otherVertex, relEntry);
        }
    }
    if (state == Change.REMOVED && relCache.hasProperties()) {
        // copy over properties
        for (LongObjectCursor<Object> entry : relCache) {
            rel.setPropertyDirect(tx.getExistingPropertyKey(entry.key), entry.value);
        }
    }
    return rel;
}
Also used : EdgeLabel(org.janusgraph.core.EdgeLabel) Change(org.janusgraph.core.log.Change) InternalRelation(org.janusgraph.graphdb.internal.InternalRelation) Entry(org.janusgraph.diskstorage.Entry) InternalVertex(org.janusgraph.graphdb.internal.InternalVertex) InternalRelationType(org.janusgraph.graphdb.internal.InternalRelationType) PropertyKey(org.janusgraph.core.PropertyKey)

Example 8 with InternalVertex

use of org.janusgraph.graphdb.internal.InternalVertex in project janusgraph by JanusGraph.

the class StandardJanusGraph method prepareCommit.

public ModificationSummary prepareCommit(final Collection<InternalRelation> addedRelations, final Collection<InternalRelation> deletedRelations, final Predicate<InternalRelation> filter, final BackendTransaction mutator, final StandardJanusGraphTx tx, final boolean acquireLocks) throws BackendException {
    ListMultimap<Long, InternalRelation> mutations = ArrayListMultimap.create();
    ListMultimap<InternalVertex, InternalRelation> mutatedProperties = ArrayListMultimap.create();
    List<IndexSerializer.IndexUpdate> indexUpdates = Lists.newArrayList();
    // 1) Collect deleted edges and their index updates and acquire edge locks
    for (InternalRelation del : Iterables.filter(deletedRelations, filter)) {
        Preconditions.checkArgument(del.isRemoved());
        for (int pos = 0; pos < del.getLen(); pos++) {
            InternalVertex vertex = del.getVertex(pos);
            if (pos == 0 || !del.isLoop()) {
                if (del.isProperty())
                    mutatedProperties.put(vertex, del);
                mutations.put(vertex.longId(), del);
            }
            if (acquireLock(del, pos, acquireLocks)) {
                Entry entry = edgeSerializer.writeRelation(del, pos, tx);
                mutator.acquireEdgeLock(idManager.getKey(vertex.longId()), entry);
            }
        }
        indexUpdates.addAll(indexSerializer.getIndexUpdates(del));
    }
    // 2) Collect added edges and their index updates and acquire edge locks
    for (InternalRelation add : Iterables.filter(addedRelations, filter)) {
        Preconditions.checkArgument(add.isNew());
        for (int pos = 0; pos < add.getLen(); pos++) {
            InternalVertex vertex = add.getVertex(pos);
            if (pos == 0 || !add.isLoop()) {
                if (add.isProperty())
                    mutatedProperties.put(vertex, add);
                mutations.put(vertex.longId(), add);
            }
            if (!vertex.isNew() && acquireLock(add, pos, acquireLocks)) {
                Entry entry = edgeSerializer.writeRelation(add, pos, tx);
                mutator.acquireEdgeLock(idManager.getKey(vertex.longId()), entry.getColumn());
            }
        }
        indexUpdates.addAll(indexSerializer.getIndexUpdates(add));
    }
    // 3) Collect all index update for vertices
    for (InternalVertex v : mutatedProperties.keySet()) {
        indexUpdates.addAll(indexSerializer.getIndexUpdates(v, mutatedProperties.get(v)));
    }
    // 4) Acquire index locks (deletions first)
    for (IndexSerializer.IndexUpdate update : indexUpdates) {
        if (!update.isCompositeIndex() || !update.isDeletion())
            continue;
        CompositeIndexType iIndex = (CompositeIndexType) update.getIndex();
        if (acquireLock(iIndex, acquireLocks)) {
            mutator.acquireIndexLock((StaticBuffer) update.getKey(), (Entry) update.getEntry());
        }
    }
    for (IndexSerializer.IndexUpdate update : indexUpdates) {
        if (!update.isCompositeIndex() || !update.isAddition())
            continue;
        CompositeIndexType iIndex = (CompositeIndexType) update.getIndex();
        if (acquireLock(iIndex, acquireLocks)) {
            mutator.acquireIndexLock((StaticBuffer) update.getKey(), ((Entry) update.getEntry()).getColumn());
        }
    }
    // 5) Add relation mutations
    for (Long vertexId : mutations.keySet()) {
        Preconditions.checkArgument(vertexId > 0, "Vertex has no id: %s", vertexId);
        final List<InternalRelation> edges = mutations.get(vertexId);
        final List<Entry> additions = new ArrayList<>(edges.size());
        final List<Entry> deletions = new ArrayList<>(Math.max(10, edges.size() / 10));
        for (final InternalRelation edge : edges) {
            final InternalRelationType baseType = (InternalRelationType) edge.getType();
            assert baseType.getBaseType() == null;
            for (InternalRelationType type : baseType.getRelationIndexes()) {
                if (type.getStatus() == SchemaStatus.DISABLED)
                    continue;
                for (int pos = 0; pos < edge.getArity(); pos++) {
                    if (!type.isUnidirected(Direction.BOTH) && !type.isUnidirected(EdgeDirection.fromPosition(pos)))
                        // Directionality is not covered
                        continue;
                    if (edge.getVertex(pos).longId() == vertexId) {
                        StaticArrayEntry entry = edgeSerializer.writeRelation(edge, type, pos, tx);
                        if (edge.isRemoved()) {
                            deletions.add(entry);
                        } else {
                            Preconditions.checkArgument(edge.isNew());
                            int ttl = getTTL(edge);
                            if (ttl > 0) {
                                entry.setMetaData(EntryMetaData.TTL, ttl);
                            }
                            additions.add(entry);
                        }
                    }
                }
            }
        }
        StaticBuffer vertexKey = idManager.getKey(vertexId);
        mutator.mutateEdges(vertexKey, additions, deletions);
    }
    // 6) Add index updates
    boolean has2iMods = false;
    for (IndexSerializer.IndexUpdate indexUpdate : indexUpdates) {
        assert indexUpdate.isAddition() || indexUpdate.isDeletion();
        if (indexUpdate.isCompositeIndex()) {
            final IndexSerializer.IndexUpdate<StaticBuffer, Entry> update = indexUpdate;
            if (update.isAddition())
                mutator.mutateIndex(update.getKey(), Lists.newArrayList(update.getEntry()), KCVSCache.NO_DELETIONS);
            else
                mutator.mutateIndex(update.getKey(), KeyColumnValueStore.NO_ADDITIONS, Lists.newArrayList(update.getEntry()));
        } else {
            final IndexSerializer.IndexUpdate<String, IndexEntry> update = indexUpdate;
            has2iMods = true;
            IndexTransaction itx = mutator.getIndexTransaction(update.getIndex().getBackingIndexName());
            String indexStore = ((MixedIndexType) update.getIndex()).getStoreName();
            if (update.isAddition())
                itx.add(indexStore, update.getKey(), update.getEntry(), update.getElement().isNew());
            else
                itx.delete(indexStore, update.getKey(), update.getEntry().field, update.getEntry().value, update.getElement().isRemoved());
        }
    }
    return new ModificationSummary(!mutations.isEmpty(), has2iMods);
}
Also used : LongArrayList(com.carrotsearch.hppc.LongArrayList) IndexTransaction(org.janusgraph.diskstorage.indexing.IndexTransaction) IndexEntry(org.janusgraph.diskstorage.indexing.IndexEntry) InternalRelation(org.janusgraph.graphdb.internal.InternalRelation) StaticArrayEntry(org.janusgraph.diskstorage.util.StaticArrayEntry) IndexEntry(org.janusgraph.diskstorage.indexing.IndexEntry) StaticArrayEntry(org.janusgraph.diskstorage.util.StaticArrayEntry) MixedIndexType(org.janusgraph.graphdb.types.MixedIndexType) AtomicLong(java.util.concurrent.atomic.AtomicLong) InternalVertex(org.janusgraph.graphdb.internal.InternalVertex) CompositeIndexType(org.janusgraph.graphdb.types.CompositeIndexType) InternalRelationType(org.janusgraph.graphdb.internal.InternalRelationType)

Example 9 with InternalVertex

use of org.janusgraph.graphdb.internal.InternalVertex in project janusgraph by JanusGraph.

the class MultiVertexCentricQueryBuilder method execute.

/* ---------------------------------------------------------------
     * Query Execution
	 * ---------------------------------------------------------------
	 */
/**
 * Constructs the BaseVertexCentricQuery through {@link BasicVertexCentricQueryBuilder#constructQuery(org.janusgraph.graphdb.internal.RelationCategory)}.
 * If the query asks for an implicit key, the resulting map is computed and returned directly.
 * If the query is empty, a map that maps each vertex to an empty list is returned.
 * Otherwise, the query is executed for all vertices through the transaction which will effectively
 * pre-load the return result sets into the associated {@link org.janusgraph.graphdb.vertices.CacheVertex} or
 * don't do anything at all if the vertex is new (and hence no edges in the storage backend).
 * After that, a map is constructed that maps each vertex to the corresponding VertexCentricQuery and wrapped
 * into a QueryProcessor. Hence, upon iteration the query will be executed like any other VertexCentricQuery
 * with the performance difference that the SliceQueries will have already been preloaded and not further
 * calls to the storage backend are needed.
 *
 * @param returnType
 * @return
 */
protected <Q> Map<JanusGraphVertex, Q> execute(RelationCategory returnType, ResultConstructor<Q> resultConstructor) {
    Preconditions.checkArgument(!vertices.isEmpty(), "Need to add at least one vertex to query");
    final Map<JanusGraphVertex, Q> result = new HashMap<>(vertices.size());
    BaseVertexCentricQuery bq = super.constructQuery(returnType);
    profiler.setAnnotation(QueryProfiler.MULTIQUERY_ANNOTATION, true);
    profiler.setAnnotation(QueryProfiler.NUMVERTICES_ANNOTATION, vertices.size());
    if (!bq.isEmpty()) {
        for (BackendQueryHolder<SliceQuery> sq : bq.getQueries()) {
            Set<InternalVertex> adjVertices = Sets.newHashSet(vertices);
            for (InternalVertex v : vertices) {
                if (isPartitionedVertex(v)) {
                    profiler.setAnnotation(QueryProfiler.PARTITIONED_VERTEX_ANNOTATION, true);
                    adjVertices.remove(v);
                    adjVertices.addAll(allRequiredRepresentatives(v));
                }
            }
            // Overwrite with more accurate size accounting for partitioned vertices
            profiler.setAnnotation(QueryProfiler.NUMVERTICES_ANNOTATION, adjVertices.size());
            tx.executeMultiQuery(adjVertices, sq.getBackendQuery(), sq.getProfiler());
        }
        for (InternalVertex v : vertices) {
            result.put(v, resultConstructor.getResult(v, bq));
        }
    } else {
        for (JanusGraphVertex v : vertices) result.put(v, resultConstructor.emptyResult());
    }
    return result;
}
Also used : InternalVertex(org.janusgraph.graphdb.internal.InternalVertex) SliceQuery(org.janusgraph.diskstorage.keycolumnvalue.SliceQuery)

Example 10 with InternalVertex

use of org.janusgraph.graphdb.internal.InternalVertex in project janusgraph by JanusGraph.

the class AbstractTypedRelation method it.

@Override
public InternalRelation it() {
    InternalVertex v = getVertex(0);
    if (v == v.it())
        return this;
    InternalRelation next = (InternalRelation) RelationIdentifier.get(this).findRelation(tx());
    if (next == null)
        throw InvalidElementException.removedException(this);
    return next;
}
Also used : InternalVertex(org.janusgraph.graphdb.internal.InternalVertex) InternalRelation(org.janusgraph.graphdb.internal.InternalRelation)

Aggregations

InternalVertex (org.janusgraph.graphdb.internal.InternalVertex)11 InternalRelation (org.janusgraph.graphdb.internal.InternalRelation)7 InternalRelationType (org.janusgraph.graphdb.internal.InternalRelationType)3 NonBlockingHashMapLong (org.cliffc.high_scale_lib.NonBlockingHashMapLong)2 JanusGraphSchemaVertex (org.janusgraph.graphdb.types.vertices.JanusGraphSchemaVertex)2 LongArrayList (com.carrotsearch.hppc.LongArrayList)1 ImmutableMap (com.google.common.collect.ImmutableMap)1 Iterator (java.util.Iterator)1 NoSuchElementException (java.util.NoSuchElementException)1 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)1 ConcurrentMap (java.util.concurrent.ConcurrentMap)1 AtomicLong (java.util.concurrent.atomic.AtomicLong)1 EdgeLabel (org.janusgraph.core.EdgeLabel)1 PropertyKey (org.janusgraph.core.PropertyKey)1 Change (org.janusgraph.core.log.Change)1 Entry (org.janusgraph.diskstorage.Entry)1 IndexEntry (org.janusgraph.diskstorage.indexing.IndexEntry)1 IndexTransaction (org.janusgraph.diskstorage.indexing.IndexTransaction)1 SliceQuery (org.janusgraph.diskstorage.keycolumnvalue.SliceQuery)1 RecordIterator (org.janusgraph.diskstorage.util.RecordIterator)1