Search in sources :

Example 1 with VirtualNode

use of apoc.result.VirtualNode in project neo4j-apoc-procedures by neo4j-contrib.

the class Grouping method group.

@Procedure
@Description("Group all nodes and their relationships by given keys, create virtual nodes and relationships for the summary information, you can provide an aggregations map [{kids:'sum',age:['min','max','avg'],gender:'collect'},{`*`,'count'}]")
public Stream<GraphResult> group(@Name("labels") List<String> labels, @Name("groupByProperties") List<String> groupByProperties, @Name(value = "aggregations", defaultValue = "[{\"*\":\"count\"},{\"*\":\"count\"}]") List<Map<String, Object>> aggregations) {
    String[] keys = groupByProperties.toArray(new String[groupByProperties.size()]);
    Map<String, List<String>> nodeAggNames = (aggregations.size() > 0) ? toStringListMap(aggregations.get(0)) : emptyMap();
    String[] nodeAggKeys = keyArray(nodeAggNames, ASTERISK);
    Map<String, List<String>> relAggNames = (aggregations.size() > 1) ? toStringListMap(aggregations.get(1)) : emptyMap();
    String[] relAggKeys = keyArray(relAggNames, ASTERISK);
    Map<NodeKey, Set<Node>> grouped = new ConcurrentHashMap<>();
    Map<NodeKey, Node> virtualNodes = new ConcurrentHashMap<>();
    Map<RelKey, Relationship> virtualRels = new ConcurrentHashMap<>();
    List<Future> futures = new ArrayList<>(1000);
    ExecutorService pool = Pools.DEFAULT;
    for (String labelName : labels) {
        Label label = Label.label(labelName);
        Label[] singleLabel = { label };
        try (ResourceIterator<Node> nodes = (labelName.equals(ASTERISK)) ? db.getAllNodes().iterator() : db.findNodes(label)) {
            while (nodes.hasNext()) {
                List<Node> batch = Util.take(nodes, BATCHSIZE);
                futures.add(Util.inTxFuture(pool, db, () -> {
                    try {
                        for (Node node : batch) {
                            NodeKey key = keyFor(node, labelName, keys);
                            grouped.compute(key, (k, v) -> {
                                if (v == null)
                                    v = new HashSet<>();
                                v.add(node);
                                return v;
                            });
                            virtualNodes.compute(key, (k, v) -> {
                                if (v == null) {
                                    v = new VirtualNode(singleLabel, propertiesFor(node, keys), db);
                                }
                                Node vn = v;
                                if (!nodeAggNames.isEmpty()) {
                                    aggregate(vn, nodeAggNames, nodeAggKeys.length > 0 ? node.getProperties(nodeAggKeys) : Collections.emptyMap());
                                }
                                return vn;
                            });
                        }
                    } catch (Exception e) {
                        log.debug("Error grouping nodes", e);
                    }
                    return null;
                }));
                Util.removeFinished(futures);
            }
        }
    }
    Util.waitForFutures(futures);
    futures.clear();
    Iterator<Map.Entry<NodeKey, Set<Node>>> entries = grouped.entrySet().iterator();
    int size = 0;
    List<Map.Entry<NodeKey, Set<Node>>> batch = new ArrayList<>();
    while (entries.hasNext()) {
        Map.Entry<NodeKey, Set<Node>> outerEntry = entries.next();
        batch.add(outerEntry);
        size += outerEntry.getValue().size();
        if (size > BATCHSIZE || !entries.hasNext()) {
            ArrayList<Map.Entry<NodeKey, Set<Node>>> submitted = new ArrayList<>(batch);
            batch.clear();
            size = 0;
            futures.add(Util.inTxFuture(pool, db, () -> {
                try {
                    for (Map.Entry<NodeKey, Set<Node>> entry : submitted) {
                        for (Node node : entry.getValue()) {
                            NodeKey startKey = entry.getKey();
                            Node v1 = virtualNodes.get(startKey);
                            for (Relationship rel : node.getRelationships(Direction.OUTGOING)) {
                                Node endNode = rel.getEndNode();
                                for (NodeKey endKey : keysFor(endNode, labels, keys)) {
                                    Node v2 = virtualNodes.get(endKey);
                                    if (v2 == null)
                                        continue;
                                    virtualRels.compute(new RelKey(startKey, endKey, rel), (rk, vRel) -> {
                                        if (vRel == null)
                                            vRel = v1.createRelationshipTo(v2, rel.getType());
                                        if (!relAggNames.isEmpty()) {
                                            aggregate(vRel, relAggNames, relAggKeys.length > 0 ? rel.getProperties(relAggKeys) : Collections.emptyMap());
                                        }
                                        return vRel;
                                    });
                                }
                            }
                        }
                    }
                } catch (Exception e) {
                    log.debug("Error grouping relationships", e);
                }
                return null;
            }));
            Util.removeFinished(futures);
        }
    }
    Util.waitForFutures(futures);
    return fixAggregates(virtualNodes.values()).stream().map(n -> new GraphResult(singletonList(n), fixAggregates(Iterables.asList(n.getRelationships()))));
}
Also used : Collections.emptyMap(java.util.Collections.emptyMap) java.util(java.util) GraphResult(apoc.result.GraphResult) Log(org.neo4j.logging.Log) VirtualNode(apoc.result.VirtualNode) Context(org.neo4j.procedure.Context) Pools(apoc.Pools) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Collectors(java.util.stream.Collectors) Collections.singletonList(java.util.Collections.singletonList) Future(java.util.concurrent.Future) Iterables(org.neo4j.helpers.collection.Iterables) Stream(java.util.stream.Stream) Description(apoc.Description) org.neo4j.graphdb(org.neo4j.graphdb) Name(org.neo4j.procedure.Name) Util(apoc.util.Util) Procedure(org.neo4j.procedure.Procedure) ExecutorService(java.util.concurrent.ExecutorService) VirtualNode(apoc.result.VirtualNode) VirtualNode(apoc.result.VirtualNode) Collections.singletonList(java.util.Collections.singletonList) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) GraphResult(apoc.result.GraphResult) ExecutorService(java.util.concurrent.ExecutorService) Future(java.util.concurrent.Future) Collections.emptyMap(java.util.Collections.emptyMap) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Description(apoc.Description) Procedure(org.neo4j.procedure.Procedure)

Example 2 with VirtualNode

use of apoc.result.VirtualNode in project neo4j-apoc-procedures by neo4j-contrib.

the class Bolt method toRelationship.

private Object toRelationship(Object value, boolean virtual, Map<Long, Object> nodesCache) {
    Value internalValue = ((InternalEntity) value).asValue();
    Relationship relationship = internalValue.asRelationship();
    if (virtual) {
        VirtualNode start = (VirtualNode) nodesCache.getOrDefault(relationship.startNodeId(), new VirtualNode(relationship.startNodeId(), db));
        VirtualNode end = (VirtualNode) nodesCache.getOrDefault(relationship.endNodeId(), new VirtualNode(relationship.endNodeId(), db));
        VirtualRelationship virtualRelationship = new VirtualRelationship(relationship.id(), start, end, RelationshipType.withName(relationship.type()), relationship.asMap());
        return virtualRelationship;
    } else
        return Util.map("entityType", internalValue.type().name(), "type", relationship.type(), "id", relationship.id(), "start", relationship.startNodeId(), "end", relationship.endNodeId(), "properties", relationship.asMap());
}
Also used : VirtualRelationship(apoc.result.VirtualRelationship) InternalEntity(org.neo4j.driver.internal.InternalEntity) VirtualNode(apoc.result.VirtualNode) VirtualRelationship(apoc.result.VirtualRelationship) Relationship(org.neo4j.driver.v1.types.Relationship)

Example 3 with VirtualNode

use of apoc.result.VirtualNode in project neo4j-apoc-procedures by neo4j-contrib.

the class Cluster method graph.

@Procedure
@Description("apoc.cluster.graph - visually displays the servers participating in the Causal Cluster, their " + "roles, and which server in the cluster you are connected to.")
public Stream<GraphResult> graph() {
    Result execute = db.execute("CALL dbms.cluster.overview()");
    List<Node> servers = new LinkedList<>();
    List<Relationship> relationships = new LinkedList<>();
    while (execute.hasNext()) {
        Map<String, Object> next = execute.next();
        String role = (String) next.get("role");
        String id = (String) next.get("id");
        Label roleLabel = Label.label(role);
        String[] addresses = ((List<String>) next.get("addresses")).toArray(new String[0]);
        Map<String, Object> properties = new HashMap<>();
        properties.put("name", shortName.get(role));
        properties.put("title", role);
        properties.put(boltAddressKey, addresses[0]);
        properties.put("http_address", addresses[1]);
        properties.put("cluster_id", id);
        Node server = new VirtualNode(new Label[] { roleLabel }, properties, db);
        servers.add(server);
    }
    Optional<Node> leaderNode = getLeaderNode(servers);
    if (leaderNode.isPresent()) {
        for (Node server : servers) {
            if (server.hasLabel(Label.label("FOLLOWER"))) {
                VirtualRelationship follows = new VirtualRelationship(server, leaderNode.get(), RelationshipType.withName("FOLLOWS"));
                relationships.add(follows);
            }
        }
    }
    VirtualNode client = new VirtualNode(new Label[] { Label.label("CLIENT") }, singletonMap("name", "Client"), db);
    Optional<Relationship> clientConnection = determineClientConnection(servers, client);
    if (clientConnection.isPresent()) {
        servers.add(client);
        relationships.add(clientConnection.get());
    }
    GraphResult graphResult = new GraphResult(servers, relationships);
    return Stream.of(graphResult);
}
Also used : HashMap(java.util.HashMap) GraphResult(apoc.result.GraphResult) VirtualNode(apoc.result.VirtualNode) VirtualNode(apoc.result.VirtualNode) Node(org.neo4j.graphdb.Node) Label(org.neo4j.graphdb.Label) LinkedList(java.util.LinkedList) GraphResult(apoc.result.GraphResult) Result(org.neo4j.graphdb.Result) VirtualRelationship(apoc.result.VirtualRelationship) VirtualRelationship(apoc.result.VirtualRelationship) Relationship(org.neo4j.graphdb.Relationship) LinkedList(java.util.LinkedList) List(java.util.List) Description(apoc.Description) Procedure(org.neo4j.procedure.Procedure)

Example 4 with VirtualNode

use of apoc.result.VirtualNode in project neo4j-apoc-procedures by neo4j-contrib.

the class Meta method mergeMetaNode.

private Node mergeMetaNode(Label label, Map<String, Node> labels, long increment) {
    String name = label.name();
    Node vNode = labels.get(name);
    if (vNode == null) {
        vNode = new VirtualNode(new Label[] { label }, Collections.singletonMap("name", name), db);
        labels.put(name, vNode);
    }
    if (increment > 0)
        vNode.setProperty("count", (((Number) vNode.getProperty("count", 0L)).longValue()) + increment);
    return vNode;
}
Also used : VirtualNode(apoc.result.VirtualNode) VirtualNode(apoc.result.VirtualNode)

Example 5 with VirtualNode

use of apoc.result.VirtualNode in project neo4j-apoc-procedures by neo4j-contrib.

the class Bolt method toNode.

private Object toNode(Object value, boolean virtual, Map<Long, Object> nodesCache) {
    Value internalValue = ((InternalEntity) value).asValue();
    Node node = internalValue.asNode();
    if (virtual) {
        List<Label> labels = new ArrayList<>();
        node.labels().forEach(l -> labels.add(Label.label(l)));
        VirtualNode virtualNode = new VirtualNode(node.id(), labels.toArray(new Label[0]), node.asMap(), db);
        nodesCache.put(node.id(), virtualNode);
        return virtualNode;
    } else
        return Util.map("entityType", internalValue.type().name(), "labels", node.labels(), "id", node.id(), "properties", node.asMap());
}
Also used : InternalEntity(org.neo4j.driver.internal.InternalEntity) VirtualNode(apoc.result.VirtualNode) VirtualNode(apoc.result.VirtualNode) Node(org.neo4j.driver.v1.types.Node) Label(org.neo4j.graphdb.Label)

Aggregations

VirtualNode (apoc.result.VirtualNode)5 Description (apoc.Description)2 GraphResult (apoc.result.GraphResult)2 VirtualRelationship (apoc.result.VirtualRelationship)2 InternalEntity (org.neo4j.driver.internal.InternalEntity)2 Label (org.neo4j.graphdb.Label)2 Procedure (org.neo4j.procedure.Procedure)2 Pools (apoc.Pools)1 Util (apoc.util.Util)1 java.util (java.util)1 Collections.emptyMap (java.util.Collections.emptyMap)1 Collections.singletonList (java.util.Collections.singletonList)1 HashMap (java.util.HashMap)1 LinkedList (java.util.LinkedList)1 List (java.util.List)1 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)1 ExecutorService (java.util.concurrent.ExecutorService)1 Future (java.util.concurrent.Future)1 Collectors (java.util.stream.Collectors)1 Stream (java.util.stream.Stream)1