Search in sources :

Example 51 with Node

use of org.graalvm.compiler.graph.Node in project graal by oracle.

the class LoopFragmentInside method patchPeeling.

private void patchPeeling(LoopFragmentInside peel) {
    LoopBeginNode loopBegin = loop().loopBegin();
    StructuredGraph graph = loopBegin.graph();
    List<PhiNode> newPhis = new LinkedList<>();
    NodeBitMap usagesToPatch = nodes.copy();
    for (LoopExitNode exit : exits()) {
        markStateNodes(exit, usagesToPatch);
        for (ProxyNode proxy : exit.proxies()) {
            usagesToPatch.markAndGrow(proxy);
        }
    }
    markStateNodes(loopBegin, usagesToPatch);
    List<PhiNode> oldPhis = loopBegin.phis().snapshot();
    for (PhiNode phi : oldPhis) {
        if (phi.hasNoUsages()) {
            continue;
        }
        ValueNode first;
        if (loopBegin.loopEnds().count() == 1) {
            // back edge value
            ValueNode b = phi.valueAt(loopBegin.loopEnds().first());
            // corresponding value in the peel
            first = peel.prim(b);
        } else {
            first = peel.mergedInitializers.get(phi);
        }
        // create a new phi (we don't patch the old one since some usages of the old one may
        // still be valid)
        PhiNode newPhi = patchPhi(graph, phi, loopBegin);
        newPhi.addInput(first);
        for (LoopEndNode end : loopBegin.orderedLoopEnds()) {
            newPhi.addInput(phi.valueAt(end));
        }
        peel.putDuplicatedNode(phi, newPhi);
        newPhis.add(newPhi);
        for (Node usage : phi.usages().snapshot()) {
            // patch only usages that should use the new phi ie usages that were peeled
            if (usagesToPatch.isMarkedAndGrow(usage)) {
                usage.replaceFirstInput(phi, newPhi);
            }
        }
    }
    // new corresponding phis
    for (PhiNode phi : newPhis) {
        for (int i = 0; i < phi.valueCount(); i++) {
            ValueNode v = phi.valueAt(i);
            if (loopBegin.isPhiAtMerge(v)) {
                PhiNode newV = peel.getDuplicatedNode((ValuePhiNode) v);
                if (newV != null) {
                    phi.setValueAt(i, newV);
                }
            }
        }
    }
    boolean progress = true;
    while (progress) {
        progress = false;
        int i = 0;
        outer: while (i < oldPhis.size()) {
            PhiNode oldPhi = oldPhis.get(i);
            for (Node usage : oldPhi.usages()) {
                if (usage instanceof PhiNode && oldPhis.contains(usage)) {
                // Do not mark.
                } else {
                    // Mark alive by removing from delete set.
                    oldPhis.remove(i);
                    progress = true;
                    continue outer;
                }
            }
            i++;
        }
    }
    for (PhiNode deadPhi : oldPhis) {
        deadPhi.clearInputs();
    }
    for (PhiNode deadPhi : oldPhis) {
        if (deadPhi.isAlive()) {
            GraphUtil.killWithUnusedFloatingInputs(deadPhi);
        }
    }
}
Also used : ProxyNode(org.graalvm.compiler.nodes.ProxyNode) ValuePhiNode(org.graalvm.compiler.nodes.ValuePhiNode) MemoryPhiNode(org.graalvm.compiler.nodes.memory.MemoryPhiNode) GuardPhiNode(org.graalvm.compiler.nodes.GuardPhiNode) PhiNode(org.graalvm.compiler.nodes.PhiNode) LoopExitNode(org.graalvm.compiler.nodes.LoopExitNode) NodeBitMap(org.graalvm.compiler.graph.NodeBitMap) CompareNode(org.graalvm.compiler.nodes.calc.CompareNode) ValuePhiNode(org.graalvm.compiler.nodes.ValuePhiNode) MemoryPhiNode(org.graalvm.compiler.nodes.memory.MemoryPhiNode) ConstantNode(org.graalvm.compiler.nodes.ConstantNode) AbstractMergeNode(org.graalvm.compiler.nodes.AbstractMergeNode) AddNode(org.graalvm.compiler.nodes.calc.AddNode) BeginNode(org.graalvm.compiler.nodes.BeginNode) MergeNode(org.graalvm.compiler.nodes.MergeNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) FixedNode(org.graalvm.compiler.nodes.FixedNode) IfNode(org.graalvm.compiler.nodes.IfNode) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) AbstractEndNode(org.graalvm.compiler.nodes.AbstractEndNode) SubNode(org.graalvm.compiler.nodes.calc.SubNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode) LogicNode(org.graalvm.compiler.nodes.LogicNode) SafepointNode(org.graalvm.compiler.nodes.SafepointNode) ValueNode(org.graalvm.compiler.nodes.ValueNode) GuardPhiNode(org.graalvm.compiler.nodes.GuardPhiNode) LoopExitNode(org.graalvm.compiler.nodes.LoopExitNode) Node(org.graalvm.compiler.graph.Node) EndNode(org.graalvm.compiler.nodes.EndNode) FixedWithNextNode(org.graalvm.compiler.nodes.FixedWithNextNode) PhiNode(org.graalvm.compiler.nodes.PhiNode) ProxyNode(org.graalvm.compiler.nodes.ProxyNode) LinkedList(java.util.LinkedList) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) StructuredGraph(org.graalvm.compiler.nodes.StructuredGraph) ValueNode(org.graalvm.compiler.nodes.ValueNode)

Example 52 with Node

use of org.graalvm.compiler.graph.Node in project graal by oracle.

the class LoopFragmentInside method placeNewSegmentAndCleanup.

private void placeNewSegmentAndCleanup(LoopEx loop) {
    CountedLoopInfo mainCounted = loop.counted();
    LoopBeginNode mainLoopBegin = loop.loopBegin();
    // Discard the segment entry and its flow, after if merging it into the loop
    StructuredGraph graph = mainLoopBegin.graph();
    IfNode loopTest = mainCounted.getLimitTest();
    IfNode newSegmentTest = getDuplicatedNode(loopTest);
    AbstractBeginNode trueSuccessor = loopTest.trueSuccessor();
    AbstractBeginNode falseSuccessor = loopTest.falseSuccessor();
    FixedNode firstNode;
    boolean codeInTrueSide = false;
    if (trueSuccessor == mainCounted.getBody()) {
        firstNode = trueSuccessor.next();
        codeInTrueSide = true;
    } else {
        assert (falseSuccessor == mainCounted.getBody());
        firstNode = falseSuccessor.next();
    }
    trueSuccessor = newSegmentTest.trueSuccessor();
    falseSuccessor = newSegmentTest.falseSuccessor();
    for (Node usage : falseSuccessor.anchored().snapshot()) {
        usage.replaceFirstInput(falseSuccessor, loopTest.falseSuccessor());
    }
    for (Node usage : trueSuccessor.anchored().snapshot()) {
        usage.replaceFirstInput(trueSuccessor, loopTest.trueSuccessor());
    }
    AbstractBeginNode startBlockNode;
    if (codeInTrueSide) {
        startBlockNode = trueSuccessor;
    } else {
        graph.getDebug().dump(DebugContext.VERBOSE_LEVEL, mainLoopBegin.graph(), "before");
        startBlockNode = falseSuccessor;
    }
    FixedNode lastNode = getBlockEnd(startBlockNode);
    LoopEndNode loopEndNode = mainLoopBegin.getSingleLoopEnd();
    FixedWithNextNode lastCodeNode = (FixedWithNextNode) loopEndNode.predecessor();
    FixedNode newSegmentFirstNode = getDuplicatedNode(firstNode);
    FixedWithNextNode newSegmentLastNode = getDuplicatedNode(lastCodeNode);
    graph.getDebug().dump(DebugContext.DETAILED_LEVEL, loopEndNode.graph(), "Before placing segment");
    if (firstNode instanceof LoopEndNode) {
        GraphUtil.killCFG(getDuplicatedNode(mainLoopBegin));
    } else {
        newSegmentLastNode.clearSuccessors();
        startBlockNode.setNext(lastNode);
        lastCodeNode.replaceFirstSuccessor(loopEndNode, newSegmentFirstNode);
        newSegmentLastNode.replaceFirstSuccessor(lastNode, loopEndNode);
        lastCodeNode.setNext(newSegmentFirstNode);
        newSegmentLastNode.setNext(loopEndNode);
        startBlockNode.clearSuccessors();
        lastNode.safeDelete();
        Node newSegmentTestStart = newSegmentTest.predecessor();
        LogicNode newSegmentIfTest = newSegmentTest.condition();
        newSegmentTestStart.clearSuccessors();
        newSegmentTest.safeDelete();
        newSegmentIfTest.safeDelete();
        trueSuccessor.safeDelete();
        falseSuccessor.safeDelete();
        newSegmentTestStart.safeDelete();
    }
    graph.getDebug().dump(DebugContext.DETAILED_LEVEL, loopEndNode.graph(), "After placing segment");
}
Also used : FixedWithNextNode(org.graalvm.compiler.nodes.FixedWithNextNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) StructuredGraph(org.graalvm.compiler.nodes.StructuredGraph) CompareNode(org.graalvm.compiler.nodes.calc.CompareNode) ValuePhiNode(org.graalvm.compiler.nodes.ValuePhiNode) MemoryPhiNode(org.graalvm.compiler.nodes.memory.MemoryPhiNode) ConstantNode(org.graalvm.compiler.nodes.ConstantNode) AbstractMergeNode(org.graalvm.compiler.nodes.AbstractMergeNode) AddNode(org.graalvm.compiler.nodes.calc.AddNode) BeginNode(org.graalvm.compiler.nodes.BeginNode) MergeNode(org.graalvm.compiler.nodes.MergeNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) FixedNode(org.graalvm.compiler.nodes.FixedNode) IfNode(org.graalvm.compiler.nodes.IfNode) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) AbstractEndNode(org.graalvm.compiler.nodes.AbstractEndNode) SubNode(org.graalvm.compiler.nodes.calc.SubNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode) LogicNode(org.graalvm.compiler.nodes.LogicNode) SafepointNode(org.graalvm.compiler.nodes.SafepointNode) ValueNode(org.graalvm.compiler.nodes.ValueNode) GuardPhiNode(org.graalvm.compiler.nodes.GuardPhiNode) LoopExitNode(org.graalvm.compiler.nodes.LoopExitNode) Node(org.graalvm.compiler.graph.Node) EndNode(org.graalvm.compiler.nodes.EndNode) FixedWithNextNode(org.graalvm.compiler.nodes.FixedWithNextNode) PhiNode(org.graalvm.compiler.nodes.PhiNode) ProxyNode(org.graalvm.compiler.nodes.ProxyNode) LogicNode(org.graalvm.compiler.nodes.LogicNode) IfNode(org.graalvm.compiler.nodes.IfNode) FixedNode(org.graalvm.compiler.nodes.FixedNode) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode)

Example 53 with Node

use of org.graalvm.compiler.graph.Node in project graal by oracle.

the class LoopFragmentInside method getDuplicationReplacement.

@Override
@SuppressWarnings("try")
protected DuplicationReplacement getDuplicationReplacement() {
    final LoopBeginNode loopBegin = loop().loopBegin();
    final StructuredGraph graph = graph();
    return new DuplicationReplacement() {

        private EconomicMap<Node, Node> seenNode = EconomicMap.create(Equivalence.IDENTITY);

        @Override
        public Node replacement(Node original) {
            try (DebugCloseable position = original.withNodeSourcePosition()) {
                if (original == loopBegin) {
                    Node value = seenNode.get(original);
                    if (value != null) {
                        return value;
                    }
                    AbstractBeginNode newValue = graph.add(new BeginNode());
                    seenNode.put(original, newValue);
                    return newValue;
                }
                if (original instanceof LoopExitNode && ((LoopExitNode) original).loopBegin() == loopBegin) {
                    Node value = seenNode.get(original);
                    if (value != null) {
                        return value;
                    }
                    AbstractBeginNode newValue = graph.add(new BeginNode());
                    seenNode.put(original, newValue);
                    return newValue;
                }
                if (original instanceof LoopEndNode && ((LoopEndNode) original).loopBegin() == loopBegin) {
                    Node value = seenNode.get(original);
                    if (value != null) {
                        return value;
                    }
                    EndNode newValue = graph.add(new EndNode());
                    seenNode.put(original, newValue);
                    return newValue;
                }
                return original;
            }
        }
    };
}
Also used : LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) StructuredGraph(org.graalvm.compiler.nodes.StructuredGraph) LoopExitNode(org.graalvm.compiler.nodes.LoopExitNode) AbstractEndNode(org.graalvm.compiler.nodes.AbstractEndNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode) EndNode(org.graalvm.compiler.nodes.EndNode) EconomicMap(org.graalvm.collections.EconomicMap) BeginNode(org.graalvm.compiler.nodes.BeginNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) CompareNode(org.graalvm.compiler.nodes.calc.CompareNode) ValuePhiNode(org.graalvm.compiler.nodes.ValuePhiNode) MemoryPhiNode(org.graalvm.compiler.nodes.memory.MemoryPhiNode) ConstantNode(org.graalvm.compiler.nodes.ConstantNode) AbstractMergeNode(org.graalvm.compiler.nodes.AbstractMergeNode) AddNode(org.graalvm.compiler.nodes.calc.AddNode) BeginNode(org.graalvm.compiler.nodes.BeginNode) MergeNode(org.graalvm.compiler.nodes.MergeNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) FixedNode(org.graalvm.compiler.nodes.FixedNode) IfNode(org.graalvm.compiler.nodes.IfNode) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) AbstractEndNode(org.graalvm.compiler.nodes.AbstractEndNode) SubNode(org.graalvm.compiler.nodes.calc.SubNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode) LogicNode(org.graalvm.compiler.nodes.LogicNode) SafepointNode(org.graalvm.compiler.nodes.SafepointNode) ValueNode(org.graalvm.compiler.nodes.ValueNode) GuardPhiNode(org.graalvm.compiler.nodes.GuardPhiNode) LoopExitNode(org.graalvm.compiler.nodes.LoopExitNode) Node(org.graalvm.compiler.graph.Node) EndNode(org.graalvm.compiler.nodes.EndNode) FixedWithNextNode(org.graalvm.compiler.nodes.FixedWithNextNode) PhiNode(org.graalvm.compiler.nodes.PhiNode) ProxyNode(org.graalvm.compiler.nodes.ProxyNode) DebugCloseable(org.graalvm.compiler.debug.DebugCloseable) DuplicationReplacement(org.graalvm.compiler.graph.Graph.DuplicationReplacement) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode)

Example 54 with Node

use of org.graalvm.compiler.graph.Node in project graal by oracle.

the class LoopFragmentWhole method getDuplicationReplacement.

@Override
protected DuplicationReplacement getDuplicationReplacement() {
    final FixedNode entry = loop().entryPoint();
    final Graph graph = this.graph();
    return new DuplicationReplacement() {

        private EndNode endNode;

        @Override
        public Node replacement(Node o) {
            if (o == entry) {
                if (endNode == null) {
                    endNode = graph.add(new EndNode());
                }
                return endNode;
            }
            return o;
        }
    };
}
Also used : Graph(org.graalvm.compiler.graph.Graph) StructuredGraph(org.graalvm.compiler.nodes.StructuredGraph) EndNode(org.graalvm.compiler.nodes.EndNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) FixedNode(org.graalvm.compiler.nodes.FixedNode) ValueNode(org.graalvm.compiler.nodes.ValueNode) LoopExitNode(org.graalvm.compiler.nodes.LoopExitNode) Node(org.graalvm.compiler.graph.Node) EndNode(org.graalvm.compiler.nodes.EndNode) FixedNode(org.graalvm.compiler.nodes.FixedNode) DuplicationReplacement(org.graalvm.compiler.graph.Graph.DuplicationReplacement)

Example 55 with Node

use of org.graalvm.compiler.graph.Node in project graal by oracle.

the class LoopFragment method markFloating.

private static void markFloating(Deque<WorkListEntry> workList, Node start, NodeBitMap loopNodes, NodeBitMap nonLoopNodes) {
    if (isLoopNode(start, loopNodes, nonLoopNodes).isKnown()) {
        return;
    }
    pushWorkList(workList, start, loopNodes);
    while (!workList.isEmpty()) {
        WorkListEntry currentEntry = workList.peek();
        if (currentEntry.usages.hasNext()) {
            Node current = currentEntry.usages.next();
            TriState result = isLoopNode(current, loopNodes, nonLoopNodes);
            if (result.isKnown()) {
                if (result.toBoolean()) {
                    currentEntry.isLoopNode = true;
                }
            } else {
                pushWorkList(workList, current, loopNodes);
            }
        } else {
            workList.pop();
            boolean isLoopNode = currentEntry.isLoopNode;
            Node current = currentEntry.n;
            if (!isLoopNode && current instanceof GuardNode) {
                /*
                     * (gd) this is only OK if we are not going to make loop transforms based on
                     * this
                     */
                assert !((GuardNode) current).graph().hasValueProxies();
                isLoopNode = true;
            }
            if (isLoopNode) {
                loopNodes.mark(current);
                for (WorkListEntry e : workList) {
                    e.isLoopNode = true;
                }
            } else {
                nonLoopNodes.mark(current);
            }
        }
    }
}
Also used : GuardNode(org.graalvm.compiler.nodes.GuardNode) ValuePhiNode(org.graalvm.compiler.nodes.ValuePhiNode) GuardProxyNode(org.graalvm.compiler.nodes.GuardProxyNode) AbstractMergeNode(org.graalvm.compiler.nodes.AbstractMergeNode) MergeNode(org.graalvm.compiler.nodes.MergeNode) FixedNode(org.graalvm.compiler.nodes.FixedNode) VirtualObjectNode(org.graalvm.compiler.nodes.virtual.VirtualObjectNode) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) CommitAllocationNode(org.graalvm.compiler.nodes.virtual.CommitAllocationNode) ValueNode(org.graalvm.compiler.nodes.ValueNode) ValueProxyNode(org.graalvm.compiler.nodes.ValueProxyNode) GuardPhiNode(org.graalvm.compiler.nodes.GuardPhiNode) LoopExitNode(org.graalvm.compiler.nodes.LoopExitNode) MonitorEnterNode(org.graalvm.compiler.nodes.java.MonitorEnterNode) Node(org.graalvm.compiler.graph.Node) EndNode(org.graalvm.compiler.nodes.EndNode) PhiNode(org.graalvm.compiler.nodes.PhiNode) ProxyNode(org.graalvm.compiler.nodes.ProxyNode) GuardNode(org.graalvm.compiler.nodes.GuardNode) TriState(jdk.vm.ci.meta.TriState)

Aggregations

Node (org.graalvm.compiler.graph.Node)189 ValueNode (org.graalvm.compiler.nodes.ValueNode)105 FixedNode (org.graalvm.compiler.nodes.FixedNode)91 AbstractMergeNode (org.graalvm.compiler.nodes.AbstractMergeNode)75 FixedWithNextNode (org.graalvm.compiler.nodes.FixedWithNextNode)74 AbstractBeginNode (org.graalvm.compiler.nodes.AbstractBeginNode)73 PhiNode (org.graalvm.compiler.nodes.PhiNode)64 ConstantNode (org.graalvm.compiler.nodes.ConstantNode)61 LoopBeginNode (org.graalvm.compiler.nodes.LoopBeginNode)53 AbstractEndNode (org.graalvm.compiler.nodes.AbstractEndNode)47 StructuredGraph (org.graalvm.compiler.nodes.StructuredGraph)43 FloatingNode (org.graalvm.compiler.nodes.calc.FloatingNode)41 ParameterNode (org.graalvm.compiler.nodes.ParameterNode)38 EndNode (org.graalvm.compiler.nodes.EndNode)37 LoopExitNode (org.graalvm.compiler.nodes.LoopExitNode)37 MethodCallTargetNode (org.graalvm.compiler.nodes.java.MethodCallTargetNode)37 MergeNode (org.graalvm.compiler.nodes.MergeNode)35 ReturnNode (org.graalvm.compiler.nodes.ReturnNode)35 VirtualObjectNode (org.graalvm.compiler.nodes.virtual.VirtualObjectNode)32 LogicNode (org.graalvm.compiler.nodes.LogicNode)31