Search in sources :

Example 1 with LoopEndNode

use of org.graalvm.compiler.nodes.LoopEndNode in project graal by oracle.

the class GraphUtil method markFixedNodes.

private static void markFixedNodes(FixedNode node, EconomicSet<Node> markedNodes, EconomicMap<AbstractMergeNode, List<AbstractEndNode>> unmarkedMerges) {
    NodeStack workStack = new NodeStack();
    workStack.push(node);
    while (!workStack.isEmpty()) {
        Node fixedNode = workStack.pop();
        markedNodes.add(fixedNode);
        if (fixedNode instanceof AbstractMergeNode) {
            unmarkedMerges.removeKey((AbstractMergeNode) fixedNode);
        }
        while (fixedNode instanceof FixedWithNextNode) {
            fixedNode = ((FixedWithNextNode) fixedNode).next();
            if (fixedNode != null) {
                markedNodes.add(fixedNode);
            }
        }
        if (fixedNode instanceof ControlSplitNode) {
            for (Node successor : fixedNode.successors()) {
                workStack.push(successor);
            }
        } else if (fixedNode instanceof AbstractEndNode) {
            AbstractEndNode end = (AbstractEndNode) fixedNode;
            AbstractMergeNode merge = end.merge();
            if (merge != null) {
                assert !markedNodes.contains(merge) || (merge instanceof LoopBeginNode && end instanceof LoopEndNode) : merge;
                if (merge instanceof LoopBeginNode) {
                    if (end == ((LoopBeginNode) merge).forwardEnd()) {
                        workStack.push(merge);
                        continue;
                    }
                    if (markedNodes.contains(merge)) {
                        continue;
                    }
                }
                List<AbstractEndNode> endsSeen = unmarkedMerges.get(merge);
                if (endsSeen == null) {
                    endsSeen = new ArrayList<>(merge.forwardEndCount());
                    unmarkedMerges.put(merge, endsSeen);
                }
                endsSeen.add(end);
                if (!(end instanceof LoopEndNode) && endsSeen.size() == merge.forwardEndCount()) {
                    assert merge.forwardEnds().filter(n -> !markedNodes.contains(n)).isEmpty();
                    // all this merge's forward ends are marked: it needs to be killed
                    workStack.push(merge);
                }
            }
        }
    }
}
Also used : FixedWithNextNode(org.graalvm.compiler.nodes.FixedWithNextNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) AbstractMergeNode(org.graalvm.compiler.nodes.AbstractMergeNode) MonitorIdNode(org.graalvm.compiler.nodes.java.MonitorIdNode) MethodCallTargetNode(org.graalvm.compiler.nodes.java.MethodCallTargetNode) LoadIndexedNode(org.graalvm.compiler.nodes.java.LoadIndexedNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) VirtualObjectNode(org.graalvm.compiler.nodes.virtual.VirtualObjectNode) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) PiNode(org.graalvm.compiler.nodes.PiNode) ValueNode(org.graalvm.compiler.nodes.ValueNode) VirtualArrayNode(org.graalvm.compiler.nodes.virtual.VirtualArrayNode) GuardNode(org.graalvm.compiler.nodes.GuardNode) ConstantNode(org.graalvm.compiler.nodes.ConstantNode) ControlSplitNode(org.graalvm.compiler.nodes.ControlSplitNode) FixedNode(org.graalvm.compiler.nodes.FixedNode) AbstractEndNode(org.graalvm.compiler.nodes.AbstractEndNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode) LoopExitNode(org.graalvm.compiler.nodes.LoopExitNode) Node(org.graalvm.compiler.graph.Node) FixedWithNextNode(org.graalvm.compiler.nodes.FixedWithNextNode) PhiNode(org.graalvm.compiler.nodes.PhiNode) ProxyNode(org.graalvm.compiler.nodes.ProxyNode) AbstractEndNode(org.graalvm.compiler.nodes.AbstractEndNode) ArrayList(java.util.ArrayList) NodeStack(org.graalvm.compiler.graph.NodeStack) ControlSplitNode(org.graalvm.compiler.nodes.ControlSplitNode) List(java.util.List) ArrayList(java.util.ArrayList) AbstractMergeNode(org.graalvm.compiler.nodes.AbstractMergeNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode)

Example 2 with LoopEndNode

use of org.graalvm.compiler.nodes.LoopEndNode in project graal by oracle.

the class NodeLIRBuilder method isPhiInputFromBackedge.

private static boolean isPhiInputFromBackedge(PhiNode phi, int index) {
    AbstractMergeNode merge = phi.merge();
    AbstractEndNode end = merge.phiPredecessorAt(index);
    return end instanceof LoopEndNode && ((LoopEndNode) end).loopBegin().equals(merge);
}
Also used : AbstractEndNode(org.graalvm.compiler.nodes.AbstractEndNode) AbstractMergeNode(org.graalvm.compiler.nodes.AbstractMergeNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode)

Example 3 with LoopEndNode

use of org.graalvm.compiler.nodes.LoopEndNode in project graal by oracle.

the class LoopFragmentInside method patchPeeling.

private void patchPeeling(LoopFragmentInside peel) {
    LoopBeginNode loopBegin = loop().loopBegin();
    StructuredGraph graph = loopBegin.graph();
    List<PhiNode> newPhis = new LinkedList<>();
    NodeBitMap usagesToPatch = nodes.copy();
    for (LoopExitNode exit : exits()) {
        markStateNodes(exit, usagesToPatch);
        for (ProxyNode proxy : exit.proxies()) {
            usagesToPatch.markAndGrow(proxy);
        }
    }
    markStateNodes(loopBegin, usagesToPatch);
    List<PhiNode> oldPhis = loopBegin.phis().snapshot();
    for (PhiNode phi : oldPhis) {
        if (phi.hasNoUsages()) {
            continue;
        }
        ValueNode first;
        if (loopBegin.loopEnds().count() == 1) {
            // back edge value
            ValueNode b = phi.valueAt(loopBegin.loopEnds().first());
            // corresponding value in the peel
            first = peel.prim(b);
        } else {
            first = peel.mergedInitializers.get(phi);
        }
        // create a new phi (we don't patch the old one since some usages of the old one may
        // still be valid)
        PhiNode newPhi = patchPhi(graph, phi, loopBegin);
        newPhi.addInput(first);
        for (LoopEndNode end : loopBegin.orderedLoopEnds()) {
            newPhi.addInput(phi.valueAt(end));
        }
        peel.putDuplicatedNode(phi, newPhi);
        newPhis.add(newPhi);
        for (Node usage : phi.usages().snapshot()) {
            // patch only usages that should use the new phi ie usages that were peeled
            if (usagesToPatch.isMarkedAndGrow(usage)) {
                usage.replaceFirstInput(phi, newPhi);
            }
        }
    }
    // new corresponding phis
    for (PhiNode phi : newPhis) {
        for (int i = 0; i < phi.valueCount(); i++) {
            ValueNode v = phi.valueAt(i);
            if (loopBegin.isPhiAtMerge(v)) {
                PhiNode newV = peel.getDuplicatedNode((ValuePhiNode) v);
                if (newV != null) {
                    phi.setValueAt(i, newV);
                }
            }
        }
    }
    boolean progress = true;
    while (progress) {
        progress = false;
        int i = 0;
        outer: while (i < oldPhis.size()) {
            PhiNode oldPhi = oldPhis.get(i);
            for (Node usage : oldPhi.usages()) {
                if (usage instanceof PhiNode && oldPhis.contains(usage)) {
                // Do not mark.
                } else {
                    // Mark alive by removing from delete set.
                    oldPhis.remove(i);
                    progress = true;
                    continue outer;
                }
            }
            i++;
        }
    }
    for (PhiNode deadPhi : oldPhis) {
        deadPhi.clearInputs();
    }
    for (PhiNode deadPhi : oldPhis) {
        if (deadPhi.isAlive()) {
            GraphUtil.killWithUnusedFloatingInputs(deadPhi);
        }
    }
}
Also used : ProxyNode(org.graalvm.compiler.nodes.ProxyNode) ValuePhiNode(org.graalvm.compiler.nodes.ValuePhiNode) MemoryPhiNode(org.graalvm.compiler.nodes.memory.MemoryPhiNode) GuardPhiNode(org.graalvm.compiler.nodes.GuardPhiNode) PhiNode(org.graalvm.compiler.nodes.PhiNode) LoopExitNode(org.graalvm.compiler.nodes.LoopExitNode) NodeBitMap(org.graalvm.compiler.graph.NodeBitMap) CompareNode(org.graalvm.compiler.nodes.calc.CompareNode) ValuePhiNode(org.graalvm.compiler.nodes.ValuePhiNode) MemoryPhiNode(org.graalvm.compiler.nodes.memory.MemoryPhiNode) ConstantNode(org.graalvm.compiler.nodes.ConstantNode) AbstractMergeNode(org.graalvm.compiler.nodes.AbstractMergeNode) AddNode(org.graalvm.compiler.nodes.calc.AddNode) BeginNode(org.graalvm.compiler.nodes.BeginNode) MergeNode(org.graalvm.compiler.nodes.MergeNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) FixedNode(org.graalvm.compiler.nodes.FixedNode) IfNode(org.graalvm.compiler.nodes.IfNode) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) AbstractEndNode(org.graalvm.compiler.nodes.AbstractEndNode) SubNode(org.graalvm.compiler.nodes.calc.SubNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode) LogicNode(org.graalvm.compiler.nodes.LogicNode) SafepointNode(org.graalvm.compiler.nodes.SafepointNode) ValueNode(org.graalvm.compiler.nodes.ValueNode) GuardPhiNode(org.graalvm.compiler.nodes.GuardPhiNode) LoopExitNode(org.graalvm.compiler.nodes.LoopExitNode) Node(org.graalvm.compiler.graph.Node) EndNode(org.graalvm.compiler.nodes.EndNode) FixedWithNextNode(org.graalvm.compiler.nodes.FixedWithNextNode) PhiNode(org.graalvm.compiler.nodes.PhiNode) ProxyNode(org.graalvm.compiler.nodes.ProxyNode) LinkedList(java.util.LinkedList) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) StructuredGraph(org.graalvm.compiler.nodes.StructuredGraph) ValueNode(org.graalvm.compiler.nodes.ValueNode)

Example 4 with LoopEndNode

use of org.graalvm.compiler.nodes.LoopEndNode in project graal by oracle.

the class LoopFragmentInside method placeNewSegmentAndCleanup.

private void placeNewSegmentAndCleanup(LoopEx loop) {
    CountedLoopInfo mainCounted = loop.counted();
    LoopBeginNode mainLoopBegin = loop.loopBegin();
    // Discard the segment entry and its flow, after if merging it into the loop
    StructuredGraph graph = mainLoopBegin.graph();
    IfNode loopTest = mainCounted.getLimitTest();
    IfNode newSegmentTest = getDuplicatedNode(loopTest);
    AbstractBeginNode trueSuccessor = loopTest.trueSuccessor();
    AbstractBeginNode falseSuccessor = loopTest.falseSuccessor();
    FixedNode firstNode;
    boolean codeInTrueSide = false;
    if (trueSuccessor == mainCounted.getBody()) {
        firstNode = trueSuccessor.next();
        codeInTrueSide = true;
    } else {
        assert (falseSuccessor == mainCounted.getBody());
        firstNode = falseSuccessor.next();
    }
    trueSuccessor = newSegmentTest.trueSuccessor();
    falseSuccessor = newSegmentTest.falseSuccessor();
    for (Node usage : falseSuccessor.anchored().snapshot()) {
        usage.replaceFirstInput(falseSuccessor, loopTest.falseSuccessor());
    }
    for (Node usage : trueSuccessor.anchored().snapshot()) {
        usage.replaceFirstInput(trueSuccessor, loopTest.trueSuccessor());
    }
    AbstractBeginNode startBlockNode;
    if (codeInTrueSide) {
        startBlockNode = trueSuccessor;
    } else {
        graph.getDebug().dump(DebugContext.VERBOSE_LEVEL, mainLoopBegin.graph(), "before");
        startBlockNode = falseSuccessor;
    }
    FixedNode lastNode = getBlockEnd(startBlockNode);
    LoopEndNode loopEndNode = mainLoopBegin.getSingleLoopEnd();
    FixedWithNextNode lastCodeNode = (FixedWithNextNode) loopEndNode.predecessor();
    FixedNode newSegmentFirstNode = getDuplicatedNode(firstNode);
    FixedWithNextNode newSegmentLastNode = getDuplicatedNode(lastCodeNode);
    graph.getDebug().dump(DebugContext.DETAILED_LEVEL, loopEndNode.graph(), "Before placing segment");
    if (firstNode instanceof LoopEndNode) {
        GraphUtil.killCFG(getDuplicatedNode(mainLoopBegin));
    } else {
        newSegmentLastNode.clearSuccessors();
        startBlockNode.setNext(lastNode);
        lastCodeNode.replaceFirstSuccessor(loopEndNode, newSegmentFirstNode);
        newSegmentLastNode.replaceFirstSuccessor(lastNode, loopEndNode);
        lastCodeNode.setNext(newSegmentFirstNode);
        newSegmentLastNode.setNext(loopEndNode);
        startBlockNode.clearSuccessors();
        lastNode.safeDelete();
        Node newSegmentTestStart = newSegmentTest.predecessor();
        LogicNode newSegmentIfTest = newSegmentTest.condition();
        newSegmentTestStart.clearSuccessors();
        newSegmentTest.safeDelete();
        newSegmentIfTest.safeDelete();
        trueSuccessor.safeDelete();
        falseSuccessor.safeDelete();
        newSegmentTestStart.safeDelete();
    }
    graph.getDebug().dump(DebugContext.DETAILED_LEVEL, loopEndNode.graph(), "After placing segment");
}
Also used : FixedWithNextNode(org.graalvm.compiler.nodes.FixedWithNextNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) StructuredGraph(org.graalvm.compiler.nodes.StructuredGraph) CompareNode(org.graalvm.compiler.nodes.calc.CompareNode) ValuePhiNode(org.graalvm.compiler.nodes.ValuePhiNode) MemoryPhiNode(org.graalvm.compiler.nodes.memory.MemoryPhiNode) ConstantNode(org.graalvm.compiler.nodes.ConstantNode) AbstractMergeNode(org.graalvm.compiler.nodes.AbstractMergeNode) AddNode(org.graalvm.compiler.nodes.calc.AddNode) BeginNode(org.graalvm.compiler.nodes.BeginNode) MergeNode(org.graalvm.compiler.nodes.MergeNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) FixedNode(org.graalvm.compiler.nodes.FixedNode) IfNode(org.graalvm.compiler.nodes.IfNode) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) AbstractEndNode(org.graalvm.compiler.nodes.AbstractEndNode) SubNode(org.graalvm.compiler.nodes.calc.SubNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode) LogicNode(org.graalvm.compiler.nodes.LogicNode) SafepointNode(org.graalvm.compiler.nodes.SafepointNode) ValueNode(org.graalvm.compiler.nodes.ValueNode) GuardPhiNode(org.graalvm.compiler.nodes.GuardPhiNode) LoopExitNode(org.graalvm.compiler.nodes.LoopExitNode) Node(org.graalvm.compiler.graph.Node) EndNode(org.graalvm.compiler.nodes.EndNode) FixedWithNextNode(org.graalvm.compiler.nodes.FixedWithNextNode) PhiNode(org.graalvm.compiler.nodes.PhiNode) ProxyNode(org.graalvm.compiler.nodes.ProxyNode) LogicNode(org.graalvm.compiler.nodes.LogicNode) IfNode(org.graalvm.compiler.nodes.IfNode) FixedNode(org.graalvm.compiler.nodes.FixedNode) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode)

Example 5 with LoopEndNode

use of org.graalvm.compiler.nodes.LoopEndNode in project graal by oracle.

the class LoopFragmentInside method getDuplicationReplacement.

@Override
@SuppressWarnings("try")
protected DuplicationReplacement getDuplicationReplacement() {
    final LoopBeginNode loopBegin = loop().loopBegin();
    final StructuredGraph graph = graph();
    return new DuplicationReplacement() {

        private EconomicMap<Node, Node> seenNode = EconomicMap.create(Equivalence.IDENTITY);

        @Override
        public Node replacement(Node original) {
            try (DebugCloseable position = original.withNodeSourcePosition()) {
                if (original == loopBegin) {
                    Node value = seenNode.get(original);
                    if (value != null) {
                        return value;
                    }
                    AbstractBeginNode newValue = graph.add(new BeginNode());
                    seenNode.put(original, newValue);
                    return newValue;
                }
                if (original instanceof LoopExitNode && ((LoopExitNode) original).loopBegin() == loopBegin) {
                    Node value = seenNode.get(original);
                    if (value != null) {
                        return value;
                    }
                    AbstractBeginNode newValue = graph.add(new BeginNode());
                    seenNode.put(original, newValue);
                    return newValue;
                }
                if (original instanceof LoopEndNode && ((LoopEndNode) original).loopBegin() == loopBegin) {
                    Node value = seenNode.get(original);
                    if (value != null) {
                        return value;
                    }
                    EndNode newValue = graph.add(new EndNode());
                    seenNode.put(original, newValue);
                    return newValue;
                }
                return original;
            }
        }
    };
}
Also used : LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) StructuredGraph(org.graalvm.compiler.nodes.StructuredGraph) LoopExitNode(org.graalvm.compiler.nodes.LoopExitNode) AbstractEndNode(org.graalvm.compiler.nodes.AbstractEndNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode) EndNode(org.graalvm.compiler.nodes.EndNode) EconomicMap(org.graalvm.collections.EconomicMap) BeginNode(org.graalvm.compiler.nodes.BeginNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) CompareNode(org.graalvm.compiler.nodes.calc.CompareNode) ValuePhiNode(org.graalvm.compiler.nodes.ValuePhiNode) MemoryPhiNode(org.graalvm.compiler.nodes.memory.MemoryPhiNode) ConstantNode(org.graalvm.compiler.nodes.ConstantNode) AbstractMergeNode(org.graalvm.compiler.nodes.AbstractMergeNode) AddNode(org.graalvm.compiler.nodes.calc.AddNode) BeginNode(org.graalvm.compiler.nodes.BeginNode) MergeNode(org.graalvm.compiler.nodes.MergeNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) FixedNode(org.graalvm.compiler.nodes.FixedNode) IfNode(org.graalvm.compiler.nodes.IfNode) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) AbstractEndNode(org.graalvm.compiler.nodes.AbstractEndNode) SubNode(org.graalvm.compiler.nodes.calc.SubNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode) LogicNode(org.graalvm.compiler.nodes.LogicNode) SafepointNode(org.graalvm.compiler.nodes.SafepointNode) ValueNode(org.graalvm.compiler.nodes.ValueNode) GuardPhiNode(org.graalvm.compiler.nodes.GuardPhiNode) LoopExitNode(org.graalvm.compiler.nodes.LoopExitNode) Node(org.graalvm.compiler.graph.Node) EndNode(org.graalvm.compiler.nodes.EndNode) FixedWithNextNode(org.graalvm.compiler.nodes.FixedWithNextNode) PhiNode(org.graalvm.compiler.nodes.PhiNode) ProxyNode(org.graalvm.compiler.nodes.ProxyNode) DebugCloseable(org.graalvm.compiler.debug.DebugCloseable) DuplicationReplacement(org.graalvm.compiler.graph.Graph.DuplicationReplacement) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode)

Aggregations

LoopEndNode (org.graalvm.compiler.nodes.LoopEndNode)20 LoopBeginNode (org.graalvm.compiler.nodes.LoopBeginNode)17 FixedNode (org.graalvm.compiler.nodes.FixedNode)15 AbstractMergeNode (org.graalvm.compiler.nodes.AbstractMergeNode)13 EndNode (org.graalvm.compiler.nodes.EndNode)12 FixedWithNextNode (org.graalvm.compiler.nodes.FixedWithNextNode)12 AbstractBeginNode (org.graalvm.compiler.nodes.AbstractBeginNode)10 AbstractEndNode (org.graalvm.compiler.nodes.AbstractEndNode)10 LoopExitNode (org.graalvm.compiler.nodes.LoopExitNode)10 Node (org.graalvm.compiler.graph.Node)8 ControlSplitNode (org.graalvm.compiler.nodes.ControlSplitNode)8 ControlSinkNode (org.graalvm.compiler.nodes.ControlSinkNode)6 MergeNode (org.graalvm.compiler.nodes.MergeNode)6 BeginNode (org.graalvm.compiler.nodes.BeginNode)5 ConstantNode (org.graalvm.compiler.nodes.ConstantNode)5 IfNode (org.graalvm.compiler.nodes.IfNode)5 PhiNode (org.graalvm.compiler.nodes.PhiNode)5 ProxyNode (org.graalvm.compiler.nodes.ProxyNode)5 SafepointNode (org.graalvm.compiler.nodes.SafepointNode)5 ValueNode (org.graalvm.compiler.nodes.ValueNode)5