Search in sources :

Example 6 with FixedNode

use of org.graalvm.compiler.nodes.FixedNode in project graal by oracle.

the class GraphUtil method markFixedNodes.

private static void markFixedNodes(FixedNode node, EconomicSet<Node> markedNodes, EconomicMap<AbstractMergeNode, List<AbstractEndNode>> unmarkedMerges) {
    NodeStack workStack = new NodeStack();
    workStack.push(node);
    while (!workStack.isEmpty()) {
        Node fixedNode = workStack.pop();
        markedNodes.add(fixedNode);
        if (fixedNode instanceof AbstractMergeNode) {
            unmarkedMerges.removeKey((AbstractMergeNode) fixedNode);
        }
        while (fixedNode instanceof FixedWithNextNode) {
            fixedNode = ((FixedWithNextNode) fixedNode).next();
            if (fixedNode != null) {
                markedNodes.add(fixedNode);
            }
        }
        if (fixedNode instanceof ControlSplitNode) {
            for (Node successor : fixedNode.successors()) {
                workStack.push(successor);
            }
        } else if (fixedNode instanceof AbstractEndNode) {
            AbstractEndNode end = (AbstractEndNode) fixedNode;
            AbstractMergeNode merge = end.merge();
            if (merge != null) {
                assert !markedNodes.contains(merge) || (merge instanceof LoopBeginNode && end instanceof LoopEndNode) : merge;
                if (merge instanceof LoopBeginNode) {
                    if (end == ((LoopBeginNode) merge).forwardEnd()) {
                        workStack.push(merge);
                        continue;
                    }
                    if (markedNodes.contains(merge)) {
                        continue;
                    }
                }
                List<AbstractEndNode> endsSeen = unmarkedMerges.get(merge);
                if (endsSeen == null) {
                    endsSeen = new ArrayList<>(merge.forwardEndCount());
                    unmarkedMerges.put(merge, endsSeen);
                }
                endsSeen.add(end);
                if (!(end instanceof LoopEndNode) && endsSeen.size() == merge.forwardEndCount()) {
                    assert merge.forwardEnds().filter(n -> !markedNodes.contains(n)).isEmpty();
                    // all this merge's forward ends are marked: it needs to be killed
                    workStack.push(merge);
                }
            }
        }
    }
}
Also used : FixedWithNextNode(org.graalvm.compiler.nodes.FixedWithNextNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) AbstractMergeNode(org.graalvm.compiler.nodes.AbstractMergeNode) MonitorIdNode(org.graalvm.compiler.nodes.java.MonitorIdNode) MethodCallTargetNode(org.graalvm.compiler.nodes.java.MethodCallTargetNode) LoadIndexedNode(org.graalvm.compiler.nodes.java.LoadIndexedNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) VirtualObjectNode(org.graalvm.compiler.nodes.virtual.VirtualObjectNode) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) PiNode(org.graalvm.compiler.nodes.PiNode) ValueNode(org.graalvm.compiler.nodes.ValueNode) VirtualArrayNode(org.graalvm.compiler.nodes.virtual.VirtualArrayNode) GuardNode(org.graalvm.compiler.nodes.GuardNode) ConstantNode(org.graalvm.compiler.nodes.ConstantNode) ControlSplitNode(org.graalvm.compiler.nodes.ControlSplitNode) FixedNode(org.graalvm.compiler.nodes.FixedNode) AbstractEndNode(org.graalvm.compiler.nodes.AbstractEndNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode) LoopExitNode(org.graalvm.compiler.nodes.LoopExitNode) Node(org.graalvm.compiler.graph.Node) FixedWithNextNode(org.graalvm.compiler.nodes.FixedWithNextNode) PhiNode(org.graalvm.compiler.nodes.PhiNode) ProxyNode(org.graalvm.compiler.nodes.ProxyNode) AbstractEndNode(org.graalvm.compiler.nodes.AbstractEndNode) ArrayList(java.util.ArrayList) NodeStack(org.graalvm.compiler.graph.NodeStack) ControlSplitNode(org.graalvm.compiler.nodes.ControlSplitNode) List(java.util.List) ArrayList(java.util.ArrayList) AbstractMergeNode(org.graalvm.compiler.nodes.AbstractMergeNode) LoopEndNode(org.graalvm.compiler.nodes.LoopEndNode)

Example 7 with FixedNode

use of org.graalvm.compiler.nodes.FixedNode in project graal by oracle.

the class MethodReturnNode method lower.

@Override
public void lower(LoweringTool tool) {
    FixedNode originalNext = next();
    replaceFirstSuccessor(originalNext, null);
    GraphUtil.killCFG(originalNext);
    replaceAtPredecessor(graph().add(new ReturnNode(result)));
    safeDelete();
}
Also used : ReturnNode(org.graalvm.compiler.nodes.ReturnNode) FixedNode(org.graalvm.compiler.nodes.FixedNode)

Example 8 with FixedNode

use of org.graalvm.compiler.nodes.FixedNode in project graal by oracle.

the class DefaultHotSpotLoweringProvider method lowerOSRStartNode.

private void lowerOSRStartNode(OSRStartNode osrStart) {
    StructuredGraph graph = osrStart.graph();
    if (graph.getGuardsStage() == StructuredGraph.GuardsStage.FIXED_DEOPTS) {
        StartNode newStart = graph.add(new StartNode());
        ParameterNode buffer = graph.addWithoutUnique(new ParameterNode(0, StampPair.createSingle(StampFactory.forKind(runtime.getTarget().wordJavaKind))));
        ForeignCallNode migrationEnd = graph.add(new ForeignCallNode(foreignCalls, OSR_MIGRATION_END, buffer));
        migrationEnd.setStateAfter(osrStart.stateAfter());
        newStart.setNext(migrationEnd);
        FixedNode next = osrStart.next();
        osrStart.setNext(null);
        migrationEnd.setNext(next);
        graph.setStart(newStart);
        final int wordSize = target.wordSize;
        // @formatter:off
        // taken from c2 locals_addr = osr_buf + (max_locals-1)*wordSize)
        // @formatter:on
        int localsOffset = (graph.method().getMaxLocals() - 1) * wordSize;
        for (OSRLocalNode osrLocal : graph.getNodes(OSRLocalNode.TYPE)) {
            int size = osrLocal.getStackKind().getSlotCount();
            int offset = localsOffset - (osrLocal.index() + size - 1) * wordSize;
            AddressNode address = createOffsetAddress(graph, buffer, offset);
            ReadNode load = graph.add(new ReadNode(address, any(), osrLocal.stamp(NodeView.DEFAULT), BarrierType.NONE));
            osrLocal.replaceAndDelete(load);
            graph.addBeforeFixed(migrationEnd, load);
        }
        // @formatter:off
        // taken from c2 monitors_addr = osr_buf + (max_locals+mcnt*2-1)*wordSize);
        // @formatter:on
        final int lockCount = osrStart.stateAfter().locksSize();
        final int locksOffset = (graph.method().getMaxLocals() + lockCount * 2 - 1) * wordSize;
        // buffer
        for (OSRMonitorEnterNode osrMonitorEnter : graph.getNodes(OSRMonitorEnterNode.TYPE)) {
            MonitorIdNode monitorID = osrMonitorEnter.getMonitorId();
            OSRLockNode lock = (OSRLockNode) osrMonitorEnter.object();
            final int index = lock.index();
            final int offsetDisplacedHeader = locksOffset - ((index * 2) + 1) * wordSize;
            final int offsetLockObject = locksOffset - index * 2 * wordSize;
            // load the displaced mark from the osr buffer
            AddressNode addressDisplacedHeader = createOffsetAddress(graph, buffer, offsetDisplacedHeader);
            ReadNode loadDisplacedHeader = graph.add(new ReadNode(addressDisplacedHeader, any(), lock.stamp(NodeView.DEFAULT), BarrierType.NONE));
            graph.addBeforeFixed(migrationEnd, loadDisplacedHeader);
            // we need to initialize the stack slot for the lock
            BeginLockScopeNode beginLockScope = graph.add(new BeginLockScopeNode(lock.getStackKind(), monitorID.getLockDepth()));
            graph.addBeforeFixed(migrationEnd, beginLockScope);
            // write the displaced mark to the correct stack slot
            AddressNode addressDisplacedMark = createOffsetAddress(graph, beginLockScope, runtime.getVMConfig().basicLockDisplacedHeaderOffset);
            WriteNode writeStackSlot = graph.add(new WriteNode(addressDisplacedMark, DISPLACED_MARK_WORD_LOCATION, loadDisplacedHeader, BarrierType.NONE));
            graph.addBeforeFixed(migrationEnd, writeStackSlot);
            // load the lock object from the osr buffer
            AddressNode addressLockObject = createOffsetAddress(graph, buffer, offsetLockObject);
            ReadNode loadObject = graph.add(new ReadNode(addressLockObject, any(), lock.stamp(NodeView.DEFAULT), BarrierType.NONE));
            lock.replaceAndDelete(loadObject);
            graph.addBeforeFixed(migrationEnd, loadObject);
        }
        osrStart.replaceAtUsagesAndDelete(newStart);
    }
}
Also used : MonitorIdNode(org.graalvm.compiler.nodes.java.MonitorIdNode) OSRStartNode(org.graalvm.compiler.nodes.extended.OSRStartNode) StartNode(org.graalvm.compiler.nodes.StartNode) OSRLocalNode(org.graalvm.compiler.nodes.extended.OSRLocalNode) FixedNode(org.graalvm.compiler.nodes.FixedNode) OSRLockNode(org.graalvm.compiler.nodes.extended.OSRLockNode) OSRMonitorEnterNode(org.graalvm.compiler.nodes.extended.OSRMonitorEnterNode) ForeignCallNode(org.graalvm.compiler.nodes.extended.ForeignCallNode) StructuredGraph(org.graalvm.compiler.nodes.StructuredGraph) ParameterNode(org.graalvm.compiler.nodes.ParameterNode) BeginLockScopeNode(org.graalvm.compiler.hotspot.nodes.BeginLockScopeNode) GetObjectAddressNode(org.graalvm.compiler.hotspot.nodes.GetObjectAddressNode) ComputeObjectAddressNode(org.graalvm.compiler.hotspot.nodes.ComputeObjectAddressNode) AddressNode(org.graalvm.compiler.nodes.memory.address.AddressNode) FloatingReadNode(org.graalvm.compiler.nodes.memory.FloatingReadNode) ReadNode(org.graalvm.compiler.nodes.memory.ReadNode) WriteNode(org.graalvm.compiler.nodes.memory.WriteNode)

Example 9 with FixedNode

use of org.graalvm.compiler.nodes.FixedNode in project graal by oracle.

the class SnippetTemplate method instantiate.

/**
 * Replaces a given fixed node with this specialized snippet.
 *
 * @param metaAccess
 * @param replacee the node that will be replaced
 * @param replacer object that replaces the usages of {@code replacee}
 * @param args the arguments to be bound to the flattened positional parameters of the snippet
 * @param killReplacee is true, the replacee node is deleted
 * @return the map of duplicated nodes (original -&gt; duplicate)
 */
@SuppressWarnings("try")
public UnmodifiableEconomicMap<Node, Node> instantiate(MetaAccessProvider metaAccess, FixedNode replacee, UsageReplacer replacer, Arguments args, boolean killReplacee) {
    DebugContext debug = replacee.getDebug();
    assert assertSnippetKills(replacee);
    try (DebugCloseable a = args.info.instantiationTimer.start(debug)) {
        args.info.instantiationCounter.increment(debug);
        // Inline the snippet nodes, replacing parameters with the given args in the process
        StartNode entryPointNode = snippet.start();
        FixedNode firstCFGNode = entryPointNode.next();
        StructuredGraph replaceeGraph = replacee.graph();
        EconomicMap<Node, Node> replacements = bind(replaceeGraph, metaAccess, args);
        replacements.put(entryPointNode, AbstractBeginNode.prevBegin(replacee));
        UnmodifiableEconomicMap<Node, Node> duplicates = inlineSnippet(replacee, debug, replaceeGraph, replacements);
        // Re-wire the control flow graph around the replacee
        FixedNode firstCFGNodeDuplicate = (FixedNode) duplicates.get(firstCFGNode);
        replacee.replaceAtPredecessor(firstCFGNodeDuplicate);
        rewireFrameStates(replacee, duplicates);
        if (replacee instanceof DeoptimizingNode) {
            DeoptimizingNode replaceeDeopt = (DeoptimizingNode) replacee;
            FrameState stateBefore = null;
            FrameState stateDuring = null;
            FrameState stateAfter = null;
            if (replaceeDeopt.canDeoptimize()) {
                if (replaceeDeopt instanceof DeoptimizingNode.DeoptBefore) {
                    stateBefore = ((DeoptimizingNode.DeoptBefore) replaceeDeopt).stateBefore();
                }
                if (replaceeDeopt instanceof DeoptimizingNode.DeoptDuring) {
                    stateDuring = ((DeoptimizingNode.DeoptDuring) replaceeDeopt).stateDuring();
                }
                if (replaceeDeopt instanceof DeoptimizingNode.DeoptAfter) {
                    stateAfter = ((DeoptimizingNode.DeoptAfter) replaceeDeopt).stateAfter();
                }
            }
            for (DeoptimizingNode deoptNode : deoptNodes) {
                DeoptimizingNode deoptDup = (DeoptimizingNode) duplicates.get(deoptNode.asNode());
                if (deoptDup.canDeoptimize()) {
                    if (deoptDup instanceof DeoptimizingNode.DeoptBefore) {
                        ((DeoptimizingNode.DeoptBefore) deoptDup).setStateBefore(stateBefore);
                    }
                    if (deoptDup instanceof DeoptimizingNode.DeoptDuring) {
                        DeoptimizingNode.DeoptDuring deoptDupDuring = (DeoptimizingNode.DeoptDuring) deoptDup;
                        if (stateDuring != null) {
                            deoptDupDuring.setStateDuring(stateDuring);
                        } else if (stateAfter != null) {
                            deoptDupDuring.computeStateDuring(stateAfter);
                        } else if (stateBefore != null) {
                            assert !deoptDupDuring.hasSideEffect() : "can't use stateBefore as stateDuring for state split " + deoptDupDuring;
                            deoptDupDuring.setStateDuring(stateBefore);
                        }
                    }
                    if (deoptDup instanceof DeoptimizingNode.DeoptAfter) {
                        DeoptimizingNode.DeoptAfter deoptDupAfter = (DeoptimizingNode.DeoptAfter) deoptDup;
                        if (stateAfter != null) {
                            deoptDupAfter.setStateAfter(stateAfter);
                        } else {
                            assert !deoptDupAfter.hasSideEffect() : "can't use stateBefore as stateAfter for state split " + deoptDupAfter;
                            deoptDupAfter.setStateAfter(stateBefore);
                        }
                    }
                }
            }
        }
        updateStamps(replacee, duplicates);
        rewireMemoryGraph(replacee, duplicates);
        // Replace all usages of the replacee with the value returned by the snippet
        ValueNode returnValue = null;
        if (returnNode != null && !(replacee instanceof ControlSinkNode)) {
            ReturnNode returnDuplicate = (ReturnNode) duplicates.get(returnNode);
            returnValue = returnDuplicate.result();
            if (returnValue == null && replacee.usages().isNotEmpty() && replacee instanceof MemoryCheckpoint) {
                replacer.replace(replacee, null);
            } else {
                assert returnValue != null || replacee.hasNoUsages();
                replacer.replace(replacee, returnValue);
            }
            if (returnDuplicate.isAlive()) {
                FixedNode next = null;
                if (replacee instanceof FixedWithNextNode) {
                    FixedWithNextNode fwn = (FixedWithNextNode) replacee;
                    next = fwn.next();
                    fwn.setNext(null);
                }
                returnDuplicate.replaceAndDelete(next);
            }
        }
        if (killReplacee) {
            // Remove the replacee from its graph
            GraphUtil.killCFG(replacee);
        }
        debug.dump(DebugContext.DETAILED_LEVEL, replaceeGraph, "After lowering %s with %s", replacee, this);
        return duplicates;
    }
}
Also used : StartNode(org.graalvm.compiler.nodes.StartNode) FixedWithNextNode(org.graalvm.compiler.nodes.FixedWithNextNode) MemoryNode(org.graalvm.compiler.nodes.memory.MemoryNode) MemoryAnchorNode(org.graalvm.compiler.nodes.memory.MemoryAnchorNode) LoopBeginNode(org.graalvm.compiler.nodes.LoopBeginNode) ControlSinkNode(org.graalvm.compiler.nodes.ControlSinkNode) ConstantNode(org.graalvm.compiler.nodes.ConstantNode) MergeNode(org.graalvm.compiler.nodes.MergeNode) ExplodeLoopNode(org.graalvm.compiler.replacements.nodes.ExplodeLoopNode) FixedNode(org.graalvm.compiler.nodes.FixedNode) LoadSnippetVarargParameterNode(org.graalvm.compiler.replacements.nodes.LoadSnippetVarargParameterNode) PhiNode(org.graalvm.compiler.nodes.PhiNode) AbstractMergeNode(org.graalvm.compiler.nodes.AbstractMergeNode) LoadIndexedNode(org.graalvm.compiler.nodes.java.LoadIndexedNode) ReturnNode(org.graalvm.compiler.nodes.ReturnNode) AbstractBeginNode(org.graalvm.compiler.nodes.AbstractBeginNode) DeoptimizingNode(org.graalvm.compiler.nodes.DeoptimizingNode) ValueNode(org.graalvm.compiler.nodes.ValueNode) FloatingNode(org.graalvm.compiler.nodes.calc.FloatingNode) MemoryPhiNode(org.graalvm.compiler.nodes.memory.MemoryPhiNode) StartNode(org.graalvm.compiler.nodes.StartNode) ParameterNode(org.graalvm.compiler.nodes.ParameterNode) StoreIndexedNode(org.graalvm.compiler.nodes.java.StoreIndexedNode) MemoryMapNode(org.graalvm.compiler.nodes.memory.MemoryMapNode) Node(org.graalvm.compiler.graph.Node) FixedWithNextNode(org.graalvm.compiler.nodes.FixedWithNextNode) DebugContext(org.graalvm.compiler.debug.DebugContext) FixedNode(org.graalvm.compiler.nodes.FixedNode) FrameState(org.graalvm.compiler.nodes.FrameState) ControlSinkNode(org.graalvm.compiler.nodes.ControlSinkNode) DeoptimizingNode(org.graalvm.compiler.nodes.DeoptimizingNode) ReturnNode(org.graalvm.compiler.nodes.ReturnNode) StructuredGraph(org.graalvm.compiler.nodes.StructuredGraph) MemoryCheckpoint(org.graalvm.compiler.nodes.memory.MemoryCheckpoint) ValueNode(org.graalvm.compiler.nodes.ValueNode) DebugCloseable(org.graalvm.compiler.debug.DebugCloseable)

Example 10 with FixedNode

use of org.graalvm.compiler.nodes.FixedNode in project graal by oracle.

the class Block method calcKillLocations.

private LocationSet calcKillLocations() {
    LocationSet result = new LocationSet();
    for (FixedNode node : this.getNodes()) {
        if (node instanceof MemoryCheckpoint.Single) {
            LocationIdentity identity = ((MemoryCheckpoint.Single) node).getLocationIdentity();
            result.add(identity);
        } else if (node instanceof MemoryCheckpoint.Multi) {
            for (LocationIdentity identity : ((MemoryCheckpoint.Multi) node).getLocationIdentities()) {
                result.add(identity);
            }
        }
        if (result.isAny()) {
            break;
        }
    }
    return result;
}
Also used : MemoryCheckpoint(org.graalvm.compiler.nodes.memory.MemoryCheckpoint) LocationIdentity(org.graalvm.word.LocationIdentity) FixedNode(org.graalvm.compiler.nodes.FixedNode)

Aggregations

FixedNode (org.graalvm.compiler.nodes.FixedNode)87 Node (org.graalvm.compiler.graph.Node)41 FixedWithNextNode (org.graalvm.compiler.nodes.FixedWithNextNode)41 AbstractBeginNode (org.graalvm.compiler.nodes.AbstractBeginNode)39 AbstractMergeNode (org.graalvm.compiler.nodes.AbstractMergeNode)39 ValueNode (org.graalvm.compiler.nodes.ValueNode)34 LoopBeginNode (org.graalvm.compiler.nodes.LoopBeginNode)33 EndNode (org.graalvm.compiler.nodes.EndNode)27 AbstractEndNode (org.graalvm.compiler.nodes.AbstractEndNode)25 StructuredGraph (org.graalvm.compiler.nodes.StructuredGraph)23 PhiNode (org.graalvm.compiler.nodes.PhiNode)22 ControlSplitNode (org.graalvm.compiler.nodes.ControlSplitNode)21 LoopExitNode (org.graalvm.compiler.nodes.LoopExitNode)21 LoopEndNode (org.graalvm.compiler.nodes.LoopEndNode)20 ControlSinkNode (org.graalvm.compiler.nodes.ControlSinkNode)17 MergeNode (org.graalvm.compiler.nodes.MergeNode)17 StartNode (org.graalvm.compiler.nodes.StartNode)17 ConstantNode (org.graalvm.compiler.nodes.ConstantNode)16 DeoptimizeNode (org.graalvm.compiler.nodes.DeoptimizeNode)13 ProxyNode (org.graalvm.compiler.nodes.ProxyNode)13