Examples with ActionAnalysisMetadata com.google.devtools.build.lib.actions.ActionAnalysisMetadata used on opensource projects

Example 11 with ActionAnalysisMetadata

use of com.google.devtools.build.lib.actions.ActionAnalysisMetadata in project bazel by bazelbuild.

the class ArtifactFunctionTest method testMiddlemanArtifact.

@Test
public void testMiddlemanArtifact() throws Throwable {
    Artifact output = createDerivedArtifact("output");
    Artifact input1 = createSourceArtifact("input1");
    Artifact input2 = createDerivedArtifact("input2");
    Action action = new DummyAction(ImmutableList.of(input1, input2), output, MiddlemanType.AGGREGATING_MIDDLEMAN);
    // Overwrite default generating action with this one.
    for (Iterator<ActionAnalysisMetadata> it = actions.iterator(); it.hasNext(); ) {
        if (it.next().getOutputs().contains(output)) {
            it.remove();
            break;
        }
    }
    actions.add(action);
    file(input2.getPath(), "contents");
    file(input1.getPath(), "source contents");
    evaluate(Iterables.toArray(ArtifactSkyKey.mandatoryKeys(ImmutableSet.of(input2, input1, input2)), SkyKey.class));
    SkyValue value = evaluateArtifactValue(output);
    assertThat(((AggregatingArtifactValue) value).getInputs()).containsExactly(Pair.of(input1, create(input1)), Pair.of(input2, create(input2)));
}

Example 12 with ActionAnalysisMetadata

use of com.google.devtools.build.lib.actions.ActionAnalysisMetadata in project bazel by bazelbuild.

the class PopulateTreeArtifactAction method checkOutputConflicts.

private void checkOutputConflicts(Collection<? extends ActionInput> outputs) throws ArtifactPrefixConflictException {
    ImmutableMap.Builder<Artifact, ActionAnalysisMetadata> generatingActions = ImmutableMap.<Artifact, ActionAnalysisMetadata>builder();
    for (ActionInput output : outputs) {
        generatingActions.put((Artifact) output, this);
    }
    Map<ActionAnalysisMetadata, ArtifactPrefixConflictException> artifactPrefixConflictMap = Actions.findArtifactPrefixConflicts(generatingActions.build());
    if (!artifactPrefixConflictMap.isEmpty()) {
        throw artifactPrefixConflictMap.values().iterator().next();
    }
}

Example 13 with ActionAnalysisMetadata

use of com.google.devtools.build.lib.actions.ActionAnalysisMetadata in project bazel by bazelbuild.

the class SkyframeActionExecutor method findAndStoreArtifactConflicts.

/**
   * Find conflicts between generated artifacts. There are two ways to have conflicts. First, if
   * two (unshareable) actions generate the same output artifact, this will result in an {@link
   * ActionConflictException}. Second, if one action generates an artifact whose path is a prefix of
   * another artifact's path, those two artifacts cannot exist simultaneously in the output tree.
   * This causes an {@link ArtifactPrefixConflictException}. The relevant exceptions are stored in
   * the executor in {@code badActionMap}, and will be thrown immediately when that action is
   * executed. Those exceptions persist, so that even if the action is not executed this build, the
   * first time it is executed, the correct exception will be thrown.
   *
   * <p>This method must be called if a new action was added to the graph this build, so
   * whenever a new configured target was analyzed this build. It is somewhat expensive (~1s
   * range for a medium build as of 2014), so it should only be called when necessary.
   *
   * <p>Conflicts found may not be requested this build, and so we may overzealously throw an error.
   * For instance, if actions A and B generate the same artifact foo, and the user first requests
   * A' depending on A, and then in a subsequent build B' depending on B, we will fail the second
   * build, even though it would have succeeded if it had been the only build. However, since
   * Skyframe does not know the transitive dependencies of the request, we err on the conservative
   * side.
   *
   * <p>If the user first runs one action on the first build, and on the second build adds a
   * conflicting action, only the second action's error may be reported (because the first action
   * will be cached), whereas if both actions were requested for the first time, both errors would
   * be reported. However, the first time an action is added to the build, we are guaranteed to find
   * any conflicts it has, since this method will compare it against all other actions. So there is
   * no sequence of builds that can evade the error.
   */
void findAndStoreArtifactConflicts(Iterable<ActionLookupValue> actionLookupValues) throws InterruptedException {
    ConcurrentMap<ActionAnalysisMetadata, ConflictException> temporaryBadActionMap = new ConcurrentHashMap<>();
    Pair<ActionGraph, SortedMap<PathFragment, Artifact>> result;
    result = constructActionGraphAndPathMap(actionLookupValues, temporaryBadActionMap);
    ActionGraph actionGraph = result.first;
    SortedMap<PathFragment, Artifact> artifactPathMap = result.second;
    Map<ActionAnalysisMetadata, ArtifactPrefixConflictException> actionsWithArtifactPrefixConflict = Actions.findArtifactPrefixConflicts(actionGraph, artifactPathMap);
    for (Map.Entry<ActionAnalysisMetadata, ArtifactPrefixConflictException> actionExceptionPair : actionsWithArtifactPrefixConflict.entrySet()) {
        temporaryBadActionMap.put(actionExceptionPair.getKey(), new ConflictException(actionExceptionPair.getValue()));
    }
    this.badActionMap = ImmutableMap.copyOf(temporaryBadActionMap);
}

Example 14 with ActionAnalysisMetadata

use of com.google.devtools.build.lib.actions.ActionAnalysisMetadata in project bazel by bazelbuild.

the class SkyframeActionExecutor method actionRegistration.

private static Runnable actionRegistration(final List<ActionLookupValue> values, final MutableActionGraph actionGraph, final ConcurrentMap<PathFragment, Artifact> artifactPathMap, final ConcurrentMap<ActionAnalysisMetadata, ConflictException> badActionMap) {
    return new Runnable() {

        @Override
        public void run() {
            for (ActionLookupValue value : values) {
                Set<ActionAnalysisMetadata> registeredActions = new HashSet<>();
                for (Map.Entry<Artifact, ActionAnalysisMetadata> entry : value.getMapForConsistencyCheck().entrySet()) {
                    ActionAnalysisMetadata action = entry.getValue();
                    // once.
                    if (registeredActions.add(action)) {
                        try {
                            actionGraph.registerAction(action);
                        } catch (ActionConflictException e) {
                            Exception oldException = badActionMap.put(action, new ConflictException(e));
                            Preconditions.checkState(oldException == null, "%s | %s | %s", action, e, oldException);
                            // error.
                            continue;
                        }
                    }
                    artifactPathMap.put(entry.getKey().getExecPath(), entry.getKey());
                }
            }
        }
    };
}

Example 15 with ActionAnalysisMetadata

use of com.google.devtools.build.lib.actions.ActionAnalysisMetadata in project bazel by bazelbuild.

the class SkyframeBuildView method configureTargets.

/**
   * Analyzes the specified targets using Skyframe as the driving framework.
   *
   * @return the configured targets that should be built along with a WalkableGraph of the analysis.
   */
public SkyframeAnalysisResult configureTargets(ExtendedEventHandler eventHandler, List<ConfiguredTargetKey> values, List<AspectValueKey> aspectKeys, EventBus eventBus, boolean keepGoing, int numThreads) throws InterruptedException, ViewCreationFailedException {
    enableAnalysis(true);
    EvaluationResult<ActionLookupValue> result;
    try {
        result = skyframeExecutor.configureTargets(eventHandler, values, aspectKeys, keepGoing, numThreads);
    } finally {
        enableAnalysis(false);
    }
    ImmutableMap<ActionAnalysisMetadata, ConflictException> badActions = skyframeExecutor.findArtifactConflicts();
    Collection<AspectValue> goodAspects = Lists.newArrayListWithCapacity(values.size());
    NestedSetBuilder<Package> packages = NestedSetBuilder.stableOrder();
    for (AspectValueKey aspectKey : aspectKeys) {
        AspectValue value = (AspectValue) result.get(aspectKey.getSkyKey());
        if (value == null) {
            // Skip aspects that couldn't be applied to targets.
            continue;
        }
        goodAspects.add(value);
        packages.addTransitive(value.getTransitivePackages());
    }
    // Filter out all CTs that have a bad action and convert to a list of configured targets. This
    // code ensures that the resulting list of configured targets has the same order as the incoming
    // list of values, i.e., that the order is deterministic.
    Collection<ConfiguredTarget> goodCts = Lists.newArrayListWithCapacity(values.size());
    for (ConfiguredTargetKey value : values) {
        ConfiguredTargetValue ctValue = (ConfiguredTargetValue) result.get(ConfiguredTargetValue.key(value));
        if (ctValue == null) {
            continue;
        }
        goodCts.add(ctValue.getConfiguredTarget());
        packages.addTransitive(ctValue.getTransitivePackages());
    }
    ImmutableMap<PackageIdentifier, Path> packageRoots = LoadingPhaseRunner.collectPackageRoots(packages.build().toCollection());
    if (!result.hasError() && badActions.isEmpty()) {
        return new SkyframeAnalysisResult(/*hasLoadingError=*/
        false, /*hasAnalysisError=*/
        false, ImmutableList.copyOf(goodCts), result.getWalkableGraph(), ImmutableList.copyOf(goodAspects), packageRoots);
    }
    // for keeping this code in parity with legacy we just report the first error for now.
    if (!keepGoing) {
        for (Map.Entry<ActionAnalysisMetadata, ConflictException> bad : badActions.entrySet()) {
            ConflictException ex = bad.getValue();
            try {
                ex.rethrowTyped();
            } catch (MutableActionGraph.ActionConflictException ace) {
                ace.reportTo(eventHandler);
                String errorMsg = "Analysis of target '" + bad.getKey().getOwner().getLabel() + "' failed; build aborted";
                throw new ViewCreationFailedException(errorMsg);
            } catch (ArtifactPrefixConflictException apce) {
                eventHandler.handle(Event.error(apce.getMessage()));
            }
            throw new ViewCreationFailedException(ex.getMessage());
        }
        Map.Entry<SkyKey, ErrorInfo> error = result.errorMap().entrySet().iterator().next();
        SkyKey topLevel = error.getKey();
        ErrorInfo errorInfo = error.getValue();
        assertSaneAnalysisError(errorInfo, topLevel);
        skyframeExecutor.getCyclesReporter().reportCycles(errorInfo.getCycleInfo(), topLevel, eventHandler);
        Throwable cause = errorInfo.getException();
        Preconditions.checkState(cause != null || !Iterables.isEmpty(errorInfo.getCycleInfo()), errorInfo);
        String errorMsg = null;
        if (topLevel.argument() instanceof ConfiguredTargetKey) {
            errorMsg = "Analysis of target '" + ConfiguredTargetValue.extractLabel(topLevel) + "' failed; build aborted";
        } else if (topLevel.argument() instanceof AspectValueKey) {
            AspectValueKey aspectKey = (AspectValueKey) topLevel.argument();
            errorMsg = "Analysis of aspect '" + aspectKey.getDescription() + "' failed; build aborted";
        } else {
            assert false;
        }
        if (cause instanceof ActionConflictException) {
            ((ActionConflictException) cause).reportTo(eventHandler);
        }
        throw new ViewCreationFailedException(errorMsg);
    }
    boolean hasLoadingError = false;
    // --keep_going : We notify the error and return a ConfiguredTargetValue
    for (Map.Entry<SkyKey, ErrorInfo> errorEntry : result.errorMap().entrySet()) {
        // TODO(ulfjack): this is quadratic - if there are a lot of CTs, this could be rather slow.
        if (!values.contains(errorEntry.getKey().argument())) {
            continue;
        }
        SkyKey errorKey = errorEntry.getKey();
        ConfiguredTargetKey label = (ConfiguredTargetKey) errorKey.argument();
        Label topLevelLabel = label.getLabel();
        ErrorInfo errorInfo = errorEntry.getValue();
        assertSaneAnalysisError(errorInfo, errorKey);
        skyframeExecutor.getCyclesReporter().reportCycles(errorInfo.getCycleInfo(), errorKey, eventHandler);
        Exception cause = errorInfo.getException();
        Label analysisRootCause = null;
        if (cause instanceof ConfiguredValueCreationException) {
            ConfiguredValueCreationException ctCause = (ConfiguredValueCreationException) cause;
            for (Label rootCause : ctCause.getRootCauses()) {
                hasLoadingError = true;
                eventBus.post(new LoadingFailureEvent(topLevelLabel, rootCause));
            }
            analysisRootCause = ctCause.getAnalysisRootCause();
        } else if (!Iterables.isEmpty(errorInfo.getCycleInfo())) {
            analysisRootCause = maybeGetConfiguredTargetCycleCulprit(topLevelLabel, errorInfo.getCycleInfo());
        } else if (cause instanceof ActionConflictException) {
            ((ActionConflictException) cause).reportTo(eventHandler);
        }
        eventHandler.handle(Event.warn("errors encountered while analyzing target '" + topLevelLabel + "': it will not be built"));
        if (analysisRootCause != null) {
            eventBus.post(new AnalysisFailureEvent(LabelAndConfiguration.of(topLevelLabel, label.getConfiguration()), analysisRootCause));
        }
    }
    Collection<Exception> reportedExceptions = Sets.newHashSet();
    for (Map.Entry<ActionAnalysisMetadata, ConflictException> bad : badActions.entrySet()) {
        ConflictException ex = bad.getValue();
        try {
            ex.rethrowTyped();
        } catch (MutableActionGraph.ActionConflictException ace) {
            ace.reportTo(eventHandler);
            eventHandler.handle(Event.warn("errors encountered while analyzing target '" + bad.getKey().getOwner().getLabel() + "': it will not be built"));
        } catch (ArtifactPrefixConflictException apce) {
            if (reportedExceptions.add(apce)) {
                eventHandler.handle(Event.error(apce.getMessage()));
            }
        }
    }
    if (!badActions.isEmpty()) {
        // In order to determine the set of configured targets transitively error free from action
        // conflict issues, we run a post-processing update() that uses the bad action map.
        EvaluationResult<PostConfiguredTargetValue> actionConflictResult = skyframeExecutor.postConfigureTargets(eventHandler, values, keepGoing, badActions);
        goodCts = Lists.newArrayListWithCapacity(values.size());
        for (ConfiguredTargetKey value : values) {
            PostConfiguredTargetValue postCt = actionConflictResult.get(PostConfiguredTargetValue.key(value));
            if (postCt != null) {
                goodCts.add(postCt.getCt());
            }
        }
    }
    return new SkyframeAnalysisResult(hasLoadingError, result.hasError() || !badActions.isEmpty(), ImmutableList.copyOf(goodCts), result.getWalkableGraph(), ImmutableList.copyOf(goodAspects), packageRoots);
}