Examples with ArtifactPrefixConflictException com.google.devtools.build.lib.actions.ArtifactPrefixConflictException used on opensource projects

Example 1 with ArtifactPrefixConflictException

use of com.google.devtools.build.lib.actions.ArtifactPrefixConflictException in project bazel by bazelbuild.

the class PopulateTreeArtifactAction method execute.

@Override
public void execute(ActionExecutionContext actionExecutionContext) throws ActionExecutionException, InterruptedException {
    Executor executor = actionExecutionContext.getExecutor();
    Spawn spawn;
    // Create a spawn to unzip the archive file into the output TreeArtifact.
    try {
        spawn = createSpawn();
    } catch (IOException e) {
        throw new ActionExecutionException(e, this, false);
    } catch (IllegalManifestFileException e) {
        throw new ActionExecutionException(e, this, true);
    }
    // case we just return without generating anything under the output TreeArtifact.
    if (spawn.getOutputFiles().isEmpty()) {
        return;
    }
    // Check spawn output TreeFileArtifact conflicts.
    try {
        checkOutputConflicts(spawn.getOutputFiles());
    } catch (ArtifactPrefixConflictException e) {
        throw new ActionExecutionException(e, this, true);
    }
    // Create parent directories for the output TreeFileArtifacts.
    try {
        for (ActionInput fileEntry : spawn.getOutputFiles()) {
            FileSystemUtils.createDirectoryAndParents(((Artifact) fileEntry).getPath().getParentDirectory());
        }
    } catch (IOException e) {
        throw new ActionExecutionException(e, this, false);
    }
    // Execute the spawn.
    try {
        getContext(executor).exec(spawn, actionExecutionContext);
    } catch (ExecException e) {
        throw e.toActionExecutionException(getMnemonic() + " action failed for target: " + getOwner().getLabel(), executor.getVerboseFailures(), this);
    }
    // Populate the output TreeArtifact with the Spawn output TreeFileArtifacts.
    for (ActionInput fileEntry : spawn.getOutputFiles()) {
        actionExecutionContext.getMetadataHandler().addExpandedTreeOutput((TreeFileArtifact) fileEntry);
    }
}

Example 2 with ArtifactPrefixConflictException

use of com.google.devtools.build.lib.actions.ArtifactPrefixConflictException in project bazel by bazelbuild.

the class PopulateTreeArtifactAction method checkOutputConflicts.

private void checkOutputConflicts(Collection<? extends ActionInput> outputs) throws ArtifactPrefixConflictException {
    ImmutableMap.Builder<Artifact, ActionAnalysisMetadata> generatingActions = ImmutableMap.<Artifact, ActionAnalysisMetadata>builder();
    for (ActionInput output : outputs) {
        generatingActions.put((Artifact) output, this);
    }
    Map<ActionAnalysisMetadata, ArtifactPrefixConflictException> artifactPrefixConflictMap = Actions.findArtifactPrefixConflicts(generatingActions.build());
    if (!artifactPrefixConflictMap.isEmpty()) {
        throw artifactPrefixConflictMap.values().iterator().next();
    }
}

Example 3 with ArtifactPrefixConflictException

use of com.google.devtools.build.lib.actions.ArtifactPrefixConflictException in project bazel by bazelbuild.

the class SkyframeActionExecutor method findAndStoreArtifactConflicts.

/**
   * Find conflicts between generated artifacts. There are two ways to have conflicts. First, if
   * two (unshareable) actions generate the same output artifact, this will result in an {@link
   * ActionConflictException}. Second, if one action generates an artifact whose path is a prefix of
   * another artifact's path, those two artifacts cannot exist simultaneously in the output tree.
   * This causes an {@link ArtifactPrefixConflictException}. The relevant exceptions are stored in
   * the executor in {@code badActionMap}, and will be thrown immediately when that action is
   * executed. Those exceptions persist, so that even if the action is not executed this build, the
   * first time it is executed, the correct exception will be thrown.
   *
   * <p>This method must be called if a new action was added to the graph this build, so
   * whenever a new configured target was analyzed this build. It is somewhat expensive (~1s
   * range for a medium build as of 2014), so it should only be called when necessary.
   *
   * <p>Conflicts found may not be requested this build, and so we may overzealously throw an error.
   * For instance, if actions A and B generate the same artifact foo, and the user first requests
   * A' depending on A, and then in a subsequent build B' depending on B, we will fail the second
   * build, even though it would have succeeded if it had been the only build. However, since
   * Skyframe does not know the transitive dependencies of the request, we err on the conservative
   * side.
   *
   * <p>If the user first runs one action on the first build, and on the second build adds a
   * conflicting action, only the second action's error may be reported (because the first action
   * will be cached), whereas if both actions were requested for the first time, both errors would
   * be reported. However, the first time an action is added to the build, we are guaranteed to find
   * any conflicts it has, since this method will compare it against all other actions. So there is
   * no sequence of builds that can evade the error.
   */
void findAndStoreArtifactConflicts(Iterable<ActionLookupValue> actionLookupValues) throws InterruptedException {
    ConcurrentMap<ActionAnalysisMetadata, ConflictException> temporaryBadActionMap = new ConcurrentHashMap<>();
    Pair<ActionGraph, SortedMap<PathFragment, Artifact>> result;
    result = constructActionGraphAndPathMap(actionLookupValues, temporaryBadActionMap);
    ActionGraph actionGraph = result.first;
    SortedMap<PathFragment, Artifact> artifactPathMap = result.second;
    Map<ActionAnalysisMetadata, ArtifactPrefixConflictException> actionsWithArtifactPrefixConflict = Actions.findArtifactPrefixConflicts(actionGraph, artifactPathMap);
    for (Map.Entry<ActionAnalysisMetadata, ArtifactPrefixConflictException> actionExceptionPair : actionsWithArtifactPrefixConflict.entrySet()) {
        temporaryBadActionMap.put(actionExceptionPair.getKey(), new ConflictException(actionExceptionPair.getValue()));
    }
    this.badActionMap = ImmutableMap.copyOf(temporaryBadActionMap);
}

Example 4 with ArtifactPrefixConflictException

use of com.google.devtools.build.lib.actions.ArtifactPrefixConflictException in project bazel by bazelbuild.

the class SkyframeBuildView method configureTargets.

/**
   * Analyzes the specified targets using Skyframe as the driving framework.
   *
   * @return the configured targets that should be built along with a WalkableGraph of the analysis.
   */
public SkyframeAnalysisResult configureTargets(ExtendedEventHandler eventHandler, List<ConfiguredTargetKey> values, List<AspectValueKey> aspectKeys, EventBus eventBus, boolean keepGoing, int numThreads) throws InterruptedException, ViewCreationFailedException {
    enableAnalysis(true);
    EvaluationResult<ActionLookupValue> result;
    try {
        result = skyframeExecutor.configureTargets(eventHandler, values, aspectKeys, keepGoing, numThreads);
    } finally {
        enableAnalysis(false);
    }
    ImmutableMap<ActionAnalysisMetadata, ConflictException> badActions = skyframeExecutor.findArtifactConflicts();
    Collection<AspectValue> goodAspects = Lists.newArrayListWithCapacity(values.size());
    NestedSetBuilder<Package> packages = NestedSetBuilder.stableOrder();
    for (AspectValueKey aspectKey : aspectKeys) {
        AspectValue value = (AspectValue) result.get(aspectKey.getSkyKey());
        if (value == null) {
            // Skip aspects that couldn't be applied to targets.
            continue;
        }
        goodAspects.add(value);
        packages.addTransitive(value.getTransitivePackages());
    }
    // Filter out all CTs that have a bad action and convert to a list of configured targets. This
    // code ensures that the resulting list of configured targets has the same order as the incoming
    // list of values, i.e., that the order is deterministic.
    Collection<ConfiguredTarget> goodCts = Lists.newArrayListWithCapacity(values.size());
    for (ConfiguredTargetKey value : values) {
        ConfiguredTargetValue ctValue = (ConfiguredTargetValue) result.get(ConfiguredTargetValue.key(value));
        if (ctValue == null) {
            continue;
        }
        goodCts.add(ctValue.getConfiguredTarget());
        packages.addTransitive(ctValue.getTransitivePackages());
    }
    ImmutableMap<PackageIdentifier, Path> packageRoots = LoadingPhaseRunner.collectPackageRoots(packages.build().toCollection());
    if (!result.hasError() && badActions.isEmpty()) {
        return new SkyframeAnalysisResult(/*hasLoadingError=*/
        false, /*hasAnalysisError=*/
        false, ImmutableList.copyOf(goodCts), result.getWalkableGraph(), ImmutableList.copyOf(goodAspects), packageRoots);
    }
    // for keeping this code in parity with legacy we just report the first error for now.
    if (!keepGoing) {
        for (Map.Entry<ActionAnalysisMetadata, ConflictException> bad : badActions.entrySet()) {
            ConflictException ex = bad.getValue();
            try {
                ex.rethrowTyped();
            } catch (MutableActionGraph.ActionConflictException ace) {
                ace.reportTo(eventHandler);
                String errorMsg = "Analysis of target '" + bad.getKey().getOwner().getLabel() + "' failed; build aborted";
                throw new ViewCreationFailedException(errorMsg);
            } catch (ArtifactPrefixConflictException apce) {
                eventHandler.handle(Event.error(apce.getMessage()));
            }
            throw new ViewCreationFailedException(ex.getMessage());
        }
        Map.Entry<SkyKey, ErrorInfo> error = result.errorMap().entrySet().iterator().next();
        SkyKey topLevel = error.getKey();
        ErrorInfo errorInfo = error.getValue();
        assertSaneAnalysisError(errorInfo, topLevel);
        skyframeExecutor.getCyclesReporter().reportCycles(errorInfo.getCycleInfo(), topLevel, eventHandler);
        Throwable cause = errorInfo.getException();
        Preconditions.checkState(cause != null || !Iterables.isEmpty(errorInfo.getCycleInfo()), errorInfo);
        String errorMsg = null;
        if (topLevel.argument() instanceof ConfiguredTargetKey) {
            errorMsg = "Analysis of target '" + ConfiguredTargetValue.extractLabel(topLevel) + "' failed; build aborted";
        } else if (topLevel.argument() instanceof AspectValueKey) {
            AspectValueKey aspectKey = (AspectValueKey) topLevel.argument();
            errorMsg = "Analysis of aspect '" + aspectKey.getDescription() + "' failed; build aborted";
        } else {
            assert false;
        }
        if (cause instanceof ActionConflictException) {
            ((ActionConflictException) cause).reportTo(eventHandler);
        }
        throw new ViewCreationFailedException(errorMsg);
    }
    boolean hasLoadingError = false;
    // --keep_going : We notify the error and return a ConfiguredTargetValue
    for (Map.Entry<SkyKey, ErrorInfo> errorEntry : result.errorMap().entrySet()) {
        // TODO(ulfjack): this is quadratic - if there are a lot of CTs, this could be rather slow.
        if (!values.contains(errorEntry.getKey().argument())) {
            continue;
        }
        SkyKey errorKey = errorEntry.getKey();
        ConfiguredTargetKey label = (ConfiguredTargetKey) errorKey.argument();
        Label topLevelLabel = label.getLabel();
        ErrorInfo errorInfo = errorEntry.getValue();
        assertSaneAnalysisError(errorInfo, errorKey);
        skyframeExecutor.getCyclesReporter().reportCycles(errorInfo.getCycleInfo(), errorKey, eventHandler);
        Exception cause = errorInfo.getException();
        Label analysisRootCause = null;
        if (cause instanceof ConfiguredValueCreationException) {
            ConfiguredValueCreationException ctCause = (ConfiguredValueCreationException) cause;
            for (Label rootCause : ctCause.getRootCauses()) {
                hasLoadingError = true;
                eventBus.post(new LoadingFailureEvent(topLevelLabel, rootCause));
            }
            analysisRootCause = ctCause.getAnalysisRootCause();
        } else if (!Iterables.isEmpty(errorInfo.getCycleInfo())) {
            analysisRootCause = maybeGetConfiguredTargetCycleCulprit(topLevelLabel, errorInfo.getCycleInfo());
        } else if (cause instanceof ActionConflictException) {
            ((ActionConflictException) cause).reportTo(eventHandler);
        }
        eventHandler.handle(Event.warn("errors encountered while analyzing target '" + topLevelLabel + "': it will not be built"));
        if (analysisRootCause != null) {
            eventBus.post(new AnalysisFailureEvent(LabelAndConfiguration.of(topLevelLabel, label.getConfiguration()), analysisRootCause));
        }
    }
    Collection<Exception> reportedExceptions = Sets.newHashSet();
    for (Map.Entry<ActionAnalysisMetadata, ConflictException> bad : badActions.entrySet()) {
        ConflictException ex = bad.getValue();
        try {
            ex.rethrowTyped();
        } catch (MutableActionGraph.ActionConflictException ace) {
            ace.reportTo(eventHandler);
            eventHandler.handle(Event.warn("errors encountered while analyzing target '" + bad.getKey().getOwner().getLabel() + "': it will not be built"));
        } catch (ArtifactPrefixConflictException apce) {
            if (reportedExceptions.add(apce)) {
                eventHandler.handle(Event.error(apce.getMessage()));
            }
        }
    }
    if (!badActions.isEmpty()) {
        // In order to determine the set of configured targets transitively error free from action
        // conflict issues, we run a post-processing update() that uses the bad action map.
        EvaluationResult<PostConfiguredTargetValue> actionConflictResult = skyframeExecutor.postConfigureTargets(eventHandler, values, keepGoing, badActions);
        goodCts = Lists.newArrayListWithCapacity(values.size());
        for (ConfiguredTargetKey value : values) {
            PostConfiguredTargetValue postCt = actionConflictResult.get(PostConfiguredTargetValue.key(value));
            if (postCt != null) {
                goodCts.add(postCt.getCt());
            }
        }
    }
    return new SkyframeAnalysisResult(hasLoadingError, result.hasError() || !badActions.isEmpty(), ImmutableList.copyOf(goodCts), result.getWalkableGraph(), ImmutableList.copyOf(goodAspects), packageRoots);
}

Example 5 with ArtifactPrefixConflictException

use of com.google.devtools.build.lib.actions.ArtifactPrefixConflictException in project bazel by bazelbuild.

the class ActionTemplateExpansionFunctionTest method testThrowsOnArtifactPrefixConflict.

@Test
public void testThrowsOnArtifactPrefixConflict() throws Exception {
    Artifact inputTreeArtifact = createAndPopulateTreeArtifact("inputTreeArtifact", "child0", "child1", "child2");
    Artifact outputTreeArtifact = createTreeArtifact("outputTreeArtifact");
    OutputPathMapper mapper = new OutputPathMapper() {

        private int i = 0;

        @Override
        public PathFragment parentRelativeOutputPath(TreeFileArtifact inputTreeFileArtifact) {
            PathFragment path;
            switch(i) {
                case 0:
                    path = new PathFragment("path_prefix");
                    break;
                case 1:
                    path = new PathFragment("path_prefix/conflict");
                    break;
                default:
                    path = inputTreeFileArtifact.getParentRelativePath();
            }
            ++i;
            return path;
        }
    };
    SpawnActionTemplate spawnActionTemplate = new SpawnActionTemplate.Builder(inputTreeArtifact, outputTreeArtifact).setExecutable(new PathFragment("/bin/cp")).setCommandLineTemplate(CustomCommandLine.builder().build()).setOutputPathMapper(mapper).build(ActionsTestUtil.NULL_ACTION_OWNER);
    try {
        evaluate(spawnActionTemplate);
        fail("Expected ArtifactPrefixConflictException");
    } catch (ArtifactPrefixConflictException e) {
    // Expected ArtifactPrefixConflictException
    }
}