Search in sources :

Example 1 with TargetGraph

use of com.facebook.buck.rules.TargetGraph in project buck by facebook.

the class TargetsCommand method computeShowTargetHash.

private void computeShowTargetHash(CommandRunnerParams params, ListeningExecutorService executor, TargetGraphAndTargetNodes targetGraphAndTargetNodes, Map<BuildTarget, ShowOptions.Builder> showRulesResult) throws IOException, InterruptedException, BuildFileParseException, BuildTargetException, CycleException {
    LOG.debug("Getting target hash for %s", targetGraphAndTargetNodes.getTargetNodes());
    TargetGraphAndTargetNodes targetGraphAndNodesWithTests = computeTargetsAndGraphToShowTargetHash(params, executor, targetGraphAndTargetNodes);
    TargetGraph targetGraphWithTests = targetGraphAndNodesWithTests.getTargetGraph();
    FileHashLoader fileHashLoader = createOrGetFileHashLoader(params);
    // Hash each target's rule description and contents of any files.
    ImmutableMap<BuildTarget, HashCode> buildTargetHashes = new TargetGraphHashing(params.getBuckEventBus(), targetGraphWithTests, fileHashLoader, targetGraphAndNodesWithTests.getTargetNodes()).setNumThreads(params.getBuckConfig().getNumThreads()).hashTargetGraph();
    ImmutableMap<BuildTarget, HashCode> finalHashes = rehashWithTestsIfNeeded(targetGraphWithTests, targetGraphAndTargetNodes.getTargetNodes(), buildTargetHashes);
    for (TargetNode<?, ?> targetNode : targetGraphAndTargetNodes.getTargetNodes()) {
        processTargetHash(targetNode.getBuildTarget(), showRulesResult, finalHashes);
    }
}
Also used : TargetGraphAndTargetNodes(com.facebook.buck.rules.TargetGraphAndTargetNodes) FileHashLoader(com.facebook.buck.hashing.FileHashLoader) HashCode(com.google.common.hash.HashCode) BuildTarget(com.facebook.buck.model.BuildTarget) TargetGraphHashing(com.facebook.buck.rules.TargetGraphHashing) TargetGraph(com.facebook.buck.rules.TargetGraph)

Example 2 with TargetGraph

use of com.facebook.buck.rules.TargetGraph in project buck by facebook.

the class TargetsCommand method computeTargetsAndGraphToShowTargetHash.

private TargetGraphAndTargetNodes computeTargetsAndGraphToShowTargetHash(CommandRunnerParams params, ListeningExecutorService executor, TargetGraphAndTargetNodes targetGraphAndTargetNodes) throws InterruptedException, BuildFileParseException, BuildTargetException, IOException {
    if (isDetectTestChanges()) {
        ImmutableSet<BuildTarget> explicitTestTargets = TargetGraphAndTargets.getExplicitTestTargets(targetGraphAndTargetNodes.getTargetGraph().getSubgraph(targetGraphAndTargetNodes.getTargetNodes()).getNodes());
        LOG.debug("Got explicit test targets: %s", explicitTestTargets);
        Iterable<BuildTarget> matchingBuildTargetsWithTests = mergeBuildTargets(targetGraphAndTargetNodes.getTargetNodes(), explicitTestTargets);
        // Parse the BUCK files for the tests of the targets passed in from the command line.
        TargetGraph targetGraphWithTests = params.getParser().buildTargetGraph(params.getBuckEventBus(), params.getCell(), getEnableParserProfiling(), executor, matchingBuildTargetsWithTests);
        return TargetGraphAndTargetNodes.builder().setTargetGraph(targetGraphWithTests).setTargetNodes(targetGraphWithTests.getAll(matchingBuildTargetsWithTests)).build();
    } else {
        return targetGraphAndTargetNodes;
    }
}
Also used : BuildTarget(com.facebook.buck.model.BuildTarget) TargetGraph(com.facebook.buck.rules.TargetGraph)

Example 3 with TargetGraph

use of com.facebook.buck.rules.TargetGraph in project buck by facebook.

the class QueryMacroExpander method extractTargets.

private Stream<BuildTarget> extractTargets(BuildTarget target, CellPathResolver cellNames, Optional<BuildRuleResolver> resolver, T input) {
    String queryExpression = CharMatcher.anyOf("\"'").trimFrom(input.getQuery().getQuery());
    final GraphEnhancementQueryEnvironment env = new GraphEnhancementQueryEnvironment(resolver, targetGraph, cellNames, BuildTargetPatternParser.forBaseName(target.getBaseName()), ImmutableSet.of());
    try {
        QueryExpression parsedExp = QueryExpression.parse(queryExpression, env);
        HashSet<String> targetLiterals = new HashSet<>();
        parsedExp.collectTargetPatterns(targetLiterals);
        return targetLiterals.stream().flatMap(pattern -> {
            try {
                return env.getTargetsMatchingPattern(pattern, executorService).stream();
            } catch (Exception e) {
                throw new HumanReadableException(e, "Error parsing target expression %s for target %s", pattern, target);
            }
        }).map(queryTarget -> {
            Preconditions.checkState(queryTarget instanceof QueryBuildTarget);
            return ((QueryBuildTarget) queryTarget).getBuildTarget();
        });
    } catch (QueryException e) {
        throw new HumanReadableException("Error executing query in macro for target %s", target, e);
    }
}
Also used : MoreExecutors(com.google.common.util.concurrent.MoreExecutors) ImmutableSet(com.google.common.collect.ImmutableSet) CellPathResolver(com.facebook.buck.rules.CellPathResolver) QueryException(com.facebook.buck.query.QueryException) TargetGraph(com.facebook.buck.rules.TargetGraph) CharMatcher(com.google.common.base.CharMatcher) Set(java.util.Set) Query(com.facebook.buck.rules.query.Query) MacroException(com.facebook.buck.model.MacroException) HumanReadableException(com.facebook.buck.util.HumanReadableException) BuildTarget(com.facebook.buck.model.BuildTarget) Collectors(java.util.stream.Collectors) HashSet(java.util.HashSet) Stream(java.util.stream.Stream) QueryExpression(com.facebook.buck.query.QueryExpression) ImmutableList(com.google.common.collect.ImmutableList) BuildTargetPatternParser(com.facebook.buck.parser.BuildTargetPatternParser) Optional(java.util.Optional) Preconditions(com.google.common.base.Preconditions) BuildRuleResolver(com.facebook.buck.rules.BuildRuleResolver) QueryBuildTarget(com.facebook.buck.query.QueryBuildTarget) QueryTarget(com.facebook.buck.query.QueryTarget) GraphEnhancementQueryEnvironment(com.facebook.buck.rules.query.GraphEnhancementQueryEnvironment) ListeningExecutorService(com.google.common.util.concurrent.ListeningExecutorService) GraphEnhancementQueryEnvironment(com.facebook.buck.rules.query.GraphEnhancementQueryEnvironment) QueryException(com.facebook.buck.query.QueryException) HumanReadableException(com.facebook.buck.util.HumanReadableException) QueryExpression(com.facebook.buck.query.QueryExpression) QueryException(com.facebook.buck.query.QueryException) MacroException(com.facebook.buck.model.MacroException) HumanReadableException(com.facebook.buck.util.HumanReadableException) QueryBuildTarget(com.facebook.buck.query.QueryBuildTarget) HashSet(java.util.HashSet)

Example 4 with TargetGraph

use of com.facebook.buck.rules.TargetGraph in project buck by facebook.

the class AppleDescriptions method createBuildRulesForCoreDataDependencies.

public static Optional<CoreDataModel> createBuildRulesForCoreDataDependencies(TargetGraph targetGraph, BuildRuleParams params, String moduleName, AppleCxxPlatform appleCxxPlatform) {
    TargetNode<?, ?> targetNode = targetGraph.get(params.getBuildTarget());
    ImmutableSet<AppleWrapperResourceArg> coreDataModelArgs = AppleBuildRules.collectTransitiveBuildRules(targetGraph, Optional.empty(), AppleBuildRules.CORE_DATA_MODEL_DESCRIPTION_CLASSES, ImmutableList.of(targetNode));
    BuildRuleParams coreDataModelParams = params.withAppendedFlavor(CoreDataModel.FLAVOR).copyReplacingDeclaredAndExtraDeps(Suppliers.ofInstance(ImmutableSortedSet.of()), Suppliers.ofInstance(ImmutableSortedSet.of()));
    if (coreDataModelArgs.isEmpty()) {
        return Optional.empty();
    } else {
        return Optional.of(new CoreDataModel(coreDataModelParams, appleCxxPlatform, moduleName, coreDataModelArgs.stream().map(input -> new PathSourcePath(params.getProjectFilesystem(), input.path)).collect(MoreCollectors.toImmutableSet())));
    }
}
Also used : OptionalCompat(com.facebook.buck.util.OptionalCompat) ProvidesLinkedBinaryDeps(com.facebook.buck.cxx.ProvidesLinkedBinaryDeps) SourcePathRuleFinder(com.facebook.buck.rules.SourcePathRuleFinder) AbstractGenruleDescription(com.facebook.buck.shell.AbstractGenruleDescription) RichStream(com.facebook.buck.util.RichStream) InternalFlavor(com.facebook.buck.model.InternalFlavor) FlavorDomain(com.facebook.buck.model.FlavorDomain) CxxCompilationDatabase(com.facebook.buck.cxx.CxxCompilationDatabase) FluentIterable(com.google.common.collect.FluentIterable) SourcePathResolver(com.facebook.buck.rules.SourcePathResolver) SourceList(com.facebook.buck.rules.coercer.SourceList) StripStyle(com.facebook.buck.cxx.StripStyle) BuildRuleParams(com.facebook.buck.rules.BuildRuleParams) Path(java.nio.file.Path) CxxDescriptionEnhancer(com.facebook.buck.cxx.CxxDescriptionEnhancer) BuildRules(com.facebook.buck.rules.BuildRules) FrameworkDependencies(com.facebook.buck.cxx.FrameworkDependencies) Function(com.google.common.base.Function) ImmutableSet(com.google.common.collect.ImmutableSet) ImmutableMap(com.google.common.collect.ImmutableMap) TargetGraph(com.facebook.buck.rules.TargetGraph) Set(java.util.Set) BuildTarget(com.facebook.buck.model.BuildTarget) Sets(com.google.common.collect.Sets) LinkerMapMode(com.facebook.buck.cxx.LinkerMapMode) SourceWithFlags(com.facebook.buck.rules.SourceWithFlags) Predicate(com.google.common.base.Predicate) PathSourcePath(com.facebook.buck.rules.PathSourcePath) Optional(java.util.Optional) BuildRuleResolver(com.facebook.buck.rules.BuildRuleResolver) CxxBinaryDescription(com.facebook.buck.cxx.CxxBinaryDescription) Joiner(com.google.common.base.Joiner) CxxStrip(com.facebook.buck.cxx.CxxStrip) CxxLibraryDescription(com.facebook.buck.cxx.CxxLibraryDescription) SourcePath(com.facebook.buck.rules.SourcePath) Either(com.facebook.buck.model.Either) BuildRule(com.facebook.buck.rules.BuildRule) HashSet(java.util.HashSet) Tool(com.facebook.buck.rules.Tool) ImmutableList(com.google.common.collect.ImmutableList) NoSuchBuildTargetException(com.facebook.buck.parser.NoSuchBuildTargetException) BuildTargetSourcePath(com.facebook.buck.rules.BuildTargetSourcePath) Predicates(com.google.common.base.Predicates) Suppliers(com.google.common.base.Suppliers) ImmutableSortedMap(com.google.common.collect.ImmutableSortedMap) MoreCollectors(com.facebook.buck.util.MoreCollectors) ImmutableSortedSet(com.google.common.collect.ImmutableSortedSet) CxxConstructorArg(com.facebook.buck.cxx.CxxConstructorArg) TargetNode(com.facebook.buck.rules.TargetNode) CxxPlatform(com.facebook.buck.cxx.CxxPlatform) HumanReadableException(com.facebook.buck.util.HumanReadableException) MorePaths(com.facebook.buck.io.MorePaths) Ordering(com.google.common.collect.Ordering) BuildRuleWithBinary(com.facebook.buck.cxx.BuildRuleWithBinary) Paths(java.nio.file.Paths) SWIFT_EXTENSION(com.facebook.buck.swift.SwiftDescriptions.SWIFT_EXTENSION) Preconditions(com.google.common.base.Preconditions) Flavor(com.facebook.buck.model.Flavor) VisibleForTesting(com.google.common.annotations.VisibleForTesting) BuildRuleParams(com.facebook.buck.rules.BuildRuleParams) PathSourcePath(com.facebook.buck.rules.PathSourcePath)

Example 5 with TargetGraph

use of com.facebook.buck.rules.TargetGraph in project buck by facebook.

the class JavaBuildGraphProcessor method run.

/**
   * Creates the appropriate target graph and other resources needed for the {@link Processor} and
   * runs it. This method will take responsibility for cleaning up the executor service after it
   * runs.
   */
static void run(final CommandRunnerParams params, final AbstractCommand command, final Processor processor) throws ExitCodeException, InterruptedException, IOException {
    final ConcurrencyLimit concurrencyLimit = command.getConcurrencyLimit(params.getBuckConfig());
    try (CommandThreadManager pool = new CommandThreadManager(command.getClass().getName(), concurrencyLimit)) {
        Cell cell = params.getCell();
        WeightedListeningExecutorService executorService = pool.getExecutor();
        // Ideally, we should be able to construct the TargetGraph quickly assuming most of it is
        // already in memory courtesy of buckd. Though we could make a performance optimization where
        // we pass an option to buck.py that tells it to ignore reading the BUCK.autodeps files when
        // parsing the BUCK files because we never need to consider the existing auto-generated deps
        // when creating the new auto-generated deps. If we did so, we would have to make sure to keep
        // the nodes for that version of the graph separate from the ones that are actually used for
        // building.
        TargetGraph graph;
        try {
            graph = params.getParser().buildTargetGraphForTargetNodeSpecs(params.getBuckEventBus(), cell, command.getEnableParserProfiling(), executorService, ImmutableList.of(TargetNodePredicateSpec.of(x -> true, BuildFileSpec.fromRecursivePath(Paths.get(""), cell.getRoot()))), /* ignoreBuckAutodepsFiles */
            true).getTargetGraph();
        } catch (BuildTargetException | BuildFileParseException e) {
            params.getBuckEventBus().post(ConsoleEvent.severe(MoreExceptions.getHumanReadableOrLocalizedMessage(e)));
            throw new ExitCodeException(1);
        }
        BuildRuleResolver buildRuleResolver = new BuildRuleResolver(graph, new DefaultTargetNodeToBuildRuleTransformer());
        CachingBuildEngineBuckConfig cachingBuildEngineBuckConfig = params.getBuckConfig().getView(CachingBuildEngineBuckConfig.class);
        LocalCachingBuildEngineDelegate cachingBuildEngineDelegate = new LocalCachingBuildEngineDelegate(params.getFileHashCache());
        BuildEngine buildEngine = new CachingBuildEngine(cachingBuildEngineDelegate, executorService, executorService, new DefaultStepRunner(), CachingBuildEngine.BuildMode.SHALLOW, cachingBuildEngineBuckConfig.getBuildDepFiles(), cachingBuildEngineBuckConfig.getBuildMaxDepFileCacheEntries(), cachingBuildEngineBuckConfig.getBuildArtifactCacheSizeLimit(), params.getObjectMapper(), buildRuleResolver, cachingBuildEngineBuckConfig.getResourceAwareSchedulingInfo(), new RuleKeyFactoryManager(params.getBuckConfig().getKeySeed(), fs -> cachingBuildEngineDelegate.getFileHashCache(), buildRuleResolver, cachingBuildEngineBuckConfig.getBuildInputRuleKeyFileSizeLimit(), new DefaultRuleKeyCache<>()));
        // Create a BuildEngine because we store symbol information as build artifacts.
        BuckEventBus eventBus = params.getBuckEventBus();
        ExecutionContext executionContext = ExecutionContext.builder().setConsole(params.getConsole()).setConcurrencyLimit(concurrencyLimit).setBuckEventBus(eventBus).setEnvironment(/* environment */
        ImmutableMap.of()).setExecutors(ImmutableMap.<ExecutorPool, ListeningExecutorService>of(ExecutorPool.CPU, executorService)).setJavaPackageFinder(params.getJavaPackageFinder()).setObjectMapper(params.getObjectMapper()).setPlatform(params.getPlatform()).setCellPathResolver(params.getCell().getCellPathResolver()).build();
        SourcePathResolver pathResolver = new SourcePathResolver(new SourcePathRuleFinder(buildRuleResolver));
        BuildEngineBuildContext buildContext = BuildEngineBuildContext.builder().setBuildContext(BuildContext.builder().setActionGraph(new ActionGraph(ImmutableList.of())).setSourcePathResolver(pathResolver).setJavaPackageFinder(executionContext.getJavaPackageFinder()).setEventBus(eventBus).build()).setClock(params.getClock()).setArtifactCache(params.getArtifactCacheFactory().newInstance()).setBuildId(eventBus.getBuildId()).setObjectMapper(params.getObjectMapper()).setEnvironment(executionContext.getEnvironment()).setKeepGoing(false).build();
        // Traverse the TargetGraph to find all of the auto-generated dependencies.
        JavaDepsFinder javaDepsFinder = JavaDepsFinder.createJavaDepsFinder(params.getBuckConfig(), params.getCell().getCellPathResolver(), params.getObjectMapper(), buildContext, executionContext, buildEngine);
        processor.process(graph, javaDepsFinder, executorService);
    }
}
Also used : BuckEventBus(com.facebook.buck.event.BuckEventBus) ActionGraph(com.facebook.buck.rules.ActionGraph) TargetNodePredicateSpec(com.facebook.buck.parser.TargetNodePredicateSpec) SourcePathRuleFinder(com.facebook.buck.rules.SourcePathRuleFinder) MoreExceptions(com.facebook.buck.util.MoreExceptions) ConsoleEvent(com.facebook.buck.event.ConsoleEvent) ExecutionContext(com.facebook.buck.step.ExecutionContext) ImmutableList(com.google.common.collect.ImmutableList) LocalCachingBuildEngineDelegate(com.facebook.buck.rules.LocalCachingBuildEngineDelegate) SourcePathResolver(com.facebook.buck.rules.SourcePathResolver) DefaultStepRunner(com.facebook.buck.step.DefaultStepRunner) DefaultTargetNodeToBuildRuleTransformer(com.facebook.buck.rules.DefaultTargetNodeToBuildRuleTransformer) ConcurrencyLimit(com.facebook.buck.util.concurrent.ConcurrencyLimit) CachingBuildEngineBuckConfig(com.facebook.buck.rules.CachingBuildEngineBuckConfig) BuildFileParseException(com.facebook.buck.json.BuildFileParseException) Cell(com.facebook.buck.rules.Cell) WeightedListeningExecutorService(com.facebook.buck.util.concurrent.WeightedListeningExecutorService) BuildFileSpec(com.facebook.buck.parser.BuildFileSpec) DefaultRuleKeyCache(com.facebook.buck.rules.keys.DefaultRuleKeyCache) ImmutableMap(com.google.common.collect.ImmutableMap) TargetGraph(com.facebook.buck.rules.TargetGraph) BuildTargetException(com.facebook.buck.model.BuildTargetException) IOException(java.io.IOException) CachingBuildEngine(com.facebook.buck.rules.CachingBuildEngine) JavaDepsFinder(com.facebook.buck.jvm.java.autodeps.JavaDepsFinder) BuildEngineBuildContext(com.facebook.buck.rules.BuildEngineBuildContext) Paths(java.nio.file.Paths) ExecutorPool(com.facebook.buck.step.ExecutorPool) BuildEngine(com.facebook.buck.rules.BuildEngine) BuildContext(com.facebook.buck.rules.BuildContext) BuildRuleResolver(com.facebook.buck.rules.BuildRuleResolver) RuleKeyFactoryManager(com.facebook.buck.rules.keys.RuleKeyFactoryManager) ListeningExecutorService(com.google.common.util.concurrent.ListeningExecutorService) BuckEventBus(com.facebook.buck.event.BuckEventBus) JavaDepsFinder(com.facebook.buck.jvm.java.autodeps.JavaDepsFinder) DefaultRuleKeyCache(com.facebook.buck.rules.keys.DefaultRuleKeyCache) TargetGraph(com.facebook.buck.rules.TargetGraph) RuleKeyFactoryManager(com.facebook.buck.rules.keys.RuleKeyFactoryManager) WeightedListeningExecutorService(com.facebook.buck.util.concurrent.WeightedListeningExecutorService) BuildFileParseException(com.facebook.buck.json.BuildFileParseException) BuildRuleResolver(com.facebook.buck.rules.BuildRuleResolver) DefaultStepRunner(com.facebook.buck.step.DefaultStepRunner) BuildEngineBuildContext(com.facebook.buck.rules.BuildEngineBuildContext) DefaultTargetNodeToBuildRuleTransformer(com.facebook.buck.rules.DefaultTargetNodeToBuildRuleTransformer) Cell(com.facebook.buck.rules.Cell) BuildTargetException(com.facebook.buck.model.BuildTargetException) ConcurrencyLimit(com.facebook.buck.util.concurrent.ConcurrencyLimit) LocalCachingBuildEngineDelegate(com.facebook.buck.rules.LocalCachingBuildEngineDelegate) ActionGraph(com.facebook.buck.rules.ActionGraph) CachingBuildEngine(com.facebook.buck.rules.CachingBuildEngine) SourcePathResolver(com.facebook.buck.rules.SourcePathResolver) SourcePathRuleFinder(com.facebook.buck.rules.SourcePathRuleFinder) CachingBuildEngine(com.facebook.buck.rules.CachingBuildEngine) BuildEngine(com.facebook.buck.rules.BuildEngine) ExecutionContext(com.facebook.buck.step.ExecutionContext) ExecutorPool(com.facebook.buck.step.ExecutorPool) WeightedListeningExecutorService(com.facebook.buck.util.concurrent.WeightedListeningExecutorService) ListeningExecutorService(com.google.common.util.concurrent.ListeningExecutorService) CachingBuildEngineBuckConfig(com.facebook.buck.rules.CachingBuildEngineBuckConfig)

Aggregations

TargetGraph (com.facebook.buck.rules.TargetGraph)267 Test (org.junit.Test)230 BuildRuleResolver (com.facebook.buck.rules.BuildRuleResolver)195 DefaultTargetNodeToBuildRuleTransformer (com.facebook.buck.rules.DefaultTargetNodeToBuildRuleTransformer)179 BuildTarget (com.facebook.buck.model.BuildTarget)157 FakeProjectFilesystem (com.facebook.buck.testutil.FakeProjectFilesystem)126 ProjectFilesystem (com.facebook.buck.io.ProjectFilesystem)119 FakeSourcePath (com.facebook.buck.rules.FakeSourcePath)96 SourcePathRuleFinder (com.facebook.buck.rules.SourcePathRuleFinder)88 SourcePathResolver (com.facebook.buck.rules.SourcePathResolver)75 BuildRule (com.facebook.buck.rules.BuildRule)60 AllExistingProjectFilesystem (com.facebook.buck.testutil.AllExistingProjectFilesystem)58 Path (java.nio.file.Path)48 PathSourcePath (com.facebook.buck.rules.PathSourcePath)45 TargetNode (com.facebook.buck.rules.TargetNode)44 SourcePath (com.facebook.buck.rules.SourcePath)37 DefaultBuildTargetSourcePath (com.facebook.buck.rules.DefaultBuildTargetSourcePath)36 ImmutableSet (com.google.common.collect.ImmutableSet)29 Optional (java.util.Optional)25 ImmutableList (com.google.common.collect.ImmutableList)24