Search in sources :

Example 1 with TargetNodeFactory

use of com.facebook.buck.rules.TargetNodeFactory in project buck by facebook.

the class BuildCommand method executeDistributedBuild.

private int executeDistributedBuild(final CommandRunnerParams params, ActionAndTargetGraphs graphs, final WeightedListeningExecutorService executorService) throws IOException, InterruptedException {
    // Distributed builds serialize and send the unversioned target graph,
    // and then deserialize and version remotely.
    TargetGraphAndBuildTargets targetGraphAndBuildTargets = graphs.unversionedTargetGraph;
    ProjectFilesystem filesystem = params.getCell().getFilesystem();
    FileHashCache fileHashCache = params.getFileHashCache();
    DistBuildTypeCoercerFactory typeCoercerFactory = new DistBuildTypeCoercerFactory(params.getObjectMapper());
    ParserTargetNodeFactory<TargetNode<?, ?>> parserTargetNodeFactory = DefaultParserTargetNodeFactory.createForDistributedBuild(new ConstructorArgMarshaller(typeCoercerFactory), new TargetNodeFactory(typeCoercerFactory));
    DistBuildTargetGraphCodec targetGraphCodec = new DistBuildTargetGraphCodec(params.getObjectMapper(), parserTargetNodeFactory, new Function<TargetNode<?, ?>, Map<String, Object>>() {

        @Nullable
        @Override
        public Map<String, Object> apply(TargetNode<?, ?> input) {
            try {
                return params.getParser().getRawTargetNode(params.getBuckEventBus(), params.getCell().getCell(input.getBuildTarget()), false, /* enableProfiling */
                executorService, input);
            } catch (BuildFileParseException e) {
                throw new RuntimeException(e);
            }
        }
    }, targetGraphAndBuildTargets.getBuildTargets().stream().map(t -> t.getFullyQualifiedName()).collect(Collectors.toSet()));
    BuildJobState jobState = computeDistributedBuildJobState(targetGraphCodec, params, targetGraphAndBuildTargets, graphs.actionGraph, executorService);
    if (distributedBuildStateFile != null) {
        Path stateDumpPath = Paths.get(distributedBuildStateFile);
        BuildJobStateSerializer.serialize(jobState, filesystem.newFileOutputStream(stateDumpPath));
        return 0;
    } else {
        BuckVersion buckVersion = getBuckVersion();
        Preconditions.checkArgument(params.getInvocationInfo().isPresent());
        try (DistBuildService service = DistBuildFactory.newDistBuildService(params);
            DistBuildLogStateTracker distBuildLogStateTracker = DistBuildFactory.newDistBuildLogStateTracker(params.getInvocationInfo().get().getLogDirectoryPath(), filesystem)) {
            DistBuildClientExecutor build = new DistBuildClientExecutor(jobState, service, distBuildLogStateTracker, 1000, /* millisBetweenStatusPoll */
            buckVersion);
            int exitCode = build.executeAndPrintFailuresToEventBus(executorService, filesystem, fileHashCache, params.getBuckEventBus());
            // TODO(shivanker): Add a flag to disable building, and only fetch from the cache.
            if (exitCode == 0) {
                exitCode = executeLocalBuild(params, graphs.actionGraph, executorService);
            }
            return exitCode;
        }
    }
}
Also used : Path(java.nio.file.Path) SourcePath(com.facebook.buck.rules.SourcePath) FileHashCache(com.facebook.buck.util.cache.FileHashCache) TargetNode(com.facebook.buck.rules.TargetNode) BuildJobState(com.facebook.buck.distributed.thrift.BuildJobState) BuildFileParseException(com.facebook.buck.json.BuildFileParseException) DistBuildService(com.facebook.buck.distributed.DistBuildService) ConstructorArgMarshaller(com.facebook.buck.rules.ConstructorArgMarshaller) BuckVersion(com.facebook.buck.distributed.thrift.BuckVersion) TargetNodeFactory(com.facebook.buck.rules.TargetNodeFactory) DefaultParserTargetNodeFactory(com.facebook.buck.parser.DefaultParserTargetNodeFactory) ParserTargetNodeFactory(com.facebook.buck.parser.ParserTargetNodeFactory) DistBuildClientExecutor(com.facebook.buck.distributed.DistBuildClientExecutor) DistBuildTargetGraphCodec(com.facebook.buck.distributed.DistBuildTargetGraphCodec) DistBuildLogStateTracker(com.facebook.buck.distributed.DistBuildLogStateTracker) DistBuildTypeCoercerFactory(com.facebook.buck.distributed.DistBuildTypeCoercerFactory) ProjectFilesystem(com.facebook.buck.io.ProjectFilesystem) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) ConcurrentMap(java.util.concurrent.ConcurrentMap) TargetGraphAndBuildTargets(com.facebook.buck.rules.TargetGraphAndBuildTargets) Nullable(javax.annotation.Nullable)

Example 2 with TargetNodeFactory

use of com.facebook.buck.rules.TargetNodeFactory in project buck by facebook.

the class GenruleDescriptionTest method testImplicitDepsAreAddedCorrectly.

@Test
public void testImplicitDepsAreAddedCorrectly() throws Exception {
    Description<GenruleDescription.Arg> genruleDescription = new GenruleDescription();
    Map<String, Object> instance = ImmutableMap.of("srcs", ImmutableList.of(":baz", "//biz:baz"), "out", "AndroidManifest.xml", "cmd", "$(exe //bin:executable) $(location :arg)");
    ProjectFilesystem projectFilesystem = new AllExistingProjectFilesystem();
    ConstructorArgMarshaller marshaller = new ConstructorArgMarshaller(new DefaultTypeCoercerFactory(ObjectMappers.newDefaultInstance()));
    ImmutableSet.Builder<BuildTarget> declaredDeps = ImmutableSet.builder();
    ImmutableSet.Builder<VisibilityPattern> visibilityPatterns = ImmutableSet.builder();
    GenruleDescription.Arg constructorArg = genruleDescription.createUnpopulatedConstructorArg();
    BuildTarget buildTarget = BuildTargetFactory.newInstance("//foo:bar");
    marshaller.populate(createCellRoots(projectFilesystem), projectFilesystem, buildTarget, constructorArg, declaredDeps, visibilityPatterns, instance);
    TargetNode<GenruleDescription.Arg, ?> targetNode = new TargetNodeFactory(new DefaultTypeCoercerFactory(ObjectMappers.newDefaultInstance())).create(Hashing.sha1().hashString(buildTarget.getFullyQualifiedName(), UTF_8), genruleDescription, constructorArg, projectFilesystem, buildTarget, declaredDeps.build(), visibilityPatterns.build(), createCellRoots(projectFilesystem));
    assertEquals("SourcePaths and targets from cmd string should be extracted as extra deps.", ImmutableSet.of("//foo:baz", "//biz:baz", "//bin:executable", "//foo:arg"), targetNode.getExtraDeps().stream().map(Object::toString).collect(MoreCollectors.toImmutableSet()));
}
Also used : DefaultTypeCoercerFactory(com.facebook.buck.rules.coercer.DefaultTypeCoercerFactory) AllExistingProjectFilesystem(com.facebook.buck.testutil.AllExistingProjectFilesystem) VisibilityPattern(com.facebook.buck.rules.VisibilityPattern) ConstructorArgMarshaller(com.facebook.buck.rules.ConstructorArgMarshaller) ImmutableSet(com.google.common.collect.ImmutableSet) BuildTarget(com.facebook.buck.model.BuildTarget) TargetNodeFactory(com.facebook.buck.rules.TargetNodeFactory) ProjectFilesystem(com.facebook.buck.io.ProjectFilesystem) AllExistingProjectFilesystem(com.facebook.buck.testutil.AllExistingProjectFilesystem) Test(org.junit.Test)

Example 3 with TargetNodeFactory

use of com.facebook.buck.rules.TargetNodeFactory in project buck by facebook.

the class DistBuildStateTest method createDefaultCodec.

public static DistBuildTargetGraphCodec createDefaultCodec(final Cell cell, final Optional<Parser> parser) {
    // NOPMD confused by lambda
    ObjectMapper objectMapper = ObjectMappers.newDefaultInstance();
    BuckEventBus eventBus = BuckEventBusFactory.newInstance();
    Function<? super TargetNode<?, ?>, ? extends Map<String, Object>> nodeToRawNode;
    if (parser.isPresent()) {
        nodeToRawNode = (Function<TargetNode<?, ?>, Map<String, Object>>) input -> {
            try {
                return parser.get().getRawTargetNode(eventBus, cell.getCell(input.getBuildTarget()), false, MoreExecutors.listeningDecorator(MoreExecutors.newDirectExecutorService()), input);
            } catch (BuildFileParseException e) {
                throw new RuntimeException(e);
            }
        };
    } else {
        nodeToRawNode = Functions.constant(ImmutableMap.<String, Object>of());
    }
    DistBuildTypeCoercerFactory typeCoercerFactory = new DistBuildTypeCoercerFactory(objectMapper);
    ParserTargetNodeFactory<TargetNode<?, ?>> parserTargetNodeFactory = DefaultParserTargetNodeFactory.createForDistributedBuild(new ConstructorArgMarshaller(typeCoercerFactory), new TargetNodeFactory(typeCoercerFactory));
    return new DistBuildTargetGraphCodec(objectMapper, parserTargetNodeFactory, nodeToRawNode, ImmutableSet.of());
}
Also used : BuckEventBus(com.facebook.buck.event.BuckEventBus) BroadcastEventListener(com.facebook.buck.event.listener.BroadcastEventListener) ActionGraph(com.facebook.buck.rules.ActionGraph) FakeProjectFilesystem(com.facebook.buck.testutil.FakeProjectFilesystem) SourcePathRuleFinder(com.facebook.buck.rules.SourcePathRuleFinder) ObjectMappers(com.facebook.buck.util.ObjectMappers) TestDataHelper(com.facebook.buck.testutil.integration.TestDataHelper) TypeCoercerFactory(com.facebook.buck.rules.coercer.TypeCoercerFactory) Assert.assertThat(org.junit.Assert.assertThat) ProjectFilesystem(com.facebook.buck.io.ProjectFilesystem) BuckConfig(com.facebook.buck.cli.BuckConfig) TargetNodeFactory(com.facebook.buck.rules.TargetNodeFactory) ProcessExecutor(com.facebook.buck.util.ProcessExecutor) SourcePathResolver(com.facebook.buck.rules.SourcePathResolver) Map(java.util.Map) DefaultTargetNodeToBuildRuleTransformer(com.facebook.buck.rules.DefaultTargetNodeToBuildRuleTransformer) DefaultTypeCoercerFactory(com.facebook.buck.rules.coercer.DefaultTypeCoercerFactory) JavaLibraryDescription(com.facebook.buck.jvm.java.JavaLibraryDescription) Cell(com.facebook.buck.rules.Cell) Path(java.nio.file.Path) JavaLibraryBuilder(com.facebook.buck.jvm.java.JavaLibraryBuilder) Function(com.google.common.base.Function) ImmutableSet(com.google.common.collect.ImmutableSet) ImmutableMap(com.google.common.collect.ImmutableMap) TargetGraph(com.facebook.buck.rules.TargetGraph) Platform(com.facebook.buck.util.environment.Platform) DefaultParserTargetNodeFactory(com.facebook.buck.parser.DefaultParserTargetNodeFactory) DefaultCellPathResolver(com.facebook.buck.rules.DefaultCellPathResolver) DefaultFileHashCache(com.facebook.buck.util.cache.DefaultFileHashCache) BuildTarget(com.facebook.buck.model.BuildTarget) Collectors(java.util.stream.Collectors) Executors(java.util.concurrent.Executors) KnownBuildRuleTypesFactory(com.facebook.buck.rules.KnownBuildRuleTypesFactory) ProjectWorkspace(com.facebook.buck.testutil.integration.ProjectWorkspace) ConstructorArgMarshaller(com.facebook.buck.rules.ConstructorArgMarshaller) PathSourcePath(com.facebook.buck.rules.PathSourcePath) Optional(java.util.Optional) BuildRuleResolver(com.facebook.buck.rules.BuildRuleResolver) DefaultProcessExecutor(com.facebook.buck.util.DefaultProcessExecutor) BuckEventBus(com.facebook.buck.event.BuckEventBus) MoreExecutors(com.google.common.util.concurrent.MoreExecutors) FakeAndroidDirectoryResolver(com.facebook.buck.android.FakeAndroidDirectoryResolver) Config(com.facebook.buck.config.Config) TemporaryPaths(com.facebook.buck.testutil.integration.TemporaryPaths) BuckEventBusFactory(com.facebook.buck.event.BuckEventBusFactory) Lists(com.google.common.collect.Lists) BuildJobState(com.facebook.buck.distributed.thrift.BuildJobState) ParserConfig(com.facebook.buck.parser.ParserConfig) ImmutableList(com.google.common.collect.ImmutableList) BuildTargetFactory(com.facebook.buck.model.BuildTargetFactory) BuildFileParseException(com.facebook.buck.json.BuildFileParseException) ExpectedException(org.junit.rules.ExpectedException) ImmutableSortedSet(com.google.common.collect.ImmutableSortedSet) Functions(com.google.common.base.Functions) Parser(com.facebook.buck.parser.Parser) TargetNode(com.facebook.buck.rules.TargetNode) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Assert.assertTrue(org.junit.Assert.assertTrue) Matchers(org.hamcrest.Matchers) Test(org.junit.Test) IOException(java.io.IOException) Architecture(com.facebook.buck.util.environment.Architecture) FakeBuckConfig(com.facebook.buck.cli.FakeBuckConfig) DefaultBuildTargetSourcePath(com.facebook.buck.rules.DefaultBuildTargetSourcePath) StackedFileHashCache(com.facebook.buck.util.cache.StackedFileHashCache) ConfigBuilder(com.facebook.buck.config.ConfigBuilder) ParserTargetNodeFactory(com.facebook.buck.parser.ParserTargetNodeFactory) Rule(org.junit.Rule) TestConsole(com.facebook.buck.testutil.TestConsole) TargetGraphFactory(com.facebook.buck.testutil.TargetGraphFactory) TestCellBuilder(com.facebook.buck.rules.TestCellBuilder) Preconditions(com.google.common.base.Preconditions) Assert.assertEquals(org.junit.Assert.assertEquals) TargetNode(com.facebook.buck.rules.TargetNode) BuildFileParseException(com.facebook.buck.json.BuildFileParseException) ConstructorArgMarshaller(com.facebook.buck.rules.ConstructorArgMarshaller) TargetNodeFactory(com.facebook.buck.rules.TargetNodeFactory) DefaultParserTargetNodeFactory(com.facebook.buck.parser.DefaultParserTargetNodeFactory) ParserTargetNodeFactory(com.facebook.buck.parser.ParserTargetNodeFactory) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper)

Example 4 with TargetNodeFactory

use of com.facebook.buck.rules.TargetNodeFactory in project buck by facebook.

the class DistBuildSlaveExecutor method createGraphCodec.

private DistBuildTargetGraphCodec createGraphCodec() {
    DistBuildTypeCoercerFactory typeCoercerFactory = new DistBuildTypeCoercerFactory(args.getObjectMapper());
    ParserTargetNodeFactory<TargetNode<?, ?>> parserTargetNodeFactory = DefaultParserTargetNodeFactory.createForDistributedBuild(new ConstructorArgMarshaller(typeCoercerFactory), new TargetNodeFactory(typeCoercerFactory));
    DistBuildTargetGraphCodec targetGraphCodec = new DistBuildTargetGraphCodec(args.getObjectMapper(), parserTargetNodeFactory, new Function<TargetNode<?, ?>, Map<String, Object>>() {

        @Nullable
        @Override
        public Map<String, Object> apply(TargetNode<?, ?> input) {
            try {
                return args.getParser().getRawTargetNode(args.getBuckEventBus(), args.getRootCell().getCell(input.getBuildTarget()), /* enableProfiling */
                false, args.getExecutorService(), input);
            } catch (BuildFileParseException e) {
                throw new RuntimeException(e);
            }
        }
    }, new HashSet<>(args.getState().getRemoteState().getTopLevelTargets()));
    return targetGraphCodec;
}
Also used : TargetNode(com.facebook.buck.rules.TargetNode) BuildFileParseException(com.facebook.buck.json.BuildFileParseException) ConstructorArgMarshaller(com.facebook.buck.rules.ConstructorArgMarshaller) TargetNodeFactory(com.facebook.buck.rules.TargetNodeFactory) DefaultParserTargetNodeFactory(com.facebook.buck.parser.DefaultParserTargetNodeFactory) ParserTargetNodeFactory(com.facebook.buck.parser.ParserTargetNodeFactory) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) Nullable(javax.annotation.Nullable)

Example 5 with TargetNodeFactory

use of com.facebook.buck.rules.TargetNodeFactory in project buck by facebook.

the class OwnersReportTest method createTargetNode.

private TargetNode<?, ?> createTargetNode(BuildTarget buildTarget, ImmutableSet<Path> inputs) {
    Description<FakeRuleDescription.FakeArg> description = new FakeRuleDescription();
    FakeRuleDescription.FakeArg arg = description.createUnpopulatedConstructorArg();
    arg.inputs = inputs;
    try {
        return new TargetNodeFactory(new DefaultTypeCoercerFactory(ObjectMappers.newDefaultInstance())).create(Hashing.sha1().hashString(buildTarget.getFullyQualifiedName(), UTF_8), description, arg, filesystem, buildTarget, ImmutableSet.of(), ImmutableSet.of(), createCellRoots(filesystem));
    } catch (NoSuchBuildTargetException e) {
        throw new RuntimeException(e);
    }
}
Also used : TargetNodeFactory(com.facebook.buck.rules.TargetNodeFactory) NoSuchBuildTargetException(com.facebook.buck.parser.NoSuchBuildTargetException) DefaultTypeCoercerFactory(com.facebook.buck.rules.coercer.DefaultTypeCoercerFactory)

Aggregations

TargetNodeFactory (com.facebook.buck.rules.TargetNodeFactory)5 ConstructorArgMarshaller (com.facebook.buck.rules.ConstructorArgMarshaller)4 ProjectFilesystem (com.facebook.buck.io.ProjectFilesystem)3 BuildFileParseException (com.facebook.buck.json.BuildFileParseException)3 DefaultParserTargetNodeFactory (com.facebook.buck.parser.DefaultParserTargetNodeFactory)3 ParserTargetNodeFactory (com.facebook.buck.parser.ParserTargetNodeFactory)3 TargetNode (com.facebook.buck.rules.TargetNode)3 DefaultTypeCoercerFactory (com.facebook.buck.rules.coercer.DefaultTypeCoercerFactory)3 ImmutableMap (com.google.common.collect.ImmutableMap)3 Map (java.util.Map)3 BuildJobState (com.facebook.buck.distributed.thrift.BuildJobState)2 BuildTarget (com.facebook.buck.model.BuildTarget)2 FakeAndroidDirectoryResolver (com.facebook.buck.android.FakeAndroidDirectoryResolver)1 BuckConfig (com.facebook.buck.cli.BuckConfig)1 FakeBuckConfig (com.facebook.buck.cli.FakeBuckConfig)1 Config (com.facebook.buck.config.Config)1 ConfigBuilder (com.facebook.buck.config.ConfigBuilder)1 DistBuildClientExecutor (com.facebook.buck.distributed.DistBuildClientExecutor)1 DistBuildLogStateTracker (com.facebook.buck.distributed.DistBuildLogStateTracker)1 DistBuildService (com.facebook.buck.distributed.DistBuildService)1