Search in sources :

Example 11 with Cell

use of com.facebook.buck.rules.Cell in project buck by facebook.

the class OwnersReport method generateOwnersReport.

@VisibleForTesting
static OwnersReport generateOwnersReport(Cell rootCell, TargetNode<?, ?> targetNode, Iterable<String> filePaths) {
    // Process arguments assuming they are all relative file paths.
    Set<Path> inputs = Sets.newHashSet();
    Set<String> nonExistentInputs = Sets.newHashSet();
    Set<String> nonFileInputs = Sets.newHashSet();
    for (String filePath : filePaths) {
        Path file = rootCell.getFilesystem().getPathForRelativePath(filePath);
        if (!Files.exists(file)) {
            nonExistentInputs.add(filePath);
        } else if (!Files.isRegularFile(file)) {
            nonFileInputs.add(filePath);
        } else {
            inputs.add(rootCell.getFilesystem().getPath(filePath));
        }
    }
    // Try to find owners for each valid and existing file.
    Set<Path> inputsWithNoOwners = Sets.newHashSet(inputs);
    SetMultimap<TargetNode<?, ?>, Path> owners = TreeMultimap.create();
    for (final Path commandInput : inputs) {
        Predicate<Path> startsWith = input -> !commandInput.equals(input) && commandInput.startsWith(input);
        Set<Path> ruleInputs = targetNode.getInputs();
        if (ruleInputs.contains(commandInput) || FluentIterable.from(ruleInputs).anyMatch(startsWith)) {
            inputsWithNoOwners.remove(commandInput);
            owners.put(targetNode, commandInput);
        }
    }
    return new OwnersReport(owners, inputsWithNoOwners, nonExistentInputs, nonFileInputs);
}
Also used : Path(java.nio.file.Path) BuckEventBus(com.facebook.buck.event.BuckEventBus) BuildFileTree(com.facebook.buck.model.BuildFileTree) ProjectFilesystem(com.facebook.buck.io.ProjectFilesystem) ImmutableList(com.google.common.collect.ImmutableList) FluentIterable(com.google.common.collect.FluentIterable) TreeMultimap(com.google.common.collect.TreeMultimap) Map(java.util.Map) BuildFileParseException(com.facebook.buck.json.BuildFileParseException) Cell(com.facebook.buck.rules.Cell) Path(java.nio.file.Path) ImmutableSetMultimap(com.google.common.collect.ImmutableSetMultimap) Parser(com.facebook.buck.parser.Parser) ImmutableSet(com.google.common.collect.ImmutableSet) Files(java.nio.file.Files) TargetNode(com.facebook.buck.rules.TargetNode) Set(java.util.Set) IOException(java.io.IOException) Console(com.facebook.buck.util.Console) Maps(com.google.common.collect.Maps) SetMultimap(com.google.common.collect.SetMultimap) Sets(com.google.common.collect.Sets) MorePaths(com.facebook.buck.io.MorePaths) Predicate(com.google.common.base.Predicate) Optional(java.util.Optional) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) ListeningExecutorService(com.google.common.util.concurrent.ListeningExecutorService) TargetNode(com.facebook.buck.rules.TargetNode) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 12 with Cell

use of com.facebook.buck.rules.Cell in project buck by facebook.

the class BuckQueryEnvironment method getBuildFiles.

@Override
public ImmutableSet<QueryTarget> getBuildFiles(Set<QueryTarget> targets) throws QueryException {
    final ProjectFilesystem cellFilesystem = rootCell.getFilesystem();
    final Path rootPath = cellFilesystem.getRootPath();
    Preconditions.checkState(rootPath.isAbsolute());
    ImmutableSet.Builder<QueryTarget> builder = ImmutableSet.builder();
    for (QueryTarget target : targets) {
        Preconditions.checkState(target instanceof QueryBuildTarget);
        BuildTarget buildTarget = ((QueryBuildTarget) target).getBuildTarget();
        Cell cell = rootCell.getCell(buildTarget);
        if (!buildFileTrees.containsKey(cell)) {
            LOG.info("Creating a new filesystem-backed build file tree for %s", cell.getRoot());
            buildFileTrees.put(cell, new FilesystemBackedBuildFileTree(cell.getFilesystem(), cell.getBuildFileName()));
        }
        BuildFileTree buildFileTree = Preconditions.checkNotNull(buildFileTrees.get(cell));
        Optional<Path> path = buildFileTree.getBasePathOfAncestorTarget(buildTarget.getBasePath());
        Preconditions.checkState(path.isPresent());
        Path buildFilePath = MorePaths.relativize(rootPath, cell.getFilesystem().resolve(path.get()).resolve(cell.getBuildFileName()));
        Preconditions.checkState(cellFilesystem.exists(buildFilePath));
        builder.add(QueryFileTarget.of(buildFilePath));
    }
    return builder.build();
}
Also used : Path(java.nio.file.Path) QueryTarget(com.facebook.buck.query.QueryTarget) FilesystemBackedBuildFileTree(com.facebook.buck.model.FilesystemBackedBuildFileTree) ImmutableSet(com.google.common.collect.ImmutableSet) QueryBuildTarget(com.facebook.buck.query.QueryBuildTarget) BuildTarget(com.facebook.buck.model.BuildTarget) BuildFileTree(com.facebook.buck.model.BuildFileTree) FilesystemBackedBuildFileTree(com.facebook.buck.model.FilesystemBackedBuildFileTree) ProjectFilesystem(com.facebook.buck.io.ProjectFilesystem) Cell(com.facebook.buck.rules.Cell) QueryBuildTarget(com.facebook.buck.query.QueryBuildTarget)

Example 13 with Cell

use of com.facebook.buck.rules.Cell in project buck by facebook.

the class DistBuildFactory method createDistBuildExecutor.

public static DistBuildSlaveExecutor createDistBuildExecutor(BuildJobState jobState, CommandRunnerParams params, WeightedListeningExecutorService executorService, DistBuildService service, DistBuildMode mode, int coordinatorPort, Optional<StampedeId> stampedeId, Optional<Path> globalCacheDir) throws IOException {
    DistBuildState state = DistBuildState.load(Optional.of(params.getBuckConfig()), jobState, params.getCell(), params.getKnownBuildRuleTypesFactory());
    Preconditions.checkArgument(state.getCells().size() > 0);
    // Create a cache factory which uses a combination of the distributed build config,
    // overridden with the local buck config (i.e. the build slave).
    Cell rootCell = Preconditions.checkNotNull(state.getCells().get(0));
    ArtifactCacheFactory distBuildArtifactCacheFactory = params.getArtifactCacheFactory().cloneWith(rootCell.getBuckConfig());
    DistBuildSlaveExecutor executor = new DistBuildSlaveExecutor(DistBuildExecutorArgs.builder().setBuckEventBus(params.getBuckEventBus()).setPlatform(params.getPlatform()).setClock(params.getClock()).setArtifactCache(distBuildArtifactCacheFactory.newInstance(true)).setState(state).setObjectMapper(params.getObjectMapper()).setRootCell(params.getCell()).setParser(params.getParser()).setExecutorService(executorService).setActionGraphCache(params.getActionGraphCache()).setCacheKeySeed(params.getBuckConfig().getKeySeed()).setConsole(params.getConsole()).setProvider(new MultiSourceContentsProvider(service, globalCacheDir)).setExecutors(params.getExecutors()).setDistBuildMode(mode).setCoordinatorPort(coordinatorPort).setStampedeId(stampedeId.orElse(new StampedeId().setId("LOCAL_FILE"))).setVersionedTargetGraphCache(params.getVersionedTargetGraphCache()).build());
    return executor;
}
Also used : MultiSourceContentsProvider(com.facebook.buck.distributed.MultiSourceContentsProvider) StampedeId(com.facebook.buck.distributed.thrift.StampedeId) DistBuildState(com.facebook.buck.distributed.DistBuildState) Cell(com.facebook.buck.rules.Cell) ArtifactCacheFactory(com.facebook.buck.artifact_cache.ArtifactCacheFactory) DistBuildSlaveExecutor(com.facebook.buck.distributed.DistBuildSlaveExecutor)

Example 14 with Cell

use of com.facebook.buck.rules.Cell in project buck by facebook.

the class DistBuildCellIndexer method apply.

@Override
public Integer apply(Path input) {
    // Non-cell Paths are just stored in the root cell data marked as absolute paths.
    Integer i = rootCell.getKnownRoots().contains(input) ? index.get(input) : ROOT_CELL_INDEX;
    if (i == null) {
        i = index.size();
        index.put(input, i);
        Cell cell = rootCell.getCellIgnoringVisibilityCheck(input);
        state.put(i, dumpCell(cell));
    }
    return i;
}
Also used : BuildJobStateCell(com.facebook.buck.distributed.thrift.BuildJobStateCell) Cell(com.facebook.buck.rules.Cell)

Example 15 with Cell

use of com.facebook.buck.rules.Cell in project buck by facebook.

the class DistBuildSlaveExecutor method createStackOfDefaultFileHashCache.

private StackedFileHashCache createStackOfDefaultFileHashCache() throws IOException {
    ImmutableList.Builder<ProjectFileHashCache> allCachesBuilder = ImmutableList.builder();
    Cell rootCell = args.getState().getRootCell();
    // 1. Add all cells (including the root cell).
    for (Path cellPath : rootCell.getKnownRoots()) {
        Cell cell = rootCell.getCell(cellPath);
        allCachesBuilder.add(DefaultFileHashCache.createDefaultFileHashCache(cell.getFilesystem()));
    }
    // 2. Add the Operating System roots.
    allCachesBuilder.addAll(DefaultFileHashCache.createOsRootDirectoriesCaches());
    return new StackedFileHashCache(allCachesBuilder.build());
}
Also used : Path(java.nio.file.Path) ImmutableList(com.google.common.collect.ImmutableList) ProjectFileHashCache(com.facebook.buck.util.cache.ProjectFileHashCache) StackedFileHashCache(com.facebook.buck.util.cache.StackedFileHashCache) Cell(com.facebook.buck.rules.Cell)

Aggregations

Cell (com.facebook.buck.rules.Cell)87 Test (org.junit.Test)57 Path (java.nio.file.Path)46 TestCellBuilder (com.facebook.buck.rules.TestCellBuilder)41 BuildTarget (com.facebook.buck.model.BuildTarget)25 BuckConfig (com.facebook.buck.cli.BuckConfig)24 FakeBuckConfig (com.facebook.buck.cli.FakeBuckConfig)22 ProjectFilesystem (com.facebook.buck.io.ProjectFilesystem)21 PathSourcePath (com.facebook.buck.rules.PathSourcePath)15 FakeProjectFilesystem (com.facebook.buck.testutil.FakeProjectFilesystem)14 ImmutableSet (com.google.common.collect.ImmutableSet)14 BuckEventBus (com.facebook.buck.event.BuckEventBus)12 ImmutableMap (com.google.common.collect.ImmutableMap)12 BroadcastEventListener (com.facebook.buck.event.listener.BroadcastEventListener)11 ParserConfig (com.facebook.buck.parser.ParserConfig)11 IOException (java.io.IOException)11 Map (java.util.Map)11 TargetNode (com.facebook.buck.rules.TargetNode)10 ProjectWorkspace (com.facebook.buck.testutil.integration.ProjectWorkspace)10 ImmutableList (com.google.common.collect.ImmutableList)10