Search in sources :

Example 41 with VisibleForTesting

use of org.apache.beam.vendor.calcite.v1_28_0.com.google.common.annotations.VisibleForTesting in project buck by facebook.

the class MiniAapt method processXmlFile.

@VisibleForTesting
void processXmlFile(ProjectFilesystem filesystem, Path xmlFile, ImmutableSet.Builder<RDotTxtEntry> references) throws IOException, XPathExpressionException, ResourceParseException {
    try (InputStream stream = filesystem.newFileInputStream(xmlFile)) {
        Document dom = parseXml(xmlFile, stream);
        NodeList nodesWithIds = (NodeList) ANDROID_ID_DEFINITION.evaluate(dom, XPathConstants.NODESET);
        for (int i = 0; i < nodesWithIds.getLength(); i++) {
            String resourceName = nodesWithIds.item(i).getNodeValue();
            if (!resourceName.startsWith(ID_DEFINITION_PREFIX)) {
                throw new ResourceParseException("Invalid definition of a resource: '%s'", resourceName);
            }
            Preconditions.checkState(resourceName.startsWith(ID_DEFINITION_PREFIX));
            resourceCollector.addIntResourceIfNotPresent(RType.ID, resourceName.substring(ID_DEFINITION_PREFIX.length()));
        }
        NodeList nodesUsingIds = (NodeList) ANDROID_ID_USAGE.evaluate(dom, XPathConstants.NODESET);
        for (int i = 0; i < nodesUsingIds.getLength(); i++) {
            String resourceName = nodesUsingIds.item(i).getNodeValue();
            int slashPosition = resourceName.indexOf('/');
            if (resourceName.charAt(0) != '@' || slashPosition == -1) {
                throw new ResourceParseException("Invalid definition of a resource: '%s'", resourceName);
            }
            String rawRType = resourceName.substring(1, slashPosition);
            String name = resourceName.substring(slashPosition + 1);
            String nodeName = nodesUsingIds.item(i).getNodeName();
            if (name.startsWith("android:") || nodeName.startsWith("tools:")) {
                continue;
            }
            if (!RESOURCE_TYPES.containsKey(rawRType)) {
                throw new ResourceParseException("Invalid reference '%s' in '%s'", resourceName, xmlFile);
            }
            RType rType = Preconditions.checkNotNull(RESOURCE_TYPES.get(rawRType));
            references.add(new FakeRDotTxtEntry(IdType.INT, rType, sanitizeName(name)));
        }
    }
}
Also used : InputStream(java.io.InputStream) NodeList(org.w3c.dom.NodeList) RType(com.facebook.buck.android.aapt.RDotTxtEntry.RType) Document(org.w3c.dom.Document) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 42 with VisibleForTesting

use of org.apache.beam.vendor.calcite.v1_28_0.com.google.common.annotations.VisibleForTesting in project buck by facebook.

the class MiniAapt method verifyReferences.

@VisibleForTesting
ImmutableSet<RDotTxtEntry> verifyReferences(ProjectFilesystem filesystem, ImmutableSet<RDotTxtEntry> references) throws IOException {
    ImmutableSet.Builder<RDotTxtEntry> unresolved = ImmutableSet.builder();
    ImmutableSet.Builder<RDotTxtEntry> definitionsBuilder = ImmutableSet.builder();
    definitionsBuilder.addAll(resourceCollector.getResources());
    for (Path depRTxt : pathsToSymbolsOfDeps) {
        Iterable<String> lines = FluentIterable.from(filesystem.readLines(depRTxt)).filter(input -> !Strings.isNullOrEmpty(input)).toList();
        for (String line : lines) {
            Optional<RDotTxtEntry> entry = RDotTxtEntry.parse(line);
            Preconditions.checkState(entry.isPresent());
            definitionsBuilder.add(entry.get());
        }
    }
    Set<RDotTxtEntry> definitions = definitionsBuilder.build();
    for (RDotTxtEntry reference : references) {
        if (!definitions.contains(reference)) {
            unresolved.add(reference);
        }
    }
    return unresolved.build();
}
Also used : SourcePath(com.facebook.buck.rules.SourcePath) Path(java.nio.file.Path) BuckEventBus(com.facebook.buck.event.BuckEventBus) XPathExpressionException(javax.xml.xpath.XPathExpressionException) Step(com.facebook.buck.step.Step) XPathConstants(javax.xml.xpath.XPathConstants) SourcePath(com.facebook.buck.rules.SourcePath) XmlDomParser(com.facebook.buck.util.XmlDomParser) IdType(com.facebook.buck.android.aapt.RDotTxtEntry.IdType) XPathExpression(javax.xml.xpath.XPathExpression) ConsoleEvent(com.facebook.buck.event.ConsoleEvent) ProjectFilesystem(com.facebook.buck.io.ProjectFilesystem) ExecutionContext(com.facebook.buck.step.ExecutionContext) Strings(com.google.common.base.Strings) ImmutableList(com.google.common.collect.ImmutableList) FluentIterable(com.google.common.collect.FluentIterable) SourcePathResolver(com.facebook.buck.rules.SourcePathResolver) Document(org.w3c.dom.Document) Node(org.w3c.dom.Node) Path(java.nio.file.Path) ImmutableSortedSet(com.google.common.collect.ImmutableSortedSet) PrintWriter(java.io.PrintWriter) ImmutableSet(com.google.common.collect.ImmutableSet) NodeList(org.w3c.dom.NodeList) ImmutableMap(com.google.common.collect.ImmutableMap) Collection(java.util.Collection) Set(java.util.Set) IOException(java.io.IOException) XPathFactory(javax.xml.xpath.XPathFactory) Element(org.w3c.dom.Element) Ordering(com.google.common.collect.Ordering) SAXException(org.xml.sax.SAXException) Optional(java.util.Optional) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) AaptStep(com.facebook.buck.android.AaptStep) StepExecutionResult(com.facebook.buck.step.StepExecutionResult) RType(com.facebook.buck.android.aapt.RDotTxtEntry.RType) Joiner(com.google.common.base.Joiner) InputStream(java.io.InputStream) ImmutableSet(com.google.common.collect.ImmutableSet) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 43 with VisibleForTesting

use of org.apache.beam.vendor.calcite.v1_28_0.com.google.common.annotations.VisibleForTesting in project buck by facebook.

the class OwnersReport method generateOwnersReport.

@VisibleForTesting
static OwnersReport generateOwnersReport(Cell rootCell, TargetNode<?, ?> targetNode, Iterable<String> filePaths) {
    // Process arguments assuming they are all relative file paths.
    Set<Path> inputs = Sets.newHashSet();
    Set<String> nonExistentInputs = Sets.newHashSet();
    Set<String> nonFileInputs = Sets.newHashSet();
    for (String filePath : filePaths) {
        Path file = rootCell.getFilesystem().getPathForRelativePath(filePath);
        if (!Files.exists(file)) {
            nonExistentInputs.add(filePath);
        } else if (!Files.isRegularFile(file)) {
            nonFileInputs.add(filePath);
        } else {
            inputs.add(rootCell.getFilesystem().getPath(filePath));
        }
    }
    // Try to find owners for each valid and existing file.
    Set<Path> inputsWithNoOwners = Sets.newHashSet(inputs);
    SetMultimap<TargetNode<?, ?>, Path> owners = TreeMultimap.create();
    for (final Path commandInput : inputs) {
        Predicate<Path> startsWith = input -> !commandInput.equals(input) && commandInput.startsWith(input);
        Set<Path> ruleInputs = targetNode.getInputs();
        if (ruleInputs.contains(commandInput) || FluentIterable.from(ruleInputs).anyMatch(startsWith)) {
            inputsWithNoOwners.remove(commandInput);
            owners.put(targetNode, commandInput);
        }
    }
    return new OwnersReport(owners, inputsWithNoOwners, nonExistentInputs, nonFileInputs);
}
Also used : Path(java.nio.file.Path) BuckEventBus(com.facebook.buck.event.BuckEventBus) BuildFileTree(com.facebook.buck.model.BuildFileTree) ProjectFilesystem(com.facebook.buck.io.ProjectFilesystem) ImmutableList(com.google.common.collect.ImmutableList) FluentIterable(com.google.common.collect.FluentIterable) TreeMultimap(com.google.common.collect.TreeMultimap) Map(java.util.Map) BuildFileParseException(com.facebook.buck.json.BuildFileParseException) Cell(com.facebook.buck.rules.Cell) Path(java.nio.file.Path) ImmutableSetMultimap(com.google.common.collect.ImmutableSetMultimap) Parser(com.facebook.buck.parser.Parser) ImmutableSet(com.google.common.collect.ImmutableSet) Files(java.nio.file.Files) TargetNode(com.facebook.buck.rules.TargetNode) Set(java.util.Set) IOException(java.io.IOException) Console(com.facebook.buck.util.Console) Maps(com.google.common.collect.Maps) SetMultimap(com.google.common.collect.SetMultimap) Sets(com.google.common.collect.Sets) MorePaths(com.facebook.buck.io.MorePaths) Predicate(com.google.common.base.Predicate) Optional(java.util.Optional) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) ListeningExecutorService(com.google.common.util.concurrent.ListeningExecutorService) TargetNode(com.facebook.buck.rules.TargetNode) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 44 with VisibleForTesting

use of org.apache.beam.vendor.calcite.v1_28_0.com.google.common.annotations.VisibleForTesting in project buck by facebook.

the class ProjectCommand method generateWorkspacesForTargets.

@VisibleForTesting
static ImmutableSet<BuildTarget> generateWorkspacesForTargets(final CommandRunnerParams params, final TargetGraphAndTargets targetGraphAndTargets, ImmutableSet<BuildTarget> passedInTargetsSet, ImmutableSet<ProjectGenerator.Option> options, ImmutableList<String> buildWithBuckFlags, Optional<ImmutableSet<UnflavoredBuildTarget>> focusModules, Map<Path, ProjectGenerator> projectGenerators, boolean combinedProject, boolean buildWithBuck) throws IOException, InterruptedException {
    ImmutableSet<BuildTarget> targets;
    if (passedInTargetsSet.isEmpty()) {
        targets = targetGraphAndTargets.getProjectRoots().stream().map(TargetNode::getBuildTarget).collect(MoreCollectors.toImmutableSet());
    } else {
        targets = passedInTargetsSet;
    }
    LOG.debug("Generating workspace for config targets %s", targets);
    ImmutableSet.Builder<BuildTarget> requiredBuildTargetsBuilder = ImmutableSet.builder();
    for (final BuildTarget inputTarget : targets) {
        TargetNode<?, ?> inputNode = targetGraphAndTargets.getTargetGraph().get(inputTarget);
        XcodeWorkspaceConfigDescription.Arg workspaceArgs;
        if (inputNode.getDescription() instanceof XcodeWorkspaceConfigDescription) {
            TargetNode<XcodeWorkspaceConfigDescription.Arg, ?> castedWorkspaceNode = castToXcodeWorkspaceTargetNode(inputNode);
            workspaceArgs = castedWorkspaceNode.getConstructorArg();
        } else if (canGenerateImplicitWorkspaceForDescription(inputNode.getDescription())) {
            workspaceArgs = createImplicitWorkspaceArgs(inputNode);
        } else {
            throw new HumanReadableException("%s must be a xcode_workspace_config, apple_binary, apple_bundle, or apple_library", inputNode);
        }
        BuckConfig buckConfig = params.getBuckConfig();
        AppleConfig appleConfig = new AppleConfig(buckConfig);
        HalideBuckConfig halideBuckConfig = new HalideBuckConfig(buckConfig);
        CxxBuckConfig cxxBuckConfig = new CxxBuckConfig(buckConfig);
        SwiftBuckConfig swiftBuckConfig = new SwiftBuckConfig(buckConfig);
        CxxPlatform defaultCxxPlatform = params.getCell().getKnownBuildRuleTypes().getDefaultCxxPlatforms();
        WorkspaceAndProjectGenerator generator = new WorkspaceAndProjectGenerator(params.getCell(), targetGraphAndTargets.getTargetGraph(), workspaceArgs, inputTarget, options, combinedProject, buildWithBuck, buildWithBuckFlags, focusModules, !appleConfig.getXcodeDisableParallelizeBuild(), new ExecutableFinder(), params.getEnvironment(), params.getCell().getKnownBuildRuleTypes().getCxxPlatforms(), defaultCxxPlatform, params.getBuckConfig().getView(ParserConfig.class).getBuildFileName(), input -> ActionGraphCache.getFreshActionGraph(params.getBuckEventBus(), targetGraphAndTargets.getTargetGraph().getSubgraph(ImmutableSet.of(input))).getResolver(), params.getBuckEventBus(), halideBuckConfig, cxxBuckConfig, appleConfig, swiftBuckConfig);
        ListeningExecutorService executorService = params.getExecutors().get(ExecutorPool.PROJECT);
        Preconditions.checkNotNull(executorService, "CommandRunnerParams does not have executor for PROJECT pool");
        generator.generateWorkspaceAndDependentProjects(projectGenerators, executorService);
        ImmutableSet<BuildTarget> requiredBuildTargetsForWorkspace = generator.getRequiredBuildTargets();
        LOG.debug("Required build targets for workspace %s: %s", inputTarget, requiredBuildTargetsForWorkspace);
        requiredBuildTargetsBuilder.addAll(requiredBuildTargetsForWorkspace);
    }
    return requiredBuildTargetsBuilder.build();
}
Also used : AppleConfig(com.facebook.buck.apple.AppleConfig) ExecutableFinder(com.facebook.buck.io.ExecutableFinder) TargetNode(com.facebook.buck.rules.TargetNode) CxxPlatform(com.facebook.buck.cxx.CxxPlatform) WorkspaceAndProjectGenerator(com.facebook.buck.apple.project_generator.WorkspaceAndProjectGenerator) CxxBuckConfig(com.facebook.buck.cxx.CxxBuckConfig) ImmutableSet(com.google.common.collect.ImmutableSet) XcodeWorkspaceConfigDescription(com.facebook.buck.apple.XcodeWorkspaceConfigDescription) CxxBuckConfig(com.facebook.buck.cxx.CxxBuckConfig) JavaBuckConfig(com.facebook.buck.jvm.java.JavaBuckConfig) SwiftBuckConfig(com.facebook.buck.swift.SwiftBuckConfig) HalideBuckConfig(com.facebook.buck.halide.HalideBuckConfig) PythonBuckConfig(com.facebook.buck.python.PythonBuckConfig) HalideBuckConfig(com.facebook.buck.halide.HalideBuckConfig) BuildTarget(com.facebook.buck.model.BuildTarget) UnflavoredBuildTarget(com.facebook.buck.model.UnflavoredBuildTarget) HumanReadableException(com.facebook.buck.util.HumanReadableException) ListeningExecutorService(com.google.common.util.concurrent.ListeningExecutorService) SwiftBuckConfig(com.facebook.buck.swift.SwiftBuckConfig) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 45 with VisibleForTesting

use of org.apache.beam.vendor.calcite.v1_28_0.com.google.common.annotations.VisibleForTesting in project buck by facebook.

the class DirArtifactCache method getParentDirForRuleKey.

@VisibleForTesting
Path getParentDirForRuleKey(RuleKey ruleKey) {
    ImmutableList<String> folders = subfolders(ruleKey);
    Path result = cacheDir;
    for (String f : folders) {
        result = result.resolve(f);
    }
    return result;
}
Also used : Path(java.nio.file.Path) BorrowablePath(com.facebook.buck.io.BorrowablePath) LazyPath(com.facebook.buck.io.LazyPath) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Aggregations

VisibleForTesting (com.google.common.annotations.VisibleForTesting)1954 IOException (java.io.IOException)284 ArrayList (java.util.ArrayList)214 Map (java.util.Map)156 HashMap (java.util.HashMap)147 List (java.util.List)113 File (java.io.File)94 ImmutableMap (com.google.common.collect.ImmutableMap)72 HashSet (java.util.HashSet)67 Path (org.apache.hadoop.fs.Path)63 ImmutableList (com.google.common.collect.ImmutableList)60 Path (java.nio.file.Path)60 Set (java.util.Set)52 Matcher (java.util.regex.Matcher)46 Collectors (java.util.stream.Collectors)46 Collection (java.util.Collection)39 Optional (java.util.Optional)38 NotNull (org.jetbrains.annotations.NotNull)37 ImmutableSet (com.google.common.collect.ImmutableSet)34 TreeMap (java.util.TreeMap)34