Search in sources :

Example 1 with DependencyGraphCombiner

use of com.synopsys.integration.bdio.graph.DependencyGraphCombiner in project hub-detect by blackducksoftware.

the class BdioCodeLocationCreator method transformDetectCodeLocationsIntoBdioCodeLocations.

private List<BdioCodeLocation> transformDetectCodeLocationsIntoBdioCodeLocations(final List<DetectCodeLocation> codeLocations, final String codeLocationName, final boolean combineCodeLocations) {
    final List<BdioCodeLocation> bdioCodeLocations;
    final IntegrationEscapeUtil integrationEscapeUtil = new IntegrationEscapeUtil();
    if (codeLocations.size() > 1) {
        // we must either combine or create a unique name.
        if (combineCodeLocations) {
            final DependencyGraphCombiner combiner = new DependencyGraphCombiner();
            logger.info("Combining duplicate code locations with name: " + codeLocationName);
            final MutableDependencyGraph combinedGraph = new MutableMapDependencyGraph();
            final DetectCodeLocation finalCodeLocation = copyCodeLocation(codeLocations.get(0), combinedGraph);
            codeLocations.stream().filter(codeLocation -> shouldCombine(logger, finalCodeLocation, codeLocation)).forEach(codeLocation -> combiner.addGraphAsChildrenToRoot(combinedGraph, codeLocation.getDependencyGraph()));
            final BdioCodeLocation bdioCodeLocation = new BdioCodeLocation(finalCodeLocation, codeLocationName, createBdioName(codeLocationName, integrationEscapeUtil));
            bdioCodeLocations = Arrays.asList(bdioCodeLocation);
        } else {
            bdioCodeLocations = new ArrayList<>();
            for (int i = 0; i < codeLocations.size(); i++) {
                final DetectCodeLocation codeLocation = codeLocations.get(i);
                final String newCodeLocationName = String.format("%s %s", codeLocationName, Integer.toString(i));
                final BdioCodeLocation bdioCodeLocation = new BdioCodeLocation(codeLocation, newCodeLocationName, createBdioName(newCodeLocationName, integrationEscapeUtil));
                bdioCodeLocations.add(bdioCodeLocation);
            }
        }
    } else if (codeLocations.size() == 1) {
        final DetectCodeLocation codeLocation = codeLocations.get(0);
        final BdioCodeLocation bdioCodeLocation = new BdioCodeLocation(codeLocation, codeLocationName, createBdioName(codeLocationName, integrationEscapeUtil));
        bdioCodeLocations = Arrays.asList(bdioCodeLocation);
    } else {
        logger.error("Created a code location name but no code locations.");
        bdioCodeLocations = new ArrayList<>();
    }
    return bdioCodeLocations;
}
Also used : MutableDependencyGraph(com.synopsys.integration.bdio.graph.MutableDependencyGraph) Arrays(java.util.Arrays) DependencyGraph(com.synopsys.integration.bdio.graph.DependencyGraph) MutableDependencyGraph(com.synopsys.integration.bdio.graph.MutableDependencyGraph) LoggerFactory(org.slf4j.LoggerFactory) HashMap(java.util.HashMap) StringUtils(org.apache.commons.lang3.StringUtils) DetectConfiguration(com.blackducksoftware.integration.hub.detect.configuration.DetectConfiguration) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) NameVersion(com.synopsys.integration.util.NameVersion) Event(com.blackducksoftware.integration.hub.detect.workflow.event.Event) PropertyAuthority(com.blackducksoftware.integration.hub.detect.configuration.PropertyAuthority) Map(java.util.Map) DirectoryManager(com.blackducksoftware.integration.hub.detect.workflow.file.DirectoryManager) MutableMapDependencyGraph(com.synopsys.integration.bdio.graph.MutableMapDependencyGraph) IntegrationEscapeUtil(com.synopsys.integration.util.IntegrationEscapeUtil) DetectProperty(com.blackducksoftware.integration.hub.detect.configuration.DetectProperty) DetectorType(com.blackducksoftware.integration.hub.detect.detector.DetectorType) Logger(org.slf4j.Logger) DetectEnumUtil(com.blackducksoftware.integration.hub.detect.util.DetectEnumUtil) Set(java.util.Set) EventSystem(com.blackducksoftware.integration.hub.detect.workflow.event.EventSystem) Collectors(java.util.stream.Collectors) File(java.io.File) DependencyGraphCombiner(com.synopsys.integration.bdio.graph.DependencyGraphCombiner) List(java.util.List) Optional(java.util.Optional) IntegrationEscapeUtil(com.synopsys.integration.util.IntegrationEscapeUtil) ArrayList(java.util.ArrayList) MutableMapDependencyGraph(com.synopsys.integration.bdio.graph.MutableMapDependencyGraph) DependencyGraphCombiner(com.synopsys.integration.bdio.graph.DependencyGraphCombiner)

Example 2 with DependencyGraphCombiner

use of com.synopsys.integration.bdio.graph.DependencyGraphCombiner in project hub-detect by blackducksoftware.

the class NugetInspectorExtractor method extract.

public Extraction extract(final File targetDirectory, File outputDirectory, NugetInspector inspector, final ExtractionId extractionId) {
    try {
        final List<String> options = new ArrayList<>(Arrays.asList("--target_path=" + targetDirectory.toString(), "--output_directory=" + outputDirectory.getCanonicalPath(), "--ignore_failure=" + detectConfiguration.getBooleanProperty(DetectProperty.DETECT_NUGET_IGNORE_FAILURE, PropertyAuthority.None)));
        final String nugetExcludedModules = detectConfiguration.getProperty(DetectProperty.DETECT_NUGET_EXCLUDED_MODULES, PropertyAuthority.None);
        if (StringUtils.isNotBlank(nugetExcludedModules)) {
            options.add("--excluded_modules=" + nugetExcludedModules);
        }
        final String nugetIncludedModules = detectConfiguration.getProperty(DetectProperty.DETECT_NUGET_INCLUDED_MODULES, PropertyAuthority.None);
        if (StringUtils.isNotBlank(nugetIncludedModules)) {
            options.add("--included_modules=" + nugetIncludedModules);
        }
        final String[] nugetPackagesRepo = detectConfiguration.getStringArrayProperty(DetectProperty.DETECT_NUGET_PACKAGES_REPO_URL, PropertyAuthority.None);
        if (nugetPackagesRepo.length > 0) {
            final String packagesRepos = Arrays.asList(nugetPackagesRepo).stream().collect(Collectors.joining(","));
            options.add("--packages_repo_url=" + packagesRepos);
        }
        final String nugetConfigPath = detectConfiguration.getProperty(DetectProperty.DETECT_NUGET_CONFIG_PATH, PropertyAuthority.None);
        if (StringUtils.isNotBlank(nugetConfigPath)) {
            options.add("--nuget_config_path=" + nugetConfigPath);
        }
        if (logger.isTraceEnabled()) {
            options.add("-v");
        }
        final ExecutableOutput executableOutput = inspector.execute(targetDirectory, options);
        if (executableOutput.getReturnCode() != 0) {
            return new Extraction.Builder().failure(String.format("Executing command '%s' returned a non-zero exit code %s", String.join(" ", options), executableOutput.getReturnCode())).build();
        }
        final List<File> dependencyNodeFiles = detectFileFinder.findFiles(outputDirectory, INSPECTOR_OUTPUT_PATTERN);
        final List<NugetParseResult> parseResults = new ArrayList<>();
        for (final File dependencyNodeFile : dependencyNodeFiles) {
            final NugetParseResult result = nugetInspectorPackager.createDetectCodeLocation(dependencyNodeFile);
            parseResults.add(result);
        }
        final List<DetectCodeLocation> codeLocations = parseResults.stream().flatMap(it -> it.codeLocations.stream()).collect(Collectors.toList());
        if (codeLocations.size() <= 0) {
            logger.warn("Unable to extract any dependencies from nuget");
        }
        final Map<String, DetectCodeLocation> codeLocationsBySource = new HashMap<>();
        final DependencyGraphCombiner combiner = new DependencyGraphCombiner();
        codeLocations.stream().forEach(codeLocation -> {
            final String sourcePathKey = codeLocation.getSourcePath().toLowerCase();
            if (codeLocationsBySource.containsKey(sourcePathKey)) {
                logger.info("Multiple project code locations were generated for: " + targetDirectory.toString());
                logger.info("This most likely means the same project exists in multiple solutions.");
                logger.info("The code location's dependencies will be combined, in the future they will exist seperately for each solution.");
                final DetectCodeLocation destination = codeLocationsBySource.get(sourcePathKey);
                combiner.addGraphAsChildrenToRoot((MutableDependencyGraph) destination.getDependencyGraph(), codeLocation.getDependencyGraph());
            } else {
                codeLocationsBySource.put(sourcePathKey, codeLocation);
            }
        });
        final List<DetectCodeLocation> uniqueCodeLocations = codeLocationsBySource.values().stream().collect(Collectors.toList());
        final Extraction.Builder builder = new Extraction.Builder().success(uniqueCodeLocations);
        final Optional<NugetParseResult> project = parseResults.stream().filter(it -> StringUtils.isNotBlank(it.projectName)).findFirst();
        if (project.isPresent()) {
            builder.projectName(project.get().projectName);
            builder.projectVersion(project.get().projectVersion);
        }
        return builder.build();
    } catch (final Exception e) {
        return new Extraction.Builder().exception(e).build();
    }
}
Also used : Arrays(java.util.Arrays) Logger(org.slf4j.Logger) MutableDependencyGraph(com.synopsys.integration.bdio.graph.MutableDependencyGraph) Extraction(com.blackducksoftware.integration.hub.detect.workflow.extraction.Extraction) LoggerFactory(org.slf4j.LoggerFactory) HashMap(java.util.HashMap) Collectors(java.util.stream.Collectors) StringUtils(org.apache.commons.lang3.StringUtils) File(java.io.File) DetectConfiguration(com.blackducksoftware.integration.hub.detect.configuration.DetectConfiguration) ArrayList(java.util.ArrayList) DependencyGraphCombiner(com.synopsys.integration.bdio.graph.DependencyGraphCombiner) List(java.util.List) ExtractionId(com.blackducksoftware.integration.hub.detect.detector.ExtractionId) ExecutableOutput(com.blackducksoftware.integration.hub.detect.util.executable.ExecutableOutput) PropertyAuthority(com.blackducksoftware.integration.hub.detect.configuration.PropertyAuthority) Map(java.util.Map) NugetInspector(com.blackducksoftware.integration.hub.detect.detector.nuget.inspector.NugetInspector) Optional(java.util.Optional) DetectFileFinder(com.blackducksoftware.integration.hub.detect.workflow.file.DetectFileFinder) DetectCodeLocation(com.blackducksoftware.integration.hub.detect.workflow.codelocation.DetectCodeLocation) DetectProperty(com.blackducksoftware.integration.hub.detect.configuration.DetectProperty) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) ExecutableOutput(com.blackducksoftware.integration.hub.detect.util.executable.ExecutableOutput) DetectCodeLocation(com.blackducksoftware.integration.hub.detect.workflow.codelocation.DetectCodeLocation) Extraction(com.blackducksoftware.integration.hub.detect.workflow.extraction.Extraction) File(java.io.File) DependencyGraphCombiner(com.synopsys.integration.bdio.graph.DependencyGraphCombiner)

Example 3 with DependencyGraphCombiner

use of com.synopsys.integration.bdio.graph.DependencyGraphCombiner in project hub-detect by blackducksoftware.

the class SbtModuleAggregator method aggregateModules.

public List<SbtDependencyModule> aggregateModules(final List<SbtDependencyModule> modules) {
    final Set<SbtAggregate> aggregates = uniqueAggregates(modules);
    logger.debug("Found unique aggregates: " + aggregates.size());
    return aggregates.stream().map(aggregate -> {
        final SbtDependencyModule aggregated = new SbtDependencyModule();
        aggregated.name = aggregate.name;
        aggregated.version = aggregate.version;
        aggregated.org = aggregate.org;
        final MutableDependencyGraph graph = new MutableMapDependencyGraph();
        aggregated.graph = graph;
        final DependencyGraphCombiner combiner = new DependencyGraphCombiner();
        modules.forEach(module -> {
            if (moduleEqualsAggregate(module, aggregate)) {
                logger.debug("Combining '" + module.name + "' with '" + aggregate.name + "'");
                combiner.addGraphAsChildrenToRoot(graph, module.graph);
            }
        });
        return aggregated;
    }).collect(Collectors.toList());
}
Also used : List(java.util.List) Logger(org.slf4j.Logger) MutableDependencyGraph(com.synopsys.integration.bdio.graph.MutableDependencyGraph) LoggerFactory(org.slf4j.LoggerFactory) Set(java.util.Set) MutableMapDependencyGraph(com.synopsys.integration.bdio.graph.MutableMapDependencyGraph) Collectors(java.util.stream.Collectors) DependencyGraphCombiner(com.synopsys.integration.bdio.graph.DependencyGraphCombiner) MutableDependencyGraph(com.synopsys.integration.bdio.graph.MutableDependencyGraph) MutableMapDependencyGraph(com.synopsys.integration.bdio.graph.MutableMapDependencyGraph) DependencyGraphCombiner(com.synopsys.integration.bdio.graph.DependencyGraphCombiner)

Aggregations

DependencyGraphCombiner (com.synopsys.integration.bdio.graph.DependencyGraphCombiner)3 MutableDependencyGraph (com.synopsys.integration.bdio.graph.MutableDependencyGraph)3 List (java.util.List)3 Collectors (java.util.stream.Collectors)3 Logger (org.slf4j.Logger)3 LoggerFactory (org.slf4j.LoggerFactory)3 DetectConfiguration (com.blackducksoftware.integration.hub.detect.configuration.DetectConfiguration)2 DetectProperty (com.blackducksoftware.integration.hub.detect.configuration.DetectProperty)2 PropertyAuthority (com.blackducksoftware.integration.hub.detect.configuration.PropertyAuthority)2 MutableMapDependencyGraph (com.synopsys.integration.bdio.graph.MutableMapDependencyGraph)2 File (java.io.File)2 ArrayList (java.util.ArrayList)2 Arrays (java.util.Arrays)2 HashMap (java.util.HashMap)2 Map (java.util.Map)2 Optional (java.util.Optional)2 Set (java.util.Set)2 StringUtils (org.apache.commons.lang3.StringUtils)2 DetectorType (com.blackducksoftware.integration.hub.detect.detector.DetectorType)1 ExtractionId (com.blackducksoftware.integration.hub.detect.detector.ExtractionId)1