Search in sources :

Example 11 with OutputArtifact

use of com.google.idea.blaze.base.command.buildresult.OutputArtifact in project intellij by bazelbuild.

the class LocalArtifactCacheTest method put_addsArtifactInDirectory.

@Test
public void put_addsArtifactInDirectory() throws IOException {
    // Create blaze artifacts in FS
    ImmutableList<OutputArtifact> outputArtifacts = ImmutableList.of(newLocalOutputArtifact("relative/path_1/artifact_1.jar"), newLocalOutputArtifact("relative/path_2/artifact_2.jar"), newLocalOutputArtifact("relative/path_3/artifact_3.jar"));
    for (OutputArtifact a : outputArtifacts) {
        File file = ((LocalFileOutputArtifact) a).getFile();
        assertThat(Paths.get(file.getParent()).toFile().mkdirs()).isTrue();
        assertThat(file.createNewFile()).isTrue();
    }
    // Put blaze artifacts in cache
    artifactCache.initialize();
    artifactCache.putAll(outputArtifacts, blazeContext, false);
    // Check that the artifacts were added to the cache.
    ImmutableList<File> expectedFiles = Stream.concat(outputArtifacts.stream().map(a -> {
        try {
            return CacheEntry.forArtifact(a);
        } catch (ArtifactNotFoundException e) {
            return null;
        }
    }).filter(Objects::nonNull).map(CacheEntry::getFileName), Stream.of(CACHE_DATA_FILENAME)).map(f -> new File(cacheDirectory.getRoot(), f)).collect(ImmutableList.toImmutableList());
    assertThat(cacheDirectory.getRoot().listFiles()).asList().containsExactlyElementsIn(expectedFiles);
}
Also used : BlazeContext(com.google.idea.blaze.base.scope.BlazeContext) RunWith(org.junit.runner.RunWith) MockBlazeProjectDataManager(com.google.idea.blaze.base.model.MockBlazeProjectDataManager) LocalFileOutputArtifact(com.google.idea.blaze.base.command.buildresult.LocalFileOutputArtifact) ImmutableList(com.google.common.collect.ImmutableList) MockBlazeProjectDataBuilder(com.google.idea.blaze.base.model.MockBlazeProjectDataBuilder) Path(java.nio.file.Path) IntellijRule(com.google.idea.testing.IntellijRule) Before(org.junit.Before) ArtifactLocation(com.google.idea.blaze.base.ideinfo.ArtifactLocation) CACHE_DATA_FILENAME(com.google.idea.blaze.android.filecache.LocalArtifactCache.CACHE_DATA_FILENAME) IOException(java.io.IOException) Test(org.junit.Test) RemoteArtifactPrefetcher(com.google.idea.blaze.base.prefetch.RemoteArtifactPrefetcher) JUnit4(org.junit.runners.JUnit4) Truth.assertThat(com.google.common.truth.Truth.assertThat) FileOperationProvider(com.google.idea.blaze.base.io.FileOperationProvider) BlazeProjectDataManager(com.google.idea.blaze.base.sync.data.BlazeProjectDataManager) File(java.io.File) OutputArtifact(com.google.idea.blaze.base.command.buildresult.OutputArtifact) MockArtifactLocationDecoder(com.google.idea.blaze.base.sync.workspace.MockArtifactLocationDecoder) Objects(java.util.Objects) Stream(java.util.stream.Stream) Rule(org.junit.Rule) Paths(java.nio.file.Paths) WorkspaceRoot(com.google.idea.blaze.base.model.primitives.WorkspaceRoot) DefaultPrefetcher(com.google.idea.blaze.base.prefetch.DefaultPrefetcher) ArtifactLocationDecoder(com.google.idea.blaze.base.sync.workspace.ArtifactLocationDecoder) Collections(java.util.Collections) TemporaryFolder(org.junit.rules.TemporaryFolder) LocalFileOutputArtifact(com.google.idea.blaze.base.command.buildresult.LocalFileOutputArtifact) Objects(java.util.Objects) LocalFileOutputArtifact(com.google.idea.blaze.base.command.buildresult.LocalFileOutputArtifact) OutputArtifact(com.google.idea.blaze.base.command.buildresult.OutputArtifact) File(java.io.File) Test(org.junit.Test)

Example 12 with OutputArtifact

use of com.google.idea.blaze.base.command.buildresult.OutputArtifact in project intellij by bazelbuild.

the class RenderJarCache method getCachedJarForBinaryTarget.

/**
 * Returns the RenderJAR corresponding to {@code target} or null if no RenderJAR corresponding to
 * {@code target} exists in cache.
 */
@Nullable
public File getCachedJarForBinaryTarget(ArtifactLocationDecoder artifactLocationDecoder, TargetIdeInfo target) {
    if (!RenderJarClassFileFinder.isEnabled()) {
        return null;
    }
    AndroidIdeInfo androidIdeInfo = target.getAndroidIdeInfo();
    if (androidIdeInfo == null) {
        return null;
    }
    ArtifactLocation jarArtifactLocation = androidIdeInfo.getRenderResolveJar();
    if (jarArtifactLocation == null) {
        return null;
    }
    BlazeArtifact jarArtifact = artifactLocationDecoder.resolveOutput(jarArtifactLocation);
    if (!(jarArtifact instanceof OutputArtifact)) {
        Logger.getInstance(RenderJarCache.class).warn("Unexpected render jar that is not an OutputArtifact: " + jarArtifactLocation);
        return null;
    }
    Path jarPath = artifactCache.get((OutputArtifact) jarArtifact);
    return jarPath == null ? null : jarPath.toFile();
}
Also used : Path(java.nio.file.Path) ArtifactLocation(com.google.idea.blaze.base.ideinfo.ArtifactLocation) BlazeArtifact(com.google.idea.blaze.base.command.buildresult.BlazeArtifact) AndroidIdeInfo(com.google.idea.blaze.base.ideinfo.AndroidIdeInfo) OutputArtifact(com.google.idea.blaze.base.command.buildresult.OutputArtifact) Nullable(org.jetbrains.annotations.Nullable)

Example 13 with OutputArtifact

use of com.google.idea.blaze.base.command.buildresult.OutputArtifact in project intellij by bazelbuild.

the class JdepsFileReader method doLoadJdepsFiles.

@Nullable
private JdepsState doLoadJdepsFiles(Project project, BlazeContext context, ArtifactLocationDecoder decoder, @Nullable JdepsState oldState, Collection<TargetIdeInfo> targetsToLoad, SyncMode syncMode) throws InterruptedException, ExecutionException {
    Map<OutputArtifact, TargetKey> fileToTargetMap = Maps.newHashMap();
    for (TargetIdeInfo target : targetsToLoad) {
        BlazeArtifact output = resolveJdepsOutput(decoder, target);
        if (output instanceof OutputArtifact) {
            fileToTargetMap.put((OutputArtifact) output, target.getKey());
        }
    }
    ArtifactsDiff diff = ArtifactsDiff.diffArtifacts(oldState != null ? oldState.getArtifactState() : null, fileToTargetMap.keySet());
    // TODO: handle prefetching for arbitrary OutputArtifacts
    List<OutputArtifact> outputArtifacts = diff.getUpdatedOutputs();
    // already. Additional logging to identify what is going wrong.
    if (!outputArtifacts.isEmpty() && !SyncMode.involvesBlazeBuild(syncMode)) {
        logger.warn("ArtifactDiff: " + outputArtifacts.size() + " outputs need to be updated during SyncMode.NO_BUILD ");
        if (oldState == null) {
            logger.warn("ArtifactDiff: oldState == null, we failed to load prior JdepsState.");
        } else {
            // Do not list all artifacts since it may be pretty long.
            if (oldState.getArtifactState().size() != fileToTargetMap.size()) {
                logger.warn("Existing artifact state does not match with target map." + " [oldState.getArtifactState().size() = " + oldState.getArtifactState().size() + ", fileToTargetMap.size() = " + fileToTargetMap.size() + "]");
            }
        }
    }
    ListenableFuture<?> downloadArtifactsFuture = RemoteArtifactPrefetcher.getInstance().downloadArtifacts(/* projectName= */
    project.getName(), /* outputArtifacts= */
    BlazeArtifact.getRemoteArtifacts(outputArtifacts));
    ListenableFuture<?> fetchLocalFilesFuture = PrefetchService.getInstance().prefetchFiles(BlazeArtifact.getLocalFiles(outputArtifacts), true, false);
    if (!FutureUtil.waitForFuture(context, Futures.allAsList(downloadArtifactsFuture, fetchLocalFilesFuture)).timed("FetchJdeps", EventType.Prefetching).withProgressMessage("Reading jdeps files...").run().success()) {
        return null;
    }
    AtomicLong totalSizeLoaded = new AtomicLong(0);
    List<ListenableFuture<Result>> futures = Lists.newArrayList();
    for (OutputArtifact updatedFile : outputArtifacts) {
        futures.add(FetchExecutor.EXECUTOR.submit(() -> {
            totalSizeLoaded.addAndGet(updatedFile.getLength());
            try (InputStream inputStream = updatedFile.getInputStream()) {
                Deps.Dependencies dependencies = Deps.Dependencies.parseFrom(inputStream);
                if (dependencies == null) {
                    return null;
                }
                List<String> deps = dependencies.getDependencyList().stream().filter(dep -> relevantDep(dep)).map(Dependency::getPath).collect(toImmutableList());
                TargetKey targetKey = fileToTargetMap.get(updatedFile);
                return new Result(updatedFile, targetKey, deps);
            } catch (IOException e) {
                logger.info("Could not read jdeps file: " + updatedFile);
                return null;
            }
        }));
    }
    JdepsState.Builder state = JdepsState.builder();
    if (oldState != null) {
        state.list.addAll(oldState.data);
    }
    state.removeArtifacts(diff.getUpdatedOutputs().stream().map(OutputArtifact::toArtifactState).collect(toImmutableList()));
    state.removeArtifacts(diff.getRemovedOutputs());
    for (Result result : Futures.allAsList(futures).get()) {
        if (result != null) {
            state.list.add(JdepsData.create(result.targetKey, result.dependencies, result.output.toArtifactState()));
        }
    }
    context.output(PrintOutput.log(String.format("Loaded %d jdeps files, total size %dkB", diff.getUpdatedOutputs().size(), totalSizeLoaded.get() / 1024)));
    return state.build();
}
Also used : BlazeContext(com.google.idea.blaze.base.scope.BlazeContext) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) PrefetchService(com.google.idea.blaze.base.prefetch.PrefetchService) Dependency(com.google.devtools.build.lib.view.proto.Deps.Dependency) JavaIdeInfo(com.google.idea.blaze.base.ideinfo.JavaIdeInfo) ArtifactsDiff(com.google.idea.blaze.base.filecache.ArtifactsDiff) ProcessCanceledException(com.intellij.openapi.progress.ProcessCanceledException) Lists(com.google.common.collect.Lists) Scope(com.google.idea.blaze.base.scope.Scope) JdepsData(com.google.idea.blaze.java.sync.jdeps.JdepsState.JdepsData) Map(java.util.Map) TargetIdeInfo(com.google.idea.blaze.base.ideinfo.TargetIdeInfo) Project(com.intellij.openapi.project.Project) Logger(com.intellij.openapi.diagnostic.Logger) Nullable(javax.annotation.Nullable) ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) Collection(java.util.Collection) FutureUtil(com.google.idea.blaze.base.async.FutureUtil) SyncState(com.google.idea.blaze.base.model.SyncState) IOException(java.io.IOException) RemoteArtifactPrefetcher(com.google.idea.blaze.base.prefetch.RemoteArtifactPrefetcher) PrintOutput(com.google.idea.blaze.base.scope.output.PrintOutput) Maps(com.google.common.collect.Maps) BlazeArtifact(com.google.idea.blaze.base.command.buildresult.BlazeArtifact) OutputArtifact(com.google.idea.blaze.base.command.buildresult.OutputArtifact) ExecutionException(java.util.concurrent.ExecutionException) AtomicLong(java.util.concurrent.atomic.AtomicLong) Futures(com.google.common.util.concurrent.Futures) List(java.util.List) Deps(com.google.devtools.build.lib.view.proto.Deps) TimingScope(com.google.idea.blaze.base.scope.scopes.TimingScope) SyncMode(com.google.idea.blaze.base.sync.SyncMode) TargetKey(com.google.idea.blaze.base.ideinfo.TargetKey) FetchExecutor(com.google.idea.blaze.base.prefetch.FetchExecutor) ArtifactLocationDecoder(com.google.idea.blaze.base.sync.workspace.ArtifactLocationDecoder) EventType(com.google.idea.blaze.base.scope.scopes.TimingScope.EventType) InputStream(java.io.InputStream) ArtifactsDiff(com.google.idea.blaze.base.filecache.ArtifactsDiff) InputStream(java.io.InputStream) BlazeArtifact(com.google.idea.blaze.base.command.buildresult.BlazeArtifact) Dependency(com.google.devtools.build.lib.view.proto.Deps.Dependency) IOException(java.io.IOException) TargetIdeInfo(com.google.idea.blaze.base.ideinfo.TargetIdeInfo) AtomicLong(java.util.concurrent.atomic.AtomicLong) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) List(java.util.List) TargetKey(com.google.idea.blaze.base.ideinfo.TargetKey) OutputArtifact(com.google.idea.blaze.base.command.buildresult.OutputArtifact) Nullable(javax.annotation.Nullable)

Aggregations

OutputArtifact (com.google.idea.blaze.base.command.buildresult.OutputArtifact)13 BlazeArtifact (com.google.idea.blaze.base.command.buildresult.BlazeArtifact)6 File (java.io.File)6 ArtifactState (com.google.idea.blaze.base.filecache.ArtifactState)5 ArtifactLocation (com.google.idea.blaze.base.ideinfo.ArtifactLocation)5 ImmutableList (com.google.common.collect.ImmutableList)4 ImmutableMap (com.google.common.collect.ImmutableMap)4 ListenableFuture (com.google.common.util.concurrent.ListenableFuture)4 ArtifactsDiff (com.google.idea.blaze.base.filecache.ArtifactsDiff)4 TargetIdeInfo (com.google.idea.blaze.base.ideinfo.TargetIdeInfo)4 TargetKey (com.google.idea.blaze.base.ideinfo.TargetKey)4 BlazeContext (com.google.idea.blaze.base.scope.BlazeContext)4 Path (java.nio.file.Path)4 HashMap (java.util.HashMap)4 Map (java.util.Map)4 ExecutionException (java.util.concurrent.ExecutionException)4 TimingScope (com.google.idea.blaze.base.scope.scopes.TimingScope)3 Logger (com.intellij.openapi.diagnostic.Logger)3 HashSet (java.util.HashSet)3 ImmutableList.toImmutableList (com.google.common.collect.ImmutableList.toImmutableList)2