Search in sources :

Example 1 with BlazeArtifact

use of com.google.idea.blaze.base.command.buildresult.BlazeArtifact in project intellij by bazelbuild.

the class UnpackedAars method refresh.

private void refresh(BlazeContext context, ProjectViewSet viewSet, BlazeProjectData projectData, RemoteOutputArtifacts previousOutputs, boolean removeMissingFiles) {
    try {
        aarCache.getOrCreateCacheDir();
    } catch (IOException e) {
        logger.warn("Could not create unpacked AAR directory", e);
        return;
    }
    ImmutableMap<String, File> cacheFiles = aarCache.readFileState();
    ImmutableMap<String, AarLibraryContents> projectState = getArtifactsToCache(viewSet, projectData);
    ImmutableMap<String, BlazeArtifact> aarOutputs = projectState.entrySet().stream().collect(toImmutableMap(Map.Entry::getKey, e -> e.getValue().aar()));
    try {
        Set<String> updatedKeys = FileCacheDiffer.findUpdatedOutputs(aarOutputs, cacheFiles, previousOutputs).keySet();
        Set<BlazeArtifact> artifactsToDownload = new HashSet<>();
        for (String key : updatedKeys) {
            artifactsToDownload.add(projectState.get(key).aar());
            BlazeArtifact jar = projectState.get(key).jar();
            // separately. Only update jar when we decide that aar need to be updated.
            if (jar != null) {
                artifactsToDownload.add(jar);
            }
        }
        // Prefetch all libraries to local before reading and copying content
        ListenableFuture<?> downloadArtifactsFuture = RemoteArtifactPrefetcher.getInstance().downloadArtifacts(/* projectName= */
        project.getName(), /* outputArtifacts= */
        BlazeArtifact.getRemoteArtifacts(artifactsToDownload));
        FutureUtil.waitForFuture(context, downloadArtifactsFuture).timed("FetchAars", EventType.Prefetching).withProgressMessage("Fetching aar files...").run();
        // manually created directory.
        if (removeMissingFiles) {
            Collection<ListenableFuture<?>> removedFiles = aarCache.retainOnly(/* retainedFiles= */
            projectState.keySet());
            Futures.allAsList(removedFiles).get();
            if (!removedFiles.isEmpty()) {
                context.output(PrintOutput.log(String.format("Removed %d AARs", removedFiles.size())));
            }
        }
        // update cache files
        Unpacker.unpack(projectState, updatedKeys, aarCache);
        if (!updatedKeys.isEmpty()) {
            context.output(PrintOutput.log(String.format("Copied %d AARs", updatedKeys.size())));
        }
    } catch (InterruptedException e) {
        context.setCancelled();
        Thread.currentThread().interrupt();
    } catch (ExecutionException e) {
        logger.warn("Unpacked AAR synchronization didn't complete", e);
    } finally {
        // update the in-memory record of which files are cached
        aarCache.readFileState();
    }
}
Also used : BlazeContext(com.google.idea.blaze.base.scope.BlazeContext) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) BlazeLibrary(com.google.idea.blaze.base.model.BlazeLibrary) FileCacheDiffer(com.google.idea.blaze.base.filecache.FileCacheDiffer) HashMap(java.util.HashMap) ProjectViewManager(com.google.idea.blaze.base.projectview.ProjectViewManager) HashSet(java.util.HashSet) FileCache(com.google.idea.blaze.base.filecache.FileCache) BlazeProjectData(com.google.idea.blaze.base.model.BlazeProjectData) BlazeDataStorage(com.google.idea.blaze.base.sync.data.BlazeDataStorage) Map(java.util.Map) AarLibrary(com.google.idea.blaze.android.sync.model.AarLibrary) Project(com.intellij.openapi.project.Project) BlazeBuildOutputs(com.google.idea.blaze.base.sync.aspects.BlazeBuildOutputs) Logger(com.intellij.openapi.diagnostic.Logger) Nullable(javax.annotation.Nullable) RemoteOutputArtifacts(com.google.idea.blaze.base.model.RemoteOutputArtifacts) ImmutableMap(com.google.common.collect.ImmutableMap) BlazeLibraryCollector(com.google.idea.blaze.base.sync.libraries.BlazeLibraryCollector) Collection(java.util.Collection) FutureUtil(com.google.idea.blaze.base.async.FutureUtil) BlazeImportSettingsManager(com.google.idea.blaze.base.settings.BlazeImportSettingsManager) Set(java.util.Set) IOException(java.io.IOException) RemoteArtifactPrefetcher(com.google.idea.blaze.base.prefetch.RemoteArtifactPrefetcher) PrintOutput(com.google.idea.blaze.base.scope.output.PrintOutput) Collectors(java.util.stream.Collectors) BlazeProjectDataManager(com.google.idea.blaze.base.sync.data.BlazeProjectDataManager) File(java.io.File) BlazeArtifact(com.google.idea.blaze.base.command.buildresult.BlazeArtifact) BlazeImportSettings(com.google.idea.blaze.base.settings.BlazeImportSettings) ExecutionException(java.util.concurrent.ExecutionException) ImmutableMap.toImmutableMap(com.google.common.collect.ImmutableMap.toImmutableMap) Futures(com.google.common.util.concurrent.Futures) List(java.util.List) ServiceManager(com.intellij.openapi.components.ServiceManager) RemoteOutputArtifact(com.google.idea.blaze.base.command.buildresult.RemoteOutputArtifact) ProjectViewSet(com.google.idea.blaze.base.projectview.ProjectViewSet) SyncMode(com.google.idea.blaze.base.sync.SyncMode) LocalFileArtifact(com.google.idea.blaze.base.command.buildresult.BlazeArtifact.LocalFileArtifact) VisibleForTesting(com.google.common.annotations.VisibleForTesting) ArtifactLocationDecoder(com.google.idea.blaze.base.sync.workspace.ArtifactLocationDecoder) EventType(com.google.idea.blaze.base.scope.scopes.TimingScope.EventType) BlazeArtifact(com.google.idea.blaze.base.command.buildresult.BlazeArtifact) IOException(java.io.IOException) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) ExecutionException(java.util.concurrent.ExecutionException) File(java.io.File) HashMap(java.util.HashMap) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) ImmutableMap.toImmutableMap(com.google.common.collect.ImmutableMap.toImmutableMap) HashSet(java.util.HashSet)

Example 2 with BlazeArtifact

use of com.google.idea.blaze.base.command.buildresult.BlazeArtifact in project intellij by bazelbuild.

the class UnpackedAarsTest method initTest.

@Override
protected void initTest(Container applicationServices, Container projectServices) {
    writingOutputSink = new WritingOutputSink();
    context = new BlazeContext();
    context.addOutputSink(PrintOutput.class, writingOutputSink);
    workspaceRoot = new WorkspaceRoot(folder.getRoot());
    localArtifactLocationDecoder = new MockArtifactLocationDecoder() {

        @Override
        public File decode(ArtifactLocation artifactLocation) {
            return new File(workspaceRoot.directory(), artifactLocation.getRelativePath());
        }
    };
    remoteArtifactLocationDecoder = new MockArtifactLocationDecoder() {

        @Override
        public File decode(ArtifactLocation artifactLocation) {
            return new File(workspaceRoot.directory(), artifactLocation.getRelativePath());
        }

        @Override
        public BlazeArtifact resolveOutput(ArtifactLocation artifact) {
            if (!artifact.isSource()) {
                File file = new File(workspaceRoot.directory(), artifact.getRelativePath());
                // when the remote artifact cannot be resolved, it will guess it as local artifact.
                return file.exists() ? new FakeRemoteOutputArtifact(file) : super.resolveOutput(artifact);
            }
            return super.resolveOutput(artifact);
        }
    };
    projectServices.register(BlazeImportSettingsManager.class, new BlazeImportSettingsManager(project));
    try {
        File projectDataDirectory = folder.newFolder("projectdata");
        BlazeImportSettings dummyImportSettings = new BlazeImportSettings("", "", projectDataDirectory.getAbsolutePath(), "", BuildSystem.Bazel);
        BlazeImportSettingsManager.getInstance(project).setImportSettings(dummyImportSettings);
    } catch (IOException e) {
        throw new AssertionError("Fail to create directory for test", e);
    }
    applicationServices.register(FileOperationProvider.class, new FileOperationProvider());
    applicationServices.register(RemoteArtifactPrefetcher.class, new DefaultPrefetcher());
    projectServices.register(UnpackedAars.class, new UnpackedAars(project));
    registerExtensionPoint(FileCache.EP_NAME, FileCache.class).registerExtension(new FileCacheAdapter());
    registerExtensionPoint(BlazeSyncPlugin.EP_NAME, BlazeSyncPlugin.class).registerExtension(new BlazeAndroidSyncPlugin());
    registerExtensionPoint(BlazeLibrarySorter.EP_NAME, BlazeLibrarySorter.class);
    applicationServices.register(ExperimentService.class, new MockExperimentService());
}
Also used : BlazeImportSettingsManager(com.google.idea.blaze.base.settings.BlazeImportSettingsManager) MockExperimentService(com.google.idea.common.experiments.MockExperimentService) BlazeImportSettings(com.google.idea.blaze.base.settings.BlazeImportSettings) BlazeAndroidSyncPlugin(com.google.idea.blaze.android.sync.BlazeAndroidSyncPlugin) BlazeArtifact(com.google.idea.blaze.base.command.buildresult.BlazeArtifact) MockArtifactLocationDecoder(com.google.idea.blaze.base.sync.workspace.MockArtifactLocationDecoder) IOException(java.io.IOException) WorkspaceRoot(com.google.idea.blaze.base.model.primitives.WorkspaceRoot) FileCache(com.google.idea.blaze.base.filecache.FileCache) FileCacheAdapter(com.google.idea.blaze.android.libraries.UnpackedAars.FileCacheAdapter) BlazeContext(com.google.idea.blaze.base.scope.BlazeContext) FileOperationProvider(com.google.idea.blaze.base.io.FileOperationProvider) ArtifactLocation(com.google.idea.blaze.base.ideinfo.ArtifactLocation) BlazeSyncPlugin(com.google.idea.blaze.base.sync.BlazeSyncPlugin) DefaultPrefetcher(com.google.idea.blaze.base.prefetch.DefaultPrefetcher) File(java.io.File)

Example 3 with BlazeArtifact

use of com.google.idea.blaze.base.command.buildresult.BlazeArtifact in project intellij by bazelbuild.

the class RenderJarCacheTest method setupProjectData.

/**
 * Sets up a mock {@link com.google.devtools.intellij.model.ProjectData} and creates the render
 * JARs in File System
 */
private void setupProjectData() throws IOException {
    TargetMap targetMap = TargetMapBuilder.builder().addTarget(TargetIdeInfo.builder().setLabel("//com/foo/bar/baz:baz").setAndroidInfo(AndroidIdeInfo.builder().setRenderResolveJar(getArtifactLocation("com/foo/bar/baz/baz_render_jar.jar"))).setKind(RuleTypes.KT_ANDROID_LIBRARY_HELPER.getKind())).addTarget(TargetIdeInfo.builder().setLabel("//com/foo/bar/qux:qux").setAndroidInfo(AndroidIdeInfo.builder().setRenderResolveJar(getArtifactLocation("com/foo/bar/qux/qux_render_jar.jar"))).setKind(RuleTypes.KT_ANDROID_LIBRARY_HELPER.getKind()).build()).build();
    intellijRule.registerProjectService(BlazeProjectDataManager.class, new MockBlazeProjectDataManager(MockBlazeProjectDataBuilder.builder().setArtifactLocationDecoder(artifactLocationDecoder).setTargetMap(targetMap).build()));
    // Create baz_render_jar.jar in FS
    BlazeArtifact bazRenderJar = artifactLocationDecoder.resolveOutput(getArtifactLocation("com/foo/bar/baz/baz_render_jar.jar"));
    File bazRenderJarFile = ((LocalFileOutputArtifact) bazRenderJar).getFile();
    assertThat(Paths.get(bazRenderJarFile.getParent()).toFile().mkdirs()).isTrue();
    assertThat(bazRenderJarFile.createNewFile()).isTrue();
    assertThat(bazRenderJarFile.setLastModified(100000L)).isTrue();
    // Create qux_render_jar.jar in FS
    BlazeArtifact quxRenderJar = artifactLocationDecoder.resolveOutput(getArtifactLocation("com/foo/bar/qux/qux_render_jar.jar"));
    File quxRenderJarFile = ((LocalFileOutputArtifact) quxRenderJar).getFile();
    assertThat(Paths.get(quxRenderJarFile.getParent()).toFile().mkdirs()).isTrue();
    assertThat(quxRenderJarFile.createNewFile()).isTrue();
    assertThat(quxRenderJarFile.setLastModified(100000L)).isTrue();
}
Also used : LocalFileOutputArtifact(com.google.idea.blaze.base.command.buildresult.LocalFileOutputArtifact) BlazeArtifact(com.google.idea.blaze.base.command.buildresult.BlazeArtifact) File(java.io.File) TargetMap(com.google.idea.blaze.base.ideinfo.TargetMap) MockBlazeProjectDataManager(com.google.idea.blaze.base.model.MockBlazeProjectDataManager)

Example 4 with BlazeArtifact

use of com.google.idea.blaze.base.command.buildresult.BlazeArtifact in project intellij by bazelbuild.

the class FileCacheDiffer method readTimestamps.

private static ImmutableMap<File, Long> readTimestamps(Map<String, ? extends BlazeArtifact> newOutputs, Map<String, File> cachedFiles) throws InterruptedException, ExecutionException {
    boolean timestampsRequired = newOutputs.values().stream().anyMatch(a -> a instanceof LocalFileArtifact);
    if (!timestampsRequired) {
        return ImmutableMap.of();
    }
    Set<File> relevantFiles = new HashSet<>();
    for (Map.Entry<String, ? extends BlazeArtifact> entry : newOutputs.entrySet()) {
        BlazeArtifact newOutput = entry.getValue();
        boolean needsTimestamp = newOutput instanceof LocalFileArtifact;
        if (!needsTimestamp) {
            continue;
        }
        relevantFiles.add(((LocalFileArtifact) newOutput).getFile());
        File cached = cachedFiles.get(entry.getKey());
        if (cached != null) {
            relevantFiles.add(cached);
        }
    }
    return ModifiedTimeScanner.readTimestamps(relevantFiles);
}
Also used : LocalFileArtifact(com.google.idea.blaze.base.command.buildresult.BlazeArtifact.LocalFileArtifact) BlazeArtifact(com.google.idea.blaze.base.command.buildresult.BlazeArtifact) File(java.io.File) ImmutableMap(com.google.common.collect.ImmutableMap) ImmutableMap.toImmutableMap(com.google.common.collect.ImmutableMap.toImmutableMap) Map(java.util.Map) HashSet(java.util.HashSet)

Example 5 with BlazeArtifact

use of com.google.idea.blaze.base.command.buildresult.BlazeArtifact in project intellij by bazelbuild.

the class BlazeIntegrationTestCase method setUp.

@Before
public final void setUp() throws Throwable {
    testFixture = createTestFixture();
    testFixture.setUp();
    fileSystem = new TestFileSystem(getProject(), testFixture.getTempDirFixture(), isLightTestCase());
    runWriteAction(() -> {
        VirtualFile workspaceRootVirtualFile = fileSystem.createDirectory("workspace");
        workspaceRoot = new WorkspaceRoot(new File(workspaceRootVirtualFile.getPath()));
        projectDataDirectory = fileSystem.createDirectory("project-data-dir");
        workspace = new WorkspaceFileSystem(workspaceRoot, fileSystem);
    });
    BlazeImportSettingsManager.getInstance(getProject()).setImportSettings(new BlazeImportSettings(workspaceRoot.toString(), "test-project", projectDataDirectory.getPath(), workspaceRoot.fileForPath(new WorkspacePath("project-view-file")).getPath(), buildSystem()));
    registerApplicationService(InputStreamProvider.class, new InputStreamProvider() {

        @Override
        public InputStream forFile(File file) throws IOException {
            VirtualFile vf = fileSystem.findFile(file.getPath());
            if (vf == null) {
                throw new FileNotFoundException();
            }
            return vf.getInputStream();
        }

        @Override
        public BufferedInputStream forOutputArtifact(BlazeArtifact output) throws IOException {
            if (output instanceof LocalFileArtifact) {
                return new BufferedInputStream(forFile(((LocalFileArtifact) output).getFile()));
            }
            throw new RuntimeException("Can't handle output artifact type: " + output.getClass());
        }
    });
    if (isLightTestCase()) {
        registerApplicationService(FileOperationProvider.class, new TestFileSystem.MockFileOperationProvider());
        registerApplicationService(VirtualFileSystemProvider.class, new TestFileSystem.TempVirtualFileSystemProvider());
    }
    String requiredPlugins = System.getProperty("idea.required.plugins.id");
    if (requiredPlugins != null) {
        VerifyRequiredPluginsEnabled.runCheck(requiredPlugins.split(","));
    }
}
Also used : VirtualFile(com.intellij.openapi.vfs.VirtualFile) WorkspacePath(com.google.idea.blaze.base.model.primitives.WorkspacePath) BlazeImportSettings(com.google.idea.blaze.base.settings.BlazeImportSettings) InputStreamProvider(com.google.idea.blaze.base.io.InputStreamProvider) BufferedInputStream(java.io.BufferedInputStream) InputStream(java.io.InputStream) FileNotFoundException(java.io.FileNotFoundException) BlazeArtifact(com.google.idea.blaze.base.command.buildresult.BlazeArtifact) IOException(java.io.IOException) WorkspaceRoot(com.google.idea.blaze.base.model.primitives.WorkspaceRoot) BufferedInputStream(java.io.BufferedInputStream) LocalFileArtifact(com.google.idea.blaze.base.command.buildresult.BlazeArtifact.LocalFileArtifact) VirtualFile(com.intellij.openapi.vfs.VirtualFile) File(java.io.File) Before(org.junit.Before)

Aggregations

BlazeArtifact (com.google.idea.blaze.base.command.buildresult.BlazeArtifact)17 File (java.io.File)9 Nullable (javax.annotation.Nullable)8 IOException (java.io.IOException)7 Map (java.util.Map)7 ImmutableMap (com.google.common.collect.ImmutableMap)6 LocalFileArtifact (com.google.idea.blaze.base.command.buildresult.BlazeArtifact.LocalFileArtifact)6 ImmutableMap.toImmutableMap (com.google.common.collect.ImmutableMap.toImmutableMap)5 ListenableFuture (com.google.common.util.concurrent.ListenableFuture)5 OutputArtifact (com.google.idea.blaze.base.command.buildresult.OutputArtifact)5 RemoteOutputArtifact (com.google.idea.blaze.base.command.buildresult.RemoteOutputArtifact)5 ArtifactLocation (com.google.idea.blaze.base.ideinfo.ArtifactLocation)5 BlazeContext (com.google.idea.blaze.base.scope.BlazeContext)5 ExecutionException (java.util.concurrent.ExecutionException)5 Futures (com.google.common.util.concurrent.Futures)4 FutureUtil (com.google.idea.blaze.base.async.FutureUtil)4 FileCache (com.google.idea.blaze.base.filecache.FileCache)4 RemoteArtifactPrefetcher (com.google.idea.blaze.base.prefetch.RemoteArtifactPrefetcher)4 PrintOutput (com.google.idea.blaze.base.scope.output.PrintOutput)4 EventType (com.google.idea.blaze.base.scope.scopes.TimingScope.EventType)4