Search in sources :

Example 1 with UnitOfWork

use of org.gradle.internal.execution.UnitOfWork in project gradle by gradle.

the class DefaultTransformerInvocationFactory method createInvocation.

@Override
public CacheableInvocation<ImmutableList<File>> createInvocation(Transformer transformer, File inputArtifact, ArtifactTransformDependencies dependencies, TransformationSubject subject, InputFingerprinter inputFingerprinter) {
    ProjectInternal producerProject = determineProducerProject(subject);
    TransformationWorkspaceServices workspaceServices = determineWorkspaceServices(producerProject);
    UnitOfWork execution;
    if (producerProject == null) {
        execution = new ImmutableTransformerExecution(transformer, inputArtifact, dependencies, buildOperationExecutor, fileCollectionFactory, inputFingerprinter, fileSystemAccess, workspaceServices);
    } else {
        execution = new MutableTransformerExecution(transformer, inputArtifact, dependencies, buildOperationExecutor, fileCollectionFactory, inputFingerprinter, workspaceServices);
    }
    return executionEngine.createRequest(execution).withIdentityCache(workspaceServices.getIdentityCache()).getOrDeferExecution(new DeferredExecutionHandler<TransformationResult, CacheableInvocation<ImmutableList<File>>>() {

        @Override
        public CacheableInvocation<ImmutableList<File>> processCachedOutput(Try<TransformationResult> cachedOutput) {
            return CacheableInvocation.cached(mapResult(cachedOutput));
        }

        @Override
        public CacheableInvocation<ImmutableList<File>> processDeferredOutput(Supplier<Try<TransformationResult>> deferredExecution) {
            return CacheableInvocation.nonCached(() -> fireTransformListeners(transformer, subject, () -> mapResult(deferredExecution.get())));
        }

        @Nonnull
        private Try<ImmutableList<File>> mapResult(Try<TransformationResult> cachedOutput) {
            return cachedOutput.map(result -> result.resolveOutputsForInputArtifact(inputArtifact)).mapFailure(failure -> new TransformException(String.format("Execution failed for %s.", execution.getDisplayName()), failure));
        }
    });
}
Also used : InputChangesInternal(org.gradle.internal.execution.history.changes.InputChangesInternal) ProjectComponentIdentifier(org.gradle.api.artifacts.component.ProjectComponentIdentifier) ProjectStateRegistry(org.gradle.api.internal.project.ProjectStateRegistry) CurrentFileCollectionFingerprint(org.gradle.internal.fingerprint.CurrentFileCollectionFingerprint) Hashing(org.gradle.internal.hash.Hashing) Function(java.util.function.Function) Supplier(java.util.function.Supplier) ComponentIdentifier(org.gradle.api.artifacts.component.ComponentIdentifier) INCREMENTAL(org.gradle.internal.execution.fingerprint.InputFingerprinter.InputPropertyType.INCREMENTAL) FileValueSupplier(org.gradle.internal.execution.fingerprint.InputFingerprinter.FileValueSupplier) Provider(org.gradle.api.provider.Provider) ExecutionEngine(org.gradle.internal.execution.ExecutionEngine) ImmutableList(com.google.common.collect.ImmutableList) InputFingerprinter(org.gradle.internal.execution.fingerprint.InputFingerprinter) FileCollectionFactory(org.gradle.api.internal.file.FileCollectionFactory) ProjectInternal(org.gradle.api.internal.project.ProjectInternal) Duration(java.time.Duration) Map(java.util.Map) DeferredExecutionHandler(org.gradle.internal.execution.DeferredExecutionHandler) WorkspaceProvider(org.gradle.internal.execution.workspace.WorkspaceProvider) Providers(org.gradle.api.internal.provider.Providers) DefaultFileNormalizationSpec(org.gradle.internal.execution.fingerprint.impl.DefaultFileNormalizationSpec) Nonnull(javax.annotation.Nonnull) OverlappingOutputs(org.gradle.internal.execution.history.OverlappingOutputs) Nullable(javax.annotation.Nullable) UnitOfWork(org.gradle.internal.execution.UnitOfWork) FileSystemLocation(org.gradle.api.file.FileSystemLocation) OverridingMethodsMustInvokeSuper(javax.annotation.OverridingMethodsMustInvokeSuper) FileSystemLocationSnapshot(org.gradle.internal.snapshot.FileSystemLocationSnapshot) InputVisitor(org.gradle.internal.execution.fingerprint.InputFingerprinter.InputVisitor) CallableBuildOperation(org.gradle.internal.operations.CallableBuildOperation) ValueSnapshot(org.gradle.internal.snapshot.ValueSnapshot) Hasher(org.gradle.internal.hash.Hasher) BuildOperationDescriptor(org.gradle.internal.operations.BuildOperationDescriptor) FileSystemAccess(org.gradle.internal.vfs.FileSystemAccess) FileCollectionFingerprinter(org.gradle.internal.execution.fingerprint.FileCollectionFingerprinter) HashCode(org.gradle.internal.hash.HashCode) File(java.io.File) CachingDisabledReason(org.gradle.internal.execution.caching.CachingDisabledReason) BuildOperationExecutor(org.gradle.internal.operations.BuildOperationExecutor) FILE(org.gradle.internal.file.TreeType.FILE) Try(org.gradle.internal.Try) CachingDisabledReasonCategory(org.gradle.internal.execution.caching.CachingDisabledReasonCategory) DIRECTORY(org.gradle.internal.file.TreeType.DIRECTORY) BuildOperationContext(org.gradle.internal.operations.BuildOperationContext) Optional(java.util.Optional) DefaultFileSystemLocation(org.gradle.api.internal.file.DefaultFileSystemLocation) NON_INCREMENTAL(org.gradle.internal.execution.fingerprint.InputFingerprinter.InputPropertyType.NON_INCREMENTAL) UnitOfWork(org.gradle.internal.execution.UnitOfWork) Nonnull(javax.annotation.Nonnull) ProjectInternal(org.gradle.api.internal.project.ProjectInternal) Try(org.gradle.internal.Try) File(java.io.File)

Example 2 with UnitOfWork

use of org.gradle.internal.execution.UnitOfWork in project gradle by gradle.

the class RemovePreviousOutputsStep method cleanupOverlappingOutputs.

private void cleanupOverlappingOutputs(BeforeExecutionContext context, UnitOfWork work) {
    context.getPreviousExecutionState().ifPresent(previousOutputs -> {
        Set<File> outputDirectoriesToPreserve = new HashSet<>();
        work.visitOutputs(context.getWorkspace(), new UnitOfWork.OutputVisitor() {

            @Override
            public void visitOutputProperty(String propertyName, TreeType type, File root, FileCollection contents) {
                switch(type) {
                    case FILE:
                        File parentFile = root.getParentFile();
                        if (parentFile != null) {
                            outputDirectoriesToPreserve.add(parentFile);
                        }
                        break;
                    case DIRECTORY:
                        outputDirectoriesToPreserve.add(root);
                        break;
                    default:
                        throw new AssertionError();
                }
            }
        });
        OutputsCleaner cleaner = new OutputsCleaner(deleter, file -> true, dir -> !outputDirectoriesToPreserve.contains(dir));
        for (FileSystemSnapshot snapshot : previousOutputs.getOutputFilesProducedByWork().values()) {
            try {
                // Previous outputs can be in a different place than the current outputs
                outputChangeListener.beforeOutputChange(SnapshotUtil.rootIndex(snapshot).keySet());
                cleaner.cleanupOutputs(snapshot);
            } catch (IOException e) {
                throw new UncheckedIOException("Failed to clean up output files for " + work.getDisplayName(), e);
            }
        }
    });
}
Also used : OutputsCleaner(org.gradle.internal.execution.history.OutputsCleaner) UnitOfWork(org.gradle.internal.execution.UnitOfWork) TreeType(org.gradle.internal.file.TreeType) UncheckedIOException(java.io.UncheckedIOException) IOException(java.io.IOException) UncheckedIOException(java.io.UncheckedIOException) FileCollection(org.gradle.api.file.FileCollection) FileSystemSnapshot(org.gradle.internal.snapshot.FileSystemSnapshot) File(java.io.File) HashSet(java.util.HashSet)

Example 3 with UnitOfWork

use of org.gradle.internal.execution.UnitOfWork in project gradle by gradle.

the class ResolveCachingStateStep method execute.

@Override
public CachingResult execute(UnitOfWork work, C context) {
    CachingState cachingState;
    if (!buildCache.isEnabled() && !buildScansEnabled) {
        cachingState = BUILD_CACHE_DISABLED_STATE;
    } else if (context.getValidationProblems().isPresent()) {
        cachingState = VALIDATION_FAILED_STATE;
    } else {
        cachingState = context.getBeforeExecutionState().map(beforeExecutionState -> calculateCachingState(work, beforeExecutionState)).orElseGet(() -> calculateCachingStateWithNoCapturedInputs(work));
    }
    cachingState.apply(enabled -> logCacheKey(enabled.getKey(), work), disabled -> logDisabledReasons(disabled.getDisabledReasons(), work));
    UpToDateResult result = delegate.execute(work, new CachingContext() {

        @Override
        public CachingState getCachingState() {
            return cachingState;
        }

        @Override
        public Optional<String> getNonIncrementalReason() {
            return context.getNonIncrementalReason();
        }

        @Override
        public WorkValidationContext getValidationContext() {
            return context.getValidationContext();
        }

        @Override
        public ImmutableSortedMap<String, ValueSnapshot> getInputProperties() {
            return context.getInputProperties();
        }

        @Override
        public ImmutableSortedMap<String, CurrentFileCollectionFingerprint> getInputFileProperties() {
            return context.getInputFileProperties();
        }

        @Override
        public UnitOfWork.Identity getIdentity() {
            return context.getIdentity();
        }

        @Override
        public File getWorkspace() {
            return context.getWorkspace();
        }

        @Override
        public Optional<ExecutionHistoryStore> getHistory() {
            return context.getHistory();
        }

        @Override
        public Optional<PreviousExecutionState> getPreviousExecutionState() {
            return context.getPreviousExecutionState();
        }

        @Override
        public Optional<ValidationResult> getValidationProblems() {
            return context.getValidationProblems();
        }

        @Override
        public Optional<BeforeExecutionState> getBeforeExecutionState() {
            return context.getBeforeExecutionState();
        }
    });
    return new CachingResult() {

        @Override
        public CachingState getCachingState() {
            return cachingState;
        }

        @Override
        public ImmutableList<String> getExecutionReasons() {
            return result.getExecutionReasons();
        }

        @Override
        public Optional<AfterExecutionState> getAfterExecutionState() {
            return result.getAfterExecutionState();
        }

        @Override
        public Optional<OriginMetadata> getReusedOutputOriginMetadata() {
            return result.getReusedOutputOriginMetadata();
        }

        @Override
        public Try<ExecutionResult> getExecutionResult() {
            return result.getExecutionResult();
        }

        @Override
        public Duration getDuration() {
            return result.getDuration();
        }
    };
}
Also used : BeforeExecutionState(org.gradle.internal.execution.history.BeforeExecutionState) CurrentFileCollectionFingerprint(org.gradle.internal.fingerprint.CurrentFileCollectionFingerprint) LoggerFactory(org.slf4j.LoggerFactory) AfterExecutionState(org.gradle.internal.execution.history.AfterExecutionState) PreviousExecutionState(org.gradle.internal.execution.history.PreviousExecutionState) Formatter(java.util.Formatter) WorkValidationContext(org.gradle.internal.execution.WorkValidationContext) CachingStateFactory(org.gradle.internal.execution.caching.CachingStateFactory) ExecutionResult(org.gradle.internal.execution.ExecutionResult) ImmutableList(com.google.common.collect.ImmutableList) ExecutionHistoryStore(org.gradle.internal.execution.history.ExecutionHistoryStore) Duration(java.time.Duration) ImmutableSortedMap(com.google.common.collect.ImmutableSortedMap) OverlappingOutputs(org.gradle.internal.execution.history.OverlappingOutputs) UnitOfWork(org.gradle.internal.execution.UnitOfWork) Logger(org.slf4j.Logger) ValueSnapshot(org.gradle.internal.snapshot.ValueSnapshot) File(java.io.File) BuildCacheKey(org.gradle.caching.BuildCacheKey) CachingDisabledReason(org.gradle.internal.execution.caching.CachingDisabledReason) List(java.util.List) BuildCacheController(org.gradle.caching.internal.controller.BuildCacheController) NOPLogger(org.slf4j.helpers.NOPLogger) Try(org.gradle.internal.Try) CachingDisabledReasonCategory(org.gradle.internal.execution.caching.CachingDisabledReasonCategory) Optional(java.util.Optional) DefaultCachingStateFactory(org.gradle.internal.execution.caching.impl.DefaultCachingStateFactory) OriginMetadata(org.gradle.caching.internal.origin.OriginMetadata) CachingState(org.gradle.internal.execution.caching.CachingState) AfterExecutionState(org.gradle.internal.execution.history.AfterExecutionState) Optional(java.util.Optional) ImmutableSortedMap(com.google.common.collect.ImmutableSortedMap) ExecutionResult(org.gradle.internal.execution.ExecutionResult) OriginMetadata(org.gradle.caching.internal.origin.OriginMetadata) File(java.io.File) CachingState(org.gradle.internal.execution.caching.CachingState) WorkValidationContext(org.gradle.internal.execution.WorkValidationContext)

Example 4 with UnitOfWork

use of org.gradle.internal.execution.UnitOfWork in project gradle by gradle.

the class ValidateStep method validateImplementations.

private void validateImplementations(UnitOfWork work, BeforeExecutionState beforeExecutionState, WorkValidationContext validationContext) {
    MutableReference<Class<?>> workClass = MutableReference.empty();
    work.visitImplementations(new UnitOfWork.ImplementationVisitor() {

        @Override
        public void visitImplementation(Class<?> implementation) {
            workClass.set(GeneratedSubclasses.unpack(implementation));
        }

        @Override
        public void visitImplementation(ImplementationSnapshot implementation) {
        }
    });
    // It doesn't matter whether we use cacheable true or false, since none of the warnings depends on the cacheability of the task.
    Class<?> workType = workClass.get();
    TypeValidationContext workValidationContext = validationContext.forType(workType, true);
    validateImplementation(workValidationContext, beforeExecutionState.getImplementation(), "Implementation of ", work);
    beforeExecutionState.getAdditionalImplementations().forEach(additionalImplementation -> validateImplementation(workValidationContext, additionalImplementation, "Additional action of ", work));
    beforeExecutionState.getInputProperties().forEach((propertyName, valueSnapshot) -> {
        if (valueSnapshot instanceof ImplementationSnapshot) {
            ImplementationSnapshot implementationSnapshot = (ImplementationSnapshot) valueSnapshot;
            validateNestedInput(workValidationContext, propertyName, implementationSnapshot);
        }
    });
}
Also used : UnitOfWork(org.gradle.internal.execution.UnitOfWork) ImplementationSnapshot(org.gradle.internal.snapshot.impl.ImplementationSnapshot) TypeValidationContext(org.gradle.internal.reflect.validation.TypeValidationContext)

Example 5 with UnitOfWork

use of org.gradle.internal.execution.UnitOfWork in project gradle by gradle.

the class ValidateStep method execute.

@Override
public R execute(UnitOfWork work, C context) {
    WorkValidationContext validationContext = context.getValidationContext();
    work.validate(validationContext);
    context.getBeforeExecutionState().ifPresent(beforeExecutionState -> validateImplementations(work, beforeExecutionState, validationContext));
    Map<Severity, List<String>> problems = validationContext.getProblems().stream().collect(groupingBy(BaseProblem::getSeverity, mapping(ValidateStep::renderedMessage, toList())));
    ImmutableCollection<String> warnings = ImmutableList.copyOf(problems.getOrDefault(Severity.WARNING, ImmutableList.of()));
    ImmutableCollection<String> errors = ImmutableList.copyOf(problems.getOrDefault(Severity.ERROR, ImmutableList.of()));
    if (!warnings.isEmpty()) {
        warningReporter.recordValidationWarnings(work, warnings);
    }
    if (!errors.isEmpty()) {
        int maxErrCount = Integer.getInteger(MAX_NB_OF_ERRORS, 5);
        ImmutableSortedSet<String> uniqueSortedErrors = ImmutableSortedSet.copyOf(errors);
        throw WorkValidationException.forProblems(uniqueSortedErrors).limitTo(maxErrCount).withSummary(helper -> String.format("%s found with the configuration of %s (%s).", helper.size() == 1 ? "A problem was" : "Some problems were", work.getDisplayName(), describeTypesChecked(validationContext.getValidatedTypes()))).get();
    }
    if (!warnings.isEmpty()) {
        LOGGER.info("Invalidating VFS because {} failed validation", work.getDisplayName());
        virtualFileSystem.invalidateAll();
    }
    return delegate.execute(work, new ValidationFinishedContext() {

        @Override
        public Optional<BeforeExecutionState> getBeforeExecutionState() {
            return context.getBeforeExecutionState();
        }

        @Override
        public Optional<ValidationResult> getValidationProblems() {
            return warnings.isEmpty() ? Optional.empty() : Optional.of(() -> warnings);
        }

        @Override
        public Optional<PreviousExecutionState> getPreviousExecutionState() {
            return context.getPreviousExecutionState();
        }

        @Override
        public File getWorkspace() {
            return context.getWorkspace();
        }

        @Override
        public Optional<ExecutionHistoryStore> getHistory() {
            return context.getHistory();
        }

        @Override
        public ImmutableSortedMap<String, ValueSnapshot> getInputProperties() {
            return context.getInputProperties();
        }

        @Override
        public ImmutableSortedMap<String, CurrentFileCollectionFingerprint> getInputFileProperties() {
            return context.getInputFileProperties();
        }

        @Override
        public UnitOfWork.Identity getIdentity() {
            return context.getIdentity();
        }

        @Override
        public Optional<String> getNonIncrementalReason() {
            return context.getNonIncrementalReason();
        }

        @Override
        public WorkValidationContext getValidationContext() {
            return context.getValidationContext();
        }
    });
}
Also used : BeforeExecutionState(org.gradle.internal.execution.history.BeforeExecutionState) TypeValidationProblemRenderer.convertToSingleLine(org.gradle.internal.reflect.validation.TypeValidationProblemRenderer.convertToSingleLine) CurrentFileCollectionFingerprint(org.gradle.internal.fingerprint.CurrentFileCollectionFingerprint) LoggerFactory(org.slf4j.LoggerFactory) Collectors.groupingBy(java.util.stream.Collectors.groupingBy) ImmutableCollection(com.google.common.collect.ImmutableCollection) Severity(org.gradle.internal.reflect.validation.Severity) PreviousExecutionState(org.gradle.internal.execution.history.PreviousExecutionState) WorkValidationContext(org.gradle.internal.execution.WorkValidationContext) ImmutableList(com.google.common.collect.ImmutableList) ExecutionHistoryStore(org.gradle.internal.execution.history.ExecutionHistoryStore) ModelType(org.gradle.model.internal.type.ModelType) Map(java.util.Map) Collectors.mapping(java.util.stream.Collectors.mapping) BaseProblem(org.gradle.problems.BaseProblem) ImmutableSortedMap(com.google.common.collect.ImmutableSortedMap) UnitOfWork(org.gradle.internal.execution.UnitOfWork) ImmutableSortedSet(com.google.common.collect.ImmutableSortedSet) ValidationProblemBuilder(org.gradle.internal.reflect.validation.ValidationProblemBuilder) GeneratedSubclasses(org.gradle.api.internal.GeneratedSubclasses) TypeValidationProblemRenderer.renderMinimalInformationAbout(org.gradle.internal.reflect.validation.TypeValidationProblemRenderer.renderMinimalInformationAbout) WorkValidationException(org.gradle.internal.execution.WorkValidationException) Logger(org.slf4j.Logger) ImplementationSnapshot(org.gradle.internal.snapshot.impl.ImplementationSnapshot) Collection(java.util.Collection) ValueSnapshot(org.gradle.internal.snapshot.ValueSnapshot) TypeValidationContext(org.gradle.internal.reflect.validation.TypeValidationContext) MutableReference(org.gradle.internal.MutableReference) Collectors(java.util.stream.Collectors) File(java.io.File) VirtualFileSystem(org.gradle.internal.vfs.VirtualFileSystem) List(java.util.List) Collectors.toList(java.util.stream.Collectors.toList) Optional(java.util.Optional) ValidationProblemId(org.gradle.internal.reflect.problems.ValidationProblemId) Optional(java.util.Optional) ImmutableSortedMap(com.google.common.collect.ImmutableSortedMap) Severity(org.gradle.internal.reflect.validation.Severity) CurrentFileCollectionFingerprint(org.gradle.internal.fingerprint.CurrentFileCollectionFingerprint) ImmutableList(com.google.common.collect.ImmutableList) List(java.util.List) Collectors.toList(java.util.stream.Collectors.toList) File(java.io.File) WorkValidationContext(org.gradle.internal.execution.WorkValidationContext)

Aggregations

UnitOfWork (org.gradle.internal.execution.UnitOfWork)7 File (java.io.File)4 ImmutableList (com.google.common.collect.ImmutableList)3 ImmutableSortedMap (com.google.common.collect.ImmutableSortedMap)3 Duration (java.time.Duration)3 Optional (java.util.Optional)3 PreviousExecutionState (org.gradle.internal.execution.history.PreviousExecutionState)3 List (java.util.List)2 Map (java.util.Map)2 Try (org.gradle.internal.Try)2 WorkValidationContext (org.gradle.internal.execution.WorkValidationContext)2 CachingDisabledReason (org.gradle.internal.execution.caching.CachingDisabledReason)2 CachingDisabledReasonCategory (org.gradle.internal.execution.caching.CachingDisabledReasonCategory)2 BeforeExecutionState (org.gradle.internal.execution.history.BeforeExecutionState)2 ExecutionHistoryStore (org.gradle.internal.execution.history.ExecutionHistoryStore)2 CurrentFileCollectionFingerprint (org.gradle.internal.fingerprint.CurrentFileCollectionFingerprint)2 TypeValidationContext (org.gradle.internal.reflect.validation.TypeValidationContext)2 ValueSnapshot (org.gradle.internal.snapshot.ValueSnapshot)2 ImplementationSnapshot (org.gradle.internal.snapshot.impl.ImplementationSnapshot)2 Logger (org.slf4j.Logger)2