Search in sources :

Example 36 with DestinationEffect

use of com.google.copybara.effect.DestinationEffect in project copybara by google.

the class Workflow method run.

@Override
public void run(Path workdir, ImmutableList<String> sourceRefs) throws RepoException, IOException, ValidationException {
    if (sourceRefs.size() > 1) {
        throw new CommandLineException(String.format("Workflow does not support multiple source_ref arguments yet: %s", ImmutableList.copyOf(sourceRefs)));
    }
    @Nullable String sourceRef = sourceRefs.size() == 1 ? sourceRefs.get(0) : null;
    validateFlags();
    try (ProfilerTask ignore = profiler().start("run/" + name)) {
        console.progress("Getting last revision: " + "Resolving " + ((sourceRef == null) ? "origin reference" : sourceRef));
        O resolvedRef = generalOptions.repoTask("origin.resolve_source_ref", () -> origin.resolve(sourceRef));
        logger.log(Level.INFO, String.format("Running Copybara for workflow '%s' and ref '%s': %s", name, resolvedRef.asString(), this.toString()));
        logger.log(Level.INFO, String.format("Using working directory : %s", workdir));
        ImmutableList.Builder<DestinationEffect> allEffects = ImmutableList.builder();
        WorkflowRunHelper<O, D> helper = newRunHelper(workdir, resolvedRef, sourceRef, event -> {
            allEffects.addAll(event.getDestinationEffects());
            eventMonitors().dispatchEvent(m -> m.onChangeMigrationFinished(event));
        });
        try (ProfilerTask ignored = profiler().start(mode.toString().toLowerCase())) {
            mode.run(helper);
        } finally {
            if (!getGeneralOptions().dryRunMode) {
                try (ProfilerTask ignored = profiler().start("after_all_migration")) {
                    ImmutableList<DestinationEffect> effects = allEffects.build();
                    ImmutableList<DestinationEffect> resultEffects = runHooks(effects, getAfterAllMigrationActions(), // Only do this once for all the actions
                    memoized(c -> helper.getOriginReader().getFeedbackEndPoint(c)), // Only do this once for all the actions
                    memoized(c -> helper.getDestinationWriter().getFeedbackEndPoint(c)), resolvedRef);
                    if (effects.size() != resultEffects.size()) {
                        console.warn("Effects where created in after_all_migrations, but they are ignored.");
                    }
                }
            }
        }
    }
}
Also used : FinishHookContext(com.google.copybara.feedback.FinishHookContext) Path(java.nio.file.Path) Profiler(com.google.copybara.profiler.Profiler) ImmutableSetMultimap(com.google.common.collect.ImmutableSetMultimap) CHANGE_REQUEST_FROM_SOT(com.google.copybara.WorkflowMode.CHANGE_REQUEST_FROM_SOT) Reader(com.google.copybara.Origin.Reader) ProfilerTask(com.google.copybara.profiler.Profiler.ProfilerTask) ImmutableMap(com.google.common.collect.ImmutableMap) Migration(com.google.copybara.config.Migration) SkylarkConsole(com.google.copybara.transform.SkylarkConsole) Set(java.util.Set) EventMonitors(com.google.copybara.monitor.EventMonitor.EventMonitors) DestinationStatus(com.google.copybara.Destination.DestinationStatus) Logger(java.util.logging.Logger) Collectors(java.util.stream.Collectors) CHANGE_REQUEST(com.google.copybara.WorkflowMode.CHANGE_REQUEST) List(java.util.List) CommandLineException(com.google.copybara.exception.CommandLineException) Writer(com.google.copybara.Destination.Writer) ConfigFile(com.google.copybara.config.ConfigFile) MigrationReference(com.google.copybara.Info.MigrationReference) ValidationException.checkCondition(com.google.copybara.exception.ValidationException.checkCondition) DestinationEffect(com.google.copybara.effect.DestinationEffect) RepoException(com.google.copybara.exception.RepoException) Callable(java.util.concurrent.Callable) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) Level(java.util.logging.Level) ChangesResponse(com.google.copybara.Origin.Reader.ChangesResponse) TokenType(com.google.copybara.templatetoken.Token.TokenType) ImmutableList(com.google.common.collect.ImmutableList) Revision(com.google.copybara.revision.Revision) Nullable(javax.annotation.Nullable) Identity(com.google.copybara.util.Identity) Token(com.google.copybara.templatetoken.Token) Action(com.google.copybara.action.Action) MoreObjects(com.google.common.base.MoreObjects) ValidationException(com.google.copybara.exception.ValidationException) LazyResourceLoader.memoized(com.google.copybara.LazyResourceLoader.memoized) Console(com.google.copybara.util.console.Console) IOException(java.io.IOException) Consumer(java.util.function.Consumer) Authoring(com.google.copybara.authoring.Authoring) Glob(com.google.copybara.util.Glob) ChangeMigrationFinishedEvent(com.google.copybara.monitor.EventMonitor.ChangeMigrationFinishedEvent) Change(com.google.copybara.revision.Change) Paths(java.nio.file.Paths) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) ProfilerTask(com.google.copybara.profiler.Profiler.ProfilerTask) DestinationEffect(com.google.copybara.effect.DestinationEffect) ImmutableList(com.google.common.collect.ImmutableList) CommandLineException(com.google.copybara.exception.CommandLineException) Nullable(javax.annotation.Nullable)

Example 37 with DestinationEffect

use of com.google.copybara.effect.DestinationEffect in project copybara by google.

the class Workflow method runHooks.

ImmutableList<DestinationEffect> runHooks(ImmutableList<DestinationEffect> effects, ImmutableList<Action> actions, LazyResourceLoader<Endpoint> originEndpoint, LazyResourceLoader<Endpoint> destinationEndpoint, Revision resolvedRef) throws ValidationException, RepoException {
    SkylarkConsole console = new SkylarkConsole(getConsole());
    List<DestinationEffect> hookDestinationEffects = new ArrayList<>();
    for (Action action : actions) {
        try (ProfilerTask ignored2 = profiler().start(action.getName())) {
            logger.log(Level.INFO, "Running after migration hook: " + action.getName());
            FinishHookContext context = new FinishHookContext(action, originEndpoint, destinationEndpoint, ImmutableList.copyOf(effects), generalOptions.labels, resolvedRef, console);
            action.run(context);
            hookDestinationEffects.addAll(context.getNewDestinationEffects());
        }
    }
    return ImmutableList.<DestinationEffect>builder().addAll(effects).addAll(hookDestinationEffects).build();
}
Also used : SkylarkConsole(com.google.copybara.transform.SkylarkConsole) Action(com.google.copybara.action.Action) DestinationEffect(com.google.copybara.effect.DestinationEffect) ProfilerTask(com.google.copybara.profiler.Profiler.ProfilerTask) ArrayList(java.util.ArrayList) FinishHookContext(com.google.copybara.feedback.FinishHookContext)

Example 38 with DestinationEffect

use of com.google.copybara.effect.DestinationEffect in project copybara by google.

the class ActionContext method recordEffect.

@StarlarkMethod(name = "record_effect", doc = "Records an effect of the current action.", parameters = { @Param(name = "summary", doc = "The summary of this effect", named = true), @Param(name = "origin_refs", allowedTypes = { @ParamType(type = Sequence.class, generic1 = OriginRef.class) }, doc = "The origin refs", named = true), @Param(name = "destination_ref", doc = "The destination ref", named = true), @Param(name = "errors", allowedTypes = { @ParamType(type = Sequence.class, generic1 = String.class) }, defaultValue = "[]", doc = "An optional list of errors", named = true), @Param(name = "type", doc = "The type of migration effect:<br>" + "<ul>" + "<li><b>'CREATED'</b>: A new review or change was created.</li>" + "<li><b>'UPDATED'</b>: An existing review or change was updated.</li>" + "<li><b>'NOOP'</b>: The change was a noop.</li>" + "<li><b>'NOOP_AGAINST_PENDING_CHANGE'</b>: The change was a noop, relative" + "to an existing pending change.</li>" + "<li><b>'INSUFFICIENT_APPROVALS'</b>: The effect couldn't happen because " + "the change doesn't have enough approvals.</li>" + "<li><b>'ERROR'</b>: A user attributable error happened that prevented " + "the destination from creating/updating the change. " + "<li><b>'STARTED'</b>: The initial effect of a migration that depends on a " + "previous one. This allows to have 'dependant' migrations defined by users.\n" + "An example of this: a workflow migrates code from a Gerrit review to a " + "GitHub PR, and a feedback migration migrates the test results from a CI in " + "GitHub back to the Gerrit change.\n" + "This effect would be created on the former one.</li>" + "</ul>", defaultValue = "\"UPDATED\"", named = true) })
public void recordEffect(String summary, // <OriginRef>
Sequence<?> originRefs, DestinationRef destinationRef, // <String>
Sequence<?> errors, String typeStr) throws EvalException {
    DestinationEffect.Type type = SkylarkUtil.stringToEnum("type", typeStr, DestinationEffect.Type.class);
    newDestinationEffects.add(new DestinationEffect(type, summary, Sequence.cast(originRefs, OriginRef.class, "origin_refs"), destinationRef, Sequence.cast(errors, String.class, "errors")));
}
Also used : DestinationEffect(com.google.copybara.effect.DestinationEffect) StarlarkMethod(net.starlark.java.annot.StarlarkMethod)

Example 39 with DestinationEffect

use of com.google.copybara.effect.DestinationEffect in project copybara by google.

the class Feedback method run.

@Override
public void run(Path workdir, ImmutableList<String> sourceRefs) throws RepoException, ValidationException {
    ImmutableList.Builder<ActionResult> allResultsBuilder = ImmutableList.builder();
    String suffix = Joiner.on('_').join(sourceRefs).replaceAll("([/ ])", "_");
    String root = "run/" + name + "/" + suffix.substring(0, Math.min(suffix.length(), 20));
    try (ProfilerTask ignore = profiler().start(root)) {
        for (Action action : actions) {
            ArrayList<DestinationEffect> effects = new ArrayList<>();
            try (ProfilerTask ignore2 = profiler().start(action.getName())) {
                SkylarkConsole console = new SkylarkConsole(generalOptions.console());
                eventMonitors().dispatchEvent(m -> m.onChangeMigrationStarted(new ChangeMigrationStartedEvent()));
                FeedbackMigrationContext context = new FeedbackMigrationContext(this, action, generalOptions.cliLabels(), sourceRefs, console);
                action.run(context);
                effects.addAll(context.getNewDestinationEffects());
                ActionResult actionResult = context.getActionResult();
                allResultsBuilder.add(actionResult);
                // First error aborts the execution of the other actions
                ValidationException.checkCondition(actionResult.getResult() != Result.ERROR, "Feedback migration '%s' action '%s' returned error: %s. Aborting execution.", name, action.getName(), actionResult.getMsg());
            } finally {
                eventMonitors().dispatchEvent(m -> m.onChangeMigrationFinished(new ChangeMigrationFinishedEvent(ImmutableList.copyOf(effects), getOriginDescription(), getDestinationDescription())));
            }
        }
    }
    ImmutableList<ActionResult> allResults = allResultsBuilder.build();
    // This check also returns true if there are no actions
    if (allResults.stream().allMatch(a -> a.getResult() == Result.NO_OP)) {
        String detailedMessage = allResults.isEmpty() ? "actions field is empty" : allResults.stream().map(ActionResult::getMsg).collect(ImmutableList.toImmutableList()).toString();
        throw new EmptyChangeException(String.format("Feedback migration '%s' was noop. Detailed messages: %s", name, detailedMessage));
    }
}
Also used : Action(com.google.copybara.action.Action) ChangeMigrationFinishedEvent(com.google.copybara.monitor.EventMonitor.ChangeMigrationFinishedEvent) ImmutableList(com.google.common.collect.ImmutableList) ArrayList(java.util.ArrayList) SkylarkConsole(com.google.copybara.transform.SkylarkConsole) ActionResult(com.google.copybara.action.ActionResult) ProfilerTask(com.google.copybara.profiler.Profiler.ProfilerTask) DestinationEffect(com.google.copybara.effect.DestinationEffect) ChangeMigrationStartedEvent(com.google.copybara.monitor.EventMonitor.ChangeMigrationStartedEvent) EmptyChangeException(com.google.copybara.exception.EmptyChangeException)

Aggregations

DestinationEffect (com.google.copybara.effect.DestinationEffect)39 DummyRevision (com.google.copybara.testing.DummyRevision)26 Test (org.junit.Test)22 WriterContext (com.google.copybara.WriterContext)19 Author (com.google.copybara.authoring.Author)13 Glob (com.google.copybara.util.Glob)13 ImmutableList (com.google.common.collect.ImmutableList)11 Changes (com.google.copybara.revision.Changes)10 Path (java.nio.file.Path)10 ValidationException (com.google.copybara.exception.ValidationException)9 RepoException (com.google.copybara.exception.RepoException)8 IOException (java.io.IOException)8 ImmutableSetMultimap (com.google.common.collect.ImmutableSetMultimap)7 ChangeMessage (com.google.copybara.ChangeMessage)7 Writer (com.google.copybara.Destination.Writer)7 Metadata (com.google.copybara.Metadata)7 MigrationInfo (com.google.copybara.MigrationInfo)7 TransformWork (com.google.copybara.TransformWork)7 CheckerException (com.google.copybara.checks.CheckerException)7 LowLevelHttpRequest (com.google.api.client.http.LowLevelHttpRequest)6