Search in sources :

Example 41 with ReceiveCommand

use of org.eclipse.jgit.transport.ReceiveCommand in project gerrit by GerritCodeReview.

the class ReviewDbBatchUpdate method executeNoteDbUpdates.

private void executeNoteDbUpdates(List<ChangeTask> tasks) throws ResourceConflictException, IOException {
    // Aggregate together all NoteDb ref updates from the ops we executed,
    // possibly in parallel. Each task had its own NoteDbUpdateManager instance
    // with its own thread-local copy of the repo(s), but each of those was just
    // used for staging updates and was never executed.
    //
    // Use a new BatchRefUpdate as the original batchRefUpdate field is intended
    // for use only by the updateRepo phase.
    //
    // See the comments in NoteDbUpdateManager#execute() for why we execute the
    // updates on the change repo first.
    logDebug("Executing NoteDb updates for {} changes", tasks.size());
    try {
        initRepository();
        BatchRefUpdate changeRefUpdate = repoView.getRepository().getRefDatabase().newBatchUpdate();
        boolean hasAllUsersCommands = false;
        try (ObjectInserter ins = repoView.getRepository().newObjectInserter()) {
            int objs = 0;
            for (ChangeTask task : tasks) {
                if (task.noteDbResult == null) {
                    logDebug("No-op update to {}", task.id);
                    continue;
                }
                for (ReceiveCommand cmd : task.noteDbResult.changeCommands()) {
                    changeRefUpdate.addCommand(cmd);
                }
                for (InsertedObject obj : task.noteDbResult.changeObjects()) {
                    objs++;
                    ins.insert(obj.type(), obj.data().toByteArray());
                }
                hasAllUsersCommands |= !task.noteDbResult.allUsersCommands().isEmpty();
            }
            logDebug("Collected {} objects and {} ref updates to change repo", objs, changeRefUpdate.getCommands().size());
            executeNoteDbUpdate(getRevWalk(), ins, changeRefUpdate);
        }
        if (hasAllUsersCommands) {
            try (Repository allUsersRepo = repoManager.openRepository(allUsers);
                RevWalk allUsersRw = new RevWalk(allUsersRepo);
                ObjectInserter allUsersIns = allUsersRepo.newObjectInserter()) {
                int objs = 0;
                BatchRefUpdate allUsersRefUpdate = allUsersRepo.getRefDatabase().newBatchUpdate();
                for (ChangeTask task : tasks) {
                    for (ReceiveCommand cmd : task.noteDbResult.allUsersCommands()) {
                        allUsersRefUpdate.addCommand(cmd);
                    }
                    for (InsertedObject obj : task.noteDbResult.allUsersObjects()) {
                        allUsersIns.insert(obj.type(), obj.data().toByteArray());
                    }
                }
                logDebug("Collected {} objects and {} ref updates to All-Users", objs, allUsersRefUpdate.getCommands().size());
                executeNoteDbUpdate(allUsersRw, allUsersIns, allUsersRefUpdate);
            }
        } else {
            logDebug("No All-Users updates");
        }
    } catch (IOException e) {
        if (tasks.stream().allMatch(t -> t.storage == PrimaryStorage.REVIEW_DB)) {
            // Ignore all errors trying to update NoteDb at this point. We've already written the
            // NoteDbChangeStates to ReviewDb, which means if any state is out of date it will be
            // rebuilt the next time it is needed.
            //
            // Always log even without RequestId.
            log.debug("Ignoring NoteDb update error after ReviewDb write", e);
        // Otherwise, we can't prove it's safe to ignore the error, either because some change had
        // NOTE_DB primary, or a task failed before determining the primary storage.
        } else if (e instanceof LockFailureException) {
            // although it happened too late for us to produce anything but a generic error message.
            throw new ResourceConflictException("Updating change failed due to conflicting write", e);
        }
        throw e;
    }
}
Also used : AllUsersName(com.google.gerrit.server.config.AllUsersName) ChangeControl(com.google.gerrit.server.project.ChangeControl) RequestId(com.google.gerrit.server.util.RequestId) OrmException(com.google.gwtorm.server.OrmException) ObjectInserter(org.eclipse.jgit.lib.ObjectInserter) Inject(com.google.inject.Inject) ReceiveCommand(org.eclipse.jgit.transport.ReceiveCommand) LoggerFactory(org.slf4j.LoggerFactory) CheckedFuture(com.google.common.util.concurrent.CheckedFuture) PrimaryStorage(com.google.gerrit.server.notedb.NoteDbChangeState.PrimaryStorage) Assisted(com.google.inject.assistedinject.Assisted) RevWalk(org.eclipse.jgit.revwalk.RevWalk) Config(org.eclipse.jgit.lib.Config) InsertedObject(com.google.gerrit.server.git.InsertedObject) ChangeIndexer(com.google.gerrit.server.index.change.ChangeIndexer) Map(java.util.Map) NoteDbUpdateManager(com.google.gerrit.server.notedb.NoteDbUpdateManager) Timer1(com.google.gerrit.metrics.Timer1) GerritServerConfig(com.google.gerrit.server.config.GerritServerConfig) TimeZone(java.util.TimeZone) Timestamp(java.sql.Timestamp) Collection(java.util.Collection) NullProgressMonitor(org.eclipse.jgit.lib.NullProgressMonitor) Preconditions.checkState(com.google.common.base.Preconditions.checkState) PersonIdent(org.eclipse.jgit.lib.PersonIdent) List(java.util.List) Nullable(com.google.gerrit.common.Nullable) ReviewDbWrapper(com.google.gerrit.reviewdb.server.ReviewDbWrapper) BatchRefUpdate(org.eclipse.jgit.lib.BatchRefUpdate) MetricMaker(com.google.gerrit.metrics.MetricMaker) NotesMigration(com.google.gerrit.server.notedb.NotesMigration) ListeningExecutorService(com.google.common.util.concurrent.ListeningExecutorService) Singleton(com.google.inject.Singleton) MoreExecutors(com.google.common.util.concurrent.MoreExecutors) ReviewDb(com.google.gerrit.reviewdb.server.ReviewDb) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) Change(com.google.gerrit.reviewdb.client.Change) NANOSECONDS(java.util.concurrent.TimeUnit.NANOSECONDS) Stopwatch(com.google.common.base.Stopwatch) MismatchedStateException(com.google.gerrit.server.notedb.NoteDbUpdateManager.MismatchedStateException) Callable(java.util.concurrent.Callable) LockFailureException(com.google.gerrit.server.git.LockFailureException) ArrayList(java.util.ArrayList) ImmutableList(com.google.common.collect.ImmutableList) Description(com.google.gerrit.metrics.Description) Field(com.google.gerrit.metrics.Field) Comparator.comparing(java.util.Comparator.comparing) RestApiException(com.google.gerrit.extensions.restapi.RestApiException) Project(com.google.gerrit.reviewdb.client.Project) CurrentUser(com.google.gerrit.server.CurrentUser) Logger(org.slf4j.Logger) Preconditions.checkNotNull(com.google.common.base.Preconditions.checkNotNull) Units(com.google.gerrit.metrics.Description.Units) Throwables(com.google.common.base.Throwables) ChangeNotes(com.google.gerrit.server.notedb.ChangeNotes) IOException(java.io.IOException) SchemaFactory(com.google.gwtorm.server.SchemaFactory) NoteDbChangeState(com.google.gerrit.server.notedb.NoteDbChangeState) ExecutionException(java.util.concurrent.ExecutionException) Collectors.toList(java.util.stream.Collectors.toList) Futures(com.google.common.util.concurrent.Futures) ChangeUpdate(com.google.gerrit.server.notedb.ChangeUpdate) GitRepositoryManager(com.google.gerrit.server.git.GitRepositoryManager) TreeMap(java.util.TreeMap) ResourceConflictException(com.google.gerrit.extensions.restapi.ResourceConflictException) PatchSet(com.google.gerrit.reviewdb.client.PatchSet) Collections(java.util.Collections) GerritPersonIdent(com.google.gerrit.server.GerritPersonIdent) GitReferenceUpdated(com.google.gerrit.server.extensions.events.GitReferenceUpdated) Repository(org.eclipse.jgit.lib.Repository) ReceiveCommand(org.eclipse.jgit.transport.ReceiveCommand) InsertedObject(com.google.gerrit.server.git.InsertedObject) IOException(java.io.IOException) RevWalk(org.eclipse.jgit.revwalk.RevWalk) LockFailureException(com.google.gerrit.server.git.LockFailureException) Repository(org.eclipse.jgit.lib.Repository) ResourceConflictException(com.google.gerrit.extensions.restapi.ResourceConflictException) ObjectInserter(org.eclipse.jgit.lib.ObjectInserter) BatchRefUpdate(org.eclipse.jgit.lib.BatchRefUpdate)

Example 42 with ReceiveCommand

use of org.eclipse.jgit.transport.ReceiveCommand in project gerrit by GerritCodeReview.

the class ReviewDbBatchUpdate method executeNoteDbUpdate.

private void executeNoteDbUpdate(RevWalk rw, ObjectInserter ins, BatchRefUpdate bru) throws IOException {
    if (bru.getCommands().isEmpty()) {
        logDebug("No commands, skipping flush and ref update");
        return;
    }
    ins.flush();
    bru.setAllowNonFastForwards(true);
    bru.execute(rw, NullProgressMonitor.INSTANCE);
    for (ReceiveCommand cmd : bru.getCommands()) {
        // TODO(dborowitz): LOCK_FAILURE for NoteDb primary should be retried.
        if (cmd.getResult() != ReceiveCommand.Result.OK) {
            throw new IOException("Update failed: " + bru);
        }
    }
}
Also used : ReceiveCommand(org.eclipse.jgit.transport.ReceiveCommand) IOException(java.io.IOException)

Example 43 with ReceiveCommand

use of org.eclipse.jgit.transport.ReceiveCommand in project gerrit by GerritCodeReview.

the class UnfusedNoteDbBatchUpdate method executeRefUpdates.

// TODO(dborowitz): Don't execute non-change ref updates separately when fusing phases.
private void executeRefUpdates(boolean dryrun) throws IOException, RestApiException {
    if (getRefUpdates().isEmpty()) {
        logDebug("No ref updates to execute");
        return;
    }
    // May not be opened if the caller added ref updates but no new objects.
    initRepository();
    batchRefUpdate = repoView.getRepository().getRefDatabase().newBatchUpdate();
    batchRefUpdate.setPushCertificate(pushCert);
    batchRefUpdate.setRefLogMessage(refLogMessage, true);
    batchRefUpdate.setAllowNonFastForwards(true);
    repoView.getCommands().addTo(batchRefUpdate);
    logDebug("Executing batch of {} ref updates", batchRefUpdate.getCommands().size());
    if (dryrun) {
        return;
    }
    // that might have access to unflushed objects.
    try (RevWalk updateRw = new RevWalk(repoView.getRepository())) {
        batchRefUpdate.execute(updateRw, NullProgressMonitor.INSTANCE);
    }
    boolean ok = true;
    for (ReceiveCommand cmd : batchRefUpdate.getCommands()) {
        if (cmd.getResult() != ReceiveCommand.Result.OK) {
            ok = false;
            break;
        }
    }
    if (!ok) {
        throw new RestApiException("BatchRefUpdate failed: " + batchRefUpdate);
    }
}
Also used : ReceiveCommand(org.eclipse.jgit.transport.ReceiveCommand) RevWalk(org.eclipse.jgit.revwalk.RevWalk) RestApiException(com.google.gerrit.extensions.restapi.RestApiException)

Example 44 with ReceiveCommand

use of org.eclipse.jgit.transport.ReceiveCommand in project gerrit by GerritCodeReview.

the class NoteDbUpdateManager method addRewrites.

private static void addRewrites(ListMultimap<String, NoteDbRewriter> rewriters, OpenRepo openRepo) throws OrmException, IOException {
    for (Map.Entry<String, Collection<NoteDbRewriter>> entry : rewriters.asMap().entrySet()) {
        String refName = entry.getKey();
        ObjectId oldTip = openRepo.cmds.get(refName).orElse(ObjectId.zeroId());
        if (oldTip.equals(ObjectId.zeroId())) {
            throw new OrmException(String.format("Ref %s is empty", refName));
        }
        ObjectId currTip = oldTip;
        try {
            for (NoteDbRewriter noteDbRewriter : entry.getValue()) {
                ObjectId nextTip = noteDbRewriter.rewriteCommitHistory(openRepo.rw, openRepo.tempIns, currTip);
                if (nextTip != null) {
                    currTip = nextTip;
                }
            }
        } catch (ConfigInvalidException e) {
            throw new OrmException("Cannot rewrite commit history", e);
        }
        if (!oldTip.equals(currTip)) {
            openRepo.cmds.add(new ReceiveCommand(oldTip, currTip, refName));
        }
    }
}
Also used : ReceiveCommand(org.eclipse.jgit.transport.ReceiveCommand) ConfigInvalidException(org.eclipse.jgit.errors.ConfigInvalidException) ObjectId(org.eclipse.jgit.lib.ObjectId) OrmException(com.google.gwtorm.server.OrmException) Collection(java.util.Collection) Map(java.util.Map) HashMap(java.util.HashMap)

Example 45 with ReceiveCommand

use of org.eclipse.jgit.transport.ReceiveCommand in project gerrit by GerritCodeReview.

the class NoteDbUpdateManager method getDraftIds.

private Table<Change.Id, Account.Id, ObjectId> getDraftIds() {
    Table<Change.Id, Account.Id, ObjectId> draftIds = HashBasedTable.create();
    if (allUsersRepo == null) {
        return draftIds;
    }
    for (ReceiveCommand cmd : allUsersRepo.getCommandsSnapshot()) {
        String r = cmd.getRefName();
        if (r.startsWith(REFS_DRAFT_COMMENTS)) {
            Change.Id changeId = Change.Id.fromRefPart(r.substring(REFS_DRAFT_COMMENTS.length()));
            Account.Id accountId = Account.Id.fromRefSuffix(r);
            checkDraftRef(accountId != null && changeId != null, r);
            draftIds.put(changeId, accountId, cmd.getNewId());
        }
    }
    return draftIds;
}
Also used : ReceiveCommand(org.eclipse.jgit.transport.ReceiveCommand) Account(com.google.gerrit.reviewdb.client.Account) ObjectId(org.eclipse.jgit.lib.ObjectId) ObjectId(org.eclipse.jgit.lib.ObjectId) Change(com.google.gerrit.reviewdb.client.Change)

Aggregations

ReceiveCommand (org.eclipse.jgit.transport.ReceiveCommand)55 IOException (java.io.IOException)18 ArrayList (java.util.ArrayList)14 Test (org.junit.Test)14 RepositoryModel (com.gitblit.models.RepositoryModel)13 ObjectId (org.eclipse.jgit.lib.ObjectId)13 BatchRefUpdate (org.eclipse.jgit.lib.BatchRefUpdate)12 RevWalk (org.eclipse.jgit.revwalk.RevWalk)12 Repository (org.eclipse.jgit.lib.Repository)9 Date (java.util.Date)8 Ref (org.eclipse.jgit.lib.Ref)7 OrmException (com.google.gwtorm.server.OrmException)6 Map (java.util.Map)6 RestApiException (com.google.gerrit.extensions.restapi.RestApiException)5 Change (com.google.gerrit.reviewdb.client.Change)5 PersonIdent (org.eclipse.jgit.lib.PersonIdent)5 TicketModel (com.gitblit.models.TicketModel)4 Change (com.gitblit.models.TicketModel.Change)4 ResourceConflictException (com.google.gerrit.extensions.restapi.ResourceConflictException)4 Account (com.google.gerrit.reviewdb.client.Account)4