Search in sources :

Example 1 with InsertedObject

use of com.google.gerrit.server.git.InsertedObject in project gerrit by GerritCodeReview.

the class ReviewDbBatchUpdate method executeNoteDbUpdates.

private void executeNoteDbUpdates(List<ChangeTask> tasks) throws ResourceConflictException, IOException {
    // Aggregate together all NoteDb ref updates from the ops we executed,
    // possibly in parallel. Each task had its own NoteDbUpdateManager instance
    // with its own thread-local copy of the repo(s), but each of those was just
    // used for staging updates and was never executed.
    //
    // Use a new BatchRefUpdate as the original batchRefUpdate field is intended
    // for use only by the updateRepo phase.
    //
    // See the comments in NoteDbUpdateManager#execute() for why we execute the
    // updates on the change repo first.
    logDebug("Executing NoteDb updates for {} changes", tasks.size());
    try {
        initRepository();
        BatchRefUpdate changeRefUpdate = repoView.getRepository().getRefDatabase().newBatchUpdate();
        boolean hasAllUsersCommands = false;
        try (ObjectInserter ins = repoView.getRepository().newObjectInserter()) {
            int objs = 0;
            for (ChangeTask task : tasks) {
                if (task.noteDbResult == null) {
                    logDebug("No-op update to {}", task.id);
                    continue;
                }
                for (ReceiveCommand cmd : task.noteDbResult.changeCommands()) {
                    changeRefUpdate.addCommand(cmd);
                }
                for (InsertedObject obj : task.noteDbResult.changeObjects()) {
                    objs++;
                    ins.insert(obj.type(), obj.data().toByteArray());
                }
                hasAllUsersCommands |= !task.noteDbResult.allUsersCommands().isEmpty();
            }
            logDebug("Collected {} objects and {} ref updates to change repo", objs, changeRefUpdate.getCommands().size());
            executeNoteDbUpdate(getRevWalk(), ins, changeRefUpdate);
        }
        if (hasAllUsersCommands) {
            try (Repository allUsersRepo = repoManager.openRepository(allUsers);
                RevWalk allUsersRw = new RevWalk(allUsersRepo);
                ObjectInserter allUsersIns = allUsersRepo.newObjectInserter()) {
                int objs = 0;
                BatchRefUpdate allUsersRefUpdate = allUsersRepo.getRefDatabase().newBatchUpdate();
                for (ChangeTask task : tasks) {
                    for (ReceiveCommand cmd : task.noteDbResult.allUsersCommands()) {
                        allUsersRefUpdate.addCommand(cmd);
                    }
                    for (InsertedObject obj : task.noteDbResult.allUsersObjects()) {
                        allUsersIns.insert(obj.type(), obj.data().toByteArray());
                    }
                }
                logDebug("Collected {} objects and {} ref updates to All-Users", objs, allUsersRefUpdate.getCommands().size());
                executeNoteDbUpdate(allUsersRw, allUsersIns, allUsersRefUpdate);
            }
        } else {
            logDebug("No All-Users updates");
        }
    } catch (IOException e) {
        if (tasks.stream().allMatch(t -> t.storage == PrimaryStorage.REVIEW_DB)) {
            // Ignore all errors trying to update NoteDb at this point. We've already written the
            // NoteDbChangeStates to ReviewDb, which means if any state is out of date it will be
            // rebuilt the next time it is needed.
            //
            // Always log even without RequestId.
            log.debug("Ignoring NoteDb update error after ReviewDb write", e);
        // Otherwise, we can't prove it's safe to ignore the error, either because some change had
        // NOTE_DB primary, or a task failed before determining the primary storage.
        } else if (e instanceof LockFailureException) {
            // although it happened too late for us to produce anything but a generic error message.
            throw new ResourceConflictException("Updating change failed due to conflicting write", e);
        }
        throw e;
    }
}
Also used : AllUsersName(com.google.gerrit.server.config.AllUsersName) ChangeControl(com.google.gerrit.server.project.ChangeControl) RequestId(com.google.gerrit.server.util.RequestId) OrmException(com.google.gwtorm.server.OrmException) ObjectInserter(org.eclipse.jgit.lib.ObjectInserter) Inject(com.google.inject.Inject) ReceiveCommand(org.eclipse.jgit.transport.ReceiveCommand) LoggerFactory(org.slf4j.LoggerFactory) CheckedFuture(com.google.common.util.concurrent.CheckedFuture) PrimaryStorage(com.google.gerrit.server.notedb.NoteDbChangeState.PrimaryStorage) Assisted(com.google.inject.assistedinject.Assisted) RevWalk(org.eclipse.jgit.revwalk.RevWalk) Config(org.eclipse.jgit.lib.Config) InsertedObject(com.google.gerrit.server.git.InsertedObject) ChangeIndexer(com.google.gerrit.server.index.change.ChangeIndexer) Map(java.util.Map) NoteDbUpdateManager(com.google.gerrit.server.notedb.NoteDbUpdateManager) Timer1(com.google.gerrit.metrics.Timer1) GerritServerConfig(com.google.gerrit.server.config.GerritServerConfig) TimeZone(java.util.TimeZone) Timestamp(java.sql.Timestamp) Collection(java.util.Collection) NullProgressMonitor(org.eclipse.jgit.lib.NullProgressMonitor) Preconditions.checkState(com.google.common.base.Preconditions.checkState) PersonIdent(org.eclipse.jgit.lib.PersonIdent) List(java.util.List) Nullable(com.google.gerrit.common.Nullable) ReviewDbWrapper(com.google.gerrit.reviewdb.server.ReviewDbWrapper) BatchRefUpdate(org.eclipse.jgit.lib.BatchRefUpdate) MetricMaker(com.google.gerrit.metrics.MetricMaker) NotesMigration(com.google.gerrit.server.notedb.NotesMigration) ListeningExecutorService(com.google.common.util.concurrent.ListeningExecutorService) Singleton(com.google.inject.Singleton) MoreExecutors(com.google.common.util.concurrent.MoreExecutors) ReviewDb(com.google.gerrit.reviewdb.server.ReviewDb) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) Change(com.google.gerrit.reviewdb.client.Change) NANOSECONDS(java.util.concurrent.TimeUnit.NANOSECONDS) Stopwatch(com.google.common.base.Stopwatch) MismatchedStateException(com.google.gerrit.server.notedb.NoteDbUpdateManager.MismatchedStateException) Callable(java.util.concurrent.Callable) LockFailureException(com.google.gerrit.server.git.LockFailureException) ArrayList(java.util.ArrayList) ImmutableList(com.google.common.collect.ImmutableList) Description(com.google.gerrit.metrics.Description) Field(com.google.gerrit.metrics.Field) Comparator.comparing(java.util.Comparator.comparing) RestApiException(com.google.gerrit.extensions.restapi.RestApiException) Project(com.google.gerrit.reviewdb.client.Project) CurrentUser(com.google.gerrit.server.CurrentUser) Logger(org.slf4j.Logger) Preconditions.checkNotNull(com.google.common.base.Preconditions.checkNotNull) Units(com.google.gerrit.metrics.Description.Units) Throwables(com.google.common.base.Throwables) ChangeNotes(com.google.gerrit.server.notedb.ChangeNotes) IOException(java.io.IOException) SchemaFactory(com.google.gwtorm.server.SchemaFactory) NoteDbChangeState(com.google.gerrit.server.notedb.NoteDbChangeState) ExecutionException(java.util.concurrent.ExecutionException) Collectors.toList(java.util.stream.Collectors.toList) Futures(com.google.common.util.concurrent.Futures) ChangeUpdate(com.google.gerrit.server.notedb.ChangeUpdate) GitRepositoryManager(com.google.gerrit.server.git.GitRepositoryManager) TreeMap(java.util.TreeMap) ResourceConflictException(com.google.gerrit.extensions.restapi.ResourceConflictException) PatchSet(com.google.gerrit.reviewdb.client.PatchSet) Collections(java.util.Collections) GerritPersonIdent(com.google.gerrit.server.GerritPersonIdent) GitReferenceUpdated(com.google.gerrit.server.extensions.events.GitReferenceUpdated) Repository(org.eclipse.jgit.lib.Repository) ReceiveCommand(org.eclipse.jgit.transport.ReceiveCommand) InsertedObject(com.google.gerrit.server.git.InsertedObject) IOException(java.io.IOException) RevWalk(org.eclipse.jgit.revwalk.RevWalk) LockFailureException(com.google.gerrit.server.git.LockFailureException) Repository(org.eclipse.jgit.lib.Repository) ResourceConflictException(com.google.gerrit.extensions.restapi.ResourceConflictException) ObjectInserter(org.eclipse.jgit.lib.ObjectInserter) BatchRefUpdate(org.eclipse.jgit.lib.BatchRefUpdate)

Example 2 with InsertedObject

use of com.google.gerrit.server.git.InsertedObject in project gerrit by GerritCodeReview.

the class OpenRepo method flushToFinalInserter.

void flushToFinalInserter() throws IOException {
    checkState(finalIns != null);
    for (InsertedObject obj : inMemIns.getInsertedObjects()) {
        finalIns.insert(obj.type(), obj.data().toByteArray());
    }
    inMemIns.clear();
}
Also used : InsertedObject(com.google.gerrit.server.git.InsertedObject)

Example 3 with InsertedObject

use of com.google.gerrit.server.git.InsertedObject in project gerrit by GerritCodeReview.

the class ChangeNotesCommit method newStagedRevWalk.

public static ChangeNotesRevWalk newStagedRevWalk(Repository repo, Iterable<InsertedObject> stagedObjs) {
    final InMemoryInserter ins = new InMemoryInserter(repo);
    for (InsertedObject obj : stagedObjs) {
        ins.insert(obj);
    }
    return new ChangeNotesRevWalk(ins.newReader()) {

        @Override
        public void close() {
            ins.close();
            super.close();
        }
    };
}
Also used : InsertedObject(com.google.gerrit.server.git.InsertedObject) InMemoryInserter(com.google.gerrit.server.git.InMemoryInserter)

Aggregations

InsertedObject (com.google.gerrit.server.git.InsertedObject)3 Preconditions.checkNotNull (com.google.common.base.Preconditions.checkNotNull)1 Preconditions.checkState (com.google.common.base.Preconditions.checkState)1 Stopwatch (com.google.common.base.Stopwatch)1 Throwables (com.google.common.base.Throwables)1 ImmutableList (com.google.common.collect.ImmutableList)1 CheckedFuture (com.google.common.util.concurrent.CheckedFuture)1 Futures (com.google.common.util.concurrent.Futures)1 ListenableFuture (com.google.common.util.concurrent.ListenableFuture)1 ListeningExecutorService (com.google.common.util.concurrent.ListeningExecutorService)1 MoreExecutors (com.google.common.util.concurrent.MoreExecutors)1 Nullable (com.google.gerrit.common.Nullable)1 ResourceConflictException (com.google.gerrit.extensions.restapi.ResourceConflictException)1 RestApiException (com.google.gerrit.extensions.restapi.RestApiException)1 Description (com.google.gerrit.metrics.Description)1 Units (com.google.gerrit.metrics.Description.Units)1 Field (com.google.gerrit.metrics.Field)1 MetricMaker (com.google.gerrit.metrics.MetricMaker)1 Timer1 (com.google.gerrit.metrics.Timer1)1 Change (com.google.gerrit.reviewdb.client.Change)1