Search in sources :

Example 1 with MultiProgressMonitor

use of com.google.gerrit.server.git.MultiProgressMonitor in project gerrit by GerritCodeReview.

the class AllChangesIndexer method indexAll.

public SiteIndexer.Result indexAll(ChangeIndex index, Iterable<ProjectHolder> projects) {
    Stopwatch sw = Stopwatch.createStarted();
    final MultiProgressMonitor mpm = new MultiProgressMonitor(progressOut, "Reindexing changes");
    final Task projTask = mpm.beginSubTask("projects", (projects instanceof Collection) ? ((Collection<?>) projects).size() : MultiProgressMonitor.UNKNOWN);
    final Task doneTask = mpm.beginSubTask(null, totalWork >= 0 ? totalWork : MultiProgressMonitor.UNKNOWN);
    final Task failedTask = mpm.beginSubTask("failed", MultiProgressMonitor.UNKNOWN);
    final List<ListenableFuture<?>> futures = new ArrayList<>();
    final AtomicBoolean ok = new AtomicBoolean(true);
    for (final ProjectHolder project : projects) {
        ListenableFuture<?> future = executor.submit(reindexProject(indexerFactory.create(executor, index), project.name, doneTask, failedTask, verboseWriter));
        addErrorListener(future, "project " + project.name, projTask, ok);
        futures.add(future);
    }
    try {
        mpm.waitFor(transform(successfulAsList(futures), x -> {
            mpm.end();
            return null;
        }, directExecutor()));
    } catch (ExecutionException e) {
        log.error("Error in batch indexer", e);
        ok.set(false);
    }
    // If too many changes failed, maybe there was a bug in the indexer. Don't
    // trust the results. This is not an exact percentage since we bump the same
    // failure counter if a project can't be read, but close enough.
    int nFailed = failedTask.getCount();
    int nDone = doneTask.getCount();
    int nTotal = nFailed + nDone;
    double pctFailed = ((double) nFailed) / nTotal * 100;
    if (pctFailed > 10) {
        log.error("Failed {}/{} changes ({}%); not marking new index as ready", nFailed, nTotal, Math.round(pctFailed));
        ok.set(false);
    }
    return new Result(sw, ok.get(), nDone, nFailed);
}
Also used : SortedSet(java.util.SortedSet) ListMultimap(com.google.common.collect.ListMultimap) ProjectCache(com.google.gerrit.server.project.ProjectCache) Futures.successfulAsList(com.google.common.util.concurrent.Futures.successfulAsList) MultimapBuilder(com.google.common.collect.MultimapBuilder) RepositoryNotFoundException(org.eclipse.jgit.errors.RepositoryNotFoundException) Inject(com.google.inject.Inject) LoggerFactory(org.slf4j.LoggerFactory) RevObject(org.eclipse.jgit.revwalk.RevObject) SiteIndexer(com.google.gerrit.server.index.SiteIndexer) RevWalk(org.eclipse.jgit.revwalk.RevWalk) ALL(org.eclipse.jgit.lib.RefDatabase.ALL) Map(java.util.Map) IndexExecutor(com.google.gerrit.server.index.IndexExecutor) BATCH(com.google.gerrit.server.git.QueueProvider.QueueType.BATCH) PrintWriter(java.io.PrintWriter) Task(com.google.gerrit.server.git.MultiProgressMonitor.Task) Collection(java.util.Collection) TextProgressMonitor(org.eclipse.jgit.lib.TextProgressMonitor) Constants(org.eclipse.jgit.lib.Constants) ComparisonChain(com.google.common.collect.ComparisonChain) MoreExecutors.directExecutor(com.google.common.util.concurrent.MoreExecutors.directExecutor) ChangeData(com.google.gerrit.server.query.change.ChangeData) List(java.util.List) Ref(org.eclipse.jgit.lib.Ref) ListeningExecutorService(com.google.common.util.concurrent.ListeningExecutorService) ReviewDb(com.google.gerrit.reviewdb.server.ReviewDb) RevCommit(org.eclipse.jgit.revwalk.RevCommit) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) Change(com.google.gerrit.reviewdb.client.Change) Stopwatch(com.google.common.base.Stopwatch) MultiProgressMonitor(com.google.gerrit.server.git.MultiProgressMonitor) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) Callable(java.util.concurrent.Callable) TreeSet(java.util.TreeSet) ArrayList(java.util.ArrayList) Lists(com.google.common.collect.Lists) ProgressMonitor(org.eclipse.jgit.lib.ProgressMonitor) Project(com.google.gerrit.reviewdb.client.Project) Logger(org.slf4j.Logger) Iterator(java.util.Iterator) ChangeNotes(com.google.gerrit.server.notedb.ChangeNotes) IOException(java.io.IOException) SchemaFactory(com.google.gwtorm.server.SchemaFactory) ObjectId(org.eclipse.jgit.lib.ObjectId) ExecutionException(java.util.concurrent.ExecutionException) GitRepositoryManager(com.google.gerrit.server.git.GitRepositoryManager) Futures.transform(com.google.common.util.concurrent.Futures.transform) Repository(org.eclipse.jgit.lib.Repository) Task(com.google.gerrit.server.git.MultiProgressMonitor.Task) Stopwatch(com.google.common.base.Stopwatch) ArrayList(java.util.ArrayList) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) MultiProgressMonitor(com.google.gerrit.server.git.MultiProgressMonitor) Collection(java.util.Collection) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) ExecutionException(java.util.concurrent.ExecutionException)

Example 2 with MultiProgressMonitor

use of com.google.gerrit.server.git.MultiProgressMonitor in project gerrit by GerritCodeReview.

the class Index method apply.

@Override
public Response.Accepted apply(ProjectResource resource, ProjectInput input) {
    Project.NameKey project = resource.getNameKey();
    Task mpt = new MultiProgressMonitor(ByteStreams.nullOutputStream(), "Reindexing project").beginSubTask("", MultiProgressMonitor.UNKNOWN);
    PrintWriter pw = new PrintWriter(CharStreams.nullWriter());
    executor.submit(allChangesIndexer.reindexProject(indexer, project, mpt, mpt, pw));
    return Response.accepted("Project " + project + " submitted for reindexing");
}
Also used : Project(com.google.gerrit.reviewdb.client.Project) Task(com.google.gerrit.server.git.MultiProgressMonitor.Task) MultiProgressMonitor(com.google.gerrit.server.git.MultiProgressMonitor) PrintWriter(java.io.PrintWriter)

Aggregations

Project (com.google.gerrit.reviewdb.client.Project)2 MultiProgressMonitor (com.google.gerrit.server.git.MultiProgressMonitor)2 Task (com.google.gerrit.server.git.MultiProgressMonitor.Task)2 PrintWriter (java.io.PrintWriter)2 Stopwatch (com.google.common.base.Stopwatch)1 ComparisonChain (com.google.common.collect.ComparisonChain)1 ListMultimap (com.google.common.collect.ListMultimap)1 Lists (com.google.common.collect.Lists)1 MultimapBuilder (com.google.common.collect.MultimapBuilder)1 Futures.successfulAsList (com.google.common.util.concurrent.Futures.successfulAsList)1 Futures.transform (com.google.common.util.concurrent.Futures.transform)1 ListenableFuture (com.google.common.util.concurrent.ListenableFuture)1 ListeningExecutorService (com.google.common.util.concurrent.ListeningExecutorService)1 MoreExecutors.directExecutor (com.google.common.util.concurrent.MoreExecutors.directExecutor)1 Change (com.google.gerrit.reviewdb.client.Change)1 ReviewDb (com.google.gerrit.reviewdb.server.ReviewDb)1 GitRepositoryManager (com.google.gerrit.server.git.GitRepositoryManager)1 BATCH (com.google.gerrit.server.git.QueueProvider.QueueType.BATCH)1 IndexExecutor (com.google.gerrit.server.index.IndexExecutor)1 SiteIndexer (com.google.gerrit.server.index.SiteIndexer)1