Search in sources :

Example 6 with OneToOneResilientIndexingListener

use of org.sonar.server.es.OneToOneResilientIndexingListener in project sonarqube by SonarSource.

the class UserIndexer method index.

/**
 * @return the number of items that have been successfully indexed
 */
@Override
public IndexingResult index(DbSession dbSession, Collection<EsQueueDto> items) {
    if (items.isEmpty()) {
        return new IndexingResult();
    }
    Set<String> uuids = items.stream().map(EsQueueDto::getDocId).collect(toHashSet(items.size()));
    BulkIndexer bulkIndexer = newBulkIndexer(Size.REGULAR, new OneToOneResilientIndexingListener(dbClient, dbSession, items));
    bulkIndexer.start();
    dbClient.userDao().scrollByUuids(dbSession, uuids, // Deactivated users are not deleted but updated.
    u -> {
        uuids.remove(u.getUuid());
        bulkIndexer.add(newIndexRequest(u));
    });
    // the remaining uuids reference rows that don't exist in db. They must
    // be deleted from index.
    uuids.forEach(uuid -> bulkIndexer.addDeletion(TYPE_USER, uuid));
    return bulkIndexer.stop();
}
Also used : IndexingResult(org.sonar.server.es.IndexingResult) BulkIndexer(org.sonar.server.es.BulkIndexer) OneToOneResilientIndexingListener(org.sonar.server.es.OneToOneResilientIndexingListener)

Example 7 with OneToOneResilientIndexingListener

use of org.sonar.server.es.OneToOneResilientIndexingListener in project sonarqube by SonarSource.

the class IssueIndexer method doIndexIssueItems.

private IndexingResult doIndexIssueItems(DbSession dbSession, ListMultimap<String, EsQueueDto> itemsByIssueKey) {
    if (itemsByIssueKey.isEmpty()) {
        return new IndexingResult();
    }
    IndexingListener listener = new OneToOneResilientIndexingListener(dbClient, dbSession, itemsByIssueKey.values());
    BulkIndexer bulkIndexer = createBulkIndexer(Size.REGULAR, listener);
    bulkIndexer.start();
    try (IssueIterator issues = issueIteratorFactory.createForIssueKeys(itemsByIssueKey.keySet())) {
        while (issues.hasNext()) {
            IssueDoc issue = issues.next();
            bulkIndexer.add(newIndexRequest(issue));
            itemsByIssueKey.removeAll(issue.getId());
        }
    }
    // the remaining uuids reference issues that don't exist in db. They must
    // be deleted from index.
    itemsByIssueKey.values().forEach(item -> bulkIndexer.addDeletion(TYPE_ISSUE.getMainType(), item.getDocId(), item.getDocRouting()));
    return bulkIndexer.stop();
}
Also used : IndexingListener(org.sonar.server.es.IndexingListener) OneToOneResilientIndexingListener(org.sonar.server.es.OneToOneResilientIndexingListener) OneToManyResilientIndexingListener(org.sonar.server.es.OneToManyResilientIndexingListener) IndexingResult(org.sonar.server.es.IndexingResult) BulkIndexer(org.sonar.server.es.BulkIndexer) OneToOneResilientIndexingListener(org.sonar.server.es.OneToOneResilientIndexingListener)

Aggregations

BulkIndexer (org.sonar.server.es.BulkIndexer)7 OneToOneResilientIndexingListener (org.sonar.server.es.OneToOneResilientIndexingListener)7 IndexingResult (org.sonar.server.es.IndexingResult)5 EsQueueDto (org.sonar.db.es.EsQueueDto)2 VisibleForTesting (com.google.common.annotations.VisibleForTesting)1 ImmutableSet (com.google.common.collect.ImmutableSet)1 Arrays (java.util.Arrays)1 Collection (java.util.Collection)1 Collections.emptyList (java.util.Collections.emptyList)1 HashMap (java.util.HashMap)1 List (java.util.List)1 Map (java.util.Map)1 Objects (java.util.Objects)1 Set (java.util.Set)1 Collectors (java.util.stream.Collectors)1 Stream (java.util.stream.Stream)1 MoreCollectors (org.sonar.core.util.stream.MoreCollectors)1 MoreCollectors.toArrayList (org.sonar.core.util.stream.MoreCollectors.toArrayList)1 DbClient (org.sonar.db.DbClient)1 DbSession (org.sonar.db.DbSession)1