Search in sources :

Example 1 with Branch

use of com.atlassian.stash.repository.Branch in project stash-codesearch-plugin by palantir.

the class SearchUpdaterImpl method reindexAll.

@Override
public boolean reindexAll() {
    GlobalSettings globalSettings = settingsManager.getGlobalSettings();
    if (!globalSettings.getIndexingEnabled()) {
        log.warn("Not performing a complete reindex since indexing is disabled");
        return false;
    }
    if (!isReindexingAll.compareAndSet(false, true)) {
        log.warn("Not performing a complete reindex since one is already occurring");
        return false;
    }
    try {
        runWithBlockedJobPool(new Runnable() {

            @Override
            public void run() {
                initializeAliasedIndex(ES_UPDATEALIAS, true);
            }
        });
        // Disable refresh for faster bulk indexing
        es.getClient().admin().indices().prepareUpdateSettings(ES_UPDATEALIAS).setSettings(ImmutableSettings.builder().put("index.refresh_interval", "-1")).get();
        // Submit and wait for each job
        List<Future<Void>> futures = new ArrayList<Future<Void>>();
        for (Repository repo : repositoryServiceManager.getRepositoryMap(null).values()) {
            for (Branch branch : repositoryServiceManager.getBranchMap(repo).values()) {
                // No need to explicitly trigger reindex, since index is empty.
                futures.add(submitAsyncUpdate(repo, branch.getId(), 0));
            }
        }
        for (Future<Void> future : futures) {
            waitForFuture(future);
        }
        // Re-enable refresh & optimize, enable searching on index
        es.getClient().admin().indices().prepareUpdateSettings(ES_UPDATEALIAS).setSettings(ImmutableSettings.builder().put("index.refresh_interval", "1s")).get();
        es.getClient().admin().indices().prepareOptimize(ES_UPDATEALIAS).get();
        redirectAndDeleteAliasedIndex(ES_SEARCHALIAS, ES_UPDATEALIAS);
    } finally {
        isReindexingAll.getAndSet(false);
    }
    return true;
}
Also used : Repository(com.atlassian.stash.repository.Repository) Branch(com.atlassian.stash.repository.Branch) ArrayList(java.util.ArrayList) Future(java.util.concurrent.Future) GlobalSettings(com.palantir.stash.codesearch.admin.GlobalSettings)

Example 2 with Branch

use of com.atlassian.stash.repository.Branch in project stash-codesearch-plugin by palantir.

the class SearchUpdaterImpl method reindexRepository.

@Override
public boolean reindexRepository(final String projectKey, final String repositorySlug) {
    GlobalSettings globalSettings = settingsManager.getGlobalSettings();
    if (!globalSettings.getIndexingEnabled()) {
        log.warn("Not performing a repository reindex triggered since indexing is disabled");
        return false;
    }
    log.warn("Manual reindex triggered for {}^{} (expensive operation, use sparingly)", projectKey, repositorySlug);
    // Delete documents for this repository
    log.warn("Deleting {}^{} for manual reindexing", projectKey, repositorySlug);
    for (int i = 0; i < 5; ++i) {
        // since ES doesn't have a refresh setting for delete by
        // query requests, we just spam multiple requests.
        es.getClient().prepareDeleteByQuery(ES_UPDATEALIAS).setRouting(projectKey + '^' + repositorySlug).setQuery(QueryBuilders.filteredQuery(QueryBuilders.matchAllQuery(), sfu.projectRepositoryFilter(projectKey, repositorySlug))).get();
        try {
            Thread.sleep(200);
        } catch (InterruptedException e) {
            log.warn("Caught InterruptedException while trying to sleep", e);
        }
    }
    log.warn("Deletion of {}^{} completed", projectKey, repositorySlug);
    // Search for repository
    Repository repository = repositoryServiceManager.getRepositoryService().getBySlug(projectKey, repositorySlug);
    if (repository == null) {
        log.warn("Repository {}^{} not found for manual reindexing", projectKey, repositorySlug);
        return false;
    }
    // Submit and wait for each job
    List<Future<Void>> futures = new ArrayList<Future<Void>>();
    for (Branch branch : repositoryServiceManager.getBranchMap(repository).values()) {
        // No need to explicitly trigger reindex, since index is empty.
        futures.add(submitAsyncUpdate(repository, branch.getId(), 0));
    }
    for (Future<Void> future : futures) {
        waitForFuture(future);
    }
    log.warn("Manual reindex of {}^{} completed", projectKey, repositorySlug);
    return true;
}
Also used : Repository(com.atlassian.stash.repository.Repository) Branch(com.atlassian.stash.repository.Branch) ArrayList(java.util.ArrayList) Future(java.util.concurrent.Future) GlobalSettings(com.palantir.stash.codesearch.admin.GlobalSettings)

Example 3 with Branch

use of com.atlassian.stash.repository.Branch in project stash-codesearch-plugin by palantir.

the class RepositoryServiceManagerImpl method getBranchMap.

@Override
public ImmutableMap<String, Branch> getBranchMap(Repository repository) {
    PageRequest req = new PageRequestImpl(0, PAGE_SIZE);
    Map<String, Branch> branchMap = new HashMap<String, Branch>();
    RepositoryBranchesRequest rbr = new RepositoryBranchesRequest.Builder().repository(repository).order(RefOrder.ALPHABETICAL).build();
    while (true) {
        Page<? extends Branch> branchPage = repositoryMetadataService.getBranches(rbr, req);
        for (Branch b : branchPage.getValues()) {
            if (branchMap.containsKey(b)) {
                log.error("Tryting to insert existing key '" + b.getId() + "' into branchMap with value '" + b + "'");
                continue;
            }
            branchMap.put(b.getId(), b);
        }
        if (branchPage.getIsLastPage()) {
            break;
        }
        req = branchPage.getNextPageRequest();
    }
    return ImmutableMap.copyOf(branchMap);
}
Also used : PageRequest(com.atlassian.stash.util.PageRequest) HashMap(java.util.HashMap) Branch(com.atlassian.stash.repository.Branch) RepositoryBranchesRequest(com.atlassian.stash.repository.RepositoryBranchesRequest) PageRequestImpl(com.atlassian.stash.util.PageRequestImpl)

Aggregations

Branch (com.atlassian.stash.repository.Branch)3 Repository (com.atlassian.stash.repository.Repository)2 GlobalSettings (com.palantir.stash.codesearch.admin.GlobalSettings)2 ArrayList (java.util.ArrayList)2 Future (java.util.concurrent.Future)2 RepositoryBranchesRequest (com.atlassian.stash.repository.RepositoryBranchesRequest)1 PageRequest (com.atlassian.stash.util.PageRequest)1 PageRequestImpl (com.atlassian.stash.util.PageRequestImpl)1 HashMap (java.util.HashMap)1