Search in sources :

Example 11 with IndexUpdateFailedException

use of org.apache.archiva.indexer.IndexUpdateFailedException in project archiva by apache.

the class ArchivaIndexManagerMock method reset.

@Override
public ArchivaIndexingContext reset(ArchivaIndexingContext context) throws IndexUpdateFailedException {
    ArchivaIndexingContext ctx;
    executeUpdateFunction(context, indexingContext -> {
        try {
            indexingContext.close(true);
        } catch (IOException e) {
            log.warn("Index close failed");
        }
        try {
            FileUtils.deleteDirectory(Paths.get(context.getPath()));
        } catch (IOException e) {
            throw new IndexUpdateFailedException("Could not delete index files");
        }
    });
    try {
        Repository repo = context.getRepository();
        ctx = createContext(context.getRepository());
        if (repo instanceof EditableRepository) {
            ((EditableRepository) repo).setIndexingContext(ctx);
        }
    } catch (IndexCreationFailedException e) {
        throw new IndexUpdateFailedException("Could not create index");
    }
    return ctx;
}
Also used : ManagedRepository(org.apache.archiva.repository.ManagedRepository) EditableRepository(org.apache.archiva.repository.EditableRepository) Repository(org.apache.archiva.repository.Repository) RemoteRepository(org.apache.archiva.repository.RemoteRepository) IndexCreationFailedException(org.apache.archiva.indexer.IndexCreationFailedException) IOException(java.io.IOException) IndexUpdateFailedException(org.apache.archiva.indexer.IndexUpdateFailedException) ArchivaIndexingContext(org.apache.archiva.indexer.ArchivaIndexingContext) EditableRepository(org.apache.archiva.repository.EditableRepository)

Example 12 with IndexUpdateFailedException

use of org.apache.archiva.indexer.IndexUpdateFailedException in project archiva by apache.

the class ArchivaIndexManagerMock method scan.

@Override
public void scan(final ArchivaIndexingContext context) throws IndexUpdateFailedException {
    executeUpdateFunction(context, indexingContext -> {
        DefaultScannerListener listener = new DefaultScannerListener(indexingContext, indexerEngine, true, null);
        ScanningRequest request = new ScanningRequest(indexingContext, listener);
        ScanningResult result = scanner.scan(request);
        if (result.hasExceptions()) {
            log.error("Exceptions occured during index scan of " + context.getId());
            result.getExceptions().stream().map(e -> e.getMessage()).distinct().limit(5).forEach(s -> log.error("Message: " + s));
        }
    });
}
Also used : ScanningResult(org.apache.maven.index.ScanningResult) AuthenticationException(org.apache.maven.wagon.authentication.AuthenticationException) ArtifactContext(org.apache.maven.index.ArtifactContext) StringUtils(org.apache.commons.lang.StringUtils) PathUtil(org.apache.archiva.common.utils.PathUtil) UnsupportedBaseContextException(org.apache.archiva.indexer.UnsupportedBaseContextException) LoggerFactory(org.slf4j.LoggerFactory) IndexCreator(org.apache.maven.index.context.IndexCreator) ArchivaIndexManager(org.apache.archiva.indexer.ArchivaIndexManager) ArchivaConfiguration(org.apache.archiva.configuration.ArchivaConfiguration) WagonFactoryException(org.apache.archiva.proxy.common.WagonFactoryException) NetworkProxyAdmin(org.apache.archiva.admin.model.networkproxy.NetworkProxyAdmin) PasswordCredentials(org.apache.archiva.repository.PasswordCredentials) Map(java.util.Map) FileUtils(org.apache.archiva.common.utils.FileUtils) IndexCreationFailedException(org.apache.archiva.indexer.IndexCreationFailedException) UnsupportedRepositoryTypeException(org.apache.archiva.repository.UnsupportedRepositoryTypeException) RemoteIndexFeature(org.apache.archiva.repository.features.RemoteIndexFeature) URI(java.net.URI) TransferFailedException(org.apache.maven.wagon.TransferFailedException) Path(java.nio.file.Path) ArchivaIndexingContext(org.apache.archiva.indexer.ArchivaIndexingContext) AbstractHttpClientWagon(org.apache.maven.wagon.shared.http.AbstractHttpClientWagon) TransferEvent(org.apache.maven.wagon.events.TransferEvent) RepositoryAdminException(org.apache.archiva.admin.model.RepositoryAdminException) IndexCreationFeature(org.apache.archiva.repository.features.IndexCreationFeature) HttpMethodConfiguration(org.apache.maven.wagon.shared.http.HttpMethodConfiguration) ProxyInfo(org.apache.maven.wagon.proxy.ProxyInfo) Collection(java.util.Collection) ConnectionException(org.apache.maven.wagon.ConnectionException) AuthorizationException(org.apache.maven.wagon.authorization.AuthorizationException) Collectors(java.util.stream.Collectors) Indexer(org.apache.maven.index.Indexer) FileNotFoundException(java.io.FileNotFoundException) ManagedRepository(org.apache.archiva.repository.ManagedRepository) IndexUpdateRequest(org.apache.maven.index.updater.IndexUpdateRequest) IndexUpdateFailedException(org.apache.archiva.indexer.IndexUpdateFailedException) List(java.util.List) HttpConfiguration(org.apache.maven.wagon.shared.http.HttpConfiguration) IndexFormatTooOldException(org.apache.maven.index_shaded.lucene.index.IndexFormatTooOldException) IndexerEngine(org.apache.maven.index.IndexerEngine) EditableRepository(org.apache.archiva.repository.EditableRepository) WagonFactoryRequest(org.apache.archiva.proxy.common.WagonFactoryRequest) Scanner(org.apache.maven.index.Scanner) NetworkProxy(org.apache.archiva.admin.model.beans.NetworkProxy) ArtifactContextProducer(org.apache.maven.index.ArtifactContextProducer) WagonFactory(org.apache.archiva.proxy.common.WagonFactory) Inject(javax.inject.Inject) Repository(org.apache.archiva.repository.Repository) RemoteRepository(org.apache.archiva.repository.RemoteRepository) Service(org.springframework.stereotype.Service) IndexPackingRequest(org.apache.maven.index.packer.IndexPackingRequest) ResourceFetcher(org.apache.maven.index.updater.ResourceFetcher) StreamWagon(org.apache.maven.wagon.StreamWagon) Logger(org.slf4j.Logger) IndexPacker(org.apache.maven.index.packer.IndexPacker) MalformedURLException(java.net.MalformedURLException) Files(java.nio.file.Files) DefaultScannerListener(org.apache.maven.index.DefaultScannerListener) ScanningRequest(org.apache.maven.index.ScanningRequest) AuthenticationInfo(org.apache.maven.wagon.authentication.AuthenticationInfo) IOException(java.io.IOException) RepositoryType(org.apache.archiva.repository.RepositoryType) IndexingContext(org.apache.maven.index.context.IndexingContext) ConcurrentSkipListSet(java.util.concurrent.ConcurrentSkipListSet) ResourceDoesNotExistException(org.apache.maven.wagon.ResourceDoesNotExistException) Paths(java.nio.file.Paths) TransferListener(org.apache.maven.wagon.events.TransferListener) Wagon(org.apache.maven.wagon.Wagon) ScanningResult(org.apache.maven.index.ScanningResult) InputStream(java.io.InputStream) ScanningRequest(org.apache.maven.index.ScanningRequest) DefaultScannerListener(org.apache.maven.index.DefaultScannerListener)

Example 13 with IndexUpdateFailedException

use of org.apache.archiva.indexer.IndexUpdateFailedException in project archiva by apache.

the class MavenIndexManager method scan.

@Override
public void scan(final ArchivaIndexingContext context) throws IndexUpdateFailedException {
    executeUpdateFunction(context, indexingContext -> {
        DefaultScannerListener listener = new DefaultScannerListener(indexingContext, indexerEngine, true, null);
        ScanningRequest request = new ScanningRequest(indexingContext, listener);
        ScanningResult result = scanner.scan(request);
        if (result.hasExceptions()) {
            log.error("Exceptions occured during index scan of " + context.getId());
            result.getExceptions().stream().map(e -> e.getMessage()).distinct().limit(5).forEach(s -> log.error("Message: " + s));
        }
    });
}
Also used : ScanningResult(org.apache.maven.index.ScanningResult) AuthenticationException(org.apache.maven.wagon.authentication.AuthenticationException) ArtifactContext(org.apache.maven.index.ArtifactContext) StringUtils(org.apache.commons.lang.StringUtils) PathUtil(org.apache.archiva.common.utils.PathUtil) UnsupportedBaseContextException(org.apache.archiva.indexer.UnsupportedBaseContextException) LoggerFactory(org.slf4j.LoggerFactory) IndexCreator(org.apache.maven.index.context.IndexCreator) ArchivaIndexManager(org.apache.archiva.indexer.ArchivaIndexManager) ArchivaConfiguration(org.apache.archiva.configuration.ArchivaConfiguration) WagonFactoryException(org.apache.archiva.proxy.common.WagonFactoryException) NetworkProxyAdmin(org.apache.archiva.admin.model.networkproxy.NetworkProxyAdmin) PasswordCredentials(org.apache.archiva.repository.PasswordCredentials) Map(java.util.Map) FileUtils(org.apache.archiva.common.utils.FileUtils) IndexCreationFailedException(org.apache.archiva.indexer.IndexCreationFailedException) UnsupportedRepositoryTypeException(org.apache.archiva.repository.UnsupportedRepositoryTypeException) RemoteIndexFeature(org.apache.archiva.repository.features.RemoteIndexFeature) URI(java.net.URI) TransferFailedException(org.apache.maven.wagon.TransferFailedException) Path(java.nio.file.Path) ArchivaIndexingContext(org.apache.archiva.indexer.ArchivaIndexingContext) AbstractHttpClientWagon(org.apache.maven.wagon.shared.http.AbstractHttpClientWagon) TransferEvent(org.apache.maven.wagon.events.TransferEvent) RepositoryAdminException(org.apache.archiva.admin.model.RepositoryAdminException) IndexCreationFeature(org.apache.archiva.repository.features.IndexCreationFeature) HttpMethodConfiguration(org.apache.maven.wagon.shared.http.HttpMethodConfiguration) ProxyInfo(org.apache.maven.wagon.proxy.ProxyInfo) Collection(java.util.Collection) ConnectionException(org.apache.maven.wagon.ConnectionException) AuthorizationException(org.apache.maven.wagon.authorization.AuthorizationException) Collectors(java.util.stream.Collectors) Indexer(org.apache.maven.index.Indexer) FileNotFoundException(java.io.FileNotFoundException) ManagedRepository(org.apache.archiva.repository.ManagedRepository) IndexUpdateRequest(org.apache.maven.index.updater.IndexUpdateRequest) IndexUpdateFailedException(org.apache.archiva.indexer.IndexUpdateFailedException) List(java.util.List) HttpConfiguration(org.apache.maven.wagon.shared.http.HttpConfiguration) IndexFormatTooOldException(org.apache.maven.index_shaded.lucene.index.IndexFormatTooOldException) IndexerEngine(org.apache.maven.index.IndexerEngine) EditableRepository(org.apache.archiva.repository.EditableRepository) WagonFactoryRequest(org.apache.archiva.proxy.common.WagonFactoryRequest) Scanner(org.apache.maven.index.Scanner) NetworkProxy(org.apache.archiva.admin.model.beans.NetworkProxy) ArtifactContextProducer(org.apache.maven.index.ArtifactContextProducer) WagonFactory(org.apache.archiva.proxy.common.WagonFactory) IndexUpdater(org.apache.maven.index.updater.IndexUpdater) Inject(javax.inject.Inject) Repository(org.apache.archiva.repository.Repository) RemoteRepository(org.apache.archiva.repository.RemoteRepository) Service(org.springframework.stereotype.Service) IndexPackingRequest(org.apache.maven.index.packer.IndexPackingRequest) ResourceFetcher(org.apache.maven.index.updater.ResourceFetcher) StreamWagon(org.apache.maven.wagon.StreamWagon) Logger(org.slf4j.Logger) IndexPacker(org.apache.maven.index.packer.IndexPacker) MalformedURLException(java.net.MalformedURLException) Files(java.nio.file.Files) DefaultScannerListener(org.apache.maven.index.DefaultScannerListener) ScanningRequest(org.apache.maven.index.ScanningRequest) AuthenticationInfo(org.apache.maven.wagon.authentication.AuthenticationInfo) IOException(java.io.IOException) RepositoryType(org.apache.archiva.repository.RepositoryType) IndexingContext(org.apache.maven.index.context.IndexingContext) ConcurrentSkipListSet(java.util.concurrent.ConcurrentSkipListSet) ResourceDoesNotExistException(org.apache.maven.wagon.ResourceDoesNotExistException) Paths(java.nio.file.Paths) TransferListener(org.apache.maven.wagon.events.TransferListener) Wagon(org.apache.maven.wagon.Wagon) ScanningResult(org.apache.maven.index.ScanningResult) InputStream(java.io.InputStream) ScanningRequest(org.apache.maven.index.ScanningRequest) DefaultScannerListener(org.apache.maven.index.DefaultScannerListener)

Example 14 with IndexUpdateFailedException

use of org.apache.archiva.indexer.IndexUpdateFailedException in project archiva by apache.

the class MavenIndexManager method update.

@Override
public void update(final ArchivaIndexingContext context, final boolean fullUpdate) throws IndexUpdateFailedException {
    log.info("start download remote index for remote repository {}", context.getRepository().getId());
    URI remoteUpdateUri;
    if (!(context.getRepository() instanceof RemoteRepository) || !(context.getRepository().supportsFeature(RemoteIndexFeature.class))) {
        throw new IndexUpdateFailedException("The context is not associated to a remote repository with remote index " + context.getId());
    } else {
        RemoteIndexFeature rif = context.getRepository().getFeature(RemoteIndexFeature.class).get();
        remoteUpdateUri = context.getRepository().getLocation().resolve(rif.getIndexUri());
    }
    final RemoteRepository remoteRepository = (RemoteRepository) context.getRepository();
    executeUpdateFunction(context, indexingContext -> {
        try {
            // create a temp directory to download files
            Path tempIndexDirectory = Paths.get(indexingContext.getIndexDirectoryFile().getParent(), ".tmpIndex");
            Path indexCacheDirectory = Paths.get(indexingContext.getIndexDirectoryFile().getParent(), ".indexCache");
            Files.createDirectories(indexCacheDirectory);
            if (Files.exists(tempIndexDirectory)) {
                org.apache.archiva.common.utils.FileUtils.deleteDirectory(tempIndexDirectory);
            }
            Files.createDirectories(tempIndexDirectory);
            tempIndexDirectory.toFile().deleteOnExit();
            String baseIndexUrl = indexingContext.getIndexUpdateUrl();
            String wagonProtocol = remoteUpdateUri.toURL().getProtocol();
            NetworkProxy networkProxy = null;
            if (remoteRepository.supportsFeature(RemoteIndexFeature.class)) {
                RemoteIndexFeature rif = remoteRepository.getFeature(RemoteIndexFeature.class).get();
                if (StringUtils.isNotBlank(rif.getProxyId())) {
                    try {
                        networkProxy = networkProxyAdmin.getNetworkProxy(rif.getProxyId());
                    } catch (RepositoryAdminException e) {
                        log.error("Error occured while retrieving proxy {}", e.getMessage());
                    }
                    if (networkProxy == null) {
                        log.warn("your remote repository is configured to download remote index trought a proxy we cannot find id:{}", rif.getProxyId());
                    }
                }
                final StreamWagon wagon = (StreamWagon) wagonFactory.getWagon(new WagonFactoryRequest(wagonProtocol, remoteRepository.getExtraHeaders()).networkProxy(networkProxy));
                int readTimeout = (int) rif.getDownloadTimeout().toMillis() * 1000;
                wagon.setReadTimeout(readTimeout);
                wagon.setTimeout((int) remoteRepository.getTimeout().toMillis() * 1000);
                if (wagon instanceof AbstractHttpClientWagon) {
                    HttpConfiguration httpConfiguration = new HttpConfiguration();
                    HttpMethodConfiguration httpMethodConfiguration = new HttpMethodConfiguration();
                    httpMethodConfiguration.setUsePreemptive(true);
                    httpMethodConfiguration.setReadTimeout(readTimeout);
                    httpConfiguration.setGet(httpMethodConfiguration);
                    AbstractHttpClientWagon.class.cast(wagon).setHttpConfiguration(httpConfiguration);
                }
                wagon.addTransferListener(new DownloadListener());
                ProxyInfo proxyInfo = null;
                if (networkProxy != null) {
                    proxyInfo = new ProxyInfo();
                    proxyInfo.setType(networkProxy.getProtocol());
                    proxyInfo.setHost(networkProxy.getHost());
                    proxyInfo.setPort(networkProxy.getPort());
                    proxyInfo.setUserName(networkProxy.getUsername());
                    proxyInfo.setPassword(networkProxy.getPassword());
                }
                AuthenticationInfo authenticationInfo = null;
                if (remoteRepository.getLoginCredentials() != null && (remoteRepository.getLoginCredentials() instanceof PasswordCredentials)) {
                    PasswordCredentials creds = (PasswordCredentials) remoteRepository.getLoginCredentials();
                    authenticationInfo = new AuthenticationInfo();
                    authenticationInfo.setUserName(creds.getUsername());
                    authenticationInfo.setPassword(new String(creds.getPassword()));
                }
                wagon.connect(new org.apache.maven.wagon.repository.Repository(remoteRepository.getId(), baseIndexUrl), authenticationInfo, proxyInfo);
                Path indexDirectory = indexingContext.getIndexDirectoryFile().toPath();
                if (!Files.exists(indexDirectory)) {
                    Files.createDirectories(indexDirectory);
                }
                ResourceFetcher resourceFetcher = new WagonResourceFetcher(log, tempIndexDirectory, wagon, remoteRepository);
                IndexUpdateRequest request = new IndexUpdateRequest(indexingContext, resourceFetcher);
                request.setForceFullUpdate(fullUpdate);
                request.setLocalIndexCacheDir(indexCacheDirectory.toFile());
                indexUpdater.fetchAndUpdateIndex(request);
                indexingContext.updateTimestamp(true);
            }
        } catch (AuthenticationException e) {
            log.error("Could not login to the remote proxy for updating index of {}", remoteRepository.getId(), e);
            throw new IndexUpdateFailedException("Login in to proxy failed while updating remote repository " + remoteRepository.getId(), e);
        } catch (ConnectionException e) {
            log.error("Connection error during index update for remote repository {}", remoteRepository.getId(), e);
            throw new IndexUpdateFailedException("Connection error during index update for remote repository " + remoteRepository.getId(), e);
        } catch (MalformedURLException e) {
            log.error("URL for remote index update of remote repository {} is not correct {}", remoteRepository.getId(), remoteUpdateUri, e);
            throw new IndexUpdateFailedException("URL for remote index update of repository is not correct " + remoteUpdateUri, e);
        } catch (IOException e) {
            log.error("IOException during index update of remote repository {}: {}", remoteRepository.getId(), e.getMessage(), e);
            throw new IndexUpdateFailedException("IOException during index update of remote repository " + remoteRepository.getId() + (StringUtils.isNotEmpty(e.getMessage()) ? ": " + e.getMessage() : ""), e);
        } catch (WagonFactoryException e) {
            log.error("Wagon for remote index download of {} could not be created: {}", remoteRepository.getId(), e.getMessage(), e);
            throw new IndexUpdateFailedException("Error while updating the remote index of " + remoteRepository.getId(), e);
        }
    });
}
Also used : AbstractHttpClientWagon(org.apache.maven.wagon.shared.http.AbstractHttpClientWagon) MalformedURLException(java.net.MalformedURLException) HttpMethodConfiguration(org.apache.maven.wagon.shared.http.HttpMethodConfiguration) AuthenticationException(org.apache.maven.wagon.authentication.AuthenticationException) ResourceFetcher(org.apache.maven.index.updater.ResourceFetcher) RemoteRepository(org.apache.archiva.repository.RemoteRepository) HttpConfiguration(org.apache.maven.wagon.shared.http.HttpConfiguration) URI(java.net.URI) StreamWagon(org.apache.maven.wagon.StreamWagon) NetworkProxy(org.apache.archiva.admin.model.beans.NetworkProxy) AuthenticationInfo(org.apache.maven.wagon.authentication.AuthenticationInfo) ProxyInfo(org.apache.maven.wagon.proxy.ProxyInfo) WagonFactoryRequest(org.apache.archiva.proxy.common.WagonFactoryRequest) IndexUpdateFailedException(org.apache.archiva.indexer.IndexUpdateFailedException) Path(java.nio.file.Path) PasswordCredentials(org.apache.archiva.repository.PasswordCredentials) IndexUpdateRequest(org.apache.maven.index.updater.IndexUpdateRequest) IOException(java.io.IOException) RepositoryAdminException(org.apache.archiva.admin.model.RepositoryAdminException) WagonFactoryException(org.apache.archiva.proxy.common.WagonFactoryException) RemoteIndexFeature(org.apache.archiva.repository.features.RemoteIndexFeature) ConnectionException(org.apache.maven.wagon.ConnectionException)

Example 15 with IndexUpdateFailedException

use of org.apache.archiva.indexer.IndexUpdateFailedException in project archiva by apache.

the class ArchivaIndexManagerMock method pack.

@Override
public void pack(final ArchivaIndexingContext context) throws IndexUpdateFailedException {
    executeUpdateFunction(context, indexingContext -> {
        try {
            IndexPackingRequest request = new IndexPackingRequest(indexingContext, indexingContext.acquireIndexSearcher().getIndexReader(), indexingContext.getIndexDirectoryFile());
            indexPacker.packIndex(request);
            indexingContext.updateTimestamp(true);
        } catch (IOException e) {
            log.error("IOException while packing index of context " + context.getId() + (StringUtils.isNotEmpty(e.getMessage()) ? ": " + e.getMessage() : ""));
            throw new IndexUpdateFailedException("IOException during update of " + context.getId(), e);
        }
    });
}
Also used : IndexPackingRequest(org.apache.maven.index.packer.IndexPackingRequest) IOException(java.io.IOException) IndexUpdateFailedException(org.apache.archiva.indexer.IndexUpdateFailedException)

Aggregations

IndexUpdateFailedException (org.apache.archiva.indexer.IndexUpdateFailedException)22 IOException (java.io.IOException)18 Path (java.nio.file.Path)15 ArchivaIndexingContext (org.apache.archiva.indexer.ArchivaIndexingContext)15 RemoteRepository (org.apache.archiva.repository.RemoteRepository)15 RepositoryAdminException (org.apache.archiva.admin.model.RepositoryAdminException)13 MalformedURLException (java.net.MalformedURLException)12 URI (java.net.URI)12 NetworkProxy (org.apache.archiva.admin.model.beans.NetworkProxy)12 IndexCreationFailedException (org.apache.archiva.indexer.IndexCreationFailedException)12 UnsupportedBaseContextException (org.apache.archiva.indexer.UnsupportedBaseContextException)12 WagonFactoryException (org.apache.archiva.proxy.common.WagonFactoryException)12 WagonFactoryRequest (org.apache.archiva.proxy.common.WagonFactoryRequest)12 EditableRepository (org.apache.archiva.repository.EditableRepository)12 ManagedRepository (org.apache.archiva.repository.ManagedRepository)12 PasswordCredentials (org.apache.archiva.repository.PasswordCredentials)12 Repository (org.apache.archiva.repository.Repository)12 RemoteIndexFeature (org.apache.archiva.repository.features.RemoteIndexFeature)12 IndexingContext (org.apache.maven.index.context.IndexingContext)12 IndexPackingRequest (org.apache.maven.index.packer.IndexPackingRequest)12