Search in sources :

Example 81 with EventMetadata

use of org.commonjava.maven.galley.event.EventMetadata in project indy by Commonjava.

the class FoloContentAccessResource method doHead.

@ApiOperation("Store and track file/artifact content under the given artifact store (type/name) and path.")
@ApiResponses({ @ApiResponse(code = 404, message = "Content is not available"), @ApiResponse(code = 200, message = "Header metadata for content (or rendered listing when path ends with '/index.html' or '/'") })
@HEAD
@Path("/{path: (.*)}")
public Response doHead(@ApiParam("User-assigned tracking session key") @PathParam("id") final String id, @ApiParam("Package type (eg. maven, npm)") @PathParam("packageType") final String packageType, @ApiParam(allowableValues = "hosted,group,remote", required = true) @PathParam("type") final String type, @PathParam("name") final String name, @PathParam("path") final String path, @QueryParam(CHECK_CACHE_ONLY) final Boolean cacheOnly, @Context final HttpServletRequest request, @Context final UriInfo uriInfo) {
    final TrackingKey tk = new TrackingKey(id);
    final String baseUri = uriInfo.getBaseUriBuilder().path(BASE_PATH).path(id).build().toString();
    EventMetadata metadata = new EventMetadata().set(TRACKING_KEY, tk).set(ACCESS_CHANNEL, AccessChannel.MAVEN_REPO);
    return handler.doHead(packageType, type, name, path, cacheOnly, baseUri, request, metadata);
}
Also used : TrackingKey(org.commonjava.indy.folo.model.TrackingKey) EventMetadata(org.commonjava.maven.galley.event.EventMetadata) Path(javax.ws.rs.Path) HEAD(javax.ws.rs.HEAD) ApiOperation(io.swagger.annotations.ApiOperation) ApiResponses(io.swagger.annotations.ApiResponses)

Example 82 with EventMetadata

use of org.commonjava.maven.galley.event.EventMetadata in project indy by Commonjava.

the class FoloContentAccessResource method doGet.

@ApiOperation("Retrieve and track file/artifact content under the given artifact store (type/name) and path.")
@ApiResponses({ @ApiResponse(code = 404, message = "Content is not available"), @ApiResponse(code = 200, response = String.class, message = "Rendered content listing (when path ends with '/index.html' or '/')"), @ApiResponse(code = 200, response = StreamingOutput.class, message = "Content stream") })
@GET
@Path("/{path: (.*)}")
public Response doGet(@ApiParam("User-assigned tracking session key") @PathParam("id") final String id, @ApiParam("Package type (eg. maven, npm)") @PathParam("packageType") final String packageType, @ApiParam(allowableValues = "hosted,group,remote", required = true) @PathParam("type") final String type, @PathParam("name") final String name, @PathParam("path") final String path, @Context final HttpServletRequest request, @Context final UriInfo uriInfo) {
    final TrackingKey tk = new TrackingKey(id);
    final String baseUri = uriInfo.getBaseUriBuilder().path(BASE_PATH).path(id).build().toString();
    EventMetadata metadata = new EventMetadata().set(TRACKING_KEY, tk).set(ACCESS_CHANNEL, AccessChannel.MAVEN_REPO);
    return handler.doGet(packageType, type, name, path, baseUri, request, metadata);
}
Also used : TrackingKey(org.commonjava.indy.folo.model.TrackingKey) EventMetadata(org.commonjava.maven.galley.event.EventMetadata) Path(javax.ws.rs.Path) GET(javax.ws.rs.GET) ApiOperation(io.swagger.annotations.ApiOperation) ApiResponses(io.swagger.annotations.ApiResponses)

Example 83 with EventMetadata

use of org.commonjava.maven.galley.event.EventMetadata in project indy by Commonjava.

the class ImpliedRepositoryDetector method addImpliedRepositories.

private void addImpliedRepositories(final ImplicationsJob job) {
    job.implied = new ArrayList<>();
    logger.debug("Retrieving repository/pluginRepository declarations from:\n  {}", new JoinString("\n  ", job.pomView.getDocRefStack()));
    final List<List<RepositoryView>> repoLists = Arrays.asList(job.pomView.getNonProfileRepositories(), job.pomView.getAllPluginRepositories());
    final ImpliedRepositoryCreator creator = createRepoCreator();
    if (creator == null) {
        logger.error("Cannot proceed without a valid ImpliedRepositoryCreator instance. Aborting detection.");
        return;
    }
    for (final List<RepositoryView> repos : repoLists) {
        if (repos == null || repos.isEmpty()) {
            continue;
        }
        for (final RepositoryView repo : repos) {
            final ProjectVersionRef gav = job.pathInfo.getProjectId();
            try {
                if (config.isBlacklisted(repo.getUrl())) {
                    logger.debug("Discarding blacklisted repository: {}", repo);
                    continue;
                } else if (!config.isIncludeSnapshotRepos() && !repo.isReleasesEnabled()) {
                    logger.debug("Discarding snapshot repository: {}", repo);
                    continue;
                }
            } catch (final MalformedURLException e) {
                logger.error(String.format("Cannot add implied remote repo: %s from: %s (transfer: %s). Failed to check if repository is blacklisted.", repo.getUrl(), gav, job.transfer), e);
            }
            logger.debug("Detected POM-declared repository: {}", repo);
            List<RemoteRepository> rrs = null;
            try {
                rrs = storeManager.query().getRemoteRepositoryByUrl(MAVEN_PKG_KEY, repo.getUrl());
            } catch (IndyDataException e) {
                logger.error(String.format("Cannot lookup remote repositories by URL: %s. Reason: %s", e.getMessage()), e);
            }
            final RemoteRepository ref = creator.createFrom(gav, repo, LoggerFactory.getLogger(creator.getClass()));
            if (ref == null) {
                logger.warn("ImpliedRepositoryCreator didn't create anything for repo: {}, specified in: {}. Skipping.", repo.getId(), gav);
                continue;
            }
            if (rrs != null && !rrs.isEmpty()) {
                rrs = rrs.stream().filter(rr -> rr.isAllowReleases() == ref.isAllowReleases()).filter(rr -> rr.isAllowSnapshots() == ref.isAllowSnapshots()).filter(rr -> (isEmpty(rr.getPathMaskPatterns()) && isEmpty(ref.getPathMaskPatterns())) || rr.getPathMaskPatterns().equals(ref.getPathMaskPatterns())).collect(Collectors.toList());
            }
            if (rrs == null || rrs.isEmpty()) {
                logger.debug("Creating new RemoteRepository for: {}", repo);
                final RemoteRepository rr = ref.copyOf();
                rr.setMetadata(METADATA_ORIGIN, IMPLIED_REPO_ORIGIN);
                try {
                    rr.setMetadata(IMPLIED_BY_STORES, mapper.writeValueAsString(new ImpliedRepoMetadataManager.ImpliedRemotesWrapper(Collections.singletonList(job.store.getKey()))));
                } catch (JsonProcessingException e) {
                    logger.error("Failed to set {}", IMPLIED_BY_STORES);
                    continue;
                }
                if (!remoteValidator.isValid(rr)) {
                    logger.warn("Implied repository to: {} is invalid! Repository created was: {}", repo.getUrl(), rr);
                    continue;
                }
                final String changelog = String.format("Adding remote repository: %s (url: %s, name: %s), which is implied by the POM: %s (at: %s/%s)", repo.getId(), repo.getUrl(), repo.getName(), gav, job.transfer.getLocation().getUri(), job.transfer.getPath());
                final ChangeSummary summary = new ChangeSummary(ChangeSummary.SYSTEM_USER, changelog);
                try {
                    final boolean result = storeManager.storeArtifactStore(rr, summary, true, false, new EventMetadata().set(StoreDataManager.EVENT_ORIGIN, IMPLIED_REPOS_DETECTION).set(IMPLIED_BY_POM_TRANSFER, job.transfer));
                    logger.debug("Stored new RemoteRepository: {}. (successful? {})", rr, result);
                    job.implied.add(rr);
                } catch (final IndyDataException e) {
                    logger.error(String.format("Cannot add implied remote repo: %s from: %s (transfer: %s). Failed to store new remote repository.", repo.getUrl(), gav, job.transfer), e);
                }
            } else {
                logger.debug("Found existing RemoteRepositories: {}", rrs);
                for (final RemoteRepository rr : rrs) {
                    rr.setMetadata(METADATA_ORIGIN, IMPLIED_REPO_ORIGIN);
                    try {
                        metadataManager.updateImpliedBy(rr, job.store);
                    } catch (ImpliedReposException e) {
                        logger.error("Failed to set {}", IMPLIED_BY_STORES);
                        continue;
                    }
                    final String changelog = String.format("Updating the existing remote repository: %s (url: %s, name: %s), which is implied by the POM: %s (at: %s/%s)", repo.getId(), repo.getUrl(), repo.getName(), gav, job.transfer.getLocation().getUri(), job.transfer.getPath());
                    final ChangeSummary summary = new ChangeSummary(ChangeSummary.SYSTEM_USER, changelog);
                    try {
                        final boolean result = storeManager.storeArtifactStore(rr, summary, false, false, null);
                        logger.debug("Updated the RemoteRepository: {}. (successful? {})", rr, result);
                        job.implied.add(rr);
                    } catch (final IndyDataException e) {
                        logger.error(String.format("Cannot add implied remote repo: %s from: %s (transfer: %s). Failed to update the remote repository.", repo.getUrl(), gav, job.transfer), e);
                    }
                }
            }
        }
    }
}
Also used : Arrays(java.util.Arrays) KeyedLocation(org.commonjava.indy.model.galley.KeyedLocation) ChangeSummary(org.commonjava.indy.audit.ChangeSummary) LoggerFactory(org.slf4j.LoggerFactory) RemoteRepository(org.commonjava.indy.model.core.RemoteRepository) StringUtils(org.apache.commons.lang3.StringUtils) WeftManaged(org.commonjava.cdi.util.weft.WeftManaged) FileStorageEvent(org.commonjava.maven.galley.event.FileStorageEvent) MavenPomReader(org.commonjava.maven.galley.maven.parse.MavenPomReader) Observes(javax.enterprise.event.Observes) ArtifactStoreValidator(org.commonjava.indy.data.ArtifactStoreValidator) CoreEventManagerConstants(org.commonjava.indy.change.event.CoreEventManagerConstants) ImpliedReposException(org.commonjava.indy.implrepo.ImpliedReposException) StoreKey(org.commonjava.indy.model.core.StoreKey) MAVEN_PKG_KEY(org.commonjava.indy.pkg.maven.model.MavenPackageTypeDescriptor.MAVEN_PKG_KEY) ArtifactPathInfo(org.commonjava.atlas.maven.ident.util.ArtifactPathInfo) IndyGroovyException(org.commonjava.indy.subsys.template.IndyGroovyException) IndyObjectMapper(org.commonjava.indy.model.core.io.IndyObjectMapper) Collection(java.util.Collection) IMPLIED_REPO_ORIGIN(org.commonjava.indy.implrepo.data.ImpliedReposStoreDataManagerDecorator.IMPLIED_REPO_ORIGIN) METADATA_ORIGIN(org.commonjava.indy.model.core.ArtifactStore.METADATA_ORIGIN) Set(java.util.Set) Collectors(java.util.stream.Collectors) List(java.util.List) EventMetadata(org.commonjava.maven.galley.event.EventMetadata) IndyDataException(org.commonjava.indy.data.IndyDataException) MavenPomView(org.commonjava.maven.galley.maven.model.view.MavenPomView) ProjectVersionRef(org.commonjava.atlas.maven.ident.ref.ProjectVersionRef) IMPLIED_BY_STORES(org.commonjava.indy.implrepo.data.ImpliedRepoMetadataManager.IMPLIED_BY_STORES) JoinString(org.commonjava.atlas.maven.ident.util.JoinString) Group(org.commonjava.indy.model.core.Group) GalleyMavenException(org.commonjava.maven.galley.maven.GalleyMavenException) ArrayList(java.util.ArrayList) Inject(javax.inject.Inject) Transfer(org.commonjava.maven.galley.model.Transfer) RepositoryView(org.commonjava.maven.galley.maven.model.view.RepositoryView) ImpliedRepoConfig(org.commonjava.indy.implrepo.conf.ImpliedRepoConfig) ExecutorConfig(org.commonjava.cdi.util.weft.ExecutorConfig) IMPLIED_STORES(org.commonjava.indy.implrepo.data.ImpliedRepoMetadataManager.IMPLIED_STORES) Location(org.commonjava.maven.galley.model.Location) ExecutorService(java.util.concurrent.ExecutorService) Logger(org.slf4j.Logger) ArtifactStore(org.commonjava.indy.model.core.ArtifactStore) MalformedURLException(java.net.MalformedURLException) Executor(java.util.concurrent.Executor) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException) ScriptEngine(org.commonjava.indy.subsys.template.ScriptEngine) ImpliedRepoMetadataManager(org.commonjava.indy.implrepo.data.ImpliedRepoMetadataManager) Collections(java.util.Collections) StoreDataManager(org.commonjava.indy.data.StoreDataManager) MalformedURLException(java.net.MalformedURLException) RemoteRepository(org.commonjava.indy.model.core.RemoteRepository) JoinString(org.commonjava.atlas.maven.ident.util.JoinString) RepositoryView(org.commonjava.maven.galley.maven.model.view.RepositoryView) EventMetadata(org.commonjava.maven.galley.event.EventMetadata) IndyDataException(org.commonjava.indy.data.IndyDataException) JoinString(org.commonjava.atlas.maven.ident.util.JoinString) ProjectVersionRef(org.commonjava.atlas.maven.ident.ref.ProjectVersionRef) List(java.util.List) ArrayList(java.util.ArrayList) ChangeSummary(org.commonjava.indy.audit.ChangeSummary) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException) ImpliedReposException(org.commonjava.indy.implrepo.ImpliedReposException)

Example 84 with EventMetadata

use of org.commonjava.maven.galley.event.EventMetadata in project indy by Commonjava.

the class ProxyResponseHelper method createRepo.

/**
 * Create repositories (group, remote, hosted) when trackingId is present. Otherwise create normal remote
 * repository with specified name.
 *
 * @param trackingId
 * @param url
 * @param name distinct remote repository name. null if trackingId is given
 */
private ProxyCreationResult createRepo(String trackingId, URL url, String name) throws IndyDataException {
    UrlInfo info = new UrlInfo(url.toExternalForm());
    UserPass up = UserPass.parse(ApplicationHeader.authorization, httpRequest, url.getAuthority());
    String baseUrl = getBaseUrl(url, false);
    logger.debug(">>>> Create repo: trackingId=" + trackingId + ", name=" + name);
    ProxyCreationResult result = repoCreator.create(trackingId, name, baseUrl, info, up, LoggerFactory.getLogger(repoCreator.getClass()));
    ChangeSummary changeSummary = new ChangeSummary(ChangeSummary.SYSTEM_USER, "Creating HTTProx proxy for: " + info.getUrl());
    RemoteRepository remote = result.getRemote();
    if (remote != null) {
        storeManager.storeArtifactStore(remote, changeSummary, false, true, new EventMetadata());
    }
    HostedRepository hosted = result.getHosted();
    if (hosted != null) {
        storeManager.storeArtifactStore(hosted, changeSummary, false, true, new EventMetadata());
    }
    Group group = result.getGroup();
    if (group != null) {
        storeManager.storeArtifactStore(group, changeSummary, false, true, new EventMetadata());
    }
    return result;
}
Also used : Group(org.commonjava.indy.model.core.Group) UrlInfo(org.commonjava.indy.util.UrlInfo) ProxyCreationResult(org.commonjava.indy.httprox.handler.ProxyCreationResult) UserPass(org.commonjava.indy.subsys.http.util.UserPass) RemoteRepository(org.commonjava.indy.model.core.RemoteRepository) ChangeSummary(org.commonjava.indy.audit.ChangeSummary) EventMetadata(org.commonjava.maven.galley.event.EventMetadata) HostedRepository(org.commonjava.indy.model.core.HostedRepository)

Example 85 with EventMetadata

use of org.commonjava.maven.galley.event.EventMetadata in project indy by Commonjava.

the class ProxyResponseHelper method doTransfer.

private void doTransfer(final HttpConduitWrapper http, final ArtifactStore store, final String path, final boolean writeBody, final UserPass proxyUserPass, final ProxyMeter meter) throws IOException, IndyWorkflowException {
    if (transferred) {
        return;
    }
    transferred = true;
    if (!http.isOpen()) {
        throw new IOException("Sink channel already closed (or null)!");
    }
    final EventMetadata eventMetadata = createEventMetadata(writeBody, proxyUserPass, path, store);
    Transfer txfr = null;
    try {
        txfr = contentController.get(store.getKey(), path, eventMetadata);
    } catch (final IndyWorkflowException e) {
        if (!(e.getCause() instanceof TransferException)) {
            throw e;
        }
        logger.debug("Suppressed exception for further handling inside proxy logic:", e);
    }
    if (txfr != null && txfr.exists()) {
        meter.reportResponseSummary();
        http.writeExistingTransfer(txfr, writeBody, path, eventMetadata);
    } else {
        http.writeNotFoundTransfer(store, path);
    }
}
Also used : TransferException(org.commonjava.maven.galley.TransferException) IndyWorkflowException(org.commonjava.indy.IndyWorkflowException) Transfer(org.commonjava.maven.galley.model.Transfer) IOException(java.io.IOException) EventMetadata(org.commonjava.maven.galley.event.EventMetadata)

Aggregations

EventMetadata (org.commonjava.maven.galley.event.EventMetadata)154 Transfer (org.commonjava.maven.galley.model.Transfer)65 ChangeSummary (org.commonjava.indy.audit.ChangeSummary)49 Test (org.junit.Test)47 IndyDataException (org.commonjava.indy.data.IndyDataException)44 StoreKey (org.commonjava.indy.model.core.StoreKey)40 IndyWorkflowException (org.commonjava.indy.IndyWorkflowException)38 ConcreteResource (org.commonjava.maven.galley.model.ConcreteResource)37 RemoteRepository (org.commonjava.indy.model.core.RemoteRepository)35 ArtifactStore (org.commonjava.indy.model.core.ArtifactStore)28 Group (org.commonjava.indy.model.core.Group)26 ApiOperation (io.swagger.annotations.ApiOperation)23 Path (javax.ws.rs.Path)23 Logger (org.slf4j.Logger)23 ApiResponses (io.swagger.annotations.ApiResponses)22 IOException (java.io.IOException)21 InputStream (java.io.InputStream)20 HostedRepository (org.commonjava.indy.model.core.HostedRepository)20 KeyedLocation (org.commonjava.indy.model.galley.KeyedLocation)20 ArrayList (java.util.ArrayList)18