Search in sources :

Example 1 with BaseEntity

use of com.epam.pipeline.entity.BaseEntity in project cloud-pipeline by epam.

the class ParameterMapper method resolveConfigurations.

public List<ResolvedConfiguration> resolveConfigurations(AnalysisConfiguration<? extends AbstractRunConfigurationEntry> configuration) {
    FolderWithMetadata project = folderManager.getProject(configuration.getConfigurationId(), AclClass.CONFIGURATION);
    Map<String, PipeConfValue> projectData = project == null ? new HashMap<>() : project.getData();
    List<? extends AbstractRunConfigurationEntry> entries = configuration.getEntries();
    if (CollectionUtils.isEmpty(configuration.getEntitiesIds())) {
        return Collections.singletonList(resolveParameters(entries, projectData));
    }
    // In case of array references one entity may be expanded to
    // list of references entities, e.g. SampleSet is expanded
    // to list of Sample entities
    // TODO: The only reason to store it as map - is to add association to run
    // TODO: to initial entity, from which link comes. Find better solution.
    Map<Long, List<MetadataEntity>> targetEntities = fetchAndExpandInputEntities(configuration);
    // resolve all parameter references in configurations
    Map<Long, ResolvedConfiguration> resolvedConfigurations = targetEntities.values().stream().flatMap(Collection::stream).collect(Collectors.toMap(BaseEntity::getId, entity -> resolveParameters(entity, entries, projectData)));
    return targetEntities.entrySet().stream().map(idToEntities -> idToEntities.getValue().stream().map(entity -> {
        ResolvedConfiguration currentConfiguration = resolvedConfigurations.get(entity.getId());
        currentConfiguration.getAssociatedEntityIds().add(idToEntities.getKey());
        return currentConfiguration;
    }).collect(Collectors.toList())).flatMap(Collection::stream).collect(Collectors.toList());
}
Also used : EntityTypeField(com.epam.pipeline.manager.metadata.parser.EntityTypeField) Setter(lombok.Setter) Getter(lombok.Getter) MessageConstants(com.epam.pipeline.common.MessageConstants) RequiredArgsConstructor(lombok.RequiredArgsConstructor) LoggerFactory(org.slf4j.LoggerFactory) HashMap(java.util.HashMap) PipeConfValueVO(com.epam.pipeline.entity.configuration.PipeConfValueVO) StringUtils(org.apache.commons.lang3.StringUtils) Function(java.util.function.Function) CollectionUtils(org.apache.commons.collections4.CollectionUtils) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) PipeConfValue(com.epam.pipeline.entity.metadata.PipeConfValue) MessageHelper(com.epam.pipeline.common.MessageHelper) AnalysisConfiguration(com.epam.pipeline.manager.pipeline.runner.AnalysisConfiguration) Service(org.springframework.stereotype.Service) AbstractRunConfigurationEntry(com.epam.pipeline.entity.configuration.AbstractRunConfigurationEntry) FolderWithMetadata(com.epam.pipeline.entity.metadata.FolderWithMetadata) Map(java.util.Map) TypeReference(com.fasterxml.jackson.core.type.TypeReference) PipelineConfiguration(com.epam.pipeline.entity.configuration.PipelineConfiguration) MapUtils(org.apache.commons.collections4.MapUtils) BaseEntity(com.epam.pipeline.entity.BaseEntity) Logger(org.slf4j.Logger) Collection(java.util.Collection) JsonMapper(com.epam.pipeline.config.JsonMapper) EqualsAndHashCode(lombok.EqualsAndHashCode) Collectors(java.util.stream.Collectors) ResolvedConfiguration(com.epam.pipeline.entity.pipeline.ResolvedConfiguration) Objects(java.util.Objects) List(java.util.List) MetadataEntity(com.epam.pipeline.entity.metadata.MetadataEntity) MetadataEntityManager(com.epam.pipeline.manager.metadata.MetadataEntityManager) AclClass(com.epam.pipeline.entity.security.acl.AclClass) PipelineStart(com.epam.pipeline.entity.pipeline.run.PipelineStart) AllArgsConstructor(lombok.AllArgsConstructor) Collections(java.util.Collections) Assert(org.springframework.util.Assert) ResolvedConfiguration(com.epam.pipeline.entity.pipeline.ResolvedConfiguration) PipeConfValue(com.epam.pipeline.entity.metadata.PipeConfValue) ArrayList(java.util.ArrayList) List(java.util.List) FolderWithMetadata(com.epam.pipeline.entity.metadata.FolderWithMetadata)

Example 2 with BaseEntity

use of com.epam.pipeline.entity.BaseEntity in project cloud-pipeline by epam.

the class FirecloudRunner method runAnalysis.

@Override
public List<PipelineRun> runAnalysis(AnalysisConfiguration<FirecloudRunConfigurationEntry> configuration) {
    Assert.state(CollectionUtils.isNotEmpty(configuration.getEntitiesIds()), "Entities are required to run analysis in Firecloud");
    FolderWithMetadata project = folderManager.getProject(configuration.getConfigurationId(), AclClass.CONFIGURATION);
    Assert.notNull(project, "Project folder must be specified to run analysis in Firecloud.");
    List<Long> entities = parameterMapper.fetchAndExpandInputEntities(configuration).values().stream().flatMap(Collection::stream).map(BaseEntity::getId).collect(Collectors.toList());
    // TODO: after merge - add check that FC class is filled
    configuration.getEntries().forEach(this::validateFirecloudConfiguration);
    FirecloudConfiguration settings = buildFirecloudConfig(configuration.getRefreshToken());
    return configuration.getEntries().stream().map(entry -> runFirecloudAnalysis(settings, entry, configuration.getConfigurationId(), entities, project.getId())).collect(Collectors.toList());
}
Also used : CredentialsManager(com.epam.pipeline.manager.google.CredentialsManager) PasswordGenerator(com.epam.pipeline.utils.PasswordGenerator) Arrays(java.util.Arrays) ParameterMapper(com.epam.pipeline.manager.pipeline.ParameterMapper) FirecloudCredentials(com.epam.pipeline.manager.google.FirecloudCredentials) PipelineRunManager(com.epam.pipeline.manager.pipeline.PipelineRunManager) RequiredArgsConstructor(lombok.RequiredArgsConstructor) SystemPreferences(com.epam.pipeline.manager.preference.SystemPreferences) HashMap(java.util.HashMap) StringUtils(org.apache.commons.lang3.StringUtils) CollectionUtils(org.apache.commons.collections4.CollectionUtils) ArrayList(java.util.ArrayList) PipelineRun(com.epam.pipeline.entity.pipeline.PipelineRun) FireCloudClass(com.epam.pipeline.entity.metadata.FireCloudClass) Service(org.springframework.stereotype.Service) FolderWithMetadata(com.epam.pipeline.entity.metadata.FolderWithMetadata) PipelineConfiguration(com.epam.pipeline.entity.configuration.PipelineConfiguration) BaseEntity(com.epam.pipeline.entity.BaseEntity) FirecloudPreferences(com.epam.pipeline.entity.pipeline.run.FirecloudPreferences) PipelineConfigurationManager(com.epam.pipeline.manager.pipeline.PipelineConfigurationManager) PreferenceManager(com.epam.pipeline.manager.preference.PreferenceManager) FolderManager(com.epam.pipeline.manager.pipeline.FolderManager) Collection(java.util.Collection) FirecloudRunConfigurationEntry(com.epam.pipeline.entity.configuration.FirecloudRunConfigurationEntry) Collectors(java.util.stream.Collectors) List(java.util.List) Builder(lombok.Builder) MetadataEntityManager(com.epam.pipeline.manager.metadata.MetadataEntityManager) Data(lombok.Data) AclClass(com.epam.pipeline.entity.security.acl.AclClass) SystemParams(com.epam.pipeline.manager.execution.SystemParams) PipelineStart(com.epam.pipeline.entity.pipeline.run.PipelineStart) MetadataClass(com.epam.pipeline.entity.metadata.MetadataClass) Assert(org.springframework.util.Assert) Collection(java.util.Collection) FolderWithMetadata(com.epam.pipeline.entity.metadata.FolderWithMetadata)

Example 3 with BaseEntity

use of com.epam.pipeline.entity.BaseEntity in project cloud-pipeline by epam.

the class GrantPermissionManager method loadAllEntitiesPermissions.

public EntityWithPermissionVO loadAllEntitiesPermissions(AclClass aclClass, Integer page, Integer pageSize, boolean expandGroups, Integer filterMask) {
    EntityWithPermissionVO result = new EntityWithPermissionVO();
    Collection<? extends AbstractSecuredEntity> entities = entityManager.loadAllWithParents(aclClass, page, pageSize);
    Map<AbstractSecuredEntity, List<AclPermissionEntry>> allPermissions = getEntitiesPermissions(entities);
    result.setTotalCount(entityManager.loadTotalCount(aclClass));
    List<EntityPermission> permissions = entities.stream().distinct().sorted(Comparator.comparingLong(BaseEntity::getId)).map(entity -> getEntityPermission(allPermissions, entity)).collect(toList());
    if (expandGroups) {
        expandGroups(permissions);
        if (filterMask != null) {
            permissions.forEach(entry -> {
                Set<AclPermissionEntry> filtered = SetUtils.emptyIfNull(entry.getPermissions()).stream().filter(permission -> permissionsService.isMaskBitSet(permission.getMask(), filterMask)).collect(toSet());
                entry.setPermissions(filtered);
            });
        }
    }
    result.setEntityPermissions(permissions);
    return result;
}
Also used : Autowired(org.springframework.beans.factory.annotation.Autowired) EntityEventServiceManager(com.epam.pipeline.manager.event.EntityEventServiceManager) StringUtils(org.apache.commons.lang3.StringUtils) PermissionGrantVO(com.epam.pipeline.controller.vo.PermissionGrantVO) PipelineApiService(com.epam.pipeline.manager.pipeline.PipelineApiService) ObjectIdentity(org.springframework.security.acls.model.ObjectIdentity) PipelineRun(com.epam.pipeline.entity.pipeline.PipelineRun) ConfigurationProviderManager(com.epam.pipeline.manager.pipeline.runner.ConfigurationProviderManager) AbstractRunConfigurationEntry(com.epam.pipeline.entity.configuration.AbstractRunConfigurationEntry) AclPermission(com.epam.pipeline.security.acl.AclPermission) Map(java.util.Map) MutableAcl(org.springframework.security.acls.model.MutableAcl) SecurityContextHolder(org.springframework.security.core.context.SecurityContextHolder) PermissionFactory(org.springframework.security.acls.domain.PermissionFactory) DefaultRoles(com.epam.pipeline.entity.user.DefaultRoles) Set(java.util.Set) Acl(org.springframework.security.acls.model.Acl) EntityWithPermissionVO(com.epam.pipeline.controller.vo.security.EntityWithPermissionVO) Tool(com.epam.pipeline.entity.pipeline.Tool) Stream(java.util.stream.Stream) CollectionUtils(org.springframework.util.CollectionUtils) EntityVO(com.epam.pipeline.controller.vo.EntityVO) AclDataAccessException(org.springframework.security.acls.model.AclDataAccessException) MetadataEntry(com.epam.pipeline.entity.metadata.MetadataEntry) ObjectIdentityImpl(org.springframework.security.acls.domain.ObjectIdentityImpl) AbstractEntityPermissionMapper(com.epam.pipeline.mapper.AbstractEntityPermissionMapper) DataStorageAction(com.epam.pipeline.entity.datastorage.DataStorageAction) Collectors.groupingBy(java.util.stream.Collectors.groupingBy) RunConfigurationVO(com.epam.pipeline.controller.vo.configuration.RunConfigurationVO) ArrayList(java.util.ArrayList) DockerRegistryManager(com.epam.pipeline.manager.docker.DockerRegistryManager) Service(org.springframework.stereotype.Service) SetUtils(org.apache.commons.collections4.SetUtils) EntityManager(com.epam.pipeline.manager.EntityManager) Sid(org.springframework.security.acls.model.Sid) BaseEntity(com.epam.pipeline.entity.BaseEntity) Pair(org.apache.commons.math3.util.Pair) FolderManager(com.epam.pipeline.manager.pipeline.FolderManager) TaskStatus(com.epam.pipeline.entity.pipeline.TaskStatus) AbstractDataStorage(com.epam.pipeline.entity.datastorage.AbstractDataStorage) PermissionEvaluator(org.springframework.security.access.PermissionEvaluator) EntityPermissionVO(com.epam.pipeline.controller.vo.EntityPermissionVO) GrantedAuthoritySid(org.springframework.security.acls.domain.GrantedAuthoritySid) PrincipalSid(org.springframework.security.acls.domain.PrincipalSid) RunConfiguration(com.epam.pipeline.entity.configuration.RunConfiguration) DockerRegistry(com.epam.pipeline.entity.pipeline.DockerRegistry) AclSecuredEntry(com.epam.pipeline.entity.security.acl.AclSecuredEntry) PipelineWithPermissions(com.epam.pipeline.entity.pipeline.PipelineWithPermissions) MetadataEntityManager(com.epam.pipeline.manager.metadata.MetadataEntityManager) AclClass(com.epam.pipeline.entity.security.acl.AclClass) ToolGroup(com.epam.pipeline.entity.pipeline.ToolGroup) AbstractHierarchicalEntity(com.epam.pipeline.entity.AbstractHierarchicalEntity) RequiredArgsConstructor(lombok.RequiredArgsConstructor) LoggerFactory(org.slf4j.LoggerFactory) EntityPermission(com.epam.pipeline.entity.security.acl.EntityPermission) Folder(com.epam.pipeline.entity.pipeline.Folder) UserContext(com.epam.pipeline.security.UserContext) MessageHelper(com.epam.pipeline.common.MessageHelper) Collectors.toMap(java.util.stream.Collectors.toMap) ListUtils(org.apache.commons.collections4.ListUtils) PipelineWithPermissionsMapper(com.epam.pipeline.mapper.PipelineWithPermissionsMapper) IssueComment(com.epam.pipeline.entity.issue.IssueComment) NodesManager(com.epam.pipeline.manager.cluster.NodesManager) IssueManager(com.epam.pipeline.manager.issue.IssueManager) Collectors.toSet(java.util.stream.Collectors.toSet) Pipeline(com.epam.pipeline.entity.pipeline.Pipeline) RepositoryTool(com.epam.pipeline.entity.pipeline.RepositoryTool) AbstractSecuredEntity(com.epam.pipeline.entity.AbstractSecuredEntity) Collection(java.util.Collection) AccessControlEntry(org.springframework.security.acls.model.AccessControlEntry) AclPermissionEntry(com.epam.pipeline.entity.security.acl.AclPermissionEntry) NodeInstance(com.epam.pipeline.entity.cluster.NodeInstance) List(java.util.List) UserManager(com.epam.pipeline.manager.user.UserManager) SidRetrievalStrategy(org.springframework.security.acls.model.SidRetrievalStrategy) Optional(java.util.Optional) Authentication(org.springframework.security.core.Authentication) AclSecuredFilter(com.epam.pipeline.entity.filter.AclSecuredFilter) JdbcMutableAclServiceImpl(com.epam.pipeline.security.acl.JdbcMutableAclServiceImpl) MessageConstants(com.epam.pipeline.common.MessageConstants) PipelineRunManager(com.epam.pipeline.manager.pipeline.PipelineRunManager) ToolManager(com.epam.pipeline.manager.pipeline.ToolManager) Permission(org.springframework.security.acls.model.Permission) HashMap(java.util.HashMap) HashSet(java.util.HashSet) Propagation(org.springframework.transaction.annotation.Propagation) Collectors.mapping(java.util.stream.Collectors.mapping) ToolGroupManager(com.epam.pipeline.manager.pipeline.ToolGroupManager) Logger(org.slf4j.Logger) AclSid(com.epam.pipeline.entity.security.acl.AclSid) Collectors.toList(java.util.stream.Collectors.toList) MetadataEntity(com.epam.pipeline.entity.metadata.MetadataEntity) PipelinesWithPermissionsVO(com.epam.pipeline.controller.vo.PipelinesWithPermissionsVO) AccessControlEntryImpl(org.springframework.security.acls.domain.AccessControlEntryImpl) Data(lombok.Data) Issue(com.epam.pipeline.entity.issue.Issue) Comparator(java.util.Comparator) Collections(java.util.Collections) Transactional(org.springframework.transaction.annotation.Transactional) Assert(org.springframework.util.Assert) EntityWithPermissionVO(com.epam.pipeline.controller.vo.security.EntityWithPermissionVO) EntityPermission(com.epam.pipeline.entity.security.acl.EntityPermission) AbstractSecuredEntity(com.epam.pipeline.entity.AbstractSecuredEntity) AclPermissionEntry(com.epam.pipeline.entity.security.acl.AclPermissionEntry) ArrayList(java.util.ArrayList) List(java.util.List) Collectors.toList(java.util.stream.Collectors.toList)

Example 4 with BaseEntity

use of com.epam.pipeline.entity.BaseEntity in project cloud-pipeline by epam.

the class ConfigurationEntryMapper method getContentBuilder.

private XContentBuilder getContentBuilder(final EntityContainer<ConfigurationEntryDoc> container) {
    RunConfiguration configuration = container.getEntity().getConfiguration();
    try (XContentBuilder jsonBuilder = XContentFactory.jsonBuilder()) {
        jsonBuilder.startObject();
        AbstractRunConfigurationEntry entry = container.getEntity().getEntry();
        jsonBuilder.field(DOC_TYPE_FIELD, SearchDocumentType.CONFIGURATION.name()).field("id", container.getEntity().getId()).field("name", Optional.ofNullable(entry).map(AbstractRunConfigurationEntry::getName).orElse(configuration.getName())).field("description", configuration.getName() + " " + StringUtils.defaultIfBlank(configuration.getDescription(), StringUtils.EMPTY)).field("createdDate", parseDataToString(configuration.getCreatedDate())).field("parentId", Optional.ofNullable(configuration.getParent()).map(BaseEntity::getId).orElse(null));
        buildUserContent(container.getOwner(), jsonBuilder);
        buildMetadata(container.getMetadata(), jsonBuilder);
        buildPermissions(container.getPermissions(), jsonBuilder);
        if (entry != null) {
            buildConfigurationEntry(entry, container.getEntity().getPipeline(), jsonBuilder);
        }
        jsonBuilder.endObject();
        return jsonBuilder;
    } catch (IOException e) {
        throw new IllegalArgumentException("Failed to create elasticsearch document for run configuration: ", e);
    }
}
Also used : RunConfiguration(com.epam.pipeline.entity.configuration.RunConfiguration) BaseEntity(com.epam.pipeline.entity.BaseEntity) AbstractRunConfigurationEntry(com.epam.pipeline.entity.configuration.AbstractRunConfigurationEntry) IOException(java.io.IOException) XContentBuilder(org.elasticsearch.common.xcontent.XContentBuilder)

Aggregations

BaseEntity (com.epam.pipeline.entity.BaseEntity)4 AbstractRunConfigurationEntry (com.epam.pipeline.entity.configuration.AbstractRunConfigurationEntry)3 AclClass (com.epam.pipeline.entity.security.acl.AclClass)3 MetadataEntityManager (com.epam.pipeline.manager.metadata.MetadataEntityManager)3 ArrayList (java.util.ArrayList)3 Collection (java.util.Collection)3 HashMap (java.util.HashMap)3 List (java.util.List)3 RequiredArgsConstructor (lombok.RequiredArgsConstructor)3 StringUtils (org.apache.commons.lang3.StringUtils)3 Service (org.springframework.stereotype.Service)3 Assert (org.springframework.util.Assert)3 MessageConstants (com.epam.pipeline.common.MessageConstants)2 MessageHelper (com.epam.pipeline.common.MessageHelper)2 PipelineConfiguration (com.epam.pipeline.entity.configuration.PipelineConfiguration)2 FolderWithMetadata (com.epam.pipeline.entity.metadata.FolderWithMetadata)2 MetadataEntity (com.epam.pipeline.entity.metadata.MetadataEntity)2 PipelineRun (com.epam.pipeline.entity.pipeline.PipelineRun)2 PipelineStart (com.epam.pipeline.entity.pipeline.run.PipelineStart)2 FolderManager (com.epam.pipeline.manager.pipeline.FolderManager)2