Search in sources :

Example 6 with MetadataEntity

use of com.epam.pipeline.entity.metadata.MetadataEntity in project cloud-pipeline by epam.

the class ParameterMapper method resolveReferences.

private List<MetadataEntity> resolveReferences(String reference, String[] fields, Map<MetadataKey, MetadataEntity> entityReferences, MetadataEntity entity, boolean includeLast) {
    List<MetadataEntity> currentEntities = Collections.singletonList(entity);
    int lastIndex = includeLast ? fields.length : fields.length - 1;
    for (int i = 0; i < lastIndex; i++) {
        List<MetadataEntity> links = new ArrayList<>();
        for (MetadataEntity currentEntity : currentEntities) {
            Map<String, PipeConfValue> currentEntityData = currentEntity.getData();
            String field = fields[i];
            if (MapUtils.isEmpty(currentEntityData) || !currentEntityData.containsKey(field)) {
                throw new IllegalArgumentException(messageHelper.getMessage(MessageConstants.ERROR_PARAMETER_MISSING_VALUE, reference, field));
            }
            PipeConfValue value = currentEntityData.get(field);
            if (StringUtils.isBlank(value.getValue())) {
                throw new IllegalArgumentException(messageHelper.getMessage(MessageConstants.ERROR_PARAMETER_MISSING_VALUE, reference, field));
            }
            String entityClassName = EntityTypeField.parseClass(value.getType());
            if (StringUtils.isBlank(entityClassName)) {
                throw new IllegalArgumentException(messageHelper.getMessage(MessageConstants.ERROR_PARAMETER_NON_REFERENCE_TYPE, reference, field));
            }
            if (EntityTypeField.isArrayType(value.getType())) {
                List<String> arrayValues = JsonMapper.parseData(value.getValue(), new TypeReference<List<String>>() {
                });
                if (CollectionUtils.isEmpty(arrayValues)) {
                    throw new IllegalArgumentException(messageHelper.getMessage(MessageConstants.ERROR_PARAMETER_INVALID_ARRAY, reference, value.getValue(), field));
                }
                arrayValues.forEach(v -> links.add(getReference(entityReferences, v, entityClassName, reference)));
            } else {
                links.add(getReference(entityReferences, value.getValue(), entityClassName, reference));
            }
        }
        currentEntities = links;
    }
    return currentEntities;
}
Also used : MetadataEntity(com.epam.pipeline.entity.metadata.MetadataEntity) ArrayList(java.util.ArrayList) PipeConfValue(com.epam.pipeline.entity.metadata.PipeConfValue) ArrayList(java.util.ArrayList) List(java.util.List)

Example 7 with MetadataEntity

use of com.epam.pipeline.entity.metadata.MetadataEntity in project cloud-pipeline by epam.

the class ParameterMapper method resolveConfigurations.

public List<ResolvedConfiguration> resolveConfigurations(AnalysisConfiguration<? extends AbstractRunConfigurationEntry> configuration) {
    FolderWithMetadata project = folderManager.getProject(configuration.getConfigurationId(), AclClass.CONFIGURATION);
    Map<String, PipeConfValue> projectData = project == null ? new HashMap<>() : project.getData();
    List<? extends AbstractRunConfigurationEntry> entries = configuration.getEntries();
    if (CollectionUtils.isEmpty(configuration.getEntitiesIds())) {
        return Collections.singletonList(resolveParameters(entries, projectData));
    }
    // In case of array references one entity may be expanded to
    // list of references entities, e.g. SampleSet is expanded
    // to list of Sample entities
    // TODO: The only reason to store it as map - is to add association to run
    // TODO: to initial entity, from which link comes. Find better solution.
    Map<Long, List<MetadataEntity>> targetEntities = fetchAndExpandInputEntities(configuration);
    // resolve all parameter references in configurations
    Map<Long, ResolvedConfiguration> resolvedConfigurations = targetEntities.values().stream().flatMap(Collection::stream).collect(Collectors.toMap(BaseEntity::getId, entity -> resolveParameters(entity, entries, projectData)));
    return targetEntities.entrySet().stream().map(idToEntities -> idToEntities.getValue().stream().map(entity -> {
        ResolvedConfiguration currentConfiguration = resolvedConfigurations.get(entity.getId());
        currentConfiguration.getAssociatedEntityIds().add(idToEntities.getKey());
        return currentConfiguration;
    }).collect(Collectors.toList())).flatMap(Collection::stream).collect(Collectors.toList());
}
Also used : EntityTypeField(com.epam.pipeline.manager.metadata.parser.EntityTypeField) Setter(lombok.Setter) Getter(lombok.Getter) MessageConstants(com.epam.pipeline.common.MessageConstants) RequiredArgsConstructor(lombok.RequiredArgsConstructor) LoggerFactory(org.slf4j.LoggerFactory) HashMap(java.util.HashMap) PipeConfValueVO(com.epam.pipeline.entity.configuration.PipeConfValueVO) StringUtils(org.apache.commons.lang3.StringUtils) Function(java.util.function.Function) CollectionUtils(org.apache.commons.collections4.CollectionUtils) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) PipeConfValue(com.epam.pipeline.entity.metadata.PipeConfValue) MessageHelper(com.epam.pipeline.common.MessageHelper) AnalysisConfiguration(com.epam.pipeline.manager.pipeline.runner.AnalysisConfiguration) Service(org.springframework.stereotype.Service) AbstractRunConfigurationEntry(com.epam.pipeline.entity.configuration.AbstractRunConfigurationEntry) FolderWithMetadata(com.epam.pipeline.entity.metadata.FolderWithMetadata) Map(java.util.Map) TypeReference(com.fasterxml.jackson.core.type.TypeReference) PipelineConfiguration(com.epam.pipeline.entity.configuration.PipelineConfiguration) MapUtils(org.apache.commons.collections4.MapUtils) BaseEntity(com.epam.pipeline.entity.BaseEntity) Logger(org.slf4j.Logger) Collection(java.util.Collection) JsonMapper(com.epam.pipeline.config.JsonMapper) EqualsAndHashCode(lombok.EqualsAndHashCode) Collectors(java.util.stream.Collectors) ResolvedConfiguration(com.epam.pipeline.entity.pipeline.ResolvedConfiguration) Objects(java.util.Objects) List(java.util.List) MetadataEntity(com.epam.pipeline.entity.metadata.MetadataEntity) MetadataEntityManager(com.epam.pipeline.manager.metadata.MetadataEntityManager) AclClass(com.epam.pipeline.entity.security.acl.AclClass) PipelineStart(com.epam.pipeline.entity.pipeline.run.PipelineStart) AllArgsConstructor(lombok.AllArgsConstructor) Collections(java.util.Collections) Assert(org.springframework.util.Assert) ResolvedConfiguration(com.epam.pipeline.entity.pipeline.ResolvedConfiguration) PipeConfValue(com.epam.pipeline.entity.metadata.PipeConfValue) ArrayList(java.util.ArrayList) List(java.util.List) FolderWithMetadata(com.epam.pipeline.entity.metadata.FolderWithMetadata)

Example 8 with MetadataEntity

use of com.epam.pipeline.entity.metadata.MetadataEntity in project cloud-pipeline by epam.

the class GrantPermissionManager method metadataEntityPermission.

public boolean metadataEntityPermission(Long entityId, String permissionName) {
    MetadataEntity entity = metadataEntityManager.load(entityId);
    if (entity.getParent() == null || entity.getParent().getId() == null) {
        return isAdmin(getSids());
    }
    AbstractSecuredEntity securedEntity = entityManager.load(AclClass.FOLDER, entity.getParent().getId());
    return permissionsHelper.isAllowed(permissionName, securedEntity);
}
Also used : MetadataEntity(com.epam.pipeline.entity.metadata.MetadataEntity) AbstractSecuredEntity(com.epam.pipeline.entity.AbstractSecuredEntity)

Example 9 with MetadataEntity

use of com.epam.pipeline.entity.metadata.MetadataEntity in project cloud-pipeline by epam.

the class ObjectCreatorUtils method createMetadataEntity.

public static MetadataEntity createMetadataEntity(Folder folder, MetadataClass metadataClass, String name, String externalId, Map<String, PipeConfValue> data) {
    MetadataEntity metadataEntity = new MetadataEntity();
    metadataEntity.setName(name);
    metadataEntity.setClassEntity(metadataClass);
    metadataEntity.setExternalId(externalId);
    metadataEntity.setParent(folder);
    metadataEntity.setData(data);
    return metadataEntity;
}
Also used : MetadataEntity(com.epam.pipeline.entity.metadata.MetadataEntity)

Example 10 with MetadataEntity

use of com.epam.pipeline.entity.metadata.MetadataEntity in project cloud-pipeline by epam.

the class FolderEventServiceTest method shouldAddFolderEvent.

@Test
@Transactional(propagation = Propagation.REQUIRES_NEW, rollbackFor = Exception.class)
public void shouldAddFolderEvent() {
    doNothing().when(entityManager).setManagers(anyListOf(SecuredEntityManager.class));
    Folder folder2 = new Folder(2L);
    Pipeline pipeline = new Pipeline(1L);
    S3bucketDataStorage dataStorage = new S3bucketDataStorage(1L, "", "");
    RunConfiguration runConfiguration = new RunConfiguration();
    runConfiguration.setId(1L);
    MetadataEntity metadataEntity = new MetadataEntity();
    metadataEntity.setClassEntity(new MetadataClass(1L, METADATA_CLASS_NAME));
    metadataEntity.setId(1L);
    Folder folder1 = new Folder(1L);
    folder1.setChildFolders(Collections.singletonList(folder2));
    folder1.setPipelines(Collections.singletonList(pipeline));
    folder1.setStorages(Collections.singletonList(dataStorage));
    folder1.setConfigurations(Collections.singletonList(runConfiguration));
    folder1.setMetadata(Collections.singletonMap(METADATA_CLASS_NAME, 1));
    when(issueManager.loadIssuesForEntity(any())).thenReturn(Arrays.asList(Issue.builder().id(1L).build(), Issue.builder().id(2L).build()));
    when(pipelineRunManager.loadAllRunsByPipeline(anyLong())).thenReturn(Arrays.asList(new PipelineRun(1L, ""), new PipelineRun(2L, "")));
    when(dataStorageManager.load(1L)).thenReturn(dataStorage);
    when(folderManager.load(1L)).thenReturn(folder1);
    when(folderManager.load(2L)).thenReturn(folder2);
    when(metadataEntityManager.loadMetadataEntityByClassNameAndFolderId(anyLong(), anyString())).thenReturn(Collections.singletonList(metadataEntity));
    doNothing().when(eventDao).insertUpdateEvent(anyString(), anyLong());
    folderEventService.updateEventsWithChildrenAndIssues(1L);
    verify(eventDao).insertUpdateEvent("folder", 1L);
    verify(eventDao).insertUpdateEvent("folder", 2L);
    verify(eventDao).insertUpdateEvent("pipeline", 1L);
    verify(eventDao).insertUpdateEvent("run", 1L);
    verify(eventDao).insertUpdateEvent("run", 2L);
    verify(eventDao).insertUpdateEvent("S3", 1L);
    verify(eventDao).insertUpdateEvent("configuration", 1L);
    verify(eventDao).insertUpdateEvent("metadata_entity", 1L);
    verify(eventDao, times(6)).insertUpdateEvent("issue", 1L);
    verify(eventDao, times(6)).insertUpdateEvent("issue", 2L);
    verifyNoMoreInteractions(eventDao);
}
Also used : PipelineRun(com.epam.pipeline.entity.pipeline.PipelineRun) MetadataEntity(com.epam.pipeline.entity.metadata.MetadataEntity) SecuredEntityManager(com.epam.pipeline.manager.security.SecuredEntityManager) RunConfiguration(com.epam.pipeline.entity.configuration.RunConfiguration) MetadataClass(com.epam.pipeline.entity.metadata.MetadataClass) S3bucketDataStorage(com.epam.pipeline.entity.datastorage.aws.S3bucketDataStorage) Folder(com.epam.pipeline.entity.pipeline.Folder) Pipeline(com.epam.pipeline.entity.pipeline.Pipeline) AbstractSpringTest(com.epam.pipeline.AbstractSpringTest) Test(org.junit.Test) Transactional(org.springframework.transaction.annotation.Transactional)

Aggregations

MetadataEntity (com.epam.pipeline.entity.metadata.MetadataEntity)48 PipeConfValue (com.epam.pipeline.entity.metadata.PipeConfValue)22 Test (org.junit.Test)22 Transactional (org.springframework.transaction.annotation.Transactional)14 MetadataClass (com.epam.pipeline.entity.metadata.MetadataClass)12 Folder (com.epam.pipeline.entity.pipeline.Folder)10 HashMap (java.util.HashMap)9 AbstractSpringTest (com.epam.pipeline.AbstractSpringTest)7 List (java.util.List)7 Map (java.util.Map)6 MetadataEntityVO (com.epam.pipeline.controller.vo.metadata.MetadataEntityVO)5 IOException (java.io.IOException)5 MetadataReadingException (com.epam.pipeline.exception.MetadataReadingException)4 FolderManager (com.epam.pipeline.manager.pipeline.FolderManager)4 InputStream (java.io.InputStream)4 ArrayList (java.util.ArrayList)4 Autowired (org.springframework.beans.factory.annotation.Autowired)4 Propagation (org.springframework.transaction.annotation.Propagation)4 AclClass (com.epam.pipeline.entity.security.acl.AclClass)3 com.epam.pipeline.manager.metadata.parser (com.epam.pipeline.manager.metadata.parser)3