Search in sources :

Example 1 with Component

use of com.dtstack.taier.dao.domain.Component in project Taier by DTStack.

the class TenantService method initDataDevelop.

@Transactional(rollbackFor = Exception.class)
public void initDataDevelop(Long clusterId, Long tenantId, Long userId, String tenantName, String tenantDesc, List<ComponentBindDBDTO> bindDBDTOList) throws Exception {
    // 初始化目录
    List<Component> components = componentService.listAllComponents(clusterId);
    List<ComponentVO> componentVOS = ComponentVO.toVOS(components);
    batchCatalogueService.initCatalogue(tenantId, userId, componentVOS);
    // 初始化数据源相关的信息
    IComponentService componentService = null;
    for (ComponentBindDBDTO componentBindDBDTO : bindDBDTOList) {
        EComponentType eComponentType = EComponentType.getByCode(componentBindDBDTO.getComponentCode());
        String componentIdentity = tenantName;
        // db相关的操作
        if (BooleanUtils.isTrue(componentBindDBDTO.getCreateFlag())) {
            componentService = multiEngineServiceFactory.getComponentService(eComponentType.getTypeCode());
            componentService.createDatabase(clusterId, eComponentType, componentIdentity, tenantDesc);
        } else {
            componentIdentity = componentBindDBDTO.getDbName();
        }
        // 初始化数据源
        datasourceService.initDefaultSource(clusterId, eComponentType, tenantId, componentIdentity, tenantDesc, userId);
        // 初始化租户引擎关系
        TenantComponent tenantEngine = new TenantComponent();
        tenantEngine.setTaskType(ComponentTypeToEScheduleJobMapping.getEScheduleTypeByComponentCode(eComponentType.getTypeCode()).getType());
        tenantEngine.setTenantId(tenantId);
        tenantEngine.setComponentIdentity(componentIdentity);
        tenantEngine.setCreateUserId(userId);
        tenantEngine.setStatus(0);
        developTenantComponentService.insert(tenantEngine);
    }
}
Also used : IComponentService(com.dtstack.taier.develop.service.develop.IComponentService) ComponentVO(com.dtstack.taier.scheduler.vo.ComponentVO) TenantComponent(com.dtstack.taier.dao.domain.TenantComponent) Component(com.dtstack.taier.dao.domain.Component) TenantComponent(com.dtstack.taier.dao.domain.TenantComponent) EComponentType(com.dtstack.taier.common.enums.EComponentType) ComponentBindDBDTO(com.dtstack.taier.develop.dto.devlop.ComponentBindDBDTO) Transactional(org.springframework.transaction.annotation.Transactional)

Example 2 with Component

use of com.dtstack.taier.dao.domain.Component in project Taier by DTStack.

the class FlinkResource method getResourceEComponentType.

public EComponentType getResourceEComponentType(JobClient jobClient) {
    long tenantId = jobClient.getTenantId();
    Long clusterId = clusterTenantMapper.getClusterIdByTenantId(tenantId);
    Component flinkComponent = componentService.getComponentByClusterId(clusterId, EComponentType.FLINK.getTypeCode(), null);
    if (null != flinkComponent) {
        if (EDeployType.STANDALONE.getType() == flinkComponent.getDeployType()) {
            return EComponentType.FLINK;
        } else if (EDeployType.YARN.getType() == flinkComponent.getDeployType()) {
            return EComponentType.YARN;
        }
    }
    throw new RdosDefineException("No found resource EComponentType");
}
Also used : RdosDefineException(com.dtstack.taier.common.exception.RdosDefineException) Component(com.dtstack.taier.dao.domain.Component)

Example 3 with Component

use of com.dtstack.taier.dao.domain.Component in project Taier by DTStack.

the class StoragePart method getPluginName.

@Override
public String getPluginName() {
    List<Component> components = componentScheduleGroup.get(EComponentScheduleType.RESOURCE);
    if (CollectionUtils.isEmpty(components)) {
        throw new RdosDefineException(ErrorCode.RESOURCE_COMPONENT_NOT_CONFIG);
    }
    Component resourceComponent = components.get(0);
    String resourceVersion = resourceComponent.getVersionName();
    EComponentType resourceType = EComponentType.getByCode(resourceComponent.getComponentTypeCode());
    Optional<JSONObject> resourceModelConfig = context.getModelConfig(resourceType, resourceVersion);
    JSONObject storageModelConfig = resourceModelConfig.map(res -> res.getJSONObject(type.name())).orElseThrow(() -> new RdosDefineException(Strings.format(ErrorCode.RESOURCE_NOT_SUPPORT_COMPONENT_VERSION.getMsg(), resourceComponent.getComponentName(), type.name(), versionName)));
    return storageModelConfig.getString(type.name());
}
Also used : Strings(com.dtstack.taier.common.util.Strings) DataSource(com.dtstack.taier.develop.model.DataSource) RdosDefineException(com.dtstack.taier.common.exception.RdosDefineException) StringUtils(org.apache.commons.lang3.StringUtils) EComponentType(com.dtstack.taier.common.enums.EComponentType) List(java.util.List) CollectionUtils(org.apache.commons.collections.CollectionUtils) Component(com.dtstack.taier.dao.domain.Component) Map(java.util.Map) Context(com.dtstack.taier.develop.model.system.Context) Optional(java.util.Optional) JSONObject(com.alibaba.fastjson.JSONObject) EComponentScheduleType(com.dtstack.taier.common.enums.EComponentScheduleType) ErrorCode(com.dtstack.taier.common.exception.ErrorCode) JSONObject(com.alibaba.fastjson.JSONObject) RdosDefineException(com.dtstack.taier.common.exception.RdosDefineException) Component(com.dtstack.taier.dao.domain.Component) EComponentType(com.dtstack.taier.common.enums.EComponentType)

Example 4 with Component

use of com.dtstack.taier.dao.domain.Component in project Taier by DTStack.

the class DependComputePart method getExtraVersionParameters.

public Long getExtraVersionParameters() {
    Component resourceComponent = componentScheduleGroup.get(EComponentScheduleType.RESOURCE).get(0);
    String resourceVersion = resourceComponent.getVersionName();
    EComponentType resourceType = EComponentType.getByCode(resourceComponent.getComponentTypeCode());
    Optional<JSONObject> resourceModelExtraConfig = context.getModelExtraVersionParameters(resourceType, resourceVersion);
    String extraTemplateId = null;
    if (resourceModelExtraConfig.isPresent()) {
        extraTemplateId = getValueInConfigWithResourceStore(resourceModelExtraConfig.get(), resourceComponent, null);
        if (StringUtils.isNotBlank(extraTemplateId)) {
            return Long.parseLong(extraTemplateId);
        }
    }
    // 依赖resource 但是不依赖resource的类型拼接额外参数信息 如hive2
    extraTemplateId = context.getModelExtraVersionParameters(type, versionName).map((extraConfig) -> extraConfig.getString(versionName)).orElse(null);
    if (StringUtils.isNotBlank(extraTemplateId)) {
        return Long.parseLong(extraTemplateId);
    }
    return null;
}
Also used : JSONObject(com.alibaba.fastjson.JSONObject) Component(com.dtstack.taier.dao.domain.Component) EComponentType(com.dtstack.taier.common.enums.EComponentType)

Example 5 with Component

use of com.dtstack.taier.dao.domain.Component in project Taier by DTStack.

the class DependComputePart method getPluginName.

@Override
public String getPluginName() {
    validDeployType(deployType);
    if (null == storageType) {
        throw new RdosDefineException(ErrorCode.STORE_COMPONENT_NOT_CONFIG);
    }
    List<Component> components = componentScheduleGroup.get(EComponentScheduleType.RESOURCE);
    if (CollectionUtils.isEmpty(components)) {
        throw new RdosDefineException(ErrorCode.RESOURCE_COMPONENT_NOT_CONFIG);
    }
    Component resourceComponent = components.get(0);
    String resourceVersion = resourceComponent.getVersionName();
    EComponentType resourceType = EComponentType.getByCode(resourceComponent.getComponentTypeCode());
    Optional<JSONObject> resourceModelConfig = context.getModelConfig(resourceType, resourceVersion);
    if (!resourceModelConfig.isPresent()) {
        throw new RdosDefineException(Strings.format(ErrorCode.RESOURCE_NOT_SUPPORT_COMPONENT_VERSION.getMsg(), resourceType, type, versionName));
    }
    // 唯一的pluginName
    return getValueInConfigWithResourceStore(resourceModelConfig.get(), resourceComponent, this::getPluginNameInModelOrByConfigVersion);
}
Also used : JSONObject(com.alibaba.fastjson.JSONObject) RdosDefineException(com.dtstack.taier.common.exception.RdosDefineException) Component(com.dtstack.taier.dao.domain.Component) EComponentType(com.dtstack.taier.common.enums.EComponentType)

Aggregations

Component (com.dtstack.taier.dao.domain.Component)17 EComponentType (com.dtstack.taier.common.enums.EComponentType)11 JSONObject (com.alibaba.fastjson.JSONObject)10 RdosDefineException (com.dtstack.taier.common.exception.RdosDefineException)7 Cluster (com.dtstack.taier.dao.domain.Cluster)4 EScheduleJobType (com.dtstack.taier.common.enums.EScheduleJobType)3 ArrayList (java.util.ArrayList)3 EnvironmentContext (com.dtstack.taier.common.env.EnvironmentContext)2 KerberosConfig (com.dtstack.taier.dao.domain.KerberosConfig)2 ComponentVO (com.dtstack.taier.scheduler.vo.ComponentVO)2 Map (java.util.Map)2 Autowired (org.springframework.beans.factory.annotation.Autowired)2 JSONPath (com.alibaba.fastjson.JSONPath)1 Wrappers (com.baomidou.mybatisplus.core.toolkit.Wrappers)1 CommonConstant (com.dtstack.taier.common.constant.CommonConstant)1 DataSourceType (com.dtstack.taier.common.enums.DataSourceType)1 DictType (com.dtstack.taier.common.enums.DictType)1 EComponentScheduleType (com.dtstack.taier.common.enums.EComponentScheduleType)1 EScheduleType (com.dtstack.taier.common.enums.EScheduleType)1 ErrorCode (com.dtstack.taier.common.exception.ErrorCode)1