Search in sources :

Example 1 with ListenerResult

use of org.apache.inlong.manager.workflow.event.ListenerResult in project incubator-inlong by apache.

the class CreateHiveTableListener method listen.

@Override
public ListenerResult listen(WorkflowContext context) {
    GroupResourceProcessForm form = (GroupResourceProcessForm) context.getProcessForm();
    String groupId = form.getInlongGroupId();
    log.info("begin to create hive table for groupId={}", groupId);
    List<SinkForSortDTO> configList = sinkMapper.selectAllConfig(groupId, null);
    List<SinkForSortDTO> needCreateList = configList.stream().filter(sinkForSortDTO -> sinkForSortDTO.getEnableCreateResource() == 1).collect(Collectors.toList());
    if (hiveTableOperator == null) {
        hiveTableOperator = (IHiveTableOperator) SpringContextUtils.getBean(IHiveTableOperator.BEAN_NAME, DefaultHiveTableOperator.class.getName());
    }
    hiveTableOperator.createHiveResource(groupId, needCreateList);
    String result = "success to create hive table for group [" + groupId + "]";
    log.info(result);
    return ListenerResult.success(result);
}
Also used : StreamSinkEntityMapper(org.apache.inlong.manager.dao.mapper.StreamSinkEntityMapper) SinkForSortDTO(org.apache.inlong.manager.common.pojo.sink.SinkForSortDTO) Autowired(org.springframework.beans.factory.annotation.Autowired) Collectors(java.util.stream.Collectors) SpringContextUtils(org.apache.inlong.manager.service.utils.SpringContextUtils) GroupResourceProcessForm(org.apache.inlong.manager.common.pojo.workflow.form.GroupResourceProcessForm) List(java.util.List) ListenerResult(org.apache.inlong.manager.workflow.event.ListenerResult) Slf4j(lombok.extern.slf4j.Slf4j) SinkOperateListener(org.apache.inlong.manager.workflow.event.task.SinkOperateListener) TaskEvent(org.apache.inlong.manager.workflow.event.task.TaskEvent) Service(org.springframework.stereotype.Service) WorkflowContext(org.apache.inlong.manager.workflow.WorkflowContext) GroupResourceProcessForm(org.apache.inlong.manager.common.pojo.workflow.form.GroupResourceProcessForm) SinkForSortDTO(org.apache.inlong.manager.common.pojo.sink.SinkForSortDTO)

Example 2 with ListenerResult

use of org.apache.inlong.manager.workflow.event.ListenerResult in project incubator-inlong by apache.

the class CreateSortConfigListener method listen.

@Override
public ListenerResult listen(WorkflowContext context) throws Exception {
    LOGGER.info("Create sort config for context={}", context);
    ProcessForm form = context.getProcessForm();
    if (form instanceof UpdateGroupProcessForm) {
        UpdateGroupProcessForm updateGroupProcessForm = (UpdateGroupProcessForm) form;
        OperateType operateType = updateGroupProcessForm.getOperateType();
        if (operateType == OperateType.SUSPEND || operateType == OperateType.DELETE) {
            return ListenerResult.success();
        }
    }
    InlongGroupInfo groupInfo = this.getGroupInfo(form);
    String groupId = groupInfo.getInlongGroupId();
    if (StringUtils.isEmpty(groupId)) {
        LOGGER.warn("GroupId is null for context={}", context);
        return ListenerResult.success();
    }
    List<SinkResponse> sinkResponseList = streamSinkService.listSink(groupId, null);
    if (CollectionUtils.isEmpty(sinkResponseList)) {
        LOGGER.warn("Sink not found by groupId={}", groupId);
        return ListenerResult.success();
    }
    Map<String, DataFlowInfo> dataFlowInfoMap = sinkResponseList.stream().map(sink -> {
        DataFlowInfo flowInfo = commonOperateService.createDataFlow(groupInfo, sink);
        return Pair.of(sink.getInlongStreamId(), flowInfo);
    }).collect(Collectors.toMap(Pair::getKey, Pair::getValue));
    String dataFlows = OBJECT_MAPPER.writeValueAsString(dataFlowInfoMap);
    InlongGroupExtInfo extInfo = new InlongGroupExtInfo();
    extInfo.setInlongGroupId(groupId);
    extInfo.setKeyName(InlongGroupSettings.DATA_FLOW);
    extInfo.setKeyValue(dataFlows);
    if (groupInfo.getExtList() == null) {
        groupInfo.setExtList(Lists.newArrayList());
    }
    upsertDataFlow(groupInfo, extInfo);
    return ListenerResult.success();
}
Also used : ProcessForm(org.apache.inlong.manager.common.pojo.workflow.form.ProcessForm) InlongGroupInfo(org.apache.inlong.manager.common.pojo.group.InlongGroupInfo) LoggerFactory(org.slf4j.LoggerFactory) Autowired(org.springframework.beans.factory.annotation.Autowired) StringUtils(org.apache.commons.lang3.StringUtils) GroupResourceProcessForm(org.apache.inlong.manager.common.pojo.workflow.form.GroupResourceProcessForm) Lists(com.google.common.collect.Lists) Pair(org.apache.commons.lang3.tuple.Pair) CollectionUtils(org.apache.commons.collections.CollectionUtils) CommonOperateService(org.apache.inlong.manager.service.CommonOperateService) StreamSinkService(org.apache.inlong.manager.service.sink.StreamSinkService) TaskEvent(org.apache.inlong.manager.workflow.event.task.TaskEvent) ObjectMapper(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper) Map(java.util.Map) OperateType(org.apache.inlong.manager.common.pojo.workflow.form.UpdateGroupProcessForm.OperateType) WorkflowContext(org.apache.inlong.manager.workflow.WorkflowContext) InlongGroupSettings(org.apache.inlong.manager.common.settings.InlongGroupSettings) Logger(org.slf4j.Logger) SinkResponse(org.apache.inlong.manager.common.pojo.sink.SinkResponse) Collectors(java.util.stream.Collectors) WorkflowListenerException(org.apache.inlong.manager.common.exceptions.WorkflowListenerException) ListenerResult(org.apache.inlong.manager.workflow.event.ListenerResult) Component(org.springframework.stereotype.Component) List(java.util.List) UpdateGroupProcessForm(org.apache.inlong.manager.common.pojo.workflow.form.UpdateGroupProcessForm) DataFlowInfo(org.apache.inlong.sort.protocol.DataFlowInfo) InlongGroupExtInfo(org.apache.inlong.manager.common.pojo.group.InlongGroupExtInfo) SortOperateListener(org.apache.inlong.manager.workflow.event.task.SortOperateListener) UpdateGroupProcessForm(org.apache.inlong.manager.common.pojo.workflow.form.UpdateGroupProcessForm) SinkResponse(org.apache.inlong.manager.common.pojo.sink.SinkResponse) ProcessForm(org.apache.inlong.manager.common.pojo.workflow.form.ProcessForm) GroupResourceProcessForm(org.apache.inlong.manager.common.pojo.workflow.form.GroupResourceProcessForm) UpdateGroupProcessForm(org.apache.inlong.manager.common.pojo.workflow.form.UpdateGroupProcessForm) InlongGroupInfo(org.apache.inlong.manager.common.pojo.group.InlongGroupInfo) OperateType(org.apache.inlong.manager.common.pojo.workflow.form.UpdateGroupProcessForm.OperateType) InlongGroupExtInfo(org.apache.inlong.manager.common.pojo.group.InlongGroupExtInfo) DataFlowInfo(org.apache.inlong.sort.protocol.DataFlowInfo)

Aggregations

List (java.util.List)2 Collectors (java.util.stream.Collectors)2 GroupResourceProcessForm (org.apache.inlong.manager.common.pojo.workflow.form.GroupResourceProcessForm)2 WorkflowContext (org.apache.inlong.manager.workflow.WorkflowContext)2 ListenerResult (org.apache.inlong.manager.workflow.event.ListenerResult)2 TaskEvent (org.apache.inlong.manager.workflow.event.task.TaskEvent)2 Autowired (org.springframework.beans.factory.annotation.Autowired)2 Lists (com.google.common.collect.Lists)1 Map (java.util.Map)1 Slf4j (lombok.extern.slf4j.Slf4j)1 CollectionUtils (org.apache.commons.collections.CollectionUtils)1 StringUtils (org.apache.commons.lang3.StringUtils)1 Pair (org.apache.commons.lang3.tuple.Pair)1 ObjectMapper (org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper)1 WorkflowListenerException (org.apache.inlong.manager.common.exceptions.WorkflowListenerException)1 InlongGroupExtInfo (org.apache.inlong.manager.common.pojo.group.InlongGroupExtInfo)1 InlongGroupInfo (org.apache.inlong.manager.common.pojo.group.InlongGroupInfo)1 SinkForSortDTO (org.apache.inlong.manager.common.pojo.sink.SinkForSortDTO)1 SinkResponse (org.apache.inlong.manager.common.pojo.sink.SinkResponse)1 ProcessForm (org.apache.inlong.manager.common.pojo.workflow.form.ProcessForm)1