Examples with Identity com.alibaba.otter.shared.etl.model.Identity used on opensource projects

Example 26 with Identity

use of com.alibaba.otter.shared.etl.model.Identity in project otter by alibaba.

the class OtterTransformerFactory method transform.

/**
     * 将一种源数据进行转化，最后得到的结果会根据DataMediaPair中定义的目标对象生成不同的数据对象 <br/>
     * 
     * <pre>
     * 返回对象格式：Map
     * key : Class对象，代表生成的目标数据对象
     * value : 每种目标数据对象的集合数据
     * </pre>
     */
public Map<Class, BatchObject> transform(RowBatch rowBatch) {
    final Identity identity = translateIdentity(rowBatch.getIdentity());
    Map<Class, BatchObject> result = new HashMap<Class, BatchObject>();
    // 初始化默认值
    result.put(EventData.class, initBatchObject(identity, EventData.class));
    for (EventData eventData : rowBatch.getDatas()) {
        // 处理eventData
        Long tableId = eventData.getTableId();
        Pipeline pipeline = configClientService.findPipeline(identity.getPipelineId());
        // 针对每个同步数据，可能会存在多路复制的情况
        List<DataMediaPair> dataMediaPairs = ConfigHelper.findDataMediaPairByMediaId(pipeline, tableId);
        for (DataMediaPair pair : dataMediaPairs) {
            if (!pair.getSource().getId().equals(tableId)) {
                // 过滤tableID不为源的同步
                continue;
            }
            OtterTransformer translate = lookup(pair.getSource(), pair.getTarget());
            // 进行转化
            Object item = translate.transform(eventData, new OtterTransformerContext(identity, pair, pipeline));
            if (item == null) {
                continue;
            }
            // 合并结果
            merge(identity, result, item);
        }
    }
    return result;
}

Example 27 with Identity

use of com.alibaba.otter.shared.etl.model.Identity in project otter by alibaba.

the class OtterTransformerFactory method transform.

/**
     * 转化FileBatch对象
     */
public Map<Class, BatchObject> transform(FileBatch fileBatch) {
    final Identity identity = translateIdentity(fileBatch.getIdentity());
    List<FileData> fileDatas = fileBatch.getFiles();
    Map<Class, BatchObject> result = new HashMap<Class, BatchObject>();
    // 初始化默认值
    result.put(FileData.class, initBatchObject(identity, FileData.class));
    for (FileData fileData : fileDatas) {
        // 进行转化
        Long tableId = fileData.getTableId();
        Pipeline pipeline = configClientService.findPipeline(identity.getPipelineId());
        // 针对每个同步数据，可能会存在多路复制的情况
        List<DataMediaPair> dataMediaPairs = ConfigHelper.findDataMediaPairByMediaId(pipeline, tableId);
        for (DataMediaPair pair : dataMediaPairs) {
            if (!pair.getSource().getId().equals(tableId)) {
                // 过滤tableID不为源的同步
                continue;
            }
            Object item = fileDataTransformer.transform(fileData, new OtterTransformerContext(identity, pair, pipeline));
            if (item == null) {
                continue;
            }
            // 合并结果
            merge(identity, result, item);
        }
    }
    return result;
}

Example 28 with Identity

use of com.alibaba.otter.shared.etl.model.Identity in project otter by alibaba.

the class FileLoadActionTest method buildIdentity.

protected Identity buildIdentity(long channelId, long pipelineId, long processId) {
    Identity identity = new Identity();
    identity.setChannelId(channelId);
    identity.setPipelineId(pipelineId);
    identity.setProcessId(processId);
    return identity;
}