Search in sources :

Example 6 with ZkNoNodeException

use of org.I0Itec.zkclient.exception.ZkNoNodeException in project otter by alibaba.

the class TerminZooKeeperArbitrateEvent method await.

/**
     * <pre>
     * 算法:
     * 1. 开始阻塞获取符合条件的processId,获取对应的data数据直接返回
     * </pre>
     */
public TerminEventData await(Long pipelineId) throws InterruptedException {
    Assert.notNull(pipelineId);
    TerminMonitor terminMonitor = ArbitrateFactory.getInstance(pipelineId, TerminMonitor.class);
    // 符合条件的processId
    Long processId = terminMonitor.waitForProcess();
    if (logger.isDebugEnabled()) {
        logger.debug("## await pipeline[{}] processId[{}] is termin", pipelineId, processId);
    }
    // 根据pipelineId+processId构造对应的path
    String path = StagePathUtils.getTermin(pipelineId, processId);
    try {
        byte[] data = zookeeper.readData(path);
        return JsonUtils.unmarshalFromByte(data, TerminEventData.class);
    } catch (ZkNoNodeException e) {
        logger.error("pipeline[{}] processId[{}] is process", pipelineId, processId);
        // modify for 2012-09-08, 发生主备切换时,await会进入死循环,针对NoNode后直接从内存队列中移除
        terminMonitor.ack(processId);
        // 再取下一个节点
        return await(pipelineId);
    } catch (ZkException e) {
        throw new ArbitrateException("Termin_await", e);
    }
}
Also used : ZkNoNodeException(org.I0Itec.zkclient.exception.ZkNoNodeException) TerminMonitor(com.alibaba.otter.shared.arbitrate.impl.setl.monitor.TerminMonitor) ZkException(org.I0Itec.zkclient.exception.ZkException) ArbitrateException(com.alibaba.otter.shared.arbitrate.exception.ArbitrateException)

Example 7 with ZkNoNodeException

use of org.I0Itec.zkclient.exception.ZkNoNodeException in project otter by alibaba.

the class TransformZooKeeperArbitrateEvent method await.

/**
     * <pre>
     * 算法:
     * 1. 检查当前的Permit,阻塞等待其授权(解决Channel的pause状态处理)
     * 2. 开始阻塞获取符合条件的processId
     * 3. 检查当前的即时Permit状态 (在阻塞获取processId过程会出现一些error信号,process节点会被删除)
     * 4. 获取Select传递的EventData数据,添加next node信息后直接返回
     * </pre>
     * 
     * @return
     */
public EtlEventData await(Long pipelineId) throws InterruptedException {
    Assert.notNull(pipelineId);
    PermitMonitor permitMonitor = ArbitrateFactory.getInstance(pipelineId, PermitMonitor.class);
    // 阻塞等待授权
    permitMonitor.waitForPermit();
    TransformStageListener transformStageListener = ArbitrateFactory.getInstance(pipelineId, TransformStageListener.class);
    // 符合条件的processId
    Long processId = transformStageListener.waitForProcess();
    ChannelStatus status = permitMonitor.getChannelPermit();
    if (status.isStart()) {
        // 即时查询一下当前的状态,状态随时可能会变
        // 根据pipelineId+processId构造对应的path
        String path = StagePathUtils.getExtractStage(pipelineId, processId);
        try {
            byte[] data = zookeeper.readData(path);
            EtlEventData eventData = JsonUtils.unmarshalFromByte(data, EtlEventData.class);
            // 下一个节点信息即为自己
            eventData.setNextNid(ArbitrateConfigUtils.getCurrentNid());
            // 只有这一条路返回
            return eventData;
        } catch (ZkNoNodeException e) {
            logger.error("pipeline[{}] processId[{}] is invalid , retry again", pipelineId, processId);
            // /出现节点不存在,说明出现了error情况,递归调用重新获取一次
            return await(pipelineId);
        } catch (ZkException e) {
            throw new ArbitrateException("transform_await", e.getMessage(), e);
        }
    } else {
        logger.info("pipelineId[{}] transform ignore processId[{}] by status[{}]", new Object[] { pipelineId, processId, status });
        // 递归调用
        return await(pipelineId);
    }
}
Also used : TransformStageListener(com.alibaba.otter.shared.arbitrate.impl.setl.zookeeper.monitor.TransformStageListener) ZkNoNodeException(org.I0Itec.zkclient.exception.ZkNoNodeException) ZkException(org.I0Itec.zkclient.exception.ZkException) PermitMonitor(com.alibaba.otter.shared.arbitrate.impl.setl.monitor.PermitMonitor) ArbitrateException(com.alibaba.otter.shared.arbitrate.exception.ArbitrateException) ChannelStatus(com.alibaba.otter.shared.common.model.config.channel.ChannelStatus) EtlEventData(com.alibaba.otter.shared.arbitrate.model.EtlEventData)

Example 8 with ZkNoNodeException

use of org.I0Itec.zkclient.exception.ZkNoNodeException in project otter by alibaba.

the class TransformZooKeeperArbitrateEvent method single.

/**
     * <pre>
     * 算法:
     * 1. 创建对应的transformed节点,标志transform已完成
     * </pre>
     * 
     * @param pipelineId 同步流id
     */
public void single(EtlEventData data) {
    Assert.notNull(data);
    String path = StagePathUtils.getTransformStage(data.getPipelineId(), data.getProcessId());
    data.setCurrNid(ArbitrateConfigUtils.getCurrentNid());
    // 序列化
    byte[] bytes = JsonUtils.marshalToByte(data, SerializerFeature.WriteClassName);
    try {
        zookeeper.create(path, bytes, CreateMode.PERSISTENT);
    } catch (ZkNoNodeException e) {
        // process节点不存在,出现了rollback/shutdown操作,直接忽略
        logger.warn("pipelineId[{}] transform ignore processId[{}] single by data:{}", new Object[] { data.getPipelineId(), data.getProcessId(), data });
    } catch (ZkNodeExistsException e) {
        // process节点已存在,出现了ConnectionLoss retry操作
        logger.warn("pipelineId[{}] transform ignore processId[{}] single by data:{}", new Object[] { data.getPipelineId(), data.getProcessId(), data });
    } catch (ZkException e) {
        throw new ArbitrateException("transform_single", e.getMessage(), e);
    }
}
Also used : ZkNodeExistsException(org.I0Itec.zkclient.exception.ZkNodeExistsException) ZkNoNodeException(org.I0Itec.zkclient.exception.ZkNoNodeException) ZkException(org.I0Itec.zkclient.exception.ZkException) ArbitrateException(com.alibaba.otter.shared.arbitrate.exception.ArbitrateException)

Example 9 with ZkNoNodeException

use of org.I0Itec.zkclient.exception.ZkNoNodeException in project otter by alibaba.

the class ExtractStageListener method stageChannged.

public void stageChannged(Long processId, List<String> stageNodes) {
    try {
        // 1. 根据pipelineId+processId构造对应的path
        String path = StagePathUtils.getProcess(getPipelineId(), processId);
        // 2.1 判断是否存在了current节点
        if (stageNodes.contains(currentNode)) {
            if (replyProcessIds.remove(processId)) {
                if (logger.isDebugEnabled()) {
                    logger.debug("## remove reply id [{}]", processId);
                }
            }
            // 不需要监听了
            return;
        }
        if (replyProcessIds.contains(processId)) {
            // 避免重复处理
            return;
        }
        // 2.2 判断是否存在了prev节点
        if (stageNodes.contains(prevNode)) {
            // 2.2.1 获取上一个节点的next node节点信息
            byte[] data = zookeeper.readData(path + "/" + prevNode);
            EtlEventData eventData = JsonUtils.unmarshalFromByte(data, EtlEventData.class);
            if (eventData.getNextNid().equals(ArbitrateConfigUtils.getCurrentNid())) {
                // 添加到返回队列,唤醒wait阻塞
                addReply(processId);
            }
        }
    } catch (ZkNoNodeException e) {
    // 出现节点不存在,说明出现了error情况
    } catch (ZkException e) {
        logger.error("ExtractStageListener", e);
    }
}
Also used : ZkNoNodeException(org.I0Itec.zkclient.exception.ZkNoNodeException) ZkException(org.I0Itec.zkclient.exception.ZkException) EtlEventData(com.alibaba.otter.shared.arbitrate.model.EtlEventData)

Example 10 with ZkNoNodeException

use of org.I0Itec.zkclient.exception.ZkNoNodeException in project otter by alibaba.

the class LoadStageListener method stageChannged.

public void stageChannged(Long processId, List<String> stageNodes) {
    try {
        // 1. 根据pipelineId+processId构造对应的path
        String path = StagePathUtils.getProcess(getPipelineId(), processId);
        if (replyProcessIds.contains(processId)) {
            // 避免重复处理
            return;
        }
        // 2.2 判断是否存在了prev节点
        if (stageNodes.contains(prevNode)) {
            // 2.2.1 获取上一个节点的next node节点信息
            byte[] data = zookeeper.readData(path + "/" + prevNode);
            EtlEventData eventData = JsonUtils.unmarshalFromByte(data, EtlEventData.class);
            if (eventData.getNextNid().equals(ArbitrateConfigUtils.getCurrentNid())) {
                List<Long> currentProcessIds = stageMonitor.getCurrentProcessIds(false);
                if (currentProcessIds.contains(processId) && currentProcessIds.get(0).equals(processId)) {
                    // 判断是否是当前最小的processId节点,轮到自己处理了
                    // 添加到返回队列,唤醒wait阻塞
                    addReply(processId);
                }
            }
        }
    } catch (ZkNoNodeException e) {
    // 出现节点不存在,说明出现了error情况
    } catch (ZkException e) {
        logger.error("LoadStageListener", e);
    }
}
Also used : ZkNoNodeException(org.I0Itec.zkclient.exception.ZkNoNodeException) ZkException(org.I0Itec.zkclient.exception.ZkException) EtlEventData(com.alibaba.otter.shared.arbitrate.model.EtlEventData)

Aggregations

ZkNoNodeException (org.I0Itec.zkclient.exception.ZkNoNodeException)27 ZkException (org.I0Itec.zkclient.exception.ZkException)22 ArbitrateException (com.alibaba.otter.shared.arbitrate.exception.ArbitrateException)17 EtlEventData (com.alibaba.otter.shared.arbitrate.model.EtlEventData)7 PermitMonitor (com.alibaba.otter.shared.arbitrate.impl.setl.monitor.PermitMonitor)5 ChannelStatus (com.alibaba.otter.shared.common.model.config.channel.ChannelStatus)5 ZkNodeExistsException (org.I0Itec.zkclient.exception.ZkNodeExistsException)5 ArrayList (java.util.ArrayList)4 PositionRange (com.alibaba.otter.canal.protocol.position.PositionRange)3 MainStemEventData (com.alibaba.otter.shared.arbitrate.model.MainStemEventData)3 Node (com.alibaba.otter.shared.common.model.config.node.Node)3 ZkInterruptedException (org.I0Itec.zkclient.exception.ZkInterruptedException)3 TerminMonitor (com.alibaba.otter.shared.arbitrate.impl.setl.monitor.TerminMonitor)2 Date (java.util.Date)2 ZkBadVersionException (org.I0Itec.zkclient.exception.ZkBadVersionException)2 Stat (org.apache.zookeeper.data.Stat)2 CanalMetaManagerException (com.alibaba.otter.canal.meta.exception.CanalMetaManagerException)1 ClientIdentity (com.alibaba.otter.canal.protocol.ClientIdentity)1 CanalClientException (com.alibaba.otter.canal.protocol.exception.CanalClientException)1 StageComparator (com.alibaba.otter.shared.arbitrate.impl.setl.helper.StageComparator)1