Examples with VirtualRoute com.bonree.brfs.common.rebalance.route.VirtualRoute used on opensource projects

Search in sources :

Example 6 with VirtualRoute

use of com.bonree.brfs.common.rebalance.route.VirtualRoute in project BRFS by zhangnianli.

the class SecondIDParser method updateRoute.

public void updateRoute() {
    // load virtual id
    String virtualPath = baseRoutesPath + Constants.SEPARATOR + Constants.VIRTUAL_ROUTE + Constants.SEPARATOR + snID;
    List<String> virtualNodes = curatorClient.getChildren(virtualPath);
    if (curatorClient.checkExists(virtualPath)) {
        if (virtualNodes != null && !virtualNodes.isEmpty()) {
            for (String virtualNode : virtualNodes) {
                String dataPath = virtualPath + Constants.SEPARATOR + virtualNode;
                byte[] data = curatorClient.getData(dataPath);
                VirtualRoute virtual = JsonUtils.toObjectQuietly(data, VirtualRoute.class);
                virtualRouteDetail.put(virtual.getVirtualID(), virtual);
            }
        }
    }
    // load normal id
    String normalPath = baseRoutesPath + Constants.SEPARATOR + Constants.NORMAL_ROUTE + Constants.SEPARATOR + snID;
    if (curatorClient.checkExists(normalPath)) {
        List<String> normalNodes = curatorClient.getChildren(normalPath);
        if (normalNodes != null && !normalNodes.isEmpty()) {
            for (String normalNode : normalNodes) {
                String dataPath = normalPath + Constants.SEPARATOR + normalNode;
                byte[] data = curatorClient.getData(dataPath);
                NormalRoute normal = JsonUtils.toObjectQuietly(data, NormalRoute.class);
                normalRouteDetail.put(normal.getSecondID(), normal);
            }
        }
    }
}
Also used : VirtualRoute(com.bonree.brfs.common.rebalance.route.VirtualRoute) NormalRoute(com.bonree.brfs.common.rebalance.route.NormalRoute)

Example 7 with VirtualRoute

use of com.bonree.brfs.common.rebalance.route.VirtualRoute in project BRFS by zhangnianli.

the class MultiRecover3 method dealFile.

private void dealFile(String perFile, String fileName, String timeFileName, int replica) {
    // 对文件名进行分割处理
    String[] metaArr = fileName.split(NAME_SEPARATOR);
    // 提取出用于hash的部分
    String namePart = metaArr[0];
    // 整理2级serverID，转换虚拟serverID
    List<String> fileServerIds = new ArrayList<>();
    for (int j = 1; j < metaArr.length; j++) {
        if (metaArr[j].charAt(0) == Constants.VIRTUAL_ID) {
            VirtualRoute virtualRoute = virtualRoutes.get(metaArr[j]);
            if (virtualRoute != null) {
                fileServerIds.add(virtualRoute.getNewSecondID());
            } else {
                LOG.error("file:" + perFile + "is exeception!!");
                return;
            }
        } else {
            fileServerIds.add(metaArr[j]);
        }
    }
    // 这里要判断一个副本是否需要进行迁移
    // 挑选出的可迁移的servers
    String selectMultiId = null;
    // 可获取的server，可能包括自身
    List<String> recoverableServerList = null;
    // 排除掉自身或已有的servers
    List<String> exceptionServerIds = null;
    // 真正可选择的servers
    List<String> selectableServerList = null;
    while (RebalanceUtils.needRecover(fileServerIds, replica, getAliveMultiIds())) {
        for (String deadServer : fileServerIds) {
            if (!getAliveMultiIds().contains(deadServer)) {
                LOG.info("deadServer:" + deadServer);
                int pot = fileServerIds.indexOf(deadServer);
                if (!StringUtils.equals(deadServer, balanceSummary.getServerId())) {
                    recoverableServerList = getRecoverRoleList(deadServer);
                } else {
                    recoverableServerList = balanceSummary.getInputServers();
                }
                LOG.info("recoverableServerList:" + recoverableServerList);
                exceptionServerIds = new ArrayList<>();
                exceptionServerIds.addAll(fileServerIds);
                exceptionServerIds.remove(deadServer);
                selectableServerList = getSelectedList(recoverableServerList, exceptionServerIds);
                int index = RebalanceUtils.hashFileName(namePart, selectableServerList.size());
                selectMultiId = selectableServerList.get(index);
                fileServerIds.set(pot, selectMultiId);
                // 判断选取的新节点是否存活
                if (isAlive(getAliveMultiIds(), selectMultiId)) {
                    // 判断选取的新节点是否为本节点
                    if (!idManager.getSecondServerID(balanceSummary.getStorageIndex()).equals(selectMultiId)) {
                        String firstID = idManager.getOtherFirstID(selectMultiId, balanceSummary.getStorageIndex());
                        FileRecoverMeta fileMeta = new FileRecoverMeta(perFile, fileName, selectMultiId, timeFileName, replica, pot + 1, firstID);
                        try {
                            fileRecoverQueue.put(fileMeta);
                        } catch (InterruptedException e) {
                            LOG.error("put file: " + fileMeta, e);
                        }
                    }
                }
            }
        }
    }
}
Also used : VirtualRoute(com.bonree.brfs.common.rebalance.route.VirtualRoute) ArrayList(java.util.ArrayList)

Example 8 with VirtualRoute

use of com.bonree.brfs.common.rebalance.route.VirtualRoute in project BRFS by zhangnianli.

the class MultiRecover3 method loadVirualRoutes.

public void loadVirualRoutes() {
    // load virtual id
    String virtualPath = baseRoutesPath + Constants.SEPARATOR + Constants.VIRTUAL_ROUTE + Constants.SEPARATOR + balanceSummary.getStorageIndex();
    List<String> virtualNodes = client.getChildren(virtualPath);
    if (client.checkExists(virtualPath)) {
        if (virtualNodes != null && !virtualNodes.isEmpty()) {
            for (String virtualNode : virtualNodes) {
                String dataPath = virtualPath + Constants.SEPARATOR + virtualNode;
                byte[] data = client.getData(dataPath);
                VirtualRoute virtual = JsonUtils.toObjectQuietly(data, VirtualRoute.class);
                virtualRoutes.put(virtual.getVirtualID(), virtual);
            }
        }
    }
    LOG.info("virtual routes:" + virtualRoutes);
    // load normal id
    String normalPath = baseRoutesPath + Constants.SEPARATOR + Constants.NORMAL_ROUTE + Constants.SEPARATOR + balanceSummary.getStorageIndex();
    if (client.checkExists(normalPath)) {
        List<String> normalNodes = client.getChildren(normalPath);
        if (normalNodes != null && !normalNodes.isEmpty()) {
            for (String normalNode : normalNodes) {
                String dataPath = normalPath + Constants.SEPARATOR + normalNode;
                byte[] data = client.getData(dataPath);
                NormalRoute normal = JsonUtils.toObjectQuietly(data, NormalRoute.class);
                normalRoutes.put(normal.getSecondID(), normal);
            }
        }
    }
    LOG.info("normal routes:" + normalRoutes);
}
Also used : VirtualRoute(com.bonree.brfs.common.rebalance.route.VirtualRoute) NormalRoute(com.bonree.brfs.common.rebalance.route.NormalRoute)

Example 9 with VirtualRoute

use of com.bonree.brfs.common.rebalance.route.VirtualRoute in project BRFS by zhangnianli.

the class MultiRecover method loadVirualRoutes.

public void loadVirualRoutes() {
    // load virtual id
    String virtualPath = baseRoutesPath + Constants.SEPARATOR + Constants.VIRTUAL_ROUTE + Constants.SEPARATOR + balanceSummary.getStorageIndex();
    List<String> virtualNodes = client.getChildren(virtualPath);
    if (client.checkExists(virtualPath)) {
        if (virtualNodes != null && !virtualNodes.isEmpty()) {
            for (String virtualNode : virtualNodes) {
                String dataPath = virtualPath + Constants.SEPARATOR + virtualNode;
                byte[] data = client.getData(dataPath);
                VirtualRoute virtual = JsonUtils.toObjectQuietly(data, VirtualRoute.class);
                virtualRoutes.put(virtual.getVirtualID(), virtual);
            }
        }
    }
    LOG.info("virtual routes:" + virtualRoutes);
    // load normal id
    String normalPath = baseRoutesPath + Constants.SEPARATOR + Constants.NORMAL_ROUTE + Constants.SEPARATOR + balanceSummary.getStorageIndex();
    if (client.checkExists(normalPath)) {
        List<String> normalNodes = client.getChildren(normalPath);
        if (normalNodes != null && !normalNodes.isEmpty()) {
            for (String normalNode : normalNodes) {
                String dataPath = normalPath + Constants.SEPARATOR + normalNode;
                byte[] data = client.getData(dataPath);
                NormalRoute normal = JsonUtils.toObjectQuietly(data, NormalRoute.class);
                normalRoutes.put(normal.getSecondID(), normal);
            }
        }
    }
    LOG.info("normal routes:" + normalRoutes);
}
Also used : VirtualRoute(com.bonree.brfs.common.rebalance.route.VirtualRoute) NormalRoute(com.bonree.brfs.common.rebalance.route.NormalRoute)

Example 10 with VirtualRoute

use of com.bonree.brfs.common.rebalance.route.VirtualRoute in project BRFS by zhangnianli.

the class MultiRecover method dealFile.

private void dealFile(String perFile, String fileName, String timeFileName, int replica) {
    // 对文件名进行分割处理
    String[] metaArr = fileName.split(NAME_SEPARATOR);
    // 提取出用于hash的部分
    String namePart = metaArr[0];
    // 整理2级serverID，转换虚拟serverID
    List<String> fileServerIds = new ArrayList<>();
    for (int j = 1; j < metaArr.length; j++) {
        if (metaArr[j].charAt(0) == Constants.VIRTUAL_ID) {
            VirtualRoute virtualRoute = virtualRoutes.get(metaArr[j]);
            if (virtualRoute != null) {
                fileServerIds.add(virtualRoute.getNewSecondID());
            } else {
                LOG.error("file:" + perFile + "is exeception!!");
                return;
            }
        } else {
            fileServerIds.add(metaArr[j]);
        }
    }
    // 这里要判断一个副本是否需要进行迁移
    // 挑选出的可迁移的servers
    String selectMultiId = null;
    // 可获取的server，可能包括自身
    List<String> recoverableServerList = null;
    // 排除掉自身或已有的servers
    List<String> exceptionServerIds = null;
    // 真正可选择的servers
    List<String> selectableServerList = null;
    while (RebalanceUtils.needRecover(fileServerIds, replica, getAliveMultiIds())) {
        for (String deadServer : fileServerIds) {
            if (!getAliveMultiIds().contains(deadServer)) {
                LOG.info("deadServer:" + deadServer);
                int pot = fileServerIds.indexOf(deadServer);
                if (!StringUtils.equals(deadServer, balanceSummary.getServerId())) {
                    recoverableServerList = getRecoverRoleList(deadServer);
                } else {
                    recoverableServerList = balanceSummary.getInputServers();
                }
                LOG.info("recoverableServerList:" + recoverableServerList);
                exceptionServerIds = new ArrayList<>();
                exceptionServerIds.addAll(fileServerIds);
                exceptionServerIds.remove(deadServer);
                selectableServerList = getSelectedList(recoverableServerList, exceptionServerIds);
                int index = RebalanceUtils.hashFileName(namePart, selectableServerList.size());
                selectMultiId = selectableServerList.get(index);
                fileServerIds.set(pot, selectMultiId);
                // 判断选取的新节点是否存活
                if (isAlive(getAliveMultiIds(), selectMultiId)) {
                    // 判断选取的新节点是否为本节点
                    if (!idManager.getSecondServerID(balanceSummary.getStorageIndex()).equals(selectMultiId)) {
                        String firstID = idManager.getOtherFirstID(selectMultiId, balanceSummary.getStorageIndex());
                        FileRecoverMeta fileMeta = new FileRecoverMeta(perFile, fileName, selectMultiId, timeFileName, replica, pot + 1, firstID);
                        try {
                            fileRecoverQueue.put(fileMeta);
                        } catch (InterruptedException e) {
                            LOG.error("put file: " + fileMeta, e);
                        }
                    }
                }
            }
        }
    }
}
Also used : VirtualRoute(com.bonree.brfs.common.rebalance.route.VirtualRoute) ArrayList(java.util.ArrayList)

Aggregations

VirtualRoute (com.bonree.brfs.common.rebalance.route.VirtualRoute)11 NormalRoute (com.bonree.brfs.common.rebalance.route.NormalRoute)9 ArrayList (java.util.ArrayList)4 CuratorClient (com.bonree.brfs.common.zookeeper.curator.CuratorClient)1 List (java.util.List)1
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial