Examples with IHdfsFile com.dtstack.dtcenter.loader.client.IHdfsFile used on opensource projects

Example 11 with IHdfsFile

use of com.dtstack.dtcenter.loader.client.IHdfsFile in project Taier by DTStack.

the class HdfsOperator method checkAndDele.

/**
 * 文件检测并删除,递归删除
 *
 * @return
 */
public static boolean checkAndDele(Map<String, Object> conf, Map<String, Object> kerberos, String fileUri) {
    HdfsSourceDTO sourceDTO = getSourceDTO(conf, kerberos);
    IHdfsFile hdfsClient = ClientCache.getHdfs(DataSourceType.HDFS.getVal());
    return hdfsClient.checkAndDelete(sourceDTO, fileUri);
}

Example 12 with IHdfsFile

use of com.dtstack.dtcenter.loader.client.IHdfsFile in project Taier by DTStack.

the class HdfsOperator method createDir.

public static boolean createDir(Map<String, Object> conf, Map<String, Object> kerberos, String dir, short permission) {
    HdfsSourceDTO sourceDTO = getSourceDTO(conf, kerberos);
    IHdfsFile hdfsClient = ClientCache.getHdfs(DataSourceType.HDFS.getVal());
    dir = uri(dir);
    return hdfsClient.createDir(sourceDTO, dir, permission);
}

Example 13 with IHdfsFile

use of com.dtstack.dtcenter.loader.client.IHdfsFile in project Taier by DTStack.

the class HdfsOperator method getContentSummary.

/**
 * 获取hdfs上指定路径集合的ContentSummary
 *
 * @param conf
 * @param kerberos
 * @param targetPaths
 * @return
 */
public static List<HDFSContentSummary> getContentSummary(Map<String, Object> conf, Map<String, Object> kerberos, List<String> targetPaths) {
    try {
        HdfsSourceDTO sourceDTO = getSourceDTO(conf, kerberos);
        IHdfsFile hdfsClient = ClientCache.getHdfs(DataSourceType.HDFS.getVal());
        return hdfsClient.getContentSummary(sourceDTO, targetPaths);
    } catch (Exception e) {
        throw new DtCenterDefException(String.format("获取hdfs文件内容摘要异常!原因是：%s", e.getMessage()));
    }
}

Example 14 with IHdfsFile

use of com.dtstack.dtcenter.loader.client.IHdfsFile in project Taier by DTStack.

the class HdfsOperator method deleteFiles.

public static boolean deleteFiles(Map<String, Object> conf, Map<String, Object> kerberos, List<String> fileNames) {
    HdfsSourceDTO sourceDTO = getSourceDTO(conf, kerberos);
    IHdfsFile hdfsClient = ClientCache.getHdfs(DataSourceType.HDFS.getVal());
    return hdfsClient.deleteFiles(sourceDTO, fileNames);
}

Example 15 with IHdfsFile

use of com.dtstack.dtcenter.loader.client.IHdfsFile in project Taier by DTStack.

the class HdfsOperator method isFileExist.

public static boolean isFileExist(Map<String, Object> conf, Map<String, Object> kerberos, String dir) {
    HdfsSourceDTO sourceDTO = getSourceDTO(conf, kerberos);
    IHdfsFile hdfsClient = ClientCache.getHdfs(DataSourceType.HDFS.getVal());
    dir = uri(dir);
    return hdfsClient.isFileExist(sourceDTO, dir);
}