Examples with HdfsParquetDataSource com.huawei.boostkit.omnidata.model.datasource.hdfs.HdfsParquetDataSource used on opensource projects

Example 1 with HdfsParquetDataSource

use of com.huawei.boostkit.omnidata.model.datasource.hdfs.HdfsParquetDataSource in project boostkit-bigdata by kunpengcompute.

the class DataIoAdapter method initDataSource.

private DataSource initDataSource(PageCandidate pageCandidate) throws UnsupportedOperationException {
    DataSource dataSource;
    String fileFormat = pageCandidate.getFileFormat();
    Long fileStartPos = pageCandidate.getStartPos();
    Long fileLen = pageCandidate.getSplitLen();
    if ("ORC".equalsIgnoreCase(fileFormat)) {
        dataSource = new HdfsOrcDataSource(filePath, fileStartPos, fileLen, false);
    } else if ("PARQUET".equalsIgnoreCase(fileFormat)) {
        dataSource = new HdfsParquetDataSource(filePath, fileStartPos, fileLen, false);
    } else {
        throw new UnsupportedOperationException("unsupported data format : " + fileFormat);
    }
    return dataSource;
}