Examples with DataMapDistributable org.apache.carbondata.core.datamap.DataMapDistributable used on opensource projects

Example 1 with DataMapDistributable

use of org.apache.carbondata.core.datamap.DataMapDistributable in project carbondata by apache.

the class DataMapExprWrapperImpl method toDistributable.

@Override
public List<DataMapDistributableWrapper> toDistributable(List<Segment> segments) throws IOException {
    List<DataMapDistributable> dataMapDistributables = dataMap.toDistributable(segments);
    List<DataMapDistributableWrapper> wrappers = new ArrayList<>();
    for (DataMapDistributable distributable : dataMapDistributables) {
        wrappers.add(new DataMapDistributableWrapper(uniqueId, distributable));
    }
    return wrappers;
}

Example 2 with DataMapDistributable

use of org.apache.carbondata.core.datamap.DataMapDistributable in project carbondata by apache.

the class BlockletDataMapFactory method toDistributable.

@Override
public List<DataMapDistributable> toDistributable(Segment segment) {
    List<DataMapDistributable> distributables = new ArrayList<>();
    try {
        CarbonFile[] carbonIndexFiles;
        if (segment.getSegmentFileName() == null) {
            carbonIndexFiles = SegmentIndexFileStore.getCarbonIndexFiles(CarbonTablePath.getSegmentPath(identifier.getTablePath(), segment.getSegmentNo()));
        } else {
            SegmentFileStore fileStore = new SegmentFileStore(identifier.getTablePath(), segment.getSegmentFileName());
            Map<String, String> indexFiles = fileStore.getIndexFiles();
            carbonIndexFiles = new CarbonFile[indexFiles.size()];
            int i = 0;
            for (String indexFile : indexFiles.keySet()) {
                carbonIndexFiles[i++] = FileFactory.getCarbonFile(indexFile);
            }
        }
        for (int i = 0; i < carbonIndexFiles.length; i++) {
            Path path = new Path(carbonIndexFiles[i].getPath());
            FileSystem fs = path.getFileSystem(FileFactory.getConfiguration());
            RemoteIterator<LocatedFileStatus> iter = fs.listLocatedStatus(path);
            LocatedFileStatus fileStatus = iter.next();
            String[] location = fileStatus.getBlockLocations()[0].getHosts();
            BlockletDataMapDistributable distributable = new BlockletDataMapDistributable(path.toString());
            distributable.setLocations(location);
            distributables.add(distributable);
        }
    } catch (IOException e) {
        throw new RuntimeException(e);
    }
    return distributables;
}

Example 3 with DataMapDistributable

use of org.apache.carbondata.core.datamap.DataMapDistributable in project carbondata by apache.

the class LuceneDataMapFactoryBase method toDistributable.

/**
 * Get all distributable objects of a segmentid
 */
@Override
public List<DataMapDistributable> toDistributable(Segment segment) {
    List<DataMapDistributable> lstDataMapDistribute = new ArrayList<DataMapDistributable>();
    DataMapDistributable luceneDataMapDistributable = new LuceneDataMapDistributable(CarbonTablePath.getSegmentPath(tableIdentifier.getTablePath(), segment.getSegmentNo()));
    lstDataMapDistribute.add(luceneDataMapDistributable);
    return lstDataMapDistribute;
}