Examples with ResourcePool org.apache.gobblin.util.request_allocation.ResourcePool used on opensource projects

Example 1 with ResourcePool

use of org.apache.gobblin.util.request_allocation.ResourcePool in project incubator-gobblin by apache.

the class CompactionSource method prioritize.

private List<Dataset> prioritize(List<Dataset> datasets, State state) {
    double maxPool = state.getPropAsDouble(MRCompactor.COMPACTION_DATASETS_MAX_COUNT, MRCompactor.DEFUALT_COMPACTION_DATASETS_MAX_COUNT);
    ResourcePool pool = ResourcePool.builder().maxResource(SimpleDatasetRequest.SIMPLE_DATASET_COUNT_DIMENSION, maxPool).build();
    Iterator<Dataset> newList = Iterators.transform(this.allocator.allocateRequests(datasets.stream().map(SimpleDatasetRequestor::new).iterator(), pool), (input) -> input.getDataset());
    return Lists.newArrayList(newList);
}