Examples with BlockIdList alluxio.grpc.BlockIdList used on opensource projects

Example 6 with BlockIdList

use of alluxio.grpc.BlockIdList in project alluxio by Alluxio.

the class BlockMasterClient method convertBlockListMapToProto.

/**
 * Converts the block list map to a proto list.
 * Because the list is flattened from a map, in the list no two {@link LocationBlockIdListEntry}
 * instances shall have the same {@link BlockStoreLocationProto}.
 * The uniqueness of {@link BlockStoreLocationProto} is determined by tier alias and medium type.
 * That means directories with the same tier alias and medium type will be merged into the same
 * {@link LocationBlockIdListEntry}.
 *
 * @param blockListOnLocation a map from block location to the block list
 * @return a flattened and deduplicated list
 */
@VisibleForTesting
public List<LocationBlockIdListEntry> convertBlockListMapToProto(Map<BlockStoreLocation, List<Long>> blockListOnLocation) {
    final List<LocationBlockIdListEntry> entryList = new ArrayList<>();
    Map<BlockStoreLocationProto, List<Long>> tierToBlocks = new HashMap<>();
    for (Map.Entry<BlockStoreLocation, List<Long>> entry : blockListOnLocation.entrySet()) {
        BlockStoreLocation loc = entry.getKey();
        BlockStoreLocationProto locationProto = BlockStoreLocationProto.newBuilder().setTierAlias(loc.tierAlias()).setMediumType(loc.mediumType()).build();
        if (tierToBlocks.containsKey(locationProto)) {
            tierToBlocks.get(locationProto).addAll(entry.getValue());
        } else {
            List<Long> blockList = new ArrayList<>(entry.getValue());
            tierToBlocks.put(locationProto, blockList);
        }
    }
    for (Map.Entry<BlockStoreLocationProto, List<Long>> entry : tierToBlocks.entrySet()) {
        BlockIdList blockIdList = BlockIdList.newBuilder().addAllBlockId(entry.getValue()).build();
        LocationBlockIdListEntry listEntry = LocationBlockIdListEntry.newBuilder().setKey(entry.getKey()).setValue(blockIdList).build();
        entryList.add(listEntry);
    }
    return entryList;
}

Example 7 with BlockIdList

use of alluxio.grpc.BlockIdList in project alluxio by Alluxio.

the class BlockMapIterator method nextBatchFromTier.

private LocationBlockIdListEntry nextBatchFromTier(BlockStoreLocationProto currentLoc, Iterator<Long> currentIterator, int spaceLeft) {
    // Generate the next batch
    List<Long> blockIdBatch = new ArrayList<>(spaceLeft);
    while (blockIdBatch.size() < spaceLeft && currentIterator.hasNext()) {
        blockIdBatch.add(currentIterator.next());
        mCounter++;
    }
    BlockIdList blockIdList = BlockIdList.newBuilder().addAllBlockId(blockIdBatch).build();
    LocationBlockIdListEntry listEntry = LocationBlockIdListEntry.newBuilder().setKey(currentLoc).setValue(blockIdList).build();
    return listEntry;
}