Search in sources :

Example 6 with IntList

use of it.unimi.dsi.fastutil.ints.IntList in project presto by prestodb.

the class PartitioningExchanger method accept.

@Override
public synchronized void accept(Page page) {
    // reset the assignment lists
    for (IntList partitionAssignment : partitionAssignments) {
        partitionAssignment.clear();
    }
    // assign each row to a partition
    for (int position = 0; position < page.getPositionCount(); position++) {
        int partition = partitionGenerator.getPartition(position, page);
        partitionAssignments[partition].add(position);
    }
    // build a page for each partition
    Block[] sourceBlocks = page.getBlocks();
    Block[] outputBlocks = new Block[sourceBlocks.length];
    for (int partition = 0; partition < buffers.size(); partition++) {
        List<Integer> positions = partitionAssignments[partition];
        if (!positions.isEmpty()) {
            for (int i = 0; i < sourceBlocks.length; i++) {
                outputBlocks[i] = sourceBlocks[i].copyPositions(positions);
            }
            Page pageSplit = new Page(positions.size(), outputBlocks);
            memoryTracker.accept(pageSplit.getRetainedSizeInBytes());
            buffers.get(partition).accept(new PageReference(pageSplit, 1, () -> memoryTracker.accept(-pageSplit.getRetainedSizeInBytes())));
        }
    }
}
Also used : Block(com.facebook.presto.spi.block.Block) Page(com.facebook.presto.spi.Page) IntList(it.unimi.dsi.fastutil.ints.IntList)

Aggregations

IntList (it.unimi.dsi.fastutil.ints.IntList)6 IntArrayList (it.unimi.dsi.fastutil.ints.IntArrayList)3 Block (com.facebook.presto.spi.block.Block)2 Test (org.junit.Test)2 Page (com.facebook.presto.spi.Page)1 ArrayBlock (com.facebook.presto.spi.block.ArrayBlock)1 InterleavedBlock (com.facebook.presto.spi.block.InterleavedBlock)1 RunLengthEncodedBlock (com.facebook.presto.spi.block.RunLengthEncodedBlock)1 Type (com.facebook.presto.spi.type.Type)1 ArrayList (java.util.ArrayList)1 ThreadLocalRandom (java.util.concurrent.ThreadLocalRandom)1 MessageType (parquet.schema.MessageType)1