Examples with PartitionCollector org.apache.hyracks.dataflow.std.collectors.PartitionCollector used on opensource projects

Example 1 with PartitionCollector

use of org.apache.hyracks.dataflow.std.collectors.PartitionCollector in project asterixdb by apache.

the class MToNPartitioningMergingConnectorDescriptor method createPartitionCollector.

@Override
public IPartitionCollector createPartitionCollector(IHyracksTaskContext ctx, RecordDescriptor recordDesc, int index, int nProducerPartitions, int nConsumerPartitions) throws HyracksDataException {
    IBinaryComparator[] comparators = new IBinaryComparator[comparatorFactories.length];
    for (int i = 0; i < comparatorFactories.length; ++i) {
        comparators[i] = comparatorFactories[i].createBinaryComparator();
    }
    INormalizedKeyComputer nmkComputer = nkcFactory == null ? null : nkcFactory.createNormalizedKeyComputer();
    IPartitionBatchManager pbm = new NonDeterministicPartitionBatchManager(nProducerPartitions);
    IFrameReader sortMergeFrameReader = new SortMergeFrameReader(ctx, nProducerPartitions, nProducerPartitions, sortFields, comparators, nmkComputer, recordDesc, pbm);
    BitSet expectedPartitions = new BitSet();
    expectedPartitions.set(0, nProducerPartitions);
    return new PartitionCollector(ctx, getConnectorId(), index, expectedPartitions, sortMergeFrameReader, pbm);
}

Example 2 with PartitionCollector

use of org.apache.hyracks.dataflow.std.collectors.PartitionCollector in project asterixdb by apache.

the class OneToOneConnectorDescriptor method createPartitionCollector.

@Override
public IPartitionCollector createPartitionCollector(IHyracksTaskContext ctx, RecordDescriptor recordDesc, int index, int nProducerPartitions, int nConsumerPartitions) throws HyracksDataException {
    BitSet expectedPartitions = new BitSet(nProducerPartitions);
    expectedPartitions.set(index);
    NonDeterministicChannelReader channelReader = new NonDeterministicChannelReader(nProducerPartitions, expectedPartitions);
    NonDeterministicFrameReader frameReader = new NonDeterministicFrameReader(channelReader);
    return new PartitionCollector(ctx, getConnectorId(), index, expectedPartitions, frameReader, channelReader);
}

Example 3 with PartitionCollector

use of org.apache.hyracks.dataflow.std.collectors.PartitionCollector in project asterixdb by apache.

the class LocalityAwareMToNPartitioningConnectorDescriptor method createPartitionCollector.

/*
     * (non-Javadoc)
     *
     * @see org.apache.hyracks.api.dataflow.IConnectorDescriptor#
     * createPartitionCollector
     * (org.apache.hyracks.api.context.IHyracksTaskContext,
     * org.apache.hyracks.api.dataflow.value.RecordDescriptor, int, int, int)
     */
@Override
public IPartitionCollector createPartitionCollector(IHyracksTaskContext ctx, RecordDescriptor recordDesc, int receiverIndex, int nProducerPartitions, int nConsumerPartitions) throws HyracksDataException {
    BitSet expectedPartitions = new BitSet(nProducerPartitions);
    for (int i = 0; i < nProducerPartitions; i++) {
        if (localityMap.isConnected(i, receiverIndex, nConsumerPartitions)) {
            expectedPartitions.set(i);
        }
    }
    NonDeterministicChannelReader channelReader = new NonDeterministicChannelReader(nProducerPartitions, expectedPartitions);
    NonDeterministicFrameReader frameReader = new NonDeterministicFrameReader(channelReader);
    return new PartitionCollector(ctx, getConnectorId(), receiverIndex, expectedPartitions, frameReader, channelReader);
}

Example 4 with PartitionCollector

use of org.apache.hyracks.dataflow.std.collectors.PartitionCollector in project asterixdb by apache.

the class MToNBroadcastConnectorDescriptor method createPartitionCollector.

@Override
public IPartitionCollector createPartitionCollector(IHyracksTaskContext ctx, RecordDescriptor recordDesc, int index, int nProducerPartitions, int nConsumerPartitions) throws HyracksDataException {
    BitSet expectedPartitions = new BitSet(nProducerPartitions);
    expectedPartitions.set(0, nProducerPartitions);
    NonDeterministicChannelReader channelReader = new NonDeterministicChannelReader(nProducerPartitions, expectedPartitions);
    NonDeterministicFrameReader frameReader = new NonDeterministicFrameReader(channelReader);
    return new PartitionCollector(ctx, getConnectorId(), index, expectedPartitions, frameReader, channelReader);
}

Example 5 with PartitionCollector

use of org.apache.hyracks.dataflow.std.collectors.PartitionCollector in project asterixdb by apache.

the class MToNPartitioningConnectorDescriptor method createPartitionCollector.

@Override
public IPartitionCollector createPartitionCollector(IHyracksTaskContext ctx, RecordDescriptor recordDesc, int index, int nProducerPartitions, int nConsumerPartitions) throws HyracksDataException {
    BitSet expectedPartitions = new BitSet(nProducerPartitions);
    expectedPartitions.set(0, nProducerPartitions);
    NonDeterministicChannelReader channelReader = new NonDeterministicChannelReader(nProducerPartitions, expectedPartitions);
    NonDeterministicFrameReader frameReader = new NonDeterministicFrameReader(channelReader);
    return new PartitionCollector(ctx, getConnectorId(), index, expectedPartitions, frameReader, channelReader);
}