Search in sources :

Example 1 with NonDeterministicFrameReader

use of org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader in project asterixdb by apache.

the class OneToOneConnectorDescriptor method createPartitionCollector.

@Override
public IPartitionCollector createPartitionCollector(IHyracksTaskContext ctx, RecordDescriptor recordDesc, int index, int nProducerPartitions, int nConsumerPartitions) throws HyracksDataException {
    BitSet expectedPartitions = new BitSet(nProducerPartitions);
    expectedPartitions.set(index);
    NonDeterministicChannelReader channelReader = new NonDeterministicChannelReader(nProducerPartitions, expectedPartitions);
    NonDeterministicFrameReader frameReader = new NonDeterministicFrameReader(channelReader);
    return new PartitionCollector(ctx, getConnectorId(), index, expectedPartitions, frameReader, channelReader);
}
Also used : IPartitionCollector(org.apache.hyracks.api.comm.IPartitionCollector) PartitionCollector(org.apache.hyracks.dataflow.std.collectors.PartitionCollector) NonDeterministicFrameReader(org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader) BitSet(java.util.BitSet) NonDeterministicChannelReader(org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader)

Example 2 with NonDeterministicFrameReader

use of org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader in project asterixdb by apache.

the class LocalityAwareMToNPartitioningConnectorDescriptor method createPartitionCollector.

/*
     * (non-Javadoc)
     *
     * @see org.apache.hyracks.api.dataflow.IConnectorDescriptor#
     * createPartitionCollector
     * (org.apache.hyracks.api.context.IHyracksTaskContext,
     * org.apache.hyracks.api.dataflow.value.RecordDescriptor, int, int, int)
     */
@Override
public IPartitionCollector createPartitionCollector(IHyracksTaskContext ctx, RecordDescriptor recordDesc, int receiverIndex, int nProducerPartitions, int nConsumerPartitions) throws HyracksDataException {
    BitSet expectedPartitions = new BitSet(nProducerPartitions);
    for (int i = 0; i < nProducerPartitions; i++) {
        if (localityMap.isConnected(i, receiverIndex, nConsumerPartitions)) {
            expectedPartitions.set(i);
        }
    }
    NonDeterministicChannelReader channelReader = new NonDeterministicChannelReader(nProducerPartitions, expectedPartitions);
    NonDeterministicFrameReader frameReader = new NonDeterministicFrameReader(channelReader);
    return new PartitionCollector(ctx, getConnectorId(), receiverIndex, expectedPartitions, frameReader, channelReader);
}
Also used : IPartitionCollector(org.apache.hyracks.api.comm.IPartitionCollector) PartitionCollector(org.apache.hyracks.dataflow.std.collectors.PartitionCollector) NonDeterministicFrameReader(org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader) BitSet(java.util.BitSet) NonDeterministicChannelReader(org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader)

Example 3 with NonDeterministicFrameReader

use of org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader in project asterixdb by apache.

the class MToNBroadcastConnectorDescriptor method createPartitionCollector.

@Override
public IPartitionCollector createPartitionCollector(IHyracksTaskContext ctx, RecordDescriptor recordDesc, int index, int nProducerPartitions, int nConsumerPartitions) throws HyracksDataException {
    BitSet expectedPartitions = new BitSet(nProducerPartitions);
    expectedPartitions.set(0, nProducerPartitions);
    NonDeterministicChannelReader channelReader = new NonDeterministicChannelReader(nProducerPartitions, expectedPartitions);
    NonDeterministicFrameReader frameReader = new NonDeterministicFrameReader(channelReader);
    return new PartitionCollector(ctx, getConnectorId(), index, expectedPartitions, frameReader, channelReader);
}
Also used : IPartitionCollector(org.apache.hyracks.api.comm.IPartitionCollector) PartitionCollector(org.apache.hyracks.dataflow.std.collectors.PartitionCollector) NonDeterministicFrameReader(org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader) BitSet(java.util.BitSet) NonDeterministicChannelReader(org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader)

Example 4 with NonDeterministicFrameReader

use of org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader in project asterixdb by apache.

the class MToNPartitioningConnectorDescriptor method createPartitionCollector.

@Override
public IPartitionCollector createPartitionCollector(IHyracksTaskContext ctx, RecordDescriptor recordDesc, int index, int nProducerPartitions, int nConsumerPartitions) throws HyracksDataException {
    BitSet expectedPartitions = new BitSet(nProducerPartitions);
    expectedPartitions.set(0, nProducerPartitions);
    NonDeterministicChannelReader channelReader = new NonDeterministicChannelReader(nProducerPartitions, expectedPartitions);
    NonDeterministicFrameReader frameReader = new NonDeterministicFrameReader(channelReader);
    return new PartitionCollector(ctx, getConnectorId(), index, expectedPartitions, frameReader, channelReader);
}
Also used : IPartitionCollector(org.apache.hyracks.api.comm.IPartitionCollector) PartitionCollector(org.apache.hyracks.dataflow.std.collectors.PartitionCollector) NonDeterministicFrameReader(org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader) BitSet(java.util.BitSet) NonDeterministicChannelReader(org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader)

Aggregations

BitSet (java.util.BitSet)4 IPartitionCollector (org.apache.hyracks.api.comm.IPartitionCollector)4 NonDeterministicChannelReader (org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader)4 NonDeterministicFrameReader (org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader)4 PartitionCollector (org.apache.hyracks.dataflow.std.collectors.PartitionCollector)4