Search in sources :

Example 61 with VSizeFrame

use of org.apache.hyracks.api.comm.VSizeFrame in project asterixdb by apache.

the class SortMergeFrameReader method open.

@Override
public void open() throws HyracksDataException {
    if (maxConcurrentMerges >= nSenders) {
        List<IFrame> inFrames = new ArrayList<>(nSenders);
        for (int i = 0; i < nSenders; ++i) {
            inFrames.add(new VSizeFrame(ctx));
        }
        List<IFrameReader> batch = new ArrayList<IFrameReader>(nSenders);
        pbm.getNextBatch(batch, nSenders);
        merger = new RunMergingFrameReader(ctx, batch, inFrames, sortFields, comparators, nmkComputer, recordDescriptor);
    } else {
        // multi level merge.
        throw new HyracksDataException("Not yet supported");
    }
    merger.open();
}
Also used : RunMergingFrameReader(org.apache.hyracks.dataflow.std.sort.RunMergingFrameReader) IFrameReader(org.apache.hyracks.api.comm.IFrameReader) IFrame(org.apache.hyracks.api.comm.IFrame) ArrayList(java.util.ArrayList) VSizeFrame(org.apache.hyracks.api.comm.VSizeFrame) HyracksDataException(org.apache.hyracks.api.exceptions.HyracksDataException)

Example 62 with VSizeFrame

use of org.apache.hyracks.api.comm.VSizeFrame in project asterixdb by apache.

the class FrameFixedFieldTupleAppenderTest method createAppender.

@Before
public void createAppender() throws HyracksDataException {
    appender = new FrameFixedFieldAppender(fields.length);
    FrameManager manager = new FrameManager(TEST_FRAME_SIZE);
    IFrame frame = new VSizeFrame(manager);
    appender.reset(frame, true);
}
Also used : IFrame(org.apache.hyracks.api.comm.IFrame) FrameFixedFieldAppender(org.apache.hyracks.dataflow.common.comm.io.FrameFixedFieldAppender) FrameManager(org.apache.hyracks.control.nc.resources.memory.FrameManager) VSizeFrame(org.apache.hyracks.api.comm.VSizeFrame) Before(org.junit.Before)

Example 63 with VSizeFrame

use of org.apache.hyracks.api.comm.VSizeFrame in project asterixdb by apache.

the class FrameFixedFieldTupleAppenderTest method prepareData.

private IFrameTupleAccessor prepareData(DATA_TYPE type) throws HyracksDataException {
    IFrameTupleAccessor accessor = new FrameTupleAccessor(recordDescriptor);
    IFrameTupleAppender appender = new FrameTupleAppender(new VSizeFrame(new FrameManager(INPUT_BUFFER_SIZE)), true);
    int i = 0;
    do {
        switch(type) {
            case NORMAL_RECORD:
                makeATuple(tupleBuilder, i++);
                break;
            case ONE_FIELD_LONG:
                makeASizeUpTuple(tupleBuilder, i++);
                break;
            case ONE_RECORD_LONG:
                makeABigObjectTuple(tupleBuilder, i++);
                break;
        }
    } while (appender.append(tupleBuilder.getFieldEndOffsets(), tupleBuilder.getByteArray(), 0, tupleBuilder.getSize()));
    accessor.reset(appender.getBuffer());
    return accessor;
}
Also used : IFrameTupleAccessor(org.apache.hyracks.api.comm.IFrameTupleAccessor) FrameTupleAppender(org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppender) IFrameTupleAppender(org.apache.hyracks.api.comm.IFrameTupleAppender) FrameManager(org.apache.hyracks.control.nc.resources.memory.FrameManager) IFrameTupleAppender(org.apache.hyracks.api.comm.IFrameTupleAppender) VSizeFrame(org.apache.hyracks.api.comm.VSizeFrame) FrameTupleAccessor(org.apache.hyracks.dataflow.common.comm.io.FrameTupleAccessor) IFrameTupleAccessor(org.apache.hyracks.api.comm.IFrameTupleAccessor)

Aggregations

VSizeFrame (org.apache.hyracks.api.comm.VSizeFrame)63 FrameTupleAppender (org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppender)32 HyracksDataException (org.apache.hyracks.api.exceptions.HyracksDataException)22 FrameTupleAccessor (org.apache.hyracks.dataflow.common.comm.io.FrameTupleAccessor)18 IHyracksTaskContext (org.apache.hyracks.api.context.IHyracksTaskContext)17 ArrayTupleBuilder (org.apache.hyracks.dataflow.common.comm.io.ArrayTupleBuilder)16 Test (org.junit.Test)16 IFrame (org.apache.hyracks.api.comm.IFrame)13 ArrayList (java.util.ArrayList)11 DataOutput (java.io.DataOutput)10 IFrameTupleAccessor (org.apache.hyracks.api.comm.IFrameTupleAccessor)9 FrameTupleReference (org.apache.hyracks.dataflow.common.data.accessors.FrameTupleReference)9 IOException (java.io.IOException)8 IFrameWriter (org.apache.hyracks.api.comm.IFrameWriter)8 RecordDescriptor (org.apache.hyracks.api.dataflow.value.RecordDescriptor)8 ConcurrentFramePool (org.apache.asterix.common.memory.ConcurrentFramePool)7 FeedRuntimeInputHandler (org.apache.asterix.external.feed.dataflow.FeedRuntimeInputHandler)7 FeedPolicyAccessor (org.apache.asterix.external.feed.policy.FeedPolicyAccessor)7 TestFrameWriter (org.apache.hyracks.api.test.TestFrameWriter)7 ISerializerDeserializer (org.apache.hyracks.api.dataflow.value.ISerializerDeserializer)6