Examples with BufferPusher org.apache.hadoop.mapred.nativetask.handlers.BufferPusher used on opensource projects

Example 1 with BufferPusher

use of org.apache.hadoop.mapred.nativetask.handlers.BufferPusher in project hadoop by apache.

the class TestBufferPushPull method testPush.

@Test
public void testPush() throws Exception {
    final byte[] buff = new byte[BUFFER_LENGTH];
    final InputBuffer input = new InputBuffer(buff);
    final OutputBuffer out = new OutputBuffer(buff);
    final Class<BytesWritable> iKClass = BytesWritable.class;
    final Class<BytesWritable> iVClass = BytesWritable.class;
    final RecordWriterForPush writer = new RecordWriterForPush() {

        @Override
        public void write(BytesWritable key, BytesWritable value) throws IOException {
            final KV expect = dataInput[count++];
            Assert.assertEquals(expect.key.toString(), key.toString());
            Assert.assertEquals(expect.value.toString(), value.toString());
        }
    };
    final BufferPushee pushee = new BufferPushee(iKClass, iVClass, writer);
    final PushTarget handler = new PushTarget(out) {

        @Override
        public void sendData() throws IOException {
            final int outputLength = out.length();
            input.rewind(0, outputLength);
            out.rewind();
            pushee.collect(input);
        }
    };
    final BufferPusher pusher = new BufferPusher(iKClass, iVClass, handler);
    writer.reset();
    for (int i = 0; i < INPUT_KV_COUNT; i++) {
        pusher.collect(dataInput[i].key, dataInput[i].value);
    }
    pusher.close();
    pushee.close();
}