Search in sources :

Example 21 with UniformBinaryRowGenerator

use of org.apache.flink.table.runtime.util.UniformBinaryRowGenerator in project flink by apache.

the class BinaryHashTableTest method testHashWithBuildSideOuterJoin1.

@Test
public void testHashWithBuildSideOuterJoin1() throws Exception {
    final int numKeys = 20000;
    final int buildValsPerKey = 1;
    final int probeValsPerKey = 1;
    // create a build input that gives 40000 pairs with 1 values sharing the same key
    MutableObjectIterator<BinaryRowData> buildInput = new UniformBinaryRowGenerator(2 * numKeys, buildValsPerKey, false);
    // create a probe input that gives 20000 pairs with 1 values sharing a key
    MutableObjectIterator<BinaryRowData> probeInput = new UniformBinaryRowGenerator(numKeys, probeValsPerKey, true);
    MemoryManager memManager = MemoryManagerBuilder.newBuilder().setMemorySize(35 * PAGE_SIZE).build();
    // allocate the memory for the HashTable
    final BinaryHashTable table = new BinaryHashTable(conf, new Object(), this.buildSideSerializer, this.probeSideSerializer, new MyProjection(), new MyProjection(), memManager, 35 * PAGE_SIZE, ioManager, 24, 200000, true, HashJoinType.BUILD_OUTER, null, true, new boolean[] { true }, false);
    int numRecordsInJoinResult = join(table, buildInput, probeInput, true);
    Assert.assertEquals("Wrong number of records in join result.", 2 * numKeys * buildValsPerKey * probeValsPerKey, numRecordsInJoinResult);
    table.close();
    table.free();
}
Also used : BinaryRowData(org.apache.flink.table.data.binary.BinaryRowData) MemoryManager(org.apache.flink.runtime.memory.MemoryManager) UniformBinaryRowGenerator(org.apache.flink.table.runtime.util.UniformBinaryRowGenerator) Test(org.junit.Test)

Aggregations

BinaryRowData (org.apache.flink.table.data.binary.BinaryRowData)21 UniformBinaryRowGenerator (org.apache.flink.table.runtime.util.UniformBinaryRowGenerator)21 Test (org.junit.Test)21 MemoryManager (org.apache.flink.runtime.memory.MemoryManager)14 MutableObjectIterator (org.apache.flink.util.MutableObjectIterator)7 HashMap (java.util.HashMap)6 Map (java.util.Map)6 ArrayList (java.util.ArrayList)5 UnionIterator (org.apache.flink.runtime.operators.testutils.UnionIterator)5 StreamOperator (org.apache.flink.streaming.api.operators.StreamOperator)2 BinaryRowWriter (org.apache.flink.table.data.writer.BinaryRowWriter)2 IOException (java.io.IOException)1 MemorySegment (org.apache.flink.core.memory.MemorySegment)1 MemoryAllocationException (org.apache.flink.runtime.memory.MemoryAllocationException)1