Examples with DictionaryBlock com.facebook.presto.common.block.DictionaryBlock used on opensource projects

Example 21 with DictionaryBlock

use of com.facebook.presto.common.block.DictionaryBlock in project presto by prestodb.

the class TestDictionaryAwarePageProjection method testDictionaryProcessingEnableDisable.

@Test(dataProvider = "forceYield")
public void testDictionaryProcessingEnableDisable(boolean forceYield) {
    DictionaryAwarePageProjection projection = createProjection();
    // function will always processes the first dictionary
    DictionaryBlock ineffectiveBlock = createDictionaryBlock(100, 20);
    testProjectRange(ineffectiveBlock, DictionaryBlock.class, projection, forceYield);
    testProjectFastReturnIgnoreYield(ineffectiveBlock, projection);
    // dictionary processing can reuse the last dictionary
    // in this case, we don't even check yield signal; make yieldForce to false
    testProjectList(ineffectiveBlock, DictionaryBlock.class, projection, false);
    // last dictionary not effective, so dictionary processing is disabled
    DictionaryBlock effectiveBlock = createDictionaryBlock(10, 100);
    testProjectRange(effectiveBlock, LongArrayBlock.class, projection, forceYield);
    testProjectList(effectiveBlock, LongArrayBlock.class, projection, forceYield);
    // last dictionary effective, so dictionary processing is enabled again
    testProjectRange(ineffectiveBlock, DictionaryBlock.class, projection, forceYield);
    testProjectFastReturnIgnoreYield(ineffectiveBlock, projection);
    // dictionary processing can reuse the last dictionary
    // in this case, we don't even check yield signal; make yieldForce to false
    testProjectList(ineffectiveBlock, DictionaryBlock.class, projection, false);
    // last dictionary not effective, so dictionary processing is disabled again
    testProjectRange(effectiveBlock, LongArrayBlock.class, projection, forceYield);
    testProjectList(effectiveBlock, LongArrayBlock.class, projection, forceYield);
}

Example 22 with DictionaryBlock

use of com.facebook.presto.common.block.DictionaryBlock in project presto by prestodb.

the class TestDictionaryAwarePageProjection method testDictionaryBlockWithFailure.

@Test(dataProvider = "forceYield")
public void testDictionaryBlockWithFailure(boolean forceYield) {
    DictionaryBlock block = createDictionaryBlockWithFailure(10, 100);
    testProjectFails(block, DictionaryBlock.class, forceYield);
}

Example 23 with DictionaryBlock

use of com.facebook.presto.common.block.DictionaryBlock in project presto by prestodb.

the class TestDictionaryAwarePageFilter method testDictionaryProcessingEnableDisable.

@Test
public void testDictionaryProcessingEnableDisable() {
    TestDictionaryFilter nestedFilter = new TestDictionaryFilter(true);
    DictionaryAwarePageFilter filter = new DictionaryAwarePageFilter(nestedFilter);
    DictionaryBlock ineffectiveBlock = createDictionaryBlock(100, 20);
    DictionaryBlock effectiveBlock = createDictionaryBlock(10, 100);
    // function will always processes the first dictionary
    nestedFilter.setExpectedType(LongArrayBlock.class);
    testFilter(filter, ineffectiveBlock, true);
    // last dictionary not effective, so dictionary processing is disabled
    nestedFilter.setExpectedType(DictionaryBlock.class);
    testFilter(filter, effectiveBlock, true);
    // last dictionary not effective, so dictionary processing is enabled again
    nestedFilter.setExpectedType(LongArrayBlock.class);
    testFilter(filter, ineffectiveBlock, true);
    // last dictionary not effective, so dictionary processing is disabled again
    nestedFilter.setExpectedType(DictionaryBlock.class);
    testFilter(filter, effectiveBlock, true);
}

Example 24 with DictionaryBlock

use of com.facebook.presto.common.block.DictionaryBlock in project urban-eureka by errir503.

the class TestPageSplitterUtil method testSplitDictionaryBlock.

@Test
public void testSplitDictionaryBlock() {
    // create a huge dictionary with a single entry
    Block dictionary = createSlicesBlock(Slices.utf8Slice(new String(new char[1_000_000])));
    // make every row in the block identical to the single entry in the dictionary
    DictionaryBlock dictionaryBlock = createRandomDictionaryBlock(dictionary, 50000);
    List<Page> pages = splitPage(new Page(dictionaryBlock), 1000);
    assertEquals(pages.size(), 2);
}

Example 25 with DictionaryBlock

use of com.facebook.presto.common.block.DictionaryBlock in project urban-eureka by errir503.

the class Page method getRelatedDictionaryBlocks.

private Map<DictionaryId, DictionaryBlockIndexes> getRelatedDictionaryBlocks() {
    Map<DictionaryId, DictionaryBlockIndexes> relatedDictionaryBlocks = new HashMap<>();
    for (int i = 0; i < blocks.length; i++) {
        Block block = blocks[i];
        if (block instanceof DictionaryBlock) {
            DictionaryBlock dictionaryBlock = (DictionaryBlock) block;
            relatedDictionaryBlocks.computeIfAbsent(dictionaryBlock.getDictionarySourceId(), id -> new DictionaryBlockIndexes()).addBlock(dictionaryBlock, i);
        }
    }
    return relatedDictionaryBlocks;
}