Search in sources :

Example 66 with PrestoException

use of com.facebook.presto.spi.PrestoException in project presto by prestodb.

the class HashSemiJoinOperator method addInput.

@Override
public void addInput(Page page) {
    requireNonNull(page, "page is null");
    checkState(!finishing, "Operator is finishing");
    checkState(channelSet != null, "Set has not been built yet");
    checkState(outputPage == null, "Operator still has pending output");
    // create the block builder for the new boolean column
    // we know the exact size required for the block
    BlockBuilder blockBuilder = BOOLEAN.createFixedSizeBlockBuilder(page.getPositionCount());
    Page probeJoinPage = new Page(page.getBlock(probeJoinChannel));
    // update hashing strategy to use probe cursor
    for (int position = 0; position < page.getPositionCount(); position++) {
        if (probeJoinPage.getBlock(0).isNull(position)) {
            throw new PrestoException(NOT_SUPPORTED, "NULL values are not allowed on the probe side of SemiJoin operator. See the query plan for details.");
        } else {
            boolean contains = channelSet.contains(position, probeJoinPage);
            if (!contains && channelSet.containsNull()) {
                blockBuilder.appendNull();
            } else {
                BOOLEAN.writeBoolean(blockBuilder, contains);
            }
        }
    }
    // add the new boolean column to the page
    Block[] sourceBlocks = page.getBlocks();
    // +1 for the single boolean output channel
    Block[] outputBlocks = new Block[sourceBlocks.length + 1];
    System.arraycopy(sourceBlocks, 0, outputBlocks, 0, sourceBlocks.length);
    outputBlocks[sourceBlocks.length] = blockBuilder.build();
    outputPage = new Page(outputBlocks);
}
Also used : Block(com.facebook.presto.spi.block.Block) Page(com.facebook.presto.spi.Page) PrestoException(com.facebook.presto.spi.PrestoException) BlockBuilder(com.facebook.presto.spi.block.BlockBuilder)

Example 67 with PrestoException

use of com.facebook.presto.spi.PrestoException in project presto by prestodb.

the class BigintGroupByHash method rehash.

private void rehash() {
    expectedHashCollisions += estimateNumberOfHashCollisions(getGroupCount(), hashCapacity);
    long newCapacityLong = hashCapacity * 2L;
    if (newCapacityLong > Integer.MAX_VALUE) {
        throw new PrestoException(GENERIC_INSUFFICIENT_RESOURCES, "Size of hash table cannot exceed 1 billion entries");
    }
    int newCapacity = (int) newCapacityLong;
    int newMask = newCapacity - 1;
    LongBigArray newValues = new LongBigArray();
    newValues.ensureCapacity(newCapacity);
    IntBigArray newGroupIds = new IntBigArray(-1);
    newGroupIds.ensureCapacity(newCapacity);
    for (int groupId = 0; groupId < nextGroupId; groupId++) {
        if (groupId == nullGroupId) {
            continue;
        }
        long value = valuesByGroupId.get(groupId);
        // find an empty slot for the address
        long hashPosition = getHashPosition(value, newMask);
        while (newGroupIds.get(hashPosition) != -1) {
            hashPosition = (hashPosition + 1) & newMask;
            hashCollisions++;
        }
        // record the mapping
        newValues.set(hashPosition, value);
        newGroupIds.set(hashPosition, groupId);
    }
    mask = newMask;
    hashCapacity = newCapacity;
    maxFill = calculateMaxFill(hashCapacity);
    values = newValues;
    groupIds = newGroupIds;
    this.valuesByGroupId.ensureCapacity(maxFill);
}
Also used : LongBigArray(com.facebook.presto.array.LongBigArray) IntBigArray(com.facebook.presto.array.IntBigArray) PrestoException(com.facebook.presto.spi.PrestoException)

Example 68 with PrestoException

use of com.facebook.presto.spi.PrestoException in project presto by prestodb.

the class Histogram method combine.

public static void combine(HistogramState state, HistogramState otherState) {
    if (state.get() != null && otherState.get() != null) {
        TypedHistogram typedHistogram = state.get();
        long startSize = typedHistogram.getEstimatedSize();
        try {
            typedHistogram.addAll(otherState.get());
        } catch (ExceededMemoryLimitException e) {
            throw new PrestoException(INVALID_FUNCTION_ARGUMENT, format("The result of histogram may not exceed %s", e.getMaxMemory()));
        }
        state.addMemoryUsage(typedHistogram.getEstimatedSize() - startSize);
    } else if (state.get() == null) {
        state.set(otherState.get());
    }
}
Also used : PrestoException(com.facebook.presto.spi.PrestoException) ExceededMemoryLimitException(com.facebook.presto.ExceededMemoryLimitException)

Example 69 with PrestoException

use of com.facebook.presto.spi.PrestoException in project presto by prestodb.

the class MapAggregationFunction method input.

public static void input(Type keyType, Type valueType, KeyValuePairsState state, Block key, Block value, int position) {
    KeyValuePairs pairs = state.get();
    if (pairs == null) {
        pairs = new KeyValuePairs(keyType, valueType);
        state.set(pairs);
    }
    long startSize = pairs.estimatedInMemorySize();
    try {
        pairs.add(key, value, position, position);
    } catch (ExceededMemoryLimitException e) {
        throw new PrestoException(INVALID_FUNCTION_ARGUMENT, format("The result of map_agg may not exceed %s", e.getMaxMemory()));
    }
    state.addMemoryUsage(pairs.estimatedInMemorySize() - startSize);
}
Also used : PrestoException(com.facebook.presto.spi.PrestoException) ExceededMemoryLimitException(com.facebook.presto.ExceededMemoryLimitException)

Example 70 with PrestoException

use of com.facebook.presto.spi.PrestoException in project presto by prestodb.

the class MultimapAggregationFunction method combine.

public static void combine(MultiKeyValuePairsState state, MultiKeyValuePairsState otherState) {
    if (state.get() != null && otherState.get() != null) {
        Block keys = otherState.get().getKeys();
        Block values = otherState.get().getValues();
        MultiKeyValuePairs pairs = state.get();
        long startSize = pairs.estimatedInMemorySize();
        for (int i = 0; i < keys.getPositionCount(); i++) {
            try {
                pairs.add(keys, values, i, i);
            } catch (ExceededMemoryLimitException e) {
                throw new PrestoException(INVALID_FUNCTION_ARGUMENT, format("The result of map_agg may not exceed %s", e.getMaxMemory()));
            }
        }
        state.addMemoryUsage(pairs.estimatedInMemorySize() - startSize);
    } else if (state.get() == null) {
        state.set(otherState.get());
    }
}
Also used : Block(com.facebook.presto.spi.block.Block) PrestoException(com.facebook.presto.spi.PrestoException) ExceededMemoryLimitException(com.facebook.presto.ExceededMemoryLimitException)

Aggregations

PrestoException (com.facebook.presto.spi.PrestoException)273 IOException (java.io.IOException)58 Type (com.facebook.presto.spi.type.Type)48 ImmutableList (com.google.common.collect.ImmutableList)40 SchemaTableName (com.facebook.presto.spi.SchemaTableName)32 ArrayList (java.util.ArrayList)32 List (java.util.List)28 Path (org.apache.hadoop.fs.Path)28 Map (java.util.Map)24 Slice (io.airlift.slice.Slice)23 TableNotFoundException (com.facebook.presto.spi.TableNotFoundException)22 SqlType (com.facebook.presto.spi.function.SqlType)22 ImmutableMap (com.google.common.collect.ImmutableMap)22 Optional (java.util.Optional)20 ColumnMetadata (com.facebook.presto.spi.ColumnMetadata)19 Block (com.facebook.presto.spi.block.Block)19 Test (org.testng.annotations.Test)19 ConnectorSession (com.facebook.presto.spi.ConnectorSession)17 Table (com.facebook.presto.hive.metastore.Table)15 Session (com.facebook.presto.Session)14