Search in sources :

Example 1 with BigintType

use of com.facebook.presto.common.type.BigintType in project presto by prestodb.

the class DruidBrokerPageSource method getNextPage.

@Override
public Page getNextPage() {
    if (finished) {
        return null;
    }
    long start = System.nanoTime();
    boolean columnHandlesHasErrorMessageField = columnHandles.stream().anyMatch(handle -> ((DruidColumnHandle) handle).getColumnName().equals("errorMessage"));
    try {
        String readLine;
        while ((readLine = responseStream.readLine()) != null) {
            // if read a blank line,it means read finish
            if (readLine.isEmpty()) {
                finished = true;
                break;
            } else {
                JsonNode rootNode = OBJECT_MAPPER.readTree(readLine);
                if (rootNode.has("errorMessage") && !columnHandlesHasErrorMessageField) {
                    throw new PrestoException(DRUID_BROKER_RESULT_ERROR, rootNode.findValue("errorMessage").asText());
                }
                for (int i = 0; i < columnHandles.size(); i++) {
                    Type type = columnTypes.get(i);
                    BlockBuilder blockBuilder = pageBuilder.getBlockBuilder(i);
                    JsonNode value = rootNode.get(((DruidColumnHandle) columnHandles.get(i)).getColumnName());
                    if (value == null) {
                        blockBuilder.appendNull();
                        continue;
                    }
                    if (type instanceof BigintType) {
                        type.writeLong(blockBuilder, value.longValue());
                    } else if (type instanceof DoubleType) {
                        type.writeDouble(blockBuilder, value.doubleValue());
                    } else if (type instanceof RealType) {
                        type.writeLong(blockBuilder, floatToRawIntBits(value.floatValue()));
                    } else if (type instanceof TimestampType) {
                        DateTimeFormatter formatter = ISODateTimeFormat.dateTimeParser().withChronology(ISOChronology.getInstanceUTC()).withOffsetParsed();
                        DateTime dateTime = formatter.parseDateTime(value.textValue());
                        type.writeLong(blockBuilder, dateTime.getMillis());
                    } else {
                        Slice slice = Slices.utf8Slice(value.textValue());
                        type.writeSlice(blockBuilder, slice);
                    }
                }
            }
            pageBuilder.declarePosition();
            if (pageBuilder.isFull()) {
                break;
            }
        }
        // if responseStream.readLine() is null, it means read finish
        if (readLine == null) {
            finished = true;
            return null;
        }
        // only return a page if the buffer is full or we are finishing
        if (pageBuilder.isEmpty() || (!finished && !pageBuilder.isFull())) {
            return null;
        }
        Page page = pageBuilder.build();
        completedPositions += page.getPositionCount();
        completedBytes += page.getSizeInBytes();
        pageBuilder.reset();
        return page;
    } catch (IOException e) {
        finished = true;
        throw new PrestoException(DRUID_BROKER_RESULT_ERROR, "Parse druid client response error", e);
    } finally {
        readTimeNanos += System.nanoTime() - start;
    }
}
Also used : JsonNode(com.fasterxml.jackson.databind.JsonNode) PrestoException(com.facebook.presto.spi.PrestoException) Page(com.facebook.presto.common.Page) IOException(java.io.IOException) RealType(com.facebook.presto.common.type.RealType) BigintType(com.facebook.presto.common.type.BigintType) DateTime(org.joda.time.DateTime) Type(com.facebook.presto.common.type.Type) BigintType(com.facebook.presto.common.type.BigintType) RealType(com.facebook.presto.common.type.RealType) DoubleType(com.facebook.presto.common.type.DoubleType) TimestampType(com.facebook.presto.common.type.TimestampType) DoubleType(com.facebook.presto.common.type.DoubleType) Slice(io.airlift.slice.Slice) TimestampType(com.facebook.presto.common.type.TimestampType) DateTimeFormatter(org.joda.time.format.DateTimeFormatter) BlockBuilder(com.facebook.presto.common.block.BlockBuilder)

Example 2 with BigintType

use of com.facebook.presto.common.type.BigintType in project presto by prestodb.

the class InCodeGenerator method checkSwitchGenerationCase.

@VisibleForTesting
static SwitchGenerationCase checkSwitchGenerationCase(Type type, List<RowExpression> values) {
    if (values.size() > 32) {
        // * Benchmark shows performance of SET_CONTAINS is better at 50, but similar at 25.
        return SwitchGenerationCase.SET_CONTAINS;
    }
    if (!(type instanceof IntegerType || type instanceof BigintType || type instanceof DateType)) {
        return SwitchGenerationCase.HASH_SWITCH;
    }
    for (RowExpression expression : values) {
        // Same argument applies for nulls.
        if (!(expression instanceof ConstantExpression)) {
            continue;
        }
        Object constant = ((ConstantExpression) expression).getValue();
        if (constant == null) {
            continue;
        }
        long longConstant = ((Number) constant).longValue();
        if (longConstant < Integer.MIN_VALUE || longConstant > Integer.MAX_VALUE) {
            return SwitchGenerationCase.HASH_SWITCH;
        }
    }
    return SwitchGenerationCase.DIRECT_SWITCH;
}
Also used : IntegerType(com.facebook.presto.common.type.IntegerType) ConstantExpression(com.facebook.presto.spi.relation.ConstantExpression) RowExpression(com.facebook.presto.spi.relation.RowExpression) DateType(com.facebook.presto.common.type.DateType) BigintType(com.facebook.presto.common.type.BigintType) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 3 with BigintType

use of com.facebook.presto.common.type.BigintType in project presto by prestodb.

the class PinotBrokerPageSourceBase method setValue.

protected void setValue(Type type, BlockBuilder blockBuilder, String value) {
    if (blockBuilder == null) {
        return;
    }
    if (value == null) {
        blockBuilder.appendNull();
        return;
    }
    if (!(type instanceof FixedWidthType) && !(type instanceof VarcharType) && !(type instanceof JsonType)) {
        throw new PinotException(PINOT_UNSUPPORTED_COLUMN_TYPE, Optional.empty(), "type '" + type + "' not supported");
    }
    if (type instanceof FixedWidthType) {
        completedBytes += ((FixedWidthType) type).getFixedSize();
        if (type instanceof BigintType) {
            type.writeLong(blockBuilder, parseDouble(value).longValue());
        } else if (type instanceof IntegerType) {
            blockBuilder.writeInt(parseDouble(value).intValue());
        } else if (type instanceof TinyintType) {
            blockBuilder.writeByte(parseDouble(value).byteValue());
        } else if (type instanceof SmallintType) {
            blockBuilder.writeShort(parseDouble(value).shortValue());
        } else if (type instanceof BooleanType) {
            type.writeBoolean(blockBuilder, parseBoolean(value));
        } else if (type instanceof DecimalType || type instanceof DoubleType) {
            type.writeDouble(blockBuilder, parseDouble(value));
        } else if (type instanceof TimestampType) {
            type.writeLong(blockBuilder, parseTimestamp(value));
        } else if (type instanceof DateType) {
            type.writeLong(blockBuilder, parseLong(value));
        } else {
            throw new PinotException(PINOT_UNSUPPORTED_COLUMN_TYPE, Optional.empty(), "type '" + type + "' not supported");
        }
    } else {
        Slice slice = Slices.utf8Slice(value);
        blockBuilder.writeBytes(slice, 0, slice.length()).closeEntry();
        completedBytes += slice.length();
    }
}
Also used : JsonType(com.facebook.presto.common.type.JsonType) VarcharType(com.facebook.presto.common.type.VarcharType) TinyintType(com.facebook.presto.common.type.TinyintType) BooleanType(com.facebook.presto.common.type.BooleanType) BigintType(com.facebook.presto.common.type.BigintType) IntegerType(com.facebook.presto.common.type.IntegerType) DoubleType(com.facebook.presto.common.type.DoubleType) Slice(io.airlift.slice.Slice) DecimalType(com.facebook.presto.common.type.DecimalType) TimestampType(com.facebook.presto.common.type.TimestampType) SmallintType(com.facebook.presto.common.type.SmallintType) DateType(com.facebook.presto.common.type.DateType) FixedWidthType(com.facebook.presto.common.type.FixedWidthType)

Example 4 with BigintType

use of com.facebook.presto.common.type.BigintType in project presto by prestodb.

the class LiteralInterpreter method evaluate.

public static Object evaluate(ConnectorSession session, ConstantExpression node) {
    Type type = node.getType();
    SqlFunctionProperties properties = session.getSqlFunctionProperties();
    if (node.getValue() == null) {
        return null;
    }
    if (type instanceof BooleanType) {
        return node.getValue();
    }
    if (type instanceof BigintType || type instanceof TinyintType || type instanceof SmallintType || type instanceof IntegerType) {
        return node.getValue();
    }
    if (type instanceof DoubleType) {
        return node.getValue();
    }
    if (type instanceof RealType) {
        Long number = (Long) node.getValue();
        return intBitsToFloat(number.intValue());
    }
    if (type instanceof DecimalType) {
        DecimalType decimalType = (DecimalType) type;
        if (decimalType.isShort()) {
            checkState(node.getValue() instanceof Long);
            return decodeDecimal(BigInteger.valueOf((long) node.getValue()), decimalType);
        }
        checkState(node.getValue() instanceof Slice);
        Slice value = (Slice) node.getValue();
        return decodeDecimal(decodeUnscaledValue(value), decimalType);
    }
    if (type instanceof VarcharType || type instanceof CharType) {
        return ((Slice) node.getValue()).toStringUtf8();
    }
    if (type instanceof VarbinaryType) {
        return new SqlVarbinary(((Slice) node.getValue()).getBytes());
    }
    if (type instanceof DateType) {
        return new SqlDate(((Long) node.getValue()).intValue());
    }
    if (type instanceof TimeType) {
        if (properties.isLegacyTimestamp()) {
            return new SqlTime((long) node.getValue(), properties.getTimeZoneKey());
        }
        return new SqlTime((long) node.getValue());
    }
    if (type instanceof TimestampType) {
        try {
            if (properties.isLegacyTimestamp()) {
                return new SqlTimestamp((long) node.getValue(), properties.getTimeZoneKey());
            }
            return new SqlTimestamp((long) node.getValue());
        } catch (RuntimeException e) {
            throw new PrestoException(GENERIC_USER_ERROR, format("'%s' is not a valid timestamp literal", (String) node.getValue()));
        }
    }
    if (type instanceof IntervalDayTimeType) {
        return new SqlIntervalDayTime((long) node.getValue());
    }
    if (type instanceof IntervalYearMonthType) {
        return new SqlIntervalYearMonth(((Long) node.getValue()).intValue());
    }
    if (type.getJavaType().equals(Slice.class)) {
        // DO NOT ever remove toBase64. Calling toString directly on Slice whose base is not byte[] will cause JVM to crash.
        return "'" + VarbinaryFunctions.toBase64((Slice) node.getValue()).toStringUtf8() + "'";
    }
    // We should not fail at the moment; just return the raw value (block, regex, etc) to the user
    return node.getValue();
}
Also used : IntervalYearMonthType(com.facebook.presto.type.IntervalYearMonthType) VarcharType(com.facebook.presto.common.type.VarcharType) TinyintType(com.facebook.presto.common.type.TinyintType) SqlVarbinary(com.facebook.presto.common.type.SqlVarbinary) SqlTime(com.facebook.presto.common.type.SqlTime) SqlIntervalYearMonth(com.facebook.presto.type.SqlIntervalYearMonth) SqlTimestamp(com.facebook.presto.common.type.SqlTimestamp) PrestoException(com.facebook.presto.spi.PrestoException) RealType(com.facebook.presto.common.type.RealType) IntervalDayTimeType(com.facebook.presto.type.IntervalDayTimeType) TimeType(com.facebook.presto.common.type.TimeType) VarbinaryType(com.facebook.presto.common.type.VarbinaryType) TimestampType(com.facebook.presto.common.type.TimestampType) SmallintType(com.facebook.presto.common.type.SmallintType) DateType(com.facebook.presto.common.type.DateType) SqlFunctionProperties(com.facebook.presto.common.function.SqlFunctionProperties) BooleanType(com.facebook.presto.common.type.BooleanType) BigintType(com.facebook.presto.common.type.BigintType) IntegerType(com.facebook.presto.common.type.IntegerType) IntervalDayTimeType(com.facebook.presto.type.IntervalDayTimeType) TinyintType(com.facebook.presto.common.type.TinyintType) BigintType(com.facebook.presto.common.type.BigintType) IntervalYearMonthType(com.facebook.presto.type.IntervalYearMonthType) VarcharType(com.facebook.presto.common.type.VarcharType) RealType(com.facebook.presto.common.type.RealType) SmallintType(com.facebook.presto.common.type.SmallintType) VarbinaryType(com.facebook.presto.common.type.VarbinaryType) TimestampType(com.facebook.presto.common.type.TimestampType) DecimalType(com.facebook.presto.common.type.DecimalType) BooleanType(com.facebook.presto.common.type.BooleanType) IntegerType(com.facebook.presto.common.type.IntegerType) CharType(com.facebook.presto.common.type.CharType) Type(com.facebook.presto.common.type.Type) TimeType(com.facebook.presto.common.type.TimeType) DateType(com.facebook.presto.common.type.DateType) DoubleType(com.facebook.presto.common.type.DoubleType) DoubleType(com.facebook.presto.common.type.DoubleType) Slices.utf8Slice(io.airlift.slice.Slices.utf8Slice) Slice(io.airlift.slice.Slice) SqlDate(com.facebook.presto.common.type.SqlDate) SqlIntervalDayTime(com.facebook.presto.type.SqlIntervalDayTime) IntervalDayTimeType(com.facebook.presto.type.IntervalDayTimeType) DecimalType(com.facebook.presto.common.type.DecimalType) CharType(com.facebook.presto.common.type.CharType)

Example 5 with BigintType

use of com.facebook.presto.common.type.BigintType in project presto by prestodb.

the class MetastoreUtil method convertRawValueToString.

public static String convertRawValueToString(Object value, Type type) {
    String val;
    if (value == null) {
        val = HIVE_DEFAULT_DYNAMIC_PARTITION;
    } else if (type instanceof CharType) {
        Slice slice = (Slice) value;
        val = padSpaces(slice, type).toStringUtf8();
    } else if (type instanceof VarcharType) {
        Slice slice = (Slice) value;
        val = slice.toStringUtf8();
    } else if (type instanceof DecimalType && !((DecimalType) type).isShort()) {
        Slice slice = (Slice) value;
        val = Decimals.toString(slice, ((DecimalType) type).getScale());
    } else if (type instanceof DecimalType && ((DecimalType) type).isShort()) {
        val = Decimals.toString((long) value, ((DecimalType) type).getScale());
    } else if (type instanceof DateType) {
        DateTimeFormatter dateTimeFormatter = ISODateTimeFormat.date().withZoneUTC();
        val = dateTimeFormatter.print(TimeUnit.DAYS.toMillis((long) value));
    } else if (type instanceof TimestampType) {
        // we don't have time zone info, so just add a wildcard
        val = PARTITION_VALUE_WILDCARD;
    } else if (type instanceof TinyintType || type instanceof SmallintType || type instanceof IntegerType || type instanceof BigintType || type instanceof DoubleType || type instanceof RealType || type instanceof BooleanType) {
        val = value.toString();
    } else {
        throw new PrestoException(NOT_SUPPORTED, format("Unsupported partition key type: %s", type.getDisplayName()));
    }
    return val;
}
Also used : Varchars.isVarcharType(com.facebook.presto.common.type.Varchars.isVarcharType) VarcharType(com.facebook.presto.common.type.VarcharType) TinyintType(com.facebook.presto.common.type.TinyintType) BooleanType(com.facebook.presto.common.type.BooleanType) PrestoException(com.facebook.presto.spi.PrestoException) ProtectMode.getProtectModeFromString(org.apache.hadoop.hive.metastore.ProtectMode.getProtectModeFromString) RealType(com.facebook.presto.common.type.RealType) BigintType(com.facebook.presto.common.type.BigintType) IntegerType(com.facebook.presto.common.type.IntegerType) Slice(io.airlift.slice.Slice) DoubleType(com.facebook.presto.common.type.DoubleType) DecimalType(com.facebook.presto.common.type.DecimalType) TimestampType(com.facebook.presto.common.type.TimestampType) SmallintType(com.facebook.presto.common.type.SmallintType) Chars.isCharType(com.facebook.presto.common.type.Chars.isCharType) CharType(com.facebook.presto.common.type.CharType) DateType(com.facebook.presto.common.type.DateType) DateTimeFormatter(org.joda.time.format.DateTimeFormatter)

Aggregations

BigintType (com.facebook.presto.common.type.BigintType)8 DoubleType (com.facebook.presto.common.type.DoubleType)7 IntegerType (com.facebook.presto.common.type.IntegerType)7 TimestampType (com.facebook.presto.common.type.TimestampType)7 BooleanType (com.facebook.presto.common.type.BooleanType)6 DateType (com.facebook.presto.common.type.DateType)6 DecimalType (com.facebook.presto.common.type.DecimalType)6 RealType (com.facebook.presto.common.type.RealType)6 SmallintType (com.facebook.presto.common.type.SmallintType)6 TinyintType (com.facebook.presto.common.type.TinyintType)6 VarcharType (com.facebook.presto.common.type.VarcharType)6 Slice (io.airlift.slice.Slice)6 CharType (com.facebook.presto.common.type.CharType)5 Type (com.facebook.presto.common.type.Type)5 PrestoException (com.facebook.presto.spi.PrestoException)4 TimestampWithTimeZoneType (com.facebook.presto.common.type.TimestampWithTimeZoneType)2 VarbinaryType (com.facebook.presto.common.type.VarbinaryType)2 Page (com.facebook.presto.common.Page)1 Subfield (com.facebook.presto.common.Subfield)1 Arrays.ensureCapacity (com.facebook.presto.common.array.Arrays.ensureCapacity)1