Search in sources :

Example 1 with SliceUtf8

use of io.airlift.slice.SliceUtf8 in project presto by prestodb.

the class StringFunctions method pad.

private static Slice pad(Slice text, long targetLength, Slice padString, int paddingOffset) {
    checkCondition(0 <= targetLength && targetLength <= Integer.MAX_VALUE, INVALID_FUNCTION_ARGUMENT, "Target length must be in the range [0.." + Integer.MAX_VALUE + "]");
    checkCondition(padString.length() > 0, INVALID_FUNCTION_ARGUMENT, "Padding string must not be empty");
    int textLength = countCodePoints(text);
    int resultLength = (int) targetLength;
    // if our target length is the same as our string then return our string
    if (textLength == resultLength) {
        return text;
    }
    // if our string is bigger than requested then truncate
    if (textLength > resultLength) {
        return SliceUtf8.substring(text, 0, resultLength);
    }
    // number of bytes in each code point
    int padStringLength = countCodePoints(padString);
    int[] padStringCounts = new int[padStringLength];
    for (int i = 0; i < padStringLength; ++i) {
        padStringCounts[i] = lengthOfCodePointSafe(padString, offsetOfCodePoint(padString, i));
    }
    // preallocate the result
    int bufferSize = text.length();
    for (int i = 0; i < resultLength - textLength; ++i) {
        bufferSize += padStringCounts[i % padStringLength];
    }
    Slice buffer = Slices.allocate(bufferSize);
    // fill in the existing string
    int countBytes = bufferSize - text.length();
    int startPointOfExistingText = (paddingOffset + countBytes) % bufferSize;
    buffer.setBytes(startPointOfExistingText, text);
    // assign the pad string while there's enough space for it
    int byteIndex = paddingOffset;
    for (int i = 0; i < countBytes / padString.length(); ++i) {
        buffer.setBytes(byteIndex, padString);
        byteIndex += padString.length();
    }
    // handle the tail: at most we assign padStringLength - 1 code points
    buffer.setBytes(byteIndex, padString.getBytes(0, paddingOffset + countBytes - byteIndex));
    return buffer;
}
Also used : Slice(io.airlift.slice.Slice) Slices.utf8Slice(io.airlift.slice.Slices.utf8Slice) Constraint(com.facebook.presto.type.Constraint) SliceUtf8.lengthOfCodePoint(io.airlift.slice.SliceUtf8.lengthOfCodePoint) SliceUtf8.offsetOfCodePoint(io.airlift.slice.SliceUtf8.offsetOfCodePoint)

Example 2 with SliceUtf8

use of io.airlift.slice.SliceUtf8 in project presto by prestodb.

the class StringFunctions method fromUtf8.

@Description("decodes the UTF-8 encoded string")
@ScalarFunction
@LiteralParameters("x")
@SqlType(StandardTypes.VARCHAR)
public static Slice fromUtf8(@SqlType(StandardTypes.VARBINARY) Slice slice, @SqlType("varchar(x)") Slice replacementCharacter) {
    int count = countCodePoints(replacementCharacter);
    if (count > 1) {
        throw new PrestoException(INVALID_FUNCTION_ARGUMENT, "Replacement character string must empty or a single character");
    }
    OptionalInt replacementCodePoint;
    if (count == 1) {
        try {
            replacementCodePoint = OptionalInt.of(getCodePointAt(replacementCharacter, 0));
        } catch (InvalidUtf8Exception e) {
            throw new PrestoException(INVALID_FUNCTION_ARGUMENT, "Invalid replacement character");
        }
    } else {
        replacementCodePoint = OptionalInt.empty();
    }
    return SliceUtf8.fixInvalidUtf8(slice, replacementCodePoint);
}
Also used : InvalidUtf8Exception(io.airlift.slice.InvalidUtf8Exception) PrestoException(com.facebook.presto.spi.PrestoException) OptionalInt(java.util.OptionalInt) Constraint(com.facebook.presto.type.Constraint) SliceUtf8.lengthOfCodePoint(io.airlift.slice.SliceUtf8.lengthOfCodePoint) SliceUtf8.offsetOfCodePoint(io.airlift.slice.SliceUtf8.offsetOfCodePoint) ScalarFunction(com.facebook.presto.spi.function.ScalarFunction) Description(com.facebook.presto.spi.function.Description) LiteralParameters(com.facebook.presto.spi.function.LiteralParameters) SqlType(com.facebook.presto.spi.function.SqlType)

Example 3 with SliceUtf8

use of io.airlift.slice.SliceUtf8 in project presto by prestodb.

the class CharacterStringCasts method codePointsToSliceUtf8.

private static Slice codePointsToSliceUtf8(List<Integer> codePoints) {
    int length = codePoints.stream().mapToInt(SliceUtf8::lengthOfCodePoint).sum();
    Slice result = Slices.wrappedBuffer(new byte[length]);
    int offset = 0;
    for (int codePoint : codePoints) {
        setCodePointAt(codePoint, result, offset);
        offset += lengthOfCodePoint(codePoint);
    }
    return result;
}
Also used : Slice(io.airlift.slice.Slice) SliceUtf8.lengthOfCodePoint(io.airlift.slice.SliceUtf8.lengthOfCodePoint)

Aggregations

SliceUtf8.lengthOfCodePoint (io.airlift.slice.SliceUtf8.lengthOfCodePoint)3 Constraint (com.facebook.presto.type.Constraint)2 Slice (io.airlift.slice.Slice)2 SliceUtf8.offsetOfCodePoint (io.airlift.slice.SliceUtf8.offsetOfCodePoint)2 PrestoException (com.facebook.presto.spi.PrestoException)1 Description (com.facebook.presto.spi.function.Description)1 LiteralParameters (com.facebook.presto.spi.function.LiteralParameters)1 ScalarFunction (com.facebook.presto.spi.function.ScalarFunction)1 SqlType (com.facebook.presto.spi.function.SqlType)1 InvalidUtf8Exception (io.airlift.slice.InvalidUtf8Exception)1 Slices.utf8Slice (io.airlift.slice.Slices.utf8Slice)1 OptionalInt (java.util.OptionalInt)1