Search in sources :

Example 21 with SliceUtf8.getCodePointAt

use of io.airlift.slice.SliceUtf8.getCodePointAt in project trino by trinodb.

the class CharacterStringCasts method toCodePoints.

private static IntList toCodePoints(Slice slice) {
    IntList codePoints = new IntArrayList(slice.length());
    for (int offset = 0; offset < slice.length(); ) {
        int codePoint = getCodePointAt(slice, offset);
        offset += lengthOfCodePoint(slice, offset);
        codePoints.add(codePoint);
    }
    return codePoints;
}
Also used : IntArrayList(it.unimi.dsi.fastutil.ints.IntArrayList) SliceUtf8.lengthOfCodePoint(io.airlift.slice.SliceUtf8.lengthOfCodePoint) IntList(it.unimi.dsi.fastutil.ints.IntList)

Example 22 with SliceUtf8.getCodePointAt

use of io.airlift.slice.SliceUtf8.getCodePointAt in project presto by prestodb.

the class CharacterStringCasts method toCodePoints.

private static List<Integer> toCodePoints(Slice slice) {
    ImmutableList.Builder<Integer> codePoints = ImmutableList.builder();
    for (int offset = 0; offset < slice.length(); ) {
        int codePoint = getCodePointAt(slice, offset);
        offset += lengthOfCodePoint(slice, offset);
        codePoints.add(codePoint);
    }
    return codePoints.build();
}
Also used : ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) ImmutableList(com.google.common.collect.ImmutableList) SliceUtf8.lengthOfCodePoint(io.airlift.slice.SliceUtf8.lengthOfCodePoint)

Example 23 with SliceUtf8.getCodePointAt

use of io.airlift.slice.SliceUtf8.getCodePointAt in project presto by prestodb.

the class CharacterStringCasts method codePointsToSliceUtf8.

private static Slice codePointsToSliceUtf8(List<Integer> codePoints) {
    int length = codePoints.stream().mapToInt(SliceUtf8::lengthOfCodePoint).sum();
    Slice result = Slices.wrappedBuffer(new byte[length]);
    int offset = 0;
    for (int codePoint : codePoints) {
        setCodePointAt(codePoint, result, offset);
        offset += lengthOfCodePoint(codePoint);
    }
    return result;
}
Also used : Slice(io.airlift.slice.Slice) SliceUtf8.lengthOfCodePoint(io.airlift.slice.SliceUtf8.lengthOfCodePoint)

Example 24 with SliceUtf8.getCodePointAt

use of io.airlift.slice.SliceUtf8.getCodePointAt in project presto by prestodb.

the class StringFunctions method castToCodePoints.

private static int[] castToCodePoints(Slice slice) {
    int[] codePoints = new int[safeCountCodePoints(slice)];
    int position = 0;
    for (int index = 0; index < codePoints.length; index++) {
        codePoints[index] = getCodePointAt(slice, position);
        position += lengthOfCodePoint(slice, position);
    }
    return codePoints;
}
Also used : Constraint(com.facebook.presto.type.Constraint) SliceUtf8.lengthOfCodePoint(io.airlift.slice.SliceUtf8.lengthOfCodePoint) SliceUtf8.offsetOfCodePoint(io.airlift.slice.SliceUtf8.offsetOfCodePoint)

Aggregations

SliceUtf8.lengthOfCodePoint (io.airlift.slice.SliceUtf8.lengthOfCodePoint)24 Slice (io.airlift.slice.Slice)12 SliceUtf8.offsetOfCodePoint (io.airlift.slice.SliceUtf8.offsetOfCodePoint)12 Constraint (com.facebook.presto.type.Constraint)6 NonStrictUTF8Encoding (io.airlift.jcodings.specific.NonStrictUTF8Encoding)4 Option (io.airlift.joni.Option)4 Regex (io.airlift.joni.Regex)4 Syntax (io.airlift.joni.Syntax)4 INEFFECTIVE_META_CHAR (io.airlift.joni.constants.MetaChar.INEFFECTIVE_META_CHAR)4 OP_ASTERISK_ZERO_INF (io.airlift.joni.constants.SyntaxProperties.OP_ASTERISK_ZERO_INF)4 OP_DOT_ANYCHAR (io.airlift.joni.constants.SyntaxProperties.OP_DOT_ANYCHAR)4 OP_LINE_ANCHOR (io.airlift.joni.constants.SyntaxProperties.OP_LINE_ANCHOR)4 DynamicSliceOutput (io.airlift.slice.DynamicSliceOutput)4 InvalidUtf8Exception (io.airlift.slice.InvalidUtf8Exception)4 SliceUtf8.getCodePointAt (io.airlift.slice.SliceUtf8.getCodePointAt)4 Slices.utf8Slice (io.airlift.slice.Slices.utf8Slice)4 UTF_8 (java.nio.charset.StandardCharsets.UTF_8)4 Optional (java.util.Optional)4 OptionalInt (java.util.OptionalInt)4 PrestoException (io.prestosql.spi.PrestoException)3