Search in sources :

Example 1 with BytesRefBuilder

use of org.apache.lucene.util.BytesRefBuilder in project elasticsearch by elastic.

the class AbstractBytesReferenceTestCase method testSliceIterator.

public void testSliceIterator() throws IOException {
    int length = randomIntBetween(10, PAGE_SIZE * randomIntBetween(2, 8));
    BytesReference pbr = newBytesReference(length);
    int sliceOffset = randomIntBetween(0, pbr.length());
    int sliceLength = randomIntBetween(0, pbr.length() - sliceOffset);
    BytesReference slice = pbr.slice(sliceOffset, sliceLength);
    BytesRefIterator iterator = slice.iterator();
    BytesRef ref = null;
    BytesRefBuilder builder = new BytesRefBuilder();
    while ((ref = iterator.next()) != null) {
        builder.append(ref);
    }
    assertArrayEquals(BytesReference.toBytes(slice), BytesRef.deepCopyOf(builder.toBytesRef()).bytes);
}
Also used : BytesRefIterator(org.apache.lucene.util.BytesRefIterator) BytesRefBuilder(org.apache.lucene.util.BytesRefBuilder) BytesRef(org.apache.lucene.util.BytesRef)

Example 2 with BytesRefBuilder

use of org.apache.lucene.util.BytesRefBuilder in project elasticsearch by elastic.

the class AbstractBytesReferenceTestCase method testRandomReads.

public void testRandomReads() throws IOException {
    int length = randomIntBetween(10, scaledRandomIntBetween(PAGE_SIZE * 2, PAGE_SIZE * 20));
    BytesReference pbr = newBytesReference(length);
    StreamInput streamInput = pbr.streamInput();
    BytesRefBuilder target = new BytesRefBuilder();
    while (target.length() < pbr.length()) {
        switch(randomIntBetween(0, 10)) {
            case 6:
            case 5:
                target.append(new BytesRef(new byte[] { streamInput.readByte() }));
                break;
            case 4:
            case 3:
                BytesRef bytesRef = streamInput.readBytesRef(scaledRandomIntBetween(1, pbr.length() - target.length()));
                target.append(bytesRef);
                break;
            default:
                byte[] buffer = new byte[scaledRandomIntBetween(1, pbr.length() - target.length())];
                int offset = scaledRandomIntBetween(0, buffer.length - 1);
                int read = streamInput.read(buffer, offset, buffer.length - offset);
                target.append(new BytesRef(buffer, offset, read));
                break;
        }
    }
    assertEquals(pbr.length(), target.length());
    BytesRef targetBytes = target.get();
    assertArrayEquals(BytesReference.toBytes(pbr), Arrays.copyOfRange(targetBytes.bytes, targetBytes.offset, targetBytes.length));
}
Also used : BytesRefBuilder(org.apache.lucene.util.BytesRefBuilder) StreamInput(org.elasticsearch.common.io.stream.StreamInput) BytesRef(org.apache.lucene.util.BytesRef)

Example 3 with BytesRefBuilder

use of org.apache.lucene.util.BytesRefBuilder in project elasticsearch by elastic.

the class CommonTermsQueryBuilder method parseQueryString.

private static Query parseQueryString(ExtendedCommonTermsQuery query, Object queryString, String field, Analyzer analyzer, String lowFreqMinimumShouldMatch, String highFreqMinimumShouldMatch) throws IOException {
    // Logic similar to QueryParser#getFieldQuery
    try (TokenStream source = analyzer.tokenStream(field, queryString.toString())) {
        source.reset();
        CharTermAttribute termAtt = source.addAttribute(CharTermAttribute.class);
        BytesRefBuilder builder = new BytesRefBuilder();
        while (source.incrementToken()) {
            // UTF-8
            builder.copyChars(termAtt);
            query.add(new Term(field, builder.toBytesRef()));
        }
    }
    query.setLowFreqMinimumNumberShouldMatch(lowFreqMinimumShouldMatch);
    query.setHighFreqMinimumNumberShouldMatch(highFreqMinimumShouldMatch);
    return query;
}
Also used : TokenStream(org.apache.lucene.analysis.TokenStream) BytesRefBuilder(org.apache.lucene.util.BytesRefBuilder) CharTermAttribute(org.apache.lucene.analysis.tokenattributes.CharTermAttribute) Term(org.apache.lucene.index.Term)

Example 4 with BytesRefBuilder

use of org.apache.lucene.util.BytesRefBuilder in project elasticsearch by elastic.

the class Uid method createUidsForTypesAndIds.

public static BytesRef[] createUidsForTypesAndIds(Collection<String> types, Collection<?> ids) {
    BytesRef[] uids = new BytesRef[types.size() * ids.size()];
    BytesRefBuilder typeBytes = new BytesRefBuilder();
    BytesRefBuilder idBytes = new BytesRefBuilder();
    int index = 0;
    for (String type : types) {
        typeBytes.copyChars(type);
        for (Object id : ids) {
            uids[index++] = Uid.createUidAsBytes(typeBytes.get(), BytesRefs.toBytesRef(id, idBytes));
        }
    }
    return uids;
}
Also used : BytesRefBuilder(org.apache.lucene.util.BytesRefBuilder) BytesRef(org.apache.lucene.util.BytesRef)

Example 5 with BytesRefBuilder

use of org.apache.lucene.util.BytesRefBuilder in project elasticsearch by elastic.

the class SortingBinaryDocValues method grow.

/**
     * Make sure the {@link #values} array can store at least {@link #count} entries.
     */
protected final void grow() {
    if (values.length < count) {
        final int oldLen = values.length;
        final int newLen = ArrayUtil.oversize(count, RamUsageEstimator.NUM_BYTES_OBJECT_REF);
        values = Arrays.copyOf(values, newLen);
        for (int i = oldLen; i < newLen; ++i) {
            values[i] = new BytesRefBuilder();
        }
    }
}
Also used : BytesRefBuilder(org.apache.lucene.util.BytesRefBuilder)

Aggregations

BytesRefBuilder (org.apache.lucene.util.BytesRefBuilder)150 BytesRef (org.apache.lucene.util.BytesRef)79 ArrayList (java.util.ArrayList)21 IOException (java.io.IOException)17 Term (org.apache.lucene.index.Term)16 HashSet (java.util.HashSet)15 ChecksumIndexInput (org.apache.lucene.store.ChecksumIndexInput)14 FieldType (org.apache.solr.schema.FieldType)14 IndexInput (org.apache.lucene.store.IndexInput)12 BytesRefIterator (org.apache.lucene.util.BytesRefIterator)10 CharsRefBuilder (org.apache.lucene.util.CharsRefBuilder)10 IntsRef (org.apache.lucene.util.IntsRef)10 SchemaField (org.apache.solr.schema.SchemaField)10 BufferedChecksumIndexInput (org.apache.lucene.store.BufferedChecksumIndexInput)9 ParseException (java.text.ParseException)8 IntsRefBuilder (org.apache.lucene.util.IntsRefBuilder)8 DecimalFormat (java.text.DecimalFormat)7 HashMap (java.util.HashMap)7 Map (java.util.Map)7 Directory (org.apache.lucene.store.Directory)7