Search in sources :

Example 66 with StoredField

use of org.apache.lucene.document.StoredField in project lucene-solr by apache.

the class TestTermVectorsWriter method testTermVectorCorruption3.

// LUCENE-1168
public void testTermVectorCorruption3() throws IOException {
    Directory dir = newDirectory();
    IndexWriter writer = new IndexWriter(dir, newIndexWriterConfig(new MockAnalyzer(random())).setMaxBufferedDocs(2).setRAMBufferSizeMB(IndexWriterConfig.DISABLE_AUTO_FLUSH).setMergeScheduler(new SerialMergeScheduler()).setMergePolicy(new LogDocMergePolicy()));
    Document document = new Document();
    FieldType customType = new FieldType();
    customType.setStored(true);
    Field storedField = newField("stored", "stored", customType);
    document.add(storedField);
    FieldType customType2 = new FieldType(StringField.TYPE_NOT_STORED);
    customType2.setStoreTermVectors(true);
    customType2.setStoreTermVectorPositions(true);
    customType2.setStoreTermVectorOffsets(true);
    Field termVectorField = newField("termVector", "termVector", customType2);
    document.add(termVectorField);
    for (int i = 0; i < 10; i++) writer.addDocument(document);
    writer.close();
    writer = new IndexWriter(dir, newIndexWriterConfig(new MockAnalyzer(random())).setMaxBufferedDocs(2).setRAMBufferSizeMB(IndexWriterConfig.DISABLE_AUTO_FLUSH).setMergeScheduler(new SerialMergeScheduler()).setMergePolicy(new LogDocMergePolicy()));
    for (int i = 0; i < 6; i++) writer.addDocument(document);
    writer.forceMerge(1);
    writer.close();
    IndexReader reader = DirectoryReader.open(dir);
    for (int i = 0; i < 10; i++) {
        reader.getTermVectors(i);
        reader.document(i);
    }
    reader.close();
    dir.close();
}
Also used : StringField(org.apache.lucene.document.StringField) StoredField(org.apache.lucene.document.StoredField) Field(org.apache.lucene.document.Field) TextField(org.apache.lucene.document.TextField) MockAnalyzer(org.apache.lucene.analysis.MockAnalyzer) Document(org.apache.lucene.document.Document) Directory(org.apache.lucene.store.Directory) FieldType(org.apache.lucene.document.FieldType)

Example 67 with StoredField

use of org.apache.lucene.document.StoredField in project lucene-solr by apache.

the class SpatialExample method newSampleDocument.

private Document newSampleDocument(int id, Shape... shapes) {
    Document doc = new Document();
    doc.add(new StoredField("id", id));
    doc.add(new NumericDocValuesField("id", id));
    // strategies; see the javadocs of the SpatialStrategy impl to see.
    for (Shape shape : shapes) {
        for (Field f : strategy.createIndexableFields(shape)) {
            doc.add(f);
        }
        //store it too; the format is up to you
        //  (assume point in this example)
        Point pt = (Point) shape;
        doc.add(new StoredField(strategy.getFieldName(), pt.getX() + " " + pt.getY()));
    }
    return doc;
}
Also used : StoredField(org.apache.lucene.document.StoredField) SortField(org.apache.lucene.search.SortField) NumericDocValuesField(org.apache.lucene.document.NumericDocValuesField) Field(org.apache.lucene.document.Field) StoredField(org.apache.lucene.document.StoredField) Shape(org.locationtech.spatial4j.shape.Shape) NumericDocValuesField(org.apache.lucene.document.NumericDocValuesField) Point(org.locationtech.spatial4j.shape.Point) Document(org.apache.lucene.document.Document)

Example 68 with StoredField

use of org.apache.lucene.document.StoredField in project lucene-solr by apache.

the class PointVectorStrategy method createIndexableFields.

/** @see #createIndexableFields(org.locationtech.spatial4j.shape.Shape) */
public Field[] createIndexableFields(Point point) {
    Field[] fields = new Field[fieldsLen];
    int idx = -1;
    if (hasStored) {
        fields[++idx] = new StoredField(fieldNameX, point.getX());
        fields[++idx] = new StoredField(fieldNameY, point.getY());
    }
    if (hasDocVals) {
        fields[++idx] = new DoubleDocValuesField(fieldNameX, point.getX());
        fields[++idx] = new DoubleDocValuesField(fieldNameY, point.getY());
    }
    if (hasPointVals) {
        fields[++idx] = new DoublePoint(fieldNameX, point.getX());
        fields[++idx] = new DoublePoint(fieldNameY, point.getY());
    }
    assert idx == fields.length - 1;
    return fields;
}
Also used : DoubleDocValuesField(org.apache.lucene.document.DoubleDocValuesField) StoredField(org.apache.lucene.document.StoredField) Field(org.apache.lucene.document.Field) StoredField(org.apache.lucene.document.StoredField) DoubleDocValuesField(org.apache.lucene.document.DoubleDocValuesField) DoublePoint(org.apache.lucene.document.DoublePoint) DoublePoint(org.apache.lucene.document.DoublePoint) Point(org.locationtech.spatial4j.shape.Point)

Example 69 with StoredField

use of org.apache.lucene.document.StoredField in project lucene-solr by apache.

the class StrategyTestCase method newDoc.

protected Document newDoc(String id, Shape shape) {
    Document doc = new Document();
    doc.add(new StringField("id", id, Field.Store.YES));
    if (shape != null) {
        for (Field f : strategy.createIndexableFields(shape)) {
            doc.add(f);
        }
        if (storeShape)
            //not to be parsed; just for debug
            doc.add(new StoredField(strategy.getFieldName(), shape.toString()));
    }
    return doc;
}
Also used : StringField(org.apache.lucene.document.StringField) StoredField(org.apache.lucene.document.StoredField) Field(org.apache.lucene.document.Field) StoredField(org.apache.lucene.document.StoredField) StringField(org.apache.lucene.document.StringField) Document(org.apache.lucene.document.Document)

Example 70 with StoredField

use of org.apache.lucene.document.StoredField in project lucene-solr by apache.

the class TestSuggestField method testSuggestOnMostlyDeletedDocuments.

@Test
public void testSuggestOnMostlyDeletedDocuments() throws Exception {
    Analyzer analyzer = new MockAnalyzer(random());
    // using IndexWriter instead of RandomIndexWriter
    IndexWriter iw = new IndexWriter(dir, iwcWithSuggestField(analyzer, "suggest_field"));
    int num = Math.min(1000, atLeast(10));
    for (int i = 1; i <= num; i++) {
        Document document = new Document();
        document.add(new SuggestField("suggest_field", "abc_" + i, i));
        document.add(new StoredField("weight_fld", i));
        document.add(new IntPoint("weight_fld", i));
        iw.addDocument(document);
        if (usually()) {
            iw.commit();
        }
    }
    iw.deleteDocuments(IntPoint.newRangeQuery("weight_fld", 2, Integer.MAX_VALUE));
    DirectoryReader reader = DirectoryReader.open(iw);
    SuggestIndexSearcher indexSearcher = new SuggestIndexSearcher(reader);
    PrefixCompletionQuery query = new PrefixCompletionQuery(analyzer, new Term("suggest_field", "abc_"));
    TopSuggestDocs suggest = indexSearcher.suggest(query, 1, false);
    assertSuggestions(suggest, new Entry("abc_1", 1));
    reader.close();
    iw.close();
}
Also used : DirectoryReader(org.apache.lucene.index.DirectoryReader) Term(org.apache.lucene.index.Term) Analyzer(org.apache.lucene.analysis.Analyzer) MockAnalyzer(org.apache.lucene.analysis.MockAnalyzer) Document(org.apache.lucene.document.Document) IntPoint(org.apache.lucene.document.IntPoint) IntPoint(org.apache.lucene.document.IntPoint) StoredField(org.apache.lucene.document.StoredField) MockAnalyzer(org.apache.lucene.analysis.MockAnalyzer) IndexWriter(org.apache.lucene.index.IndexWriter) RandomIndexWriter(org.apache.lucene.index.RandomIndexWriter) Test(org.junit.Test)

Aggregations

StoredField (org.apache.lucene.document.StoredField)110 Document (org.apache.lucene.document.Document)98 Directory (org.apache.lucene.store.Directory)72 StringField (org.apache.lucene.document.StringField)44 Field (org.apache.lucene.document.Field)40 MockAnalyzer (org.apache.lucene.analysis.MockAnalyzer)39 RandomIndexWriter (org.apache.lucene.index.RandomIndexWriter)36 BytesRef (org.apache.lucene.util.BytesRef)35 NumericDocValuesField (org.apache.lucene.document.NumericDocValuesField)34 TextField (org.apache.lucene.document.TextField)31 IndexReader (org.apache.lucene.index.IndexReader)29 IndexSearcher (org.apache.lucene.search.IndexSearcher)26 IntPoint (org.apache.lucene.document.IntPoint)24 SortedDocValuesField (org.apache.lucene.document.SortedDocValuesField)23 TopDocs (org.apache.lucene.search.TopDocs)23 SortField (org.apache.lucene.search.SortField)22 SortedNumericDocValuesField (org.apache.lucene.document.SortedNumericDocValuesField)21 Sort (org.apache.lucene.search.Sort)21 BinaryDocValuesField (org.apache.lucene.document.BinaryDocValuesField)18 MatchAllDocsQuery (org.apache.lucene.search.MatchAllDocsQuery)18