Search in sources :

Example 61 with FieldType

use of org.apache.lucene.document.FieldType in project lucene-solr by apache.

the class TestTermVectors method beforeClass.

@BeforeClass
public static void beforeClass() throws Exception {
    directory = newDirectory();
    RandomIndexWriter writer = new RandomIndexWriter(random(), directory, newIndexWriterConfig(new MockAnalyzer(random(), MockTokenizer.SIMPLE, true)).setMergePolicy(newLogMergePolicy()));
    //writer.infoStream = System.out;
    for (int i = 0; i < 1000; i++) {
        Document doc = new Document();
        FieldType ft = new FieldType(TextField.TYPE_STORED);
        int mod3 = i % 3;
        int mod2 = i % 2;
        if (mod2 == 0 && mod3 == 0) {
            ft.setStoreTermVectors(true);
            ft.setStoreTermVectorOffsets(true);
            ft.setStoreTermVectorPositions(true);
        } else if (mod2 == 0) {
            ft.setStoreTermVectors(true);
            ft.setStoreTermVectorPositions(true);
        } else if (mod3 == 0) {
            ft.setStoreTermVectors(true);
            ft.setStoreTermVectorOffsets(true);
        } else {
            ft.setStoreTermVectors(true);
        }
        doc.add(new Field("field", English.intToEnglish(i), ft));
        //test no term vectors too
        doc.add(new TextField("noTV", English.intToEnglish(i), Field.Store.YES));
        writer.addDocument(doc);
    }
    reader = writer.getReader();
    writer.close();
}
Also used : Field(org.apache.lucene.document.Field) TextField(org.apache.lucene.document.TextField) MockAnalyzer(org.apache.lucene.analysis.MockAnalyzer) TextField(org.apache.lucene.document.TextField) Document(org.apache.lucene.document.Document) RandomIndexWriter(org.apache.lucene.index.RandomIndexWriter) FieldType(org.apache.lucene.document.FieldType) BeforeClass(org.junit.BeforeClass)

Example 62 with FieldType

use of org.apache.lucene.document.FieldType in project lucene-solr by apache.

the class TestTermVectorsReader method testIllegalVectorPayloadsWithoutVectors.

public void testIllegalVectorPayloadsWithoutVectors() throws Exception {
    Directory dir = newDirectory();
    MockAnalyzer a = new MockAnalyzer(random());
    a.setEnableChecks(false);
    RandomIndexWriter w = new RandomIndexWriter(random(), dir, a);
    FieldType ft = new FieldType(TextField.TYPE_NOT_STORED);
    ft.setStoreTermVectors(false);
    ft.setStoreTermVectorPayloads(true);
    Document doc = new Document();
    doc.add(new Field("field", "value", ft));
    IllegalArgumentException expected = expectThrows(IllegalArgumentException.class, () -> {
        w.addDocument(doc);
    });
    assertEquals("cannot index term vector payloads when term vectors are not indexed (field=\"field\")", expected.getMessage());
    w.close();
    dir.close();
}
Also used : StoredField(org.apache.lucene.document.StoredField) Field(org.apache.lucene.document.Field) TextField(org.apache.lucene.document.TextField) Document(org.apache.lucene.document.Document) Directory(org.apache.lucene.store.Directory) FieldType(org.apache.lucene.document.FieldType)

Example 63 with FieldType

use of org.apache.lucene.document.FieldType in project lucene-solr by apache.

the class TestTermVectorsReader method testIllegalOffsetsWithoutVectors.

public void testIllegalOffsetsWithoutVectors() throws Exception {
    Directory dir = newDirectory();
    MockAnalyzer a = new MockAnalyzer(random());
    a.setEnableChecks(false);
    RandomIndexWriter w = new RandomIndexWriter(random(), dir, a);
    FieldType ft = new FieldType(TextField.TYPE_NOT_STORED);
    ft.setStoreTermVectors(false);
    ft.setStoreTermVectorOffsets(true);
    Document doc = new Document();
    doc.add(new Field("field", "value", ft));
    IllegalArgumentException expected = expectThrows(IllegalArgumentException.class, () -> {
        w.addDocument(doc);
    });
    assertEquals("cannot index term vector offsets when term vectors are not indexed (field=\"field\")", expected.getMessage());
    w.close();
    dir.close();
}
Also used : StoredField(org.apache.lucene.document.StoredField) Field(org.apache.lucene.document.Field) TextField(org.apache.lucene.document.TextField) Document(org.apache.lucene.document.Document) Directory(org.apache.lucene.store.Directory) FieldType(org.apache.lucene.document.FieldType)

Example 64 with FieldType

use of org.apache.lucene.document.FieldType in project lucene-solr by apache.

the class TestTermVectorsReader method testIllegalVectorPayloadsWithoutIndexed.

public void testIllegalVectorPayloadsWithoutIndexed() throws Exception {
    Directory dir = newDirectory();
    MockAnalyzer a = new MockAnalyzer(random());
    a.setEnableChecks(false);
    RandomIndexWriter w = new RandomIndexWriter(random(), dir, a);
    FieldType ft = new FieldType(StoredField.TYPE);
    ft.setStoreTermVectorPayloads(true);
    Document doc = new Document();
    doc.add(new Field("field", "value", ft));
    IllegalArgumentException expected = expectThrows(IllegalArgumentException.class, () -> {
        w.addDocument(doc);
    });
    assertEquals("cannot store term vector payloads for a field that is not indexed (field=\"field\")", expected.getMessage());
    w.close();
    dir.close();
}
Also used : StoredField(org.apache.lucene.document.StoredField) Field(org.apache.lucene.document.Field) TextField(org.apache.lucene.document.TextField) Document(org.apache.lucene.document.Document) Directory(org.apache.lucene.store.Directory) FieldType(org.apache.lucene.document.FieldType)

Example 65 with FieldType

use of org.apache.lucene.document.FieldType in project lucene-solr by apache.

the class TestTermVectorsReader method testIllegalVectorOffsetsWithoutIndexed.

public void testIllegalVectorOffsetsWithoutIndexed() throws Exception {
    Directory dir = newDirectory();
    MockAnalyzer a = new MockAnalyzer(random());
    a.setEnableChecks(false);
    RandomIndexWriter w = new RandomIndexWriter(random(), dir, a);
    FieldType ft = new FieldType(StoredField.TYPE);
    ft.setStoreTermVectorOffsets(true);
    Document doc = new Document();
    doc.add(new Field("field", "value", ft));
    IllegalArgumentException expected = expectThrows(IllegalArgumentException.class, () -> {
        w.addDocument(doc);
    });
    assertEquals("cannot store term vector offsets for a field that is not indexed (field=\"field\")", expected.getMessage());
    w.close();
    dir.close();
}
Also used : StoredField(org.apache.lucene.document.StoredField) Field(org.apache.lucene.document.Field) TextField(org.apache.lucene.document.TextField) Document(org.apache.lucene.document.Document) Directory(org.apache.lucene.store.Directory) FieldType(org.apache.lucene.document.FieldType)

Aggregations

FieldType (org.apache.lucene.document.FieldType)262 Document (org.apache.lucene.document.Document)229 Field (org.apache.lucene.document.Field)191 Directory (org.apache.lucene.store.Directory)172 TextField (org.apache.lucene.document.TextField)166 MockAnalyzer (org.apache.lucene.analysis.MockAnalyzer)125 StringField (org.apache.lucene.document.StringField)72 StoredField (org.apache.lucene.document.StoredField)65 IndexReader (org.apache.lucene.index.IndexReader)49 IndexWriter (org.apache.lucene.index.IndexWriter)49 BytesRef (org.apache.lucene.util.BytesRef)47 RandomIndexWriter (org.apache.lucene.index.RandomIndexWriter)45 IndexSearcher (org.apache.lucene.search.IndexSearcher)45 Term (org.apache.lucene.index.Term)40 NumericDocValuesField (org.apache.lucene.document.NumericDocValuesField)38 RAMDirectory (org.apache.lucene.store.RAMDirectory)37 TermQuery (org.apache.lucene.search.TermQuery)33 TopDocs (org.apache.lucene.search.TopDocs)32 IndexWriterConfig (org.apache.lucene.index.IndexWriterConfig)30 Analyzer (org.apache.lucene.analysis.Analyzer)27