Search in sources :

Example 1 with Lucene62Codec

use of org.apache.lucene.codecs.lucene62.Lucene62Codec in project Anserini by castorini.

the class IndexTopics method run.

private void run() throws IOException, InterruptedException {
    final long start = System.nanoTime();
    LOG.info("Starting indexer...");
    final Directory dir = FSDirectory.open(indexPath);
    final SimpleAnalyzer analyzer = new SimpleAnalyzer();
    final IndexWriterConfig config = new IndexWriterConfig(analyzer);
    config.setOpenMode(IndexWriterConfig.OpenMode.CREATE);
    config.setCodec(new Lucene62Codec(Lucene50StoredFieldsFormat.Mode.BEST_SPEED));
    config.setUseCompoundFile(false);
    final IndexWriter writer = new IndexWriter(dir, config);
    index(writer, collectionPath);
    int numIndexed = writer.maxDoc();
    try {
        writer.commit();
    } finally {
        try {
            writer.close();
        } catch (IOException e) {
            LOG.error(e);
        }
    }
    LOG.info("Indexed documents: " + counters.indexedDocuments.get());
    final long durationMillis = TimeUnit.MILLISECONDS.convert(System.nanoTime() - start, TimeUnit.NANOSECONDS);
    LOG.info("Total " + numIndexed + " documents indexed in " + DurationFormatUtils.formatDuration(durationMillis, "HH:mm:ss"));
}
Also used : SimpleAnalyzer(org.apache.lucene.analysis.core.SimpleAnalyzer) IndexWriter(org.apache.lucene.index.IndexWriter) IOException(java.io.IOException) Lucene62Codec(org.apache.lucene.codecs.lucene62.Lucene62Codec) Directory(org.apache.lucene.store.Directory) FSDirectory(org.apache.lucene.store.FSDirectory) IndexWriterConfig(org.apache.lucene.index.IndexWriterConfig)

Example 2 with Lucene62Codec

use of org.apache.lucene.codecs.lucene62.Lucene62Codec in project Anserini by castorini.

the class IndexObjectTriples method run.

private void run() throws IOException, InterruptedException {
    final long start = System.nanoTime();
    LOG.info("Starting indexer...");
    final Directory dir = FSDirectory.open(indexPath);
    final EnglishAnalyzer analyzer = new EnglishAnalyzer();
    final IndexWriterConfig config = new IndexWriterConfig(analyzer);
    config.setOpenMode(IndexWriterConfig.OpenMode.CREATE);
    config.setCodec(new Lucene62Codec(Lucene50StoredFieldsFormat.Mode.BEST_SPEED));
    config.setUseCompoundFile(false);
    final IndexWriter writer = new IndexWriter(dir, config);
    index(writer, collectionPath);
    int numIndexed = writer.maxDoc();
    try {
        writer.commit();
    } finally {
        try {
            writer.close();
        } catch (IOException e) {
            LOG.error(e);
        }
    }
    LOG.info("Indexed documents: " + counters.indexedDocuments.get());
    final long durationMillis = TimeUnit.MILLISECONDS.convert(System.nanoTime() - start, TimeUnit.NANOSECONDS);
    LOG.info("Total " + numIndexed + " documents indexed in " + DurationFormatUtils.formatDuration(durationMillis, "HH:mm:ss"));
}
Also used : IndexWriter(org.apache.lucene.index.IndexWriter) EnglishAnalyzer(org.apache.lucene.analysis.en.EnglishAnalyzer) IOException(java.io.IOException) Lucene62Codec(org.apache.lucene.codecs.lucene62.Lucene62Codec) Directory(org.apache.lucene.store.Directory) FSDirectory(org.apache.lucene.store.FSDirectory) IndexWriterConfig(org.apache.lucene.index.IndexWriterConfig)

Aggregations

IOException (java.io.IOException)2 Lucene62Codec (org.apache.lucene.codecs.lucene62.Lucene62Codec)2 IndexWriter (org.apache.lucene.index.IndexWriter)2 IndexWriterConfig (org.apache.lucene.index.IndexWriterConfig)2 Directory (org.apache.lucene.store.Directory)2 FSDirectory (org.apache.lucene.store.FSDirectory)2 SimpleAnalyzer (org.apache.lucene.analysis.core.SimpleAnalyzer)1 EnglishAnalyzer (org.apache.lucene.analysis.en.EnglishAnalyzer)1