Search in sources :

Example 21 with PerfRunData

use of org.apache.lucene.benchmark.byTask.PerfRunData in project lucene-solr by apache.

the class WriteLineDocTaskTest method testBZip2.

/* Tests WriteLineDocTask with a bzip2 format. */
public void testBZip2() throws Exception {
    // Create a document in bz2 format.
    Path file = getWorkDir().resolve("one-line.bz2");
    PerfRunData runData = createPerfRunData(file, false, WriteLineDocMaker.class.getName());
    WriteLineDocTask wldt = new WriteLineDocTask(runData);
    wldt.doLogic();
    wldt.close();
    doReadTest(file, Type.BZIP2, "title", "date", "body");
}
Also used : Path(java.nio.file.Path) PerfRunData(org.apache.lucene.benchmark.byTask.PerfRunData)

Example 22 with PerfRunData

use of org.apache.lucene.benchmark.byTask.PerfRunData in project lucene-solr by apache.

the class WriteLineDocTaskTest method testJustDate.

/** Fail by default when there's only date */
public void testJustDate() throws Exception {
    Path file = getWorkDir().resolve("one-line");
    PerfRunData runData = createPerfRunData(file, false, JustDateDocMaker.class.getName());
    WriteLineDocTask wldt = new WriteLineDocTask(runData);
    wldt.doLogic();
    wldt.close();
    try (BufferedReader br = Files.newBufferedReader(file, StandardCharsets.UTF_8)) {
        String line = br.readLine();
        assertHeaderLine(line);
        line = br.readLine();
        assertNull(line);
    }
}
Also used : Path(java.nio.file.Path) BufferedReader(java.io.BufferedReader) PerfRunData(org.apache.lucene.benchmark.byTask.PerfRunData)

Example 23 with PerfRunData

use of org.apache.lucene.benchmark.byTask.PerfRunData in project lucene-solr by apache.

the class Sample method main.

public static void main(String[] args) throws Exception {
    Properties p = initProps();
    Config conf = new Config(p);
    PerfRunData runData = new PerfRunData(conf);
    // 1. top sequence
    // top level, not parallel
    TaskSequence top = new TaskSequence(runData, null, null, false);
    // 2. task to create the index
    CreateIndexTask create = new CreateIndexTask(runData);
    top.addTask(create);
    // 3. task seq to add 500 docs (order matters - top to bottom - add seq to top, only then add to seq)
    TaskSequence seq1 = new TaskSequence(runData, "AddDocs", top, false);
    seq1.setRepetitions(500);
    seq1.setNoChildReport();
    top.addTask(seq1);
    // 4. task to add the doc
    AddDocTask addDoc = new AddDocTask(runData);
    //addDoc.setParams("1200"); // doc size limit if supported
    // order matters 9see comment above)
    seq1.addTask(addDoc);
    // 5. task to close the index
    CloseIndexTask close = new CloseIndexTask(runData);
    top.addTask(close);
    // task to report
    RepSumByNameTask rep = new RepSumByNameTask(runData);
    top.addTask(rep);
    // print algorithm
    System.out.println(top.toString());
    // execute
    top.doLogic();
}
Also used : CloseIndexTask(org.apache.lucene.benchmark.byTask.tasks.CloseIndexTask) RepSumByNameTask(org.apache.lucene.benchmark.byTask.tasks.RepSumByNameTask) TaskSequence(org.apache.lucene.benchmark.byTask.tasks.TaskSequence) Config(org.apache.lucene.benchmark.byTask.utils.Config) CreateIndexTask(org.apache.lucene.benchmark.byTask.tasks.CreateIndexTask) PerfRunData(org.apache.lucene.benchmark.byTask.PerfRunData) Properties(java.util.Properties) AddDocTask(org.apache.lucene.benchmark.byTask.tasks.AddDocTask)

Example 24 with PerfRunData

use of org.apache.lucene.benchmark.byTask.PerfRunData in project lucene-solr by apache.

the class SearchWithCollectorTask method setup.

@Override
public void setup() throws Exception {
    super.setup();
    //check to make sure either the doc is being stored
    PerfRunData runData = getRunData();
    Config config = runData.getConfig();
    clnName = config.get("collector.class", "");
}
Also used : Config(org.apache.lucene.benchmark.byTask.utils.Config) PerfRunData(org.apache.lucene.benchmark.byTask.PerfRunData)

Example 25 with PerfRunData

use of org.apache.lucene.benchmark.byTask.PerfRunData in project lucene-solr by apache.

the class OpenIndexTask method doLogic.

@Override
public int doLogic() throws IOException {
    PerfRunData runData = getRunData();
    Config config = runData.getConfig();
    final IndexCommit ic;
    if (commitUserData != null) {
        ic = OpenReaderTask.findIndexCommit(runData.getDirectory(), commitUserData);
    } else {
        ic = null;
    }
    final IndexWriter writer = CreateIndexTask.configureWriter(config, runData, OpenMode.APPEND, ic);
    runData.setIndexWriter(writer);
    return 1;
}
Also used : IndexWriter(org.apache.lucene.index.IndexWriter) Config(org.apache.lucene.benchmark.byTask.utils.Config) IndexWriterConfig(org.apache.lucene.index.IndexWriterConfig) PerfRunData(org.apache.lucene.benchmark.byTask.PerfRunData) IndexCommit(org.apache.lucene.index.IndexCommit)

Aggregations

PerfRunData (org.apache.lucene.benchmark.byTask.PerfRunData)40 Config (org.apache.lucene.benchmark.byTask.utils.Config)13 Path (java.nio.file.Path)12 Properties (java.util.Properties)10 BufferedReader (java.io.BufferedReader)4 AddDocTask (org.apache.lucene.benchmark.byTask.tasks.AddDocTask)3 CloseIndexTask (org.apache.lucene.benchmark.byTask.tasks.CloseIndexTask)3 CreateIndexTask (org.apache.lucene.benchmark.byTask.tasks.CreateIndexTask)3 TaskSequence (org.apache.lucene.benchmark.byTask.tasks.TaskSequence)3 IndexWriterConfig (org.apache.lucene.index.IndexWriterConfig)3 WhitespaceAnalyzer (org.apache.lucene.analysis.core.WhitespaceAnalyzer)2 DirectoryTaxonomyWriter (org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter)2 IndexReader (org.apache.lucene.index.IndexReader)2 IndexWriter (org.apache.lucene.index.IndexWriter)2 Term (org.apache.lucene.index.Term)2 IndexSearcher (org.apache.lucene.search.IndexSearcher)2 TermQuery (org.apache.lucene.search.TermQuery)2 TopDocs (org.apache.lucene.search.TopDocs)2 ByteArrayOutputStream (java.io.ByteArrayOutputStream)1 PrintStream (java.io.PrintStream)1