Search in sources :

Example 1 with NullInfoStream

use of org.apache.lucene.util.NullInfoStream in project crate by crate.

the class RecoverySourcePruneMergePolicyTests method testPruneAll.

public void testPruneAll() throws IOException {
    try (Directory dir = newDirectory()) {
        IndexWriterConfig iwc = newIndexWriterConfig();
        RecoverySourcePruneMergePolicy mp = new RecoverySourcePruneMergePolicy("extra_source", MatchNoDocsQuery::new, newLogMergePolicy());
        iwc.setMergePolicy(new ShuffleForcedMergePolicy(mp));
        try (IndexWriter writer = new IndexWriter(dir, iwc)) {
            for (int i = 0; i < 20; i++) {
                if (i > 0 && randomBoolean()) {
                    writer.flush();
                }
                Document doc = new Document();
                doc.add(new StoredField("source", "hello world"));
                doc.add(new StoredField("extra_source", "hello world"));
                doc.add(new NumericDocValuesField("extra_source", 1));
                writer.addDocument(doc);
            }
            writer.forceMerge(1);
            writer.commit();
            try (DirectoryReader reader = DirectoryReader.open(writer)) {
                for (int i = 0; i < reader.maxDoc(); i++) {
                    Document document = reader.document(i);
                    assertEquals(1, document.getFields().size());
                    assertEquals("source", document.getFields().get(0).name());
                }
                assertEquals(1, reader.leaves().size());
                LeafReader leafReader = reader.leaves().get(0).reader();
                NumericDocValues extra_source = leafReader.getNumericDocValues("extra_source");
                if (extra_source != null) {
                    assertEquals(DocIdSetIterator.NO_MORE_DOCS, extra_source.nextDoc());
                }
                if (leafReader instanceof CodecReader && reader instanceof StandardDirectoryReader) {
                    CodecReader codecReader = (CodecReader) leafReader;
                    StandardDirectoryReader sdr = (StandardDirectoryReader) reader;
                    SegmentInfos segmentInfos = sdr.getSegmentInfos();
                    MergePolicy.MergeSpecification forcedMerges = mp.findForcedDeletesMerges(segmentInfos, new MergePolicy.MergeContext() {

                        @Override
                        public int numDeletesToMerge(SegmentCommitInfo info) {
                            return info.info.maxDoc() - 1;
                        }

                        @Override
                        public int numDeletedDocs(SegmentCommitInfo info) {
                            return info.info.maxDoc() - 1;
                        }

                        @Override
                        public InfoStream getInfoStream() {
                            return new NullInfoStream();
                        }

                        @Override
                        public Set<SegmentCommitInfo> getMergingSegments() {
                            return Collections.emptySet();
                        }
                    });
                    // don't wrap if there is nothing to do
                    assertSame(codecReader, forcedMerges.merges.get(0).wrapForMerge(codecReader));
                }
            }
        }
    }
}
Also used : NumericDocValues(org.apache.lucene.index.NumericDocValues) CodecReader(org.apache.lucene.index.CodecReader) SegmentInfos(org.apache.lucene.index.SegmentInfos) Set(java.util.Set) LeafReader(org.apache.lucene.index.LeafReader) SegmentCommitInfo(org.apache.lucene.index.SegmentCommitInfo) StandardDirectoryReader(org.apache.lucene.index.StandardDirectoryReader) DirectoryReader(org.apache.lucene.index.DirectoryReader) StandardDirectoryReader(org.apache.lucene.index.StandardDirectoryReader) MatchNoDocsQuery(org.apache.lucene.search.MatchNoDocsQuery) Document(org.apache.lucene.document.Document) NullInfoStream(org.apache.lucene.util.NullInfoStream) InfoStream(org.apache.lucene.util.InfoStream) StoredField(org.apache.lucene.document.StoredField) NumericDocValuesField(org.apache.lucene.document.NumericDocValuesField) NullInfoStream(org.apache.lucene.util.NullInfoStream) IndexWriter(org.apache.lucene.index.IndexWriter) MergePolicy(org.apache.lucene.index.MergePolicy) ShuffleForcedMergePolicy(org.apache.lucene.index.ShuffleForcedMergePolicy) ShuffleForcedMergePolicy(org.apache.lucene.index.ShuffleForcedMergePolicy) Directory(org.apache.lucene.store.Directory) IndexWriterConfig(org.apache.lucene.index.IndexWriterConfig)

Aggregations

Set (java.util.Set)1 Document (org.apache.lucene.document.Document)1 NumericDocValuesField (org.apache.lucene.document.NumericDocValuesField)1 StoredField (org.apache.lucene.document.StoredField)1 CodecReader (org.apache.lucene.index.CodecReader)1 DirectoryReader (org.apache.lucene.index.DirectoryReader)1 IndexWriter (org.apache.lucene.index.IndexWriter)1 IndexWriterConfig (org.apache.lucene.index.IndexWriterConfig)1 LeafReader (org.apache.lucene.index.LeafReader)1 MergePolicy (org.apache.lucene.index.MergePolicy)1 NumericDocValues (org.apache.lucene.index.NumericDocValues)1 SegmentCommitInfo (org.apache.lucene.index.SegmentCommitInfo)1 SegmentInfos (org.apache.lucene.index.SegmentInfos)1 ShuffleForcedMergePolicy (org.apache.lucene.index.ShuffleForcedMergePolicy)1 StandardDirectoryReader (org.apache.lucene.index.StandardDirectoryReader)1 MatchNoDocsQuery (org.apache.lucene.search.MatchNoDocsQuery)1 Directory (org.apache.lucene.store.Directory)1 InfoStream (org.apache.lucene.util.InfoStream)1 NullInfoStream (org.apache.lucene.util.NullInfoStream)1