Search in sources :

Example 11 with Version

use of org.apache.lucene.util.Version in project lucene-solr by apache.

the class BaseSegmentInfoFormatTestCase method testVersions.

/** Test versions */
public void testVersions() throws Exception {
    Codec codec = getCodec();
    for (Version v : getVersions()) {
        for (Version minV : new Version[] { v, null }) {
            Directory dir = newDirectory();
            byte[] id = StringHelper.randomId();
            SegmentInfo info = new SegmentInfo(dir, v, minV, "_123", 1, false, codec, Collections.<String, String>emptyMap(), id, new HashMap<>(), null);
            info.setFiles(Collections.<String>emptySet());
            codec.segmentInfoFormat().write(dir, info, IOContext.DEFAULT);
            SegmentInfo info2 = codec.segmentInfoFormat().read(dir, "_123", id, IOContext.DEFAULT);
            assertEquals(info2.getVersion(), v);
            if (supportsMinVersion()) {
                assertEquals(info2.getMinVersion(), minV);
            } else {
                assertEquals(info2.getMinVersion(), null);
            }
            dir.close();
        }
    }
}
Also used : Codec(org.apache.lucene.codecs.Codec) Version(org.apache.lucene.util.Version) Directory(org.apache.lucene.store.Directory)

Example 12 with Version

use of org.apache.lucene.util.Version in project lucene-solr by apache.

the class Lucene50RWSegmentInfoFormat method write.

@Override
public void write(Directory dir, SegmentInfo si, IOContext ioContext) throws IOException {
    final String fileName = IndexFileNames.segmentFileName(si.name, "", Lucene50SegmentInfoFormat.SI_EXTENSION);
    assert si.getIndexSort() == null;
    try (IndexOutput output = dir.createOutput(fileName, ioContext)) {
        // Only add the file once we've successfully created it, else IFD assert can trip:
        si.addFile(fileName);
        CodecUtil.writeIndexHeader(output, Lucene50SegmentInfoFormat.CODEC_NAME, Lucene50SegmentInfoFormat.VERSION_CURRENT, si.getId(), "");
        Version version = si.getVersion();
        if (version.major < 5) {
            throw new IllegalArgumentException("invalid major version: should be >= 5 but got: " + version.major + " segment=" + si);
        }
        // Write the Lucene version that created this segment, since 3.1
        output.writeInt(version.major);
        output.writeInt(version.minor);
        output.writeInt(version.bugfix);
        assert version.prerelease == 0;
        output.writeInt(si.maxDoc());
        output.writeByte((byte) (si.getUseCompoundFile() ? SegmentInfo.YES : SegmentInfo.NO));
        output.writeMapOfStrings(si.getDiagnostics());
        Set<String> files = si.files();
        for (String file : files) {
            if (!IndexFileNames.parseSegmentName(file).equals(si.name)) {
                throw new IllegalArgumentException("invalid files: expected segment=" + si.name + ", got=" + files);
            }
        }
        output.writeSetOfStrings(files);
        output.writeMapOfStrings(si.getAttributes());
        CodecUtil.writeFooter(output);
    }
}
Also used : Version(org.apache.lucene.util.Version) IndexOutput(org.apache.lucene.store.IndexOutput)

Example 13 with Version

use of org.apache.lucene.util.Version in project lucene-solr by apache.

the class Lucene50RWSegmentInfoFormat method read.

@Override
public SegmentInfo read(Directory dir, String segment, byte[] segmentID, IOContext context) throws IOException {
    final String fileName = IndexFileNames.segmentFileName(segment, "", Lucene50SegmentInfoFormat.SI_EXTENSION);
    try (ChecksumIndexInput input = dir.openChecksumInput(fileName, context)) {
        Throwable priorE = null;
        SegmentInfo si = null;
        try {
            CodecUtil.checkIndexHeader(input, Lucene50SegmentInfoFormat.CODEC_NAME, Lucene50SegmentInfoFormat.VERSION_START, Lucene50SegmentInfoFormat.VERSION_CURRENT, segmentID, "");
            final Version version = Version.fromBits(input.readInt(), input.readInt(), input.readInt());
            final int docCount = input.readInt();
            if (docCount < 0) {
                throw new CorruptIndexException("invalid docCount: " + docCount, input);
            }
            final boolean isCompoundFile = input.readByte() == SegmentInfo.YES;
            final Map<String, String> diagnostics = input.readMapOfStrings();
            final Set<String> files = input.readSetOfStrings();
            final Map<String, String> attributes = input.readMapOfStrings();
            si = new SegmentInfo(dir, version, null, segment, docCount, isCompoundFile, null, diagnostics, segmentID, attributes, null);
            si.setFiles(files);
        } catch (Throwable exception) {
            priorE = exception;
        } finally {
            CodecUtil.checkFooter(input, priorE);
        }
        return si;
    }
}
Also used : ChecksumIndexInput(org.apache.lucene.store.ChecksumIndexInput) Version(org.apache.lucene.util.Version) SegmentInfo(org.apache.lucene.index.SegmentInfo) CorruptIndexException(org.apache.lucene.index.CorruptIndexException)

Example 14 with Version

use of org.apache.lucene.util.Version in project lucene-solr by apache.

the class TestBackwardsCompatibility method testOldVersions.

/** 
   * Test that we didn't forget to bump the current Constants.LUCENE_MAIN_VERSION.
   * This is important so that we can determine which version of lucene wrote the segment.
   */
public void testOldVersions() throws Exception {
    // first create a little index with the current code and get the version
    Directory currentDir = newDirectory();
    RandomIndexWriter riw = new RandomIndexWriter(random(), currentDir);
    riw.addDocument(new Document());
    riw.close();
    DirectoryReader ir = DirectoryReader.open(currentDir);
    SegmentReader air = (SegmentReader) ir.leaves().get(0).reader();
    Version currentVersion = air.getSegmentInfo().info.getVersion();
    // only 3.0 segments can have a null version
    assertNotNull(currentVersion);
    ir.close();
    currentDir.close();
    // now check all the old indexes, their version should be < the current version
    for (String name : oldNames) {
        Directory dir = oldIndexDirs.get(name);
        DirectoryReader r = DirectoryReader.open(dir);
        for (LeafReaderContext context : r.leaves()) {
            air = (SegmentReader) context.reader();
            Version oldVersion = air.getSegmentInfo().info.getVersion();
            // only 3.0 segments can have a null version
            assertNotNull(oldVersion);
            assertTrue("current Version.LATEST is <= an old index: did you forget to bump it?!", currentVersion.onOrAfter(oldVersion));
        }
        r.close();
    }
}
Also used : Version(org.apache.lucene.util.Version) Document(org.apache.lucene.document.Document) Directory(org.apache.lucene.store.Directory) RAMDirectory(org.apache.lucene.store.RAMDirectory) FSDirectory(org.apache.lucene.store.FSDirectory) SimpleFSDirectory(org.apache.lucene.store.SimpleFSDirectory) NIOFSDirectory(org.apache.lucene.store.NIOFSDirectory)

Example 15 with Version

use of org.apache.lucene.util.Version in project lucene-solr by apache.

the class Lucene62RWSegmentInfoFormat method write.

@Override
public void write(Directory dir, SegmentInfo si, IOContext ioContext) throws IOException {
    final String fileName = IndexFileNames.segmentFileName(si.name, "", Lucene62SegmentInfoFormat.SI_EXTENSION);
    try (IndexOutput output = dir.createOutput(fileName, ioContext)) {
        // Only add the file once we've successfully created it, else IFD assert can trip:
        si.addFile(fileName);
        CodecUtil.writeIndexHeader(output, Lucene62SegmentInfoFormat.CODEC_NAME, Lucene62SegmentInfoFormat.VERSION_CURRENT, si.getId(), "");
        Version version = si.getVersion();
        if (version.major < 5) {
            throw new IllegalArgumentException("invalid major version: should be >= 5 but got: " + version.major + " segment=" + si);
        }
        // Write the Lucene version that created this segment, since 3.1
        output.writeInt(version.major);
        output.writeInt(version.minor);
        output.writeInt(version.bugfix);
        assert version.prerelease == 0;
        output.writeInt(si.maxDoc());
        output.writeByte((byte) (si.getUseCompoundFile() ? SegmentInfo.YES : SegmentInfo.NO));
        output.writeMapOfStrings(si.getDiagnostics());
        Set<String> files = si.files();
        for (String file : files) {
            if (!IndexFileNames.parseSegmentName(file).equals(si.name)) {
                throw new IllegalArgumentException("invalid files: expected segment=" + si.name + ", got=" + files);
            }
        }
        output.writeSetOfStrings(files);
        output.writeMapOfStrings(si.getAttributes());
        Sort indexSort = si.getIndexSort();
        int numSortFields = indexSort == null ? 0 : indexSort.getSort().length;
        output.writeVInt(numSortFields);
        for (int i = 0; i < numSortFields; ++i) {
            SortField sortField = indexSort.getSort()[i];
            SortField.Type sortType = sortField.getType();
            output.writeString(sortField.getField());
            int sortTypeID;
            switch(sortField.getType()) {
                case STRING:
                    sortTypeID = 0;
                    break;
                case LONG:
                    sortTypeID = 1;
                    break;
                case INT:
                    sortTypeID = 2;
                    break;
                case DOUBLE:
                    sortTypeID = 3;
                    break;
                case FLOAT:
                    sortTypeID = 4;
                    break;
                case CUSTOM:
                    if (sortField instanceof SortedSetSortField) {
                        sortTypeID = 5;
                        sortType = SortField.Type.STRING;
                    } else if (sortField instanceof SortedNumericSortField) {
                        sortTypeID = 6;
                        sortType = ((SortedNumericSortField) sortField).getNumericType();
                    } else {
                        throw new IllegalStateException("Unexpected SortedNumericSortField " + sortField);
                    }
                    break;
                default:
                    throw new IllegalStateException("Unexpected sort type: " + sortField.getType());
            }
            output.writeVInt(sortTypeID);
            if (sortTypeID == 5) {
                SortedSetSortField ssf = (SortedSetSortField) sortField;
                if (ssf.getSelector() == SortedSetSelector.Type.MIN) {
                    output.writeByte((byte) 0);
                } else if (ssf.getSelector() == SortedSetSelector.Type.MAX) {
                    output.writeByte((byte) 1);
                } else if (ssf.getSelector() == SortedSetSelector.Type.MIDDLE_MIN) {
                    output.writeByte((byte) 2);
                } else if (ssf.getSelector() == SortedSetSelector.Type.MIDDLE_MAX) {
                    output.writeByte((byte) 3);
                } else {
                    throw new IllegalStateException("Unexpected SortedSetSelector type: " + ssf.getSelector());
                }
            } else if (sortTypeID == 6) {
                SortedNumericSortField snsf = (SortedNumericSortField) sortField;
                if (snsf.getNumericType() == SortField.Type.LONG) {
                    output.writeByte((byte) 0);
                } else if (snsf.getNumericType() == SortField.Type.INT) {
                    output.writeByte((byte) 1);
                } else if (snsf.getNumericType() == SortField.Type.DOUBLE) {
                    output.writeByte((byte) 2);
                } else if (snsf.getNumericType() == SortField.Type.FLOAT) {
                    output.writeByte((byte) 3);
                } else {
                    throw new IllegalStateException("Unexpected SortedNumericSelector type: " + snsf.getNumericType());
                }
                if (snsf.getSelector() == SortedNumericSelector.Type.MIN) {
                    output.writeByte((byte) 0);
                } else if (snsf.getSelector() == SortedNumericSelector.Type.MAX) {
                    output.writeByte((byte) 1);
                } else {
                    throw new IllegalStateException("Unexpected sorted numeric selector type: " + snsf.getSelector());
                }
            }
            output.writeByte((byte) (sortField.getReverse() ? 0 : 1));
            // write missing value 
            Object missingValue = sortField.getMissingValue();
            if (missingValue == null) {
                output.writeByte((byte) 0);
            } else {
                switch(sortType) {
                    case STRING:
                        if (missingValue == SortField.STRING_LAST) {
                            output.writeByte((byte) 1);
                        } else if (missingValue == SortField.STRING_FIRST) {
                            output.writeByte((byte) 2);
                        } else {
                            throw new AssertionError("unrecognized missing value for STRING field \"" + sortField.getField() + "\": " + missingValue);
                        }
                        break;
                    case LONG:
                        output.writeByte((byte) 1);
                        output.writeLong(((Long) missingValue).longValue());
                        break;
                    case INT:
                        output.writeByte((byte) 1);
                        output.writeInt(((Integer) missingValue).intValue());
                        break;
                    case DOUBLE:
                        output.writeByte((byte) 1);
                        output.writeLong(Double.doubleToLongBits(((Double) missingValue).doubleValue()));
                        break;
                    case FLOAT:
                        output.writeByte((byte) 1);
                        output.writeInt(Float.floatToIntBits(((Float) missingValue).floatValue()));
                        break;
                    default:
                        throw new IllegalStateException("Unexpected sort type: " + sortField.getType());
                }
            }
        }
        CodecUtil.writeFooter(output);
    }
}
Also used : SortedSetSortField(org.apache.lucene.search.SortedSetSortField) IndexOutput(org.apache.lucene.store.IndexOutput) SortedSetSortField(org.apache.lucene.search.SortedSetSortField) SortField(org.apache.lucene.search.SortField) SortedNumericSortField(org.apache.lucene.search.SortedNumericSortField) SortedNumericSortField(org.apache.lucene.search.SortedNumericSortField) Version(org.apache.lucene.util.Version) Sort(org.apache.lucene.search.Sort)

Aggregations

Version (org.apache.lucene.util.Version)22 Directory (org.apache.lucene.store.Directory)7 Sort (org.apache.lucene.search.Sort)6 SortField (org.apache.lucene.search.SortField)5 SortedNumericSortField (org.apache.lucene.search.SortedNumericSortField)5 SortedSetSortField (org.apache.lucene.search.SortedSetSortField)5 IOException (java.io.IOException)4 HashSet (java.util.HashSet)4 Codec (org.apache.lucene.codecs.Codec)4 Document (org.apache.lucene.document.Document)4 CorruptIndexException (org.apache.lucene.index.CorruptIndexException)4 SegmentInfo (org.apache.lucene.index.SegmentInfo)4 ChecksumIndexInput (org.apache.lucene.store.ChecksumIndexInput)4 ArrayList (java.util.ArrayList)3 HashMap (java.util.HashMap)3 Collectors (java.util.stream.Collectors)3 IntStream (java.util.stream.IntStream)3 Store (org.apache.lucene.document.Field.Store)3 TextField (org.apache.lucene.document.TextField)3 DirectoryReader (org.apache.lucene.index.DirectoryReader)3