Search in sources :

Example 1 with SleepingLockWrapper

use of org.apache.lucene.store.SleepingLockWrapper in project crate by crate.

the class FsDirectoryFactoryTests method doTestPreload.

private void doTestPreload(String... preload) throws IOException {
    Settings build = Settings.builder().put(IndexModule.INDEX_STORE_TYPE_SETTING.getKey(), "mmapfs").putList(IndexModule.INDEX_STORE_PRE_LOAD_SETTING.getKey(), preload).build();
    Directory directory = newDirectory(build);
    try (Directory dir = directory) {
        // prevent warnings
        assertSame(dir, directory);
        assertFalse(directory instanceof SleepingLockWrapper);
        if (preload.length == 0) {
            assertTrue(directory.toString(), directory instanceof MMapDirectory);
            assertFalse(((MMapDirectory) directory).getPreload());
        } else if (Arrays.asList(preload).contains("*")) {
            assertTrue(directory.toString(), directory instanceof MMapDirectory);
            assertTrue(((MMapDirectory) directory).getPreload());
        } else {
            assertTrue(directory.toString(), directory instanceof FsDirectoryFactory.PreLoadMMapDirectory);
            FsDirectoryFactory.PreLoadMMapDirectory preLoadMMapDirectory = (FsDirectoryFactory.PreLoadMMapDirectory) directory;
            for (String ext : preload) {
                assertTrue("ext: " + ext, preLoadMMapDirectory.useDelegate("foo." + ext));
                assertTrue("ext: " + ext, preLoadMMapDirectory.getDelegate().getPreload());
            }
            assertFalse(preLoadMMapDirectory.useDelegate("XXX"));
            assertFalse(preLoadMMapDirectory.getPreload());
            preLoadMMapDirectory.close();
            expectThrows(AlreadyClosedException.class, () -> preLoadMMapDirectory.getDelegate().openInput("foo.bar", IOContext.DEFAULT));
        }
    }
    expectThrows(AlreadyClosedException.class, () -> directory.openInput(randomBoolean() && preload.length != 0 ? "foo." + preload[0] : "foo.bar", IOContext.DEFAULT));
}
Also used : SleepingLockWrapper(org.apache.lucene.store.SleepingLockWrapper) AlreadyClosedException(org.apache.lucene.store.AlreadyClosedException) MMapDirectory(org.apache.lucene.store.MMapDirectory) Settings(org.elasticsearch.common.settings.Settings) IndexSettings(org.elasticsearch.index.IndexSettings) MMapDirectory(org.apache.lucene.store.MMapDirectory) Directory(org.apache.lucene.store.Directory) SimpleFSDirectory(org.apache.lucene.store.SimpleFSDirectory) NIOFSDirectory(org.apache.lucene.store.NIOFSDirectory)

Example 2 with SleepingLockWrapper

use of org.apache.lucene.store.SleepingLockWrapper in project elasticsearch by elastic.

the class FsDirectoryService method newDirectory.

@Override
public Directory newDirectory() throws IOException {
    final Path location = path.resolveIndex();
    final LockFactory lockFactory = indexSettings.getValue(INDEX_LOCK_FACTOR_SETTING);
    Files.createDirectories(location);
    Directory wrapped = newFSDirectory(location, lockFactory);
    Set<String> preLoadExtensions = new HashSet<>(indexSettings.getValue(IndexModule.INDEX_STORE_PRE_LOAD_SETTING));
    wrapped = setPreload(wrapped, location, lockFactory, preLoadExtensions);
    if (indexSettings.isOnSharedFilesystem()) {
        wrapped = new SleepingLockWrapper(wrapped, 5000);
    }
    return wrapped;
}
Also used : ShardPath(org.elasticsearch.index.shard.ShardPath) Path(java.nio.file.Path) SleepingLockWrapper(org.apache.lucene.store.SleepingLockWrapper) SimpleFSLockFactory(org.apache.lucene.store.SimpleFSLockFactory) NativeFSLockFactory(org.apache.lucene.store.NativeFSLockFactory) LockFactory(org.apache.lucene.store.LockFactory) SimpleFSDirectory(org.apache.lucene.store.SimpleFSDirectory) MMapDirectory(org.apache.lucene.store.MMapDirectory) Directory(org.apache.lucene.store.Directory) NIOFSDirectory(org.apache.lucene.store.NIOFSDirectory) FSDirectory(org.apache.lucene.store.FSDirectory) FileSwitchDirectory(org.apache.lucene.store.FileSwitchDirectory) HashSet(java.util.HashSet)

Example 3 with SleepingLockWrapper

use of org.apache.lucene.store.SleepingLockWrapper in project elasticsearch by elastic.

the class FsDirectoryServiceTests method testHasSleepWrapperOnSharedFS.

public void testHasSleepWrapperOnSharedFS() throws IOException {
    Settings build = randomBoolean() ? Settings.builder().put(IndexMetaData.SETTING_SHARED_FILESYSTEM, true).build() : Settings.builder().put(IndexMetaData.SETTING_SHADOW_REPLICAS, true).build();
    ;
    IndexSettings settings = IndexSettingsModule.newIndexSettings("foo", build);
    IndexStore store = new IndexStore(settings);
    Path tempDir = createTempDir().resolve(settings.getUUID()).resolve("0");
    Files.createDirectories(tempDir);
    ShardPath path = new ShardPath(false, tempDir, tempDir, new ShardId(settings.getIndex(), 0));
    FsDirectoryService fsDirectoryService = new FsDirectoryService(settings, store, path);
    Directory directory = fsDirectoryService.newDirectory();
    assertTrue(directory.getClass().toString(), directory instanceof SleepingLockWrapper);
}
Also used : ShardPath(org.elasticsearch.index.shard.ShardPath) Path(java.nio.file.Path) ShardId(org.elasticsearch.index.shard.ShardId) ShardPath(org.elasticsearch.index.shard.ShardPath) IndexSettings(org.elasticsearch.index.IndexSettings) SleepingLockWrapper(org.apache.lucene.store.SleepingLockWrapper) Settings(org.elasticsearch.common.settings.Settings) IndexSettings(org.elasticsearch.index.IndexSettings) SimpleFSDirectory(org.apache.lucene.store.SimpleFSDirectory) MMapDirectory(org.apache.lucene.store.MMapDirectory) Directory(org.apache.lucene.store.Directory) FileSwitchDirectory(org.apache.lucene.store.FileSwitchDirectory)

Example 4 with SleepingLockWrapper

use of org.apache.lucene.store.SleepingLockWrapper in project elasticsearch by elastic.

the class FsDirectoryServiceTests method doTestPreload.

private void doTestPreload(String... preload) throws IOException {
    Settings build = Settings.builder().put(IndexModule.INDEX_STORE_TYPE_SETTING.getKey(), "mmapfs").putArray(IndexModule.INDEX_STORE_PRE_LOAD_SETTING.getKey(), preload).build();
    IndexSettings settings = IndexSettingsModule.newIndexSettings("foo", build);
    IndexStore store = new IndexStore(settings);
    Path tempDir = createTempDir().resolve(settings.getUUID()).resolve("0");
    Files.createDirectories(tempDir);
    ShardPath path = new ShardPath(false, tempDir, tempDir, new ShardId(settings.getIndex(), 0));
    FsDirectoryService fsDirectoryService = new FsDirectoryService(settings, store, path);
    Directory directory = fsDirectoryService.newDirectory();
    assertFalse(directory instanceof SleepingLockWrapper);
    if (preload.length == 0) {
        assertTrue(directory.toString(), directory instanceof MMapDirectory);
        assertFalse(((MMapDirectory) directory).getPreload());
    } else if (Arrays.asList(preload).contains("*")) {
        assertTrue(directory.toString(), directory instanceof MMapDirectory);
        assertTrue(((MMapDirectory) directory).getPreload());
    } else {
        assertTrue(directory.toString(), directory instanceof FileSwitchDirectory);
        FileSwitchDirectory fsd = (FileSwitchDirectory) directory;
        assertTrue(fsd.getPrimaryDir() instanceof MMapDirectory);
        assertTrue(((MMapDirectory) fsd.getPrimaryDir()).getPreload());
        assertTrue(fsd.getSecondaryDir() instanceof MMapDirectory);
        assertFalse(((MMapDirectory) fsd.getSecondaryDir()).getPreload());
    }
}
Also used : ShardPath(org.elasticsearch.index.shard.ShardPath) Path(java.nio.file.Path) ShardId(org.elasticsearch.index.shard.ShardId) ShardPath(org.elasticsearch.index.shard.ShardPath) IndexSettings(org.elasticsearch.index.IndexSettings) SleepingLockWrapper(org.apache.lucene.store.SleepingLockWrapper) FileSwitchDirectory(org.apache.lucene.store.FileSwitchDirectory) MMapDirectory(org.apache.lucene.store.MMapDirectory) Settings(org.elasticsearch.common.settings.Settings) IndexSettings(org.elasticsearch.index.IndexSettings) SimpleFSDirectory(org.apache.lucene.store.SimpleFSDirectory) MMapDirectory(org.apache.lucene.store.MMapDirectory) Directory(org.apache.lucene.store.Directory) FileSwitchDirectory(org.apache.lucene.store.FileSwitchDirectory)

Example 5 with SleepingLockWrapper

use of org.apache.lucene.store.SleepingLockWrapper in project elasticsearch by elastic.

the class FsDirectoryServiceTests method testHasNoSleepWrapperOnNormalFS.

public void testHasNoSleepWrapperOnNormalFS() throws IOException {
    Settings build = Settings.builder().put(IndexModule.INDEX_STORE_TYPE_SETTING.getKey(), "simplefs").build();
    IndexSettings settings = IndexSettingsModule.newIndexSettings("foo", build);
    IndexStore store = new IndexStore(settings);
    Path tempDir = createTempDir().resolve(settings.getUUID()).resolve("0");
    Files.createDirectories(tempDir);
    ShardPath path = new ShardPath(false, tempDir, tempDir, new ShardId(settings.getIndex(), 0));
    FsDirectoryService fsDirectoryService = new FsDirectoryService(settings, store, path);
    Directory directory = fsDirectoryService.newDirectory();
    assertFalse(directory instanceof SleepingLockWrapper);
    assertTrue(directory instanceof SimpleFSDirectory);
}
Also used : ShardPath(org.elasticsearch.index.shard.ShardPath) Path(java.nio.file.Path) ShardId(org.elasticsearch.index.shard.ShardId) ShardPath(org.elasticsearch.index.shard.ShardPath) IndexSettings(org.elasticsearch.index.IndexSettings) SleepingLockWrapper(org.apache.lucene.store.SleepingLockWrapper) SimpleFSDirectory(org.apache.lucene.store.SimpleFSDirectory) Settings(org.elasticsearch.common.settings.Settings) IndexSettings(org.elasticsearch.index.IndexSettings) SimpleFSDirectory(org.apache.lucene.store.SimpleFSDirectory) MMapDirectory(org.apache.lucene.store.MMapDirectory) Directory(org.apache.lucene.store.Directory) FileSwitchDirectory(org.apache.lucene.store.FileSwitchDirectory)

Aggregations

Directory (org.apache.lucene.store.Directory)5 MMapDirectory (org.apache.lucene.store.MMapDirectory)5 SimpleFSDirectory (org.apache.lucene.store.SimpleFSDirectory)5 SleepingLockWrapper (org.apache.lucene.store.SleepingLockWrapper)5 Path (java.nio.file.Path)4 FileSwitchDirectory (org.apache.lucene.store.FileSwitchDirectory)4 Settings (org.elasticsearch.common.settings.Settings)4 IndexSettings (org.elasticsearch.index.IndexSettings)4 ShardPath (org.elasticsearch.index.shard.ShardPath)4 ShardId (org.elasticsearch.index.shard.ShardId)3 NIOFSDirectory (org.apache.lucene.store.NIOFSDirectory)2 HashSet (java.util.HashSet)1 AlreadyClosedException (org.apache.lucene.store.AlreadyClosedException)1 FSDirectory (org.apache.lucene.store.FSDirectory)1 LockFactory (org.apache.lucene.store.LockFactory)1 NativeFSLockFactory (org.apache.lucene.store.NativeFSLockFactory)1 SimpleFSLockFactory (org.apache.lucene.store.SimpleFSLockFactory)1