Search in sources :

Example 1 with PersistentGlobalState

use of com.yelp.nrtsearch.server.luceneserver.state.PersistentGlobalState in project nrtsearch by Yelp.

the class LocalStateBackend method loadOrCreateGlobalState.

@Override
public PersistentGlobalState loadOrCreateGlobalState() throws IOException {
    logger.info("Loading local state");
    Path statePath = globalStatePath.resolve(GLOBAL_STATE_FILE);
    File stateFile = statePath.toFile();
    if (stateFile.isDirectory()) {
        throw new IllegalStateException("State file: " + stateFile + " is a directory");
    }
    if (!stateFile.exists()) {
        logger.info("Local state not present, initializing default");
        PersistentGlobalState state = new PersistentGlobalState();
        commitGlobalState(state);
        return state;
    } else {
        PersistentGlobalState persistentGlobalState = StateUtils.readStateFromFile(statePath);
        logger.info("Loaded local state: " + MAPPER.writeValueAsString(persistentGlobalState));
        return persistentGlobalState;
    }
}
Also used : Path(java.nio.file.Path) PersistentGlobalState(com.yelp.nrtsearch.server.luceneserver.state.PersistentGlobalState) File(java.io.File)

Example 2 with PersistentGlobalState

use of com.yelp.nrtsearch.server.luceneserver.state.PersistentGlobalState in project nrtsearch by Yelp.

the class RemoteStateBackendTest method testLoadsSavedState.

@Test
public void testLoadsSavedState() throws IOException {
    StateBackend stateBackend = new RemoteStateBackend(getMockGlobalState(false));
    Path localFilePath = getLocalStateFilePath();
    assertFalse(localFilePath.toFile().exists());
    Map<String, IndexInfo> indicesMap = new HashMap<>();
    indicesMap.put("test_index", new IndexInfo());
    indicesMap.put("test_index_2", new IndexInfo());
    PersistentGlobalState initialState = new PersistentGlobalState(indicesMap);
    writeStateToS3(initialState);
    PersistentGlobalState loadedState = stateBackend.loadOrCreateGlobalState();
    assertEquals(initialState, loadedState);
    assertTrue(localFilePath.toFile().exists());
    assertTrue(localFilePath.toFile().isFile());
    PersistentGlobalState loadedLocalState = StateUtils.readStateFromFile(localFilePath);
    assertEquals(initialState, loadedLocalState);
}
Also used : Path(java.nio.file.Path) HashMap(java.util.HashMap) PersistentGlobalState(com.yelp.nrtsearch.server.luceneserver.state.PersistentGlobalState) IndexInfo(com.yelp.nrtsearch.server.luceneserver.state.PersistentGlobalState.IndexInfo) Test(org.junit.Test)

Example 3 with PersistentGlobalState

use of com.yelp.nrtsearch.server.luceneserver.state.PersistentGlobalState in project nrtsearch by Yelp.

the class RemoteStateBackendTest method testReadOnlyCommit.

@Test
public void testReadOnlyCommit() throws IOException {
    StateBackend stateBackend = new RemoteStateBackend(getMockGlobalState(true));
    Map<String, IndexInfo> indicesMap = new HashMap<>();
    indicesMap.put("test_index", new IndexInfo());
    indicesMap.put("test_index_2", new IndexInfo());
    PersistentGlobalState initialState = new PersistentGlobalState(indicesMap);
    writeStateToS3(initialState);
    PersistentGlobalState loadedState = stateBackend.loadOrCreateGlobalState();
    assertEquals(initialState, loadedState);
    indicesMap = new HashMap<>();
    indicesMap.put("test_index_3", new IndexInfo());
    indicesMap.put("test_index_4", new IndexInfo());
    indicesMap.put("test_index_5", new IndexInfo());
    PersistentGlobalState updatedState = new PersistentGlobalState(indicesMap);
    try {
        stateBackend.commitGlobalState(updatedState);
        fail();
    } catch (IllegalStateException e) {
        assertEquals("Cannot update remote state when configured as read only", e.getMessage());
    }
}
Also used : HashMap(java.util.HashMap) PersistentGlobalState(com.yelp.nrtsearch.server.luceneserver.state.PersistentGlobalState) IndexInfo(com.yelp.nrtsearch.server.luceneserver.state.PersistentGlobalState.IndexInfo) Test(org.junit.Test)

Example 4 with PersistentGlobalState

use of com.yelp.nrtsearch.server.luceneserver.state.PersistentGlobalState in project nrtsearch by Yelp.

the class LocalStateBackendTest method testLoadsSavedState.

@Test
public void testLoadsSavedState() throws IOException {
    StateBackend stateBackend = new LocalStateBackend(getMockGlobalState());
    Path filePath = getStateFilePath();
    Map<String, IndexInfo> indicesMap = new HashMap<>();
    indicesMap.put("test_index", new IndexInfo());
    indicesMap.put("test_index_2", new IndexInfo());
    PersistentGlobalState initialState = new PersistentGlobalState(indicesMap);
    StateUtils.writeStateToFile(initialState, Paths.get(folder.getRoot().getAbsolutePath(), StateUtils.GLOBAL_STATE_FOLDER), StateUtils.GLOBAL_STATE_FILE);
    assertTrue(filePath.toFile().exists());
    assertTrue(filePath.toFile().isFile());
    PersistentGlobalState loadedState = stateBackend.loadOrCreateGlobalState();
    assertEquals(initialState, loadedState);
}
Also used : Path(java.nio.file.Path) HashMap(java.util.HashMap) PersistentGlobalState(com.yelp.nrtsearch.server.luceneserver.state.PersistentGlobalState) IndexInfo(com.yelp.nrtsearch.server.luceneserver.state.PersistentGlobalState.IndexInfo) Test(org.junit.Test)

Example 5 with PersistentGlobalState

use of com.yelp.nrtsearch.server.luceneserver.state.PersistentGlobalState in project nrtsearch by Yelp.

the class LocalStateBackendTest method testCreatesDefaultState.

@Test
public void testCreatesDefaultState() throws IOException {
    StateBackend stateBackend = new LocalStateBackend(getMockGlobalState());
    Path filePath = getStateFilePath();
    assertFalse(filePath.toFile().exists());
    PersistentGlobalState globalState = stateBackend.loadOrCreateGlobalState();
    assertEquals(globalState, new PersistentGlobalState());
    assertTrue(filePath.toFile().exists());
    assertTrue(filePath.toFile().isFile());
    PersistentGlobalState loadedState = StateUtils.readStateFromFile(filePath);
    assertEquals(globalState, loadedState);
}
Also used : Path(java.nio.file.Path) PersistentGlobalState(com.yelp.nrtsearch.server.luceneserver.state.PersistentGlobalState) Test(org.junit.Test)

Aggregations

PersistentGlobalState (com.yelp.nrtsearch.server.luceneserver.state.PersistentGlobalState)11 Path (java.nio.file.Path)9 Test (org.junit.Test)8 IndexInfo (com.yelp.nrtsearch.server.luceneserver.state.PersistentGlobalState.IndexInfo)6 HashMap (java.util.HashMap)6 DataInputStream (java.io.DataInputStream)1 File (java.io.File)1 FileInputStream (java.io.FileInputStream)1 LZ4FrameInputStream (net.jpountz.lz4.LZ4FrameInputStream)1 TarArchiveEntry (org.apache.commons.compress.archivers.tar.TarArchiveEntry)1 TarArchiveInputStream (org.apache.commons.compress.archivers.tar.TarArchiveInputStream)1