Search in sources :

Example 1 with ProcessKeyCache

use of com.walmartlabs.concord.server.process.queue.ProcessKeyCache in project concord by walmartlabs.

the class ProcessStateManagerTest method testUpdateState.

@Test
public void testUpdateState() throws Exception {
    ProcessKey processKey = new ProcessKey(UUID.randomUUID(), OffsetDateTime.now());
    Path baseDir = Files.createTempDirectory("testImport");
    writeTempFile(baseDir.resolve("file-1"), "123".getBytes());
    writeTempFile(baseDir.resolve("file-2"), "456".getBytes());
    // 
    ProcessKeyCache processKeyCache = new ProcessKeyCache(new ProcessQueueDao(getConfiguration(), new ConcordObjectMapper(new ObjectMapper())));
    ProcessConfiguration stateCfg = new ProcessConfiguration(Duration.of(24, ChronoUnit.HOURS), Collections.singletonList(Constants.Files.CONFIGURATION_FILE_NAME));
    ProcessStateManager stateManager = new ProcessStateManager(getConfiguration(), mock(SecretStoreConfiguration.class), stateCfg, mock(PolicyManager.class), mock(ProcessLogManager.class), processKeyCache);
    stateManager.importPath(processKey, null, baseDir, (p, attrs) -> true);
    Path tmpDir = Files.createTempDirectory("testExport");
    boolean result = stateManager.export(processKey, copyTo(tmpDir));
    assertTrue(result);
    assertFileContent("123", tmpDir.resolve("file-1"));
    assertFileContent("456", tmpDir.resolve("file-2"));
    // --- update
    writeTempFile(baseDir.resolve("file-1"), "123-up".getBytes());
    stateManager.importPath(processKey, null, baseDir, (p, attrs) -> true);
    result = stateManager.export(processKey, copyTo(tmpDir));
    assertTrue(result);
    assertFileContent("123-up", tmpDir.resolve("file-1"));
    assertFileContent("456", tmpDir.resolve("file-2"));
}
Also used : Path(java.nio.file.Path) ProcessKeyCache(com.walmartlabs.concord.server.process.queue.ProcessKeyCache) PolicyManager(com.walmartlabs.concord.server.policy.PolicyManager) ProcessConfiguration(com.walmartlabs.concord.server.cfg.ProcessConfiguration) ConcordObjectMapper(com.walmartlabs.concord.server.ConcordObjectMapper) SecretStoreConfiguration(com.walmartlabs.concord.server.cfg.SecretStoreConfiguration) ProcessLogManager(com.walmartlabs.concord.server.process.logs.ProcessLogManager) ProcessKey(com.walmartlabs.concord.server.sdk.ProcessKey) ConcordObjectMapper(com.walmartlabs.concord.server.ConcordObjectMapper) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) ProcessQueueDao(com.walmartlabs.concord.server.process.queue.ProcessQueueDao) Test(org.junit.jupiter.api.Test) AbstractDaoTest(com.walmartlabs.concord.server.AbstractDaoTest)

Example 2 with ProcessKeyCache

use of com.walmartlabs.concord.server.process.queue.ProcessKeyCache in project concord by walmartlabs.

the class ProcessStateManagerTest method testLargeImport.

@Test
public void testLargeImport() throws Exception {
    ProcessKey processKey = new ProcessKey(UUID.randomUUID(), OffsetDateTime.now());
    int files = 100;
    int chunkSize = 1024 * 1024;
    int fileSize = 10 * chunkSize;
    byte[] ab = new byte[chunkSize];
    Arrays.fill(ab, (byte) 0);
    Path baseDir = Files.createTempDirectory("test");
    for (int i = 0; i < files; i++) {
        Path p = baseDir.resolve("file" + i);
        try (OutputStream out = Files.newOutputStream(p, StandardOpenOption.CREATE)) {
            for (int j = 0; j < fileSize; j += chunkSize) {
                out.write(ab);
            }
        }
    }
    ProcessKeyCache processKeyCache = new ProcessKeyCache(new ProcessQueueDao(getConfiguration(), new ConcordObjectMapper(new ObjectMapper())));
    ProcessConfiguration stateCfg = new ProcessConfiguration(Duration.of(24, ChronoUnit.HOURS), Collections.singletonList(Constants.Files.CONFIGURATION_FILE_NAME));
    ProcessStateManager stateManager = new ProcessStateManager(getConfiguration(), mock(SecretStoreConfiguration.class), stateCfg, mock(PolicyManager.class), mock(ProcessLogManager.class), processKeyCache);
    stateManager.importPath(processKey, "/", baseDir, (p, attrs) -> true);
}
Also used : Path(java.nio.file.Path) ProcessKeyCache(com.walmartlabs.concord.server.process.queue.ProcessKeyCache) PolicyManager(com.walmartlabs.concord.server.policy.PolicyManager) SecretStoreConfiguration(com.walmartlabs.concord.server.cfg.SecretStoreConfiguration) OutputStream(java.io.OutputStream) ProcessKey(com.walmartlabs.concord.server.sdk.ProcessKey) ProcessQueueDao(com.walmartlabs.concord.server.process.queue.ProcessQueueDao) ProcessConfiguration(com.walmartlabs.concord.server.cfg.ProcessConfiguration) ConcordObjectMapper(com.walmartlabs.concord.server.ConcordObjectMapper) ProcessLogManager(com.walmartlabs.concord.server.process.logs.ProcessLogManager) ConcordObjectMapper(com.walmartlabs.concord.server.ConcordObjectMapper) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Test(org.junit.jupiter.api.Test) AbstractDaoTest(com.walmartlabs.concord.server.AbstractDaoTest)

Aggregations

ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)2 AbstractDaoTest (com.walmartlabs.concord.server.AbstractDaoTest)2 ConcordObjectMapper (com.walmartlabs.concord.server.ConcordObjectMapper)2 ProcessConfiguration (com.walmartlabs.concord.server.cfg.ProcessConfiguration)2 SecretStoreConfiguration (com.walmartlabs.concord.server.cfg.SecretStoreConfiguration)2 PolicyManager (com.walmartlabs.concord.server.policy.PolicyManager)2 ProcessLogManager (com.walmartlabs.concord.server.process.logs.ProcessLogManager)2 ProcessKeyCache (com.walmartlabs.concord.server.process.queue.ProcessKeyCache)2 ProcessQueueDao (com.walmartlabs.concord.server.process.queue.ProcessQueueDao)2 ProcessKey (com.walmartlabs.concord.server.sdk.ProcessKey)2 Path (java.nio.file.Path)2 Test (org.junit.jupiter.api.Test)2 OutputStream (java.io.OutputStream)1