Examples with ConsistencyGuard org.apache.hudi.common.fs.ConsistencyGuard used on opensource projects

Example 1 with ConsistencyGuard

use of org.apache.hudi.common.fs.ConsistencyGuard in project hudi by apache.

the class TestConsistencyGuard method testCheckFailingAppearFailSafe.

@Test
public void testCheckFailingAppearFailSafe() throws Exception {
    FileCreateUtils.createBaseFile(basePath, "partition/path", "000", "f1");
    ConsistencyGuard passing = new FailSafeConsistencyGuard(fs, getConsistencyGuardConfig());
    assertThrows(TimeoutException.class, () -> {
        passing.waitTillAllFilesAppear(basePath + "/partition/path", Arrays.asList(basePath + "/partition/path/f1_1-0-2_000" + BASE_FILE_EXTENSION, basePath + "/partition/path/f2_1-0-2_000" + BASE_FILE_EXTENSION));
    });
}

Example 2 with ConsistencyGuard

use of org.apache.hudi.common.fs.ConsistencyGuard in project hudi by apache.

the class TestConsistencyGuard method testCheckFailingDisappearFailSafe.

@Test
public void testCheckFailingDisappearFailSafe() throws Exception {
    FileCreateUtils.createBaseFile(basePath, "partition/path", "000", "f1");
    ConsistencyGuard passing = new FailSafeConsistencyGuard(fs, getConsistencyGuardConfig());
    assertThrows(TimeoutException.class, () -> {
        passing.waitTillAllFilesDisappear(basePath + "/partition/path", Arrays.asList(basePath + "/partition/path/f1_1-0-1_000" + BASE_FILE_EXTENSION, basePath + "/partition/path/f2_1-0-2_000" + BASE_FILE_EXTENSION));
    });
}

Example 3 with ConsistencyGuard

use of org.apache.hudi.common.fs.ConsistencyGuard in project hudi by apache.

the class TestConsistencyGuard method testCheckFailingAppearsFailSafe.

@Test
public void testCheckFailingAppearsFailSafe() throws Exception {
    FileCreateUtils.createBaseFile(basePath, "partition/path", "000", "f1");
    ConsistencyGuard passing = new FailSafeConsistencyGuard(fs, getConsistencyGuardConfig());
    assertThrows(TimeoutException.class, () -> {
        passing.waitTillFileAppears(new Path(basePath + "/partition/path/f1_1-0-2_000" + BASE_FILE_EXTENSION));
    });
}

Example 4 with ConsistencyGuard

use of org.apache.hudi.common.fs.ConsistencyGuard in project hudi by apache.

the class TestConsistencyGuard method testCheckFailingAppearTimedWait.

@Test
public void testCheckFailingAppearTimedWait() throws Exception {
    FileCreateUtils.createBaseFile(basePath, "partition/path", "000", "f1");
    ConsistencyGuard passing = new OptimisticConsistencyGuard(fs, getConsistencyGuardConfig());
    passing.waitTillAllFilesAppear(basePath + "/partition/path", Arrays.asList(basePath + "/partition/path/f1_1-0-2_000" + BASE_FILE_EXTENSION, basePath + "/partition/path/f2_1-0-2_000" + BASE_FILE_EXTENSION));
}

Example 5 with ConsistencyGuard

use of org.apache.hudi.common.fs.ConsistencyGuard in project hudi by apache.

the class TestConsistencyGuard method testCheckFailingDisappearsFailSafe.

@Test
public void testCheckFailingDisappearsFailSafe() throws Exception {
    FileCreateUtils.createBaseFile(basePath, "partition/path", "000", "f1");
    FileCreateUtils.createBaseFile(basePath, "partition/path", "000", "f1");
    ConsistencyGuard passing = new FailSafeConsistencyGuard(fs, getConsistencyGuardConfig());
    assertThrows(TimeoutException.class, () -> {
        passing.waitTillFileDisappears(new Path(basePath + "/partition/path/f1_1-0-1_000" + BASE_FILE_EXTENSION));
    });
}