Search in sources :

Example 6 with FailSafeConsistencyGuard

use of org.apache.hudi.common.fs.FailSafeConsistencyGuard in project hudi by apache.

the class HoodieTableMetaClient method getFs.

/**
 * Get the FS implementation for this table.
 */
public HoodieWrapperFileSystem getFs() {
    if (fs == null) {
        FileSystem fileSystem = FSUtils.getFs(metaPath, hadoopConf.newCopy());
        if (fileSystemRetryConfig.isFileSystemActionRetryEnable()) {
            fileSystem = new HoodieRetryWrapperFileSystem(fileSystem, fileSystemRetryConfig.getMaxRetryIntervalMs(), fileSystemRetryConfig.getMaxRetryNumbers(), fileSystemRetryConfig.getInitialRetryIntervalMs(), fileSystemRetryConfig.getRetryExceptions());
        }
        ValidationUtils.checkArgument(!(fileSystem instanceof HoodieWrapperFileSystem), "File System not expected to be that of HoodieWrapperFileSystem");
        fs = new HoodieWrapperFileSystem(fileSystem, consistencyGuardConfig.isConsistencyCheckEnabled() ? new FailSafeConsistencyGuard(fileSystem, consistencyGuardConfig) : new NoOpConsistencyGuard());
    }
    return fs;
}
Also used : HoodieRetryWrapperFileSystem(org.apache.hudi.common.fs.HoodieRetryWrapperFileSystem) HoodieWrapperFileSystem(org.apache.hudi.common.fs.HoodieWrapperFileSystem) HoodieRetryWrapperFileSystem(org.apache.hudi.common.fs.HoodieRetryWrapperFileSystem) FileSystem(org.apache.hadoop.fs.FileSystem) HoodieWrapperFileSystem(org.apache.hudi.common.fs.HoodieWrapperFileSystem) FailSafeConsistencyGuard(org.apache.hudi.common.fs.FailSafeConsistencyGuard) NoOpConsistencyGuard(org.apache.hudi.common.fs.NoOpConsistencyGuard)

Aggregations

FailSafeConsistencyGuard (org.apache.hudi.common.fs.FailSafeConsistencyGuard)6 ConsistencyGuard (org.apache.hudi.common.fs.ConsistencyGuard)5 OptimisticConsistencyGuard (org.apache.hudi.common.fs.OptimisticConsistencyGuard)5 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)5 Test (org.junit.jupiter.api.Test)4 Path (org.apache.hadoop.fs.Path)3 FileSystem (org.apache.hadoop.fs.FileSystem)1 ConsistencyGuardConfig (org.apache.hudi.common.fs.ConsistencyGuardConfig)1 HoodieRetryWrapperFileSystem (org.apache.hudi.common.fs.HoodieRetryWrapperFileSystem)1 HoodieWrapperFileSystem (org.apache.hudi.common.fs.HoodieWrapperFileSystem)1 NoOpConsistencyGuard (org.apache.hudi.common.fs.NoOpConsistencyGuard)1 MethodSource (org.junit.jupiter.params.provider.MethodSource)1