Search in sources :

Example 16 with UfsFileStatus

use of alluxio.underfs.UfsFileStatus in project alluxio by Alluxio.

the class DefaultFileSystemMaster method checkConsistencyInternal.

/**
 * Checks if a path is consistent between Alluxio and the underlying storage.
 * <p>
 * A path without a backing under storage is always consistent.
 * <p>
 * A not persisted path is considered consistent if:
 * 1. It does not shadow an object in the underlying storage.
 * <p>
 * A persisted path is considered consistent if:
 * 1. An equivalent object exists for its under storage path.
 * 2. The metadata of the Alluxio and under storage object are equal.
 *
 * @param inodePath the path to check. This must exist and be read-locked
 * @return true if the path is consistent, false otherwise
 */
private boolean checkConsistencyInternal(LockedInodePath inodePath) throws InvalidPathException, IOException {
    Inode inode;
    try {
        inode = inodePath.getInode();
    } catch (FileDoesNotExistException e) {
        // already checked existence when creating the inodePath
        throw new RuntimeException(e);
    }
    MountTable.Resolution resolution = mMountTable.resolve(inodePath.getUri());
    try (CloseableResource<UnderFileSystem> ufsResource = resolution.acquireUfsResource()) {
        UnderFileSystem ufs = ufsResource.get();
        String ufsPath = resolution.getUri().getPath();
        if (ufs == null) {
            return true;
        }
        if (!inode.isPersisted()) {
            return !ufs.exists(ufsPath);
        }
        UfsStatus ufsStatus;
        try {
            ufsStatus = ufs.getStatus(ufsPath);
        } catch (FileNotFoundException e) {
            return !inode.isPersisted();
        }
        // TODO(calvin): Evaluate which other metadata fields should be validated.
        if (inode.isDirectory()) {
            return ufsStatus.isDirectory();
        } else {
            String ufsFingerprint = Fingerprint.create(ufs.getUnderFSType(), ufsStatus).serialize();
            return ufsStatus.isFile() && (ufsFingerprint.equals(inode.asFile().getUfsFingerprint())) && ufsStatus instanceof UfsFileStatus && ((UfsFileStatus) ufsStatus).getContentLength() == inode.asFile().getLength();
        }
    }
}
Also used : FileDoesNotExistException(alluxio.exception.FileDoesNotExistException) UfsFileStatus(alluxio.underfs.UfsFileStatus) Inode(alluxio.master.file.meta.Inode) UfsStatus(alluxio.underfs.UfsStatus) FileNotFoundException(java.io.FileNotFoundException) MountTable(alluxio.master.file.meta.MountTable) UnderFileSystem(alluxio.underfs.UnderFileSystem)

Aggregations

UfsFileStatus (alluxio.underfs.UfsFileStatus)16 UfsDirectoryStatus (alluxio.underfs.UfsDirectoryStatus)7 UfsStatus (alluxio.underfs.UfsStatus)6 UnderFileSystem (alluxio.underfs.UnderFileSystem)4 IOException (java.io.IOException)4 AlluxioURI (alluxio.AlluxioURI)3 File (java.io.File)3 FileNotFoundException (java.io.FileNotFoundException)3 PosixFileAttributes (java.nio.file.attribute.PosixFileAttributes)3 FileStatus (org.apache.hadoop.fs.FileStatus)3 Test (org.junit.Test)3 ConsistentUnderFileSystem (alluxio.underfs.ConsistentUnderFileSystem)2 FileInfo (alluxio.wire.FileInfo)2 CephStat (com.ceph.fs.CephStat)2 FileSystemException (java.nio.file.FileSystemException)2 Nullable (javax.annotation.Nullable)2 FileSystem (org.apache.hadoop.fs.FileSystem)2 Path (org.apache.hadoop.fs.Path)2 DistributedFileSystem (org.apache.hadoop.hdfs.DistributedFileSystem)2 PrepareForTest (org.powermock.core.classloader.annotations.PrepareForTest)2