Examples with RawLocalFileSystem org.apache.hadoop.fs.RawLocalFileSystem used on opensource projects

Example 1 with RawLocalFileSystem

use of org.apache.hadoop.fs.RawLocalFileSystem in project hadoop by apache.

the class TestFileSystemApplicationHistoryStore method testInitNonExistingWorkingDirectoryInSafeMode.

@Test
public void testInitNonExistingWorkingDirectoryInSafeMode() throws Exception {
    LOG.info("Starting testInitNonExistingWorkingDirectoryInSafeMode");
    tearDown();
    // Setup file system to inject startup conditions
    FileSystem fs = spy(new RawLocalFileSystem());
    FileStatus fileStatus = Mockito.mock(FileStatus.class);
    doReturn(false).when(fileStatus).isDirectory();
    doReturn(fileStatus).when(fs).getFileStatus(any(Path.class));
    doThrow(new IOException()).when(fs).mkdirs(any(Path.class));
    try {
        initAndStartStore(fs);
        Assert.fail("Exception should have been thrown");
    } catch (Exception e) {
    // Expected failure
    }
    // Make sure that directory creation was attempted
    verify(fileStatus, never()).isDirectory();
    verify(fs, times(1)).mkdirs(any(Path.class));
}

Example 2 with RawLocalFileSystem

use of org.apache.hadoop.fs.RawLocalFileSystem in project hadoop by apache.

the class TestRawlocalContractRename method testRenameWithNonEmptySubDirPOSIX.

/**
   * Test fallback rename code <code>handleEmptyDstDirectoryOnWindows()</code>
   * even on not Windows platform where the normal <code>File.renameTo()</code>
   * is supposed to work well. This test has been added for HADOOP-9805.
   * 
   * @see AbstractContractRenameTest#testRenameWithNonEmptySubDirPOSIX()
   */
@Test
public void testRenameWithNonEmptySubDirPOSIX() throws Throwable {
    final Path renameTestDir = path("testRenameWithNonEmptySubDir");
    final Path srcDir = new Path(renameTestDir, "src1");
    final Path srcSubDir = new Path(srcDir, "sub");
    final Path finalDir = new Path(renameTestDir, "dest");
    FileSystem fs = getFileSystem();
    ContractTestUtils.rm(fs, renameTestDir, true, false);
    fs.mkdirs(srcDir);
    fs.mkdirs(finalDir);
    ContractTestUtils.writeTextFile(fs, new Path(srcDir, "source.txt"), "this is the file in src dir", false);
    ContractTestUtils.writeTextFile(fs, new Path(srcSubDir, "subfile.txt"), "this is the file in src/sub dir", false);
    ContractTestUtils.assertPathExists(fs, "not created in src dir", new Path(srcDir, "source.txt"));
    ContractTestUtils.assertPathExists(fs, "not created in src/sub dir", new Path(srcSubDir, "subfile.txt"));
    RawLocalFileSystem rlfs = (RawLocalFileSystem) fs;
    rlfs.handleEmptyDstDirectoryOnWindows(srcDir, rlfs.pathToFile(srcDir), finalDir, rlfs.pathToFile(finalDir));
    // Accept only POSIX rename behavior in this test
    ContractTestUtils.assertPathExists(fs, "not renamed into dest dir", new Path(finalDir, "source.txt"));
    ContractTestUtils.assertPathExists(fs, "not renamed into dest/sub dir", new Path(finalDir, "sub/subfile.txt"));
    ContractTestUtils.assertPathDoesNotExist(fs, "not deleted", new Path(srcDir, "source.txt"));
}

Example 3 with RawLocalFileSystem

use of org.apache.hadoop.fs.RawLocalFileSystem in project tez by apache.

the class ExternalSorter method sameVolRename.

/**
 * Rename srcPath to dstPath on the same volume. This is the same as
 * RawLocalFileSystem's rename method, except that it will not fall back to a
 * copy, and it will create the target directory if it doesn't exist.
 */
protected void sameVolRename(Path srcPath, Path dstPath) throws IOException {
    RawLocalFileSystem rfs = (RawLocalFileSystem) this.rfs;
    File src = rfs.pathToFile(srcPath);
    File dst = rfs.pathToFile(dstPath);
    if (!dst.getParentFile().exists()) {
        if (!dst.getParentFile().mkdirs()) {
            throw new IOException("Unable to rename " + src + " to " + dst + ": couldn't create parent directory");
        }
    }
    if (!src.renameTo(dst)) {
        throw new IOException("Unable to rename " + src + " to " + dst);
    }
}

Example 4 with RawLocalFileSystem

use of org.apache.hadoop.fs.RawLocalFileSystem in project flink by apache.

the class HadoopLocalFileSystemBehaviorTest method getFileSystem.

@Override
public FileSystem getFileSystem() throws Exception {
    org.apache.hadoop.fs.FileSystem fs = new RawLocalFileSystem();
    fs.initialize(LocalFileSystem.getLocalFsURI(), new Configuration());
    return new HadoopFileSystem(fs);
}

Example 5 with RawLocalFileSystem

use of org.apache.hadoop.fs.RawLocalFileSystem in project kylo by Teradata.

the class FileSystemUtilTest method registerFileSystems.

/**
 * Verify adding file systems to Hadoop configuration.
 */
@Test
public void registerFileSystems() {
    final Configuration conf = new Configuration(false);
    FileSystemUtil.registerFileSystems(Arrays.asList(new LocalFileSystem(), new MockFileSystem(), new RawLocalFileSystem()), conf);
    Assert.assertEquals(LocalFileSystem.class.getName(), conf.get("fs.file.impl"));
    Assert.assertEquals(MockFileSystem.class.getName(), conf.get("fs.mock.impl"));
}